draft-mondal-llm-serving-workload-profiles

Benchmarking Workload Profiles for Large Language Model Serving

groupnone stateactive pages56 rev00 expires2026-10-11 canonicalhttps://datatracker.ietf.org/doc/draft-mondal-llm-serving-workload-profiles/
This document defines standard workload profiles for benchmarking Large Language Model (LLM) inference serving systems. Each profile represents one class of production workload. A profile is defined by its input and output token distribution, output structure, latency sensitivity, concurrency pattern, and caching behavior. Profiles are organized into six groups: Non-Generative, Minimal Output, Interactive Streaming, Prefill-Heavy Generative, Decode-Heavy Generative, and Multi-Step Chained. This document works with "Benchmarking Methodology for Large Language Model Serving" [LLM-METHOD]. It specifies the workloads that the methodology's tests SHOULD be run against.