draft-mondal-llm-serving-workload-profiles
Benchmarking Workload Profiles for Large Language Model Serving
This document defines standard workload profiles for benchmarking
Large Language Model (LLM) inference serving systems. Each profile
represents one class of production workload. A profile is defined by
its input and output token distribution, output structure, latency
sensitivity, concurrency pattern, and caching behavior. Profiles are
organized into six groups: Non-Generative, Minimal Output,
Interactive Streaming, Prefill-Heavy Generative, Decode-Heavy
Generative, and Multi-Step Chained. This document works with
"Benchmarking Methodology for Large Language Model Serving"
[LLM-METHOD]. It specifies the workloads that the methodology's
tests SHOULD be run against.