LLM serving · request trace
A Year in LLM Serving
One year of request-level production traffic from Chutes for LLM workload, caching, and load-balancing research.
- 6.12Brequests
- 9,174models
- 91 GBParquet
Dataset details
Request-level timing, model and user identifiers, serving instances, token counts, latency, time-to-first-token, and prefix-cache reuse.