What does time to first token (TTFT) measure?

Question

Accepted Answer

The latency from sending a request until the first output token appears — TTFT captures how responsive a stream feels — it covers queuing and prefill before generation starts, and is separate from total or per-token speed.

Answer

The total time taken to generate every token in a full response

Answer

The time the model spends loading its weights into GPU memory

Answer

The average number of tokens the model produces every second

What does time to first token (TTFT) measure?

Why this is the answer

More Deploying & Serving LLMs in Production flashcards