What does it mean to "serve" a model?

Question

Accepted Answer

Make a trained model available to answer live requests in production — Serving exposes an already-trained model so clients can send inputs and get outputs — it is the inference stage, distinct from training, compression, or evaluation.

Answer

Train a model on fresh data until its accuracy stops improving more

Answer

Compress a model so its stored weights take far less space on disk

Answer

Evaluate a model on a held-out dataset to measure its final quality

What does it mean to "serve" a model?

Why this is the answer

More Deploying & Serving LLMs in Production flashcards