glossary
AI Glossary
AI inference
Also known as: Inference workload, Model serving
Inference is the stage where a trained model actually gets used: answering a prompt, classifying an image, suggesting a product. Training happens in a few huge clusters. Inference is spread across many sites, follows user demand and is sensitive to latency, so it pulls compute toward regional and edge facilities.