GlossaryAI

AI inference

Also known as: Inference workload, Model serving

Inference is the stage where a trained model actually gets used: answering a prompt, classifying an image, suggesting a product. Training happens in a few huge clusters. Inference is spread across many sites, follows user demand and is sensitive to latency, so it pulls compute toward regional and edge facilities.