What Is LLM Inference? How AI Models Generate Responses
LLM inference is the process of running a trained model against new input. It involves tokenization, continuous batching, and two phases: compute-bound prefill and memory bandwidth-bound decode.
Read article