1.1 · Step 2 · tools go at the front
What survives the cache
This is your coding agent's last request after the chat template rendered it: one system prompt, two tools and four turns, measured with the real Qwen2.5 tokenizer. The width of each block is its token count.
The request the model actually sees
Rendered text
Predict first
Your server reuses the longest unchanged token prefix, as vLLM does. You add one tool, git_diff, to the end of the tools list. Nothing else changes. What can the next call reuse from the prefix cache?
How sure are you?
Pick an answer and how sure you are.
Now edit the request
Each edit is measured against the request above: the next call reuses the longest run of identical tokens from the start.
–tokens reused
–tokens processed again
–reused in 16-token blocks
Tell your tutor what you predicted and what surprised you.