The tool-use loop

The tool-use loop, one step at a time

A scripted run of the loop an application drives. No live model is called; the control flow is the real thing.

Steps

Conversation so far, sent in full on every model call

Tools offeredget_weather(city: string)calculator(expression: string)

Instructions
You can call tools. Reply with a tool call or a final answer.
User
What's the temperature right now in the city where the Eiffel Tower is, in Fahrenheit?

Model first passes a landmark instead of a city

Weather service is down

Step limit (model calls)6

Model calls so far0 of 6 allowed

Messages in the conversation2

OutcomeRunning

Challenges

0 of 3 complete

Get to a correct answer even though the model's first tool call fails. (not yet)
With the landmark error on, find the smallest step limit that still ends in an answer. (not yet)
End a run without an answer in two different ways. (not yet)

Real tool-use runs, recorded

Three sizes of one open model, Qwen2.5-Instruct, run through the same loop with four real tools. Every message is exactly what was recorded.

Loading the recorded outputs...

Try this

Step through a normal run and note who acts at each step: the model only ever writes messages.
Turn on the landmark error and watch the model use the error message to correct itself.
Lower the step limit until the program stops the loop before an answer.
In the recorded runs, compare what the three model sizes do with the same task, and turn on “Show exactly what the model wrote” to see the tool call as the text it really is.

The tool-use loop in codePython with the standard library

import ast
import operator

# Tools: ordinary functions that the program, not the model, runs.
def get_weather(city):
    if city != "Paris":
        return {"error": f'unknown city: "{city}"'}
    return {"city": "Paris", "temp_c": 18, "conditions": "cloudy"}

OPS = {ast.Add: operator.add, ast.Sub: operator.sub,
       ast.Mult: operator.mul, ast.Div: operator.truediv}

def calculator(expression):
    def ev(node):  # arithmetic only; never eval() text a model wrote
        if isinstance(node, ast.BinOp) and type(node.op) in OPS:
            return OPS[type(node.op)](ev(node.left), ev(node.right))
        if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
            return node.value
        raise ValueError
    try:
        return {"value": round(ev(ast.parse(expression, mode="eval").body), 2)}
    except (SyntaxError, ValueError):
        return {"error": "invalid expression"}

TOOLS = {"get_weather": get_weather, "calculator": calculator}

def model(messages):
    # A scripted stand-in for the model: it reads the conversation and
    # replies with a tool call or a final answer. It first passes a landmark.
    results = [m["content"] for m in messages if m["role"] == "tool"]
    weather = next((r for r in results if "temp_c" in r), None)
    value = next((r["value"] for r in results if "value" in r), None)
    if weather and value is not None:
        return {"answer": f"About {value} degrees Fahrenheit in Paris."}
    if weather:
        expression = f"{weather['temp_c']} * 9 / 5 + 32"
        return {"tool": "calculator", "args": {"expression": expression}}
    return {"tool": "get_weather", "args": {"city": "Paris" if results else "Eiffel Tower"}}

def run(limit):
    messages = [{"role": "user", "content": "Temperature where the Eiffel Tower is, in F?"}]
    for call in range(1, limit + 1):  # the program enforces the step limit
        reply = model(messages)
        if "answer" in reply:
            print(f"call {call}: answer: {reply['answer']}")
            return
        print(f"call {call}: asks for {reply['tool']} with {reply['args']}")
        tool = TOOLS.get(reply["tool"])
        result = tool(**reply["args"]) if tool else {"error": "no such tool"}
        print(f"  program runs it: {result}")
        messages += [{"role": "assistant", "content": reply}, {"role": "tool", "content": result}]
    print(f"limit of {limit} model calls reached: stopped without an answer")

run(limit=6)
run(limit=3)

call 1: asks for get_weather with {'city': 'Eiffel Tower'}
  program runs it: {'error': 'unknown city: "Eiffel Tower"'}
call 2: asks for get_weather with {'city': 'Paris'}
  program runs it: {'city': 'Paris', 'temp_c': 18, 'conditions': 'cloudy'}
call 3: asks for calculator with {'expression': '18 * 9 / 5 + 32'}
  program runs it: {'value': 64.4}
call 4: answer: About 64.4 degrees Fahrenheit in Paris.
call 1: asks for get_weather with {'city': 'Eiffel Tower'}
  program runs it: {'error': 'unknown city: "Eiffel Tower"'}
call 2: asks for get_weather with {'city': 'Paris'}
  program runs it: {'city': 'Paris', 'temp_c': 18, 'conditions': 'cloudy'}
call 3: asks for calculator with {'expression': '18 * 9 / 5 + 32'}
  program runs it: {'value': 64.4}
limit of 3 model calls reached: stopped without an answer

The first demo’s model responses are scripted for teaching; the loop, the message formats, and the failure handling mirror how real tool-using applications work. The second shows real runs of three open models through the same kind of loop, recorded message by message.