Tool Calling

Tool & Function Calling - Use tools in your prompts

Tool calling (also known as function calls) give an LLM access to external tools. The LLM does not call the tools directly. Instead, it suggests the tool to call. The user then calls the tool separately and provides the results back to the LLM. Finally, the LLM formats the response into an answer to the user's original question.

Tool calling provides LLMs with a powerful and flexible way to interface with external systems and access data beyond their training set. This guide shows how to connect a model to the data and actions provided by your application.

Tool calling is a multi-turn conversation between your application and the model. The tool calling flow has five main steps:

1. Send a request to the model, including the tools it can call

2. Receive tool calls from the model

3. Execute code on the application side using the tool call inputs

4. Send a new request to the model, including the tool outputs

5. Receive the final response from the model (or additional tool calls)

Infron supports tool calling across multiple API protocols:

API protocols	parameters
OpenAI Chat Completion API	Use the tools and tool_choice parameters
OpenAI Responses API (Beta)	Use the tools parameter; responses include the function_call type
Anthropic Messages API (Beta)	Use the tools parameter; tool definitions use input_schema

Protocol comparison

Feature	Chat Completion	Responses API	Anthropic Messages	Vertex AI
Tool parameter name	tools	tools	tools	tools
Schema field	parameters	parameters	input_schema	parameters
Tool call identifier	tool_calls	function_call	tool_use	functionCall
Result field	tool role	function_call_output	tool_result	functionResponse
Parallel calls	✅	✅	✅	✅
Forced calling	tool_choice	tool_choice	tool_choice	functionCallingConfig
Strict mode	strict: true	✅	strict: true	VALIDATED mode
Streaming support	✅	✅	✅	✅

Supported Models: You can find models that support tool calling by filtering on https://infron.ai/models?supported_parameters=tools

If you prefer to learn from a full end-to-end example, keep reading.

OpenAI Chat Completion API

Tool calling example

Let’s look at a complete tool calling flow, using get_horoscope to fetch a daily horoscope for a zodiac sign.

Python

from openai import OpenAI
import json

client = OpenAI(
  base_url="https://llm.onerouter.pro/v1",
  api_key="<API_KEY>",
)

# 1. Define the list of callable tools for the model
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_horoscope",
            "description": "Get today's horoscope for a zodiac sign.",
            "parameters": {
                "type": "object",
                "properties": {
                    "sign": {
                        "type": "string",
                        "description": "Zodiac sign name, e.g., Taurus or Aquarius",
                    },
                },
                "required": ["sign"],
            },
        },
    },
]

# Create the message list; we'll append messages to it over time
input_list = [
    {"role": "user", "content": "How's my horoscope? I'm an Aquarius."}
]

# 2. Prompt the model with the defined tools
response = client.chat.completions.create(
    model="moonshotai/kimi-k2-0905",
    tools=tools,
    messages=input_list,
)

# Save the function call output for subsequent requests
function_call = None
function_call_arguments = None
input_list.append({
  "role": "assistant",
  "content": response.choices[0].message.content,
  "tool_calls": [tool_call.model_dump() for tool_call in response.choices[0].message.tool_calls] if response.choices[0].message.tool_calls else None,
})

for item in response.choices[0].message.tool_calls:
    if item.type == "function":
        function_call = item
        function_call_arguments = json.loads(item.function.arguments)

def get_horoscope(sign):
    return f"{sign}: Next Tuesday you'll meet a baby otter."

# 3. Execute the get_horoscope function logic
result = {"horoscope": get_horoscope(function_call_arguments["sign"])}

# 4. Provide the function call result to the model
input_list.append({
    "role": "tool",
    "tool_call_id": function_call.id,
    "name": function_call.function.name,
    "content": json.dumps(result),
})

print("Final input:")
print(json.dumps(input_list, indent=2, ensure_ascii=False))

response = client.chat.completions.create(
    model="moonshotai/kimi-k2-0905",
    tools=tools,
    messages=input_list,
)

# 5. The model should now be able to respond!
print("Final output:")
print(response.model_dump_json(indent=2))
print("\n" + response.choices[0].message.content)

TypeScript

Defining a function tool (function)

Function tools can be configured via the tools parameter. A function tool is defined by its schema, which tells the model what the function does and what input parameters it expects. A function tool definition includes the following fields:

Field	Description
type	Must always be function
function	The tool object
function.name	Function name (e.g., get_weather)
function.description	Detailed information on when and how to use the function
function.parameters	JSON Schema defining the function input parameters
function.strict	Whether to enable strict schema adherence when generating function calls

Below is the definition for a get_weather function tool:

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Retrieve the current weather for a given location.",
    "parameters": {
      "type": "object",
      "properties": {
        "location": {
          "type": "string",
          "description": "City and country, for example: Bogotá, Colombia"
        },
        "units": {
          "type": "string",
          "enum": ["celsius", "fahrenheit"],
          "description": "The unit for the returned temperature."
        }
      },
      "required": ["location", "units"],
      "additionalProperties": false
    },
    "strict": true
  }
}

Handling tool calls (Tool calling)

When the model calls a tool in tools, you must execute that tool and return the result. Since tool calling may include zero, one, or multiple calls, best practice is to assume there may be multiple.

Response format

When the model needs to call tools, the response finish_reason is "tool_calls", and message includes a tool_calls array:

{
  "id": "chatcmpl_xxx",
  "model": "openai/gpt-4.1-nano",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": null,
        "tool_calls": [
          {
            "id": "call_abc123",
            "type": "function",
            "function": {
              "name": "get_weather",
              "arguments": "{\"location\":\"Beijing\"}"
            }
          }
        ]
      },
      "finish_reason": "tool_calls"
    }
  ]
}

Each call in the tool_calls array contains:

• id: a unique identifier used when submitting the function result later

• type: the tool type, typically function or custom

• function: the function object

  • name: the function name

  • arguments: JSON-encoded function arguments

Example tool_calls containing multiple tool calls:

[
  {
    "id": "fc_12345xyz",
    "type": "function",
    "function": {
      "name": "get_weather",
      "arguments": "{\"location\":\"Paris, France\"}"
    }
  },
  {
    "id": "fc_67890abc",
    "type": "function",
    "function": {
      "name": "get_weather",
      "arguments": "{\"location\":\"Bogotá, Colombia\"}"
    }
  },
  {
    "id": "fc_99999def",
    "type": "function",
    "function": {
      "name": "send_email",
      "arguments": "{\"to\":\"[e-mail]\",\"body\":\"Hi Andrew\"}"
    }
  }
]

Execute tool calls and append results

Python

for choice in response.choices:
    for tool_call in choice.message.tool_calls or []:
        if tool_call.type != "function":
            continue

        name = tool_call.function.name
        args = json.loads(tool_call.function.arguments)

        result = call_function(name, args)
        input_list.append({
            "role": "tool",
            "name": name,
            "tool_call_id": tool_call.id,
            "content": str(result)
        })

Controlling tool calling behavior (tool_choice)

By default, the model decides when and how many tools to call. You can control tool calling behavior using the tool_choice parameter.

1. Auto: (default) Call zero, one, or multiple tools. tool_choice: "auto"

2. Required: Call one or more tools. tool_choice: "required"

When to use (allowed_tools)

If you want the model to use only a subset of the tool list in a given request—without modifying the tool list you pass in, to maximize prompt caching—you can configure allowed_tools.

"tool_choice": {
    "type": "allowed_tools",
    "mode": "auto",
    "tools": [
        { "type": "function", "function": { "name": "get_weather" } },
        { "type": "function", "function": { "name": "get_time" } }
    ]
}

You can also set tool_choice to "none" to force the model not to call any tools.

Streaming

Streaming tool calling is very similar to streaming normal responses: set stream to true and receive a stream of events.

Python

from openai import OpenAI

client = OpenAI(
  base_url="https://llm.onerouter.pro/v1",
  api_key="<API_KEY>",
)

tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Get the current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country, e.g. Bogotá, Colombia"
                }
            },
            "required": [
                "location"
            ],
            "additionalProperties": False
        }
    }
}]

stream = client.chat.completions.create(
    model="moonshotai/kimi-k2-0905",
    messages=[{"role": "user", "content": "What's the weather like in Paris today?"}],
    tools=tools,
    stream=True
)

for event in stream:
    print(event.choices[0].delta.model_dump_json())

Output events

{"content":"","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":"I'll","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":" check","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":" the","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":" current","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":" weather","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":" in","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":" Paris","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":" for","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":" you","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":".","function_call":null,"refusal":null,"role":"assistant","tool_calls":null}
{"content":"","function_call":null,"refusal":null,"role":"assistant","tool_calls":[{"index":0,"id":"functions.get_weather:0","function":{"arguments":"{\"location\": \", France\"}","name":"get_weather"},"type":"function"}]}

When the model calls one or more tools, an event will be emitted for each tool call where tool_calls.type is not empty:

{
  "content": "",
  "role": "assistant",
  "tool_calls": [
    {
      "index": 0,
      "id": "get_weather:0",
      "function": { "arguments": "", "name": "get_weather" },
      "type": "function"
    }
  ]
}

Below is a snippet showing how to aggregate delta values into the final tool_call object.

Accumulate tool_call content

Python

final_tool_calls = {}

for event in stream:
    delta = event.choices[0].delta
    if delta.tool_calls and len(delta.tool_calls) > 0:
        tool_call = delta.tool_calls[0]
        if tool_call.type == "function":
            final_tool_calls[tool_call.index] = tool_call
        else:
            final_tool_calls[tool_call.index].function.arguments += tool_call.function.arguments

print("Final tool calls:")
for index, tool_call in final_tool_calls.items():
    print(f"Tool Call {index}:")
    print(tool_call.model_dump_json(indent=2))

Accumulated final_tool_calls[0]

{
  "index": 0,
  "id": "get_weather:0",
  "function": {
    "arguments": "{\"location\": \"Paris, France\"}",
    "name": "get_weather"
  },
  "type": "function"
}