Uncensored AI Atlas
Menu

Developer guide

Tool calling with an OpenAI-compatible LLM API

Published 14 November 2026Facts checked 14 November 20262 min readBy the Atlas editors
Short answer

Describe each function with a JSON Schema in the tools array. When the model wants one, the reply has finish_reason tool_calls and a list of calls with JSON arguments. Run the function yourself, append the result as a tool message with the matching tool_call_id, and call the API again until the model answers in text. Validate arguments and cap the number of rounds.

Test status, 14 November 2026: The Python and Node.js loops were run unchanged against the mock OpenAI-compatible server in this site's repository, which returns a tool call and then a final answer. Tool-calling quality of real models was not tested. Environment: Python 3.12.4 with openai 3.51.0; Node.js 22.19.0 with openai 7.57.0. Not yet run against a live provider endpoint.

How the loop works

Tool calling, also called function calling, lets a model ask your code to do something it cannot do itself, such as look up data or run a calculation. The model never executes anything. A round trip looks like this:

  1. You send messages plus a tools array, each tool with a name, a description and a JSON Schema for its parameters.
  2. The model either answers in text, or returns finish_reason: "tool_calls" with one or more calls, each with an id, a function name and arguments as a JSON string.
  3. Your code validates the arguments, runs the function, and appends two things to the conversation: the assistant message with the calls, and a tool message per call containing the result and the matching tool_call_id.
  4. You call the API again with the longer conversation. Repeat until the model answers in text.

Python

python code
"""A complete tool-calling loop: the model asks for a function, we run it,
send the result back, and the model writes the final answer."""
import json
import os

from openai import OpenAI

client = OpenAI(base_url=os.environ["LLM_BASE_URL"], api_key=os.environ["LLM_API_KEY"], timeout=60.0)
MODEL = os.environ["LLM_MODEL"]

tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Current weather for a city.",
            "parameters": {
                "type": "object",
                "properties": {"city": {"type": "string", "description": "City name, e.g. Lisbon"}},
                "required": ["city"],
                "additionalProperties": False,
            },
        },
    }
]


def get_weather(city: str) -> dict:
    # Replace with a real lookup. Never execute code or shell commands that a model writes.
    return {"city": city, "temperature_c": 19, "conditions": "clear"}


messages = [{"role": "user", "content": "Should I take an umbrella in Lisbon today?"}]

for _ in range(5):  # hard limit on round trips
    reply = client.chat.completions.create(model=MODEL, messages=messages, tools=tools, tool_choice="auto")
    message = reply.choices[0].message
    if not message.tool_calls:
        print(message.content)
        break

    messages.append(message.model_dump(exclude_none=True))
    for call in message.tool_calls:
        if call.function.name != "get_weather":
            result = {"error": f"unknown tool {call.function.name}"}
        else:
            try:
                args = json.loads(call.function.arguments or "{}")
                result = get_weather(str(args["city"]))
            except (json.JSONDecodeError, KeyError) as e:
                result = {"error": f"bad arguments: {e}"}
        messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})
else:
    print("Stopped after 5 rounds without a final answer.")

Node.js

javascript code
// A complete tool-calling loop with the OpenAI Node SDK.
import OpenAI from 'openai'

const client = new OpenAI({ baseURL: process.env.LLM_BASE_URL, apiKey: process.env.LLM_API_KEY })
const model = process.env.LLM_MODEL

const tools = [
  {
    type: 'function',
    function: {
      name: 'get_weather',
      description: 'Current weather for a city.',
      parameters: {
        type: 'object',
        properties: { city: { type: 'string', description: 'City name, e.g. Lisbon' } },
        required: ['city'],
        additionalProperties: false,
      },
    },
  },
]

// Replace with a real lookup. Never execute code or shell commands that a model writes.
const handlers = {
  get_weather: ({ city }) => ({ city, temperature_c: 19, conditions: 'clear' }),
}

const messages = [{ role: 'user', content: 'Should I take an umbrella in Lisbon today?' }]

let answered = false
for (let round = 0; round < 5 && !answered; round++) {
  const reply = await client.chat.completions.create({ model, messages, tools, tool_choice: 'auto' })
  const message = reply.choices[0].message
  if (!message.tool_calls?.length) {
    console.log(message.content)
    answered = true
    break
  }
  messages.push(message)
  for (const call of message.tool_calls) {
    let result
    try {
      const handler = handlers[call.function.name]
      if (!handler) throw new Error(`unknown tool ${call.function.name}`)
      result = handler(JSON.parse(call.function.arguments || '{}'))
    } catch (err) {
      result = { error: err.message }
    }
    messages.push({ role: 'tool', tool_call_id: call.id, content: JSON.stringify(result) })
  }
}
if (!answered) console.log('Stopped after 5 rounds without a final answer.')

Both versions do the same things on purpose:

  • Cap the rounds. A model can keep calling tools; five rounds is a reasonable ceiling for most tasks.
  • Parse arguments defensively. The arguments are a string the model wrote. It can be invalid JSON or miss a required field. Return an error message as the tool result instead of crashing, so the model can try again.
  • Allow-list functions. Only names you defined are run; anything else gets an error result.
  • Never run model-written code or shell commands without a sandbox and a human decision. A refusal-reduced model will follow instructions it finds in documents or web pages, which makes prompt injection through tool results a real risk.

Writing good tool definitions

  • Give each tool one job and a description that says when to use it.
  • Use required and additionalProperties: false in the schema. Some servers enforce the schema; many open-weight stacks do not, which is why validation in your code still matters.
  • Keep the result small and factual. Large tool results use context and cost input tokens on every later round.

Provider and model differences

Whether tool calling works depends on three layers: the model’s training, the chat template, and the server’s parser that turns the model’s text into structured tool_calls. Model cards often document the parser: GLM-4.7-Flash’s card, for example, serves it with a dedicated GLM tool-call parser in vLLM and SGLang. A provider can accept the tools parameter while the model behind it handles tools poorly.

Documented differences we found:

  • MolochAI supports tools and tool_choice, says quality can vary by server, and sends streamed tool arguments whole rather than in fragments.
  • Featherless, Venice and OpenRouter flag tool support per model in their model lists, so check the specific model.
  • In the OpenAI streaming format, arguments can arrive in fragments to concatenate by index; see the streaming guide.

Test tool calls with your own tools before committing to a model. Our coding model page lists which catalogued models’ cards mention tool use, and how to evaluate an LLM for coding covers testing tool calls as part of a task set.

Questions

Does a refusal-reduced model call tools better?

There is no evidence that it does. Tool calling depends on the base model's training and on the server's parser; reducing refusals does not add either.

Can the model run my functions directly?

No. The model only proposes a call with arguments. Your code decides whether to run it, which is where you enforce limits and permissions.

Uncensored AI for coding

Coding criteria and which claims are measured or not.

OpenAI-compatible LLM APIs, and where they differ

What OpenAI-compatible means for uncensored LLM APIs: base URLs, ignored parameters, reasoning fields, usage in streams, tool calls, plus a script to probe any endpoint.

Streaming LLM responses with server-sent events

How LLM API streaming works: the server-sent events format, parsing chunks, the final usage chunk, cancelling a stream, timeouts and errors that arrive mid-stream.

How to evaluate an LLM for coding

A practical method to evaluate an LLM for coding: real tasks, unit tests, pass@k, contamination checks, cost per solved task, latency, tool calls and run logs.