Tool Use and Function Calling: Language Models Invoking External Functions

Tool use (function calling) enables language models to invoke external functions — calculators, search engines, code interpreters, databases, and APIs — extending beyond the limitations of parametric knowledge stored in weights. The model generates a structured description of the function call; an external executor runs the function and returns the result, which is appended to the context for the next generation step.

The Tool Use Loop

User query
    ↓
LM generates: {"name": "calculator", "arguments": {"expr": "24 * 365"}}
    ↓
Executor: runs calculator("24 * 365") → 8760
    ↓
Context append: [TOOL_RESULT]: 8760
    ↓
LM continues: "There are 8760 hours in a year."

Toolformer: Self-Supervised Tool Learning (Schick et al., 2023)

Toolformer trains a model to self-supervise when and how to insert tool calls:

1. Sample positions in training text where a tool call might reduce prediction loss
2. Generate candidate API calls via few-shot prompting
3. Filter: keep only calls where executing the tool and inserting the result reduces loss on the following text
4. Fine-tune on the filtered dataset with API calls embedded inline

Tool	Example Use Case	Perplexity Benefit
Calculator	Arithmetic problems in training text	Reduces loss on following numbers
Calendar	Date arithmetic and temporal reasoning	Correct date computations
Wikipedia search	Factual entity lookups	Grounded factual claims
Machine translation	Non-English text processing	Correct multilingual handling
QA system	Knowledge retrieval	Factual question answering

ReAct vs Direct Tool Calling (Yao et al., 2022)

Approach	HotpotQA EM	ALFWorld Success
Standard prompting (no tools)	29.0%	25%
Act-only (tool calls, no reasoning)	28.7%	45%
CoT-only (reasoning, no tools)	28.7%	—
ReAct (reasoning + tool calls)	35.1%	71%

The act-only baseline (tool calls without reasoning) performs similarly to no-tools prompting on multi-hop QA, confirming that reasoning traces are essential for effective tool selection and sequencing.

Structured Output Formats

Function calls require well-formed JSON within the text generation stream:

Format	Mechanism	Parsing
Inline (Toolformer)	Special tokens wrap API call syntax	Token-level detection
Dedicated turn	Entire output is a JSON object	Message-level parsing
JSON schema constrained	Constrained decoding to valid JSON	Grammar-based sampling

Trust and Safety Boundaries

Tool use introduces a trust boundary: the model-generated function call is untrusted input to the executor. Sandbox requirements depend on tool capabilities:

• Read-only tools (search, calculator): low risk; broad access acceptable
• Write tools (database, email): require explicit user authorization per call
• Code execution: requires full sandboxing; never run directly on host

See chain-of-thought for reasoning traces that improve tool selection quality, rag for read-only retrieval-based augmentation, and context-window for how tool results consume the available token budget.