Tool calling data preparation

The example on this page makes a pizza: given the recipe and the current state of the pizza, the model calls the next action needed to finish it.

The shared rules for the input directory, row shapes and validation are in Overview. This page covers what’s specific to tool calling.

Job description

What you expect the model to do, in the words you’d use to prompt an LLM. Tool calling needs two fields:

task_description. The task itself.
tools. Every tool the model can call, in OpenAI function-calling format, with unique names. Every call in your training and test data validates against these schemas, so a call to a tool you didn’t declare fails the job.

llm_as_a_judge_instructions isn’t valid here. Tool calls are scored against the reference call, so there’s no judge to instruct.

Constrain arguments in the schema wherever the real tool constrains them. The enum on add_topping and the minimum/maximum on set_oven_temp below are what stop the teacher generating calls your tool rejects.

{
  "task_description": "Respond with the next tool call to make a pizza",
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "take_out_of_oven",
        "description": "Remove the pizza from the oven",
        "parameters": {
          "type": "object",
          "properties": {},
          "required": [],
          "additionalProperties": false
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "put_in_oven",
        "description": "Place the pizza in the oven",
        "parameters": {
          "type": "object",
          "properties": {},
          "required": [],
          "additionalProperties": false
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "set_oven_temp",
        "description": "Set the oven temperature in degrees Celsius",
        "parameters": {
          "type": "object",
          "properties": {
            "degrees_celsius": {
              "type": "integer",
              "description": "Temperature in degrees Celsius",
              "minimum": 50,
              "maximum": 300
            }
          },
          "required": ["degrees_celsius"],
          "additionalProperties": false
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "wait",
        "description": "Wait for a specified number of seconds",
        "parameters": {
          "type": "object",
          "properties": {
            "seconds": {
              "type": "integer",
              "description": "Number of seconds to wait",
              "minimum": 1
            }
          },
          "required": ["seconds"],
          "additionalProperties": false
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "add_topping",
        "description": "Add a topping ingredient to the pizza",
        "parameters": {
          "type": "object",
          "properties": {
            "ingredient": {
              "type": "string",
              "description": "The topping ingredient to add",
              "enum": [
                "mozzarella",
                "gorgonzola",
                "Parmigiano-Reggiano",
                "ricotta",
                "rucola",
                "pineapple",
                "ham",
                "salami",
                "jalapeños",
                "onions",
                "black olives",
                "green olives",
                "anchovies",
                "tuna",
                "mushrooms"
              ]
            }
          },
          "required": ["ingredient"],
          "additionalProperties": false
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "add_sauce",
        "description": "Add sauce to the pizza base",
        "parameters": {
          "type": "object",
          "properties": {
            "sauce": {
              "type": "string",
              "description": "The type of sauce to add",
              "enum": [
                "tomato",
                "alfredo"
              ]
            }
          },
          "required": ["sauce"],
          "additionalProperties": false
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "fold",
        "description": "Fold the pizza in half",
        "parameters": {
          "type": "object",
          "properties": {},
          "required": [],
          "additionalProperties": false
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "cut",
        "description": "Cut the pizza into slices",
        "parameters": {
          "type": "object",
          "properties": {},
          "required": [],
          "additionalProperties": false
        }
      }
    },
    {
      "type": "function",
      "function": {
        "name": "serve",
        "description": "Serve the finished pizza",
        "parameters": {
          "type": "object",
          "properties": {},
          "required": [],
          "additionalProperties": false
        }
      }
    }
  ]
}

Training data

Each row is a messages conversation: a user turn holding the recipe and the current state, and an assistant turn whose tool_calls array holds the expected call. The assistant turn omits content, and an empty string is also accepted.

arguments is a real JSON object, not a JSON-encoded string. That’s the HuggingFace format, and it differs from OpenAI’s chat completions, where arguments is a string. The schemas in job_description.json still use the OpenAI shape, and only the emitted call changes.

Aim for 20+ diverse rows. Every tool needs examples, and the ones that take arguments need several, so the model learns which argument goes with which state rather than which tool comes next.

{"messages": [{"role": "user", "content": "Margherita recipe: spread tomato sauce on the base, add fresh mozzarella, bake at 250°C for 9-12 minutes. Margherita assembled. Oven heating to 250°C, currently at 100°C."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "wait", "arguments": {"seconds": 150}}}]}]}
{"messages": [{"role": "user", "content": "Quattro Formaggi recipe: spread alfredo sauce on the base, add mozzarella, gorgonzola, Parmigiano-Reggiano, and ricotta, bake at 220°C for 9 minutes. Quattro Formaggi ready to bake. Oven heated to 220°C."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "put_in_oven", "arguments": {}}}]}]}
{"messages": [{"role": "user", "content": "Ham recipe: spread tomato sauce on the base, add mozzarella and diced ham, bake at 220°C for 10 minutes. Ham pizza with tomato sauce spread to the edges."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "add_topping", "arguments": {"ingredient": "mozzarella"}}}]}]}
{"messages": [{"role": "user", "content": "Marinara recipe: spread tomato sauce on the base, bake at 230°C for 10 minutes. Marinara base stretched and bare. Oven is cold."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "set_oven_temp", "arguments": {"degrees_celsius": 230}}}]}]}
{"messages": [{"role": "user", "content": "Diavola recipe: spread tomato sauce on the base, add mozzarella and spicy salami, bake at 240°C for 9 minutes. Diavola dough stretched, nothing on it yet."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "add_sauce", "arguments": {"sauce": "tomato"}}}]}]}
{"messages": [{"role": "user", "content": "Meat calzone recipe: on half of the dough circle, spread tomato sauce, add mozzarella, salami, and ham, fold over and seal edges, bake at 215°C for 16 minutes. Meat calzone with all fillings on one half of the dough."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "fold", "arguments": {}}}]}]}
{"messages": [{"role": "user", "content": "Anchovy recipe: spread tomato sauce on the base, add mozzarella and anchovies, bake at 240°C for 8 minutes. Anchovy pizza in the oven for 8 minutes. Crust golden and cheese bubbling."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "take_out_of_oven", "arguments": {}}}]}]}
{"messages": [{"role": "user", "content": "Margherita recipe: spread tomato sauce on the base, add fresh mozzarella, bake at 250°C for 9-12 minutes. Margherita out of the oven, resting whole on the board."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "cut", "arguments": {}}}]}]}
{"messages": [{"role": "user", "content": "Meat calzone recipe: on half of the dough circle, spread tomato sauce, add mozzarella, salami, and ham, fold over and seal edges, bake at 215°C for 16 minutes. Hearty meat calzone steaming on plate, already sliced."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "serve", "arguments": {}}}]}]}

Test data

Same format as train.jsonl, held out for evaluation. This is what every score you’ll see is measured against: the teacher evaluation, the base student and the tuned student.

Two rules fail validation if you break them:

No row may be identical to a training row.
Every call has to validate against your tools schemas.

Cover each tool at least once. States that look similar but need different calls are what this set is for, since a pizza in the oven and a pizza out of the oven differ by one clause in the prompt.

{"messages": [{"role": "user", "content": "Capricciosa recipe: spread tomato sauce on the base, add mozzarella, ham and black olives, bake at 230°C for 11 minutes. Capricciosa assembled. Oven climbing through 180°C toward 230°C."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "wait", "arguments": {"seconds": 120}}}]}]}
{"messages": [{"role": "user", "content": "Marinara recipe: spread tomato sauce on the base, bake at 230°C for 10 minutes. Marinara topped and ready. Oven heated to 230°C."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "put_in_oven", "arguments": {}}}]}]}
{"messages": [{"role": "user", "content": "Hawaiian recipe: spread tomato sauce on the base, add mozzarella and pineapple and ham, bake at 230°C for 12 minutes. Hawaiian base sauced, no cheese on it yet."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "add_topping", "arguments": {"ingredient": "mozzarella"}}}]}]}
{"messages": [{"role": "user", "content": "Funghi recipe: spread tomato sauce on the base, add mozzarella and mushrooms, bake at 235°C for 10 minutes. Funghi in the oven for 10 minutes. Crust golden."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "take_out_of_oven", "arguments": {}}}]}]}
{"messages": [{"role": "user", "content": "Quattro Stagioni recipe: spread tomato sauce on the base, add mozzarella, mushrooms, ham and black olives, bake at 235°C for 11 minutes. Quattro Stagioni cooling on the board, still whole."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "cut", "arguments": {}}}]}]}
{"messages": [{"role": "user", "content": "Tuna recipe: spread tomato sauce on the base, add mozzarella, tuna and onions, bake at 240°C for 10 minutes. Tuna pizza dough stretched on the peel, bare."}, {"role": "assistant", "tool_calls": [{"type": "function", "function": {"name": "add_sauce", "arguments": {"sauce": "tomato"}}}]}]}

Unstructured data (optional)

Unstructured data steers the teacher toward diverse, domain-specific examples. It can be documentation, unlabelled examples, or industry material covering the same ground.

Here it’s a set of pizza recipes. The teacher draws new states and sequences from them, so the generated examples cover pizzas your handwritten rows never mention. Rows carry a single context field.

{"context": "Classic Margherita Pizza: Start by stretching your dough into a 12-inch circle. Spread 3-4 tablespoons of tomato sauce evenly, leaving a 1-inch border for the crust. Tear fresh mozzarella into small pieces and distribute over the sauce. Drizzle with olive oil, add a pinch of salt, and bake in a preheated oven at 250°C for 9-12 minutes until the crust is golden and cheese is bubbly. Finish with fresh basil leaves after removing from oven."}
{"context": "Quattro Formaggi (Four Cheese) Pizza: This white pizza celebrates Italian cheeses. Begin with a base of creamy alfredo or bechamel sauce instead of tomato. Layer four cheeses: mozzarella for stretch, gorgonzola for tang, Parmigiano-Reggiano for sharpness, and dollops of ricotta for creaminess. The key is balance - use mozzarella as the base, then add the others sparingly. Bake at 220°C for about 9 minutes. The different melting points create a complex texture."}
{"context": "Calzone Preparation Technique: A calzone is essentially a folded pizza that creates a pocket of delicious filling. Roll your dough into an oval shape, slightly thicker than for regular pizza. Place fillings only on one half, leaving a 1-inch border. Common fillings include ricotta, mozzarella, Italian sausage, and vegetables. Brush the edges with water, fold the empty half over, and seal by pressing and crimping the edges. Cut 2-3 small vents on top to release steam. Bake at 215°C for 15-18 minutes until golden brown."}
{"context": "Hawaiian Pizza Assembly: Despite the controversy, Hawaiian pizza has its devoted fans. Start with a traditional tomato sauce base, spreading it evenly across the dough. Add a layer of mozzarella cheese, then distribute cubed ham evenly across the surface. The key to good Hawaiian pizza is properly preparing the pineapple - use fresh or well-drained canned chunks, and pat them dry to prevent excess moisture. Arrange pineapple pieces between the ham. Bake at 230°C for 10-12 minutes."}
{"context": "Pizza Timing and Temperature Guide: Different pizza styles require different baking approaches. Neapolitan pizzas cook best at extremely high temperatures (450°C+) for just 60-90 seconds. New York style benefits from 280-300°C for 5-7 minutes. Thick crust or deep dish pizzas need lower temperatures (200-220°C) for 15-25 minutes to cook through without burning the top. Always preheat your oven fully - at least 30 minutes for home ovens. Use a pizza stone or steel if available, preheating it with the oven for optimal crust development."}

Config

The task type, plus the two models:

base:
  task: tool-calling-closed-book
  student_model_name: Qwen3-1.7B
  teacher_model_name: openai.gpt-oss-120b

Tool calling restricts both. Students are limited to the Qwen3, Qwen3.5, Llama 3-family, LFM2/LFM2.5, FunctionGemma and Gemma 4 families, and teachers to those marked in the tool-calling column of Supported models. The default teacher, openai.gpt-oss-120b, works here.

Every other field has a default. See Config file for the full table and Supported models for the values you can use.

Create the seed dataset, which validates your files at the same time:

distil seed-dataset create --data ./your-data-dir

Then run teacher evaluation.