Using the REST API

Everything the CLI does, the REST API does. Use it when the CLI can’t be installed: Windows without WSL, an environment that permits no new binaries, or when your work is already scripted in Python.

Both create the same entities on the same platform, accept the same config overrides, and identify each entity by the same UUID. A project can move between them freely, since an id from one works in the other.

The complete endpoint schema is at api.distillabs.ai/docs. This page is the working guide.

Prerequisites

pip install requests pyyaml
export DL_USERNAME="you@example.com"
export DL_PASSWORD="…"

The API is https://api.distillabs.ai, with Cognito client id 4569nvlkn8dm0iedo54nbta6fd in eu-central-1.

Access tokens last one hour, which is shorter than a generation or training run, so the helpers below re-authenticate per request rather than holding a token.

Helpers

Every example on this page assumes this block.

import json
import os
import time
from pathlib import Path

import requests
import yaml

PLATFORM_URL = os.getenv("DL_PLATFORM_URL", "https://api.distillabs.ai")
COGNITO_CLIENT_ID = os.getenv("DL_COGNITO_CLIENT_ID", "4569nvlkn8dm0iedo54nbta6fd")
COGNITO_URL = "https://cognito-idp.eu-central-1.amazonaws.com"
POLL_INTERVAL_SECONDS = 20

# A staging response names each file by extension; a create body names it by
# field. These maps are the only place that difference is spelled out.
STAGING_FIELDS = {
    "train_data": "train_data_jsonl",
    "test_data": "test_data_jsonl",
    "config": "config_yaml",
    "job_description": "job_description_json",
    "unstructured_data": "unstructured_data_jsonl",
    "model": "model_tar",
}
PREPARED_TRACES_FIELDS = {
    "traces_jsonl": "traces_jsonl",
    "config": "config_yaml",
    "job_description_json": "job_description_json",
    "test_jsonl": "test_jsonl",
}


def auth():
    response = requests.post(
        COGNITO_URL,
        headers={
            "X-Amz-Target": "AWSCognitoIdentityProviderService.InitiateAuth",
            "Content-Type": "application/x-amz-json-1.1",
        },
        data=json.dumps({
            "AuthParameters": {
                "USERNAME": os.environ["DL_USERNAME"],
                "PASSWORD": os.environ["DL_PASSWORD"],
            },
            "AuthFlow": "USER_PASSWORD_AUTH",
            "ClientId": COGNITO_CLIENT_ID,
        }),
    )
    response.raise_for_status()
    return {"Authorization": response.json()["AuthenticationResult"]["AccessToken"]}


def raise_with_body(response):
    """raise_for_status hides the response body, which is where validation
    errors live. Print it before raising."""
    try:
        response.raise_for_status()
    except requests.exceptions.HTTPError:
        print(f"{response.status_code}: {response.text[:2000]}")
        raise


def get(path):
    response = requests.get(f"{PLATFORM_URL}{path}", headers=auth())
    raise_with_body(response)
    return response.json()


def post(path, body):
    response = requests.post(
        f"{PLATFORM_URL}{path}",
        data=json.dumps(body),
        headers={"content-type": "application/json", **auth()},
    )
    raise_with_body(response)
    return response.json()


def stage(staging_route, files, fields=STAGING_FIELDS):
    """PUT each local file to its presigned URL and return the create body."""
    urls = get(staging_route)
    body = {}
    for field, filepath in files.items():
        url = urls[fields[field]]
        requests.put(url, data=Path(filepath).read_bytes()).raise_for_status()
        body[field] = url
    return body


def config_of(collection, entity_id):
    """Fetch a parent's whole config as a dict. An override replaces the config
    rather than merging into it, so every override starts here. Costs nothing."""
    urls = get(f"/{collection}/{entity_id}/download-metadata")
    if urls["config_url"] is None:
        status = get(f"/{collection}/{entity_id}/status")["status"]
        raise RuntimeError(f"{collection}/{entity_id}: not available yet (status {status})")
    response = requests.get(urls["config_url"])
    response.raise_for_status()
    return yaml.safe_load(response.text)


def poll(collection, entity_id, timeout_seconds, status_field="status"):
    deadline = time.time() + timeout_seconds
    while time.time() < deadline:
        status = get(f"/{collection}/{entity_id}/status")[status_field]
        if status == "JOB_SUCCESS":
            return
        if status in ("JOB_FAILURE", "JOB_STOPPED"):
            logs = get(f"/{collection}/{entity_id}/logs")["logs"]
            raise RuntimeError(f"{collection}/{entity_id} {status}:\n{logs[-4000:]}")
        print(f"{collection}/{entity_id}: {status}", flush=True)
        time.sleep(POLL_INTERVAL_SECONDS)
    raise TimeoutError(f"{collection}/{entity_id} did not finish in {timeout_seconds}s")

Keep raise_with_body. Validation errors arrive in the response body, and raise_for_status() throws them away, so a 400 that names the exact row and field at fault reads as a bare HTTPError.

The entity model

Stage	Submit	Read
Trace processing	`POST /prepared-traces`, then `POST /seed-datasets/from-prepared-traces`	`GET /seed-datasets/<id>/{status,logs,metrics,download,download-metadata}`
Job input	`POST /seed-datasets`	`GET /seed-datasets/<id>/{status,logs,metrics,download,download-metadata}`
Teacher evaluation	`POST /teacher-evaluations/from-seed-datasets`	`GET /teacher-evaluations/<id>/{status,logs,metrics,download-metadata}`
Synthetic data generation	`POST /training-datasets/from-seed-datasets`	`GET /training-datasets/<id>/{status,logs,metrics,sample,download,download-metadata}`
Model training	`POST /slms/from-training-datasets`	`GET /slms/<id>/{status,logs,metrics,download,download-metadata}`
Deployment	`POST /deployments/from-slms`	`GET /deployments/<id>/{status,endpoint,logs}`, `DELETE /deployments/<id>`

The /uploads, /staging-uploads-s3-urls, /teacher-evaluations/from-uploads and /training-datasets/from-uploads paths answer 410 Gone, naming the route to use instead. A 410 means the path is the problem, not the payload.

Staging files

Staging is a three-step exchange: GET /staging-<kind>-s3-urls returns a presigned PUT URL per file, each file is PUT to its URL, and those URLs are posted back as the create body. The stage() helper does all three.

The staging response names each file by extension (train_data_jsonl) while the create body names it by field (train_data). The helper hides that difference.

Staged bundles expire after seven days if they never become an entity.

Creating a seed dataset

body = stage("/staging-seed-datasets-s3-urls", {
    "train_data": "train.jsonl",
    "test_data": "test.jsonl",
    "config": "config.yaml",
    "job_description": "job_description.json",
})
seed_dataset_id = post("/seed-datasets", body)["id"]

Omit unstructured_data for tasks that don’t use it.

POST /seed-datasets validates the bundle and returns 400 with the validation error when it fails. That’s this backend’s dry run. The route is metered, but only a successful create spends a credit, so validating a broken bundle repeatedly is free. A 402 here means the balance was already zero before the bundle was read.

Trace processing

body = stage("/staging-prepared-traces-s3-urls", {
    "traces_jsonl": "traces.jsonl",
    "config": "config.yaml",
    "job_description_json": "job_description.json",
    # Add "test_jsonl" only for a curated test set - supplying one replaces
    # the generated test split and makes num_traces_as_testing_base inert.
}, PREPARED_TRACES_FIELDS)
prepared_traces_id = post("/prepared-traces", body)["id"]

seed_dataset_id = post(
    "/seed-datasets/from-prepared-traces", {"from": prepared_traces_id}
)["id"]
poll("seed-datasets", seed_dataset_id, 60 * 45)

Submitting the rest of the pipeline

Every job is created by posting the id of the entity before it. Nothing uploads at this point - the parent’s files are already on the platform.

te_id = post("/teacher-evaluations/from-seed-datasets", {"from": seed_dataset_id})["id"]
poll("teacher-evaluations", te_id, 60 * 30)

dataset_id = post("/training-datasets/from-seed-datasets", {"from": seed_dataset_id})["id"]
poll("training-datasets", dataset_id, 60 * 90)

slm_id = post("/slms/from-training-datasets", {"from": dataset_id})["id"]
poll("slms", slm_id, 60 * 90)

deployment_id = post("/deployments/from-slms", {"from": slm_id})["id"]
poll("deployments", deployment_id, 60 * 40, "deployment_status")

Stage	Collection	Typical timeout
Trace processing	`seed-datasets`	45 min
Teacher evaluation	`teacher-evaluations`	30 min
Synthetic data generation	`training-datasets`	90 min
Model training	`slms`	90 min
Deployment	`deployments`	40 min

Overrides

The four job creates each accept optional config and job_description objects inline, alongside from. There’s no staging step for either.

config = config_of("seed-datasets", seed_dataset_id)
config["synthgen"]["generation_target"] = 64

dataset_id = post("/training-datasets/from-seed-datasets", {
    "from": seed_dataset_id,
    "config": config,
})["id"]

Each override replaces the parent’s file whole. Anything you leave out reverts to a library default rather than the parent’s value, so an override is always read-edit-resend, never hand-built. Two shapes to know:

{"config": {"base": {"student_model_name": …}}} is a 400. base is required and has no default, so a config naming only one field under it is not a config.
A config carrying a valid base but omitting synthgen or tuning is accepted, and those sections silently take defaults. Dropping one field from a section you do send reverts that field the same way. Nothing errors.

See How the platform works.

GET /<collection>/<id>/download-metadata returns presigned URLs for an entity’s config.yaml and job_description.json at no credit cost. It fills only once the entity’s own job has finished - while it runs, both are null. The config_of helper raises a message naming the entity and its status rather than passing a None to requests.get, which otherwise fails with Invalid URL 'None': No scheme supplied and sends you hunting for a bug in your URL building.

GET /slms/<id>/download-metadata returns a third field, model_client_url. Every other entity returns exactly two.

Credits

balances = get("/credits/endpoints")["balances"]
print(balances["training_datasets_from_seed_datasets_post"])

The read is free and answers at zero balance. Routes the platform never charges for are absent from the response, so read an unfamiliar key with .get() rather than assuming a zero. A submission against an exhausted route fails 402.

The route table and starting balances: How the platform works.

Reading results

teacher = get(f"/teacher-evaluations/{te_id}/metrics")["teacher_performance"]

slm_metrics = get(f"/slms/{slm_id}/metrics")
base, tuned = slm_metrics["base_model_performance"], slm_metrics["tuned_model_performance"]

Each metrics response also carries a presigned *_download_url for the per-example predictions. Those URLs expire after an hour.

Reading a training dataset:

rows = get(f"/training-datasets/{dataset_id}/sample")["rows"]     # free, 128 rows max
size = get(f"/training-datasets/{dataset_id}/metrics")["train_data_size_bytes"]

sample returns at most 128 train rows drawn from the first 384, and never test rows. GET /training-datasets/<id>/download returns them all and is metered on training_datasets_download_get, which starts at zero.

What the numbers mean: Metrics.

Deployment

endpoint = get(f"/deployments/{deployment_id}/endpoint")
url, api_key = endpoint["url"], endpoint["api_key"]

# ... use it ...

requests.delete(f"{PLATFORM_URL}/deployments/{deployment_id}", headers=auth())

Before the job reaches JOB_SUCCESS, endpoint answers {"url": null, "api_key": null}, so poll the status rather than probing the endpoint.

A deployment is a session, not a permanent endpoint. It stops after six hours, or after one hour with no traffic, and it can’t be restarted. It bills until that idle timeout, so delete it when you’re finished. Replacing a stopped one spends another deployments_from_slms_post credit, so have your test inputs ready before you create it.

See distil labs inference for querying it.

Inference endpoints

An inference endpoint is an OpenAI-compatible gateway that fronts the model you already run in production and keeps a copy of every call it serves. Those copies are traces, and trace processing turns them into a dataset for the training pipeline. Later, a second endpoint serves the trained model with the first model as its fallback. The same thing from the terminal: CLI reference.

endpoint = post("/inference-endpoints", {
    "name-prefix": "support",
    "fallback": {"model": "openai/gpt-4.1-mini"},
    "trace-sample-rate": 1,
})
name = endpoint["unique_endpoint_name"]     # eg. "support-yeOdAS"

name-prefix is appended with a randomized suffix and returned as unique_endpoint_name, which is what every other route takes and what a request body carries.

fallback.model is an OpenRouter model slug in owner/model form, named from the model directory. The CLI reference lists the slugs we run today if you want a starting point.

trace-sample-rate is the fraction of calls the endpoint records, 0 to 1. Send 1 unless your traffic is very high. An endpoint created without it records one call in a hundred, and the response reports what an endpoint does as trace_sampling_rate.

primary puts a model of your own in front of the fallback, usually a deployment of the trained model:

served = get(f"/deployments/{deployment_id}/endpoint")      # {"url": ..., "api_key": ...}
endpoint = post("/inference-endpoints", {
    "name-prefix": "support-slm",
    "fallback": {"model": "openai/gpt-4.1-mini"},
    "primary": {"url": served["url"], "api-key": served["api_key"]},
    "trace-sample-rate": 1,
})

primary.url is the base URL of an OpenAI-compatible server, without /v1 since the endpoint appends /v1/chat/completions itself, and primary.api-key the key that authenticates it; both or neither. The endpoint calls the primary first and falls back whenever the primary fails, a timeout included. primary.readiness-gate-timeout-ms is optional and for internal use. An endpoint can’t be changed after creation, and the response carries primary_url. See Serving behind the endpoint.

endpoints = get("/inference-endpoints")             # newest first
endpoint = get(f"/inference-endpoints/{name}")      # one, by unique name

POST /inference-endpoints is metered (inference_endpoints_post) and answers 402 once the balance for it is spent, like the other metered routes above.

API keys

Before you can use your endpoint you need to create an API key for it and attach it to your inference endpoint:

key = post("/api-keys", {"api-key-name": "support-prod"})
secret = key["secret"]      # returned by this response and by nothing else

Store secret before moving on as it can not be retrieved again. GET /api-keys lists ids, names and creation dates but never the secret itself. You can attach several API keys to a single inference endpoint so if you lose your key, simply create a new one and delete the old one. Note that the total number of API keys you can have in your account is limited to a maximum of 5.

A key authenticates nothing until it is linked. PUT links it, DELETE on the same path unlinks it, and both answer 204. DELETE /api-keys/<name> revokes the key everywhere it was linked.

requests.delete(
    f"{PLATFORM_URL}/inference-endpoints/{name}/api-keys/support-prod", headers=auth()
)
requests.delete(f"{PLATFORM_URL}/api-keys/support-prod", headers=auth())

Key changes take up to a minute to propagate. A 409 from either link route means the endpoint’s datastore has not caught up with a write yet, usually a create moments earlier, and a link that answered 204 can take a moment longer before the endpoint honours the key. Wait and retry rather than treating either as a failure, and give a new key that minute before you move traffic onto it or revoke the key it replaces.

Calling the endpoint

Our OpenAI compatible inference endpoints are hosted in its own distillabs.ai subdomain and can be called as follows:

requests.post(
    "https://inference.distillabs.ai/v1/chat/completions",
    headers={"Authorization": f"Bearer {secret}", "Content-Type": "application/json"},
    json={"model": name, "messages": [{"role": "user", "content": "…"}]},
)

The model field carries the unique endpoint name. Everything else is an ordinary chat completions request, so any OpenAI-compatible client works against https://inference.distillabs.ai/v1.

Reading the traces

def traces(endpoint_name):
    """Yield an endpoint's traces, newest first, a page at a time."""
    cursor = None
    while True:
        query = f"?{urlencode({'pagination-cursor': cursor})}" if cursor else ""
        page = get(f"/inference-endpoints/{endpoint_name}/traces{query}")
        yield from page["traces"]
        cursor = page["pagination_cursor"]
        if cursor is None:
            return

Terminate on pagination_cursor being null or when you’ve accumulated the number of traces you need. The cursor is opaque, so pass it back as given rather than parsing it.

Use from-start-time and to-start-time narrow the window, as ISO 8601 timestamps. Sending neither leaves the default, which is 90 days.

Each trace is the platform’s record of one call. input and output hold the request and the response as JSON strings, and metadata carries the HTTP status and source, which names whether the fallback or the primary answered. Endpoint records to trace inputs has the full shape and the conversion into a trace processing input.