Troubleshooting

Find your symptom, or search this page for the exact error text you were given.

Quick index

Symptom	Section
A job has sat in JOB_PENDING for a long time	Jobs that don’t seem to be moving
A job failed and I need the cause	A job failed
Insufficient credits for this operation, or a 402	Credits
Only the following tasks are supported: [...]	The seed dataset was rejected
Required file not found: ...	The seed dataset was rejected
1 validation error for JobInputParser	The seed dataset was rejected
I changed one config field and other settings reverted	My config changes didn’t stick
Not logged in or session expired	Authentication
Invalid email or password.	Authentication
Email confirmation required.	Authentication
Timed out waiting for browser authorization	Authentication
distil deployment endpoint returned nulls but exited 0	The endpoint isn’t answering
The model answers badly, but the eval scores were fine	Good scores, bad answers
I lost an entity id	I lost an id
Not enough disk space to download the model	Downloads
Failed to upload file: ...	Downloads
Failed to parse YAML file: / Failed to parse JSON file:	The seed dataset was rejected
Training ran out of memory	Out of memory
I can’t install the CLI	Setup

Jobs that don’t seem to be moving

JOB_PENDING means the job was accepted and is waiting for capacity. That wait is normal and it is not counted against you.

distil <group> status --output json <id> | jq -r .status

Poll every 20 seconds or so. The end-to-end times to plan for are in the stages table: up to 45 minutes for trace processing, 30 for teacher evaluation, and 90 each for generation and training. Those are whole-job budgets and they include queue time, so a job that has been pending for a while but is still returning JOB_PENDING is fine.

If a job is still pending well past the budget for its stage, contact contact@distillabs.ai with the entity id. There is no cancel command, so don’t wait on one.

JOB_STOPPED means the platform stopped the job deliberately. You can’t produce this state yourself. Treat it like JOB_FAILURE and read the log.

A job failed

distil <group> logs --output json <id> | jq -r .logs

Search for the first error, not the last one. A crash often unwinds into a second, unrelated error, so an out-of-memory failure can finish with a pickling error thousands of characters after the OutOfMemoryError that actually caused it. The end of the log is usually the wrong place to look.

A failed job produces no metrics and still spends its credit. Fix the cause and submit again from the same parent: the failed entity is spent, and the retry is a new one.

Credits

Insufficient credits for this operation

This is an HTTP 402, surfaced by the CLI as a TokensExhaustedError. Credits are metered per endpoint rather than from one pool, so running out on one route says nothing about the others.

distil credits-balance

Read that against your whole plan, not just the next command: N training runs need N credits. See How the platform works for the route table and what each new account starts with.

Two routes start at zero: training_datasets_download_get and slms_post. The stages they gate are unavailable until you ask for a grant.

To get more, email contact@distillabs.ai.

Check your training balance before generating, not after. A generation run with no training credit left spends a credit on data you then can’t train on.

The seed dataset was rejected

distil seed-dataset create validates your directory before creating anything, and a rejection costs no credit. The message names the cause.

Required file not found: test.jsonl in input-dir

A file is missing from the directory. This is caught locally and nothing uploads. See the input directory.

Only the following tasks are supported: ['classification', 'question-answering-open-book', …]

base.task isn’t a task the platform has. The valid values are in the task table.

1 validation error for JobInputParser
job_input.1.data.question-answering.train_dataset.0.messages
  Field required [type=missing, input_value={'wrong': 'shape'}, input_type=dict]

A row is the wrong shape, named down to the row index. Check it against the row format for your task.

Failed to parse YAML file: <path>
Failed to parse JSON file: <path>

config.yaml or job_description.json isn’t valid. A trailing comma in JSON is the usual cause.

Each of these exits 1 and creates nothing, so fixing and re-running is free.

My config changes didn’t stick

An override replaces config.yaml and job_description.json whole. Anything you leave out reverts to the library default, not to the parent’s value, and nothing errors.

A config carrying a complete base but no synthgen or tuning section is accepted, and those sections silently revert. Submitted against a parent that set generation_target: 512, output_is_json: true and per_device_train_batch_size: 8, a base-only override ran with generation_target: 10000, output_is_json: false and per_device_train_batch_size: 1.

Always read the parent’s config back, change one field, and send the whole file:

distil <group> download-metadata -d ./iter-2 <id>   # free
# edit one field in ./iter-2/config.yaml
distil <group> create-from-<parent> --config ./iter-2/config.yaml <parent-id>

To check what a submission actually ran with, diff it:

distil training-dataset download-metadata -d ./check <training-dataset-id>
diff ./iter-2/config.yaml ./check/config.yaml

Authentication

Not logged in or session expired. Run `distil login` to authenticate or `distil register` to create an account.

Run distil auth (distil login is an alias). If you have no account yet, distil signup (aliases register, join).

Invalid email or password.

Wrong credentials for distil auth --email <email> --password <password>.

Email confirmation required. Please check your email and confirm your account.

Your account exists but isn’t confirmed. Open the confirmation email, then authenticate again.

Timed out waiting for browser authorization

The browser flow didn’t complete. On a machine with no browser, authenticate headlessly instead:

distil auth --email <email> --password <password>

See Account and authentication.

The endpoint isn’t answering

distil deployment endpoint answers {"url": null, "api_key": null} and exits 0 while the deployment is still coming up. Poll the status rather than probing the endpoint:

distil deployment status --output json <deployment-id> | jq -r .deployment_status

Deployments report deployment_status and endpoint_status, and have no status field at all.

If it answered before and has stopped: a deployment is a session, not a permanent endpoint. It stops after six hours, or after one hour with no traffic, and it can’t be restarted. Create a new one, which comes with a new URL and a new key.

Good scores, bad answers

Your evaluation metrics looked fine, but the deployed model produces reasoning text, wrong formatting, or noticeably worse answers than the scores promised.

You are almost certainly calling it wrong. A hand-built chat-completions request fails quietly: the endpoint answers 200 and the quality just drops.

Each model ships with its own client, generated at training time, which reproduces the exact setup the model was evaluated under: the trained system prompt, temperature=0, thinking disabled, tool_choice="required" for tool-calling models, and QA context inlined as <context>…</context>.

Fetch the client, then point it at your deployment with the URL and API key from distil deployment endpoint <deployment-id>:

distil slm download-metadata --destination ./model <slm-id>
uv run ./model/model_client.py --base-url https://<endpoint>/v1 --api-key <api-key> \
  --conversation '[{"role": "user", "content": "..."}]'

See distil labs inference.

If you’re already using the client, check that the scores you’re comparing against came from the same test set, and remember the ±0.03 noise band in Metrics.

I lost an id

Every entity records its parent, so nothing is unrecoverable.

distil <group> list --output json

Returns one object per entity, newest first, with id, created_at and the parent’s id under its own key (seed_dataset_id, training_dataset_id, slm_id). list carries no status, so checking state costs one status call per id.

distil <group> show <id> walks backwards from a child to its parent.

Downloads

Not enough disk space to download the model. Required: <size>, available: <size>

distil slm download writes a tarball of several gigabytes, about 1.2 GB for a Qwen3-0.6B run, and checks free space before it starts. Free some up, or fetch just the client with distil slm download-metadata, which is a few kilobytes.

Failed to upload file: <status>

A staged file didn’t reach object storage. Re-run the create. Nothing partial is kept.

A download of something the job hasn’t produced yet names the reason and exits 1, so read the message: SLM is still training, SLM training failed, The model is not available for this SLM.

Staged files that never became an entity are deleted after seven days and have to be supplied again. Once the entity exists it owns its own copy.

Out of memory

Apply these in order. Each costs more speed or quality than the one before.

1. Lower tuning.per_device_train_batch_size. Halve it, down to a floor of 1. Raise num_train_epochs alongside it.
2. tuning.memory_optimized_training: true. Activation offloading and gradient checkpointing. Significantly slower.
3. tuning.use_qlora: true with memory_optimized_training: true. A 4-bit base model, roughly 3x less VRAM.
4. Drop the longest ~1% of training examples. The long tail drives peak memory.

The first three are config changes that go through an ordinary override. Only the fourth touches the data, which means a new seed dataset. See Model training.

Still stuck

Email contact@distillabs.ai with the entity id and the command you ran.