How the platform works

This page describes how the platform behaves. It’s true whether you drive it with the CLI or the REST API, since those only differ in how you phrase a request.

Entities and jobs

Each stage of the pipeline produces one entity, and each entity is created either from files you supply or by running a job over the entity before it. That chain is the pipeline:

PreparedTraces ─► SeedDataset ─┬─► TeacherEvaluation
                               └─► TrainingDataset ─► SLM ─► Deployment

A seed dataset is the job input. It comes either from trace processing or directly from a prepared directory. Everything downstream is created by naming the id of its parent, so entity ids are the currency of the whole pipeline. Keep each one as you go, because every later read resolves against it.

If you do lose one, nothing is gone: every entity records its parent, so show walks backwards and a list filtered on the parent field walks forwards.

Supplying files is a separate step from creating the entity. Files that never became an entity are deleted after seven days and have to be supplied again. Once the entity exists it owns its own copy and is unaffected.

Job status

Status	Meaning
JOB_NOT_STARTED	accepted, not yet scheduled
JOB_PENDING	scheduled, waiting for capacity
JOB_RUNNING	running
JOB_SUCCESS	finished; outputs readable
JOB_FAILURE	failed
JOB_STOPPED	stopped deliberately

Deployments report deployment_status and endpoint_status instead of status.

Runs take up to hours. Creating a job returns its id at once, so submit in the foreground and poll in the background. A creation that succeeds tells you nothing about the job, only the status does.

Reading a failed job

When a job fails, its log holds the cause, but not always at the end. A crash often unwinds into a second, unrelated error, so an out-of-memory failure can finish with a pickling error thousands of characters after the OutOfMemoryError that caused it. Search the log for the first error rather than reading the last one.

A failed job produces no metrics. Fix the cause and submit again: the entity is spent, and the retry is a new one from the same parent.

Overrides

A job’s config.yaml and job_description.json can each be varied at submission. That means an iteration is the same parent submitted again rather than a new job input, so several attempts can all point at one seed dataset.

Two rules govern this. Neither raises an error when you get it wrong.

Each file is replaced whole, never merged

Anything you don’t send takes the library default, not the parent’s value, and that applies to individual fields as well as whole sections. Sending only the section you’re changing silently reverts everything else. So every override starts by reading the parent’s file, editing one field, and sending it back entire.

A config carrying a complete base but no synthgen or tuning section is accepted, and those sections revert. Submitted against a parent that set generation_target: 512, output_is_json: true and per_device_train_batch_size: 8, a base-only override ran with generation_target: 10000, output_is_json: false and per_device_train_batch_size: 1. Nothing errored. See Config file.

An override reaches the config and the job description and nothing else. Changing the data means a new entity, not an override.

Credits

Credits are metered per endpoint rather than from one pool, so a balance is a count of remaining calls to that endpoint. Reading the balance is free and answers at zero.

Stage	Route key	New account starts with
Trace processing	prepared_traces_post	100
Trace processing	seed_datasets_from_prepared_traces_post	20
Seed dataset, also the validator	seed_datasets_post	100
Teacher evaluation	teacher_evaluations_post	20
Synthetic data generation	training_datasets_from_seed_datasets_post	5
Training dataset download	training_datasets_download_get	0
Training dataset from files	training_datasets_post	100
Model training	slms_from_training_datasets_post	2
Model re-upload	slms_post	0
Deployment	deployments_from_slms_post	2

Two routes start at zero, so the stage they gate is unavailable until you ask for a grant. Read a balance against your whole plan rather than the next submission, since N training runs need N credits.

A call against an exhausted route is refused. A failed job spends its credit like any other, but invalid input costs nothing: the balance is checked before validation, and the call is recorded only once the entity exists. That’s what makes creating a seed dataset usable as a free validator.

Only metered routes appear in a balance. A route the platform never charges for is absent rather than reported as unlimited, so a missing key isn’t a zero.

Read the balance before synthetic data generation rather than before training. A generation run with no training credit left spends a credit on data you then can’t train on.

What each stage produces

Entity	Metrics	Files	Config + job description
PreparedTraces	-	traces, test data	free
SeedDataset	original-model score and predictions, trace-derived only	train, test, unstructured	free
TeacherEvaluation	teacher score, its predictions	-	free
TrainingDataset	train and test size in bytes	credit gated; a free sample instead	free
SLM	base and tuned scores, the tuned model’s predictions	model tarball	free; also the inference client
Deployment	-	-	endpoint URL and key

Metrics and files answer once that entity’s own job reaches JOB_SUCCESS, and before that they’re null. Treat a null as “not yet” and check the status to be sure.

Three conditions are worth knowing before you plan around them:

• A directly created seed dataset has no metrics. It runs no job, since validation is synchronous, so there’s nothing to report. Those fields fill only on a trace-derived one, and only when trace processing ran with evaluate_original_model: true.
• An SLM reports both scores but only the tuned model’s predictions. The base-versus-tuned gap is available as numbers; a base-model failure case isn’t.
• The free training dataset sample is capped: a deterministic 128 rows at most, drawn from the first 384, train rows only. It never contains test rows, so take those from the parent seed dataset, whose download is free.

What the scores mean, and how to judge them: Metrics.

Runs aren’t reproducible

The teacher generates and the judge scores at non-zero temperature, so the same config submitted twice gives different data and different numbers. Two runs of one generation config produced 512 and 634 examples. One untrained model scored 0.64, 0.60 and 0.58 on the same 50-row test set.

base.random_seed doesn’t pin this.

Treat every score as a sample. Quote the run it came from, and never resolve a decision on a difference smaller than the noise band in Metrics.