From Coding Traces To Training Data

BashGym starts from a simple idea: coding agents already leave behind training material. The useful part is not the final answer alone. It is the full trail of context, commands, observations, edits, verifier output, and recovery decisions that led to the result.

That trail is what BashGym calls a trace. A trace can teach imitation, preference, reward scoring, or executable terminal behavior, but only if the system preserves enough evidence to know what actually happened.

What Becomes Training Data

A successful coding session can become an SFT example because it shows the student how a good agent moves through a task. A failed session can still be useful, but not as a success example. It belongs in failure analysis, DPO rejected examples, reward labels, or curriculum design.

This distinction is the spine of the platform. BashGym should never flatten every session into the same bucket. A gold trace teaches "do this." A lower-quality trace often teaches "prefer something else." A terminal environment gives the model a task it can actually attempt, fail, retry, and pass under a verifier.

The Trace-To-Training Flow

The first step is capture or import. BashGym can work from current coding-agent sessions and from existing histories. That matters because the platform should have data on day one, before a user has installed every future hook or built every workflow around it.

After capture, the platform curates. It scrubs secrets and PII, records provenance, scores quality, and separates training-safe data from eval-only or unsafe sources. This is where the platform earns trust. If provenance is missing or an eval source leaks into training, the later model claim is already compromised.

Only then should BashGym generate artifacts: SFT JSONL, DPO pairs, reward examples, terminal environment specs, replay records, source manifests, and RunCards. Each artifact serves a different training method, so each needs its own contract rather than a generic pile of "examples."

Why Boundaries Matter

Most training problems look like model problems at first. In practice, many are data boundary problems. The student cannot learn reliable behavior from traces that lost the verifier output, pairs where chosen and rejected answers do not share the same prompt, or reward examples that fail to declare label source and split.

BashGym's job is to keep those boundaries visible. Gold data stays gold. Failures stay useful without becoming fake successes. Eval-only sources stay out of training. Terminal environments preserve protected-file and verifier checks. RunCards hold the evidence together so a future reviewer can understand what was trained, what was tested, and what still remains a claim.

The First Useful Run

A first run should be boring on purpose. Start with a small SFT baseline that proves data loading, chat template behavior, loss masking, metric logging, and checkpoint writing. The goal is not to beat a frontier model. The goal is to prove the loop is intact.

Once the baseline exists, attach behavior evidence. Run heldout trace evals or executable pass@k. Inspect truncation. Check that the final verifier and recovery steps are present. If pass@k is still zero, improve the curriculum or use distillation before trying RL.

That is the flywheel BashGym is trying to make repeatable: capture real work, curate it into the right artifacts, train the simplest viable student, evaluate behavior, route narrow wins, and feed the new traces back into the next cycle.