Trace Capture
Capture or import coding sessions from Claude Code, Codex, Gemini, OpenCode, Copilot CLI, and similar tools.
Appears in the workspace, terminal grid, canvas view, and trace browser.
Turn Coding Traces
Into Training Loops
BashGym captures real coding-agent work, scrubs and scores the traces, converts them into SFT, DPO, reward, and terminal-RL artifacts, then evaluates the model before you route work to it.
Self-host:
$
git clone https://github.com/GhostPeony/bashgym && cd bashgym && pip install -r requirements.txt
Turn agent work into training data.
BashGym captures real coding sessions, cleans the traces, and turns the useful parts into trainable artifacts with evidence attached.
Capture
Import or record coding sessions from the tools your agents already use.
Prepare
Scrub secrets, score quality, and create SFT, DPO, reward, or environment data.
Prove
Use RunCards, heldout tests, and pass@k before routing work to a model.
Platform Walkthrough
See BashGym in action — from trace capture to training evidence.
Capability Map
Eleven capabilities, organized the same way BashGym works: a core trace-to-training loop, support rails, and research tracks that stay evidence-gated.
Core trace-to-training loop
The main path is deliberately conservative: capture real work, clean it, train against a stated method, prove behavior, then route only the tasks the student model has earned.
Privacy by Design
PII detection, secret scrubbing, replay redaction, path anonymization, and provenance metadata keep traces reviewable.
Replay scrubbers, source provenance, and guardrail checks run before training handoff.
Training Methods
SFT, DPO, reward modeling, distillation, GRPO/RLVR, cascade RL, and DPPO planning with starter settings and metrics.
Training config, guidance tables, loss curves, and model-family profiles keep method choices explicit.
Evaluation Gates
Heldout traces, executable pass@k, holdout comparisons, benchmark ingest, spurious-reward controls, and tamper canaries.
The evaluator dashboard, heldout gate panel, reward controls, and benchmark manifests provide the proof.
Progressive Routing
A student handles narrow work only after heldout behavior, safety controls, and release evidence support the claim.
Router and promotion logic stay downstream of RunCard evidence.
Support rails for repeatable runs
These are the operational layers users need once training becomes a system: sources, evidence, education, and compute targets.
Source Library
Curated source cards identify what is safe for training, what is eval-only, and which adapters can convert records into BashGym artifacts.
This produces source manifests, dataset cards, and environment specs.
RunCards & Evidence
RunCards tie together source manifests, compute targets, configs, metrics, release evidence, known limits, and claim-tier blockers.
Each serious run can produce a run_card.json with findings and next actions.
Training Guidance
Guides, capability maps, metric runbooks, and agent-readable CLI commands explain settings and failure modes.
Users get recommended methods, starter settings, and metric interpretation instead of blank inputs.
Compute Targets
Local GPU, SSH/GX10, and cloud launch planning make backend runs reproducible while remote or billable work stays explicit.
Compute targets, smoke bundles, and launch plans make heavier runs easier to review before they start.
Advanced methods without overclaiming
Reward and terminal-RL work are part of the platform, but BashGym separates diagnostics from release evidence so users know what is proven and what still needs backend smoke tests.
Reward Modeling
DPO pairs, reward examples, ORM/PRM artifacts, fixture reward-model smokes, and reward_eval.json expose whether a learned scoring signal is trustworthy.
Promotion should depend on preference accuracy, reward margin, fixture smokes, and no heldout regression.
Terminal RL Research
DPPO replay, Binary-TV/KL telemetry, ECHO/RWML world-model payloads, and smoke bundles support terminal-agent RL research without overstating gains.
Promotion should depend on backend-emitted telemetry, pass@k evidence, tamper checks, and RunCard findings.
The same structure appears inside BashGym as Workspace, Data Factory, Training Guidance, RunCard Evidence, Evaluator, Router, Source Library, and Compute Target surfaces.
The Flywheel
A self-reinforcing coding-agent loop: capture traces, curate evidence, train a student, evaluate behavior, route narrowly, repeat.
Capture
Record agent work
Curate
Scrub and score
Generate
Build artifacts
Train
Tune a student
Evaluate
Run gates
Route
Ship narrow wins
Training Guidance & Evidence
Plan from the CLI or UI, prepare source artifacts, validate reward data, build smoke bundles, and attach evidence before making model claims.
- Agent-readable training docs and capability maps
- Source manifests, dataset cards, and artifact validators
- RunCards with claim-tier blockers and next actions
- Reward-model evals, heldout gates, and pass@k evidence
- DPPO/ECHO/RWML smoke readiness before GX10 or cloud runs
BashGym Evidence
$
bashgym training capabilities --json
methods: sft, dpo, reward-model, grpo, dppo
sources: gold traces, source cards, environments
gates: heldout, pass@k, spurious, tamper
world-model: diagnostic only
next: bashgym training plan --strategy sft
Evidence-first training path ready.
Save RunCard before promotion.
Three Steps to a Proven Training Loop
1
Capture Traces
Import existing agent history or install hooks for Claude Code, Codex, Gemini, OpenCode, Copilot CLI, and similar tools.
2
Create Artifacts
Scrub replay output, score trace quality, attach source metadata, and generate SFT examples, DPO pairs, reward records, or environment specs.
3
Train, Evaluate, Route
Run a focused training plan, capture metrics in a RunCard, prove behavior with heldout gates, then route only the work the model has earned.
8-Layer Architecture
A modular system from trace capture to training evidence and conservative routing.
Arena
Trace Capture
Hook into coding-agent work
Replay Import
Conversation, tool, terminal, and diff context
Judge
Quality Scoring
Verifier and rubric evidence
Replay Scrubbing
Secrets, PII, and path redaction
Factory
Source Library
Training-safe and eval-only catalogs
Data Designer
Adapters and artifact generation
Gym
SFT / DPO / Reward / RL
Open-model training recipes
Compute Targets
Local, GX10, SSH, and cloud plans
Models
RunCards
Configs, metrics, limits, and claims
Lineage
Trace-to-model provenance
Observability
Guidance
Settings, metric guides, and next actions
Evaluation Gates
Heldout, pass@k, benchmark, and safety evidence
Integrations
Agent CLIs
Claude Code, Codex, Gemini, OpenCode
Research Tools
Data Designer, AutoResearch, BashBros
API
Agent CLI
Capabilities, plans, smokes, and docs
Routing
Evidence-gated model promotion
Works With Your Stack
Claude Code / Codex
Hugging Face
Data Designer
Ollama / GGUF
verl / SkyRL
GX10 / Cloud GPUs
Start the Trace-to-Training Loop
Capture traces, prepare artifacts, run a small SFT or preference baseline, then attach evidence before routing any real work.
Get BashGym Updates
New features, training strategies, and model releases. No spam.
Frequently Asked Questions
What is BashGym?
BashGym is a trace-to-training platform for coding agents. It captures real coding sessions, scrubs and scores them, turns them into SFT, DPO, reward, and terminal-environment artifacts, and attaches the evaluation evidence needed before routing an open model to real work.
What does BashGym train from?
BashGym trains from verified traces: task context, tool calls, terminal output, file edits, diffs, tests, verifier results, quality labels, and source metadata. Gold traces become SFT examples, failed traces can become DPO negatives, and executable environments become pass@k and RL evidence.
How does the flywheel work?
The BashGym flywheel is capture or import, curate, generate examples, train, evaluate, and route. Supporting flywheels handle Source Library data, reward modeling, terminal RL, JEPA-style ECHO/RWML diagnostics, AutoResearch recipes, compute targets, and RunCards.
What models can I train with BashGym?
BashGym is model-family aware, with profiles for Qwen3/Qwen3.6, Qwen2.5, Gemma 4, Llama 3, and generic Hugging Face causal LMs. Local LoRA/QLoRA, SSH/GX10, and cloud paths are supported or planned through explicit compute targets and RunCards.
Do I need ML expertise to use BashGym?
BashGym is designed to teach the training process instead of hiding it. The CLI and UI expose starter plans, setting explanations, metric guides, source guardrails, smoke bundles, and release gates so developers can start small and know what evidence is still missing.
How does BashGym capture training data?
BashGym captures or imports traces from coding tools, normalizes them into a trace event schema, records structured agent-status events, scrubs replay output for secrets, scores quality, and preserves provenance before any training handoff.
Is BashGym free?
Yes. BashGym is open source under the MIT License. You can self-host it by cloning the GitHub repository. Cloud GPU costs for training are separate and depend on your chosen provider.