Frost2o24/bash-instruct-II-55k

Bash Instruct II 55k

A synthetic instruction-tuning dataset that maps natural-language requests to correct Bash — single commands, short pipelines, and multi-line scripts — for fine-tuning small LLMs to turn plain requests into runnable shell code.

Each example is a chat conversation (system / user / assistant). Rows carry metadata for slicing (category, utility) and for grouping equivalent answers to one request (variant_group). This is the second-generation build: chat format, grouped alternative answers, real human phrasing seeded from tldr-pages, a strict utility cap, and validation by both static analysis and real execution on Linux.

🐙 GitHub (clone + tooling)	https://github.com/ya5h-P/bash-instruct-55k
🤗 Hugging Face (viewer)	https://huggingface.co/datasets/Frost2o24/bash-instruct-II-55k

from datasets import load_dataset
ds = load_dataset("Frost2o24/bash-instruct-II-55k", split="train")

---

At a glance

Metric	Value
Rows	54,803
Distinct requests (variant_groups)	45,142
single / pipeline / script	40% / 35% / 25% (exact)
Distinct primary utilities	183
bash -n valid	100% (0 syntax errors)
Executed on real Linux (correctness on gradable)	≈100% clean
Exact-duplicate (request, command) pairs	0
Unique request text per request	≈97%
Requests with ≥2 equivalent answers	≈17%
Max share of any one utility (requests / rows)	≤ 4.9% / ≤ 6.0%

---

Format

One JSON object per line (bash_dataset.jsonl):

{
  "messages": [
    {"role": "system",    "content": "You are a Bash expert."},
    {"role": "user",      "content": "Show the last 20 lines of error.log."},
    {"role": "assistant", "content": "tail -n 20 error.log"}
  ],
  "category": "single",
  "utility": "tail",
  "variant_group": "3f9a1c0b7e2d4a86"
}

Field	Meaning
messages	The training conversation. The system prompt is one of 5 equivalent Bash-assistant prompts, rotated so the model doesn't overfit a single string.
category	single (one command) · pipeline (pipes, &&/||, $(...), xargs) · script (multi-line: set -euo pipefail, functions, getopts, trap, loops, here-docs).
utility	The primary command of the solution (grep, awk, find, for, …).
variant_group	16-hex id shared by rows that answer the same request with different-but-equivalent commands (e.g. wc -l f vs cat f | wc -l). Solo requests get a unique id.

System / info utilities that small models often fumble are covered with correct, idiomatic usage and guaranteed minimum counts: journalctl, pstree, lsof, vmstat, w, nice, renice, free, hostname, ss, iostat, netstat, dmesg, uptime.

---

Loading

from datasets import load_dataset
ds = load_dataset("Frost2o24/bash-instruct-II-55k", split="train")   # or: json, data_files="bash_dataset.jsonl"
print(ds[0]["messages"])

import collections
print(collections.Counter(ds["category"]))
print(collections.Counter(ds["utility"]).most_common(15))

# collapse to one canonical answer per request
seen, canonical = set(), []
for r in ds:
    if r["variant_group"] not in seen:
        seen.add(r["variant_group"]); canonical.append(r)
print(len(canonical), "distinct requests")

Peek without Python:

head -n 1 bash_dataset.jsonl | jq .
jq -r '.utility' bash_dataset.jsonl | sort | uniq -c | sort -rn | head

---

Quality & validation

Three layers, in increasing strength:

1. bash -n on every command — 0 / 54,803 syntax failures.
2. shellcheck (warning level) on a large sample — the few findings are style nits (SC2010 ls | grep, SC2164 cd without check).
3. Real execution of the runnable subset in a disposable Linux sandbox (Debian 13 via WSL): 43,257 commands run; after cleaning (below), essentially all gradable commands execute cleanly.

"Gradable" excludes failures that are environmental, not command defects — a blank sandbox legitimately lacks referenced files, users, services, CLI args, root, or systemd (e.g. ps -C postgres when postgres isn't running, chown ec2-user: when that user is absent, rmdir on a populated fixture dir). Those commands are correct; only the sandbox lacks the target, so they are kept.

Data cleaning

Whole-set execution flagged 43 genuinely-malformed commands — bad argument count/format that fail on any system (mostly tldr-seed placeholder fills that slipped the filter). They were removed (55,000 → 54,957). What was cut:

• echo "host" | git credential fill/approve/reject — needs key=value input, not a bare host
• split 10 10 file, shuf 3 5 file — too many operands
• date --rfc-3339 20, date 2026-03-01 @<ts> — bad / extra operand
• uname java, cp --parents f1 f2, lspci -s notes.xml, head 20 -3 file
• echo -e "appdb" | tsort — odd number of tokens

A second pass (manual analysis + a stricter validator) caught two generator bugs and fixed them at the source (55,000 → 54,808):

• Invalid long-form flag variant. The answer-diversity generator applied a command's short→long flag map across a whole pipeline, leaking e.g. grep's -n → --line-number onto a downstream tail -n → tail --line-number (invalid). 146 rows removed; the generator now restricts substitution to the owning command's segment. (tail --lines / head --lines are valid on GNU and were kept.) Also removed 3 kill --CONT <name>.
• find -size missing its comparison operator. "larger than N" requests emitted bare -size N (means exactly N). 113 rows repaired to -size +N; the template now sets +/- from the request direction.

Re-running the hardened validator (now with set -o pipefail + a per-stage stderr scan for invalid/unrecognized option) then surfaced 5 more malformed rows the old exit-status check had missed — journalctl <service> <service> (bare units need -u) — which were removed (→ 54,803).

Running the validator yourself (validate.py)

Validation is decoupled from generation — nothing in generate.py calls it.

# instant static checks: bash -n on every command + shellcheck (if installed),
# broken down by category and utility
python validate.py --data bash_dataset.jsonl

# execute the SAFE self-contained subset in a disposable fixture dir
python validate.py --data bash_dataset.jsonl --execute --n 3000

# whole-set execution on a THROWAWAY Linux box (WSL etc.): runs nearly every
# single-line command AND scripts, then reports the genuine-error rate and offers
# to delete those rows (backs up the dataset first)
python validate.py --data bash_dataset.jsonl --execute --permissive --n 60000 --timeout 5

Two execution modes:

• strict (default) — only self-contained, read-only coreutils on relative paths.
• --permissive — for a disposable Linux box. Runs ≈43k rows including scripts and system-info / fs-mutating tools, while hard-blocking genuinely dangerous commands (dd/mkfs/shutdown/reboot/mount, fork bombs, kill/pkill, writes to /etc·/var·$HOME, block devices) and skipping anything that would hang (tail -f, watch, while true, sleep, installers, network, yes). Every command runs in a temp sandbox with a timeout, stdin from /dev/null, sudo stripped, and stdout discarded so infinite-output commands can't exhaust memory.

The report separates ran-clean, environmental (not correctness failures), and genuine errors (bad flag/option/logic). --dump-errors FILE writes every genuine error for inspection; --yes / --no control the delete prompt non-interactively.

---

Reproducing (generate.py)

python generate.py plan            # quota table + registered recipes
python generate.py run             # generate to ./bash_dataset.jsonl (resumable)
python generate.py status          # where a run stands
python generate.py validate-all    # re-run bash -n over the whole file
python generate.py seeds           # build/inspect the real-phrasing seed pool

Resumable: bash_dataset.jsonl is ground truth and a derived bash_dataset.progress.json snapshot is rewritten after every batch, so re-running continues where it stopped.

How it's built

• Recipe engine — ≈150 hand-written recipe families emit (request, command) pairs by combining varied phrasings with realistic parameter pools (filenames, dirs, ports, services, users, patterns). Every parameter is drawn once per example and reused in both the request and the command, so they never disagree.
• Argument-fidelity gate (argcheck.py) — a pair is rejected unless the command references the concrete nouns the request names (filename, extension, user, group, service, process, port).
• Real phrasing (seeds) — a share of requests are seeded from tldr-pages: the human description becomes the request, its command the solution, with {{placeholders}} filled fresh from the pools. Heavily quality-filtered (subcommand multiplexers and non-fillable placeholders dropped) down to ≈600 clean coreutils/text/net templates, cached in seeds/_seedcache.json.
• Answer diversity — a share of requests also emit 1–2 genuinely-equivalent alternates sharing a variant_group (flag reorderings, long/short options, wc -l f ↔ cat f | wc -l, sort -u ↔ sort | uniq, …), each re-checked before it's kept.
• Guards — SHA-1 dedup; a degeneracy guard rejecting no-op transforms (sed 's/x/x/', mv f f); category quotas (40/35/25); a 4% per-utility request cap + 6% row cap; minimum floors for the system/info utilities; and a bash -n gate on every command.

Tunable flags

Flag	Default	Effect
--target	55000	Total rows (rescales quotas, caps, floors).
--seed-frac	0.40	Target share of seedable requests drawn from real seeds.
--seed-reuse	8	Max reuse of one tldr template (higher = more real phrasing, less request variety).
--variant-frac	0.30	Share of requests that attempt alternate answers.
--row-cap-pct	0.06	Max total-row share per utility (trades off answer diversity).
--no-net	off	Never download seeds; use the local cache only.

Seed and answer-diversity fractions are soft (they land under target when the clean seed pool or the caps bind). The hard invariants — category split, caps, floors, argument fidelity, bash -n, dedup — always hold.

---

Files

File	What it is
bash_dataset.jsonl	The dataset — 54,803 rows (≈22 MB, no Git LFS needed).
generate.py	The generator (reproducible, resumable).
validate.py	Standalone validator (strict + --permissive execution).
argcheck.py	Argument-fidelity checker (generation gate + standalone auditor).
seeds/_seedcache.json	Cleaned real-phrasing templates so generation runs offline.

Intended use & limitations

Good for teaching a small model to map natural-language requests to correct single commands, short pipelines, and small scripts — including the fumble-prone system utilities.

• Synthetic. Variety comes from recombining templates + token pools and a quality-filtered slice of real tldr phrasings; request shapes come from a finite family set (phrasing is diversified — request text ≈97% unique).
• Valid ≠ fully semantically correct. bash -n, shellcheck, sandbox execution, and the argument gate show commands parse, lint, run, and use the request's nouns — none prove the command's logic perfectly matches intent (e.g. an awk column index assumes a particular layout).
• Single-turn. No explanations or negative examples, aside from the equivalent answers grouped by variant_group.

License

MIT.