thinkthink-dev/bashgemma-270m

BashGemma 270M

Fine-tuned FunctionGemma 270M for natural language to bash command translation.

Paper: Zenodo DOI: 10.5281/zenodo.18058613

Model Description

BashGemma translates natural language queries into structured JSON tool calls representing bash commands. It achieves 57.4% NLC2CMD accuracy on NL2Bash test data—a 52.9 percentage point improvement over the base FunctionGemma model.

Metric	Baseline	BashGemma
NLC2CMD Accuracy	0.045	0.574
Utility Match	0.000	0.595
Parse Rate	0.000	0.995

Usage

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load model
base_model = AutoModelForCausalLM.from_pretrained(
    "unsloth/functiongemma-270m-it",
    torch_dtype=torch.float32,
)
model = PeftModel.from_pretrained(base_model, "thinkthink-dev/bashgemma-270m")
tokenizer = AutoTokenizer.from_pretrained("thinkthink-dev/bashgemma-270m")

# Generate
prompt = "<start_of_turn>user\nFind all Python files in the current directory<end_of_turn>\n<start_of_turn>model\n"
inputs = tokenizer(prompt, return_tensors="pt")

with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=150, do_sample=False)

print(tokenizer.decode(outputs[0], skip_special_tokens=False))
# {"name": "find", "arguments": {"name": "'*.py'", "path": "."}}

Capabilities

Strong performance on:

• find with name, type, size, mtime filters
• Simple pipelines (2-3 commands)
• Common utilities: grep, ls, cat, wc, sort

Limitations:

• Complex multi-step operations
• Utilities not in training data (rsync, tar, etc.)
• Complex sed/awk patterns

Training

• Base Model: unsloth/functiongemma-270m-it
• Method: LoRA (rank=64, alpha=128)
• Training Data: 9,153 NL2Bash examples
• Training Time: 36 minutes on Apple M4 Max
• Approach: Response-only training (mask instruction tokens)

Citation

@software{large2024bashgemma,
  author = {Large, Jack},
  title = {BashGemma: Fine-tuning a 270M Parameter Model for Natural Language to Bash Translation},
  year = {2024},
  publisher = {Zenodo},
  doi = {10.5281/zenodo.18058613},
  url = {https://zenodo.org/records/18058613}
}

License

Apache 2.0 (same as base Gemma model)