Llama-3.2-3B-NuminaQA

Links

• 📜 Paper
• 💻 GitHub
• 🤗 Oat-Zero Collection

Introduction

This model serves as a 3B base in our minimalist R1-Zero recipe.

Training details:

• Base model: HuggingFaceTB/FineMath-Llama-3B
• Dataset: lkevinzc/numia-1.5-qa-concatenated
• Epochs: 2
• Learning rate: 1e-5

Citation

@article{liu2025understanding,
  title={Understanding r1-zero-like training: A critical perspective},
  author={Liu, Zichen and Chen, Changyu and Li, Wenjun and Qi, Penghui and Pang, Tianyu and Du, Chao and Lee, Wee Sun and Lin, Min},
  journal={arXiv preprint arXiv:2503.20783},
  year={2025}
}