Update README.md

README.md

@@ -11,6 +11,7 @@ tags:
 pipeline_tag: text-generation
 datasets:
 - HuggingFaceTB/smol-smoltalk
+- karpathy/fineweb-edu-100b-shuffle
 ---
 
 # nanochat-d20
@@ -19,10 +20,14 @@ datasets:
 
 
 ### Training Pipeline
-1.**Base-training** General PreTraining using nanochat framework
-2. **Mid-training**: General instruction tuning on SmolTalk, MMLU, GSM8K, Spelling tasks
-3. **SFT (Supervised Fine-Tuning)**: Chat-specific training on ARC, GSM8K, SmolTalk
-4. **RL (Reinforcement Learning)**: Optional GRPO-style training on GSM8K (if included)
+
+  1.**Base-training** PreTraining on FineWeb-EDU dataset using nanochat framework
+
+  2. **Mid-training**: General instruction tuning on SmolTalk, MMLU, GSM8K, Spelling tasks
+
+  3. **SFT (Supervised Fine-Tuning)**: Chat-specific training on ARC, GSM8K, SmolTalk
+
+  4. **RL (Reinforcement Learning)**: Optional GRPO-style training on GSM8K (if included)
 
 
 ## Repository Structure
@@ -34,13 +39,13 @@ datasets:
 ├── mid_checkpoints/d34/       # Mid-training checkpoint
 │   ├── model_*.pt
 │   └── meta_*.json
-├── chatsft_checkpoints/d34/   # SFT checkpoint
+├── chatsft_checkpoints/d20/   # SFT checkpoint
 │   ├── model_*.pt
 │   └── meta_*.json
-├── chatsft_checkpoints_int8/d34/   # SFT checkpoint
+├── chatsft_checkpoints_int8/d20/   # SFT checkpoint
 │   ├── model_*.pt
 │   └── meta_*.json
-├── chatrl_checkpoints/d34/    # RL checkpoint (if available)
+├── chatrl_checkpoints/d20/    # RL checkpoint (if available)
 │   ├── model_*.pt
 │   └── meta_*.json
 ├── report/                    # Evaluation reports
@@ -55,7 +60,7 @@ MIT License (same as nanochat)
 
 ## Acknowledgments
 
-- [Andrej Karpathy](https://github.com/karpathy) for the nanochat framework and pre-trained base model
+- [Andrej Karpathy](https://github.com/karpathy) for the nanochat framework
 
 ```bibtex
 @misc{nanochat,