Update README.md

README.md

@@ -31,13 +31,9 @@ inference: false
 
 ## Model Summary
 
-**TabiBERT** is a modernized encoder-only Transformer model (BERT-style) based on the [ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) architecture.  
-It has been **pre-trained on 86 billion tokens** of diverse data, primarily:
-
-- A large-scale **Turkish corpus** covering literature, news, social media, Wikipedia, and academic 
-texts.  
-- **English text**, ** code with English commentary**, and **math problems in English** — together making up about **13% non-Turkish** tokens.  
-
+**TabiBERT** is a modernized encoder-only Transformer model (BERT-style) based on the [ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) architecture.
+TabiBERT is pre-trained on **1 trillion tokens** of a diverse dataset including Turkish, English, Code, Math with a native context length of up to 8,192 tokens. 
+ 
 TabiBERT inherits ModernBERT’s architectural improvements, such as:
 
 - **Rotary Positional Embeddings (RoPE)** for long-context support.  
@@ -75,13 +71,17 @@ pip install flash-attn
 Example usage with `AutoModelForMaskedLM`:
 ```py
 from transformers import AutoTokenizer, AutoModelForMaskedLM
+import torch
 
 model_id = "boun-tabilab/TabiBERT"
 tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = AutoModelForMaskedLM.from_pretrained(model_id)
 
+device = "cuda" if torch.cuda.is_available() else "cpu"
+model = model.to(device)
+
 text = "[MASK] Sistemi'ndeki en büyük gezegen Jüpiter'dir."
-inputs = tokenizer(text, return_tensors="pt")
+inputs = tokenizer(text, return_tensors="pt").to(device)
 outputs = model(**inputs)
 
 masked_index = inputs["input_ids"][0].tolist().index(tokenizer.mask_token_id)
@@ -96,10 +96,19 @@ from transformers import pipeline
 
 pipe = pipeline("fill-mask", model="boun-tabilab/TabiBERT")
 
-print(pipe("[MASK], Türkiye Cumhuriyeti'nin ilk başkentidir."))
+print(pipe("[MASK], Türkiye Cumhuriyeti'nin başkentidir."))
 
 ```
 
+## Pre-training Data
+
+TabiBERT has been **pre-trained on 86 billion tokens** of diverse data, primarily:
+
+- A large-scale **Turkish corpus** covering literature, news, social media, Wikipedia, and academic 
+texts.  
+- **English text**, ** code with English commentary**, and **math problems in English** — together making up about **13% non-Turkish** tokens.  
+
+
 ## Evaluation
 
 Evaluations are in progress.