Upload folder using huggingface_hub

1_Dense/config.json

@@ -0,0 +1,7 @@
+{
+    "in_features": 128,
+    "out_features": 128,
+    "bias": false,
+    "activation_function": "torch.nn.modules.linear.Identity",
+    "use_residual": false
+}

1_Dense/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:65869626726db1e3eb9cb199a7d5e6627c77ad384504a8902802c0cfc6e6e2f6
+size 65624

README.md

@@ -0,0 +1,455 @@
+---
+language:
+- tr
+tags:
+- ColBERT
+- PyLate
+- sentence-transformers
+- sentence-similarity
+- feature-extraction
+- generated_from_trainer
+- dataset_size:910904
+- loss:Contrastive
+base_model: ozayezerceli/bert-hash-nano-TR
+datasets:
+- parsak/msmarco-tr
+pipeline_tag: sentence-similarity
+library_name: PyLate
+---
+
+# PyLate model based on ozayezerceli/bert-hash-nano-TR
+
+This is a [PyLate](https://github.com/lightonai/pylate) model finetuned from [ozayezerceli/bert-hash-nano-TR](https://huggingface.co/ozayezerceli/bert-hash-nano-TR) on the [msmarco-tr](https://huggingface.co/datasets/parsak/msmarco-tr) dataset. It maps sentences & paragraphs to sequences of 128-dimensional dense vectors and can be used for semantic textual similarity using the MaxSim operator.
+
+## Model Details
+
+### Model Description
+- **Model Type:** PyLate model
+- **Base model:** [ozayezerceli/bert-hash-nano-TR](https://huggingface.co/ozayezerceli/bert-hash-nano-TR) <!-- at revision dee6adc907398084b08a433d9e49e79690f6ee20 -->
+- **Document Length:** 180 tokens
+- **Query Length:** 32 tokens
+- **Output Dimensionality:** 128 tokens
+- **Similarity Function:** MaxSim
+- **Training Dataset:**
+    - [msmarco-tr](https://huggingface.co/datasets/parsak/msmarco-tr)
+- **Language:** tr
+<!-- - **License:** Unknown -->
+
+### Model Sources
+
+- **Documentation:** [PyLate Documentation](https://lightonai.github.io/pylate/)
+- **Repository:** [PyLate on GitHub](https://github.com/lightonai/pylate)
+- **Hugging Face:** [PyLate models on Hugging Face](https://huggingface.co/models?library=PyLate)
+
+### Full Model Architecture
+
+```
+ColBERT(
+  (0): Transformer({'max_seq_length': 179, 'do_lower_case': False, 'architecture': 'BertHashModel'})
+  (1): Dense({'in_features': 128, 'out_features': 128, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'use_residual': False})
+)
+```
+
+## Usage
+First install the PyLate library:
+
+```bash
+pip install -U pylate
+```
+
+### Retrieval
+
+Use this model with PyLate to index and retrieve documents. The index uses [FastPLAID](https://github.com/lightonai/fast-plaid) for efficient similarity search.
+
+#### Indexing documents
+
+Load the ColBERT model and initialize the PLAID index, then encode and index your documents:
+
+```python
+from pylate import indexes, models, retrieve
+
+# Step 1: Load the ColBERT model
+model = models.ColBERT(
+    model_name_or_path="pylate_model_id",
+)
+
+# Step 2: Initialize the PLAID index
+index = indexes.PLAID(
+    index_folder="pylate-index",
+    index_name="index",
+    override=True,  # This overwrites the existing index if any
+)
+
+# Step 3: Encode the documents
+documents_ids = ["1", "2", "3"]
+documents = ["document 1 text", "document 2 text", "document 3 text"]
+
+documents_embeddings = model.encode(
+    documents,
+    batch_size=32,
+    is_query=False,  # Ensure that it is set to False to indicate that these are documents, not queries
+    show_progress_bar=True,
+)
+
+# Step 4: Add document embeddings to the index by providing embeddings and corresponding ids
+index.add_documents(
+    documents_ids=documents_ids,
+    documents_embeddings=documents_embeddings,
+)
+```
+
+Note that you do not have to recreate the index and encode the documents every time. Once you have created an index and added the documents, you can re-use the index later by loading it:
+
+```python
+# To load an index, simply instantiate it with the correct folder/name and without overriding it
+index = indexes.PLAID(
+    index_folder="pylate-index",
+    index_name="index",
+)
+```
+
+#### Retrieving top-k documents for queries
+
+Once the documents are indexed, you can retrieve the top-k most relevant documents for a given set of queries.
+To do so, initialize the ColBERT retriever with the index you want to search in, encode the queries and then retrieve the top-k documents to get the top matches ids and relevance scores:
+
+```python
+# Step 1: Initialize the ColBERT retriever
+retriever = retrieve.ColBERT(index=index)
+
+# Step 2: Encode the queries
+queries_embeddings = model.encode(
+    ["query for document 3", "query for document 1"],
+    batch_size=32,
+    is_query=True,  #  # Ensure that it is set to False to indicate that these are queries
+    show_progress_bar=True,
+)
+
+# Step 3: Retrieve top-k documents
+scores = retriever.retrieve(
+    queries_embeddings=queries_embeddings,
+    k=10,  # Retrieve the top 10 matches for each query
+)
+```
+
+### Reranking
+If you only want to use the ColBERT model to perform reranking on top of your first-stage retrieval pipeline without building an index, you can simply use rank function and pass the queries and documents to rerank:
+
+```python
+from pylate import rank, models
+
+queries = [
+    "query A",
+    "query B",
+]
+
+documents = [
+    ["document A", "document B"],
+    ["document 1", "document C", "document B"],
+]
+
+documents_ids = [
+    [1, 2],
+    [1, 3, 2],
+]
+
+model = models.ColBERT(
+    model_name_or_path="pylate_model_id",
+)
+
+queries_embeddings = model.encode(
+    queries,
+    is_query=True,
+)
+
+documents_embeddings = model.encode(
+    documents,
+    is_query=False,
+)
+
+reranked_documents = rank.rerank(
+    documents_ids=documents_ids,
+    queries_embeddings=queries_embeddings,
+    documents_embeddings=documents_embeddings,
+)
+```
+
+<!--
+### Direct Usage (Transformers)
+
+<details><summary>Click to see the direct usage in Transformers</summary>
+
+</details>
+-->
+
+<!--
+### Downstream Usage (Sentence Transformers)
+
+You can finetune this model on your own dataset.
+
+<details><summary>Click to expand</summary>
+
+</details>
+-->
+
+<!--
+### Out-of-Scope Use
+
+*List how the model may foreseeably be misused and address what users ought not to do with the model.*
+-->
+
+<!--
+## Bias, Risks and Limitations
+
+*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
+-->
+
+<!--
+### Recommendations
+
+*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
+-->
+
+## Training Details
+
+### Training Dataset
+
+#### msmarco-tr
+
+* Dataset: [msmarco-tr](https://huggingface.co/datasets/parsak/msmarco-tr) at [ffad30a](https://huggingface.co/datasets/parsak/msmarco-tr/tree/ffad30a7b0648f1c789c639db6c1d4720c22274c)
+* Size: 910,904 training samples
+* Columns: <code>query</code>, <code>positive</code>, and <code>negative</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                             | positive                                                                           | negative                                                                           |
+  |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                             | string                                                                             |
+  | details | <ul><li>min: 6 tokens</li><li>mean: 16.58 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 25 tokens</li><li>mean: 31.99 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 20 tokens</li><li>mean: 31.99 tokens</li><li>max: 32 tokens</li></ul> |
+* Samples:
+  | query                                                          | positive                                                                                                                                                                                                                                                                                                                                                                                                                       | negative                                                                                                                                                                                                                                                                                                                                                                                                                      |
+  |:---------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>sinir dokusundaki miyelin kılıfı nerede</code>           | <code>Miyelin, bir tabaka oluşturan akson dielektrik (elektriksel olarak yalıtkan) malzemeyi çevreleyen yağlı bir beyaz maddedir, miyelin kılıfı, genellikle sadece bir nöronun aksonu etrafında bulunur. Sinir sisteminin düzgün çalışması için gereklidir. Bir tür glial hücrenin bir dış büyümesidir. Miyelin kılıfının üretimi miyelinasyon olarak adlandırılır. İnsanlarda, miyelin kılıfı 14'üncü haftada başlar.</code> | <code>İnsanlarda, dört temel doku tipi vardır: epitel dokusu, bağ dokusu, kas dokusu ve sinir dokusu. Her genel doku tipi içinde, belirli doku tipleri vardır. Bunu bir futbol takımı gibi düşünün.Her biri sahada kendi 'iş' olan bireysel oyuncular vardır.n insanlar, dört temel doku tipi vardır: epitel dokusu, bağ dokusu, kas dokusu ve sinir dokusu. Bu genel doku tipinde, her bir genel doku tipinde vardır.</code> |
+  | <code>Okulların Makine Mühendisliğini Sundukları Şeyler</code> | <code>Makine Mühendisliği Teknolojisi Dereceleri için Üst Okullar. Pennsylvania Eyalet Üniversitesi - Harrisburg, Purdue Üniversitesi ve Houston Üniversitesi, makine mühendisliği teknolojisi (MET) alanında lisans derecesi sunan üç okuldur. Bu üniversitelerdeki MET programları hakkında daha fazla bilgi edinmek için okumaya devam edin.</code>                                                                         | <code>Mühendis tanımı, motorların veya makinelerin tasarımında, yapımında ve kullanımında veya çeşitli mühendislik dallarından herhangi birinde eğitimli ve yetenekli bir kişi: bir makine mühendisi; bir inşaat mühendisi. Daha fazla bilgi için bkz.</code>                                                                                                                                                                 |
+  | <code>kim navigatör karıştırma valfleri taşır</code>           | <code>BRADLEY THERMOSTATIC MIXING VANAS. Bradley Corporation, armatür ve sıhhi tesisat ürünlerinin üretiminde lider, dört hat üretir. termostatik karıştırma valfleri (TMVs). Bradley Navigator Yüksek Düşük termostatik karıştırma valfleri vardır. Dıştan gelen talebin çok düşük olduğu uygulamalar için idealdir.</code>                                                                                                   | <code>Hidrolik Valfler. Eaton valfleri, tüm dünyadaki pazarlarda müşterilerimiz için rekabet avantajı sağlar. Geniş bir seçenek yelpazesinde benzersiz kalite sunan yüksek değerli hidrolik valf ürünlerimiz, gerçek endüstri liderlerinin tüm özelliklerini ve performans seviyelerini içerir. Endüstriyel Valfler.</code>                                                                                                   |
+* Loss: <code>pylate.losses.contrastive.Contrastive</code>
+
+### Evaluation Dataset
+
+#### msmarco-tr
+
+* Dataset: [msmarco-tr](https://huggingface.co/datasets/parsak/msmarco-tr) at [ffad30a](https://huggingface.co/datasets/parsak/msmarco-tr/tree/ffad30a7b0648f1c789c639db6c1d4720c22274c)
+* Size: 9,202 evaluation samples
+* Columns: <code>query</code>, <code>positive</code>, and <code>negative</code>
+* Approximate statistics based on the first 1000 samples:
+  |         | query                                                                             | positive                                                                           | negative                                                                           |
+  |:--------|:----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------|
+  | type    | string                                                                            | string                                                                             | string                                                                             |
+  | details | <ul><li>min: 6 tokens</li><li>mean: 16.35 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 25 tokens</li><li>mean: 31.99 tokens</li><li>max: 32 tokens</li></ul> | <ul><li>min: 25 tokens</li><li>mean: 31.99 tokens</li><li>max: 32 tokens</li></ul> |
+* Samples:
+  | query                                                                           | positive                                                                                                                                                                                                                                                                                                                       | negative                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+  |:--------------------------------------------------------------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+  | <code>Ermin hangi hayvandır</code>                                              | <code>1 Aslında ermine kelimesi beyaz kürklü bir hayvanı ifade ederken, sırt üstü kahverengi kürklü ve karnında baş ve beyaz kürklü bireyler için stoat kullanılır.</code>                                                                                                                                                     | <code>Dünyada kaç hayvan türü var? İşte kaba bir sayım ve bilim adamlarının sayılara nasıl ulaştıklarına dair kısa bir açıklama. Dünyada kaç hayvan türü var? İşte kaba bir sayım ve bilim adamlarının sayılara nasıl ulaştıklarına dair kısa bir açıklama. Kaç hayvan türü var? https://www.thoughtco.com/how-many-animal-türleri-on-planet-130923 Strauss, Bob.</code>                                                                                                                           |
+  | <code>Abacus nereden çıktı</code>                                               | <code>Abacus: Kısa Bir Tarih. Abacus, kökeni Yunanca abax veya abakon (masa veya tablet anlamına gelir) kelimelerinden gelen ve muhtemelen kum anlamına gelen Semitik abq kelimesinden kaynaklanan Latince bir kelimedir. Abacus, büyük sayıları saymak için kullanılan birçok sayma cihazından biridir.</code>                | <code>Hücre apeksinde, bir flagellum için çapa alanı olan bazal gövdedir. Bazal cisimler, dokuz periferik mikrotübül üçlüsü ile centrioles'inkine benzer bir alt yapıya sahiptir (görüntünün alt merkezindeki yapıya bakınız).</code>                                                                                                                                                                                                                                                              |
+  | <code>Başın arkasında radyasyon tedavisi yüz kızarıklığına neden olur mu</code> | <code>Radyasyon Terapisinin En Yaygın Yan Etkileri. Cilt reaksiyonu: Radyasyon tedavisinin yaygın bir yan etkisi, tedavi edilen vücut bölgesinde cilt tahrişidir. Cilt reaksiyonu, hafif kızarıklık ve kuruluktan (güneş yanığına benzer) bazı hastalarda cildin şiddetli soyulmasına (desquamation) kadar değişebilir.</code> | <code>Bu açıklama amfizemi işaret edebilir. Bu, sigara içme geçmişiniz varsa daha da muhtemeldir. Radyasyon terapisi bilinen nedenlerden biri değildir. Bu konuda daha fazla cevap almak ve semptomlarınızı çözmeye yardımcı olmak için bir pulmonologla takip etmenizi isteyeceğim. Umarım bu, sorgunuzu tamamen ele alır. Sigara içme geçmişiniz varsa, daha da fazla umut eder. Radyasyon terapisi, bu sorunun çözümüne yardımcı olmanızı ve bu sorunun cevabını takip etmenizi isterim.</code> |
+* Loss: <code>pylate.losses.contrastive.Contrastive</code>
+
+### Training Hyperparameters
+#### Non-Default Hyperparameters
+
+- `learning_rate`: 3e-06
+- `num_train_epochs`: 1
+- `fp16`: True
+
+#### All Hyperparameters
+<details><summary>Click to expand</summary>
+
+- `overwrite_output_dir`: False
+- `do_predict`: False
+- `eval_strategy`: no
+- `prediction_loss_only`: True
+- `per_device_train_batch_size`: 8
+- `per_device_eval_batch_size`: 8
+- `per_gpu_train_batch_size`: None
+- `per_gpu_eval_batch_size`: None
+- `gradient_accumulation_steps`: 1
+- `eval_accumulation_steps`: None
+- `torch_empty_cache_steps`: None
+- `learning_rate`: 3e-06
+- `weight_decay`: 0.0
+- `adam_beta1`: 0.9
+- `adam_beta2`: 0.999
+- `adam_epsilon`: 1e-08
+- `max_grad_norm`: 1.0
+- `num_train_epochs`: 1
+- `max_steps`: -1
+- `lr_scheduler_type`: linear
+- `lr_scheduler_kwargs`: {}
+- `warmup_ratio`: 0.0
+- `warmup_steps`: 0
+- `log_level`: passive
+- `log_level_replica`: warning
+- `log_on_each_node`: True
+- `logging_nan_inf_filter`: True
+- `save_safetensors`: True
+- `save_on_each_node`: False
+- `save_only_model`: False
+- `restore_callback_states_from_checkpoint`: False
+- `no_cuda`: False
+- `use_cpu`: False
+- `use_mps_device`: False
+- `seed`: 42
+- `data_seed`: None
+- `jit_mode_eval`: False
+- `use_ipex`: False
+- `bf16`: False
+- `fp16`: True
+- `fp16_opt_level`: O1
+- `half_precision_backend`: auto
+- `bf16_full_eval`: False
+- `fp16_full_eval`: False
+- `tf32`: None
+- `local_rank`: 0
+- `ddp_backend`: None
+- `tpu_num_cores`: None
+- `tpu_metrics_debug`: False
+- `debug`: []
+- `dataloader_drop_last`: False
+- `dataloader_num_workers`: 0
+- `dataloader_prefetch_factor`: None
+- `past_index`: -1
+- `disable_tqdm`: False
+- `remove_unused_columns`: True
+- `label_names`: None
+- `load_best_model_at_end`: False
+- `ignore_data_skip`: False
+- `fsdp`: []
+- `fsdp_min_num_params`: 0
+- `fsdp_config`: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
+- `fsdp_transformer_layer_cls_to_wrap`: None
+- `accelerator_config`: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
+- `parallelism_config`: None
+- `deepspeed`: None
+- `label_smoothing_factor`: 0.0
+- `optim`: adamw_torch_fused
+- `optim_args`: None
+- `adafactor`: False
+- `group_by_length`: False
+- `length_column_name`: length
+- `ddp_find_unused_parameters`: None
+- `ddp_bucket_cap_mb`: None
+- `ddp_broadcast_buffers`: False
+- `dataloader_pin_memory`: True
+- `dataloader_persistent_workers`: False
+- `skip_memory_metrics`: True
+- `use_legacy_prediction_loop`: False
+- `push_to_hub`: False
+- `resume_from_checkpoint`: None
+- `hub_model_id`: None
+- `hub_strategy`: every_save
+- `hub_private_repo`: None
+- `hub_always_push`: False
+- `hub_revision`: None
+- `gradient_checkpointing`: False
+- `gradient_checkpointing_kwargs`: None
+- `include_inputs_for_metrics`: False
+- `include_for_metrics`: []
+- `eval_do_concat_batches`: True
+- `fp16_backend`: auto
+- `push_to_hub_model_id`: None
+- `push_to_hub_organization`: None
+- `mp_parameters`: 
+- `auto_find_batch_size`: False
+- `full_determinism`: False
+- `torchdynamo`: None
+- `ray_scope`: last
+- `ddp_timeout`: 1800
+- `torch_compile`: False
+- `torch_compile_backend`: None
+- `torch_compile_mode`: None
+- `include_tokens_per_second`: False
+- `include_num_input_tokens_seen`: False
+- `neftune_noise_alpha`: None
+- `optim_target_modules`: None
+- `batch_eval_metrics`: False
+- `eval_on_start`: False
+- `use_liger_kernel`: False
+- `liger_kernel_config`: None
+- `eval_use_gather_object`: False
+- `average_tokens_across_devices`: False
+- `prompts`: None
+- `batch_sampler`: batch_sampler
+- `multi_dataset_batch_sampler`: proportional
+- `router_mapping`: {}
+- `learning_rate_mapping`: {}
+
+</details>
+
+### Training Logs
+| Epoch  | Step | Training Loss |
+|:------:|:----:|:-------------:|
+| 0.0000 | 1    | 1.9342        |
+| 0.0088 | 1000 | 1.3908        |
+| 0.0176 | 2000 | 1.1178        |
+| 0.0263 | 3000 | 1.0264        |
+| 0.0351 | 4000 | 0.9816        |
+| 0.0439 | 5000 | 0.9313        |
+| 0.0527 | 6000 | 0.9324        |
+| 0.0615 | 7000 | 0.8896        |
+| 0.0703 | 8000 | 0.8703        |
+| 0.0790 | 9000 | 0.8665        |
+
+
+### Framework Versions
+- Python: 3.12.1
+- Sentence Transformers: 5.1.1
+- PyLate: 1.3.4
+- Transformers: 4.56.2
+- PyTorch: 2.8.0+cu128
+- Accelerate: 1.10.1
+- Datasets: 4.0.0
+- Tokenizers: 0.22.1
+
+
+## Citation
+
+### BibTeX
+
+#### Sentence Transformers
+```bibtex
+@inproceedings{reimers-2019-sentence-bert,
+    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
+    author = "Reimers, Nils and Gurevych, Iryna",
+    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
+    month = "11",
+    year = "2019",
+    publisher = "Association for Computational Linguistics",
+    url = "https://arxiv.org/abs/1908.10084"
+}
+```
+
+#### PyLate
+```bibtex
+@misc{PyLate,
+title={PyLate: Flexible Training and Retrieval for Late Interaction Models},
+author={Chaffin, Antoine and Sourty, Raphaël},
+url={https://github.com/lightonai/pylate},
+year={2024}
+}
+```
+
+<!--
+## Glossary
+
+*Clearly define terms in order to be accessible across audiences.*
+-->
+
+<!--
+## Model Card Authors
+
+*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
+-->
+
+<!--
+## Model Card Contact
+
+*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
+-->

added_tokens.json

@@ -0,0 +1,4 @@
+{
+  "[D] ": 30523,
+  "[Q] ": 30522
+}

config.json

@@ -0,0 +1,31 @@
+{
+  "architectures": [
+    "BertHashModel"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "auto_map": {
+    "AutoConfig": "configuration_bert_hash.BertHashConfig",
+    "AutoModel": "modeling_bert_hash.BertHashModel",
+    "AutoModelForMaskedLM": "modeling_bert_hash.BertHashForMaskedLM",
+    "AutoModelForSequenceClassification": "modeling_bert_hash.BertHashForSequenceClassification"
+  },
+  "classifier_dropout": null,
+  "dtype": "float32",
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 128,
+  "initializer_range": 0.02,
+  "intermediate_size": 512,
+  "layer_norm_eps": 1e-12,
+  "max_position_embeddings": 512,
+  "model_type": "bert_hash",
+  "num_attention_heads": 2,
+  "num_hidden_layers": 2,
+  "pad_token_id": 0,
+  "position_embedding_type": "absolute",
+  "projections": 16,
+  "transformers_version": "4.56.2",
+  "type_vocab_size": 2,
+  "use_cache": true,
+  "vocab_size": 30524
+}

config_sentence_transformers.json

@@ -0,0 +1,54 @@
+{
+  "model_type": "SentenceTransformer",
+  "__version__": {
+    "sentence_transformers": "5.1.1",
+    "transformers": "4.56.2",
+    "pytorch": "2.8.0+cu128"
+  },
+  "prompts": {
+    "query": "",
+    "document": ""
+  },
+  "default_prompt_name": null,
+  "similarity_fn_name": "MaxSim",
+  "query_prefix": "[Q] ",
+  "document_prefix": "[D] ",
+  "query_length": 32,
+  "document_length": 180,
+  "attend_to_expansion_tokens": false,
+  "skiplist_words": [
+    "!",
+    "\"",
+    "#",
+    "$",
+    "%",
+    "&",
+    "'",
+    "(",
+    ")",
+    "*",
+    "+",
+    ",",
+    "-",
+    ".",
+    "/",
+    ":",
+    ";",
+    "<",
+    "=",
+    ">",
+    "?",
+    "@",
+    "[",
+    "\\",
+    "]",
+    "^",
+    "_",
+    "`",
+    "{",
+    "|",
+    "}",
+    "~"
+  ],
+  "do_query_expansion": true
+}

configuration_bert_hash.py

@@ -0,0 +1,14 @@
+from transformers.models.bert.configuration_bert import BertConfig
+
+
+class BertHashConfig(BertConfig):
+    """
+    Extension of Bert configuration to add projections parameter.
+    """
+
+    model_type = "bert_hash"
+
+    def __init__(self, projections=5, **kwargs):
+        super().__init__(**kwargs)
+
+        self.projections = projections

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bc90b4bd7de9e1f1dada8731ef68a818939fec7b64c40b238f24ea64ce890d35
+size 3883056

modeling_bert_hash.py

@@ -0,0 +1,519 @@
+from typing import Optional, Union
+
+import torch
+from torch import nn
+from torch.nn import BCEWithLogitsLoss, CrossEntropyLoss, MSELoss
+
+from transformers.cache_utils import Cache
+from transformers.models.bert.modeling_bert import BertEncoder, BertPooler, BertPreTrainedModel, BertOnlyMLMHead
+from transformers.modeling_attn_mask_utils import _prepare_4d_attention_mask_for_sdpa, _prepare_4d_causal_attention_mask_for_sdpa
+from transformers.modeling_outputs import (
+    BaseModelOutputWithPoolingAndCrossAttentions,
+    MaskedLMOutput,
+    SequenceClassifierOutput,
+)
+from transformers.utils import auto_docstring, logging
+
+from .configuration_bert_hash import BertHashConfig
+
+logger = logging.get_logger(__name__)
+
+
+class BertHashTokens(nn.Module):
+    """
+    Module that embeds token vocabulary to an intermediate embeddings layer then projects those embeddings to the
+    hidden size.
+
+    The number of projections is like a hash. Setting the projections parameter to 5 is like generating a
+    160-bit hash (5 x float32) for each token. That hash is then projected to the hidden size.
+
+    This significantly reduces the number of parameters necessary for token embeddings.
+
+    For example:
+        Standard token embeddings:
+          30,522 (vocab size) x 768 (hidden size) = 23,440,896 parameters
+          23,440,896 x 4 (float32) = 93,763,584 bytes
+
+        Hash token embeddings:
+          30,522 (vocab size) x 5 (hash buckets) + 5 x 768 (projection matrix)= 156,450 parameters
+          156,450 x 4 (float32) = 625,800 bytes    
+    """
+
+    def __init__(self, config):
+        super().__init__()
+        self.config = config
+
+        # Token embeddings
+        self.embeddings = nn.Embedding(config.vocab_size, config.projections, padding_idx=config.pad_token_id)
+
+        # Token embeddings projections
+        self.projections = nn.Linear(config.projections, config.hidden_size)
+
+    def forward(self, input_ids):
+        # Project embeddings to hidden size
+        return self.projections(self.embeddings(input_ids))
+
+
+class BertHashEmbeddings(nn.Module):
+    """Construct the embeddings from word, position and token_type embeddings."""
+
+    def __init__(self, config):
+        super().__init__()
+        self.word_embeddings = BertHashTokens(config)
+        self.position_embeddings = nn.Embedding(config.max_position_embeddings, config.hidden_size)
+        self.token_type_embeddings = nn.Embedding(config.type_vocab_size, config.hidden_size)
+
+        # self.LayerNorm is not snake-cased to stick with TensorFlow model variable name and be able to load
+        # any TensorFlow checkpoint file
+        self.LayerNorm = nn.LayerNorm(config.hidden_size, eps=config.layer_norm_eps)
+        self.dropout = nn.Dropout(config.hidden_dropout_prob)
+        # position_ids (1, len position emb) is contiguous in memory and exported when serialized
+        self.position_embedding_type = getattr(config, "position_embedding_type", "absolute")
+        self.register_buffer(
+            "position_ids", torch.arange(config.max_position_embeddings).expand((1, -1)), persistent=False
+        )
+        self.register_buffer(
+            "token_type_ids", torch.zeros(self.position_ids.size(), dtype=torch.long), persistent=False
+        )
+
+    def forward(
+        self,
+        input_ids: Optional[torch.LongTensor] = None,
+        token_type_ids: Optional[torch.LongTensor] = None,
+        position_ids: Optional[torch.LongTensor] = None,
+        inputs_embeds: Optional[torch.FloatTensor] = None,
+        past_key_values_length: int = 0,
+    ) -> torch.Tensor:
+        if input_ids is not None:
+            input_shape = input_ids.size()
+        else:
+            input_shape = inputs_embeds.size()[:-1]
+
+        seq_length = input_shape[1]
+
+        if position_ids is None:
+            position_ids = self.position_ids[:, past_key_values_length : seq_length + past_key_values_length]
+
+        # Setting the token_type_ids to the registered buffer in constructor where it is all zeros, which usually occurs
+        # when its auto-generated, registered buffer helps users when tracing the model without passing token_type_ids, solves
+        # issue #5664
+        if token_type_ids is None:
+            if hasattr(self, "token_type_ids"):
+                buffered_token_type_ids = self.token_type_ids[:, :seq_length]
+                buffered_token_type_ids_expanded = buffered_token_type_ids.expand(input_shape[0], seq_length)
+                token_type_ids = buffered_token_type_ids_expanded
+            else:
+                token_type_ids = torch.zeros(input_shape, dtype=torch.long, device=self.position_ids.device)
+
+        if inputs_embeds is None:
+            inputs_embeds = self.word_embeddings(input_ids)
+        token_type_embeddings = self.token_type_embeddings(token_type_ids)
+
+        embeddings = inputs_embeds + token_type_embeddings
+        if self.position_embedding_type == "absolute":
+            position_embeddings = self.position_embeddings(position_ids)
+            embeddings += position_embeddings
+        embeddings = self.LayerNorm(embeddings)
+        embeddings = self.dropout(embeddings)
+        return embeddings
+
+
+@auto_docstring(
+    custom_intro="""
+    The model can behave as an encoder (with only self-attention) as well as a decoder, in which case a layer of
+    cross-attention is added between the self-attention layers, following the architecture described in [Attention is
+    all you need](https://huggingface.co/papers/1706.03762) by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit,
+    Llion Jones, Aidan N. Gomez, Lukasz Kaiser and Illia Polosukhin.
+
+    To behave as an decoder the model needs to be initialized with the `is_decoder` argument of the configuration set
+    to `True`. To be used in a Seq2Seq model, the model needs to initialized with both `is_decoder` argument and
+    `add_cross_attention` set to `True`; an `encoder_hidden_states` is then expected as an input to the forward pass.
+    """
+)
+class BertHashModel(BertPreTrainedModel):
+    config_class = BertHashConfig
+
+    _no_split_modules = ["BertEmbeddings", "BertLayer"]
+
+    def __init__(self, config, add_pooling_layer=True):
+        r"""
+        add_pooling_layer (bool, *optional*, defaults to `True`):
+            Whether to add a pooling layer
+        """
+        super().__init__(config)
+        self.config = config
+
+        self.embeddings = BertHashEmbeddings(config)
+        self.encoder = BertEncoder(config)
+
+        self.pooler = BertPooler(config) if add_pooling_layer else None
+
+        self.attn_implementation = config._attn_implementation
+        self.position_embedding_type = config.position_embedding_type
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    def get_input_embeddings(self):
+        return self.embeddings.word_embeddings.embeddings
+
+    def set_input_embeddings(self, value):
+        self.embeddings.word_embeddings.embeddings = value
+
+    def _prune_heads(self, heads_to_prune):
+        """
+        Prunes heads of the model. heads_to_prune: dict of {layer_num: list of heads to prune in this layer} See base
+        class PreTrainedModel
+        """
+        for layer, heads in heads_to_prune.items():
+            self.encoder.layer[layer].attention.prune_heads(heads)
+
+    @auto_docstring
+    def forward(
+        self,
+        input_ids: Optional[torch.Tensor] = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        token_type_ids: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.Tensor] = None,
+        head_mask: Optional[torch.Tensor] = None,
+        inputs_embeds: Optional[torch.Tensor] = None,
+        encoder_hidden_states: Optional[torch.Tensor] = None,
+        encoder_attention_mask: Optional[torch.Tensor] = None,
+        past_key_values: Optional[list[torch.FloatTensor]] = None,
+        use_cache: Optional[bool] = None,
+        output_attentions: Optional[bool] = None,
+        output_hidden_states: Optional[bool] = None,
+        return_dict: Optional[bool] = None,
+        cache_position: Optional[torch.Tensor] = None,
+    ) -> Union[tuple[torch.Tensor], BaseModelOutputWithPoolingAndCrossAttentions]:
+        output_attentions = output_attentions if output_attentions is not None else self.config.output_attentions
+        output_hidden_states = (
+            output_hidden_states if output_hidden_states is not None else self.config.output_hidden_states
+        )
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+
+        if self.config.is_decoder:
+            use_cache = use_cache if use_cache is not None else self.config.use_cache
+        else:
+            use_cache = False
+
+        if input_ids is not None and inputs_embeds is not None:
+            raise ValueError("You cannot specify both input_ids and inputs_embeds at the same time")
+        elif input_ids is not None:
+            self.warn_if_padding_and_no_attention_mask(input_ids, attention_mask)
+            input_shape = input_ids.size()
+        elif inputs_embeds is not None:
+            input_shape = inputs_embeds.size()[:-1]
+        else:
+            raise ValueError("You have to specify either input_ids or inputs_embeds")
+
+        batch_size, seq_length = input_shape
+        device = input_ids.device if input_ids is not None else inputs_embeds.device
+
+        past_key_values_length = 0
+        if past_key_values is not None:
+            past_key_values_length = (
+                past_key_values[0][0].shape[-2]
+                if not isinstance(past_key_values, Cache)
+                else past_key_values.get_seq_length()
+            )
+
+        if token_type_ids is None:
+            if hasattr(self.embeddings, "token_type_ids"):
+                buffered_token_type_ids = self.embeddings.token_type_ids[:, :seq_length]
+                buffered_token_type_ids_expanded = buffered_token_type_ids.expand(batch_size, seq_length)
+                token_type_ids = buffered_token_type_ids_expanded
+            else:
+                token_type_ids = torch.zeros(input_shape, dtype=torch.long, device=device)
+
+        embedding_output = self.embeddings(
+            input_ids=input_ids,
+            position_ids=position_ids,
+            token_type_ids=token_type_ids,
+            inputs_embeds=inputs_embeds,
+            past_key_values_length=past_key_values_length,
+        )
+
+        if attention_mask is None:
+            attention_mask = torch.ones((batch_size, seq_length + past_key_values_length), device=device)
+
+        use_sdpa_attention_masks = (
+            self.attn_implementation == "sdpa"
+            and self.position_embedding_type == "absolute"
+            and head_mask is None
+            and not output_attentions
+        )
+
+        # Expand the attention mask
+        if use_sdpa_attention_masks and attention_mask.dim() == 2:
+            # Expand the attention mask for SDPA.
+            # [bsz, seq_len] -> [bsz, 1, seq_len, seq_len]
+            if self.config.is_decoder:
+                extended_attention_mask = _prepare_4d_causal_attention_mask_for_sdpa(
+                    attention_mask,
+                    input_shape,
+                    embedding_output,
+                    past_key_values_length,
+                )
+            else:
+                extended_attention_mask = _prepare_4d_attention_mask_for_sdpa(
+                    attention_mask, embedding_output.dtype, tgt_len=seq_length
+                )
+        else:
+            # We can provide a self-attention mask of dimensions [batch_size, from_seq_length, to_seq_length]
+            # ourselves in which case we just need to make it broadcastable to all heads.
+            extended_attention_mask = self.get_extended_attention_mask(attention_mask, input_shape)
+
+        # If a 2D or 3D attention mask is provided for the cross-attention
+        # we need to make broadcastable to [batch_size, num_heads, seq_length, seq_length]
+        if self.config.is_decoder and encoder_hidden_states is not None:
+            encoder_batch_size, encoder_sequence_length, _ = encoder_hidden_states.size()
+            encoder_hidden_shape = (encoder_batch_size, encoder_sequence_length)
+            if encoder_attention_mask is None:
+                encoder_attention_mask = torch.ones(encoder_hidden_shape, device=device)
+
+            if use_sdpa_attention_masks and encoder_attention_mask.dim() == 2:
+                # Expand the attention mask for SDPA.
+                # [bsz, seq_len] -> [bsz, 1, seq_len, seq_len]
+                encoder_extended_attention_mask = _prepare_4d_attention_mask_for_sdpa(
+                    encoder_attention_mask, embedding_output.dtype, tgt_len=seq_length
+                )
+            else:
+                encoder_extended_attention_mask = self.invert_attention_mask(encoder_attention_mask)
+        else:
+            encoder_extended_attention_mask = None
+
+        # Prepare head mask if needed
+        # 1.0 in head_mask indicate we keep the head
+        # attention_probs has shape bsz x n_heads x N x N
+        # input head_mask has shape [num_heads] or [num_hidden_layers x num_heads]
+        # and head_mask is converted to shape [num_hidden_layers x batch x num_heads x seq_length x seq_length]
+        head_mask = self.get_head_mask(head_mask, self.config.num_hidden_layers)
+
+        encoder_outputs = self.encoder(
+            embedding_output,
+            attention_mask=extended_attention_mask,
+            head_mask=head_mask,
+            encoder_hidden_states=encoder_hidden_states,
+            encoder_attention_mask=encoder_extended_attention_mask,
+            past_key_values=past_key_values,
+            use_cache=use_cache,
+            output_attentions=output_attentions,
+            output_hidden_states=output_hidden_states,
+            return_dict=return_dict,
+            cache_position=cache_position,
+        )
+        sequence_output = encoder_outputs[0]
+        pooled_output = self.pooler(sequence_output) if self.pooler is not None else None
+
+        if not return_dict:
+            return (sequence_output, pooled_output) + encoder_outputs[1:]
+
+        return BaseModelOutputWithPoolingAndCrossAttentions(
+            last_hidden_state=sequence_output,
+            pooler_output=pooled_output,
+            past_key_values=encoder_outputs.past_key_values,
+            hidden_states=encoder_outputs.hidden_states,
+            attentions=encoder_outputs.attentions,
+            cross_attentions=encoder_outputs.cross_attentions,
+        )
+
+
+@auto_docstring
+class BertHashForMaskedLM(BertPreTrainedModel):
+    _tied_weights_keys = ["predictions.decoder.bias", "cls.predictions.decoder.weight"]
+    config_class = BertHashConfig
+
+    def __init__(self, config):
+        super().__init__(config)
+
+        if config.is_decoder:
+            logger.warning(
+                "If you want to use `BertForMaskedLM` make sure `config.is_decoder=False` for "
+                "bi-directional self-attention."
+            )
+
+        self.bert = BertHashModel(config, add_pooling_layer=False)
+        self.cls = BertOnlyMLMHead(config)
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    @auto_docstring
+    def forward(
+        self,
+        input_ids: Optional[torch.Tensor] = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        token_type_ids: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.Tensor] = None,
+        head_mask: Optional[torch.Tensor] = None,
+        inputs_embeds: Optional[torch.Tensor] = None,
+        encoder_hidden_states: Optional[torch.Tensor] = None,
+        encoder_attention_mask: Optional[torch.Tensor] = None,
+        labels: Optional[torch.Tensor] = None,
+        output_attentions: Optional[bool] = None,
+        output_hidden_states: Optional[bool] = None,
+        return_dict: Optional[bool] = None,
+    ) -> Union[tuple[torch.Tensor], MaskedLMOutput]:
+        r"""
+        labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):
+            Labels for computing the masked language modeling loss. Indices should be in `[-100, 0, ...,
+            config.vocab_size]` (see `input_ids` docstring) Tokens with indices set to `-100` are ignored (masked), the
+            loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`
+        """
+
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+
+        outputs = self.bert(
+            input_ids,
+            attention_mask=attention_mask,
+            token_type_ids=token_type_ids,
+            position_ids=position_ids,
+            head_mask=head_mask,
+            inputs_embeds=inputs_embeds,
+            encoder_hidden_states=encoder_hidden_states,
+            encoder_attention_mask=encoder_attention_mask,
+            output_attentions=output_attentions,
+            output_hidden_states=output_hidden_states,
+            return_dict=return_dict,
+        )
+
+        sequence_output = outputs[0]
+        prediction_scores = self.cls(sequence_output)
+
+        masked_lm_loss = None
+        if labels is not None:
+            loss_fct = CrossEntropyLoss()  # -100 index = padding token
+            masked_lm_loss = loss_fct(prediction_scores.view(-1, self.config.vocab_size), labels.view(-1))
+
+        if not return_dict:
+            output = (prediction_scores,) + outputs[2:]
+            return ((masked_lm_loss,) + output) if masked_lm_loss is not None else output
+
+        return MaskedLMOutput(
+            loss=masked_lm_loss,
+            logits=prediction_scores,
+            hidden_states=outputs.hidden_states,
+            attentions=outputs.attentions,
+        )
+
+    def prepare_inputs_for_generation(self, input_ids, attention_mask=None, **model_kwargs):
+        input_shape = input_ids.shape
+        effective_batch_size = input_shape[0]
+
+        #  add a dummy token
+        if self.config.pad_token_id is None:
+            raise ValueError("The PAD token should be defined for generation")
+
+        attention_mask = torch.cat([attention_mask, attention_mask.new_zeros((attention_mask.shape[0], 1))], dim=-1)
+        dummy_token = torch.full(
+            (effective_batch_size, 1), self.config.pad_token_id, dtype=torch.long, device=input_ids.device
+        )
+        input_ids = torch.cat([input_ids, dummy_token], dim=1)
+
+        return {"input_ids": input_ids, "attention_mask": attention_mask}
+
+    @classmethod
+    def can_generate(cls) -> bool:
+        """
+        Legacy correction: BertForMaskedLM can't call `generate()` from `GenerationMixin`, even though it has a
+        `prepare_inputs_for_generation` method.
+        """
+        return False
+
+
+@auto_docstring(
+    custom_intro="""
+    Bert Model transformer with a sequence classification/regression head on top (a linear layer on top of the pooled
+    output) e.g. for GLUE tasks.
+    """
+)
+class BertHashForSequenceClassification(BertPreTrainedModel):
+    config_class = BertHashConfig
+
+    def __init__(self, config):
+        super().__init__(config)
+        self.num_labels = config.num_labels
+        self.config = config
+
+        self.bert = BertHashModel(config)
+        classifier_dropout = (
+            config.classifier_dropout if config.classifier_dropout is not None else config.hidden_dropout_prob
+        )
+        self.dropout = nn.Dropout(classifier_dropout)
+        self.classifier = nn.Linear(config.hidden_size, config.num_labels)
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    @auto_docstring
+    def forward(
+        self,
+        input_ids: Optional[torch.Tensor] = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        token_type_ids: Optional[torch.Tensor] = None,
+        position_ids: Optional[torch.Tensor] = None,
+        head_mask: Optional[torch.Tensor] = None,
+        inputs_embeds: Optional[torch.Tensor] = None,
+        labels: Optional[torch.Tensor] = None,
+        output_attentions: Optional[bool] = None,
+        output_hidden_states: Optional[bool] = None,
+        return_dict: Optional[bool] = None,
+    ) -> Union[tuple[torch.Tensor], SequenceClassifierOutput]:
+        r"""
+        labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*):
+            Labels for computing the sequence classification/regression loss. Indices should be in `[0, ...,
+            config.num_labels - 1]`. If `config.num_labels == 1` a regression loss is computed (Mean-Square loss), If
+            `config.num_labels > 1` a classification loss is computed (Cross-Entropy).
+        """
+        return_dict = return_dict if return_dict is not None else self.config.use_return_dict
+
+        outputs = self.bert(
+            input_ids,
+            attention_mask=attention_mask,
+            token_type_ids=token_type_ids,
+            position_ids=position_ids,
+            head_mask=head_mask,
+            inputs_embeds=inputs_embeds,
+            output_attentions=output_attentions,
+            output_hidden_states=output_hidden_states,
+            return_dict=return_dict,
+        )
+
+        pooled_output = outputs[1]
+
+        pooled_output = self.dropout(pooled_output)
+        logits = self.classifier(pooled_output)
+
+        loss = None
+        if labels is not None:
+            if self.config.problem_type is None:
+                if self.num_labels == 1:
+                    self.config.problem_type = "regression"
+                elif self.num_labels > 1 and (labels.dtype == torch.long or labels.dtype == torch.int):
+                    self.config.problem_type = "single_label_classification"
+                else:
+                    self.config.problem_type = "multi_label_classification"
+
+            if self.config.problem_type == "regression":
+                loss_fct = MSELoss()
+                if self.num_labels == 1:
+                    loss = loss_fct(logits.squeeze(), labels.squeeze())
+                else:
+                    loss = loss_fct(logits, labels)
+            elif self.config.problem_type == "single_label_classification":
+                loss_fct = CrossEntropyLoss()
+                loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
+            elif self.config.problem_type == "multi_label_classification":
+                loss_fct = BCEWithLogitsLoss()
+                loss = loss_fct(logits, labels)
+        if not return_dict:
+            output = (logits,) + outputs[2:]
+            return ((loss,) + output) if loss is not None else output
+
+        return SequenceClassifierOutput(
+            loss=loss,
+            logits=logits,
+            hidden_states=outputs.hidden_states,
+            attentions=outputs.attentions,
+        )

modules.json

@@ -0,0 +1,14 @@
+[
+  {
+    "idx": 0,
+    "name": "0",
+    "path": "",
+    "type": "sentence_transformers.models.Transformer"
+  },
+  {
+    "idx": 1,
+    "name": "1",
+    "path": "1_Dense",
+    "type": "pylate.models.Dense.Dense"
+  }
+]

optimizer.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:97245b9ef2b05399f151f4eeba08819b3a584a079c260e9abf70ab6c9a70bfad
+size 7790987

rng_state.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d46800e6e3ce4bcc9b3ba403638c28e1036a2fca5005396a4c64f753ed44d765
+size 14645

scaler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:125dd9a754f313ede3918b2838113df93582e51376581e56793338dd7645bd93
+size 1383

scheduler.pt

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b198d63af5c6563399a525397894e9b3d1958651f2304ed8bae0db0183a0164
+size 1465

sentence_bert_config.json

@@ -0,0 +1,4 @@
+{
+    "max_seq_length": 179,
+    "do_lower_case": false
+}

special_tokens_map.json

@@ -0,0 +1,31 @@
+{
+  "cls_token": {
+    "content": "[CLS]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "mask_token": {
+    "content": "[MASK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "pad_token": "[MASK]",
+  "sep_token": {
+    "content": "[SEP]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  },
+  "unk_token": {
+    "content": "[UNK]",
+    "lstrip": false,
+    "normalized": false,
+    "rstrip": false,
+    "single_word": false
+  }
+}

tokenizer_config.json

@@ -0,0 +1,79 @@
+{
+  "added_tokens_decoder": {
+    "0": {
+      "content": "[PAD]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "100": {
+      "content": "[UNK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "101": {
+      "content": "[CLS]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "102": {
+      "content": "[SEP]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "103": {
+      "content": "[MASK]",
+      "lstrip": false,
+      "normalized": false,
+      "rstrip": false,
+      "single_word": false,
+      "special": true
+    },
+    "30522": {
+      "content": "[Q] ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    },
+    "30523": {
+      "content": "[D] ",
+      "lstrip": false,
+      "normalized": true,
+      "rstrip": false,
+      "single_word": false,
+      "special": false
+    }
+  },
+  "clean_up_tokenization_spaces": false,
+  "cls_token": "[CLS]",
+  "do_lower_case": true,
+  "extra_special_tokens": {},
+  "mask_token": "[MASK]",
+  "max_length": 512,
+  "model_max_length": 512,
+  "pad_to_multiple_of": null,
+  "pad_token": "[MASK]",
+  "pad_token_type_id": 0,
+  "padding_side": "right",
+  "sep_token": "[SEP]",
+  "stride": 0,
+  "strip_accents": null,
+  "tokenize_chinese_chars": true,
+  "tokenizer_class": "BertTokenizer",
+  "truncation_side": "right",
+  "truncation_strategy": "longest_first",
+  "unk_token": "[UNK]"
+}

trainer_state.json

@@ -0,0 +1,104 @@
+{
+  "best_global_step": null,
+  "best_metric": null,
+  "best_model_checkpoint": null,
+  "epoch": 0.08343360002810395,
+  "eval_steps": 1000,
+  "global_step": 9500,
+  "is_hyper_param_search": false,
+  "is_local_process_zero": true,
+  "is_world_process_zero": true,
+  "log_history": [
+    {
+      "epoch": 8.782484213484626e-06,
+      "grad_norm": NaN,
+      "learning_rate": 3e-06,
+      "loss": 1.9342,
+      "step": 1
+    },
+    {
+      "epoch": 0.008782484213484626,
+      "grad_norm": 11.566916465759277,
+      "learning_rate": 2.9736788948121867e-06,
+      "loss": 1.3908,
+      "step": 1000
+    },
+    {
+      "epoch": 0.017564968426969252,
+      "grad_norm": 13.525375366210938,
+      "learning_rate": 2.9473314421717326e-06,
+      "loss": 1.1178,
+      "step": 2000
+    },
+    {
+      "epoch": 0.026347452640453878,
+      "grad_norm": 13.769821166992188,
+      "learning_rate": 2.920983989531279e-06,
+      "loss": 1.0264,
+      "step": 3000
+    },
+    {
+      "epoch": 0.035129936853938504,
+      "grad_norm": 14.206275939941406,
+      "learning_rate": 2.8946365368908247e-06,
+      "loss": 0.9816,
+      "step": 4000
+    },
+    {
+      "epoch": 0.04391242106742313,
+      "grad_norm": 15.29100513458252,
+      "learning_rate": 2.868289084250371e-06,
+      "loss": 0.9313,
+      "step": 5000
+    },
+    {
+      "epoch": 0.052694905280907756,
+      "grad_norm": 19.36515235900879,
+      "learning_rate": 2.8419416316099173e-06,
+      "loss": 0.9324,
+      "step": 6000
+    },
+    {
+      "epoch": 0.06147738949439238,
+      "grad_norm": 16.350460052490234,
+      "learning_rate": 2.8155941789694636e-06,
+      "loss": 0.8896,
+      "step": 7000
+    },
+    {
+      "epoch": 0.07025987370787701,
+      "grad_norm": 8.30239200592041,
+      "learning_rate": 2.7892467263290094e-06,
+      "loss": 0.8703,
+      "step": 8000
+    },
+    {
+      "epoch": 0.07904235792136163,
+      "grad_norm": 17.16596794128418,
+      "learning_rate": 2.7628992736885557e-06,
+      "loss": 0.8665,
+      "step": 9000
+    }
+  ],
+  "logging_steps": 1000,
+  "max_steps": 113863,
+  "num_input_tokens_seen": 0,
+  "num_train_epochs": 1,
+  "save_steps": 500,
+  "stateful_callbacks": {
+    "TrainerControl": {
+      "args": {
+        "should_epoch_stop": false,
+        "should_evaluate": false,
+        "should_log": false,
+        "should_save": true,
+        "should_training_stop": false
+      },
+      "attributes": {}
+    }
+  },
+  "total_flos": 0.0,
+  "train_batch_size": 8,
+  "trial_name": null,
+  "trial_params": null
+}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0015d449c32a217387af36a8fd35ed78fb8c69b20b3934f06a042ba284240858
+size 6161