---
pipeline_tag: translation
language: multilingual
library_name: transformers
base_model:
- FacebookAI/xlm-roberta-large
license: apache-2.0
---
đź“Š Estimating Machine Translation Difficulty
This repository contains one of the two **SENTINELSRC** metric models analyzed in our paper **Estimating Machine Translation Difficulty**.
## Usage
To run this model, install the following git repository:
```bash
pip install git+https://github.com/prosho-97/guardians-mt-eval
```
After that, you can use this model within Python in the following way:
```python
from sentinel_metric import download_model, load_from_checkpoint
model_path = download_model("Prosho/sentinel-src-24")
model = load_from_checkpoint(model_path)
data = [
{"src": "Please sign the form."},
{"src": "He spilled the beans, then backpedaled—talk about mixed signals!"}
]
output = model.predict(data, batch_size=8, gpus=1)
```
Output:
```python
# Segment scores
>>> output.scores
[0.5726182460784912, -0.12408381700515747]
# System score
>>> output.system_score
0.22426721453666687
```
Where the higher the output score, the easier it is to translate the input source text.
## Cite this work
This work has been accepted at [EMNLP 2025](https://2025.emnlp.org/). If you use any part, please consider citing our paper as follows:
```bibtex
@misc{proietti2025estimatingmachinetranslationdifficulty,
title={Estimating Machine Translation Difficulty},
author={Lorenzo Proietti and Stefano Perrella and Vilém Zouhar and Roberto Navigli and Tom Kocmi},
year={2025},
eprint={2508.10175},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2508.10175},
}
``` 
