---
license: apache-2.0
language:
- en
metrics:
- bleu
- rouge
tags:
- causal-lm
- code
- cypher
- graph
- neo4j
inference: false
---

## Model Description

A finetune of https://huggingface.co/stabilityai/stable-code-instruct-3b trained on https://github.com/neo4j-labs/text2cypher/tree/main/datasets/synthetic_opus_demodbs to generate CYPHER statements for GraphDB queries such as neo4j.

## Usage

### Safetensors (recommended)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load the model and tokenizer
print("Loading model...")
model_name = "path/to/your/safetensors/model" #./stable-cypher-instruct-3b
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16, device_map="auto")

# Define your question
'''instruction is VERY IMPORTANT the model was finetuned on this particular system prompt. 
Except bad performance without it'''
instruction = "Create a Cypher statement to answer the following question:"
question = "List the first 3 articles mentioning organizations with a revenue less than 5 million."

# Create the full prompt
full_prompt = f"{instruction}\n\nHuman: {question}\n\nAssistant:"

# Tokenize input
inputs = tokenizer(full_prompt, return_tensors="pt")

# Generate response
print("Generating response...")
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=True,
        top_p=0.9,
        temperature=0.2,
        pad_token_id=tokenizer.eos_token_id,
    )

# Decode and print the generated response
answer = tokenizer.decode(outputs[0], skip_special_tokens=True)
answer = answer[len(full_prompt):].strip()  # Remove the input prompt from the output

print("\nQuestion:", question)
print("\nGenerated Cypher statement:")
print(answer)
```

### GGUF

```python
from llama_cpp import Llama

# Load the GGUF model
print("Loading model...")
model = Llama(
    model_path="stable-cypher-instruct-3b.Q4_K_M.gguf",
    n_ctx=512,
    n_batch=512,
    n_gpu_layers=-1,  # Use all available GPU layers
    max_tokens=128,
    top_p=0.9,
    temperature=0.2,
    verbose=False 
)

# Define your question
'''instruction is VERY IMPORTANT the model was finetuned on this particular system prompt. 
Except bad performance without it'''
instruction = "Create a Cypher statement to answer the following question:" 
question = "List the first 3 articles mentioning organizations with a revenue less than 5 million."

# Create the full prompt
full_prompt = f"{instruction}\n\nHuman: {question}\n\nAssistant:"

# Generate response
print("Generating response...")
response = model(
    full_prompt,
    max_tokens=128,
    stop=["Human:", "\n\n"],
    echo=False
)

# Extract and print the generated response
answer = response['choices'][0]['text'].strip()
print("\nQuestion:", question)
print("\nGenerated Cypher statement:")
print(answer)
```

## Performance


| Metric  | stable-code-instruct-3b | stable-cypher-instruct-3b |
| --------- | ------------------------- | --------------------------- |
| BLEU-4  | 19.07                   | 88.63                     |
| ROUGE-1 | 39.49                   | 95.09                     |
| ROUGE-2 | 24.82                   | 90.71                     |
| ROUGE-L | 29.63                   | 91.51                     |

### Example

#### Stable Cypher
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6504bb76423b46492e7f38c7/pweL4qgmFaknLBYp-CGHm.png)

#### Stable Code
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6504bb76423b46492e7f38c7/YwMENiOk6JU14xT_wfAeN.png)

### Eval params
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6504bb76423b46492e7f38c7/AT80-09XrHNz-dJs9TH3M.png)

## Reproducability

This is the config file from Llama Factory :

```json
{
  "top.model_name": "Custom",
  "top.finetuning_type": "lora",
  "top.adapter_path": [],
  "top.quantization_bit": "none",
  "top.template": "default",
  "top.rope_scaling": "none",
  "top.booster": "none",
  "train.training_stage": "Supervised Fine-Tuning",
  "train.dataset_dir": "data",
  "train.dataset": [
    "cypher_opus"
  ],
  "train.learning_rate": "2e-4",
  "train.num_train_epochs": "5.0",
  "train.max_grad_norm": "1.0",
  "train.max_samples": "5000",
  "train.compute_type": "fp16",
  "train.cutoff_len": 256,
  "train.batch_size": 16,
  "train.gradient_accumulation_steps": 2,
  "train.val_size": 0.1,
  "train.lr_scheduler_type": "cosine",
  "train.logging_steps": 10,
  "train.save_steps": 100,
  "train.warmup_steps": 20,
  "train.neftune_alpha": 0,
  "train.optim": "adamw_torch",
  "train.resize_vocab": false,
  "train.packing": false,
  "train.upcast_layernorm": false,
  "train.use_llama_pro": false,
  "train.shift_attn": false,
  "train.report_to": false,
  "train.num_layer_trainable": 3,
  "train.name_module_trainable": "all",
  "train.lora_rank": 64,
  "train.lora_alpha": 64,
  "train.lora_dropout": 0.1,
  "train.loraplus_lr_ratio": 0,
  "train.create_new_adapter": false,
  "train.use_rslora": false,
  "train.use_dora": true,
  "train.lora_target": "",
  "train.additional_target": "",
  "train.dpo_beta": 0.1,
  "train.dpo_ftx": 0,
  "train.orpo_beta": 0.1,
  "train.reward_model": null,
  "train.use_galore": false,
  "train.galore_rank": 16,
  "train.galore_update_interval": 200,
  "train.galore_scale": 0.25,
  "train.galore_target": "all"
}
```

I used llama.cpp to merge the LoRa and generate the quants.

The progress achieved from the base model is significant but this is just a first draft.
I ran a few batches of training tickering with some of the values but was far from being exhaustive and thourough.
Main concern is the capability of the model to expand on unseen fields and syntax.
I'm open to the idea of making a v2 which (should) be production ready and a full tutorial if there is enough interest in this project.