Spaces:

Azure99
/

Blossom-V6.3-30B-A3B-Demo

Running on Zero

Azure99 commited on 7 days ago

Commit

c0d05d6

verified ·

1 Parent(s): 544f103

Update app.py

Files changed (1) hide show

app.py CHANGED Viewed

@@ -11,13 +11,13 @@ from transformers import (
 )
 MAX_NEW_TOKENS = 8192
-MODEL_NAME = "Azure99/Blossom-V6.2-32B"
 model = AutoModelForCausalLM.from_pretrained(
     MODEL_NAME,
     torch_dtype="auto",
     device_map="auto",
-    quantization_config=FineGrainedFP8Config(),
 )
 tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)

 )
 MAX_NEW_TOKENS = 8192
+MODEL_NAME = "Azure99/Blossom-V6.3-30B-A3B"
 model = AutoModelForCausalLM.from_pretrained(
     MODEL_NAME,
     torch_dtype="auto",
     device_map="auto",
+    #quantization_config=FineGrainedFP8Config(),
 )
 tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)