Concern About Standard Quants
#1
by
davidpfarrell - opened
Greetings!
Not sure if you've seen comments posted by the Tesslate team that they don't think standard quants perform well:
We found that standard quantization significantly degrades quality and can break the model's reasoning chains. For the best results, we highly recommend running it in BF16 or FP8.
Your quants appear to be standard, so they may be subject to the same concerns ... Just wanted to make sure you were aware. Thanks for these and for your contributions to the community!
Hi thanks for sharing
Just looked the Reddit post. I can definitely share this info on the model card!
Thanks again