unsloth

/

gpt-oss-120b-GGUF

like 193

Follow

Unsloth AI 12k

[Moving here since I accidentally filed it in the non-gguf repo]

Something is wrong with Q5_K_M and maybe others. So many tokens are just "missing". I am running the llama.cpp webui but also see it from directly interacting with the model. Some tokens just appear to be dropped or something! It affects the output quality, since it produces Python programs which do not work. Spot the two missing tokens in this snippet:

    def delete_task(self, task_id: int) ->:
        """
 Delete a task by its primary key.

        Args:
            task_id: The integer identifier of the task.
        """
        sql = "DELETE FROM tasks WHERE id = ?"
        with self._lock, self._conn:  # type: ignore[misc]
            try:
                cur = self._conn.execute(sql, (task_id,))  # type: ignore[union-attr]
                if cur.rowcount == 0:
                    raise DatabaseError(f"No task with id={task_id} to delete.")
                logging.getLogger(__name__).info("Deleted task=%s", task_id)
            except sqlite3.Error as exc:
                raise DatabaseError(f"Failed to delete task: {exc}") from exc

I am not able to do any coding with the model because the dropped tokens frequently cause syntax errors.