Update tokenizer
#9
by
alvarobartt HF Staff - opened
Description
Updates both tokenizer.json and special_tokens_map.json as in https://huggingface.co/meta-llama/Meta-Llama-3.1-70B-Instruct/commit/846357c7ee5e3f50575fd4294edb3d898c8ea100
Closes #6
alvarobartt changed pull request title from
Update special_tokens_map.json
to Update tokenizer
alvarobartt changed pull request status to
merged