Update README.md
Browse files
README.md
CHANGED
|
@@ -24,6 +24,7 @@ The model utilizes **fp8 dynamic (w8a8)** for optimal performance and deployment
|
|
| 24 |
## Just Run It (vLLM serve)
|
| 25 |
|
| 26 |
You can serve the model using vLLM's OpenAI-compatible API server.
|
|
|
|
| 27 |
*Warning: this model uses Gpt-oss as the base language model, and seems to have some issues running in vllm. Still digging in*
|
| 28 |
```bash
|
| 29 |
vllm serve brandonbeiler/InternVL3_5-GPT-OSS-20B-A4B-Preview-FP8-Dynamic \
|
|
|
|
| 24 |
## Just Run It (vLLM serve)
|
| 25 |
|
| 26 |
You can serve the model using vLLM's OpenAI-compatible API server.
|
| 27 |
+
|
| 28 |
*Warning: this model uses Gpt-oss as the base language model, and seems to have some issues running in vllm. Still digging in*
|
| 29 |
```bash
|
| 30 |
vllm serve brandonbeiler/InternVL3_5-GPT-OSS-20B-A4B-Preview-FP8-Dynamic \
|