Im getting 50tk/s with qwen3.6 35B FP8 on vllm with speculative decoding. 250k tk context window.
Have to do a lot of configs to get things working right
npub1gv…9vyex
2026-06-20 03:19:28 UTC
in reply to nevent1q…tgsf
Author Public Key
npub1gvfmd977m580mmeegfrkljc2phulcs8gcwf5xs7q3mpx0zr88u7q09vyexPublished at
2026-06-20 03:19:28 UTCEvent JSON
{
"id": "52aa98660c307faff875196f320029237d0bad34e2946f62e625ba8170fe2689",
"pubkey": "4313b697dedd0efdef3942476fcb0a0df9fc40e8c3934343c08ec26788673f3c",
"created_at": 1781925568,
"kind": 1,
"tags": [
[
"e",
"e3322c5fbd7e23aef355a88c3688c2f7f76d63cb62beb8092ec2e558ea4edc9d",
"",
"root"
],
[
"e",
"a0d89fb5c066e03db0e836f84dd1a51e4467330f117dded5e0f0b4c118171a1f",
"",
"reply"
],
[
"p",
"c21b1a6cdb247ccbd938dcb16b15a4fa382d00ffd7b12d5cbbad172a0cd4d170",
"",
"mention"
]
],
"content": "Im getting 50tk/s with qwen3.6 35B FP8 on vllm with speculative decoding. 250k tk context window.\nHave to do a lot of configs to get things working right",
"sig": "590d6b4307aa63842893337958edd23508147438ab1c940affb218ba950c4deb48d0d40b0372cb9b1e7222315481bddc4dd2a87106298082f46a7f48e3c11940"
}