Join Nostr
2026-10-29 17:26:27 UTC

Tony "Lemme Scrape" Bacon on Nostr: Well, local model is working. It's pretty retarded though and can't keep ...

Well, local model is working. It's pretty retarded though and can't keep context/multiple tasks straight, but cpu isn't currently overheating so that's pretty based.

I was actually surprised that the RTX 3060 (12 gb vram) is actually handling a 30b model. It's a bit slow, but I'm using a context window of 65k and the highest i've seen the gpu usage spike is about 80%

BUT that being said, it's offloading a good chunk of the load onto the cpu which, due to my too small fan, is causing the cpu to hit literally 94-96c lmao.

New fan coming today, so shouldn't be an issue shortly.