Tony "Lemme Scrape" Bacon on Nostr: Well, local model is working. It's pretty retarded though and can't keep ...
Well, local model is working. It's pretty retarded though and can't keep context/multiple tasks straight, but cpu isn't currently overheating so that's pretty based.
I was actually surprised that the RTX 3060 (12 gb vram) is actually handling a 30b model. It's a bit slow, but I'm using a context window of 65k and the highest i've seen the gpu usage spike is about 80%
BUT that being said, it's offloading a good chunk of the load onto the cpu which, due to my too small fan, is causing the cpu to hit literally 94-96c lmao.
New fan coming today, so shouldn't be an issue shortly.
Published at
2026-10-29 17:26:27 UTCEvent JSON
{
"id": "905198124a7c41c8e423f36c932311200a88174f88ce4c6faabc9f18f5197839",
"pubkey": "8639304d3dab1042ac8d11f9849284c018bf282d1d447f878af5e391205c3898",
"created_at": 1793294787,
"kind": 1,
"tags": [
[
"imeta",
"url https://clew.lol/media/9dcbaba8-95b1-4cd5-af95-8240cba84014/image.png",
"m image/png"
],
[
"imeta",
"url https://clew.lol/media/2f54a3a5-7717-4cd6-ab71-1eea86705b1e/image.png",
"m image/png"
],
[
"imeta",
"url https://clew.lol/media/232398d9-5ba5-43c3-8b39-a54da7135ec3/image.png",
"m image/png"
],
[
"proxy",
"https://clew.lol/objects/48b8198a-d261-417e-a535-ff603a5e9eeb",
"activitypub"
],
[
"L",
"pink.momostr"
],
[
"l",
"pink.momostr.activitypub:https://clew.lol/objects/48b8198a-d261-417e-a535-ff603a5e9eeb",
"pink.momostr"
],
[
"-"
]
],
"content": "Well, local model is working. It's pretty retarded though and can't keep context/multiple tasks straight, but cpu isn't currently overheating so that's pretty based.\r\n\r\nI was actually surprised that the RTX 3060 (12 gb vram) is actually handling a 30b model. It's a bit slow, but I'm using a context window of 65k and the highest i've seen the gpu usage spike is about 80% \r\n\r\nBUT that being said, it's offloading a good chunk of the load onto the cpu which, due to my too small fan, is causing the cpu to hit literally 94-96c lmao. \r\n\r\nNew fan coming today, so shouldn't be an issue shortly.\nhttps://clew.lol/media/9dcbaba8-95b1-4cd5-af95-8240cba84014/image.png\nhttps://clew.lol/media/2f54a3a5-7717-4cd6-ab71-1eea86705b1e/image.png\nhttps://clew.lol/media/232398d9-5ba5-43c3-8b39-a54da7135ec3/image.png\n",
"sig": "19740931e56c1d9ee8d07141c1406bff74ed73219f79975d910fe82d3848c363c6dde90e49a7755d14c6f6d456e9bdc5cc4de656fc5e86ac12641e0e21853267"
}