jsr on Nostr: NEW: Cost to 'poison' an LLM and insert backdoors is relatively constant. Even as ...
NEW: Cost to 'poison' an LLM and insert backdoors is relatively constant. Even as models grow.
Implication: scaling security is orders-of-magnitude harder than scaling LLMs.
Prior work had suggested that as model sizes grew, it would make them cost-prohibitive to poison.
So, in LLM training-set-land, dilution isn't the solution to pollution.
Just about the same size of poisoned training data that works on a 1B model could also work on a 1T model.
I feel like this is something that cybersecurity folks will find intuitive: lots of attacks scale. Most defenses don't
PAPER: POISONING ATTACKS ON LLMS REQUIRE A NEAR-CONSTANT NUMBER OF POISON SAMPLES
https://arxiv.org/pdf/2510.07192Published at
2025-10-09 17:26:26 UTCEvent JSON
{
"id": "46b0237c96c1f41d6c499bb920b4f6bb7c9fbf048fa461bfbf0f46e2de4431b1",
"pubkey": "609f186ca023d658c0fe019570472f59565c8be1dc163b1541fac9d90aa4e8af",
"created_at": 1760030786,
"kind": 1,
"tags": [],
"content": "NEW: Cost to 'poison' an LLM and insert backdoors is relatively constant. Even as models grow. \n\nImplication: scaling security is orders-of-magnitude harder than scaling LLMs.\n\nhttps://blossom.primal.net/1bdbe13fe20b39f757d6d440b416a74a2099c63cb50bc344cc1d2e96f7c4646b.png \n\nPrior work had suggested that as model sizes grew, it would make them cost-prohibitive to poison.\n\nhttps://blossom.primal.net/d44c301ef8c297ee3eb30c7e8a161b5dcecc8618dee83607d1532d9d9ad63b02.png \n\nSo, in LLM training-set-land, dilution isn't the solution to pollution. \n\nJust about the same size of poisoned training data that works on a 1B model could also work on a 1T model. \nhttps://blossom.primal.net/2c635801a74e4ddc0628adb7d1f1942cb4431550474696a7a7e36702ecb042b7.png \nI feel like this is something that cybersecurity folks will find intuitive: lots of attacks scale. Most defenses don't\n\nPAPER: POISONING ATTACKS ON LLMS REQUIRE A NEAR-CONSTANT NUMBER OF POISON SAMPLES https://arxiv.org/pdf/2510.07192",
"sig": "2c52e58ae07ff47250aac0b26a208bda60b4f08c88d5cfe9a3d5bf7acaad55cb342df1632fa562760cf07a91f547ac4098f3dc7b98dea036c41feeadddd68c30"
}
