Peter Bloem on Nostr: Let me go point by point here. - Curation doesn't mean manual selection. It just ...

Let me go point by point here.
- Curation doesn't mean manual selection. It just means that you don't blindly ingest everything from the internet. This has been done since GPT2, when they used social media upvotes as a proxy for article quality. In GPT3, they included sources proportional to the quality of the source which gives you the best of both worlds.
- All sorts of junk sneaks into the data, but so long as the majority is high quality, it won't lead to model collapse.
- "Glue on Pizza" came from the Google AI summary which is a poor proxy for current AI capability.
- Watermarking can and does 100% work as advertised. For a simple insight to the idea, consider the bitstream used to feed the PRNG. If you have the complete model output of one session and the model, you can reconstruct that. Save it, use the same stream each session and you have your watermark with no changes to the model (some extra tricks are necessary to watermark subsets of the conversation).
- It is indeed relatively easy to circumvent, although the precise method they use seems to be somewhat robust to basic rephrasing. Still, it's going to catch out a lot of lazy people.