Peter Bloem on Nostr: My understanding is that it's mostly happening because the EU requires it. ...

My understanding is that it's mostly happening because the EU requires it.

Pre-training datasets are pretty carefully curated and they have strong quality filters, so I don't think there is much of a risk of model collapse in practice.

It's also not so much about special characters or word choice as about the choices between equally good options (by the models' own probabilities). Every time you choose between two equally likely options you are essentially "encoding" one bit (i.e. a coinflip) of information. You can use these bits to create a watermark without losing quality.