Anthony on Nostr: It seems to me that it's a bit worse than this, because the variability is probably ...

It seems to me that it's a bit worse than this, because the variability is probably long-tailed and there are buffering/queuing effects. When things go well, they go OK; when things go badly, they can go really, really badly. Anything downstream of something going well can also go well; anything downstream of something going really, really badly is unlikely to be able to compensate, and they risk "buffer overflow" if they try. I think in knowledge work and similar long-tailed tasks, increased volatility tends to mean things fail when they didn't need to.

This overall imbalance means that there are negative outcomes that are unlikely to be counterbalanced by corresponding positive outcomes at the system level, leading to net negative value. I think this partly explains why insurance companies are less and less likely to sell coverage to companies that heavily use AI in commercial policies: the actuaries have worked this out, and not willing to take on the extra risk.