I went through something similar. With Vim, I realized I was spending a fair amount of time tweaking and adjusting my configuration for relatively little benefit.
The turning point for me was learning to live without syntax highlighting. Once I got used to that, moving to nvi felt natural, and there was really no going back.
These days I prefer having a small editor that I don't have to think about or constantly configure. I just open the file and edit it.