I’ve worked at places that were incredibly slow at delivering software and spent massive amounts of time and money re-inventing wheels because the self-organizing team thought it was better.

I think all your criticisms hit the nail on the head. You need real users giving feedback, good loops, NONE of which require anyone to be in the same office, and you need good leadership. Not all self-organizing teams organize around good leaders or even have leaders. Some will say “we don’t need leaders!”, except you work in enough organizations and you see that yes, you do, and before that you were probably just getting lucky.

Good point about requirements. The only people who can truly say a product is correct are the users. Everything else is in the imagination.