I do not like or endorse the idea of wearing headphones while riding, especially on roads and streets, however, I really enjoy listening to music. My alternative has been a relatively small bluetooth stereo speaker. It is shaped almost cylindrically less than 2 inches diameter (like 4 cm) by 8 inches (20 cm).
I mounted it on the handlebar for riding on roads. I can clearly hear the music when car traffic is far, but as soon as a car approaches, it's noise overcomes the sound of my speaker. Granted, that "interrupts" music, but I'd rather hear that big truck approaching or that racer-wannabe revving the engine, as that alerts me of their presence and prompts me to glance at them to check if they pose danger for me.
For city riding, I've used the same speaker but inside a top pocket in my backpack, so it is near the back of my head. The result is almost the same. Of course the public also hears my music, but since the speaker is near my head and the streets are noisy, most people didn't notice until I get close to them (A little bit over normal conversation range). Here I set the volume such that I could hear a car approaching, claxons, etc.
I do not think that would be annoying to pedestrians since the street is already filled with too many other sounds. On some quieter streets with cycle paths that where usually invaded by pedestrians, sometimes the music served as a sort of "here I am/here I come" announcer, causing pedestrians to turn their head and acknowledge my presence before I rang my bike bell. (Not that I relied on that, but It happened)
With this setup, sometimes a friend rode along with me and we could sustain a conversation without me having to stop the music, just turned volume down a bit.
I preferred this way of listening to music because as nothing physically blocks my ears, I could easily hear many danger-alerting sounds, so I felt not isolated from the environment. Earbuds by themselves tend to block part of ambient sound, even with no music playing, and over the ear headphones tend to interfere with helmet straps and do not sound as good as my speaker at close range.
There are even commercially available speakers that fit in a standard bottle cage.
On the other hand, I wouldn't use a speaker while mountainbiking with other people, because in a more quiet environment where other people will remain near, I would not force them to hear music they don't want to. That is, normal courtesy should be observed regarding speakers.