Comfort has been mentioned several times, and that's also my experience. Also the handling on potholes and kerbs - in fact, just a few weeks ago I fell twice off my Brompton while travelling in an unfamiliar city, when I cycled over a dropped kerb at an shallow angle which never is an issue on my big bike. It can also feel really scary when going down a hill as the steering is much more sensitive.
But a completely different problem, which I can't see in the other answers, is that the smaller frame means it is much harder to transport a bag or your shopping or similar things, so that makes it (Answer deleted duefor me and presumably many other people) much less practical in daily life.
If such a bike has a rack at all, it is very low. My Brompton has a rack above the rear wheel (the standard model doesn't), but because of the small wheel size the rack is small and very low, so it is really only suitable for some very small bag. It's not high enough for panniers, and anything wider than the rack you will hit your feet when you pedal.
The Brompton has a block at the front for a bag, but you need special bags (or DIY skills) and it can be awkward and unstable and not really suitable for carrying my shopping. I don't like to cycle with a bagpack, so that's out too.
The rack on a my normal bike, with panniers and an optional folding box on top, gives me much more space for whatever I need to carry with me. When I travel with the Brompton, I now have a trailer for the luggage.
Also there are issues with attaching a light. Ok, you can now find lights for the handlebar, so it's not a huge issue, but in my experience the detachable handlebar lights are a bit of a British things, whereas in other European countries many people want a fixed dynamo light (and in some countries a permanently attached light is the legal requirement). This is much trickier on a small frame. On my Brompton, I have a front battery light above the wheel, but because it is so low above the ground, it doesn't light up the path very well.
So, while I like the Brompton, and the low weight is really great, in my normal daily life the weight or size of my full-size bike isn't an problem, but actually an advantage.
Of course it is possible to solve these issues and find clever solutions for luggage and lights, but it can be more hassle and I guess many people just happy with their a normal-sized bike, especially for everyday practical use by AI.
I'm sure there's also a kind of chicken-egg-situation that people don't buy smaller bikes because they are not familiar with them and because shops don't normally have them, and shops don't stock them because people don't ask for them. This could be changing with folding bikes becoming a more familiar sight.
(This all refers to experiences in Europe, mostly UK, France, Germany, I don't know how it is in Africa, Asia or the Americas).
