M5 is used far on far heavier luggage too, like rear racks. You don't want too many sizes to keep in your spare parts (I carry a couple of M5 and a couple of M6 screws on the bike in my toolkit, plus nuts, and have used them).
Standardisation is useful even when you've got more room for spares and don't care about the weight - the vast majority of the screws in my optics lab are M4 or M6, though I've used everything from M1 to M10 there when necessary, and not just in my own designs.This holds at manufacturers too - they don't want a massive parts inventory, even of cheap parts. Similarly on bikes, other sizes are used if necessary:there are M3 and perhaps smaller screws in gear assemblies; I think I've seen M2.5 holding covers on housings, but axles are big.
We should assume they're screwed into aluminium, where fatigue on the mating thread is an issue, and, we should also assume they're taken in and out a few times, torqued quite hard each time, as you would fitting a luggage rack. A challenge for you is to try stripping a range of threads in aluminium, all with the same depth engaged - perhaps drill and tap into 3mm aluminium plate. Maybe a thought experiment would do, though I have the tools to try it here. With a few mm of engagement, M3 will strip with some effort, M4 possibly, M5 unlikely and M6 only with a cheater bar. But each time they're done up the aluminium gets weaker. That will soon make up for the thicker plate you'd have in reality - as would using a screw that's a little short.
One way to get round the limit of threads stripping is to specify a maximum torque - and that's what we do on bike stuff. Typical engagement calculations don't treat this as a variable. Engagement length is reduced with bolt size, but the number of turns remains roughly constant. Note that the usual calculations for minimum engagement length are based on the screw breaking before the thread strips. That's only one failure mode and in many applications there simply isn't that much material. This table starts at M6 but expects over 12mm into aluminium alloy, 6mm into steel. You'd be lucky to get that much on disc brake mounts. At that point the larger threads should allow more holding force in the same depth, up to a point. The deeper threads of larger sizes mean far more material which will deform less.
Another factor, common on steel into aluminium, is the threads seizing (grease helps avoid this). Then you want a big enough hex socket head to avoid stripping the thread. A big head means a big thread. Bike bolts, especially for accessories, are often button head, which normally uses a smaller hex key than cap head (disc brake attachment bolts are often cap head). on bikes, special button head screws with bigger sockets than you'd expect from other fields are common. It's quite easy to round out the head in a normal M4 button head, far, far harder in a typical M5 on a bike.
Another reason is tolerancing. Going from M5 to M4 wouldn't be too much of an issue, but going to M3 would. Those bottle cage screws don't line up perfectly between bikes; there's a reason cages have slots. You'd need unusual wide heads to give as much freedom. Washers would provide a bit, but not as much and add hassle. This is also very true when parts are being modified in a workshop, where M5-M6 threads are at a sweet spot: easy to cut the thread, not too easy to break the tap, and taps are cheap. This would apply in traditional frame building as well.
Rust makes much o this worse, but it holds for stainless fasteners too.
