TLDR:
It is the aerodynamically most efficient position in a range of available positions derived from a typical road bike geometry, where the main focus is to sustainably ride fast (comfort is considered but never the main priority) - this varies depending on the type of bike but you'll never want to sit as upright as on a cruiser bike. You could change the bar height but that would also impact other positions, so it is a bad compromise, in my opinion. Your normal hoods position would be even more upright and so is the regular drops position. If you change a bike as suggested, you could ride full aero in the drops but everything else gets worse.
The longer version ...
A drop bar bike gives you multiple grip positions, you typically choose it based on the demands of your current riding situation and pick what suits you best. Unless you have a very monotonic riding profile, you'll very likely use various positions during your rides and they have obvious advantages and disadvantages.
You can think of it as one of those triangular diagrams in which you can either get extremes or compromises but never the best of all dimensions.
In terms of (road) cycling with drop bars, these attributes could be:
- Speed/aerodynamics
- Control/handling
- Comfort
Let's think about the main positions and score the three attributes from one 1 to 3 points (worst to best)
Hoods
- Speed/aero - 1 point - simply the most upright position and thus the slowest/least aero one
- Control/handling - 2 points - You have a solid grip and can reach the brakes but the upright position might not be ideal while cornering.
- Comfort - 3 points - You are sitting as upright as you can get on your bike and should be able to ride like this for hours
Aero hoods
- Speed/aero - 3 points - the most aero position on dropbar bikes
- Control/handling - 1 point - You don't have good leverage on the brake levers and no good grip to the bars. Steering in this position might also feel akward
- Comfort - 1 point - It is aero but the hunched-over position has the highest demands on your back and core muscles
Drops
- Speed/aero - 2 points - Aero but allegedly not as aero as the aero hoods
- Control/handling - 3 points - The best grip on the bars and access to the brakes, so you will opt for the drops when descending technical roads. The low position also provides a lower center of gravity and more weight/grip on the front wheel. When sprinting, you are still aero but very unlikely to slip off the bars and crash.
- Comfort - 1 point - Pretty much the same as the aero hoods, hard on the back
(Positions like the 'puppy grip' are not listed because the UCI doesn't approve of...)
A good and comprehensive listing on aerodynamic efficiency of bar positions can be found here: https://silca.cc/blogs/silca/body-position-and-aerodynamics-on-a-bike
So, people will typically ride the aero hoods on non-demanding straight roads because you want to go fast and can compromise on handling abilities. That doesn't mean you can't corner in this position but you can only reach the very top of the brake levers and thus, input force is limited. You may also get knocked off the bars when you hit a pothole, at least more likely than in the other grips.
Changing the bar height as suggested would only compromise your options because you compromise on aspects like lower center of gravity or your general bike fit, at least when you are an performance-orientated cyclist. If you're looking at the height difference between the top of the hoods and your drops, that would easily be 10 cm, I guess many pre-build road bikes don't even have enough steerer tube to accomplish this without extenders, etc... You could do this when your back demands an upright position and only ride in flat terrain but then you're potentially better off with a flat-bar sports/cruiser type of bike, anyway.
I'd personally find it odd for somebody to look for a super-aero drops position (so, clearly aiming at performance) but sitting like a sail in the wind in any other case and compromising on pretty much all other riding positions for it...
P.S.: Perhaps I've overshot a bit with this answer but I wanted to give some context on the pros and cons of various riding positions to outline the "why ride aero hoods")