I just switched to a road bike from a hybrid. On my first time out commuting into work this morning I arrived in 32 mins as opposed to 37 previously.
Weight is a big issue. Newton's 2nd law F= ma or acceleration = Force / mass, so if your bike is twice as heavy then it will take twice as much effort to get up to a given speed.
Then you have resistance, rolling resistance and wind resistance. If resistance is greater then you will find it harder to accelerate and you will require additional effort to maintain a given speed. Rolling resistance is a function of the area of contact with the road and also the weight. Wind resistance is a function of the speed you are going, the area you are presenting as you travel forward and aero-dynamic nuances that are beyond our worries. You can see it's all worse for a hybrid.
Efficiency plays a factor which is a measure of what proportion of effort you put in goes into forward momentum of the bike. Road bikes are optimised for this with stiff frames and an efficient riding position.
We drop down gears when effort becomes too great and because the effort is greater on a hybrid you will not be able to maintain the same gear as the road guys. When you're in a lower gear you aren't going as fast, unless you compensate with a higher cadence (difficult to maintain if you're already struggling to keep up).
There is good advice on here, but minimising the resistance, optimising your efficiency and improving your strength and fitness will only take you so far.
Hybrids are great, but they aren't road bikes. If speed over tarmac is your main aim then you'll never equal a road bike. That's the conclusion I came to anyway and so far I'm not changing my mind!