They are unstable because of how your weight shifts on the bike when you move to the aerobars. Hybrids are not really designed to have aerobars, the geometry doesn't really lend itself to that.
My suggestion would be to get rid of the aerobars, and get handlebar extensions for the ends of the bars, much like you see on many mountain bikes. This will allow you to switch between different hand positions, while still keeping control and weight distribution where it should be.
Bike geometry- The main difference is in the seat tube and head tube angles. As you go from a hybrid (upright riding posture, slack seat tube angle) to a road bike (slightly more leaning forward, steeper seat tube angle) to a time trial frame (leaning on forearms, even steeper seat tubes), you progressively move the weight on the frame further forward.
There are also differences in how the bottom bracket is placed in relation to the ground/seat, etc. that are designed to produce a more comfortable ride.
As the weight shift forward happens, the handling of the bike is different. TT frames are notoriously poor handlers compared to road frames, and hybrids handle much different than a traditional road frame.
So when you move forward on the hybrid like you would for aerobars, you are moving the weight/center of gravity out of the "sweet spot" for the frame, and handling deteriorates as a result.