Many saddlebags work with the German KLICKfix system. The KLICKfix system is a series of brackets including one that goes under the seat and another that goes on the seatpost. In my experience the one that goes on the seat rails is more durable and less wobbly. It also seems to be the one preferred by manufacturers of bags.
One interesting option are the bags made by Blackburn. They have a lifetime warranty on them - a quirk of the Blackburn company. Get one and it won't be throwaway.
Here is one they do for the Klickfix bracket, the Zayante large. It has an expanding main compartment, little pocket for valuables, loop for putting the light on, reflective trim and so on:
Here is the KLICKfix 'Contour' model with the other bracket type. This one is supposed to be aero:
I suggest you go to one of your better local bike shops and specifically look for bags with the KLICKfix adaptor, regardless of brand, they should have some. Ordering online can be tricky as you need to see the product first to see if you will be able to get your coat, tools and whatever else in there.
In former times (50 years ago) panniers were the rarity and saddlebags were the norm. I am a proud owner of an extremely retro saddlebag and I use it in preference to panniers.
I believe such bags are bigger than your current requirements, however, I mention that for others finding this question. There are smaller 'real saddlebags' and there are also brackets to go with them if you don't have saddlebag loops.
Another 'out there' solution you might want to look into is a handlebar-bag mounted behind the seat. Klickfix adaptors are available for the handlebar or the seatpost and you can put both brackets on if you want for marginal weight penalty. When you want to do a bit of map reading, eating, using the camera and so on, you can put the bag on the front, when you want lighter steering and better aero then you can put it on the back.
I don't think bike bags are a truly international product area, so the UK tip of 'Altura + Klickfix' probably is not what you would want to look for in the U.S.
In summary, the German 'Klickfix' brackets are brilliant, easy to use, well constructed and used by many different bag manufacturers. The under-seat bracket for 'saddle stuffer' bags is very good, for your requirements I would suggest having a look at what bags you can get that use them.