For you, on your mileage, are definitely a worthy investment. Before explaining why I must correct you on your low to medium usage comment - in the UK the average cyclist manages to average all of 17 miles per week. Maybe you are not part of the high-miler club but you can fairly claim better than average usage when amongst lycra-clad folk and high usage when with civilians.
As for your question, the problem with single-walled rims is what happens to them when they wear out. Rims only wear out on bikes with brakes that work on the rim, so maybe this point might not apply. However, with single-walled rims, when the brakes wear through the rim, the problem is that the rim deforms, to bulge out (due to the pressure of the tyre). This can result in a blowout, although in practice this is a rare occurrence.
Meanwhile, with the double-walled rim, if it wears completely through, the inner wall is still strong enough to hold it together.
There are some single-walled rims with a wear indicator on them. This is a groove that should be visible in the side wall. When this is worn off and no longer visible it is new wheel time.
As for pot-holes, really any rim should be up to the job as it is the tyre and not the rim that takes the impact. If you damage the rims by hitting anything (even a kerb) then that is due to not enough air in the tyres.
Keeping the tyres at the right pressure is also important for puncture resistance and tyre longevity. The best plan is to invest in a track-pump with gauge and to check the tyres fortnightly. You can stretch this out to monthly, however, having tyres inflated at less than the side wall stated value also means you are going slower, not really with a more comfy ride, just slower. Stretch your budget to get a decent track-pump if you have not got one already.
The other top tip for looking after your wheels is to clean the rims. It doesn't matter if you clean nothing else on the bike, but do clean the rims. The build up of dirt and grime works as an abrasive paste that also makes your brake pads wear away quickly. To clean them is only a five-minute job, any damp cloth will do - hold the cloth against the braking surfaces and spin the wheels around. Maybe build this into your fortnightly pressure-check routine.
There are untold treatments for rims (e.g. anodizing), different sections (e.g. aero and box), different ways of holding the spokes (e.g. eyelets), different ways to do the join (e.g. welded or with an extra bit riveted in), sidewall finishings (including the wear indicator) and materials (variants of aluminium or ultra-lame steel). The options are far too many to make a sensible choice, however, if there is one thing you should fork out for, then that is the double-wall for the safety reasons that will only come into play a long way down the line when wear takes it's toll. That is what you are investing in.
Fundamentally the double-walled rim is a different product than the single-wall version, with heat-treatment and other process that make the extra money worth the while. You also can get decent eyelets on them and better joins, i.e. without a rough seam. The extra lightening of the wallet for them may be painful at the time but this will soon pass.
