Buy a cheap and ugly bike to start with, but one that fits you. Fix up the gears/tyres/brakes to good enough. If it gets stolen you haven't lost much, but hopefully it won't. I'm in the process of doing something similar (including overnight storage outside).
Some suggestions:
- No drop bars and no suspension make it "just a bike" and less valuable (assuming that a thief would try to sell to someone who doesn't know much about bikes).
- Very mismatched tyres - one city bike, one "urban MTB" style - will look more different than they feel.
- Don't fit fancy pedals, especially not clipless - they're a giveaway.
- Don't forget component theft:
- Nutted axles are good.
- Security hardware for quick-release axles or the seat post could help, or could act as a giveaway.
- Normal screws (e.g. for pannier racks) can be replaced with anti-tamper torx.
- Don't be mean with the locks.
- Look around and you'll probably see cheap bikes locked up with decent locks -- a decent lock doesn't mean a decent bike
- Generally this means a D-lock through the frame and round the back wheel (or round both chainstays).
- I'd always have a second independent (cable) lock through the other wheel, to something else solid if at all possible.
- Think of ways to make it hard to ride off on the bike even once it's unlocked. This could be anything from a a rear wheel lock/padlock round the rim to taking the saddle off. Do you know if the thieves in your area ride off or put bikes in vans? Removing the saddle reduces the resale value in the latter case.
You may not get the riding quality you'd ideally like, especially if you like to go fast on the open road but actually this sort of thing can be quite fun to throw around.
You might have the option of going for a folding bike and taking it indoors. But that will almost certainly affect the riding quality even if you spend a lot of money on it.