The "pure" answer to the question as asked is probably as others have said, climb as much as possible. But perhaps a better answer is to admit that cycling is awesome for aerobic fitness and leg strength but not as great on the upper body.
Obviously, cross-training is an option, but even if you are 100% committed to your bike, you can probably get significant results from two common calisthenics pre and/or post ride: Push-ups and Pull-ups. The key is that you want to develop your chest/shoulders/triceps (pushing) and your back/biceps (pulling), and doing multi-joint exercises is likely more efficient/helpful than isolation exercises. If you can't do pull-ups and don't have access to a lat pulldown machine you can help yourself by using your legs to give yourself a gentle assist. If you have or can scrounge some dumbbells you can do a bit more.
Try this:
- Do a set of 20-25 pushups, then 8-12 pull-ups, either just before or just after you put on your shorts/jersey.
- Do a second set 5 minutes later after you've made up your water bottles, etc., put on your shoes, whatever.
- Pump up your tires and get your bike "ready," then do a third set.
Go for a ride, and try to get in as much climbing as possible. You'll probably be amazed at how quickly your strength will improve just with three sets of good form concentrated upper body calisthenics pre-ride. Again, if you have dumbbells or a weight bench you could do bench presses/military presses, or laterals, and you could do rows to help with the "pull," but the goal isn't really do to a full on weightlifting session, just some quick calisthenics with one effective push and one effective pull exercise and at least two, preferably three sets as you prep for your ride.