You could perhaps consider (if you go for the upgrade instead of the different bike route) to only replace some (the most important) parts with ultegra(or 105 if you want to save some money) and leaving the rest of the drive train parts as is (spec wise) or replacing them with higher spec but lower than ultegra/105.
In my opinion the rear derailleur/shifter are the most important/first parts to upgrade since you shift a lot more with the rear derailleur and adjustment and such needs to be a lot more precise to produce clean shifts than in the front. Shifters will last you a very long time and shifting performance is very much related to proper shifters. Also make sure your shift cables are in good condition to get optimal shifting performance.
Rear derailleur: ultegra
Front derailleur: 105/tiagra
In my opinion this is a good compromise in terms of price/performance. You could change the example by replacing Integra by 105 and 105 by tiagra to get a cheaper but still adequate option. The modern tiagra parts are quite good imo.
Regarding ultegra being 11 speed and 105 being 10 speed: in my experience derailleurs are not speed specific and Shimano derailleurs with different 'rated' speeds can be used interchangably when using the correct shifter (same number of gears) for the casette/freewheel. So you could also consider 105 shifters, ultegra RD and 10 speed chain as an option
In regards to cutting costs: try finding the parts used but new (bought but never used) on eBay or another second hand marketplace. Good deals can occasionally be found :)