The most common cause by far is the cable tension needs to be adjusted. The other potential causes and contributing factors are cable/housing friction, housing damage, derailer hanger misalignment, derailer damage, barrel adjuster unscrewed all the way and flopping loose, cable anchored incorrectly, cable donut caught in housing stop, wheel crooked in frame, b-tension screw adjustment, extreme cassette/chain wear. Since for you the problem is happening at the high end of the cassette, the limit screws should also be checked. The chain should be reasonably lubricated as well.
The basic troubleshooting would be make sure the wheel is seated squarely in the frame, check (or really sense) that the cable is moving through the housings with minimal friction and lube or otherwise address it if needed, eyeball that the limit screws are doing their thing by constraining the derailer's range of movement correctly, while doing so also eyeball for a bent hanger or derailer, and then check that when shifted onto the smallest cog position you have cable tension in the ballpark of being just taut with no slack but not much tension beyond that. That sounds like a lot but once you know what you're looking for it's pretty automatic. From there you fine tune the cable tension using the barrel adjusters so that it shifts smoothly in both directions. If that's not possible, the most common cause is hanger misalignment.
The shifter itself is typically only the problem if it's clearly broken, or if it's missing clicks due to lack of internal lubrication, which is correctable. Either of those occurring is fairly common. A Shimano Rapidfire type shifter clicking through fine but shifting poorly due to inadequate precision, acquired or not, is a distant theoretical possibility, but not really something you see.