I'm getting some chain rub on the hardest gear in the front and 2 hardest gears in the back where the chain hits the front derailer cage outer side. When I spin the pedal it seems like the chain is not moving straight e.g. in certain points of the revolution the chain comes closer to the front derailer cage to the point of barely touching. I have tried adjusting the derailer at the LBS but still have this issue of rubbing the derailer cage. Can I go for a 9 speed chain on my 8 speed Claris since the chain is thinner and reduce this issue?
When I spin the pedal it seems like the chain is not moving straight e.g. in certain points of the revolution the chain comes closer to the front derailer cage to the point of barely touching.
You have to fix that first. The front rings are bent and need to be trued so that they track straight. The lateral motion from a bent ring is more significant than the differences in chain width.
No matter how true you have your front rings, under heavy pedaling, like during hill climbing, there is frame flex, and derailleur rub may show up which is absent under light pedaling. If you lower your expectations to accept this, you will have an easier time.
Make sure you understand the limit screws. Front and rear derailleurs have them. The limit being set too tight on the front derailleur can promote rubbing. To loose a limit can result in the chain coming off. You have to fine tune it.
Check the rotation of the front derailleur around its mounting axis (the seat tube). The cage should be "aimed" toward the center sprockets of the cassette. When the derailleur is rotated around the axis, because it has a curved profile to match the rings, under rotation, its aperture will appear slanted. That creates an offset between how it tracks the smaller and larger rings. Even if it looks well positioned, it's one more parameter at your disposal; it may be worth playing with to optimize it.
The height of the derailleur (up and down position on the seat tube) is another parameter to tweak that can interact with the rubbing issue.
I can personally attest that swapping a 6-7-8 KMC chain for a 8-9-10 speed SRAM on an 8-speed bike stopped front derailleur rub in the extreme positions. (This was in a bike where the rear derailleur is a Shimano Sora, which is rated for 9 speeds, so at least that component specifically "likes" the thinner chain.) The bike's rings were true, and everything was adjusted as well as could possibly be. In the end, the narrower chain made final difference in eliminating the remaining faint rub.
This might work, but it will likely be trading one problem for another.
9-speed chains do shift on 8-speed cassettes, but there tends to be noticeable lag to complete the shift compared to using the correct chain.
In front, the mismatch between chain width and cage inner width can cause issues where, for example, by the time the starting position of the inner cage is close enough to the chain to shift it smoothly from the small to large ring (or middle on a triple), the gap with the outer cage when in the high gear is too large, which can result in derailing or poor shifting in the other direction.
This issue is usually the result of a front derailleur that's either tweaked or needs re-positioning.
This feels like patching the symptom and not the root cause.
Yes you could use 9 speed chain on an 8 speed bike, but shifting will be a little sloppier. You'll be moving from 7.1mm wide chain to 6.7mm wide chain, saving 0.4mm ( 1/64th of an inch or 15 thousandths of an inch) which is not a lot. Personally I wouldn't bother.
Does your front derailleur support Trim? That is more stop positions than your crank's total number of chainrings, and you an access them with smaller/shorter pushes of the shifter.
Another option is to try adusting the "low" limiter on the front derailleur to get a little more clearance.
Check the inside of the front derailleur cage too - it may have some dings and burrs which are artificially narrowing the entry. Use a ruler to check if the cage is bent - it may simply need straightening.
Front derailleurs are fiddly and can still function even with minor damage like a bend.