My current cassette is 11-36 and I want to change it to 11-32.
Can I stay with my current chain or I need to modify my current chain to fit the new cassette?
As the smallest cog and smallest chainring have not change changed size, the chain length will be no worse than it was previously. With the smaller large cassette cog, you may find you can shorten the chain, giving better small/small tension. The difference (presuming a reasonably sized chain to begin with) of dropping a link will be minor enough I doubt I would bother.
However, as pointed out in comments by @Craggie - when fitting a new cassette, its usually false economy not to fit a new chain. In that case, make the new chain the best size for the new cassette.
You can try it since it will not cost you any time or money to try it other than swapping the cassette and putting the wheel back on.
You might just be able to get away with adjusting the B screw slightly.
Here is a link to sizing a chain.
Usually you wrap the chain around the biggest cog and chainring to size it. Pull it tight and add 2 pairs of inside/outside links (2"). That's supposedly ideal in most cases.
As far as just eyeballing it, if your derailleur/chain doesn't have any tension on it when you're in the highest gear, it's too loose. It's going to bang around, drop, or slip/jump.
I run two different sets of wheels on an old hardtail. I use an 11-28t with 28mm tires on endurance rides and 11-36t with 2.0" tires on mellow muddy doubletrack. I run two different lengths chains but sometimes i get lazy and run the shorter chain with the larger cassette. The chain tension is a little high but doesn't require b screw adjustment. I wouldn't do it the other way around, though. I wouldn't run the longer chain with the smaller cassette since that bike runs 1X narrow-wide chainring.