I currently have a folding bike (think Dahon like) with 20" wheels and a front chainring of 52T and a rear cassette of 14-24T. Although 14 provides me with good speed I'd love to get faster speed for less spinning. I found a 7 speed cassette that is 11t-28t (Shimano HG30-7 which is 11t,13t,15t,18t,21t,24t,28t) and wondering if just switching the cassette would work or if I'd also need to switch out the rear derailleur as well (I think it is a Shimano TZ-21) and potentially also the chain (longer?)?
I can't say for sure, but you have a decent chance of being able to just adjust the body screw on the derailleur to increase its clearance for the larger cassette (see "B-screw Adjustment" on this page).
As you've also noted, the chain will be tighter when in that lower rear gear, which might work but more likely than not you're going to have some difficulty shifting into low rear gear and/or front big chainring without making the chain longer. You can potentially just splice in another link or three (most LBSs will give them out for free, although make sure you know what type of chain you have -- probably standard seven speed) rather than purchasing a whole new chain.