You can get quick release pedals (some folding bikes use them) with the advantage that the mount point left behind is too small to effectively pedal on. Pedals are also small and relatively clean if you carry them with you, so this works fairly well.
(MKS Ezy quick release pedal)
Commonly people with concerns like yours will install a quick release seatpost bolt and take the seat with them instead/as well. But seats are large and the seatpost is often greasy or dirty, so I am not a fan of that approach. Re-setting the seat height every time is also annoying.
The other awful thing people often remove for security is their front wheel. It's even bigger and dirtier than the seatpost.
Some not-quite-folding bikes are designed to be taken apart easily. Bike Friday, for example, and anything using S&S couplings. You could conceivably take one of the smaller pieces with you every time.
A more extreme variant is simply to remove parts and leave them off. Many fixie riders claim that people won't steal a fixie because they're hard to ride, for example. Riding a unicycle can be an extension of that idea... people won't steal a "bicycle" that has one wheel, no gears and very few valuable components, and that they can't ride.
There are a few types of bike thieves, and you need to deter them in different ways. The lazy thief who just wants transport can be deterred by almost anything (bike locked with any non-trivial lock); the opportunistic/unorganised thief who will steal anything easy and hope to sell it quickly, for whom you just need a decent lock and use it to attach your bike to something solid; and the professional or organised thief, for whom nothing short of a security guard will do as long as your bike is worth stealing. Both steal-for-sale thieves will also steal components off your bike (as will other cyclists who don't think of themselves as criminals). Normally only the easily removed parts like wheels or pumps, but I've had brakes and handlebars stolen off my bikes in the past.
A general solution is to take everything of value off the bike and replace it with cheap crap. That's a junker bike and they're very popular for this reason.