I think you'd benefit from an assistant here. Get someone technical to look closely around the bike while you lean on the pedals.
Based on "it gets worse speeding downhill" the problem is speed-related - that suggests wheel axles/bearings. Check for horizontal play on each wheel at the rim.
Another possibility is a frame crack which has weakened a specific point, but not enough to fail yet. Wash the bike, then inspect all over for cracks. If you find something questionable, put your hand across the crack and flex the bike as you have done. The sides will move subtly-different and you can often feel that.
Good luck !
Additional: I've recently ridden a folding bike that had a sloppy pin in the main hinge. It rode fine until one brakes hard when the hinge area "bucked upward" like a rearing cat, a particularly horrible sensation that made me walk.
Check your frame pivots for play in all directions and adjust it out where possible. If the pin is worn or the holes are enlarged, you may need replacements or bushings.