We don't normally do product recommendations here, but it may be possible to address the issues in a general sense.
The eTrex 30 does not actually appear to be a bike-specific GPS, at least not in the way many cyclists understand the term. In the Garmin line, the entire Edge series are bike-specific GPSes, but their navigation capabilities are usually more limited. The eTrex appears to be a multipurpose GPS. It appears to me that it would usually be handheld, or put in a bracket mount in a vehicle. Garmin's site does say that it can be used in all terrain vehicles and on bicycles. The mount they describe as a bicycle mount is pictured below.
I assume the GPS has rails on its body that slide into the mount, and the mount zip ties to the bicycle. In contrast, the bike-specific GPS units use a twist mount. There's a mount molded into the computer's body. That interfaces with a corresponding puck on a more dedicated computer mount, e.g. a Garmin out-front mount pictured below. I think Garmin sometimes calls this mount a quarter-turn mount.
This style of mount, or the equivalents made by other manufacturers like Lezyne and Wahoo, are secure enough for mountain biking. GPS makers do often supply a stem mount secured with zip ties alone. Without having ever used an eTrex, I would guess that the rail mounting system is less secure than the bike-specific mounting system, assuming the OP used a genuine Garmin mount as pointed out by @ChrisH in comments. Maybe the rails suffice for users on smooth roads but not off-road. I have no personal experience, and these details weren't given in the question.
It is possible that there are more rugged third party mounts for bicycles. This is something the OP would need to investigate, as I don't keep track of this space. It's possible some specific communities may have come up with better mounting solutions, e.g. as discussed on this thread on bikepacking.com, or solutions that involve enhancing the existing mount as discussed in other answers.
Alternatives to the eTrex do exist. The Quad Lock mount system for smartphones, reviewed here by Cyclingtips, is actually quite secure and provides a phone case for major smartphone models. However, using your phone for navigation will drain the battery relatively fast, as the phone has to power the screen plus continuously search for GPS signals in addition to running the usual apps in the background. This may not be a problem on shorter rides in familiar areas. If the OP needs to do a lot of navigation, it is not ideal.
Garmin's more dedicated cycling computers have some options. I realize that OP prefers not to spend more money, but this option is worth mentioning because most bike-specific GPS mounts are pretty secure. The Edge Explore appears to be a touring-oriented GPS with a touchscreen and better navigation functions than, say, the Edge 130 or 530. These are more oriented to leisure or performance cyclists who either are on roads they know and are just tracking rides, or who are following a pre-provided course (e.g. given by a ride organizer, downloaded from a site like Ridewithgps.com). Their on-unit navigation functions are a lot more basic. I think the Edge 830 and 1030 have more advanced on-unit navigation functions as well. I am focusing on navigation capability because I assume the OP is looking for this because they referenced the eTrex in the original question. For alternative brands, I believe Lezyne and Bryton make good value for money GPS units, but I'm uncertain how extensive their navigation capabilities are.
If the OP knows their area well and just wants to track miles and ride leisurely, then actually, putting the eTrex in your back pocket would suffice. For that matter, starting Strava on your phone and putting that away will also suffice; this does raise battery consumption, but not excessively if you aren't continuously referring to the phone on the ride. I remember recording a full century ride on my old iPhone 4 and still having about 20% battery at the finish, although this will obviously depend on any other apps you have running in the background plus your battery health. Riding without any electronic input in this fashion can free you up to concentrate on the joys of riding, although this is obviously a subjective consideration.
A side note on non-original mounts
Each manufactured part has a set of dimensions. For example, the rails on the mount are x mm long, and they are y mm apart, and they are z mm thick. Also, a manufacturing process will produce parts with a certain variance around those dimensions. Sometimes the process is highly variable, and you often get, for example, rails that are too thick or too thin. The result of that could be that the eTrex is too hard to slide on, or that it slides on (and off!) too easily. We usually say the tolerances are poor.
With knock-off mounts, you do run the risk that a) they didn't exactly replicate the average dimensions of the rails and other important bits, and b) that their manufacturing tolerances are poor. I assume this is the core of @ChrisH's objection in comments. I agree with this in general, especially for cheap knock off components.
Of course, we don't know that Garmin's own manufacturing process for the eTrex mounts has that tight tolerances. All else equal, tighter tolerances should be more expensive to manufacture. Garmin may not be assuming that many people cycle with the eTrex. They may simply not have designed the mount to be very secure to begin with.
