The word is "repacking" where you swap out the rechargable batteries inside a pack. Generally speaking the pack's design does not make this easy, but its not impossible. Its normally done at the end of life for the batteries, so after 500-4000 charge cycles.
Note this is quite different to fitting primary cells (ie disposable batteries) that you can throw away once they are discharged
First budget for a replacement battery, in case you stuff up the original one. Can you afford $40 for the new one?
Then closely examine the pack. There will be a seam somewhere, either at the end or down the side where the original battery was inserted. If you're positive its an 18650 then the hole will need to allow that size through. 18mm in diameter and 65mm long.
Crunch up the seam in a bench vise, they're normally built with an inside and an outside clip. You want to compress the "inside" part of the shell. Then use a wood chisel directly on the seam. A light but sharp tap with a hammer should pop the seam without damaging the old battery.
Figure out how to get the cell out, and eyeball it. Its probably unlabelled.
The battery on the left has protection, the cell on the right probably doesn't. The rim you see is a small circuit board that does the voltage sensing. You need the same kind as you removed. Its possible the voltage protection circuit is built into the plastic housing already, and two of them in series is bad.
The vent holes around the positive end are NOT a sign of electric protection. Instead they're vents for any gas buildup.
Reconnect your new battery to the wires, reinsert in the housing, and test. Then use superglue to close and seal the housing. This will be a one-time repack, it should give you another couple years of usage.