It appears that is some kind of fancy CNC'd aluminum ferrule. If so it might be a little chunkier than a regular one and randomly be able to encounter fit issues. If that is the case, assuming you're using normal spiral housing, all you have to do is get a normal plated brake ferrule like the one you had and you should be fine. If you're using compressionless aka braided brake housing, choose from the other options suitable for compressionless, like the Jagwire nosed ones.
If every ferrule you put in there is going in tight, you'll need to figure out why. Usually it's because of excess paint, or the two sides of the stop got squished together, or the stop got smashed shut at the opening a little. On an aluminum frame one must proceed with caution to find the best way of repairing it. I have very gently used a screwdriver to widen back out smooshed housing stops on aluminum frames, but there is some risk of it breaking completely. I've also used a small file to take away the dented-in part of the opening, which is typically a better way as long as the end result is workable because you really don't want to be bending anything non-replaceable around on aluminum frames.
4mm, 4.5mm, and 5mm refer to the outside diameter of the housing that goes into the ferrule. In contemporary times, all brake housing is 5mm, and shift is usually 4mm. 4.5mm is seen in a few fancy kit type systems, but is pretty uncommon.