Silicone isn't impossible to remove and most RV manufacturers use it for some of the external sealant needs, e.g. around window frames and such. Just need to wipe it down with a solvent after peeling the old stuff off. Mineral spirits is usually good enough solvent if the silicone was old and dried out.
As with most things, there are places where silicone is appropriate and others where it is not. It's a really poor choice for roof seams and openings, or for joints that flex a lot, e.g. sidewall seams and such. It works well, though, for sealing around the edges of things applied to vertical surfaces, e.g. window frames, water or electrical ports, that sort of thing. Those already have another sealant behind them, so the edge sealer is an extra preventative and trim seal.
Another point: don't confuse pure (100%) silicone sealants with silicone blends, e.g. silicone-acrylic. Any silicone-based sealant that claims to be paintable will NOT have the oily residue that makes silicone unsuitable for some uses.