Some, but not all, RVs have a valve that allows you to fill the fresh water tank when hooked to city water. Look in your water/services bay to see if you have one and if you inadvertently left it in the "fill tank" position.
If it is not the fill valve, suspect either a check valve in the plumbing between the pump and the water inlet or the pump itself. If it is the pump, take the head apprat and clean it with chlorine solution as you were sanitizing the tank and put it back together. Notice when you take it appart if the head pieces are slippery. If they are that was your problem, a bacteria biofilm.
Do you have a charcoal filter on the inlet to the MH??
I'ts hard to guess, since every manufacturer plumbs the water system in their own way and you didn't mention the year/make/model of RV. Generally there is a fill valve, ss Tom mentioned, and it may have failed or accidentally been left in the tank fill position. Make sure the valve itself turns when you turn the handle - it may have stripped.
Most systems have a check valve that keeps water from reverse flowing from the city side to the tank side and that could have failed (stuck open). Sometimes the check valve in the pump (as Jim Godward described) is the one that also prevents back flow from the city pressure side. In others there may be a separate connection and check valve. You will just have to look. If you have a schematic of the water system, you can usually figure out where the interconects are. If not, it's time to dig in behind the panels to see what's where.
I have a 94 30 SunSport. There is no valve that I can find. I drained off the water & reconnected it. Can't hear any water running in... An RV serviceman told me the checkvalve is probably (as earlier mentioned) in the pump head. I take it out & clean it. Thanks everybody.