I'ts hard to guess, since every manufacturer plumbs the water system in their own way and you didn't mention the year/make/model of RV. Generally there is a fill valve, ss Tom mentioned, and it may have failed or accidentally been left in the tank fill position. Make sure the valve itself turns when you turn the handle - it may have stripped.
Most systems have a check valve that keeps water from reverse flowing from the city side to the tank side and that could have failed (stuck open). Sometimes the check valve in the pump (as Jim Godward described) is the one that also prevents back flow from the city pressure side. In others there may be a separate connection and check valve. You will just have to look. If you have a schematic of the water system, you can usually figure out where the interconects are. If not, it's time to dig in behind the panels to see what's where.