There are several variations in packaging of the three main power distribution components: the 120v load center (breaker panel), the 12v fuse panel, and the converter/charger. They can be all in one box, all separate, or paired in various ways.
The converter needs 120v from your RV's supply to operate and it converts that to 12v for charging and general usage in lieu of the battery. Sometimes the converter is put in the same enclosure with the 120v breaker panel, but yours apparently is not that way. When separate from the 120v system, the converter will be wired to one of the breakers in the 120v panel. It may just be plugged into a standard outlet.
Sometimes the 12v power center is put in the box with the converter and the fuses you found in it suggest that is so in your RV. However, there is no reason the 12v output from the converter would run back to a separate 120v breaker panel.
If this did not answer your question, try me again.