DEFINITIVE ANSWER:
The Bargeman Supply "+12V" does not normally have sufficient voltage to charge Trailer Batteries. In modern vehicles, including the Tundra, the Alternator contains an electronic controller which adjusts output (Voltage, and therefore Amps) according to the measured "State of Charge" found via the "Sense" wire. For a short time after starting, the voltage WILL be sufficient to charge the batteries - but the "State of Charge" recovers pretty quickly, and the alternator reduces its voltage to a "float" level of 13.5 - 13.8V.
Even if all this voltage could be seen at the Trailer batteries, it's not enough to drive much current - and the batteries will not charge in a reasonable amount of time. But the situation at the Trailer batteries is even worse than that, for two reasons: Voltage Drop in the wiring, and lots of power being consumed by the refrigerator.
Typical RV refrigerators, built using heater-based "absorption" technology, consume 10-15A while running the DC Heater. While the refrigerator tries to draw this power, from both the Trailer batteries and the TV-Bargeman connection, the Voltage at the end of the wiring from the Tundra connection wiring by amounts which Internet "Voltage Drop Calculators" will estimate for you. "Voltage Drop" values of 5% and higher are not uncommon when the Fridge activates the heater. When the original "13.5V" drops below the Voltage available from the Trailer batteries, the Fridge will pull power from the Batteries instead:
They are being discharged, rather than charged.
- - - - -
The solutions to this problem all involve devices which can consume lots of Power from the Bargeman Connector (even at lower voltage) and convert to higher Output Voltage, capable of charging batteries. Ctek makes (one the "D250S"), and Redarc devices from Australia are very popular for solving this problem. The boxes install in the Trailer, not the TV. You could also build DIY, although multiple boxes would be involved: A Voltage Booster (a 12V->24V "boost" converter, with high current capabilities; A "smart" Solar charger, to convert the Voltage back down (so that the Trailer batteries are charged properly, and not overcharged); and a small pair of "12V" batteries wired in series, in the middle, to stabilize the high-frequency "PWM Mode" square wave demand profile which the Solar Controller will present to the upstream device.
Costs for DIY are (roughly) $60-80 for a nice, weatherproof converter; about $30 for the pair of batteries; and $40-60 for "half decent" Solar Controller. Costs for Ctek and Redarc devices are much higher, but they can also support the use of Solar Panels.