Is this a trip you're planning now or for the future?
During camping season, I would stay at Croton Point Park and take the train into the city. It's a county park on the Hudson River north of NYC, right on the rail line.
Incidentally, the Statue of Liberty is okay, but Ellis Island is the real attraction.