I've been there several times but only once for sightseeing. We stayed in New Brunswick and took the train to NYC. The subway got us close to everywhere we wanted to go but we also walked a whole lot. We hit Manhattan, Ellis Island, Carnegie Hall, Central Park, The Stock Exchange, Rockefeller Center, a famous Deli, the 911 memorial and probably a bunch of things I've forgotten. All that in one day!
Other times there I was too busy working. I did manage to go to the Hooters in Manhattan once, I wouldn't recommend it. I took my tech team there because it was a tradition.