OK, so I went back in and looked at the little spray head, and there seemed to be a twist knob below it. I turned that, and it came down, and the spray head was removeable. Ta da. A couple months ago I got some really really neat new stuff I saw on TV or someplace that is CLR's new "enhanced bathroom cleaner." Yellow spray bottle. It's good for chrome and everything and does a fantastic job on my kitchen sinks (sorry about the kitchen reference, Tom) and smells good too. I sprayed a tiny puddle of it on the nozzle for a half minute and rinsed off, sprayed and cleaned a second time and rinsed. The chrome trim on the spray nozzle got even shinier. The nozzle does spray significantly better than it did, so I imagine continued treatment will help. I don't need to remove the nozzle, but I was able to clean the washer in it while I had it apart. I imagine there's all sorts of hard water crud in the nozzle, and eventually a new one from Home Depot will be a much better solution.
It's great to know that I can replace this nozzle easily. It does leak a little, but I think I had let that cover cap loosen over time. This is one more thing about my RV that I didn't know and feel a ton better understanding.
Thanks.
--pat