When I worked at a fly fishing store (Ozark Anglers in Little Rock, AR), I always recommended people buy the best ROD they could afford, and yes, there is a difference, particularly in weight if you're going to fish all day. I would take customers to the parking lot, let them cast several different rods, and recommend the one that felt best to them. If their upper limit was $250, would take several rods at or under that price, etc. Sage makes an excellent starter rod in that price range, as does Cortland, Cabela's, and other vendors. If you fish for trout, try a 8'6" to a 9' rod in a 5 weight. One thing to consider is that a good rod, even a starter, will generally have a lifetime warranty, and I've sent back a bunch of rods that were accidentally broken. I personally prefer Winston, and have a dozen or so, but they are pricey. As for reels, very good ones can be purchased for under $100. I'd recommend the Orvis Battenkill with disc drag. I think they are in the $80 to $90 range. Don't skimp on the rod or the fly line!
Interestingly as a golfer, you probably don't use just one club. As a fly fisher, you probably won't use just one rod! At least that's the argument I made to DW!
Good luck, and you've been give a lot of good advice by other forum members.