No, I agree, going to nice places is good too.
However, after going to so many restaurants that charged a considerable amount per person (~$30+) and finding that the food cooked at home was better, I got disillusioned about "classy" restaurants. When you're paying premium for something that is cooked worse than a recipe out of The Joy of Cooking, it's just not worth it.
I guess I'm just low-bred and lacking refined taste.