That is so so so well said.
I think as soon as you think you are too old for LEGO, then it's time for you to quit.
I am 28 and I love collecting LEGO sets. I may not play with them as a young child would, but I can almost guarantee I appreciate them more.
Anything can be a hobby, if you are afraid of what people think, you'll spend your whole life worrying. That's no fun.
I don't mean for this to be rude or an insult, but that's really sad. You hide the LEGO bag? Something you claim to really enjoy. I think if I was that ashamed of something, I just wouldnt participate in it anymore. I'm an adult and when I get to the closet LEGO store to me, I love the big yellow bag, because I know I get to go home and enjoy some new LEGO, I have to go to the mall of america for my LEGO store needs, so thousands of people see me and my yellow bag, do I care, no. Let them think what they want, it won't hurt me.