Can you take a picnic to a restaurant?

I rarely take the little man clothes shopping. Like most men, he is not keen on retail therapy so it is usually best to go solo but last week I realised I hadn't bought any new underwear since said toddler arrived on the scene which is really rather enbarrassing, so to the shops we went. I presuaded my mum to come along for back up (she didn't need much arm twisting really, what with shopping and her grandson being her two most favourite things) and off we went the three of us on a mission to buy some pants.
After an hour of very satisfying credit card bashing - Marks & Spark's Per Una Perfect Fit Comfort bras and undies may not sound very sexy but turned out to be, like all the best marriages, the perfect combination of sexy and comfyness - we headed to their restuarant for a bowl of soup each and a cuppa.
As grandma queued, I got the little man set up in his high chair and started unwrapping his lunch. As I spread out all his favourites in front of him - cream cheese sandwich, avocado, grapes, smoothie - a large man in an M&S uniform plus apron and cap approached and told me that I couldn't bring food into the restaurant. I was completely stunned. It hadn't even occured to me that toddler food would be a no-no. I have been doing it since he was weaned and never batted an eyelid. I quickly explained the mini picnic in front of me was for my son not me, and pointed to my mum who was busy arranging bowls and pots of tea onto a tray. He wavered for a second, unsure of his ground, and I was just about to tell him I was going to feed my child and he could just try and stop me (the thought of snatching back the half eaten sandwich from his little hands would have been too much to bear) when he relented and moved on.
Now I can completely see his point of view - I had brought a lot of food for the little man and it must have looked at first glance like I might be about to happily start munching away with him, although I think it only takes a bit of common sense to work out what was really going on - and it did make me pose this question: at what age does it become unacceptable to bring food for your child into a restaurant? He is only two and to me that seems fine. By three, perhaps not? They did have kiddies' lunch packs which probably would have been fine for him but it is so much easier to bring along something I know he will tuck into and when I am out and about having a good time, I don't want the hassle of trying to tempt him to eat food he doesn't like or is unsure of.
Perhaps I will just have to keep doing it until I am finally chucked out of the restaurant in disgrace. Until then, happy picnicking.