The Story of the Lazy-Loading Lunchbox

Many years ago when I was a lot smaller and still thought 20 GOTO 10 was pretty cool, my long-suffering mother would pack me off to school every day, with a lovingly-packed rucksack containing my homework, gym kit, packed lunch, pencil case and various other educational essentials. At lunchtime, all the packed-lunch kids would go outside and play at being spies and Thundercats until the sandwich-bell rang, which meant it was time to go inside and eat. We'd all troop into the dinner-hall and get out our plastic MASK and He-Man lunchboxes and unwrap our squashed-looking sandwiches and warm bananas and little plastic Thermos flasks of Ribena. Ham sandwiches, jam sandwiches, or - for special occasions - ham and jam, all the better for spending three hours in a warm airtight plastic box alongside a banana. 

Once in a while, something would liven up the lunch break. I'll never forget what happened when Jeremy's parents went on holiday and left the babysitter in charge. Next day, Jeremy opened his lunchbox and instead of sandwiches, there was a pizza menu, and a £10 note in there. He had to ask for special permission to use the phone in the office... good thing Chris had an older sister who worked at Speedy Pizza otherwise I don't think they'd have believed us. Next day the headmistress shouted at everyone in assembly,  and there were letters sent home to parents. (My dad laughed about it all weekend.)

Then Jim joined our class. I happened to be lunch monitor on his first day, which meant I got to hand out all the lunchboxes when the sandwich bell rang. As I lined them up to hand out, I realized that Jim's was empty.  Light as a feather, no noise when you shook it - clearly there was nothing inside. Poor kid, I thought, forgetting his packed lunch on his very first day! I figured I could give him half of my sandwich, and if Chris had bacon crisps he'd probably give them to Jim 'cos Chris hated bacon crisps... it was OK, I told myself. We'd work something out.

Then Jim sat down with his empty lunchbox... and before we knew it, he was tucking into a full three-course roast lunch, piping hot - with gravy, and a glass of milk, and ice-cream for afters. We sat around with envious eyes, eating our banana-scented ham sandwiches, aware that something funny was clearly going on ("why doesn't it melt?") Next day - same thing. On Fridays, he'd get roast pork and dumplings (his favourite, apparently). We asked Jim one day how he did this, but he didn't know. "My mum always does my lunchbox", he said, between mouthfuls of potatoes and gravy, and so the mystery went on. Finally, just before the end of term, Jim asked if I wanted to go over and play after school one afternoon.  I jumped at the chance - finally, I could ask his mum about those amazing packed lunches.

The long-awaited day came, and after Jim and I were all done playing Thundercats vs. Ghostbusters in the garden, Jim's mum called us in for tea. We sat around their big dining table and we ate raspberry jelly and ice-cream, and eventually I plucked up the courage to ask about the lunches.

"Mrs. Fowler", I said politely (for that was her name), "me'n'the other children were wondering about Jim's packed lunch, please... how come the ice-cream doesn't melt from being next to the roast potatoes all day? An' how does Jim get all that food into such a tiny box?"

"Oh, that's easy", she said. "See, we bought Jim a lazy-loaded lunchbox.  He's a growing lad, and he needs plenty to eat, but I think carrying all that food around all day would give him a bad back. Besides, most days he's off out the door before I've even put the potatoes in the oven - and if he kept turning up late because I didn't have his packed lunch ready for him, I'm sure his teacher would complain. So I get his lunch ready for him and keep it warm in the oven, and then when it's time to eat it, I just pop it into his lunchbox - easy peasy! Now, who'd like more ice-cream?"

I went home even more confused than when I arrived. She must be winding me up, I thought - that's impossible - but Jim's amazing lunches just kept on going. One by one, we all tried to get OUR mothers to do us roast-beef-and-ice-cream packed lunches, but it never worked - from the day Pete's bag dripped gravy and raspberry ripple all over the nature corner, to the day Rob's mum got in trouble for trying to break into the school kitchen, none of us ever got a three-course roast packed lunch except Jim.

Hang on... you're making this up!

OK, ok, busted. It's not a true story. Except the bit about Thundercats. And Chris getting gravy in the nature corner, come to think of it. But there's a couple of details in here that might help explain a few things if you're trying to get your head round OO architecture and enterprise design patterns.

Separation of Concerns - aka "Why is a Eight-Year-Old Ordering Pizza?"

The various parts of an object-oriented application should have their own distinct responsibilities, and should depend on each other for help when they need to do something outside their immediate remit. Web pages shouldn't send e-mail. Database queries shouldn't return HTML. When objects end up doing things they're not supposed to, that's a warning sign - just like an eight-year-old ordering a pizza.

If you're relying on a eight-year-old to arrange their own lunch, you're not separating your concerns properly. To make lunch, we need to go shopping, handle money - and get the Penguin bars from their secret hiding place on top of the fridge. Kids don't know where the food comes from - and they shouldn't have to; if they were in charge of this, it would lead to all sorts of chaos.

Kids have clearly defined responsibilities. They torment the guinea pig, throw up during maths, eat lunch, play Thundercats, have nap time and go home. They shouldn't be shopping, they shouldn't be cooking, and they get really funny looks if they tell the teacher they're just popping out for falafel and a skinny latté. (Incidentally, the the teachers, Mr. Namespace and Mrs. Interface, understand all this, and the reason they don't let kids go out for falafel is that they're enforcing this separation of concerns.)

Aggregates and Repositories - aka "Mum, Did You Pack My Gym Kit?"

That rucksack that my mum would pack for me before sending me off for a day in the big wide world was an aggregate - a top-level container, containing a whole collection of objects that make sense when you put them inside each other, as in the following example:

When I was growing up, kids didn't carry money, and they wouldn't let us out of school to go to the shops. I had no way of getting another pencil or a bag of crisps if I realized I'd forgotten it halfway through the day. I was completely dependent on mum making sure all of these things were in my bag to start with.

This is one of the distinguishing features of the repository, as opposed to just cherry-picking bits of data using ad-hoc queries. When using a repository, you are retrieving deep object graphs. Once I had "retrieved" my rucksack from the repository, I could assume that I also had all the rucksack contents - the lunchbox, the pencil tin, the curiosities and the yo-yo.  (The tantrums I'd throw when I got to school and my yo-yo was missing were really just an early form of NullReferenceException.)

So, assuming our repository pattern is returning nicely-populated object graphs, we now have a whole different set of problems. 

• Memory consumption. I had to carry everything, all day - and there was only so much space in the locker room at school. The day everyone brought their skateboards in, there wasn't enough space for them all.
• Long-running queries.  If I'd lost my gym shoes or we'd forgotten to defrost any bread to make sandwiches, I'd always end up being late for school. Even though I didn't need the gym shoes until after morning break and the bread until lunchtime, I couldn't leave home without them.  Some days it would rain and games would be cancelled and I wouldn't even need the gym shoes in the end - I'd just carry them around all day for no reason.
• Stale objects. By the time I actually got around to eating lunch, it had been knocking around in my bag for a couple of hours and invariably smelled of bananas.

Obviously Jim's lazy-loading lunchbox - and presumably the lazy-loading gym-bag, the lazy-loading pencil-case and the lazy-loading gym kit - could solve all these problems, but first let's look at another approach.

Value Holders - aka "Forget lunch, just give 'em a pizza menu"

The day Jeremy's parents were away and the babysitter sent him to school with a pizza menu - that's what PoEAA calls a Value Holder. That particular lunchbox didn't contain any lunch, but it did contain a very simple method that Jeremy could use to get lunch when he needed it. This works, sure - but asking Jeremy to order pizza and pay for it is violating our separation of concerns. Sometimes, this is OK - but be aware you're doing it. You'd should have a damn good reason for letting a eight-year-old order pizza. I suspect Jeremy's babysitter has no such reason; maybe OO principles aren't required reading at babysitter school any more.

In OO, this will typically show up as a method like Customer.GetOrders() - the customer doesn't actually contain the orders, but it knows how to get them. This obviously requires that GetOrders() knows something about the data store (repository, data access layer, whatever) - so our object is not persistence ignorant.

Virtual Proxy - aka "The Magic Lazy-Loading Lunchbox"

Jim's magic lunchbox is something much smarter, though. Jim's lunchbox is a virtual proxy - a lightweight, portable "pretend lunchbox" that, as long as we don't open it, looks and behaves just like a regular lunchbox. Jim's lunch isn't in there. It's at home, keeping warm (or cold, ), but - and this is the kicker - the virtual proxy holds a reference back to it. Jim can't go home and get his lunch - because lunch-retrieval is not his responsibility - but the magic lunchbox can not only fetch lunches on demand, it can do so completely seamlessly, without Jim being aware that it's even happening. All Jim knows is that every day his mum gives him a rucksack, and when he takes his lunchbox out at lunchtime and opens it, he gets a three-course dinner.

This is much harder to implement seamlessly, but once it's working, it's an absolutely joy to use. Just fire up Customer, get back an apparently fully-populated object, and then do something with Customer.Orders and watch as the database quietly retrieves the exact records you need - no more, no less - and seamlessly weaves the new data into your existing object graph. Almost as cool as watching someone get a three-course roast lunch out of a plastic lunchbox.

Finally... Eager Loading - aka "Colin"

Colin carried a huge sports bag with him, everywhere he ever went, with half-a-dozen paperback books, drinks, snacks, sweets, a Spiderman outfit, poster paints, his father's old harmonica... the kid was like a walking junkyard. He'd carry absolutely everything he could think of with him, all the time, just in case he needed it.

Colin's a great example of what we call eager loading - where you actually load more than you need at initialization time, because (for whatever reason) it's going to be really, really hard to get it later on. If you've ever run Google Reader in offline mode, where it uses the Google Gears engine to get all your blog posts now so you can read them later on the train - that's eager loading. As with all these techniques, it's worth knowing that it exists (so you know when it might help you), and what it's called (so you know what to Google when you need to use it for real).

Further Reading - aka "stop with the metaphors already!"

The principles behind aggregates, domain models and the repository pattern are covered in depth by Eric Evans' work on Domain-Driven Design and there are good explanations of the supporting patterns in both Eric's book and Martin Fowler's Patterns of Enterprise Application Architecture (PoEAA)

The Value Holder and Virtual Proxy patterns are described under "Lazy Load" in PoEAA, and the Castle Project includes a .NET implementation of the virtual proxy pattern called DynamicProxy, which is used by the lazy-loading features provided in several object-relational mappers, including NHibernate and Castle ActiveRecord (which is built on NHibernate)

Many LINQ providers implement IQueryable<T> as a variation on the lazy-loading pattern; take a look at Mike Hadlow's Linq-to-SQL implementation of IRepository<T> to see an example of this in action (and don't miss the controversy over whether IQueryable<T> is violating separation of concerns or not!)

Oh, and the trick is to eat the banana on the way to school, and then at lunchtime you put the bacon crisps inside the ham-and-jam sandwiches. Honest.

For the record, names are fictional, any resemblance to any persons living or dead is purely coincidental; the He-Man lunchbox is from www.retrojunk.com , and tweetfu rocks.