Mathias Verraes 2021-09-13T20:45:52+00:00 https://verraes.net/ Mathias Verraes mathias@verraes.net Design and Reality 2021-09-13T00:00:00+00:00 https://verraes.net/2021/09/design-and-reality <blockquote> <p>“The transition to a really deep model is a profound shift in your thinking and demands a major change to the design.” — <a href="https://amzn.to/3ljYdp5">Domain-Driven Design</a>, Eric Evans</p> </blockquote> <p>There is a fallacy about how domain modelling works. The misconception is that we can design software by discovering all the relevant concepts in the domain, turn them into concepts in our design, add some behaviours, and voilà, we’ve solved our problem. It’s a simplistic perception of how design works: a linear path from A to B:</p> <ol> <li>understand the problem,</li> <li>apply design,</li> <li>end up with a solution.</li> </ol> <p>That idea was so central to early Object-Oriented Design, that one of us (Rebecca) thought to refute it in her book:</p> <blockquote> <p>“Early object design books including [my own] Designing Object-Oriented Software [Wirfs-Brock 1990], speak of finding objects by identifying things (noun phrases) written in a design specification. In hindsight, this approach seems naive. Today, we don’t advocate underlining nouns and simplistically modeling things in the real world. It’s much more complicated than that. Finding good objects means identifying abstractions that are part of your application’s domain and its execution machinery. Their correspondence to real-world things may be tenuous, at best. Even when modeling domain concepts, you need to look carefully at how those objects fit into your overall application design.” — <a href="https://www.informit.com/promotions/object-design-142314">Object Design</a>, Rebecca Wirfs-Brock</p> </blockquote> <h2 id="domain-language-vs-ubiquitous-language">Domain language vs Ubiquitous Language</h2> <p>The idea has persisted in many naive interpretations of Domain-Driven Design as well. Domain language and Ubiquitous Language are often conflated. They’re not the same.</p> <p>Domain language is what is used by people working in the domain. It’s a natural language, and therefore messy. It’s organic: concepts are introduced out of necessity, without deliberation, without agreement, without precision. Terminology spreads across the organisation or fades out. Meaning shifts. People adapt old terms into new meanings, or terms acquire multiple, ambiguous meanings. It exists because it works, at least well enough for human-to-human communication. A domain language (like all language) only works in the specific context it evolved in.</p> <p>For us system designers, messy language is not good enough. We need precise language with well understood concepts, and explicit context. This is what a Ubiquitous Language is: a constructed, formalised language, agreed upon by stakeholders and designers, to serve the needs of our design. We need more control over this language than we have over the domain language. The Ubiquitous Language has to be deeply connected to the domain language, or there will be discord. The level of formality and precision in any Ubiquitous Language depends on its environment: a meme sharing app and an oil rig control system have different needs.</p> <h2 id="drilling-mud">Drilling Mud</h2> <p>Talking of oil rigs:</p> <p>Rebecca was invited to consult for a company that makes hardware and software for oil rigs. She was asked to help with object design and modelling, working on redesigning the control system that monitors and manages sensors and equipment on the oil rig. Drilling causes a lot of friction, and “drilling mud” (a proprietary chemical substance) is used as a lubricant. It’s also used as a carrier for the rocks and debris you get from drilling, lifting it all up and out of the hole. Equipment monitors the drilling mud pressure, and by changing the composition of the mud during drilling, you can control that pressure. Too much pressure is a really bad thing.</p> <p>And then an oil rig in the gulf exploded.</p> <p>As the news stories were coming out, the team found out that the rig was using a competitor’s equipment. Whew! The team started speculating about what could have happened, and were thinking about how something like that could happen with their own systems. Was it faulty equipment, sensors, the telemetry, communications between various components, the software?</p> <h2 id="scenarios">Scenarios</h2> <p>When in doubt, look for examples. The team ran through scenarios. What happens when a catastrophic condition occurs? How do people react? When something fails, it’s a noisy environment for the oil rig engineers: sirens blaring, alarms going off, … We discovered that when a problem couldn’t be fixed immediately, the engineers, in order to concentrate, would turn off the alarms after a while. When a failure is easy to fix, the control system logs reflect that the alarm went on and was turned off a few minutes later.</p> <p>But for more consequential failures, even though these problems take much longer to resolve, it still shows up on the logs as being resolved within minutes. Then, when people study the logs, it looks like the failure was resolved quickly. But that’s totally inaccurate. This may look like a software bug, but it’s really a flaw in the model. And we should use it as an opportunity to improve that model.</p> <p>The initial modelling assumption is that alarms are directly connected to the emergency conditions in the world. However, the system’s perception of the world is distorted: when the engineers turn off the alarm, the system believes the emergency is over. But it’s not, turning an alarm off doesn’t change the emergency condition in the world. The alarms are only indirectly connected to the emergency. If it’s indirectly connected, there’s something else in between, that doesn’t exist in our model. The model is an incomplete representation of a fact of the world, and that could be catastrophic.</p> <h2 id="a-breakthrough">A Breakthrough</h2> <p>The team explored scenarios, specifically the weird ones, the awkward edge cases where nobody really knows how the system behaves, or even how it should behave. One such scenario is when two separate sensor measurements raise alarms at the same time. The alarm sounds, an engineer turns it off, but what happens to the second alarm? Should the alarm still sound or not? Should turning off one turn off the other? If it didn’t turn off, would the engineers think the off switch didn’t work and just push it again?</p> <p>By working through these scenarios, the team figured out there was a distinction between the alarm sounding, and the state of alertness. Now, in this new model, when measurements from the sensors exceed certain thresholds or exhibit certain patterns, the system doesn’t sound the alarm directly anymore. Instead, it raises an alert condition, which is also logged. It’s this alert condition that is associated with the actual problem. The new alert concept is now responsible for sounding the alarm (or not). The alarm can still be turned off, but the alert condition remains. Two alert conditions with different causes can coexist without being confused by the single alarm. This model decouples the emergency from the sounding of the alarm.</p> <p>The old model didn’t make that distinction, and therefore it couldn’t handle edge cases very well. When at last the team understood the need for separating alert conditions from the alarms, they couldn’t unsee it. It’s one of those aha-moments that seem obvious in retrospect. Such distinctions are not easily unearthed. It’s what Eric Evans calls a Breakthrough.</p> <h2 id="an-act-of-creation">An Act of Creation</h2> <p>There was a missing concept, and at the first the team didn’t know something was missing. It wasn’t obvious at first, because there wasn’t a name for “alert condition” in the domain language. The oil rig engineers’ job isn’t designing software or creating a precise language, they just want to be able to respond to alarms and fix problems in peace. Alert conditions didn’t turn up in a specification document, or in any communication between the oil rig engineers. The concept was not used implicitly by the engineers or the software; no, the whole concept did not exist.</p> <p>Then where did the concept come from?</p> <p>People in the domain experienced the problem, but without explicit terminology, they couldn’t express the problem to the system designers. So it’s us, the designers, who created it. It’s an act of creative modelling. The concept is invented. In our oil rig monitoring domain, it was a novel way to perceive reality.</p> <p>Of course, in English, alert and alarm exist. They are almost synonymous. But in our Ubiquitous Language, we agreed to make them distinct. We designed our Ubiquitous Language to fit our purpose, and it’s different from the domain language. After we introduced “alert conditions”, the oil rig engineers incorporated it in their language. This change in the domain is driven by the design. This is a break with the linear, unidirectional understanding of moving from problem to solution through design. Instead, through design, we reframed the problem.</p> <h2 id="is-it-a-better-model">Is it a better model?</h2> <p>How do we know that this newly invented model is in fact better (specifically, more fit for purpose)? We find realistic scenarios and test them against the alert condition model, as well as other candidate models. In our case, with the new model, the logs will be more accurate, which was the original problem.</p> <p>But in addition to helping with the original problem, a deeper model often opens new possibilities. This alert conditions model suggests several:</p> <ul> <li>Different measurements can be associated with the same alert.</li> <li>Alert conditions can be qualified.</li> <li>We can define alarm behaviours for simultaneous alert conditions, for example by spacing the alarms, or picking different sound patterns.</li> <li>Critical alerts could block less critical ones from hogging the alarm.</li> <li>Alert conditions can be lowered as the situation improves, without resolving them.</li> <li>…</li> </ul> <p>These new options are relevant, and likely to bring value. Yet another sign we’d hit on a better model is that we had new conversations with the domain experts. A lot of failure scenarios became easier to detect and respond to. We started asking, what other alert conditions could exist? What risks aren’t we mitigating yet? How should we react?</p> <h2 id="design-creates-new-realities">Design Creates New Realities</h2> <p>In a world-centric view of design, only the sensors and the alarms existed in the real world, and the old software model reflected that accurately. Therefore it was an accurate model. The new model that includes alerts isn’t more “accurate” than the old one, it doesn’t come from the real world,it’s not more realistic, and it isn’t more “domain-ish”. But it is more useful. Sensors and alarms are objective, compared to alert conditions. Something is an alert condition because in this environment, we believe it should be an alert condition, and that’s subjective.</p> <p>The model works for the domain and is connected to it, but it is not purely a model <em>of</em> the problem domain. It addresses the problems in the contexts we envision better. The solution clarified the problem. Having only a real world focus for modelling blinds us to better options and innovations.</p> <p>These creative introductions of novel concepts into the model are rarely discussed in literature about modelling. Software design books talk about turning concepts into types and data structures, but what if the concept isn’t there yet? Forming distinctions, not just abstractions, however, can help clarify a model. These distinctions create opportunities.</p> <p>The model must be radically about its utility in solving the problem.</p> <blockquote> <p>“Our measure of success lies in how clearly we invent a software reality that satisfies our application’s requirements—and not in how closely it resembles the real world.” — <a href="https://www.informit.com/promotions/object-design-142314">Object Design</a>, Rebecca Wirfs-Brock</p> </blockquote> <p><em>Written by <a href="https://twitter.com/mathiasverraes">Mathias Verraes</a> and <a href="https://twitter.com/rebeccawb">Rebecca Wirfs-Brock</a>. Special thanks to <a href="https://twitter.com/ericevans0">Eric Evans</a> for the spot on feedback and constructive advice.</em></p> Splitting a Domain Across Multiple Bounded Contexts 2021-06-14T00:00:00+00:00 https://verraes.net/2021/06/split-domain-across-bounded-contexts <p>Imagine a wholesaler of parts for agricultural machines. They’ve built a B2B webshop that resellers and machine servicing companies use to order. In their Ubiquitous Language, an Order represents this automated process. It enables customers to pick products, apply the right discounts, and push it to Shipment.</p> <p>Our wholesaler merges with a competitor: They’re an older player with a solid customer base and a huge catalog. They also have an ordering system, but it’s much more traditional: customers call, and an account manager enters the order, applies an arbitrary discount, and pushes it to Shipment.</p> <p>The merged company still has a single Sales Subdomain, but it now has two Sales Bounded Contexts. They both have concepts like Order and Discount in their models, and these concepts have fundamentally the same meaning. The employees from both wholesalers agree on what an order or a discount is. But they have different processes for using them, they enter different information in the forms, and there are different business rules.</p> <p>In the technical sense that difference is expressed in the object design, the methods, the workflows, the logic for applying discounts, the database representation, and some of the language. It runs deeper though: for a software designer to be productive in either Bounded Context, they’d have to understand the many distinctions between the two models. Bounded Contexts represent Understandability Boundaries.</p> <p>In a perfectly designed system, our ideal Bounded Contexts would usually align nicely with the boundaries of the subdomains. In reality though, Bounded Contexts follow the contours of the evolution of the system. Systems evolve along with the needs and opportunities of organisations. And unfortunately, needs and opportunities don’t often arise in ways that match our design sensibilities. We’re uncomfortable with code that could be made more consistent, if only we had the time to do so. We want to unify concepts and craft clean abstractions, because we think that is what we should do to create a well-designed system. But that might not be the better option.</p> <h2 id="deliberate-design-choices">Deliberate Design Choices</h2> <p>The example above trivially shows that a single subdomain may be represented by multiple Bounded Contexts.</p> <p>Bounded Contexts may or may not align with app or service boundaries. Similarly, they may or may not align with domain boundaries. Domains live in the problem space. They are how an organisation perceives its areas of activity and expertise. Bounded Contexts are part of the solution space; they are deliberate design choices. As a systems designer, you choose these boundaries to manage the understandability of the system, by using different models to solve different aspects of the domain.</p> <p>You might argue that in the wholesaler merger, the designers didn’t have a choice. It’s true that the engineers didn’t choose to merge the companies. And there will always be external triggers that put constraints on our designs. At this point however, the systems designers can make a case for:</p> <ul> <li>merging the two Sales Contexts,</li> <li>migrating one to the other,</li> <li>building a new Sales Context to replace both,</li> <li>postponing this effort,</li> <li>or not doing anything and keeping the existing two Contexts.</li> </ul> <p>These are design choices, even if ultimately the CEO picks from those options (because of the expected ROI for example). Perhaps, after considering the trade-offs, keeping the two Sales Contexts side by side is the best strategic design choice for now, as it allows the merged company to focus on new opportunities. After all, the existing systems do serve their purpose. The takeaway here is that having two Bounded Contexts for a single Subdomain can be a perfectly valid choice.</p> <h2 id="twenty-commodities-traders">Twenty Commodities Traders</h2> <p>When there is no external trigger (as in the wholesaler merger), would you ever choose to split a single domain over multiple Contexts, deliberately?</p> <p>One of us was brought in to consult for a small commodities trader. They were focused on a few specialty commodities, and consisted of 20 traders, some operational support roles, and around 10 developers. The lead engineer gave a tour of the system, which consisted of 20 Bounded Contexts for the single Trading Domain. There was one Context for each trader.</p> <p>This seemed odd, and our instinct was to identify similarities, abstract them, and make them reusable. The developers were doing none of that. At best they were copy-pasting chunks of each other’s code. The lead engineer’s main concern was <em>“Are we doing the optimal design for our situation?”</em> They were worried they were in a big mess.</p> <p>Were they?</p> <p>Every trader had their own representation of a trade. There were even <a href="https://verraes.net/2019/06/emergent-contexts-through-refinement/">multiple representations of money</a> throughout the company. The traders’ algorithms were different, although many were doing somewhat similar things. Each trader had a different dashboard. The developers used the same third party libraries, but when they shared their own code between each other, they made no attempt at unifying it. Instead, they copied code, and modified it over time as they saw fit. A lot of the work involved mathematical algorithms, more than typical business oriented IT.</p> <p>It turned out that every trader had unique needs. They needed to move fast: they experimented with different algorithms, projections, and ways of looking at the market. The developers were serving the traders, and worked in close collaboration with them, constantly turning their ideas into new code. The traders were highly valued, they were the prima donnas in a high stress, highly competitive environment. You couldn’t be a slow coder or a math slacker if you wanted to be part of this. There were no (Jira) tickets, no feature backlogs. It was the ultimate fast feedback loop between domain experts and programmers.</p> <p>Things were changing very rapidly, every day. Finding the right abstractions would have taken a lot of coordination, it would slow development down drastically. An attempt at unifying the code would have destroyed the company.</p> <p>This design was not full of technical debt. It also wasn’t a legacy problem, where the design had accidentally grown this way over the years. The code was working. This lack of unifying abstractions was a deliberate design choice, fit for purpose, even if it seems like a radical choice at first. And all the developers and traders were happy with it.</p> <p>These weren’t merely separate programs with a lot of repeated code either. This was a single domain, split over 20 Bounded Contexts, each with their own domain model, their own Ubiquitous Language, and their own rate of change. Coordinating the language and concepts in the models, would have increased the friction in this high speed environment. By deliberately choosing to design individual Contexts, they eliminated that friction.</p> <h2 id="trade-offs">Trade-offs</h2> <p>There are consequences of this design choice: When a developer wanted help with a problem, they had to bring that other developer up to speed. Each developer, when working in another bounded context, expected that they’d have to make a context switch. After all, their terms and concepts were different from each other, even though they shared similar terminology. Context-switching has a cost, which you’ve probably experienced if you’ve work on different projects throughout a day. But here, because the Contexts were clearly well-bounded, this didn’t cause many problems. And sometimes, by explaining a problem to another developer with a similar background (but a different Bounded Context), solutions became obvious.</p> <h2 id="multiple-bounded-contexts-in-ordinary-it-systems">Multiple Bounded Contexts in Ordinary IT Systems</h2> <p>The trading system is an extreme example, and you won’t come across many environments where a single Subdomain with 20 Bounded Contexts would make sense. But there are many common situations where you should consider splitting a domain. If in your company, the rules about pricing for individual and corporate customers are different, perhaps efforts to unify these rules in a single domain model will cost more than it is worth. Or in a payroll system, where the rules and processes for salaried and hourly employees are different, you might be better off splitting this domain.</p> <h2 id="conclusion">Conclusion</h2> <p>The question is not: Can I unify this? Of course you can. But should you? Perhaps not. The right Context boundaries follow the contours of the business. Different areas change at different times and at different speeds. And over time what appears similar may diverge in surprising and unexpectedly productive ways (if given the opportunity). Squeezing concepts into a single model is constraining. You’re complicating your model by making it serve two distinct business needs, and taking a continued coordination cost. It’s a hidden dependency debt.</p> <p>There’s two heuristics we can derive here:</p> <ol> <li>Bounded Contexts shouldn’t serve the designer’s sensibilities and need for perfection, but enable business opportunities.</li> <li>The Rate of Change Heuristic: Consider organising Bounded Contexts so they manage related concepts that change at the same pace.</li> </ol> <p><em>Written by Mathias Verraes <a href="https://twitter.com/mathiasverraes">@mathiasverraes</a> and Rebecca Wirfs-Brock <a href="https://twitter.com/rebeccawb">@rebeccawb</a>.</em></p> The “It's Just Like...” Heuristic 2021-05-22T00:00:00+00:00 https://verraes.net/2021/05/its-just-like-heuristic <p>There’s a common bias in the tech industry, and I imagine everywhere. When there’s a new idea, and some very smart people will say that the new idea X is really just the old idea Y. The idea could be a model for looking at things, a way of doing things, a technology, a pattern. People will make the case that X is nothing but Y in new packaging.</p> <h2 id="brain-rewards">Brain Rewards</h2> <p>Sometimes it really is just new packaging. But:</p> <p>Your brain rewards you for classifying new information into existing buckets. Looking for similarities costs less energy than understanding new differences. When you say “Oh I get it, it’s just like …”, then it probably is <em>not</em> “just like”. That’s very counterintuitive, because that <strong>“oh I get it” feeling is intuition, not reason</strong>.<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> You feel good about the insight, so it’s very tricky to then go “Oh I get it so I must be wrong”.</p> <p>Proclaiming that the new idea is the same as the old idea, <strong>gives you permission not to investigate it deeply</strong><sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>. You can stick to your existing worldview, in which you are an expert in the old idea. New ideas are merely repackaged old ones, not worthy of our consideration. Everything was already invented, nothing to see here, move along. Balance has been restored.</p> <p>Sometimes this habit is intellectual laziness, sometimes it’s intellectual dishonesty. Sometimes there’s the undertone of superiority (“I’m still the smartest person in the room”). Sometimes it’s an unconscious attempt to preserve the power structures that are threatened by new ideas. In the worst cases, it’s a <em>deliberate</em> attempt.</p> <p>But mostly, there’s such a firehose of information, that staying fully up to date on even a single topic is impossible. We should be forgiven for not knowing and understanding and keeping up with everything, even if our job is knowing and understanding and keeping up with everything.</p> <h2 id="applying-the-its-just-like-heuristic">Applying the “It’s Just Like” Heuristic</h2> <p>Still, there’s a remedy. We can use a low energy heuristic<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>, to catch our brain in the act of using a low energy classification. It works like this:</p> <p>Be aware when you are presented with new information. <strong>When you feel “Oh I get it, it’s just like X”, then ask: “So what makes the new thing different from X?”</strong> You ask it to yourself, as a starting point for deeper analysis, and to keep you intellectually honest. And you ask it to others, because by giving them your frame of reference, they can explain the value of the idea better. And you can ask it when others dismiss an idea too eagerly. This question has been a powerful learning tool for me.</p> <h2 id="the-packaging-heuristic">The Packaging Heuristic</h2> <p>A bonus heuristic: <strong>Sometimes the packaging <em>is</em> the new idea</strong>. New packaging gives old ideas new wings. The packaging in itself can be a valuable contribution. It can re-frame the problem, make the solution more appealing, highlight the benefits better, and get it in front of the right people. By dismissing it as mere packaging, we rob others of discovering it. We overvalue originality, but not everything needs to be original to be valuable. It takes a village to raise an idea.</p> <h2 id="more-heuristics">More Heuristics</h2> <ul> <li><a href="/2021/03/heuristic-legacy-mirror/">The Legacy Mirror Heuristic</a> helps you evaluate new ideas by swapping their chronology.</li> <li>I wrote about the “It’s Just Like…” Heuristic <a href="/2014/06/information-overload/">in 2014</a> as well, but the 2021 one here is much better.</li> </ul> <p><strong>Footnotes</strong></p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:1" role="doc-endnote"> <p>The low energy classification of information and the the higher energy thinking fit in Daniel Kahneman’s two modes of thoughts. He calls them System 1 (fast, instinctive and emotional) and System 2 (slow, deliberative, logical). Coherence bias is at play here, and Reductionism, as well as Substitution bias, and “What You See Is All There Is”. See <a href="https://amzn.to/3vc9An1">“Thinking Fast &amp; Slow” by Kahneman</a> <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:2" role="doc-endnote"> <p>That was the polite version. In my notes, I had a line about “a bunch of old farts sitting around congratulating each other on knowing everything”, which happens a lot on social media. Usually someone brings up an obscure author who was the misunderstood genius that already invented the thing. I wasn’t going to publish this bit but meh, YOLO. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:3" role="doc-endnote"> <p>Fast and Frugal Heuristics limit the information search, and do not involve much computation. See <a href="https://amzn.to/3bINCQE">“Simple Heuristics That Make Us Smart” Gigerenzer, Todd et al.</a> <a href="#fnref:3" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div> Simple Models, Scaffolding, Enabling Constraints 2021-04-23T00:00:00+00:00 https://verraes.net/2021/04/simple-models-scaffolding-enabling-constraints <p>All models are wrong but simple models are more wrong than complex ones. Simple models are more appealing, easier to teach and spread and apply. Because of that, they can bring value faster, and they can cause harm faster. “Everything should be made as simple as possible, but no simpler.” (Einstein) is great advice, but the risk is that we forget the last bit. It’s not a joke, it’s a serious warning against reductionism.</p> <h2 id="spotting-reductionism">Spotting reductionism</h2> <p>Reductionism is looking at a problem space as a simple combination of parts, and ignoring hidden, but essential, complexities (eg in the relations between the parts). You end up with what looks like simple model but is actually a lossy model.</p> <p>Luckily, reductionism is often easy to spot. It starts with “[Complex thing] is just a [simple thing]”. The “is just” gives it away. Reductionist models have “5 easy steps” or “3 secrets” or a convenient mnemonic acronym like E.A.S.Y.</p> <p>Simple models that are not (overly) reductionist can also be spotted: they accept to be changed over time. They accept exceptions as inputs to refine the model. When an exception doesn’t fit, it isn’t discarded but embraced.</p> <h2 id="how-to-use-a-simple-model">How to use a simple model</h2> <p>However (and this is where I’m changing my opinion) <em>even</em> reductionist models have their use<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>, and not just for the teaching and spreading and applying part. There are some important ingredients here, which are not part of the reductionist model itself, but are about how <em>we</em> use the model: scaffolding and enabling constraints.</p> <h2 id="enabling-constraints">Enabling Constraints</h2> <p>Say a simple, reductionist model is expected to be good enough to cover 90% of cases. To evaluate our case, we try to fit it into the model. If it works, great, if not, we are either in a 10% case; or we are in a 90% case but we’re doing something wrong. In the 10% case, we need to put in the work to find better, richer, more suitable models. But because we’re more likely to be in the 90%, first we must look for problems in our information and in how we use the model.</p> <p>This is how it becomes an enabling constraint then: The model offers a constraint on how to look at the problem. That enables you to see if you’re in a standard or exceptional situation, raises red flags if something is missing on your side, and helps you pick the right approach. Reductionist models aren’t usually presented to you that way. They should come with a sticker on the box: “Only applicable in 90% of cases. Use as enabling constraint only.”</p> <h2 id="scaffolding">Scaffolding</h2> <p>Scaffolding is the other ingredient. Scaffolding means that use the model to make progress fast in the beginning, use it as the fire starter for finding your own better models, and then get rid of the simple model.</p> <p>This is basically a superpower. You start by not re-inventing the wheel, but you end up discovering novel solutions that address novel problems that the simple model’s author couldn’t possible foresee. Again, most of the time simple models aren’t sold to us explicitly as scaffolding. Perhaps the author, being a great problem solver, uses the model as scaffolding themselves, but it’s not labeled as such.</p> <p>It is up to us to use them that way. Model authors can help though: they can include ways to get rid of the model as part of the model. “Here’s how you apply it, and here’s how you look for opportunities to evolve it, here’s how you get rid of it.”</p> <h2 id="our-relation-to-simple-models">Our relation to simple models</h2> <p>Ironically, by adding these ingredients to the model itself, the model becomes more complex (that is, it addresses more of the inherent complexity in the problem space). And complex models are, although less wrong, harder to teach, spread, and apply!</p> <p>I used to blame the authors, but (and this is also something where I’m changing my mind) I think all these effects are inevitable. Simple models are more likely to become popular.</p> <p><strong>The work of using simple models as enabling constraints and scaffolding is up to us.</strong></p> <p>Sometimes you’ll want to follow the recipe. A recipe can be a great tool to understand why the soufflé doesn’t rise, and a kickstarter to create your own dishes. And if you follow the recipe, remember to taste the food before you serve it.</p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:1" role="doc-endnote"> <p>This blog post was triggered by a conversation with <a href="https://twitter.com/matthewpskelton">Matthew Skelton</a> last night, and a conversation with <a href="https://twitter.com/conways_law">Dr. Mel Conway</a> years ago that took a bit longer to digest :-) <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div> Tensions when Designing Evolvable Bounded Contexts 2021-04-09T00:00:00+00:00 https://verraes.net/2021/04/tensions-when-designing-evolvable-bounded-contexts <p>(This is a small snapshot from a longer conversation I’m having with <a href="https://twitter.com/rebeccawb">Rebecca Wirfs-Brock</a> about Bounded Contexts. It’s very incomplete but I’m posting it anybody who get some value from it in its present form.)</p> <h3 id="context">Context</h3> <ul> <li>A Bounded Context is an “understandability boundary”, a boundary around a model and its language. You can understand the model and the language in isolation, without having to understand other Bounded Contexts.</li> <li>An Interface is the set of contracts or message types or APIs between Bounded Contexts. They translate from one model and language to another.</li> <li>A Bounded Context or Interface is evolvable if we are able to make frequent, safe, and cheap changes to it.</li> </ul> <h3 id="tensions">Tensions</h3> <p>When designing evolvable Bounded Contexts (BCs), there are fundamental tensions that affect our design.</p> <ol> <li>A BC is evolvable on the inside if it is small and understandable and has a small stable Interface.</li> <li>A system of BCs is evolvable (adding, removing, reconfiguring BCs) if the Interfaces are small and understandable.</li> <li>The tension between 1 and 2 is that choosing many small BCs implies having larger Interfaces, but choosing many small Interfaces implies having larger BCs.</li> <li>The Interfaces themselves are more evolvable if they integrate fewer BCs, and less evolvable if they integrate more BCs.</li> <li>The consequence of 2 and 4 is that a small evolvable Interface makes adding BCs easier, but becomes less evolvable in the process.</li> <li>It’s easier for a BC to offer a single generic shared Interface, than many specialised Interfaces for each consumer. Having many specialised Interfaces is harder to maintain when the BC changes.</li> <li>It’s easier for a BC to consume a specialised Interface, adapted to that BC. Generic interfaces are bloated because they support other consumers, and therefore require more integration work.</li> <li>Trading off 6 and 7 involves finding a small set of semi-specialised Interfaces, that are shared between some consumers with similar needs.</li> <li>Coordination between teams (which is in fact coordination of understanding, aka sharing models and language) is expensive, which is one of the reasons we have BCs to begin with.</li> <li>Achieving 8. requires more coordination between more teams than either a single generic Interface or individual specialised Interface.</li> </ol> <p>There are no solutions or hard rules to make these trade-offs. Instead, you need to use heuristics for making the trade-offs, and heuristics for knowing when to choose different trade-offs than the ones you originally made.</p> The Legacy Mirror Heuristic 2021-03-14T00:00:00+00:00 https://verraes.net/2021/03/heuristic-legacy-mirror <blockquote> <p><em>Decision-making heuristics are patterns and ideas that can help us find another angle, generate new options, evaluate them, or otherwise address the problem. They don’t guarantee a good answer, they only give you some possible progress towards an answers.</em></p> </blockquote> <p>The Legacy Mirror Heuristic<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> helps you to evaluate whether to adopt a new idea, method, or technology. It works by imagining the new idea has actually been the mainstream choice for the past 30 years, and you just learned about the old idea for the first time.</p> <h2 id="example-eventsourcing">Example: EventSourcing</h2> <p>(EventSourcing is a relatively niche way to store and work with data, based on a very old<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup> idea.)</p> <p>A recurring question is whether to adopt EventSourcing in favour of more mainstream data storage. The common hot take is some form of “Use a relational database unless you have a very specific reason for EventSourcing”. The lack of literature, the slow adoption curve, the learning cost, and the availability of tooling, are cited as reasons to avoid it. That specific reason, then, is that the problem involves some temporal aspect.</p> <p>Using the Legacy Mirror, we swap EventSourcing and relational databases in time. It’s speculative of course, but I think we’d be getting hot takes like “Use EventSourcing everywhere, unless you have a very specific use case, involving relations between data, that can only be solved using a relational database. Is the extra complexity that relational brings worth the efforts, such as the need for ORMs with their impedance mismatch and leaky abstractions, the costly migrations, the lack of tooling and literature, and the learning curve?” There would of course be plenty of EventSourcing tooling and literature, students would learn it in school, the <a href="https://verraes.net/2019/05/ddd-msg-arch/">patterns and anti-patterns would be widely documented</a>.</p> <p>I’m not saying the answer is now automatically to choose EventSourcing. Debates on technology choices are often blinded: one side sees the shiny new silver bullet, the other side is so invested in and well adapted to the downsides of the old method that they forgot that it, well, sucks.</p> <h2 id="discussion">Discussion</h2> <p>I first used this to help clients decide on a technical choice. Like many of the heuristics I use, it started organically. Later I saw that it was a reusable pattern, and can be applied to all kinds of decision-making.</p> <p>Don’t use the Legacy Mirror as a rhetoric device though. As a heuristic, a tool to introduce new lines of thought into the debate. But it evens the playing field: both the new and the old idea can be evaluated on merit, instead of purely the benefits and downsides of age.</p> <p>And, often, the fact that something is new, is precisely the reason not to do it: you may not want to be a trailblazer on something that is not core to your business. (That’s another heuristic for you.)</p> <p>As always, identify competing heuristics: “X is not our core business, but doing X will attract engineering talent, which will help us achieve our core business goals” is also valid.</p> <h2 id="example-work-from-home">Example: Work from home</h2> <p>“Should we go back to our offices post-COVID?” What if working from home was the norm, and the hot new trend was this new-fangled idea called “offices”?</p> <p>People would think it’s silly to commute all the time to a different location, when you have a nice office set up at home. They’d cite the waste of time, the environmental impact, the fact that offices eat up real estate, the risks of transmitting diseases. Others would report on the benefits on in-person collaboration, the social contact, buying furniture in bulk, getting your laptop repaired by the IT person on the next floor. The outcome would not be “offices with special accommodation for working from home”, but “work from home with occasional in-person meetings”.</p> <h2 id="example-crypto">Example: Crypto</h2> <ul> <li>Decision: Should I trade fiat currency for cryptocurrencies?</li> <li>Legacy Mirror: What if all money had been crypto for a long time, and fiat money was new on the scene?</li> <li>Scenario: Massive amounts of fossil fuel would be burnt, and causing irreparable damage to the ecosystem and life on earth. (I don’t accept the notion that we would have evolved and massively adopted ecological alternatives, for the simple reason that there is no evidence so far that humanity is capable of massively adopting ecological alternatives for anything. That’s the Historical Perspective Heuristic.) Conclusion: Don’t buy cryptocurrencies.</li> </ul> <h2 id="more-heuristics">More Heuristics</h2> <ul> <li><a href="/2021/05/its-just-like-heuristic/">The “It’s Just Like…” Heuristic</a> for figuring out if you understood a new idea well enough.</li> </ul> <h2 id="footnotes">Footnotes</h2> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:1" role="doc-endnote"> <p>Thanks to <a href="https://twitter.com/CiaranMcNulty/status/1371083860323278849">Ciaran McNulty</a> for naming the heuristic. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:2" role="doc-endnote"> <p>The oldest known recorded ledgers are 7000 years old and were found in Mesopotamia (today’s Iraq). <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div> The Wall of Technical Debt 2020-01-22T00:00:00+00:00 https://verraes.net/2020/01/wall-of-technical-debt <p>“Technical debt” is a metaphor for all software design choices that turn out to be suboptimal, no longer valid, or just plain wrong. These choices incur a cost on future development, and the shortcuts taken today will later slow you down until you “pay back” the debt by fixing the problems. And it’s not only code: Artifacts like architecture, documentation, tests, and domain models can all suffer from technical debt.</p> <p>Technical debt can severely drag down development. And paying back all debt by replacing all software isn’t usually very feasible either. But, like financial debt, technical debt is not always a bad thing. <strong>Debt allows you to invest in something before it makes money.</strong> Taking shortcuts when bringing a product to market allows you to test a business model cheaply, and it makes it easier to throw away the code if it turns out to be a bad idea. And often, the debt exists because the business has grown and evolved, and the design choices that were valid early on no longer are.</p> <p>All this to say that the problem isn’t technical debt; it’s <em>unmanaged</em> technical debt. In any company, the CFO knows exactly how much financial debt there is. There are spreadsheets, quarterly reports, payment plans, and options to refinance or sell debt. But ask your CTO how much technical debt your organization has, and you’ll get an awkward “uh… a lot?” as an answer.</p> <p>So how do you manage debt in a fast-moving agile project?</p> <p>The <strong>Wall of Technical Debt</strong> is a surface in your office where you visualize issues on sticky notes. It’s easy to start and maintain, and yet it has a profound impact on how you choose to add, reduce, pay back, or ignore debt. It’s by no means intended as a complete solution to scale the management of debt, but it works precisely because it requires no buy-in.</p> <h3 id="creating-your-own-wall">Creating Your Own Wall</h3> <p>Creating your own Wall of Technical Debt is simple: Make the debt visible, build a habit, negotiate regularly, and give away control.</p> <p><img style="margin-right: 10px" src="/img/posts/2020-01-22-wall-of-technical-debt/pelshoff2-edit.jpeg" alt="Example of a Wall of Technical Debt, from @Pelshoff" /> <br /><br /> <small>Image by <a href="https://twitter.com/Pelshoff/status/1220291781188866049?s=20">Pim Elshoff</a>. Licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</small></p> <p><br /><br /></p> <h4 id="make-it-visible">Make It Visible</h4> <p>Pick a wall in the space where your team works. Make sure it’s publicly visible. Label it the “Wall of Technical Debt.” Draw a nice dramatic logo. Make sure there are sticky notes and markers around so people can easily add to it.</p> <h4 id="build-a-habit">Build a Habit</h4> <p>Whenever you work on code and encounter technical debt:</p> <ul> <li>Ask yourself: <strong>What made you slow? What made the code difficult to understand? What caused a specific bug to be hard to track down? What should have been better documented or tested?</strong> Write down any technical debt you encountered on a sticky note. Keep it short, but make sure you’ll be able to understand it later.</li> <li>Estimate the opportunity cost, i.e. the time you would have spent on something else if the issue at hand didn’t exist. Write it on the sticky note. Agree on a notation for time, such as tally marks or dots per half day. Being visual is more important than being precise.</li> <li>Estimate how long it would take to fix it, and write that down as well.</li> <li>Optionally, write down how to fix the technical debt. You’ll probably want to do this part in an issue tracker, so if you do, write the issue ID on the sticky note.</li> <li>Put the sticky note on the Wall of Technical Debt.</li> <li>Whenever someone runs into the same issue, add more marks to represent the time they lost.</li> <li>Over time, you’ll want to add some categorisation and structure. Let these emerge, resist organising it prematurely.</li> </ul> <p>Don’t forget to also add sticky notes whenever you introduce new technical debt yourself. And be honest. There’s no shame in this, because <em>you are not your code</em>. Remember technical debt is an investment, not a crime — unless you’re hiding debt, in which case you’re cooking the books!</p> <p>When this becomes a habit, the wall becomes an information radar of the state of your system.</p> <h4 id="negotiate-regularly">Negotiate Regularly</h4> <p>As the Wall of Technical Debt grows, people in your organization might get a little nervous. That’s OK! The debt was always there; you only made it visible. Hopefully some of the habits start changing.</p> <p>Whenever someone gets up to add another dot to an existing sticky note, discuss it with the team. Compare the cost of the measured wasted time to the estimated time to fix the debt. Sure, it might take a day to fix the code, but if you keep losing two hours over it every week, the time spent fixing it will pay for itself soon enough.</p> <p>Or, perhaps fixing it isn’t worth it. That’s no longer a matter of aesthetics or opinion. Rather, it’s a matter of empirically collected metrics. Congratulations, you’re negotiating tradeoffs as a team.</p> <p>Meanwhile, when someone adds a sticky for newly discovered existing debt, consider if it’s something the team could fix right away. Of course, it’s perfectly fine to put it on the wall; that’s what it’s there for. However, when you’re introducing new debt, it’s different. Consider if the team could choose not to take the shortcut and instead fix the code while it’s still cheap. The question shouldn’t be “Do we have time now?” It should be “Is it worth paying interest on this later? How much?” The answer could range anywhere from “It doesn’t matter, just commit it,” to “Lots of things will depend on this, so let’s get it right before moving on.”</p> <p>The point is not to fix all technical debt. <strong>The point is to learn to negotiate when to add it, when to fix it, and when to avoid it.</strong> Before, this happened mostly invisibly: Individual team members added or fixed debt without knowing all the facts. Now, the collective brain power of the team can be applied, using real data.</p> <h4 id="give-away-control">Give Away Control</h4> <p>Now we get to the real magic. Because the Wall of Technical Debt is so visible, managers should start to take notice. (If they still don’t, use a fat red marker to add some € or $ signs to the Wall. Money is the universal language, and <em>lost money</em> stings twice as hard.)</p> <p>Normally, when a team asks for time to spend on technical debt, there’s no direct tangible benefit to the users or the business. A manager can’t decide whether those fixes are of critical importance, an attempt to play with shiny new tech, or a desire to have prettier code. Even for a manager with a deep technical background, if they’re not directly involved with the code, it’s impossible for them to judge the importance of a fix.</p> <p>But with the Wall of Technical Debt, it’s different. Now it’s not about opinions. It’s not about the desire to be a craftsperson who writes elegant code. It’s not about solving challenging puzzles. This time, it’s about facts, numbers, debt and interest payments, and returns on investment. You’re speaking the manager’s language. Good code as an asset, bad code as a liability. They’re trained to look at the numbers and make decisions. They’ve got the methods for it: CBA, OODA, PCDA, SWOT, …</p> <p>So now, give away control. The manager knows more about the company strategy, the budgets, the risks. Let them decide, with your help, what to fix and when to plan it. The numbers enable that.</p> <h3 id="tips-and-traps">Tips and Traps</h3> <ul> <li>Don’t start with a complete audit of all your debt. Sure, you could spend a few days identifying all debt. But then you need to get buy-in, and set up dates, and you’re making it a big thing. “We’ll do it during Summer” or “after the big project is finished” or whatever the local euphemism for “never” is. Instead, a single sticky note on the wall today brings value today. You can always do your big audit later; just don’t make it a bottleneck.</li> <li>Without tally marks or dots that represent real incurred costs, you’re back at opinion, and some of the things that bother you may not actually impact future development. This is why real-time tracking is important. By only adding problems when you run into them, your Wall of Technical Debt will represent actual time wasted and not aesthetic problems with no impact on development. This is important because you want to build a habit of negotiating debt, and not simply make a one-off effort and then go back to the old ways.</li> <li>It works the other way around as well. If you encounter ugly or strange code but you didn’t lose any time over it, don’t mark it. We’re not measuring whether you like it, only whether it costs time.</li> <li>Don’t skip the physical wall. Resist the urge to put everything in a digital tool such as a bug tracker. Logs and metrics only have an impact when people look at them. Digital tools make it easy to hide things, whereas the wall is visible — confrontational even.</li> </ul> <h3 id="conclusion">Conclusion</h3> <p>I started writing and speaking about the Wall of Technical Debt in 2013, after my team and I had developed a first version. One of the first companies that adapted our approach, was a small startup I was consulting for. The little wall space they had was already used by business plans and app mockups. So the team put the sticky notes on the office’s only window. The joke became that whenever the room got too dark, they knew it was time to refactor!</p> <p>More importantly, they had been stuck alternating between perfectionism and hacking things to get to market faster. The Wall of Technical Debt helped them break out of that cycle, and they launched soon after. I’ve since heard stories of teams in all sorts of companies using it successfully. Perhaps you could be next.</p> <h3 id="credits">Credits</h3> <p>I wrote this article for <a href="https://leanpub.com/visualcollaborationtools"><em>Visual Collaboration Tools</em></a>, a book that Kenny Baas-Schwegler and João Rosa are editing. It’s loosely based on <a href="https://verraes.net/2013/07/managed-technical-debt/">“Managed Technical Debt”, Verraes 2013</a>. Thanks to Indu Alagarsamy, Rebecca Wirfs-Brock, and Natalye Childress, for reviewing the drafts.</p> Eventsourcing: State from Events or Events as State? 2019-08-24T00:00:00+00:00 https://verraes.net/2019/08/eventsourcing-state-from-events-vs-events-as-state <p>Trying to get people to agree on a single unambiguous definition of a software concept is usually impossible. In the very least, I can point out that such ambiguity exists when it comes to eventsourcing.</p> <h2 id="state-from-events">State from Events</h2> <p>For a given stream of events, I can process them to derive some state. This is a very common practice, because it makes a lot of sense to do this. You’ll encounter this basic concept referenced with different names, such as stream processing, event processing, real-time analytics, complex event processing, streaming analytics, real-time streaming analytics, projections, all depending on the environment you’re in. That’s normal, communities evolve language to fit their needs, and we can’t expect the DDD crowd (mostly people developing backend for line of business applications) to synchronize language with, say, data scientists.</p> <p>An example:</p> <p>Events:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>// (some fields omitted for clarity) ItemWasAddedToCart {cartid = 1, itemid = 5, timestamp = 2018-01-01} ItemWasAddedToCart {cartid = 2, itemid = 6, timestamp = 2019-01-01} CartWasCheckedOut {cartid = 2, timestamp = 2019-01-02} </code></pre></div></div> <p>Queries<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>IsCartCheckedOut {cartid = 2} HowManyCartsAreAbandoned {} </code></pre></div></div> <p>To answer the queries, two consumers can listen to the same stream of events. The first consumer projects a map of <code class="language-plaintext highlighter-rouge">cartid =&gt; boolean</code> resulting in <code class="language-plaintext highlighter-rouge">[ 1 =&gt; false, 2 =&gt; true ]</code>. The second consumer is a bit more involved. It has a business rule that says “A cart is abandoned if it hasn’t been checked out and the last operations is more than 7 days ago”. The projection is a map of <code class="language-plaintext highlighter-rouge">cartid =&gt; lastModified</code>, and every time a cart is checked out, we remove it from the map. Our stream projects as <code class="language-plaintext highlighter-rouge">[ 1 =&gt; 2018-01-01 ]</code> To answer the query, we count all records with <code class="language-plaintext highlighter-rouge">lastModified &gt; 7 days</code>.</p> <p>Should we call this Eventsourcing? It makes sense, as the <strong>state of the system is sourced from events.</strong></p> <p>Let’s look at the other side.</p> <h2 id="events-as-state">Events as State</h2> <p>The previous example does not make any assumptions about where events come from. In all kinds of analytics and data processing contexts, this is all there is: data or events are provided as is, and the question of where they came from is irrelevant. I’m pointing this out because it explains why, in that context, you would call the previous example eventsourcing.</p> <p>So where do the events come from? The producer could be modelled as a state based system. Every cart is represented by a document (or an equivalent set of records) that has a status, line items, timestamps… Changing the state of such a document is usually governed by constraints, such as <code class="language-plaintext highlighter-rouge">A cart with 0 items cannot be checked out</code> and <code class="language-plaintext highlighter-rouge">Items can not be added by a cart that was checked out</code>. Whenever we update the state, we can emit an event that represents what we did. But it’s important to know that often these events can only be emitted after the constraints are imposed. We might even store the events in a log, but the source of truth is the state in the database.</p> <p>Instead of storing state as documents or records with an optional log, we can get rid of the database and promote the event log to be our single source of truth. I wouldn’t call it a log anymore, but an eventstore. So now, all updates are only expressed as new events, and no longer as state changes. Before storing a new event, all relevant constraints still need to be enforced. To enforce <code class="language-plaintext highlighter-rouge">Items can not be added by a cart that was checked out</code>, we project the state of the cart from the event history.</p> <p>Should we call this Eventsourcing? Now <strong>the event store is the single source of truth for the system</strong>.</p> <h2 id="state-from-events-vs-events-as-state">State from Events vs Events as State</h2> <p>The distinction is important. “State from Events” assumes an existing event stream, regardless of how it was produced, and projects state from it. No new events are added to the stream<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>. “Events as State” is about events as the single source of truth. In other words, new events are added to the stream, but they’re constrained by business rules, and these rules depend on previous events as their input (as opposed to state as the input).</p> <h2 id="defining-eventsourcing">Defining Eventsourcing</h2> <p>Now how should we define Eventsourcing? Both make sense. But there are already many terms for “State from Events”. If we choose to call that Eventsourcing, what should we call the second type?</p> <p>That’s why I use the term <strong>Eventsourcing</strong> only when I specifically talk about a system where state is stored as events and used for decision-making (aka enforcing constraints). We can now try to define Eventsourcing unambiguously<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>:</p> <blockquote> <p><strong>A system is eventsourced when</strong></p> <ul> <li><strong>the single source of truth is a persisted history of the system’s events;</strong></li> <li><strong>and that history is taken into account for enforcing constraints on new events.</strong></li> </ul> </blockquote> <p>I’m pretty sure this definition can improved. In the meantime, that’s the definition I’m using for <a href="https://verraes.net/2019/05/ddd-msg-arch/">my articles on Eventsourcing and Messaging patterns</a>.</p> <p><strong>Footnotes</strong></p> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:1" role="doc-endnote"> <p><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-domain-query">Queries can be modelled as domain messages.</a> <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:2" role="doc-endnote"> <p>New events <em>can</em> be added to the eventstore, such as <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Events</a>, but these are derived. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:3" role="doc-endnote"> <p>Greg Young is often quoted that “Eventsourcing is just a fold over a list of data”, which is correct, but reductionist. (I’m not sure the quote is how he actually phrased it, or what the context was.) In any case, a more accurate phrasing is that “a projection can be implemented as a fold over a stream of events”. <a href="#fnref:3" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div> Patterns Are Not Defined by Their Implementation 2019-07-02T00:00:00+00:00 https://verraes.net/2019/07/patterns-are-not-defined-by-their-implementation <p>I wrote about something that I called the <em><a href="https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads/">Forgettable Payload</a></em> pattern. The abstract reads “Store the sensitive payload of an event in a separate store to control access and removal.” Please read the full pattern first. In short, it’s about being able to remove sensitive information from an eventstore, for example upon receiving a deletion request under the GDPR.</p> <p>Some readers have remarked that <em>Forgettable Payloads</em> is the same the <em>Claim Check</em> pattern. <a href="http://amzn.to/1dFVRMd">Enterprise Integration Patterns</a> describes the problem that <em>Claim Check</em> solves as:</p> <blockquote> <p>“How can we reduce the data volume of a message sent across the system without sacrificing information content?”</p> </blockquote> <p>The solution of both patterns is pretty much identical:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>UserUploadedImage { userId:12, image: "...huge binary blob goes here" } // Refactored using Claim Check pattern: UserUploadedImage { userId:12, imageUrl: "http://...photo12.png" } </code></pre></div></div> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>UserHasRegistered { userId:12, name: "Mathias Verraes", address: "..." } // Refactored using Forgettable Payloads UserHasRegistered { userId:12, payload: "b167a587-1f1f-45bf-bd41-af3e7a1aabd1" } </code></pre></div></div> <p>In both cases, the actual payload of the message is replaced by a reference to a separate store that holds the payload.</p> <p>Does this mean that <em>Claim Check</em> and <em>Forgettable Payloads</em> are the exact same pattern? I’d argue they’re not. A pattern consists of a problem and a reusable solution. If the problem is different, the pattern is a different one.</p> <p><em>Forgettable Payloads</em> is specifically about being able to delete information in an eventsourced system, without touching the eventstore. The same problem can be solved in different ways, for example using <em><a href="https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key/">Crypto-Shredding</a></em>.</p> <p><em>Claim Check</em> is specifically about reducing message sizes (and is not limited to eventsourced systems). We can also reduce messages sizes by splitting them into smaller ones, called <em>Message Sequence</em> in Enterprise Integration Patterns.</p> <h3 id="caveat">Caveat</h3> <p>(<a href="http://amzn.to/1dFVRMd">Enterprise Integration Patterns</a> does mention that <em>Claim Check</em> could be used to hide information. It makes no mention of deleting the information. The book was written many years before GDPR, so the authors can be forgiven for not recognising <em>Forgettable Payloads</em> as a separate pattern. In all fairness, using <em>Claim Check</em> for sensitive information is only mentioned as an option in passing, and does not seem to be part of the actual pattern.)</p> <h3 id="conclusion">Conclusion</h3> <p>We must remember to think of a design pattern as “a problem in a context with a reusable solution”, not merely “a reusable solution”. If two patterns have identical solutions, we should not think of them as the same pattern. A solution that is great for one problem, could be a terrible idea for another problem. As our understanding of the problems evolve, their solutions could diverge in entirely separate directions. The problems themselves could evolve separately as well: the technical restrictions to large message sizes may disappear, while the need for robust privacy solutions may increase.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Temporal Modelling 2019-06-04T00:00:00+00:00 https://verraes.net/2019/06/talk-temporal-modelling <p>The versions of this talk are all part overlapping, part different. You should probably pick one at random to watch, and if you want to find out more, skip around a bit in the others.</p> <ul> <li><a href="https://vimeo.com/339925511">NewCrafts 2019, Paris</a></li> <li><a href="https://skillsmatter.com/skillscasts/13787-keynote-temporal-modelling#video">MuCon 2019, London</a></li> <li><a href="https://gotoams.nl/2019/sessions/817">GOTO 2019, Amsterdam</a></li> <li><a href="https://www.youtube.com/watch?v=hh5lskgATCk&amp;feature=youtu.be">ExploreDDD 2019, Denver</a></li> </ul> Emergent Contexts through Refinement 2019-06-04T00:00:00+00:00 https://verraes.net/2019/06/emergent-contexts-through-refinement <p><em>This article was first published in the book <a href="https://dddeurope.com/15years">Domain-Driven Design: The First 15 Years. Essays from the DDD Community</a>. The e-book is available for free in English and Traditional Chinese.</em></p> <p><a href="https://dddeurope.com/15years"> <img src="https://verraes.net/img/posts/DDD_the_first_15_years_small.png" alt="DDD The First 15 Years" width="200" /> </a></p> <p>Which Bounded Context owns a particular concept? One way to find out is by evolving your model until everything finds a natural place. All models are wrong, especially the early ones. Let’s look at some simple requirements, and explore how we can evolve the model over time. As we learn more about the problem we’re solving, we can bring that clarity into new iterations of the model.</p> <h2 id="the-problem">The problem</h2> <p>Imagine working on a business application, that deals with sales, accounting, reporting, that sort of thing. The existing software has some serious issues. For example, monetary values are represented as scalars. In many places, values are calculated at a high precision, and then rounded down to 2 decimals, and later used again for high precision calculations. These rounding errors are all over the code. It doesn’t make a huge difference on a single amount and a single rounding error, but eventually it could add up, and cost the business millions. The monetary values can represent different currencies, but the financial reporting is always in EUR. It is unclear if the code always correctly converts to EUR when needed, or accidentally adds up amounts from different currencies.</p> <p>To solve these problems, the first thing to do is to have conversations with the domain experts from sales and accounting. As a result, we can come to a common agreement on some requirements.</p> <h2 id="requirements">Requirements</h2> <ol> <li>Business needs to support about 10 currencies, possibly more in the future. When business wants to support a new currency, say Japanese Yen, then it is assumed that the developers will add support in the code. There’s no need for a UI for adding new currencies.</li> <li>All price calculations need to be done with a precision of 8 decimals. This is a business decision.</li> <li>When showing amounts to users, or passing them along over an API, the software will always stick to the currency’s official division. For example, in the case of EUR or USD, that’s 2 decimals. In the case of Bitcoin, a precision of 1 satoshi aka BTC 10^-8^.</li> <li>In some markets, the software will need to do specific compliance reporting.</li> <li>All internal reporting needs to be in EUR, no matter what the original currency was.</li> <li>There will be some legacy and third-party systems, that also publish revenues to the internal reporting tool. Most of them only support the currency’s official division, and can’t deal with higher precision.</li> </ol> <blockquote> <p>Some programming languages don’t deal well with highly precise calculations. There are workarounds which won’t be discussed in this article. Just assume that here “number” means a suitable datatype, such as float or Decimal in C# or BigDecimal in Java.</p> </blockquote> <h2 id="the-first-step">The first step</h2> <p>Are there existing design patterns that can help here? A good pattern to apply when dealing with monetary values is the Value Object pattern from <a href="http://amzn.to/1CdXXP9">Domain-Driven Design</a>. In fact, a couple of years before Eric Evans published his book, Martin Fowler described an implementation for <code class="language-plaintext highlighter-rouge">Money</code> in <a href="http://amzn.to/1TN7Tq4">Patterns of Enterprise Application Architecture</a>. The pattern describes an object consisting of two properties, a number for the amount, and another property for the associated currency (which could be a Value Object as well). The <code class="language-plaintext highlighter-rouge">Money</code> Value Object is immutable, so all operations will return a new instance.</p> <p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/Money.png" alt="Figure 1" /></p> <p>The <code class="language-plaintext highlighter-rouge">Currency</code> type supports the 10 different currencies that the business is interested in. Enums can be used, but a simple assertion will do if the language doesn’t have support for enums. The following constraints can be added:</p> <ul> <li>The type can’t be initialised with a value different from any of the supported 10 currencies.</li> <li>It only uses the 3-letter ISO symbols, anything else is an error. That satisfies requirement 1.</li> </ul> <p><code class="language-plaintext highlighter-rouge">Money</code>’s constructor rounds the numbers to 8 decimals. This gives it enough precision to deal with any currency. Operations like <code class="language-plaintext highlighter-rouge">Money.add(Money other) : Money</code> and <code class="language-plaintext highlighter-rouge">Money.multiply(Number operand) : Money</code> etc can be added to the <code class="language-plaintext highlighter-rouge">Money</code> class. The operations automatically round to 8 decimals. In addition, there’s also a <code class="language-plaintext highlighter-rouge">Money.round() : Money</code> method that returns a new Money object, rounded to 2 decimals.</p> <pre><code class="language-`java">Money { // ... round() : Money { return new Money(round(this.amount, 2), this.currency) } } </code></pre> <p>Now that we have modeled the types based on the requirements, the next step is to refactor all the places in the old code that do things with money to use the new <code class="language-plaintext highlighter-rouge">Money</code> object.</p> <h2 id="composition">Composition</h2> <p>An interesting aspect of Value Objects, is that they are the ultimate composable objects. That helps to make implicit concepts more explicit. For example, we could naively use the money type to represent prices. But what if a price is not so simple? In Europe, prices have a Value Added Tax component. You can now naturally compose a new <code class="language-plaintext highlighter-rouge">Price</code> Value Object from a <code class="language-plaintext highlighter-rouge">Money</code> and a <code class="language-plaintext highlighter-rouge">VAT</code> object, the latter representing a percentage.</p> <p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/Price.png" alt="Figure 2" /></p> <p>A Product could have different prices in different markets, and we can make this concept explicit in a another Value Object.</p> <p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/ProductPrice.png" alt="Figure 3" /></p> <p>… and so on. We can push a lot of logic that we’d traditionally put in services, down into these Value Objects.</p> <p>Value Objects make it easy to build abstractions that can handle lots of complexity, at a low cognitive cost to the developer using it. Finding the abstractions can be hard, but spending the effort here usually impacts code quality and maintainability so much in the long run, that it’s very often worth it.</p> <h2 id="look-for-weaknesses">Look for weaknesses</h2> <p>Even if the current model feels intuitively good, there is always room for improvement. Eric often suggests to look for aspects that are awkward or painful to use. These are smells, pointing to opportunities for further refinement.</p> <p>Looking closely at the model, there are two potential weaknesses:</p> <ol> <li>While the scalars in the code have been replaced by the <code class="language-plaintext highlighter-rouge">Money</code> type, the software is still at risk that money is being rounded and then reused in a higher precision calculation. This problem of rounding errors still has not been fully addressed.</li> <li>The software supports 8 decimal precision only, and the model assumes this is fine. However, when <code class="language-plaintext highlighter-rouge">Money.round()</code> gets invoked, the model doesn’t really deal well with the fact that some currencies don’t have two but three decimals by default (such as Bahraini and Kuwaiti Dinar) or more (like Bitcoin), and some have none (like the Japanese Yen).</li> </ol> <p>This is where we should start feeling an itch to stop and rethink our model.</p> <p>Have a close look at: <code class="language-plaintext highlighter-rouge">Money.round() : Money</code></p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">a</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="o">(</span><span class="mf">1.987654321</span><span class="o">,</span> <span class="n">eur</span><span class="o">)</span> <span class="c1">// The constructor rounds it down to 1.98765432</span> <span class="n">b</span> <span class="o">=</span> <span class="n">a</span><span class="o">.</span><span class="na">round</span><span class="o">()</span> <span class="c1">// round() rounds it up to 1.99 and instantiates a new Money</span> <span class="c1">// Money’s constructor rounds it to 1.99000000</span> </code></pre></div></div> <p>Technically, most programming languages don’t distinguish between 1.99 and 1.99000000, but logically, there is an important nuance here. <code class="language-plaintext highlighter-rouge">b</code> is not just any <code class="language-plaintext highlighter-rouge">Money</code> , <strong>it is a fundamentally different type of money</strong>. The current design doesn’t make that distinction, and just mixes up money, whether it was rounded or not.</p> <h2 id="make-the-implicit-explicit">Make the implicit explicit.</h2> <p>A good heuristic when modelling, is to consider if we can be more explicit in the naming, and tease out subtly different concepts. We can rename <code class="language-plaintext highlighter-rouge">Money</code> to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>, and add a new type called <code class="language-plaintext highlighter-rouge">RoundedMoney</code>. The latter always rounds to whatever the currency’s default division is.</p> <p>The <code class="language-plaintext highlighter-rouge">Money.round()</code> method now becomes very simple:</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">PreciseMoney</span> <span class="o">{</span> <span class="n">round</span><span class="o">()</span> <span class="o">:</span> <span class="nc">RoundedMoney</span> <span class="o">{</span> <span class="k">return</span> <span class="k">new</span> <span class="nf">RoundedMoney</span><span class="o">(</span><span class="k">this</span><span class="o">.</span><span class="na">amount</span><span class="o">,</span> <span class="k">this</span><span class="o">.</span><span class="na">currency</span><span class="o">)</span> <span class="o">}</span> <span class="o">}</span> </code></pre></div></div> <p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/PreciseMoney.png" alt="Figure 4" /></p> <p>The chief benefit is strong guarantees. We can now typehint against <code class="language-plaintext highlighter-rouge">PreciseMoney</code> in most of our domain model, and typehint against <code class="language-plaintext highlighter-rouge">RoundedMoney</code> where we explicitly want or need it.</p> <p>It’s easy to underestimate how valuable this style of granular types can be.</p> <ul> <li><strong>It’s defensive coding, against a whole category of bugs</strong>. Methods and their callers now have an explicit contract, about what kind of money they are talking about. Good design communicates intent.</li> <li><strong>Contracts beat tests</strong>. Obviously correct code doesn’t need tests. If <code class="language-plaintext highlighter-rouge">RoundedMoney</code> is well-tested, and some client code typehints for that type, we don’t need a test to verify that this money is in fact rounded. This is why proponents of strong static type systems like to talk about <em>Type Driven Development</em> as an alternative to <em>Test Driven Development</em>: Declare your intent through types, and have the type checker do the work.</li> <li><strong>It communicates to different people working on the code, that we care specifically about the difference.</strong> A developer who is unfamiliar with this problem space, might look for a <code class="language-plaintext highlighter-rouge">Money</code> type. But the IDE tells them no such type exists, and suggests <code class="language-plaintext highlighter-rouge">RoundedMoney</code> and <code class="language-plaintext highlighter-rouge">PreciseMoney</code>. This forces the developer to consider which type to use, and learn something about how the domain deals with precision.</li> <li><strong>It introduces a concept from the domain into the Ubiquitous Language and the model</strong>. Precision and Rounding was fundamental to the domain but clearly ignored in the original model. Co-evolving the language, the models, and the implementation, is central to Domain-Driven Design. These model refinements can have exponential payoffs in the long run.</li> <li>This design also <strong>helps to apply the Interface Segregation Principle</strong>. Client code will not depend on an large set of APIs, only on the ones relevant to the type.</li> </ul> <h2 id="dealing-with-different-precision">Dealing with different precision</h2> <p>We can add a <code class="language-plaintext highlighter-rouge">round()</code> method to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>, but we wouldn’t add a <code class="language-plaintext highlighter-rouge">toPrecise()</code> method to <code class="language-plaintext highlighter-rouge">RoundedMoney</code>. Just like in physics, we can’t create more precision out of nothing. In other words, you can cast <code class="language-plaintext highlighter-rouge">PreciseMoney</code> to <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, but you can’t cast <code class="language-plaintext highlighter-rouge">RoundedMoney</code> to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>. It’s a one way operation.</p> <p>There’s an elegance to that constraint. Once you round something, the precision is lost forever. The lack of <code class="language-plaintext highlighter-rouge">RoundedMoney.toPrecise()</code> fits the understanding of the domain.</p> <h2 id="common-interfaces">Common Interfaces</h2> <p>You may have noticed that in the current design, there is no <code class="language-plaintext highlighter-rouge">Money</code> interface at the top of the object graph. Aren’t <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code> both a kind of <code class="language-plaintext highlighter-rouge">Money</code> or <code class="language-plaintext highlighter-rouge">AbstractMoney</code>? Don’t they share a lot of methods?</p> <p>If the goal is to try and build a model inspired by the real-world, this would make sense. However, don’t judge your models by how well they fit into hierarchical categorisations. A Domain Model is not an taxonomy — in contrast to the OOP books that teach you that a <code class="language-plaintext highlighter-rouge">Cat extends Animal</code>. Judge your models based on usefulness instead. A top-level <code class="language-plaintext highlighter-rouge">Money</code> interface adds no value at all; in fact it takes away value.</p> <p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/BadMoney.png" alt="Figure 5" /></p> <p>This may be a bit counterintuitive. <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, although somewhat related, are fundamentally different types. The model is designed for clarity, for the guarantee that rounded and precise values are not mixed up. By allowing client code the typehint for the generic <code class="language-plaintext highlighter-rouge">Money</code>, you’ve taken away that clarity. There’s now no way of knowing which <code class="language-plaintext highlighter-rouge">Money</code> you’re getting. All responsibility for passing the correct type is now back in the hands of the caller. The caller could do <code class="language-plaintext highlighter-rouge">money instanceof PreciseMoney</code>, but that’s a serious code smell.</p> <h2 id="dealing-with-conversions">Dealing with Conversions</h2> <p>Converting between different currencies depends on today’s exchange rates. The exchange rates probably come from some third party API or a database. To avoid leaking these technical details into the model, we can have an <code class="language-plaintext highlighter-rouge">CurrencyService</code> interface, with a <code class="language-plaintext highlighter-rouge">convert</code> method. It takes a <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and a target <code class="language-plaintext highlighter-rouge">Currency</code>, and does the conversion.</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">interface</span> <span class="nc">CurrencyService</span> <span class="o">{</span> <span class="n">convert</span><span class="o">(</span><span class="nc">PreciseMoney</span> <span class="n">source</span><span class="o">,</span> <span class="nc">Currency</span> <span class="n">target</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span> <span class="o">}</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">CurrencyService</code> implementations might deal with concerns such as caching today’s rates, to avoid unnecessary traffic on each call.</p> <p>If fetching + caching + converting sounds like a lot of responsibility for one service, that’s because it is. Service classes like are <code class="language-plaintext highlighter-rouge">CurrencyService</code> fundamentally procedural code wrapped in an object. Even though leakage is reduced, thanks to the <code class="language-plaintext highlighter-rouge">CurrencyService</code> interface, the rest of the code still needs to depend on this interface. This makes testing harder, as all those tests will need to mock <code class="language-plaintext highlighter-rouge">CurrencyService</code>. And finally, all implementations of <code class="language-plaintext highlighter-rouge">CurrencyService</code> will need to duplicate the actual conversion, or somehow share that logic. A heuristic that helps, is to figure out if we can somehow isolate the actual domain logic code from the part of the code that is just trying to prepare the data for the domain logic to be applied. For example, differentiate the domain logic (conversion) from the infrastructure code (fetching, caching).</p> <p>There’s a missing concept here. Instead of having the <code class="language-plaintext highlighter-rouge">CurrencyService</code> do the conversion, we can make it return a <code class="language-plaintext highlighter-rouge">ConversionRate</code> instead. This is a Value Object that represents a source <code class="language-plaintext highlighter-rouge">Currency</code>, a target <code class="language-plaintext highlighter-rouge">Currency</code>, and a factor (a float). Value Objects attract behaviour, and in this case, it’s the <code class="language-plaintext highlighter-rouge">ConversionRate</code> object that becomes the natural place for doing the actual calculation.</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">interface</span> <span class="nc">CurrencyService</span> <span class="o">{</span> <span class="n">getRate</span><span class="o">(</span><span class="nc">Currency</span> <span class="n">source</span><span class="o">,</span> <span class="nc">Currency</span> <span class="n">target</span><span class="o">)</span> <span class="o">::</span> <span class="nc">ConversionRate</span> <span class="o">}</span> <span class="nc">ConversionRate</span> <span class="o">{</span> <span class="o">-</span> <span class="n">sourceCurrency</span> <span class="o">:</span> <span class="nc">Currency</span> <span class="o">-</span> <span class="n">targetCurrency</span> <span class="o">:</span> <span class="nc">Currency</span> <span class="o">-</span> <span class="n">factor</span> <span class="o">:</span> <span class="nc">Float</span> <span class="nf">convert</span><span class="o">(</span><span class="nc">PreciseMoney</span> <span class="n">source</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span> <span class="o">}</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">convert</code> method makes sure that we never accidentally convert USD to EUR using the rate that was actually meant to convert GBP to JPY. It can throw an exception if the arguments have the wrong currency.</p> <p>The other cool thing here is that there’s no need to pass around <code class="language-plaintext highlighter-rouge">CurrencyService</code> interface. Instead, we pass around the much smaller, simpler, <code class="language-plaintext highlighter-rouge">ConversionRate</code> objects. They are, once again, more composable. Already the possibilities for reuse become obvious: for example, a transaction log can store a copy of the <code class="language-plaintext highlighter-rouge">ConversionRate</code> instance that was used for a conversion, so we get accountability.</p> <h2 id="simpler-elements">Simpler Elements</h2> <p>An <code class="language-plaintext highlighter-rouge">CurrencyService</code> is now something that represents the collection of <code class="language-plaintext highlighter-rouge">ConversionRate</code> objects, and provides access (and filters) on that collection. Sounds familiar? This is really just the <em>Repository</em> pattern! Repositories are not just for Entities or Aggregates, but for all domain objects, including Value Objects.</p> <p>We can now rename <code class="language-plaintext highlighter-rouge">CurrencyService</code> to <code class="language-plaintext highlighter-rouge">ConversionRateRepository</code>. It has the benefit of being more explicit, and more narrowly defined. Having the pattern name <code class="language-plaintext highlighter-rouge">Repository</code> in the class name is a bit of an annoyance. As Repositories in DDD really represent collections of domain objects, calling it <code class="language-plaintext highlighter-rouge">ConversionRates</code> is a good alternative. But we should take the opportunity to look for terms in the Ubiquitous Language. A place where you can get today’s conversion rates, is a foreign exchange, so we can rename <code class="language-plaintext highlighter-rouge">ConversionRateRepository</code> to <code class="language-plaintext highlighter-rouge">ForeignExchange</code> (or <code class="language-plaintext highlighter-rouge">Forex</code>). Rather than put the word Repository in the classname, we can annotate <code class="language-plaintext highlighter-rouge">ForeignExchange</code> as a @Repository, or have it implement a marker interface called <code class="language-plaintext highlighter-rouge">Repository</code>.</p> <p>The original procedural <code class="language-plaintext highlighter-rouge">CurrencyService</code> is now split into the two simpler patterns, Repository and Value Object. Notice how we have at no point removed any essential complexity, and yet each element, each object, is very simple in its own right. To understand this code, the only pattern knowledge a junior developer would need, is in a few pages of chapters 5 &amp; 6 of <a href="http://amzn.to/1LrjmZF">Domain-Driven Design</a>.</p> <h2 id="currency-type-safety">Currency Type Safety</h2> <p>There are still have some issues left. Remember that some currencies have a division of 1/00, 1/1000, or 1/100000000. <code class="language-plaintext highlighter-rouge">RoundedMoney</code> needs to support this, and in this case, for 10 different currencies. The constructor can start to look somewhat ugly:</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">switch</span><span class="o">(</span><span class="n">currency</span><span class="o">)</span> <span class="k">case</span> <span class="nl">EUR:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span> <span class="k">case</span> <span class="nl">USD:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span> <span class="k">case</span> <span class="nl">BTC:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">8</span><span class="o">)</span> <span class="n">etc</span> </code></pre></div></div> <p>Also, every time business needs to support a new currency, new code needs to get added here, and possibly in other places in the system. While not a serious issue, it’s not exactly ideal either. A switch statement (or a key/value pairs) can be a smell for missing types.</p> <p>One way to achieve this is to make <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code> abstracts or interfaces, and factor the variation out into subtypes for each currency.</p> <p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/FactorOutCurrency.png" alt="Figure 6" /></p> <p>Each of the <code class="language-plaintext highlighter-rouge">PreciseEUR</code>, <code class="language-plaintext highlighter-rouge">PreciseBTC</code>, <code class="language-plaintext highlighter-rouge">RoundedEUR</code>, <code class="language-plaintext highlighter-rouge">RoundedBTC</code> etc classes have local knowledge about how they go about their business, such as the rounding switch example above.</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">RoundedEUR</span> <span class="o">{</span> <span class="nc">RoundedEUR</span> <span class="o">(</span><span class="n">amount</span><span class="o">)</span> <span class="o">{</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span> <span class="o">}</span> <span class="o">}</span> </code></pre></div></div> <p>Again, we can put the type system to work here. Remember the requirement that the reporting needs to be in EUR? We can now typehint for that, making it impossible to pass any other currency into our reporting. Similarly, the different compliance reporting strategies for different markets can each be limited to the currencies they support.</p> <p>So, when should you <em>not</em> want to lift values to types?</p> <p>Let’s say you’re not dealing with 5 or 10 values, but there is an infinite or very large set of potential values. Or when you’re expecting each of these implementations to change often. In this case, having to support 10 currencies is somewhat on the edge. But again, it’s very unlikely that you’re having to add support for all 180 currencies. You’re probably going to only support the 10 or 15 most relevant ones.</p> <h2 id="minimalist-interfaces">Minimalist Interfaces</h2> <p>At this point, we can start seeing how concepts from our requirements, and concepts from our refined Money model, start clustering. We can start breaking up this model, moving and copying parts to different Bounded Contexts. The name Money fooled us: Money means different things to different Contexts.</p> <p>A benefit of having lots of small classes, is that we can get rid of a lot of code. Perhaps the Sales Bounded Context deals with the 10 <code class="language-plaintext highlighter-rouge">PreciseXYZ</code> types, but the Internal Reporting Bounded Context only needs <code class="language-plaintext highlighter-rouge">RoundedEUR</code>. That means there’s no need to support <code class="language-plaintext highlighter-rouge">RoundedUSD</code> etc, as there’s no need for it. This also implies that we don’t need <code class="language-plaintext highlighter-rouge">round()</code> methods on any of the <code class="language-plaintext highlighter-rouge">PreciseXYZ</code> classes, apart from EUR. Less code means less boilerplate, less bugs, less tests, and less maintenance.</p> <p>Not supporting a way back from <code class="language-plaintext highlighter-rouge">RoundedEUR</code> to <code class="language-plaintext highlighter-rouge">PreciseEUR</code> is another example of a minimalist interface. Don’t build behaviours that you don’t need or want to discourage.</p> <h2 id="single-responsibility">Single Responsibility</h2> <p>Another benefit of these small, ultra-single-purpose classes, is that they very rarely need to change, and only when something in the domain changes. Currencies are very stable concepts, and our model mimics this slow rate of change. A good design allows you to easily add or remove elements, or change the composition of the elements, but rarely requires you to actually change existing code. This in turn leads to fewer bugs and less work and more maintainable code.</p> <h2 id="ledger">Ledger</h2> <p>One advantage of expressing our code in terms of rich, well-adapted domain models, is that sometimes this can lead to finding opportunities for other features. It also makes the implementation of those features very easy.</p> <p>In this case, a goal of this modelling exercise was to solve the precision problem. Every time a precise value is rounded, what happens to the fractions of cents that the business gains or loses? Is this important to the domain?</p> <p>If so, we can keep a separate ledger for rounding. Every time a fraction of the value is gained or lost by rounding, we record it in the ledger. When the fractions add up to more than a cent, we can add it to the next payment.</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// rounding now separates the rounded part from the leftover fraction</span> <span class="nc">PreciseMoney</span><span class="o">.</span><span class="na">round</span><span class="o">()</span> <span class="o">:</span> <span class="o">(</span><span class="nc">RoundedMoney</span><span class="o">,</span> <span class="nc">PreciseMoney</span><span class="o">)</span> <span class="nc">Ledger</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="nc">PreciseMoney</span><span class="o">)</span> <span class="o">:</span> <span class="kt">void</span> </code></pre></div></div> <p>You might not have to deal with this in most domains, but when you have high volumes of small transactions, it could make a significant difference. Consider this a pattern for your toolbox.</p> <p>The point here is that our model is easy to extend. In the original code, where rounded and precise amounts were not clear, keeping track of lost fractions of cents would have been a nightmare. Because the change in requirements is reflected in the types, we can rely on the type checker to find all places in the code that need to be adapted.</p> <h2 id="refining-bounded-contexts">Refining Bounded Contexts</h2> <p>Different capabilities of the system have different requirements. This is what Bounded Contexts are good for: they allow us to reason about the system as a number of cooperating models, as opposed to one unified model. Perhaps the Product Catalog needs a really simple model for money, because it doesn’t really do anything other than displaying prices. Sales may need precise price calculations. Our internal Reporting might be fine with reporting with rounded numbers, or even with reporting with no decimals. Company financials are sometimes expressed in thousands or millions. The Compliance Reporting again has different needs.</p> <p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/Contexts.png" alt="Figure 7" /></p> <p>Cleaning up the language:</p> <p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/Contexts2.png" alt="Figure 8" /></p> <p>If the Sales context always deals with high precision, does it then make sense to call the type <code class="language-plaintext highlighter-rouge">PreciseMoney</code>? Why not just <code class="language-plaintext highlighter-rouge">Money</code>? It should then be clearly understood that in the Sales Ubiquitous Language, there is only one <code class="language-plaintext highlighter-rouge">Money</code> and it doesn’t tolerate rounding. In the Reporting Context, money is always rounded and in EUR. Again the type doesn’t have to be <code class="language-plaintext highlighter-rouge">RoundedMoney</code> or <code class="language-plaintext highlighter-rouge">RoundedEUR</code>, it can just be <code class="language-plaintext highlighter-rouge">Money</code>.</p> <p>Every Bounded Context now gets its own domain models for Money. Some are simple, some have more complexity. Some have more features, others have less. We’ve already written all that code, we’re just separating it into the Contexts where we need it. Each Bounded Context has a small, unique model for Money, highly adapted to its specific needs. A new team member working on a Bounded Context can now learn this model quickly, because it’s unburdened by the needs of other Contexts. And we can rest assured that they won’t make rounding errors or swap EUR and USD. Different teams don’t need to coordinate on the features of Money, and don’t need to worry about breaking changes.</p> <p>In some environments, a single Generic Subdomain for money would have been perfectly fine. In others, we want highly specialized models. Determining which is which is your job as a domain modeller.</p> <p><strong>We didn’t do upfront analysis of what goes into which Bounded Context. If we had tried that, early on in the project, we would have naively built a shared Money library, assuming that the concept of Money is universal between all Contexts. Instead, we kept on refining our model based on our needs and new insights, and saw how these explicit concepts naturally found a place. Cultivate a healthy obsession with language, and keep refactoring towards deeper insight, one step at a time. All good design is redesign.</strong></p> Messaging Patterns: Natural Language Message Names 2019-06-02T00:00:00+00:00 https://verraes.net/2019/06/messaging-patterns-natural-language-message-names <h3 id="problem">Problem</h3> <p>Our industry has a tradition of unnatural, compact, lossy naming. This may save on typing, but costs in understandability.</p> <h3 id="solution">Solution</h3> <p>Naming is hard, because we <em>name</em> things. If we understand the domain and have discussed with domain experts, we don’t have to invent names, we can use the names from that domain. And no domain experts says <code class="language-plaintext highlighter-rouge">Payment Event</code> or <code class="language-plaintext highlighter-rouge">Invoice Paid</code> or <code class="language-plaintext highlighter-rouge">OnAfterPayment</code>, they say <code class="language-plaintext highlighter-rouge">The invoice was paid</code>.</p> <p><strong>Use descriptive, natural language names. Do not remove the verbs. Use past tense sentences for events, imperative sentences for commands, questions for queries, negative statements for exceptions.</strong> (In English, I have a slight preference to leave of the article (“the”, “a”), but I might be convinced otherwise.)</p> <h3 id="examples">Examples</h3> <p>Events: <code class="language-plaintext highlighter-rouge">CustomerWasInvoiced</code>, <code class="language-plaintext highlighter-rouge">InvoiceWasPaid</code> Commands: <code class="language-plaintext highlighter-rouge">InvoiceCustomer</code>, <code class="language-plaintext highlighter-rouge">FulfilOrder</code> Queries: <code class="language-plaintext highlighter-rouge">WhichInvoicesAreUnpaid</code>, <code class="language-plaintext highlighter-rouge">HowManyInvoicesAreUnpaid</code> Responses: <code class="language-plaintext highlighter-rouge">UnpaidInvoices</code>, <code class="language-plaintext highlighter-rouge">NumberOfUnpaidInvoices</code> Exceptions: <code class="language-plaintext highlighter-rouge">UnpaidOrdersMayNotBeFulfilled</code></p> <h3 id="discussion">Discussion</h3> <p>Using natural language names helps us to see communication in our systems as mimicking human communication. This is how people tell stories: This has happened, that has happened, I did this, I expected that to happen. Embedding that language in our code and artefacts, is a core idea in Domain-Driven Design, because it enables two-way conversations. We may not expect a domain expert to actually read this code, but we can write it in a way that they would understand. Later, when more work needs to be done on the code, the intent is still very clear, and we can easily go abck and forth between code and domain.</p> <h3 id="test-scenarios">Test scenarios</h3> <p>We can test scenarios by structuring example stories:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scenario .given(new OrderWasPlaced(orderId=5, ...)) .when(new FulfilOrder(orderId=5)) .then(new UnpaidOrdersMayNotBeFulfilled()) // exception scenario .given(new OrderWasPlaced(orderId=5, ...)) .and(new InvoiceWasPaid(invoiceId=6, orderId=5, ...)) .when(new FulfilOrder(orderId=5)) .then(new OrderWasFulfilled(orderId=5)) </code></pre></div></div> <p>This style of writing tests as example stories is common in Behaviour-Driven Development, often using a DSL called Gherkin. But in a messaging architecture, it can be written in plain code as illustrated above.</p> <p>If we see a persona as a history, we can write reusable scenarios:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>angryCustomerJim = scenario .given(new OrderWasPlaced(...)) .given(new InvoiceWasPaid(...)) .given(new ShipmentWasDelayed(...)) .given(new OrderWasPlaced(...)) .given(new InvoiceWasPaid(...)) .given(new ShipmentWasDelayed(...)) </code></pre></div></div> <p>The actual test can now reuse this story:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scenario .involving(angryCustomerJim) .when(new ExpediteShipment(...)) .then(...) </code></pre></div></div> <p>The pattern for scenarios that change the state of the system is “given events, when command, then events”. The pattern for queries is “given events, when query, then response”.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scenario .given(new CustomerWasInvoiced(...)) .given(new CustomerWasInvoiced(...)) .given(new InvoiceWasPaid(...)) .given(new CustomerWasInvoiced(...)) .when(new HowManyInvoicesAreUnpaid()) .then(new NumberOfUnpaidInvoices(5)) </code></pre></div></div> <h3 id="code">Code</h3> <p>The benefits are not limited to tests. By using well-chosen class and method names, intent becomes even more explicit.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class KeepingTrackOfUnpaidInvoices implements Projector when(CustomerWasInvoiced e) unpaidInvoices++ when(InvoiceWasPaid e) unpaidInvoices-- ask(HowManyInvoicesAreUnpaid q) return new NumberOfUnpaidInvoices(unpaidInvoices) </code></pre></div></div> <h3 id="going-further">Going further</h3> <p>In F#, identifiers wrapped in three backticks can contain spaces and other characters. <code class="language-plaintext highlighter-rouge">'''Invoice was paid'''</code> (This blog can’t print backticks, just imagine them.) This makes it even more natural. A problem arises when integrating with other systems, where those characters might not be supported.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Eventsourcing Patterns: Migration Events in a Ghost Context 2019-06-01T00:00:00+00:00 https://verraes.net/2019/06/eventsourcing-patterns-migration-events-ghost-context <p>Explicitly conform to the legacy component’s model in an eventsourcing migration.</p> <h3 id="problem">Problem</h3> <p>We are replacing a Legacy component with a new Eventsourced component. Besides being fully Eventsourced, the new component has a clear domain model with a well-defined <a href="http://amzn.to/1CdXXP9">Ubiquitous Language</a>. The domain events are granular, and named in this language, and track the entire history of the component.</p> <p>The new component needs to retain and use data produced by the old component. The old component stores state, but no history. There is not enough information in that state to deduce all the domain events reproduce the real history. In other words, we know the state but not how we got there. There is no way for new component to have a real complete history from the old component.</p> <h3 id="common-approaches">Common approaches</h3> <p>One solution is to keep the database of the Legacy component, mark it as read-only, and query it from the Eventsourced system. The downside is that the Eventsourced component is now a hybrid, and relies on two entirely different persistence strategies.</p> <p>Another common solution is to make a best effort to force the data from the legacy component, into the domain events of the Eventsourced component. The appeal of this approach is that we have a single, straightforward way of doing things: one canonical model, that fits everything, and therefore easy to work with and reason about. The downside is that the domain events are lying. They didn’t happen in that way, we forced them into the new model. We’re hiding essential complexity, which often causes accidental complexity as a side-effect.</p> <p>That said, both solutions may be perfectly acceptable, especially if the underlying conceptual models for the legacy and new component are very close.</p> <h3 id="migration-events">Migration Events</h3> <blockquote> <p><em>“The old system is now the new problem.”</em></p> <p>— <a href="http://amzn.to/1LMl99l">John Gall; Systemantics</a></p> </blockquote> <p>The mindshift is to see the old system as a domain. The new system is not only solving for the business domain, but for the old system’s domain as well. If the old system is az domain, we can apply domain modelling to it, draw a <em>Bounded Context</em> around it, agree on a <em>Ubiquitous Language</em>, and reason about its relation to other <em>Bounded Contexts</em>.</p> <p><em>Migration Events</em> are now domain events in the new component, but they use the <em>Ubiquitous Language</em> of the old component. They’re not shy about the fact that they are <em>Migration Events</em>, and they don’t try to pretend they happened in the new component. Typical <em>Migration Events</em> names are <code class="language-plaintext highlighter-rouge">LegacyCustomerWasImported</code>, <code class="language-plaintext highlighter-rouge">LegacyOrderWasPlaced</code>. If concepts existed in the Legacy component, that do not exist in the new component, the <em>Migration Events</em> reflect those legacy concepts. No attempt is made to deduce more history than what is available in the Legacy component.</p> <p>This may sound more complex than the alternatives: we have more event types, that we are stuck with forever. All our code, such as projections, will need to deal with both the new event types and with the <em>Migration Events</em>. But by hiding the fact the we did have a Legacy component, we are hiding complexity and end up with a system that is harder to understand for people who where not involved in the old component.</p> <h3 id="ghost-context">Ghost Context</h3> <p>In DDD, we express the domain in the design, it’s just not always obvious what the domain is. <em>Bounded Contexts</em> are essentially about lifting a domain model and its language into a first-class concept in our designs. By treating a model+language as a unit, we can do higher-order reasoning about them: replace models, protect against models, find patterns for relationships between models. In our case, the new Bounded Context is <em><a href="http://amzn.to/1CdXXP9">Conformist</a></em> to the Legacy Bounded Context. The <em>Conformist</em> pattern has an unfortunate negative connotation, but is actually a good strategy when applied judiciously.</p> <p>Even after we delete the Legacy component, the Legacy Bounded Context stays alive. It just isn’t tied to a physical component anymore. The language and model exist, but the implementation has disappeared. But it is still a Bounded Context: there’s language in there, like <code class="language-plaintext highlighter-rouge">LegacyCustomerWasImported</code>, and it is internally consistent to a model, even if most of that model has no implementation anymore. (Note that the <code class="language-plaintext highlighter-rouge">LegacyCustomerWasImported</code> did not exist in the old model, but it represents a <code class="language-plaintext highlighter-rouge">Customer</code> as it was defined in that model.) This approach of seeing Bounded Contexts as first-class concepts, autonomous from implementations, is one of the superpowers of Domain-Driven Design in my opinion, and one of its most unique innovations.</p> <p>I propose <em>Ghost Context</em> as the pattern name for a legacy Bounded Context with no implementation, but that still has traces in other Bounded Contexts.</p> <blockquote> <p><em>“The ghost of the old system continues to haunt the new.”</em></p> <p>— <a href="http://amzn.to/1LMl99l">John Gall; Systemantics</a></p> </blockquote> <ul> <li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li> <li>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</li> </ul> Eventsourcing Patterns: Decision Tracking 2019-05-29T00:00:00+00:00 https://verraes.net/2019/05/eventsourcing-patterns-decision-tracking <h2 id="decision-tracking">Decision Tracking</h2> <p>Store the outcome of a decision to guard against rule changes.</p> <h3 id="problem">Problem</h3> <p>In an eventsourced system, a consumer listens to a stream of events. IT has some business logic that makes decisions, and acts upon those, for example in the form of a side effect or a new event. Because we’ve already stored the input events in the event store, we can recalculate that decision. Should our event store persist an event that represents that decision?</p> <h3 id="solution">Solution</h3> <p>Often, the reasoning is that the event that represents the decision, is redundant. We could store it, but we can recalculate it, so we don’t <em>have to</em> store it.</p> <p>The problem with this line of reasoning is that it doesn’t take into account model changes. The business rule that is applied by the consumer, could change over time, perhaps because of a new requirement, or a bug fix.</p> <p>If we now recalculate the decisions, using the stored events, we get a different outcome than the one that was decided the first time. The old outcome may have been wrong, but it was the one that was calculated at the time, and that caused the side effect to happen. Our present system is therefore lying, as it shows us “facts” about the past that didn’t actually happen. This goes against the idea that an eventstore serves as a historical record of the system.</p> <p>One solution is to track decisions as events, and store these just like other events. We can now inspect the event store, and not only see domain events for the things that happened, but also for the decisions our system has made, whether these are correct or not.</p> <p>This is much more explicit. It can, for example, help us to mitigate the consequences of a bug, because we know exactly what happened. In fact, many problems arise from attempts to make a system look like a bug never happened.</p> <p><em>Decision Tracking</em> isn’t going to be worth the effort in all cases, for example if the we don’t expect anything to be affected by that decision long after it was made.</p> <h3 id="example">Example</h3> <p>A producer emits a stream of <code class="language-plaintext highlighter-rouge">CustomerHasPurchasedTeddyBear</code> events. A business rule states that “At every 1000th purchase, we give away a free teddy bear hat”. A consumer increments a counter for each event. Alice is the 1000th buyer, so the consumer stores <code class="language-plaintext highlighter-rouge">FreeHatWasAwarded{customer=Alice}</code> in the event store.</p> <p>Later, after seeing how successful the campaign was, marketing wants every 750th customer to get a hat now. On top of that, they want to prevent people from getting more than one hat. Because we have a historical record of customers who received the hat, we are certain not to award hats twice.</p> <h3 id="alternative">Alternative</h3> <p>A different approach is <em>Model Change Tracking</em>. Here changes in the model are tracked in the event store, as regular events. When we want to recalculate decisions, we know exactly which version of the model was active at the time. We then need to recalculate using that specific version of the code, so that the outcome is identical with the original outcome (even if that outcome was wrong).</p> <p>These could be generic model change events, such as <code class="language-plaintext highlighter-rouge">ModelWasDeployed{version=7}</code>, or domain-specific such as <code class="language-plaintext highlighter-rouge">NewFreeHatRuleWasDeployed{version=7}</code>, or, if possible, they could be configuration changes such as <code class="language-plaintext highlighter-rouge">FreeHatTresholdWasChanged{treshold=750}</code>.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Messaging Patterns: Throttling 2019-05-23T00:00:00+00:00 https://verraes.net/2019/05/messaging-patterns-throttling <h2 id="throttling">Throttling</h2> <p>Only keep one event per time unit from a high frequency stream.</p> <h3 id="problem">Problem</h3> <p>A stream of <em><a href="https://verraes.net/2019/05/messaging-patterns-ephemeral-events">Ephemeral Events</a></em> is too much to handle for a consumer.</p> <h3 id="solution">Solution</h3> <p>An intermediary consumer is placed in front of the original consumer. It drops all events, except once per time unit, where it passes the event to the original consumer.</p> <h3 id="example">Example</h3> <p>This pattern is very common in user interfaces. An input device, such as a mouse, emits it’s position at 500 Hz. Instead of rendering at the same frequency, the stream is <em>throttled</em> at 5Hz, and therefore the consumer only needs to rerender 5 times per second.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Messaging Patterns: Ephemeral Events 2019-05-23T00:00:00+00:00 https://verraes.net/2019/05/messaging-patterns-ephemeral-events <h2 id="ephemeral-events">Ephemeral Events</h2> <p>An event’s lifetime lasts until the next event.</p> <h3 id="problem">Problem</h3> <p>Some producers, such as sensors, emit realtime or near-realtime measurements, expressed as events. A new measurement makes the previous measurement obsolete. These events can be sent at regular or unpredictable intervals, but they are often high frequency. Consumers aren’t capable of keeping up with all events, or aren’t interested in processing all of them.</p> <h3 id="solution">Solution</h3> <p>Mark the events as being <em>Ephemeral Events</em>. The infrastructure knows that only the latest event matters. When a consumer can’t keep up with the event stream coming in, the infrastructure prefers to drop older events and skip to the most recent one. The consumer can still produce valuable outcomes with a subset of all events.</p> <p>In an eventsourced system, the <em>Ephemeral Events</em> are not persisted. Partly this is because there are too many of them. More importantly, they have no lasting business value, or at least not at their original level of granularity. An <code class="language-plaintext highlighter-rouge">@ephemeral</code> annotation in the code can indicate to the eventsourcing infrastructure that this event must not be persisted.</p> <h3 id="example">Example</h3> <p>A thermometer emits <code class="language-plaintext highlighter-rouge">TemperatureWasMeasured { temperature, timestamp }</code> events multiple times per second. A consumer updates a graph on a TV monitor once a minute, and ignores the other events. The consumer is turned off during the night. In the morning, it doesn’t catch up with the nighttime events, but picks up from the latest one.</p> <h3 id="patterns">Patterns</h3> <p><em>Ephemeral Events</em> are often combined with other patterns:</p> <ol> <li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Event</a>. For example, consumer receives a continuous stream of stock prices. At <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-passage-of-time-event/">the close of trading</a>, the consumer emits an event containing statistics such as min/max/avg.</li> <li><a href="https://verraes.net/2019/05/messaging-patterns-throttling/">Throttling</a></li> <li><a href="https://verraes.net/2019/05/messaging-patterns-change-detection-events">Change Detection Events</a></li> </ol> <p><em>Ephemeral Events</em> should not be confused with <em><a href="http://amzn.to/1dFVRMd">Message Expiration</a></em>, as described in <a href="http://amzn.to/1dFVRMd">Enterprise Integration Patterns</a>, where the message itself contains an expiration date by which it is no longer relevant.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Messaging Patterns: Change Detection Events 2019-05-23T00:00:00+00:00 https://verraes.net/2019/05/messaging-patterns-change-detection-events <h2 id="change-detection-events">Change Detection Events</h2> <p>Listen to a stream of events and produce a new event when a value changes.</p> <h3 id="problem">Problem</h3> <p>A consumer is interested in an event stream, but the consumer only wants to know when a particular value changes, and doesn’t care about the events where that value didn’t change. Optionally, the consumer only cares when the value changes more than a given interval.</p> <h3 id="solution">Solution</h3> <p>A second intermediary consumer is placed in front of the original consumer. It acts as a filter, emitting <em>Change Detection Events</em> when the observed value is different from the previous event. It may also ignore changes that are too small for the original consumer.</p> <p><em>Change Detection Events</em> are often used in combination with <em><a href="https://verraes.net/2019/05/messaging-patterns-ephemeral-events">Ephemeral Events</a></em>.</p> <h3 id="example">Example</h3> <p>The producer emits <code class="language-plaintext highlighter-rouge">TemperatureWasMeasured</code> multiple times per second. A consumer is only interested in changes of 1°C or more. An intermediate consumer listens to the <code class="language-plaintext highlighter-rouge">TemperatureWasMeasured</code> events. When an event arrives, it compares the temperature with the temperature from the previous event. If it differs by more than 1°C, it emits a <code class="language-plaintext highlighter-rouge">TemperatureHasChanged</code> event.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> DDD and Messaging Architectures 2019-05-21T00:00:00+00:00 https://verraes.net/2019/05/ddd-msg-arch <p>My aim is to identify, name, and document patterns, that I’ve used or encountered in my consulting practice. All patterns are anti-patterns when used in the wrong context, so I try to add heuristics, trade-offs, and caveats.</p> <p>You can help: tell me your experiences, point me to other people’s writings on patterns, distill your own heuristics, …</p> <h3 id="meta">Meta</h3> <ol> <li><a href="https://verraes.net/2019/08/eventsourcing-state-from-events-vs-events-as-state">Eventsourcing: State from Events or Events as State?</a></li> <li><a href="https://verraes.net/2019/07/patterns-are-not-defined-by-their-implementation/">Patterns Are Not Defined by Their Implementation</a></li> </ol> <h3 id="patterns-for-decoupling-in-distributed-systems">Patterns for Decoupling in Distributed Systems</h3> <ol> <li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-domain-query">Domain Queries</a></li> <li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Event</a></li> <li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee/">Completeness Guarantee</a></li> <li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-passage-of-time-event/">Passage of Time Event</a></li> <li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event/">Fat Event</a></li> <li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-explicit-public-events/">Explicit Public Events</a></li> <li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-segregated-event-layers/">Segregated Event Layers</a></li> </ol> <h3 id="eventsourcing-patterns">Eventsourcing Patterns</h3> <ol> <li><a href="https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads">Forgettable Payloads</a></li> <li><a href="https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key">Crypto-Shredding</a></li> <li><a href="https://verraes.net/2019/05/eventsourcing-patterns-decision-tracking/">Decision Tracking</a></li> <li><a href="https://verraes.net/2019/06/eventsourcing-patterns-migration-events-ghost-context/">Migration Events in a Ghost Context</a></li> </ol> <h3 id="general-messaging-patterns">General Messaging Patterns</h3> <ol> <li><a href="https://verraes.net/2019/05/messaging-patterns-ephemeral-events/">Ephemeral Events</a></li> <li><a href="https://verraes.net/2019/05/messaging-patterns-throttling/">Throttling</a></li> <li><a href="https://verraes.net/2019/05/messaging-patterns-change-detection-events">Change Detection Events</a></li> <li><a href="https://verraes.net/2019/06/messaging-patterns-natural-language-message-names/">Natural Language Message Names</a></li> </ol> <p><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></p> Eventsourcing Patterns: Crypto-Shredding 2019-05-13T00:00:00+00:00 https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key <h2 id="crypto-shredding">Crypto-Shredding</h2> <p>Encrypt sensitive information in an event and delete the key.</p> <h3 id="problem">Problem</h3> <p>The problem is the same as for <a href="https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads">Forgettable Payloads</a>: some attributes of an event should not be read by all consumers, and we should be able to delete them, without touching the event store.</p> <h3 id="solution">Solution</h3> <p>Encrypt the sensitive attributes, with a different encryption key for each resource (such as a customer). Only give the key to consumers that require it. When the sensitive information needs to be erased, delete the encryption key instead, to ensure the information can never be accessed again. This effectively makes all copies and backups of the sensitive data unusable.</p> <h3 id="discussion">Discussion</h3> <p>The <em>Crypto-Shredding</em> pattern is of course only as good as your encryption and your key management practices. Today’s unbreakable encryption could be tomorrow’s infosec disaster.</p> <p>If the consumers (such as projections) consistently store all sensitive information using encryption, it makes deletions extremely cheap, as only the key must be deleted, as opposed to deleting all copies. But this benefit can be achieved using <em><a href="https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads">Forgettable Payloads</a></em>. Both patterns do not solve the problem of consumer storing encrypted data, or using the encrypted data to compute some new value (that may still be sensitive).</p> <p>There is also a concern of legality: is <em>Crypto-Shredding</em> a sufficient implementation of delete requests under GDPR, or does the law consider the data not truly deleted? EDIT: Read <a href="https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key/#comment-4491442914">Harrison J. Brown’s comment on Crypto-Shredding and GDPR</a></p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Eventsourcing Patterns: Forgettable Payloads 2019-05-13T00:00:00+00:00 https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads <h2 id="forgettable-payloads">Forgettable Payloads</h2> <p>Store the sensitive payload of an event in a separate store to control access and removal.</p> <h3 id="problem">Problem</h3> <p>An <em>Event Store</em> is an append-only chronological database of <em>Domain Events</em>, so ideally we never remove any event or event data. This poses a problem when data needs to be deleted, for example after a customer invokes their right to be forgotten under the GDPR.</p> <p>A different problem (with a similar solution) exists when the events in the <em>Event Store</em> are emitted to outside consumers, but some events contain some information that must remain private to the producer.</p> <h3 id="solution">Solution</h3> <p>(Note: the second problem can also be solved using <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-explicit-public-events/">Explicit Public Events</a> or <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-segregated-event-layers/">Segregated Event Layers</a>.)</p> <p>Remove the sensitive information from the definition of the event. Instead, store that information in a separate (logical) database. In the event, store a reference or URL that points to the payload’s location in the database.</p> <p>Consumers still receive the events, but to read the sensitive data, they need to query the payload database. Tightly control the access to that database, so that only consumers with the right permissions can read the sensitive information. Consumers without access, can still depend on the rest of the event’s non-sensitive payload.</p> <p>When a deletion request is received, remove the payload from the payload database, and leave the events in the Event Store untouched. Consumers that query the payload, get no result for a deleted payload, and show a blank instead.</p> <p>Instead of deleting the payloads, they can also be replaced with a John Doe; that is, a set of fake data to substitute the real data.</p> <h3 id="example">Example</h3> <p>Original event:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerHasSignedUp { customerId: 123, name: "Isaac Newton", organisation: "Trinity College, Cambridge", birthday: 1643-01-04, hobbies: "Physics", signedUpOn: 2019-05-13 status: "Premium member" } </code></pre></div></div> <p>Redesigned event:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerHasSignedUp { customerId: 123, personalData: "example.com/personalData/789" signedUpOn: 2019-05-13 status: "Premium member" } Body of example.com/personalData/789: { personalDataId: 789, customerId: 123, name: "Isaac Newton", organisation: "Trinity College, Cambridge", birthday: 1643-01-04, hobbies: "Physics", } After a removal request: { personalDataId: 789, customerId: 123, name: "John Doe", organisation: "(removed under GDPR removal request #456)", birthday: null, hobbies: "(removed under GDPR removal request #456)" } </code></pre></div></div> <h3 id="weaknesses">Weaknesses</h3> <p>The <code class="language-plaintext highlighter-rouge">Forgettable Payloads</code> pattern requires to maintain a separate storage, so it breaks the concept of an <em>Event Store</em> as the Single Source of Truth in an eventsourced system. Consumers of the events can now no longer depend on the events alone. Instead, after receiving an event, they need to query the payload database. This increases coupling and complexity.</p> <p>A second weakness is that a consumer might receive the event, query the sensitive payload, and then store that information locally When a delete request arrives, the consumer also needs to remove its local copy. This problem is not unique to eventsourcing, but there’s an elegant solution: the producer emits a <code class="language-plaintext highlighter-rouge">GDPRDeletionWasRequested {customerId}</code>, and the consumers react to it. A good habit is to never store the sensitive payloads, and make sure consumers always query the payload database.</p> <p>Finally, sometimes a person can be identified not only through personal information, but through the associations and relations with other information. Say Alice exchanges messages with both Bob and Charlie. Alice requests a deletion and becomes Jane Doe in our records. We don’t know who Jane Doe is, but we now that Jane Doe is a friend of both Bob and Charlie. The <code class="language-plaintext highlighter-rouge">Forgettable Payloads</code> pattern offers no solution here.</p> <h3 id="alternatives">Alternatives</h3> <p>The <a href="https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key">Crypto-Shredding</a> pattern is an alternative solution to the <code class="language-plaintext highlighter-rouge">Forgettable Payloads</code> pattern.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Patterns for Decoupling in Distributed Systems: Segregated Event Layers 2019-05-11T00:00:00+00:00 https://verraes.net/2019/05/patterns-for-decoupling-distsys-segregated-event-layers <h2 id="segregated-event-layers">Segregated Event Layers</h2> <p>Explicitly segregate a Bounded Context’s events in visibility layers, with their own language.</p> <h3 id="problem">Problem</h3> <p>The problem is the same as described in <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-explicit-public-events/">Explicit Public Events</a></em>. Sometimes marking some events as public is not enough. You really need to be able to evolve the internal events separately, while designing the public events differently to suit the clients. The clients may need <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Events</a></em> or <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event/">Fat Events</a></em>, but you don’t want to muddy your <em>Bounded Context</em> with all these consumers’ needs.</p> <p>There’s also the problem that if too many consumers impose their needs on a producer, and all depend on the same events, the consumers are implicitly logically coupled to each other.</p> <h3 id="solution">Solution</h3> <p>Keep all internal events strictly private. Set up an adapter that listens to internal events, and emits a new stream of different events. This may be on stream for all public events, but we can go as far as setting up a separate adapter and stream for each consumer, highly adapted to their need.</p> <p>The new event stream is effectively a different <em>Bounded Context</em> from the one we’re in. The stream has its own language, event types, and names. This makes the <em>Segregated Event Layers</em> effectively an implementation pattern to build an <em>Anti-Corruption Layer</em>. Some names may overlap or conflict with the names in the original <em>Bounded Context</em>, but this is fine: Having the freedom of defining concepts locally without trying to define them globally, is precisely why we use <em>Bounded Contexts</em>.</p> <h3 id="example">Example</h3> <p>In the private layer, we emit:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>OrderWasInitiated {orderId} LineItemWasAdded {orderId, productId, quantity} OrderWasPlaced {orderId, customerId} StockWasReserved {orderId, stockId} TransportWasBooked {transportId, orderId} OrderWasPacked {orderId} OrderWasShipped {orderId} </code></pre></div></div> <p>The adapter is a type of <em>Projector</em> that listens for these events, and emits a new stream for public consumption:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>OrderWasPlaced {orderId, customerName, lineItems} OrderWasConfirmed {orderId, customerName, lineItems} OrderWasShipped {orderId, customerName, lineItems} </code></pre></div></div> <p>The public stream is clearly designed to suit the clients: <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> is a <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Event</a>, the others are <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event/">Fat Events</a>, and there’s no <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee/">Completeness Guarantee</a> because <code class="language-plaintext highlighter-rouge">stockId</code> and <code class="language-plaintext highlighter-rouge">transportId</code> are not in that stream.</p> <p>The public stream is a different <em>Bounded Context</em>: Some of the language in the public stream does not exist in the private stream, like <code class="language-plaintext highlighter-rouge">OrderWasConfirmed</code>; and <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> and <code class="language-plaintext highlighter-rouge">OrderWasShipped</code> have a slightly different meaning from their equivalents in the private stream.</p> <p>We could take this example further by making a separate adapter, stream, and language, for each consumer, such as a customer notification system, the sales department, the shipping company, etc.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Patterns for Decoupling in Distributed Systems: Fat Event 2019-05-11T00:00:00+00:00 https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event <h2 id="fat-event">Fat Event</h2> <p>Add redundant information to a Domain Event to reduce complexity in the consumer.</p> <h3 id="problem">Problem</h3> <p>A consumer is interested in one event type from a producer, to react to it or report information to a user. The producer’s events are designed with a <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee/">Completeness Guarantee</a>. The event only contains the attributes that have changed, and none of the other state of the resources that the consumer is interested in. So the consumer has to listen to a number of other events as well, that each contain some of the state changes of the resource. Most of the work the consumer is now doing, is building state based on events, and doing lookups to associate things with each other. There’s a lot of complexity in the consumer, just to achieve a small task.</p> <h3 id="solution">Solution</h3> <p>A <em>Fat Event</em> is an event that not only contains the attributes that have changed, but also a number of attributes that have not changed at the time the event is emitted. These are specifically chosen to lighten the work that the consumer has to do: it doesn’t have to persist intermediate states from other event types, as everything it needs to do its core job is the one (or few) <em>Fat Events</em>.</p> <p>The consumer now needs to be aware of fewer event types, and is therefore better isolated from changes in the producer.</p> <h3 id="example">Example</h3> <p>A consumer wants to display a list of overdue invoices. It listens to <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue {invoiceId}</code>. This event only contains the <code class="language-plaintext highlighter-rouge">invoiceId</code>, but the user needs to see the customer’s name as well (and other informationbut let’s keep this example simple). So the consumer listens to <code class="language-plaintext highlighter-rouge">CustomerWasInvoiced {invoiceId, customerId, amount, lineItems...}</code>. Now the consumer knows which customer the invoice belongs to, but to get the name, it needs to listen to <code class="language-plaintext highlighter-rouge">CustomerHasSignedUp {customerId, customerName, ...}</code> and <code class="language-plaintext highlighter-rouge">CustomerWasRenamed {customerId, newCustomerName}</code>. The consumer keeps lookup tables:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class OverdueInvoices implements Projector when(CustomerHasSignedUp e) INSERT INTO CustomerNames (e.customerId, e.customerName) when(CustomerWasRenamed e) UPDATE CustomerNames (e.customerId, e.customerName) when(CustomerWasInvoiced e) INSERT INTO Invoices (e.invoiceId, e.customerId) when(InvoiceBecameOverdue e) INSERT INTO OverdueInvoices ( SELECT invoiceId, customerName FROM Invoices i LEFT JOIN CustomerNames cn ON cn.customerId = i.customerId WHERE invoiceId = e.invoiceId ) when(WhichInvoicesAreOverdue q) SELECT * FROM OverdueInvoices </code></pre></div></div> <p>In the new design, we adapt the producer first. <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> will now carry the attribute <code class="language-plaintext highlighter-rouge">customerName</code> as well. The consumer becomes very simple:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class OverdueInvoices implements Projector when(InvoiceBecameOverdue e) INSERT INTO OverdueInvoices (e.invoiceId, e.customerName) when(WhichInvoicesAreOverdue q) SELECT * FROM OverdueInvoices </code></pre></div></div> <h3 id="anti-pattern">Anti-pattern</h3> <p>All patterns become anti-patterns when applied incorrectly, but <em>Fat Events</em> certainly have their share of risks.</p> <p>Adapting the API of a producer to fit the needs of the consumer, should always be done with care. With lots of consumers that each have their own specific demands, it could quickly become mess. Events become huge, and as consumers come and go, it becomes unclear wich attributes are still relevant. Changing the producer becomes harder. This is exactly the kind of situation that the <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee/">Completeness Guarantee</a></em> pattern was supposed to avoid. A useful heuristic is to consider the number of consumers and their ownership. If a single team owns both the producer and consumer, <em>Fat Events</em> pose less of a risk than if a large number of teams each want different redundant attributes in the messages.</p> <h3 id="eventual-consistency-and-immutability">Eventual consistency and immutability</h3> <p>In many cases, there’s a risk that the consumer’s state becomes inconsistent with the producer’s state. The invoice example above is chosen to illustrate this. In the second design, where we use the fat <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> event, suppose the customer is renamed <em>after</em> the invoice became overdue. Now the consumer has a list of overdue invoices with some outdated customer names. In the original design, this problem did not exist, because there it doesn’t matter if the customer name changes before or after the invoice became overdue, the state is always updated.</p> <p>This inconsistency may not be a problem. If, say, only a very small number of customers do in fact change their name, and the invoies only stay unpaid for a short while, the chances of actual inconsistency become slim. And in the rare case it happens, the user might see the old name in one place and the new name in another place, and work out easily that it’s the same invoice. As always, it helps to carefully consider the domain, the users, their expectations and behaviours.</p> <p>A very powerful heuristic is that immutable values are always safe to copy. If in the producer, a value can never change after it was initially set, there is no risk that the copy in the consumer becomes inconsistent. In our example, if we don’t allow customers to change their name, the <em>Fat Event</em> that carries the name is a fine solution to simplify the consumer.</p> <h3 id="identifiers">Identifiers</h3> <p>A special case of immutability are identifiers. In a well-designed system, IDs never change. So assuming that the customers can change their name, but that all IDs are stable, the <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> event can be designed to contain only <code class="language-plaintext highlighter-rouge">invoiceId</code> and <code class="language-plaintext highlighter-rouge">customerId</code>, but no <code class="language-plaintext highlighter-rouge">customerName</code>. The consumer now needs to track customer names again, but doesn’t have to deal with a separate <code class="language-plaintext highlighter-rouge">Invoices</code> lookup table.</p> <p>Because of the safety of copying IDs, as a rule of thumb I recommend to always include redundant IDs in events. The cost is extremely low, and it simplifies consumers. It becomes especially important when dealing with multiple relations between the entities described by the events.</p> <p>Consider:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerHasSignedUp {customerId, customerName} OrderWasPlaced {orderId, customerId} InvoiceWasMadeForOrder {invoiceId, orderId} PaymentWasReceived {paymentId, invoiceId} </code></pre></div></div> <p>In this example, a consumer interested in payment, needs three lookup tables to associate a customer name with a payment. With <em>Fat Events</em>, each event has the redundant, but immutable, IDs, and only a single lookup table is required:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerHasSignedUp {customerId, customerName} OrderWasPlaced {orderId, customerId} InvoiceWasMadeForOrder {invoiceId, orderId, customerId} PaymentWasReceived {paymentId, invoiceId, orderId, customerId} </code></pre></div></div> <h3 id="separated-payload">Separated payload</h3> <p>If you’re concerned about putting redundant attributes in a an event, a nice trick is to separate the payload into the actual state changes, and the “fat”. This makes it obvious to other developers that these attributes are not part of the core event.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>InvoiceBecameOverdue { core: {invoiceId}, fat: {customerName} } </code></pre></div></div> <h3 id="summary-event">Summary Event</h3> <p>The <em>Fat Event</em> pattern is distinct from the <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Event</a> in a few ways:</p> <ol> <li>A <em>Summary Event</em> usually happens at the end of a business process, whereas all events can be <em>Fat Events</em>.</li> <li>A <em>Summary Event</em> is often (but not necessarily) a new event type, whereas <em>Fat Events</em> are existing event types with additional attributes.</li> <li>A <em>Summary Event</em> contains often a complete representation of the resource or artefact that resulted from the business process, whereas <em>Fat Events</em> usually only contain a few specific extra attributes the consumer cares about. <em>Summary Events</em> are more likely to be useful for a range of different consumers.</li> </ol> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Patterns for Decoupling in Distributed Systems: Explicit Public Events 2019-05-11T00:00:00+00:00 https://verraes.net/2019/05/patterns-for-decoupling-distsys-explicit-public-events <h2 id="explicit-public-events">Explicit Public Events</h2> <p>Mark a small subset of events as public, keep the rest private by default.</p> <h3 id="problem">Problem</h3> <p><em>Domain Events</em> are not only useful for communicating to other <em>Bounded Contexts</em>, but also for organising and decoupling code inside the <em>Bounded Context</em>. In CQRS/Eventsourcing architectures, this is even the primary way of organising the code. It’s very convenient to publish all these events using a single bus, effectively making them all public. This is a problem when some of these events contain sensitive data.</p> <p>More importantly, the outside API is tightly coupled to the internal structure of the Bounded Context. Changing the internals would force an API change.</p> <h3 id="solution">Solution</h3> <p>Set up separate messaging channels for inside the Bounded Context and outside. Keep all events private by default, and indicate the ones you want to make public with an explicit <code class="language-plaintext highlighter-rouge">@Public</code> annotation, a marker interface, or an <code class="language-plaintext highlighter-rouge">isPublic():bool</code> method. When emitting events, the event publishing mechanism knows to read the annotation and either send the event on the private channel only, or on both the private and the public channel.</p> <h3 id="discussion">Discussion</h3> <p>In general, my feeling is that the problem seldom occurs when the <em>Domain Events</em> have been chosen carefully to reflect the business domain, using <em>Ubiquitous Language</em>, and at the right granularity. These events then tend to become stable very quickly during development, and rarely need to be altered. When there are significant changes in the domain, the events may need to change, but in these cases you’ll need an API change anyway.</p> <p>That said, it’s a universally useful heuristic in software design to keep everything as closed off as possible, and only open up things where there’s a good case for it.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Patterns for Decoupling in Distributed Systems: Passage of Time Event 2019-05-10T00:00:00+00:00 https://verraes.net/2019/05/patterns-for-decoupling-distsys-passage-of-time-event <h2 id="passage-of-time-event">Passage of Time Event</h2> <p>Replace cron jobs and scheduled commands, with an agnostic event to indicate the passage of time.</p> <h3 id="problem">Problem</h3> <p>Many business processes involve some action or job or workload that needs to be performed at a future date. It can be a one-off action or a repeated action, it can be scheduled for a specific date (eg Christmas), a recurring date (the last weekday of the month) or a timeout (thirty days from now).</p> <p>We want to keep the domain logic for the entire process nicely isolated in a single (micro)service, so that’s also where the logic for this action goes. But there is a small amount of logic that is not inside this service: the fact itself that the action needs to happen, and when it needs to happen.</p> <p>Cron is the most troublesome: it only works for repeated actions, and offers little control to outside tools. In large systems, the cron file can be a huge mess. More modern solutions allow us to schedule using by calling an API from within our code, so that logic moves to our own service. But fundamentally, the scheduler is still a separate service that knows more than it should. In DDD lingo, the Ubiquitous Language of our business process is leaking. We’ve got more types of elements, and more moving parts in our system.</p> <h3 id="solution">Solution</h3> <p>The mind switch is to think of the passage of time as just another <em>Domain Event</em>, exactly like all the other events. After all, if we define a <em>Domain Event</em> as a granular point in time where something happened that is relevant to the business, then certainly the next business day, month, or quarter, is extremely relevant.</p> <p>In the new design, a cron or scheduler emits generic <em>Passage of Time Events</em> at regular intervals, such as a <code class="language-plaintext highlighter-rouge">DayHasPassed {date}</code> event at midnight, or a <code class="language-plaintext highlighter-rouge">QuarterHasPassed {year, quarter}</code>. All interested services listen for this event. They can react to it, by performing an action, by increasing a counter, or by querying some database and filter by date, to find items that have some work to be done.</p> <h3 id="example">Example</h3> <p>Time governs everything. It’s hard to look at a business and not find examples where the <code class="language-plaintext highlighter-rouge">Passage of Time</code> pattern is useful: billable hours, subscriptions, resource usage, rentals, accruing interest, payments, reporting, salaries, maintenance schedules, and everything that is cyclic.</p> <p>On an invoice’s due date, we need to send a reminder to the client. In the old design, we can set up a cron job that calls <code class="language-plaintext highlighter-rouge">CheckForOverdueInvoices</code>. In the new design, the cron generates <code class="language-plaintext highlighter-rouge">DayHasPassed</code> every midnight. The <code class="language-plaintext highlighter-rouge">InvoiceDebtCollection</code> listens to <code class="language-plaintext highlighter-rouge">DayHasPassed</code>. Whenever this event arrives, <code class="language-plaintext highlighter-rouge">InvoiceDebtCollection</code> queries <code class="language-plaintext highlighter-rouge">SELECT * FROM Invoice AS i WHERE DATEDIFF(i.dueDate, NOW()) &gt;= 30</code>. It can either send the reminder directly, but even better would be to emit a new <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> event. Now, the service can listen to its own event and send the reminder. Other services can react to <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> as well, for example to adjust a revenue projection or suspend an account.</p> <p><em>Passage of Time Events</em> can also be a little more domain-specific. The NASDAQ’s pre-market trading hours are from 04:00 to 09:30, followed by normal trading hours until 16:00, and after-hours trading until 20:00. On holidays there’s no trading. A service can produce events for each start and close, that can be consumed by many interested services.</p> <h3 id="benefits">Benefits</h3> <p>It’s very explicit. In the above example, an event log would show:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerWasInvoiced DayHasPassed DayHasPassed ... InvoiceBecameOverdue ReminderWasSent AccountWasSuspended </code></pre></div></div> <p>It makes writing tests for business processes with a time element very elegant:</p> <div class="language-gherkin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">Scenario</span><span class="p">:</span> If no payment is received after 15 days, we suspend the account <span class="nf">Given </span>CustomerWasInvoiced <span class="nf">And </span>DayHasPassed <span class="nf">And </span>DayHasPassed <span class="nf">And </span>(...) <span class="nf">When </span>DayHasPassed <span class="nf">Then </span>InvoiceBecameOverdue <span class="nf">And </span>AccountWasSuspended <span class="kn">Scenario</span><span class="p">:</span> If payment is received in time, we do nothing <span class="nf">Given </span>CustomerWasInvoiced <span class="nf">And </span>DayHasPassed <span class="nf">And </span>DayHasPassed <span class="nf">When </span>PaymentWasMade <span class="nf">Then </span>Nothing </code></pre></div></div> <p>(You’d probably want to add a shortcut so you don’t need to write the <code class="language-plaintext highlighter-rouge">DayHasPassed</code> line 15 times.)</p> <p>More importantly, it’s a nicely reactive approach. When a service sends a <em>Command</em> to another service, it needs to know that this other service accepts that <em>Command</em>. When all we do is send generic <em>Passage of Time Events</em>, the scheduler doesn’t need to know who’s listening, how the consumers should react, or if there’s even any service left that listens. All decisions and domain knowledge lie with the reciever. That’s great decoupling.</p> <p>It’s also temporal decoupling: The scheduler can put this <em>Passage of Time Event</em> on a queue, and it doesn’t matter if the consumers are available at this time to handle the event. For all it cares, a consumer could be down for a few days, and then simply catch up and process all the <code class="language-plaintext highlighter-rouge">DayHasPassed</code> events on the queue.</p> <p>In Event Sourcing, you simply store the <code class="language-plaintext highlighter-rouge">DayHasPassed</code> events in the event store, so you can play back entire histories exactly as they happened over time, with no dependency on an external source.</p> <h3 id="implementation">Implementation</h3> <p>The <em>Passage of Time Events</em> use the exact same format and protocol as your other <em>Domain Events</em>. In code, you can make classes for them, which are exactly like the classes for other <em>Domain Events</em>. And you send them over the same messaging infrastructure as everything else. This makes this pattern very cheap in terms of implementation.</p> <h3 id="weaknesses">Weaknesses</h3> <p>There is some domain knowledge in the other direction: The domain logic to calculate when something needs to happen, such as “every 10th of the month”, has now moved effectively from the cron or scheduler, to the service. In practice, it’s not a big deal, as you can find time libraries that can do the work for you. On the upside, “every 10th of the month except in weekends, during full moons, or during the yearly office party” is something no scheduler will do for you anyway.</p> <p>It’s worth noting that you don’t want to use the <em>Passage of Time Event</em> pattern for realtime systems. We can easily spare the 365 events per year for <code class="language-plaintext highlighter-rouge">DayHasPassed</code> but for systems that deal with minutes or seconds or less, it’s not feasible. Luckily for programmers, in most businesses, the word “immediately” means by “by the end of business day”, or “by the end of the week”, or “before the end of the quarter following the month where the thing happened”.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Patterns for Decoupling in Distributed Systems: Summary Event 2019-05-08T00:00:00+00:00 https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event <h2 id="summary-event">Summary Event</h2> <p>Instead of emitting a stream of Domain Events, emit a single Summary.</p> <h3 id="problem">Problem</h3> <p>A business process involves a number of steps that each produce domain events. A consumer depends on information in those events, and listens to all of them to make meaningful decisions. This in itself is perfectly fine, especially when it’s only a small number of event types. When the number of event types is large, we can get concerned about the amount of knowledge the consumer now has of the producer’s business process. The consumer is now almost like a micromanager, looking over the shoulder of the producer and getting immediate reports of everything the producer does.</p> <p>This has a risk: when the original business process evolves, event types might be added, removed, or altered. This requires a change in the consumer, which needs to be adapted to the new types, and either be deployed at the same time or have a good versioning strategy.</p> <p>And it gets worse if there are multiple consumers, that all need to be aware of every type of event the producer emits. There’s often also quite some duplication: each consumer projects similar or identical state off of the events.</p> <h3 id="solution">Solution</h3> <p>Instead of emitting each event, the original business process can keep all the events private. At the end of the process, the process emits a single <em>Summary Event</em>. This event is redundant, in the sense that it contains only information that was already available in the preceding events. Consumers, instead of being aware of every event, are now only listening to this <em>Summary Event</em>, which tells them everything they need to know, with little to no irrelevant (to them) information. Consumers do not need to track state changes during the process, because they get everything at the end, and can then act on them.</p> <p>You can recognize opportunities to apply the <em>Summary Event</em> pattern, by looking at the consumer. If the consumer listens to multiple event types, but there’s only one event type that causes the consumer to do some meaningful domain action (besides keeping state), it’s a good sign. The producer offers clues as well: the business process involves creating some kind of artefact, and the business value is not necessarily in the artefact itself, but in the usage of the artefact after it has been created. (@TODO Cfr Three Archetypes by Cyrille)</p> <h3 id="example">Example</h3> <p>Many business processes involve creating an artefact for later productive use. During an <code class="language-plaintext highlighter-rouge">Ordering</code> process, different users might add and remove line items of items they wish to order. They might change the quantities of the items they want. And an account manager might change the prices, change the quantities based on available stock, and so on. In the original design, the consumer listens to <code class="language-plaintext highlighter-rouge">LineItemWasAdded</code>, <code class="language-plaintext highlighter-rouge">LineItemWasRemoved</code>, <code class="language-plaintext highlighter-rouge">QuantityWasChanged</code>, <code class="language-plaintext highlighter-rouge">PriceWasChanged</code>, as well as events such as <code class="language-plaintext highlighter-rouge">ShippingAddressWasAdded</code>. The consumer, say an <code class="language-plaintext highlighter-rouge">OrderFulfilment</code> process, listens to most of these events, and projects a list of of items that need to be packed and shipped.</p> <p>But <code class="language-plaintext highlighter-rouge">OrderFulfilmentProcess</code> doesn’t do anything until the <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> event arrives. That’s when it kicks into action. The <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> doesn’t have redundant information, so it mostly consists of an <code class="language-plaintext highlighter-rouge">OrderId</code>, and not much else.</p> <p>We can now refactor the <code class="language-plaintext highlighter-rouge">Ordering</code> process, to keep all its events private. Instead, we make a single public event for <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code>, but this one contains the complete information about the order. In fact, one way to look at it is that <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> is the actual order document. (@TODO I believe this is the Document Message pattern in Enterprise Integration Patterns, I need to look it up).</p> <p>This is of course very familiar, precisely because this is how information travels in businesses, and has traveled for ages before computers: Someone working in the Order Fulfilment department is usually not aware of all the negotiations and changes that happen to a draft order. They only get a message with the resulting order, and act on that.</p> <p>The point of the <em>Summary Event</em> pattern is to look for opportunities to apply it in domains that are not as mature as ordering, or in situations where it is not obvious that the two highly coupled systems could reduce the communication between them using this pattern.</p> <h3 id="implementation">Implementation</h3> <p>You can use a <em>Projection</em> to generate the <em>Summary Event</em>. A <em>Projection</em> is an event listener that persists state from events it listens to, and either exposes that state by responding to queries, or by emitting new events.</p> <h3 id="weaknesses">Weaknesses</h3> <ol> <li> <p>Sometimes the consumer does need to know a lot of internals of the producer to do its job. In that case, the <em>Summary Event</em> pattern is not helpful, as it would not only contain the resulting information, but also metadata about each step in the process.</p> </li> <li> <p>In the ordering example above, <code class="language-plaintext highlighter-rouge">OrderFulfilment</code> doesn’t care about the price of the order. Another consumer, say <code class="language-plaintext highlighter-rouge">RevenueReporting</code>, cares about the prices but not the exact items ordered. Yet they both receive the same <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> event with everything in there. There might be cases where this is a problem, for example if some of the information is sensitive.</p> </li> <li> <p>The new <code class="language-plaintext highlighter-rouge">Summary Event</code> adds yet another event type, a “new standard to replace all standards”.</p> </li> <li> <p>The <code class="language-plaintext highlighter-rouge">Summary Event</code> might not naturally belong to the domain of the producer. In that case, we can add an intermediate service, that consumes the producer’s events, summarizes them, and then emits the <code class="language-plaintext highlighter-rouge">Summary Event</code> for consumption.</p> </li> </ol> <h3 id="diagram">Diagram</h3> <p>Original design:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Producer: OrderWasInitiated LineItemWasAdded PriceWasChanged ... OrderWasPlaced | | | | v v v v Consumer: (update state) (update state) (update state) ...-&gt; (start fulfilment) </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> is refactored to a <em>Summary Event</em>, the other events become private:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Producer: OrderWasInitiated LineItemWasAdded PriceWasChanged ... OrderWasPlaced | v Consumer: (start fulfilment) </code></pre></div></div> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Patterns for Decoupling in Distributed Systems: Domain Query 2019-05-08T00:00:00+00:00 https://verraes.net/2019/05/patterns-for-decoupling-distsys-domain-query <h2 id="domain-query">Domain Query</h2> <p>Replace Free Queries with Domain Queries to decouple from knowledge of the server’s internals.</p> <h3 id="problem">Problem</h3> <p>The word query is usually associated with database queries. There are however other ways we can query a system that we don’t perceive as a database. REST and GraphQL come to mind. I propose the term <em>Free Query</em> for every type of message that knows about the structure, schema, endpoints, … and has a great freedom in accessing and combining data from that system, typically using a rich query language.</p> <p>In a very simple setup with a single client and server, owned by a single team, this isn’t a great problem, as changes to the server and client can be applied simultaneously.</p> <p>In a more complex environments, where multiple clients are outside the control of the server’s team, it is. The database schema or service endpoints are essentially a public API. The clients have intimate knowledge of the schema. We don’t know what queries they will send us, so the only way to guarantee backwards compatibility of that API is to keep the schema intact. Sure, we could log the queries, but that doesn’t guarantee there are yet-to-be sent queries hiding in the client.</p> <h3 id="solution">Solution</h3> <p>Instead of exposing the schema or endpoints and allowing <em>Free Queries</em>, define a set of domain-specific messages and responses. These <em>Domain Queries</em> consist of a name and a small number of arguments. Both the <em>Domain Queries</em> and their responses are highly tailored to the use case and only use the <em>Ubiquitous Language</em>. I recommend the name to be a question expressed in natural language.</p> <p>The server provides a single endpoint for all <code class="language-plaintext highlighter-rouge">Domain Queries</code>, and hides any implementation details. Knowledge about where and how to fetch and compute the relevant data, is encapsulated in the server.</p> <h3 id="example">Example</h3> <p>The client sends the following Free Query:</p> <pre><code class="language-SQL">SELECT * FROM Cars AS c INNER JOIN LeasingContracts AS lc ON c.CarId = lc.CarId WHERE c.RegistrationYear = 2014; </code></pre> <p>It is unclear from the query alone what the purpose is. Presumably not all of the fields from the two tables are used by the client, but we can’t change our schema without risking to break things.</p> <p>Some investigation learns that the client wants to know which cars are up for replacement, and uses somes fields from <code class="language-plaintext highlighter-rouge">LeasingContracts</code> to compute the status. A better design is to close off all access to the database, and provide a <em>Domain Query</em> using natural language. The server fetches and computes the results, and the <em>Response</em> message contains only the data that the client actually needs.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>WhichCarsAreUpForReplacement : Query { registrationYear: 2014 } CarsThatAreUpForReplacement : Response { cars: [CarId] } </code></pre></div></div> <h3 id="discussion">Discussion</h3> <p>It may feel strange to use names like <code class="language-plaintext highlighter-rouge">WhichCarsAreUpForReplacement</code> instead of programmer-language like <code class="language-plaintext highlighter-rouge">CarReplacementQuery</code>. I assert that this is only a matter of familiarity. The benefits are that we stay much closer to the language of the business, and that it puts us in the mindset of making very deliberate expressions of use cases.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Patterns for Decoupling in Distributed Systems: Completeness Guarantee 2019-05-08T00:00:00+00:00 https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee <h2 id="completeness-guarantee">Completeness Guarantee</h2> <p>Design the set of Domain Events from a producer so that they can be used to rebuild the producer’s state.</p> <h3 id="problem">Problem</h3> <p>Often, the events emitted by a producer, are designed haphazardly. New event types are added whenever a new feature requires them. A consumer needs to be aware of an event, so we come up with an event in the producer, give it a name, and add some attributes the consumer needs. Then perhaps later another consumer is added, and it reuses some existing events, but it needs just a little bit of additional information. This is added as new attributes in an existing event. Later, a consumer is removed, but the events still exist.</p> <p>After a while it becomes hard to understand which events to use where, what they contain, and what they mean exactly. Consumers become indirectly coupled to each other, and the producer needs to support backwards compatibility to a growing sets of messy events.</p> <h3 id="solution">Solution</h3> <p>We need an organizing principle that answers the question “What goes in an event?”, looking holistically at the set of events from a producer.</p> <p>A <em>Completeness Guarantee</em> is the idea that if a consumer listens to the entire stream of <em>Domain Events</em>, it must be able to faithfully reproduce the entire state of the producer. Every action in the producer that changes state, causes an event to be emitted, and each event contains every unit of information that is affected by the state change. On top of that, ideally the events contain no redundant information: the event only has the attributes that have changed, and not a single one more. (Note: <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event/">Fat Events</a></em> discusses cases where you’d want to add redundant attributes.)</p> <p>The effect of a <em>Completeness Guarantee</em> is that it becomes clear and obvious which events are relevant, what they mean, what attributes they contain, etc. There is the certainty that any feature that could be implemented in the producer, can be implemented in a consumer, with no missing information. This enables a system designer to choose where to move certain responsibilities.</p> <p>The <em>Completeness Guarantee</em> gives a great degree of decoupling to consumers: they never need to query the producer, as they already have access to all information.</p> <h3 id="implementation">Implementation</h3> <p>In systems that use Event Sourcing, you get the <em>Completeness Guarantee</em> for free. I define Event Sourcing as a persistence style where a store of domain events is the single source of truth for the system’s state. State may stored in other ways, but this state is discardable, and can be rebuilt from the event store. If the events are the single source of truth, it follows that all state can be rebuild, so the guarantee is sound.</p> <p>For system designers used to Event Sourcing, the <code class="language-plaintext highlighter-rouge">Completeness Guarantee</code> comes naturally. The point of the pattern is to show that it is useful outside of Event Sourcing as well.</p> <p>How do you actually achieve that guarantee, when the single source of truth of a system is a mode traditional database, which stores state instead of events? One approach is to write a test for it. This test listens to all events from production, and projects them into an identical schema as the producer’s real schema. The test compares to projected state to the production state, and alerts when there are differences. However, instead of going through the effort of building and running this test, you might be better off just using Event Sourcing in the first place.</p> <ul> <li><strong>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></strong></li> <li><strong>Intrigued? I teach workshops about <a href="https://training.dddeurope.com/ddd-messaging-architectures-mathias-verraes/">DDD for Messaging Architectures</a>.</strong></li> </ul> Design Heuristics 2018-04-26T00:00:00+00:00 https://verraes.net/2018/04/design-heuristics <p><a href="https://skillsmatter.com/skillscasts/11685-we-are-delighted-to-have-mathias-verraes-at-ddd-exchange-2018#video">Video for my DDD eXchange 2018 keynote in London</a></p> <p><a href="https://skillsmatter.com/skillscasts/11685-we-are-delighted-to-have-mathias-verraes-at-ddd-exchange-2018#video"> <img style="float:left;margin-right: 10px" src="/img/posts/2018-04-26-design-heuristics/27918963728_6a70e4c83d_edit.jpg" alt="" /> </a></p> <h2 id="abstract">Abstract</h2> <p>Software design principles aspire to be universal. And yet, when you create software, you sometimes intentionally violate principles.</p> <p>You might not be able to explain why this “wrong” design somehow “feels” better. You’re applying your own, unspoken design heuristics. Bringing them out in the open, improves design discussions, and helps you build a larger toolkit for making better tradeoffs.</p> <p>In this session, Mathias will explore the benefits of distilling your design heuristics using practical examples.</p> Emergent Boundaries 2017-04-27T00:00:00+00:00 https://verraes.net/2017/04/emergent-boundaries <ul> <li><a href="https://skillsmatter.com/skillscasts/9847-emergent-boundaries#video">Video of my DDD eXchange 2017 talk in London</a></li> <li><a href="https://www.youtube.com/watch?v=ECM1rPYxvD4">Video of my ExploreDDD 2017 talk in Denver</a></li> </ul> <iframe width="560" height="315" src="https://www.youtube.com/embed/ECM1rPYxvD4" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe> <h2 id="abstract">Abstract</h2> <p>Modelling is more than knowledge management. It’s complexity management. To reduce cognitive load, you split things up, whether at small scale, in code, or in the large, such as Bounded Contexts. But if you choose large boundaries upfront, you risk getting it wrong, and being stuck to them for a long time. Join Mathias in a discussion of better ways of empirically discovering boundaries instead.</p> Messages Over Structure 2017-04-03T00:00:00+00:00 https://verraes.net/2017/04/messages-over-structure <p><a href="https://www.youtube.com/watch?v=2nDNMB_wXcE">Video of my keynote at Neos Conference 2017 in Hamburg</a></p> <iframe width="560" height="315" src="https://www.youtube.com/embed/2nDNMB_wXcE" frameborder="0" allow="autoplay; encrypted-media" allowfullscreen=""></iframe> On Being Explicit 2016-11-09T00:00:00+00:00 https://verraes.net/2016/11/on-being-explicit <p><a href="https://skillsmatter.com/skillscasts/8915-on-being-explicit">Video of my talk at Agile Testing &amp; BDD eXchange 2016 in London</a> <a href="https://skillsmatter.com/skillscasts/8806-ddd-meetup">Video of my talk at the DDD London meetup</a></p> <p>Slides:</p> <script async="" class="speakerdeck-embed" data-id="70c97069d81543deb57b8cf974f5b58b" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script> <h2 id="abstract">Abstract</h2> <p>“Make the implicit explicit” must be one of the most valuable advices I ever got about software modelling and design. Gather around for some tales from the trenches: stories from software projects where identifying a missing concept, and bringing it front and centre, turned the model inside out. Our tools: metaphors, pedantry, type systems, the age old heuristic of “Follow the money”, visual models, and a healthy obsession with language.</p> Designed Stickiness 2016-05-24T00:00:00+00:00 https://verraes.net/2016/05/designed-stickiness <p>Video &amp; slides for my NCrafts talk on software metaphors.</p> <iframe src="https://player.vimeo.com/video/167722762" width="640" height="360" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe> <script async="" class="speakerdeck-embed" data-id="ebac02f6dafb48baa086edb30fdd3a49" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script> <h2 id="abstract">Abstract</h2> <p>When we model, we tend to do it for ourselves: gain understanding, capture the business language, and turn it into running code. But are we missing opportunities to do something more with our models? What if, instead of mirroring the business, our models could transform it? Let’s find out how a healthy obsession with language and metaphor can step up from “domain-driven” to “driving the domain”.</p> Lightning Talks 2016-05-05T00:00:00+00:00 https://verraes.net/2016/05/lightning-talks <p>I submitted proposals to conferences for almost two years before getting accepted. When it finally happened, I found myself in front of a 250 head audience, realising <em>I didn’t actually know how to do proper presentations</em>. I wish I knew then that there are better ways.</p> <p><strong>Lightning talks at meetups are ideal to practice public speaking</strong>, and experiment with explaining or pitching something in a short time. I’m trying to write one each month for the Software Craftsmanship Belgium meetups. It’s only 30 attendees, so it’s an opportunity to <strong>try things that I wouldn’t do as a proper conference talk yet</strong>. And I can recycle the ones that do work for other meetups or conferences.</p> <p>My self-inflicted constraints:</p> <ul> <li><strong>10 minutes max</strong>, or 5 minutes + Q&amp;A.</li> <li>Write the slides in a <strong>single evening</strong>.</li> <li>Take a topic that I haven’t talked or written about before. Trying to explaining a topic you don’t know well, is a great way to learn.</li> <li><strong>Assume the audience doesn’t know the topic</strong> either, so it needs to cover the basics.</li> <li>When showing code, it needs to be <strong>understandable for people unfamiliar with the language</strong>, and within the 30 seconds or so that the code is on the screen. I might write unidiomatic Haskell, or leave out anything complicated, or just show pseudo-code.</li> </ul> <p>Not constrained by:</p> <ul> <li><strong>Being original</strong>. It doesn’t have to be my own inventions, my own examples, etc. Presenting a blog post written by somebody else is perfectly fine.</li> <li><strong>Being precise, perfect, or nuanced</strong>. First get it out there, then perhaps iterate later.</li> </ul> <p>I’ll post the slides here (reverse chronology). They’re probably useless without the explanation, but perhaps someone gets something out of them.</p> <h2 id="how-to-find-the-bar">How to Find the Bar</h2> <p>August 2016, SoCraTes Belgium <script async="" class="speakerdeck-embed" data-id="b72a0afd9eff4a5db78be607bbdfdb01" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script></p> <h2 id="the-monty-hall-problem">The Monty Hall Problem</h2> <p>May 2016, SoCraTes Belgium <script async="" class="speakerdeck-embed" data-id="b0c29a5d8501415ca9b4edb66e397e02" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script></p> <h2 id="property-based-testing">Property Based Testing</h2> <p>April 2016, SoCraTes Belgium and CukeUp London <script async="" class="speakerdeck-embed" data-id="eedd62be375f4802b61e3a6afc683c84" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script></p> <h2 id="the-worlds-shortest-and-most-chaotic-introduction-to-event-storming">The World’s Shortest and Most Chaotic Introduction to Event Storming</h2> <p>April 2016, CukeUp London This was a hands-on Event Storming session with a short introduction.) <script async="" class="speakerdeck-embed" data-id="dc2dac4a332f44d6bc338f3a58660b8e" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script></p> The Repair/Replace Heuristic for Legacy Software 2016-04-26T00:00:00+00:00 https://verraes.net/2016/04/repair-replace-heuristic-for-legacy-software <p><img style="float:right;margin-left: 10px" src="/img/posts/2016-04-27-repair-replace-heuristic-for-legacy-software/david-d-angers-small.jpg" alt="Galerie David d'Angers" /></p> <p>Technical Debt is a great metaphor. It shares many analogous properties with financial debt: loans, accrued interest, token payments, bankrupty… There is a key difference however. <strong>We take financial debt with another party.</strong> This party desires to get their money back eventually, and then some. Over centuries, a system was developed to measure debt fairly unambiguously. We called it money, and we’re still iterating and experimenting with new designs, like Blockchain. A second system was developed, to create strong incentives for parties to honour debt agreements. We called this system Property Laws.</p> <p>Technical Debt has no measure like money, and no ruleset like Property law, and, more importantly, <strong>with Technical Debt there is no other party</strong>. The organisation is <strong>both the creditor and debtor</strong>. This takes away all incentive to control how Technical Debt is taken out and paid back. If you ask your CFO how much financial debt your organisation has, she’ll show you spreadsheets, graphs, payment plans, and projections. She makes quarterly reports for the shareholders. Ask your CTO about the Technical Debt in your company, and you’ll get a vague “um… a lot”, or “lol idk, u can’t measure that bro”.</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2016-04-27-repair-replace-heuristic-for-legacy-software/tied_by_its_shoelaces.png" alt="Technical Debt by Ruth Malan" /></p> <p>In “<a href="/2013/07/managed-technical-debt/">Managed Technical Debt</a>”, I propose a cheap, imprecise, but surprisingly effective method for mapping and measuring debt. In short, it involves posting stickies whenever progress is impeded by debt, and keep marking the stickies for every incident. Over time, we build a map that is not based on perceived quality, but on the actual cost of debt. The <em>Wall of Technical Debt</em> serves as a decision tool for focusing our repayment efforts.</p> <h2 id="heuristics">Heuristics</h2> <p>So now we have identified our worst Technical Debt, and made estimations on its relative size. What should we do, replace it or repair it?</p> <p>The Wall of Technical Debt is a <strong>structural model</strong>, but I like to build <strong>temporal models</strong>. Move beyond “<em>What debt do we have?</em>” towards “<em>How did we get here? What happened before? What was it like back then?</em>”. Call it archaeology, or historical thinking if you will. The point is, you learn a lot from temporal models that you don’t get from a mere structural model of the here and now.</p> <p>When evaluating our legacy systems, these questions become “<em>What was this system like before? Did people like building it? Did users like using it? Did it meet goals, did it balance the trade-offs well? Has the purpose of the system changed, or have the constraints changed?</em>” Perhaps the system was well-designed, but not designed for scale, or for mobile, or for multi-tenancy. Businesses change, markets change, regulation changes, technology changes.</p> <blockquote><p>A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.<small>Gall's Law, from <a href="http://amzn.to/1Qz1Yhg">Systemantics, John Gall, 1975</a></small></p></blockquote> <p>Based on this, I propose:</p> <p><strong>The Repair/Replace Heuristic for Legacy Software:</strong></p> <ul> <li><strong>If the system had good design, repair it;</strong></li> <li><strong>If the system had bad design, replace it.</strong></li> </ul> <p>If the system used to work well, and it was well-designed, meaning its structure was congruent with the purpose and the trade-offs, there might still be a lot of value in it. There’s a lot of <strong>learning encapsulated in its design</strong>, and we don’t want to throw out the (middle-aged) baby with the bathwater. We compare the purpose and the constraints the system had back then with the current ones. From that, we might learn which adaptations can give it a second life. With luck, the system was designed with flexibility in mind, and can be changed cheaply.</p> <p>On the other hand, a system that never worked really well, is likely beyond saving. Replacement is a more attractive alternative.</p> <p>That said, it’s a <strong>heuristic, not a rule</strong>. There are many other forces that affect the outcome of a Replacement project, and you’ll need to understand them to succeed. <a href="http://amzn.to/1VQSNRW">Second-System Syndrome</a> is real, and this industry has spent insane amounts of money on failed attempts.</p> <p>Even when you’re not trying to decide on <strong>Repair/Replace</strong> — perhaps the decision was already made by others — the <em>process</em> of mapping its history will teach you more about the system and and its design. And one deep insight you learn from temporal modelling, is what is captured in <em>Boulding’s Backward Basis</em>:</p> <blockquote><p>Things are the way they are because they got that way.<small> <a href="http://amzn.to/234iIUW">The Secrets of Consulting, Gerald M. Weinberg, 1985</a></small></p></blockquote> <p>Or, an ounce of prevention is better than a pound of cure.</p> <p>(Update May 5 2016:)</p> <p>When I say system, I don’t necessarily mean “the entire system”. Any large system, no matter if it is monolithic, can be thought of as a composition of a systems. They may be tightly coupled, and hard to replace or repair individually, but they have their own purpose, history, forces, and authors. And, just to drive home the point: what I propose above is a heuristic. Good decision making uses a multitude of heuristics and balances all forces.</p> <p><small><a href="http://www.ruthmalan.com/Journal/2013/2013JournalMarch.htm#Technical_Debt">Shoelaces cartoon by Ruth Malan</a></small></p> <h2 id="read-more">Read More</h2> <ul> <li><a href="/2013/07/managed-technical-debt/">Managed Technical Debt</a></li> </ul> Property-based testing 2016-04-14T00:00:00+00:00 https://verraes.net/2016/04/property-based-testing <p><a href="https://skillsmatter.com/skillscasts/8088-property-based-testing">Video for my CukeUp 2016 lightning talk in London</a></p> Code Reviews and Blame Culture 2016-04-09T00:00:00+00:00 https://verraes.net/2016/04/code-reviews-and-blame-culture <p><em>Gated pre-merge code reviews are bad. Always refactor on master. Always do Pair Programming. Don’t use branches. Apply Continuous Integration.</em></p> <p>Best practices likes these are great. They <strong>encode experiences of many individuals and teams into memorable slogans, and help others to make decisions on how to run their teams processes</strong>. Unfortunately, the not-so-great practices also get their share of the slogans. Phrases like <em>“TDD is dead”</em> and <em>“Test-induced design damage”</em> are just as catchy.</p> <p>The human brain likes to rationalise. We like to <strong>mix and match the advice that fits our narrative</strong>. When for example, our code makes writing unit tests hard, adopting <em>“TDD is dead”</em> as a slogan is much more comfortable than learning how to do safe refactors and introduce a test harness. Combine a lack of automated tests with the advice to refactor on master, and you’ll get a recipe for disaster.</p> <p>As a consultant, sometimes I get hired to help out in high performing teams that deploy great quality software daily. More often though, the environment is less than ideal. If the idea of bringing in external help was a tough sell to management, I’m rarely there for more than a few days. I could come in, recite all the best practices, and be gone (and I’m afraid I’ve made that mistake on occasion). More valuable is to <strong><a href="/2014/03/small-controlled-experiments/">look for the small improvements that can have a lasting impact</a></strong> and put the team on the path to improvement. If that’s just enough wind in the sails to show that progress is possible, that might be enough to get them to take control of their own situation.</p> <p><img style="float:right;margin-left: 10px; margin-bottom: 20px" src="/img/posts/2016-04-09-code-reviews-and-blame-culture/treasure-medium.jpg" alt="Gradual improvement" /></p> <p>I advocated <a href="/2013/10/pre-merge-code-reviews/">Pre-merge Code Reviews</a> on this blog a couple of years ago, and I still stand by it. A common believe is that this kind of gated reviews lead to blame culture. If you work in a high performing team, and do a lot of Pair Programming (or even Mob Programming!), then shifting to reviews is indeed a step down, that removes trust. But simply shouting on Twitter that peer reviews are bad, displays <strong>a lack of appreciation for the contexts of others</strong>.</p> <p>Many teams have no tests, no refactoring skills, and a high degree of blame culture, where individuals get blamed for breaking the code. Not coincidentally, managers in these teams tend to believe Pair Programming will cut the productivity in half. Refactoring is high risk, so nobody does it out of fear of getting blamed.</p> <p>In my experience, reviews are often sellable here. And, counter-intuitively, <strong>Pre-merge Code Reviews can actually reduce blame culture!</strong> You need some simple rules:</p> <ol> <li><strong>Open pull requests early;</strong></li> <li><strong>Never merge your own code;</strong></li> <li><strong>Never merge code you don’t understand or disagree with;</strong></li> <li><strong>Always review others’ code before starting on new tasks;</strong></li> <li><strong>Commenting on others’ pull requests is good, adding commits is better.</strong></li> </ol> <p>The interesting effect is that bugs, regressions, bad design, and lack of tests, are now no longer the fault of the person who wrote the code. <strong>It is the team that has failed to catch the issues. If you fail collectively, you’ll be induced to find solutions collectively.</strong> To reduce the effort of reviews, perhaps you’ll start adding some automated smoke tests. The senior team members might put more effort into teaching the juniors. Maybe two people will spend a Friday afternoon figuring out how to get that test framework up and running.</p> <p>I believe this is a valuable lesson for all of us: When we advocate a best practice, or take advice from someone else, we need to assume there’s always a context. There’s a history of gradual improvement, a path taken from where we were to where we are. Others haven’t taken this path yet, or might end up on a different one. Best practices show a destination, but we can all put some more effort into putting up signposts along the way.</p> <h2 id="read-more">Read More</h2> <ul> <li><a href="/2013/10/pre-merge-code-reviews/">Pre-merge Code Reviews</a></li> <li><a href="/2014/03/small-controlled-experiments/">Small Uncontrolled Experiments</a></li> </ul> Type Safety and Money 2016-02-29T00:00:00+00:00 https://verraes.net/2016/02/type-safety-and-money <!-- photo credit http://s1153.photobucket.com/user/campbellchevelle/media/MoneyTree5_zpszzjjrpdw.jpg.html --> <p>Below is an attempt at illustrating a design/redesign process I went through at a client, who’s started refactoring the core systems their business depends on. Design is the part of software development that is the most messy, the hardest to fit into rules or well-defined processes. In fact, while writing this post, I tweeted:</p> <blockquote> <p><a href="https://twitter.com/mathiasverraes/status/704250689107709952">“There are surprisingly few software design books that recommend taking a walk, a shower, or a nap, as an important step.”</a></p> </blockquote> <p>None of the solutions offered below should be taken as truth. I may have already changed my mind on some of them by the time you read them.</p> <h2 id="history">History</h2> <p>In 2010, I wrote a small library with some Value Objects to represent money, and in 2011, I published a first version. It was a typical open source scratch-your-own-itch thing, that solved a problem with incorrect rounding and conversion in an e-commerce project I was working on. It seems to have scratched many itches, as it got quite popular.</p> <p>I’ve been a bad parent to the project lately. It’s in a <a href="https://github.com/moneyphp/money">new GitHub organisation</a> now, with @sagikazarmark as the new maintainer, who’s blowing new life into it.</p> <p>Over the years, a bunch of forks and variations appeared. I welcomed this, as many of them serve a need that my library didn’t. It freed me from having to support all possible use cases, and avoided feature bloat.</p> <h2 id="design-flaws">Design Flaws</h2> <p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/MoneyTree-small.jpg" alt="MoneyTree" /></p> <p>Apart from being incomplete, the library suffers from fundamental design flaws, some of which can’t be fixed without breaking it.</p> <ul> <li>It doesn’t support higher precision than the currency’s default precision.</li> <li>As a result of this, it always rounds, so if you chain operations, you end up with rounding errors.</li> <li>It doesn’t support blockchain currencies, or virtual, vendor-specific currencies, like Facebook Credits (now discontinued), casino chips, game credit…</li> <li>Having to enter values in integers was often impractical (eg €1 had to be written as 100.)</li> <li>Some currencies have mills (1/1000<sup>th</sup> of a unit) as their official division (although they are becoming rare). Bitcoin supports 10<sup>−3</sup> (a millibitcoin), 10<sup>−6</sup> (a microbitcoin or a bit), and 10<sup>−8</sup> (a satoshi). Reportedly, the community is considering to introduce even smaller divisions. The library doesn’t support any of this.</li> </ul> <h2 id="requirements">Requirements</h2> <p>This is what we expect from our model:</p> <ol> <li>We need to support about 10 currencies, possibly more in the future. When we do add new currencies, it is assumed that the in-house developers will add support in the code. There’s no UI for adding new currencies.</li> <li>All operations need to done with a precision of 8 decimals. This is a business decision.</li> <li>When showing amounts to users, or pass them along over an API, we always stick to the currency’s official division. In the case of Bitcoin, that does indeed mean we’re keeping a precision of 1 satoshi aka BTC 10<sup>−8</sup>.</li> <li>In some markets, our software will need to do specific compliance reporting.</li> <li>All internal reporting needs to be in EUR, no matter what the original currency was.</li> <li>We’re using some legacy and third-party systems, that also publish revenues to our internal reporting tool. Most of them only support the currency’s official division, and can’t deal with higher precision.</li> </ol> <h2 id="assumptions">Assumptions</h2> <p>To understand this post, I’m assuming you know:</p> <ul> <li>what a Value Object is, and why you’d want some;</li> <li>how you can make your objects immutable;</li> <li>why they <em>should</em> be immutable;</li> <li>how you can make operations safer, eg by disallowing $1+€1 without explicit conversion;</li> <li>why splitting €0.05 between two parties is not €0.025+€0.025 but €0.03+€0.02.</li> </ul> <h2 id="fowler-style">Fowler-style</h2> <p>Let’s design!</p> <p>Our first instinct is to have the <code class="language-plaintext highlighter-rouge">Money</code> object from <a href="http://amzn.to/1TN7Tq4">Fowler’s PoEAA</a>, consisting of a float<sup>*</sup> and a currency, which in turn could be an object as well. Both are immutable.</p> <p>(<sup> * </sup> You may not want to use the actual float type in your programming language for dealing with money, as you may run into issues with precision. Do your research.)</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/Money.png" alt="Money" /></p> <p><code class="language-plaintext highlighter-rouge">Currency</code> is an enum type supporting our 10 currencies (A simple assertion will do if your language doesn’t have enums). We can’t initialise the <code class="language-plaintext highlighter-rouge">Currency</code> object with any other value than our 10 currencies. It only uses the 3-letter ISO symbols, anything else is an error. That satisfies requirement 1.</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/MoneyFormatter.png" alt="MoneyFormatter" /></p> <p><code class="language-plaintext highlighter-rouge">Money</code>’s constructor can round floats with more than 8 decimals. We can add some operations, like <code class="language-plaintext highlighter-rouge">add(Money other) : Money</code> and <code class="language-plaintext highlighter-rouge">multiply(float operand) : Money</code>. They also round to 8 decimals if needed.</p> <p>Requirement 3 (showing pretty-printed, rounded values to the users), is clearly presentation logic. We should avoid muddying up our domain model for this. So we add a <code class="language-plaintext highlighter-rouge">MoneyFormatter</code> in the presentation layer, which takes a <code class="language-plaintext highlighter-rouge">Money</code> argument and returns a string, such as €5.00 or 5,00€, depending on local standards. Perhaps we even make a HTML widget or helper of sorts, so that our templates don’t need to worry about it.</p> <h2 id="itch">Itch</h2> <p>Our current design doesn’t satisfy the problem of exposing the rounded values over an API. Of course, we could <code class="language-plaintext highlighter-rouge">round(money.getAmount())</code>, but <code class="language-plaintext highlighter-rouge">money.round() : Money</code> seems more in line with our existing methods.</p> <p>This is where you should start feeling the itch. Have a close look at the type of this method. <code class="language-plaintext highlighter-rouge">money.round() : Money</code></p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">a</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="o">(</span><span class="mf">1.987654321</span><span class="o">,</span> <span class="n">eur</span><span class="o">)</span> <span class="c1">// The constructor rounds it own to 1.98765432</span> <span class="n">b</span> <span class="o">=</span> <span class="n">a</span><span class="o">.</span><span class="na">round</span><span class="o">()</span> <span class="c1">// round() rounds it up to 1. 99 and instantiates a new Money</span> <span class="c1">// Money’s constructor rounds it to 1.99000000</span> </code></pre></div></div> <p>Technically, most languages don’t distinguish between 1.99 and 1.99000000, but logically, there is an important nuance here. <code class="language-plaintext highlighter-rouge">b</code> is not just any <code class="language-plaintext highlighter-rouge">Money</code> , it is a fundamentally different type of money. Our design doesn’t make that distinction, and just mixes up money, whether it was rounded or not.</p> <p>Let’s make the implicit explicit. We can rename <code class="language-plaintext highlighter-rouge">Money</code> to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>, and add a new type called <code class="language-plaintext highlighter-rouge">RoundedMoney</code>. The latter always rounds to whatever the currency’s default division is.</p> <p>The <code class="language-plaintext highlighter-rouge">round()</code> method now becomes very simple:</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">PreciseMoney</span> <span class="o">{</span> <span class="n">round</span><span class="o">()</span> <span class="o">:</span> <span class="nc">RoundedMoney</span> <span class="o">{</span> <span class="k">return</span> <span class="k">new</span> <span class="nf">RoundedMoney</span><span class="o">(</span><span class="k">this</span><span class="o">.</span><span class="na">amount</span><span class="o">,</span> <span class="k">this</span><span class="o">.</span><span class="na">currency</span><span class="o">)</span> <span class="o">}</span> <span class="o">}</span> </code></pre></div></div> <p>The chief benefit is strong guarantees. We can now typehint against <code class="language-plaintext highlighter-rouge">PreciseMoney</code> in most of our domain model, and typehint against <code class="language-plaintext highlighter-rouge">RoundedMoney</code> where we explicitly want or need it.</p> <p>It’s easy to underestimate how valuable this style of granular types can be.</p> <ul> <li><strong>It’s defensive coding, against a whole category of bugs</strong>. Methods and their callers now have an explicit contract, about what kind of money they are talking about.</li> <li><strong>It’s declarative coding</strong>. Declarative style require a lot less tests. There’s no point in testing something that is obviously correct. Code with an explicit contract like this, is obviously correct. This is why proponents of strong static type systems like to talk about <em>Type Driven Development</em> as an alternative to <em>Test Driven Development</em>.</li> <li><strong>It communicates to different people working on the code, that we care specifically about the difference.</strong></li> <li><strong>It introduces a concept from the domain into the Ubiquitous Language and the model</strong>. Before we did that, it would likely go unnoticed that terms like “precision” were in fact part of our language. Co-evolving the language, the models, and the implementation, is central to Domain-Driven Design.</li> <li>This design also <strong>helps to apply the Interface Segregation Principle</strong>.</li> </ul> <h2 id="type-juggling">Type Juggling</h2> <p>We’re not building this as a generic library, so we don’t need to supply a “complete” API. We can (and we should) only add the methods that we are actually going to use. In our case, <code class="language-plaintext highlighter-rouge">PreciseMoney</code> has a <code class="language-plaintext highlighter-rouge">round()</code>, but <code class="language-plaintext highlighter-rouge">RoundedMoney</code> has no <code class="language-plaintext highlighter-rouge">toPrecise()</code>. In other words, we can cast <code class="language-plaintext highlighter-rouge">PreciseMoney</code> to <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, but we can’t cast <code class="language-plaintext highlighter-rouge">RoundedMoney</code> to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>. It’s a one way operation.</p> <p>There’s an elegance to that constraint. Once you round something, the precision is lost forever. The lack of <code class="language-plaintext highlighter-rouge">RoundedMoney.toPrecise()</code> fits our understanding of our domain.</p> <p>Perhaps we’ll eventually need some operations on <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, (although I’d rather avoid having them). We can choose some smart return types:</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">RoundedMoney</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="nc">RoundedMoney</span> <span class="n">other</span><span class="o">)</span> <span class="o">:</span> <span class="nc">RoundedMoney</span> <span class="nc">RoundedMoney</span><span class="o">.</span><span class="na">multiply</span><span class="o">(</span><span class="nc">Float</span> <span class="n">operand</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span> </code></pre></div></div> <p>Notice the different return types.</p> <p>Perhaps you’re writing some client code that wants to do multiplication, but doesn’t care about high precision. Chaining methods would suffice here:</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">someCash</span><span class="o">.</span><span class="na">multiply</span><span class="o">(</span><span class="mf">0.3333</span><span class="o">).</span><span class="na">round</span><span class="o">()</span> </code></pre></div></div> <p>Again, this style is very explicit. The compiler or type checker can protect us when we make mistakes against the types, such as passing <code class="language-plaintext highlighter-rouge">PreciseMoney</code> when <code class="language-plaintext highlighter-rouge">RoundedMoney</code> is required.</p> <h2 id="conversions">Conversions</h2> <p>Converting between different currencies depends on today’s exchange rates. We’ll need to fetch those from some third party web API. Or perhaps, another process is putting that information in a file or database somewhere. We don’t want to leak that kind of detail into our model, so we can have an Exchange interface, and have one or more implementations of our choice. The method on that interface that we’re interested in, takes a <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and a target <code class="language-plaintext highlighter-rouge">Currency</code>, and does the conversion.</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">interface</span> <span class="nc">Exchange</span> <span class="o">{</span> <span class="n">convert</span><span class="o">(</span><span class="nc">PreciseMoney</span> <span class="n">source</span><span class="o">,</span> <span class="nc">Currency</span> <span class="n">target</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span> <span class="o">}</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">Exchange</code> implementations will also need to deal with concerns such as caching today’s rates, to avoid unnecessary traffic on each call.</p> <p>If fetching + caching + converting sounds like a lot of responsibility for one service, that’s because it is (and this one is actually still very clean as compared to most service classes I see in actual code). These kinds of service classes are fundamentally procedural code wrapped in an object. Even though leakage is reduced thanks to the Exchange interface, the rest of our code still needs to depend on that interface. This makes testing harder, as all those tests will need to mock <code class="language-plaintext highlighter-rouge">Exchange</code>. And finally, (although I agree it’s subtle here), all implementations of <code class="language-plaintext highlighter-rouge">Exchange</code> will need to duplicate the actual conversion, or somehow share code. A heuristic that helps, is to figure out if we can somehow isolate the side-effectful code (fetching, caching) from the domain logic (conversion).</p> <p>You guessed it, we need to identify a missing concept. Instead of having the <code class="language-plaintext highlighter-rouge">Exchange</code> do the conversion, we can make it return a <code class="language-plaintext highlighter-rouge">ConversionRate</code> instead. This is a Value Object that represents a source <code class="language-plaintext highlighter-rouge">Currency</code>, a target <code class="language-plaintext highlighter-rouge">Currency</code>, and a factor (a float). Value Objects attract behaviour, and in this case, it’s the <code class="language-plaintext highlighter-rouge">ConversionRate</code> object that becomes the natural place for doing the actual calculation.</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">interface</span> <span class="nc">Exchange</span> <span class="o">{</span> <span class="n">getRate</span><span class="o">(</span><span class="nc">Currency</span> <span class="n">source</span><span class="o">,</span> <span class="nc">Currency</span> <span class="n">target</span><span class="o">)</span> <span class="o">::</span> <span class="nc">ConversionRate</span> <span class="o">}</span> <span class="nc">ConversionRate</span> <span class="o">{</span> <span class="n">convert</span><span class="o">(</span><span class="nc">PreciseMoney</span> <span class="n">source</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span> <span class="o">}</span> </code></pre></div></div> <p>The <code class="language-plaintext highlighter-rouge">convert</code> method makes sure that we never accidentally convert USD to EUR using the rate that was actually meant for GBP to BTC. All it needs to do is throw an exception if the arguments have the wrong currency.</p> <p>The other cool thing here is that we don’t need to pass around <code class="language-plaintext highlighter-rouge">Exchange</code>. Instead, we pass around the much smaller, simpler, <code class="language-plaintext highlighter-rouge">ConversionRate</code> objects. They are, once again, more composable. Already the possibilities for reuse become obvious: for example, a transaction log can store a copy of the <code class="language-plaintext highlighter-rouge">ConversionRate</code> instance that was used for a conversion, so you get accountability.</p> <h2 id="simpler-elements">Simpler Elements</h2> <p>An <code class="language-plaintext highlighter-rouge">Exchange</code> is now something that represents the collection of <code class="language-plaintext highlighter-rouge">ConversionRate</code> objects, and provides access (and filters) on that collection. Sounds familiar? This is really just the <em>Repository</em> pattern! Repositories are not just for Entities or Aggregates, but for all domain objects, including Value Objects.</p> <p>(We could rename <code class="language-plaintext highlighter-rouge">Exchange</code> to <code class="language-plaintext highlighter-rouge">ConversionRateRepository</code>, which would make the pattern more explicit, but remove a name from our Ubiquitous Language. It’s helpful in some contexts. Personally, I’d be more in favour of keeping the name. We can annotate <code class="language-plaintext highlighter-rouge">Exchange</code> as a @Repository, or have it implement a marker interface called <code class="language-plaintext highlighter-rouge">Repository</code>.)</p> <p>We split up our procedural original <code class="language-plaintext highlighter-rouge">Exchange</code> service into the two simpler patterns, Repository and Value Object. Notice how we have at no point removed any essential complexity, and yet each element, each object, is very simple in its own right. To understand this code, the only pattern knowledge a junior developer would need, is in a few pages of chapters 5 &amp; 6 of <a href="http://amzn.to/1LrjmZF">Domain-Driven Design</a>.</p> <h2 id="currency-type-safety">Currency Type Safety</h2> <p>We still have some issues. Remember that some currencies have a division of 1/00, 1/1000, or 1/100000000. <code class="language-plaintext highlighter-rouge">RoundedMoney</code> needs to support this, in our case for 10 currencies. The constructor is starting to look pretty ugly:</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">switch</span><span class="o">(</span><span class="n">currency</span><span class="o">)</span> <span class="k">case</span> <span class="nl">EUR:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span> <span class="k">case</span> <span class="nl">USD:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span> <span class="k">case</span> <span class="nl">BTC:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">8</span><span class="o">)</span> <span class="n">etc</span> </code></pre></div></div> <p>Every time we need to support a new currency, we need to add code here, and possibly in other places in our system. Not a serious issue, but not ideal. A switch statement is often a smell for missing types. (Yes, we could replace this with a map, but it’s really the same thing. And a map wouldn’t fit if we’d be dealing with more interesting behavioural differences between currencies than rounding.)</p> <p>What happens if we make <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code> abstracts or interfaces, and factor the variation out into subtypes for each currency?</p> <p><img src="/img/posts/2016-02-29-type-safety-and-money/FactorOutCurrency.png" alt="Factoring out currency" /></p> <p>Each of the <code class="language-plaintext highlighter-rouge">PreciseEUR</code>, <code class="language-plaintext highlighter-rouge">PreciseBTC</code>, <code class="language-plaintext highlighter-rouge">RoundedEUR</code>, <code class="language-plaintext highlighter-rouge">RoundedBTC</code> etc classes have very local knowledge about how they go about their business, such as the rounding switch example above.</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">RoundedEUR</span> <span class="o">{</span> <span class="nc">RoundedEUR</span> <span class="o">(</span><span class="n">amount</span><span class="o">)</span> <span class="o">{</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span> <span class="o">}</span> <span class="o">}</span> </code></pre></div></div> <p>Again, we can put the type system to work here. Remember that, per requirement 5, our reporting needs to be in EUR? We can now typehint for that, making it impossible to pass any other currency into our reporting. Similarly, the different compliance reporting strategies we need for requirement 4 can each be limited to the currencies they support.</p> <p>Sure, we get a bit of class explosion. But so what? It’s much preferable to long methods.</p> <h2 id="minimalist-interfaces">Minimalist Interfaces</h2> <p>A benefit of having lots of small classes, is that we can get rid of a lot of code. Perhaps our main Bounded Context deals with the 10 <code class="language-plaintext highlighter-rouge">PreciseXYZ</code> types, but our Reporting Bounded Context only supports <code class="language-plaintext highlighter-rouge">RoundedEUR</code>. That means there’s no need to support <code class="language-plaintext highlighter-rouge">RoundedUSD</code> etc, as we are not using it. This also implies that we don’t need <code class="language-plaintext highlighter-rouge">round()</code> methods on any of the <code class="language-plaintext highlighter-rouge">PreciseXYZ</code> classes, apart from EUR. Less code means less boilerplate, less bugs, less tests, and less maintenance.</p> <p>Not supporting a way back from <code class="language-plaintext highlighter-rouge">RoundedEUR</code> to <code class="language-plaintext highlighter-rouge">PreciseEUR</code> is another example of a minimalist interface. Don’t build behaviours that you don’t need or want to discourage.</p> <h2 id="single-responsibility">Single Responsibility</h2> <p>Another benefit of these small, ultra-single-purpose classes, is that they very rarely need to change. This is Robert C. Martin’s heuristic for the Single Responsibility Principle:</p> <blockquote> <p>“A class should have only one reason to change.”</p> </blockquote> <p>A good design allows you to easily add or remove elements, or change the composition of the elements, but rarely requires you to actually change existing code. This in turn leads to fewer bugs and less work.</p> <h2 id="parent-interface">Parent interface?</h2> <p>You may have noticed that in the current design, I have no <code class="language-plaintext highlighter-rouge">Money</code> interface at the top of the object graph. Aren’t <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code> both a kind of <code class="language-plaintext highlighter-rouge">Money</code>? Don’t they share a lot of methods?</p> <p>If we are trying to build a model inspired by the real-world, this would make sense. However, we shouldn’t judge our models by how well they fit our hierarchical categorisations. We judge our models by usefulness. A top-level <code class="language-plaintext highlighter-rouge">Money</code> interface adds no value at all; in fact it takes away value.</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/BadMoney.png" alt="Bad Money" /></p> <p>This may be a bit counterintuitive. <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, although somewhat related, are fundamentally different types. We’ve designed our model for clarity, for the guarantee that we don’t mix up rounded and precise values. By allowing client code the typehint for the generic Money, we’ve taken away that clarity. There’s now no way of knowing which we’re getting. All responsibility for passing the correct type is now back in the hands of the caller.</p> <h2 id="overdesign">Overdesign</h2> <p>Is this overdesigned? Perhaps, depending on your context. When you only want to display some prices, it’s likely overkill. With code that decides about huge amounts of money, a design like this helps me sleep at night.</p> <p>There’s a hint in the language. We don’t say “this thing is designed”, we say “this thing is designed for <strong>that purpose</strong>”. Overdesign (and underdesign) simply mean “not designed for purpose”, or “designed for a different purpose than the current one”. The model presented here is designed for the purpose of high precision and confidence in our operations with money.</p> <p>Often, different aspects of our system have different requirements. This is precisely what Bounded Contexts are for: allow us to reason about our system as a a number of cooperating models, as opposed to one unified model. Perhaps our Product Catalog needs a really simple model for money, because it doesn’t really do anything other than displaying prices. Sales and Reporting on the other hand might benefit from our more intricate design.</p> <p><img src="/img/posts/2016-02-29-type-safety-and-money/Hexagonal.png" alt="Hexagonal" /></p> <h2 id="practicality">Practicality</h2> <p>It’s important to distinguish between “overdesigned” and “impractical”. The solution to overdesign is to remove elements, make it simpler until it just fits. The solution to impracticality, can be to add more abstractions and shortcuts.</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">new</span> <span class="nc">PreciseMoney</span><span class="o">(</span><span class="mf">5.00</span><span class="o">,</span> <span class="k">new</span> <span class="nc">Currency</span><span class="o">(</span><span class="no">EUR</span><span class="o">))</span> </code></pre></div></div> <p>is a handful to write. There’s no reason why you shouldn’t add a static factory method, like</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">PreciseMoney</span><span class="o">.</span><span class="na">EUR</span><span class="o">(</span><span class="mf">5.00</span><span class="o">)</span> </code></pre></div></div> <p>Or just a naked function if your language supports it</p> <div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">EUR</span><span class="o">(</span><span class="mf">5.00</span><span class="o">)</span> </code></pre></div></div> <p>Or a unicode function</p> <p><code class="language-plaintext highlighter-rouge">€(5.00)</code></p> <p>(Unfortunately, $ is used in many languages to mean all kinds of things, none of which are USD. Be creative!)</p> <h2 id="defensive-model">Defensive model</h2> <p>There is of course no way to defend your code against bad coders. Unless you’re doing <a href="/2013/10/pre-merge-code-reviews/">code reviews</a>, someone could change the Value Objects to be mutable, and delete the tests. (Funny story: I was once told that the new developer hired to replace me after I left a job, replaced all private methods by public ones, “because it’s more convenient”.)</p> <p>I do believe though that good design communicates intent. No matter the level of the developers using your code, if they see two types for rounded and precise money, chances are they are at least going to consider that perhaps we can’t just mix the two in operations without thinking about it.</p> <h2 id="going-further">Going Further</h2> <p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/Price.png" alt="Price" /></p> <p>Value Objects are the ultimate composable objects, so we can keep looking for implicit concepts that we can make explicit. For example, we can naively use our money type to represent prices. But what if a price is not so simple? Maybe <code class="language-plaintext highlighter-rouge">Price</code> is composed of a <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and a <code class="language-plaintext highlighter-rouge">VATRate</code> or <code class="language-plaintext highlighter-rouge">Tax</code> object of sorts. Maybe a <code class="language-plaintext highlighter-rouge">ProductPrice</code> is composed multiple <code class="language-plaintext highlighter-rouge">Price</code> objects for different amounts, for example if you offer a discount when buying in bulk.</p> <p>Value Objects make it easy to build abstractions that can handle lots of complexity at a very small cognitive cost to the developer using it. Finding the abstractions can be costly and hard, but spending the effort here usually impacts code quality and maintainability so much in the long run, that it’s very often worth it.</p> <h2 id="timezone-type-safety">Timezone Type Safety</h2> <p>The concept of using the type system to reduce a whole classes of errors in code without resorting to excessive tests, can be applied to many domains. Recently, I was late for a talk I was doing at a meetup in London, because a certain website’s “Export to Google Calendar” feature didn’t take timezones into account. Timezone bugs are not only a nuisance to a regular traveler like myself, but can have dire consequences.</p> <p>Haskell’s Time library has separate types for <code class="language-plaintext highlighter-rouge">UTCTime</code> and <code class="language-plaintext highlighter-rouge">ZonedTime</code>, and offers functions to convert between them. This gives guarantees from the compiler that a programmer can not easily mix them up, or implicitly converts between them. Ideally, your code should use <code class="language-plaintext highlighter-rouge">UTCTime</code> everywhere, and only convert to and from <code class="language-plaintext highlighter-rouge">ZonedTime</code> when human users are involved. It should be easy enough to build something like this in object oriented languages.</p> <h2 id="conclusion">Conclusion</h2> <p>In many environments, dealing with money is too critical to be regarded as a Generic Subdomain. Different projects have different needs and expectations of how money will be handled. If money matters, you need to build a model that fits your specific problem space like a glove. Some of the tools in our belt are small composable building blocks, making the implicit explicit, and using the type system to our advantage. You might end up with something entirely different.</p> <p><em>All good design is redesign.</em></p> Domain-Driven Design workshops 2015-10-20T00:00:00+00:00 https://verraes.net/2015/10/domain-driven-design-workshops <p>We’re bringing you some exciting workshops in the coming months. Register today to ensure your spot!</p> <h2 id="experiencing-domain-driven-design">Experiencing Domain-Driven Design</h2> <p>The full three day advanced workshop, with lots of hands on modelling, Event Storming, Context Mapping, CQRS/ES, heuristics, and much more.</p> <ul> <li><a href="/workshops">Experiencing Domain-Driven Design</a> Brussels (Leuven), Belgium, March 2-4 2016</li> <li><a href="/workshops">Experiencing Domain-Driven Design</a> Berlin, Germany, March 23-25 2016</li> <li><a href="/workshops">Experiencing Domain-Driven Design</a> Paris, France, date TBD</li> </ul> <h2 id="temporal-modelling">Temporal Modelling</h2> <p>New one day workshop, with part theory about Event Sourcing and CQRS, and part practical Event Storming.</p> <ul> <li><del><a href="http://dddeurope.com">Temporal Modelling</a> Brussels, Belgium, January 26 2016</del> Sold out</li> </ul> <h2 id="domain-driven-design-europe">Domain-Driven Design Europe</h2> <p>Pre-conference workshops by leading practitioners.</p> <ul> <li><del><a href="http://dddeurope.com">Strategic Domain-Driven Design with <strong>Eric Evans</strong></a> Brussels, Belgium, January 26-27 2016</del> Sold out</li> <li><a href="http://dddeurope.com">DCI Systems on a DDD Foundation with <strong>Jim Coplien</strong></a> Brussels, Belgium, January 27 2016</li> <li><a href="http://dddeurope.com">Critical Systems that Go Boom with <strong>Brian L. Troutwine</strong></a> Brussels, Belgium, January 27 2016</li> <li><a href="http://dddeurope.com">Essential Domain-Driven Design with <strong>Paul Rayner</strong></a> Brussels, Belgium, January 26-27 2016</li> </ul> <h2 id="ddd-in-php">DDD in PHP</h2> <p>This is an introductory training, teaching you the tactical design patterns and the modelling insights to get started applying DDD in your PHP projects.</p> <ul> <li><a href="http://www.phpcon.pl/2015/en/agenda">DDD in PHP</a> Poland, November 13 2015</li> <li><a href="http://laracon.eu/2015/workshop/domain-driven-design-in-php-mathias-verraes">DDD in PHP</a> Amsterdam, Netherlands, December 14, 2015</li> </ul> <p><a href="/workshops/newsletter/">Register for the workshops newsletter to receive notifications</a>.</p> Towards Modelling Processes 2015-05-22T00:00:00+00:00 https://verraes.net/2015/05/towards-modelling-processes <p>The video &amp; slides for my keynote at NCrafts 2015 in Paris.</p> <iframe src="https://player.vimeo.com/video/131658140" width="640" height="360" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe> <p><a href="https://vimeo.com/131658140"><small>Watch the video on Vimeo</small></a></p> <script async="" class="speakerdeck-embed" data-id="ff4ed4d875c246f3996807942a09df4d" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><a href="https://speakerdeck.com/mathiasverraes/towards-modelling-processes"><small>See the slides on Speakerdeck</small></a></p> <p>I should probably blog about it… just pester me until I do :-)</p> <h2 id="abstract">Abstract</h2> <p>Processes are at the heart of how many businesses operate. The processes are governed by policies, agreed upon with contracts, and guided by documents. In software systems on the other hand, we often bury the processes. There’s a domain model for sure, but it’s all about “things” and “actions”. Let’s explore how Event Sourcing allows us to explicitly model the effects of changes over time.</p> Context Mapping: Life Expectancy 2015-04-26T00:00:00+00:00 https://verraes.net/2015/04/context-mapping-life-expectancy <p>I was at a client recently, a large insurance and banking company. The organization is over 200 years old. Some of the software is only slightly younger: they first started automating some 40 years ago. Some of the people I spoke with, have been around almost as long. They have, depending on who you’re asking, 600 applications in production. The majority is written in Java and COBOL. For them, dealing with legacy is not just about removing some code, it’s about phasing out entire systems.</p> <p>We did some Context Mapping, but due to time constraints, it was mostly as an exercise, rather than a serious attempt to make sense of it all. Yet already some interesting ideas arose, so I thought it might be valuable to describe them here. If there’s any prior art about these patterns, please let me know.</p> <h2 id="reuse---buy---rebuild">Reuse - Buy - Rebuild</h2> <p>Many systems serve a common need, so a lot of other systems depend on them. When making strategic decisions about adding a new system, the company has a mantra of “Reuse - Buy - Build”. It means of course that reusing an existing system is preferred to buying a new one, which in turn is preferred to building a system. A Context Map is good place to make these forces explicit, for example through some color code or symbol.</p> <h2 id="life-expectancy">Life Expectancy</h2> <p>When reusing an existing system, there is a risk of creating a lot of rework: when we phase out an old system, we need to rewrite the integration to work with the newer system. We started indicating the life expectancy of each Bounded Context on the map. It helped us to spot whether making a new integration with an old system still makes sense. Knowing that in reality, large organizations can be slow, we even went as far as to write down both the “official” life expectancy of a system (as decided by the powers that be), and the “realistic” expectation (often 2-4 times as long).</p> <h2 id="sponsorship">Sponsorship</h2> <p>You apply Context Mapping in part to figure out how the company’s communication structures can influence the success or failure of a project. A system is usually sponsored by one specific department, and in some cases by a number of different departments. It makes a lot of sense to bring the sponsors or owners of each system into the picture. Knowing who’s paying for what (or not) can make all the difference, when you want to determine whether a project is feasible.</p> <h2 id="tragedy-of-the-commons">Tragedy of the Commons</h2> <p>Different departments often come together to solve a common problem. With the best of intentions, they jointly build a system to address that. (This pattern is called a “Partnership” in the <a href="http://www.amazon.com/gp/product/0321125215/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0321125215&amp;linkCode=as2&amp;tag=verraesnet-20&amp;linkId=DW5WV5IVDVK7SMFN">Domain-Driven Design book</a>.) Over time, the effect of the Tragedy of the Commons kicks in. More parties join the Partnership, they all adapt the system to their own needs, but no one invests in keeping the system maintainable. Quality declines. By indicating which systems suffer from this effect, we can make decisions about phasing those systems out, or forming a standalone team to take ownership. The Partnership dissolves and is replaced by a Customer/Supplier Relationship (another pattern from the book).</p> <h2 id="visualization">Visualization</h2> <p>I’m moving away from using whiteboards for Context Mapping. They are generally too small, and quickly become unrefactorable anyway. Find a large wall or glass surface, or move some tables together. Use stickies, agree on a color code, and buy some sticky shapes. <a href="http://www.post-it.com/wps/portal/3M/en_US/PostItNA/Home/Products/~/Post-it-Arrow-Flags-Bright-Colors-1-2-in-Wide-100-On-the-Go-Dispenser?N=4327+3294647258&amp;rt=rud">Arrows</a> are especially useful of course.</p> <p>Trying to get everything on a single Context Map is impossible. A Context Map is a model, and, as any model, it’s a problem solving tool. We judge a model by its usefulness. Of course, when you’re new to the organization, or you’ve never visualized it, your problem is a lack of insight. When the goal is discovery, I don’t mind if we end up with an unreadable model. Throw the map away afterwards, because the new shared understanding is the greatest artifact here. Over time, you might grow Context Maps that are stable. Only then you should treat the map as a document that you keep around.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/01/bandwidth-and-context-mapping/">Bandwidth and Context Mapping</a></li> <li><a href="/2014/02/buzzword-free-bounded-contexts/">Buzzword-free Bounded Contexts</a></li> </ul> Event Storming, Storytelling, Visualisations 2015-03-14T00:00:00+00:00 https://verraes.net/2015/03/event-storming-storytelling-visualisations <h2 id="visual-storytelling-through-event-storming">Visual storytelling through Event Storming</h2> <p>Last week, my client’s Chief Product Owner announced to the team they would introduce a set of new requirements using Event Storming, “as Mathias has taught us”. (I’ve visited there regularly for the past half year or so). The session took about ten minutes, and included no brainstorming or collaboration of any kind. The only thing it had in common with Event Storming, was that the PO used colorful stickies.</p> <p>I was of course sceptical at first, but it turned out to be quite interesting in its own right, as a presentation technique. The Chief PO and his team were well prepared. They had researched the regulations they had to comply with to enter a new market, and third party services they’d need to integrate with. They had done Event Storming (or something similar) amongst themselves, cleaned up the resulting artifact, and ordered and collected the stickies. During the short session with the development team, the Chief PO told the story, while the assistants supported it by adding stickies to wall as the story evolved. It was not a linear story. You could easily understand were a new branch in the story began, because the PO’s attached new stickies at that location.</p> <p>Surely, for the developers, it didn’t create the collaborative deep learning experience that Event Storming usually does. They were only spectators. But this style of visually supported presentation, was definitely much better than some of the alternatives, like purely oral storytelling, or a written set of requirements.</p> <p><img src="/img/posts/2015-03-14-event-storming-storytelling-visualisations/storytelling.jpg" alt="Storytelling" /></p> <h2 id="model-storming-visualisations">Model Storming visualisations</h2> <p>Event Storming as described by Alberto Brandolini is rather prescriptive, and that’s a good thing. A fixed set of steps allows you to get started on exploring and modelling from the first minute. There’s no long introductions, no discussions on what the right approach is, and no analysis paralysis.</p> <p>The first few steps go something like this:</p> <ul> <li>Write down a Domain Event on a sticky (preferably something that happens at the end, like “Bill was paid”).</li> <li>Ask what happened before that (“Bill was sent”) and before that, until you get to the beginning.</li> <li>Ask what caused the events to happen. Those are usually Commands (aka user intentions), other events, or simply the passage of time.</li> <li>Ask for what reason a Command can fail, and find the rules that impact the outcome of the Commands. This pushes you to discover alternative scenarios, missing events, connections between distant events…</li> </ul> <p>The list goes on for a bit (I should probably do a proper writeup of how I run Event Storming sessions). The point I want to make here though, is that it’s ok to deviate. When a team has some maturity with the domain, more complex scenarios might emerge that are not easily captured with basic Event Storming. At this point, you should invest some energy in coming with new visualisations, new elements that capture the essence of what you’re trying to achieve. This brainstorming about the modelling itself, is crucial to evolving deeper insight.</p> <p>Describing the experience of reaching a breakthrough, is hard. Good models are always obvious in retrospect. As an example, at this client, we were cutting our teeth on a complex customer accreditation process that varies highly in different markets. To add to the complexity, the process varies depending on whether the customer is using the site, talking to customer support by phone, or visiting an office. And finally, a process that started through one medium, could be finished through another.</p> <p>For our first model, we focused too much on the design we thought we wanted: Reusable Specification and Policy objects, Adapters and Strategies, with the intent of injecting the behaviour for the different markets into the process. The next morning, while attempting a PoC implementation, we realized we had painted ourselves into a corner. Event Storming to the rescue. Based on nothing but a gut feeling, I proposed we started with Commands instead of Events though. Then we evolved that in to some sort of matrix, with one swim lane per Market x Medium combo. Finally it hit us that the implementation shouldn’t start from a single accreditation process where we injected strategies. Our visual model suggested that we needed multiple processes, implemented independently, and only share elements between those processes if it absolutely made sense. I don’t have a good picture of the resulting model, but it wouldn’t have made sense anyway. As with all brainstorming, you had to be there.</p> <p>I don’t believe we would have come to this conclusion so fast in any other way. As developers, we are so often blinded by the way the requirements are described, and our own propensity to design according to preconceived notions of what a good design must look like. Find the elements that complicate a problem, find a visualisation, put it front and center, and brainstorm from there. Without visualisation, you spend too much energy synchronizing each other’s mental models, usually unsuccessfully.</p> <h2 id="read-more">Read More</h2> <ul> <li><a href="/2014/11/domain-events/">Domain Events</a></li> <li><a href="/2013/08/facilitating-event-storming/">Notes on facilitating Event Storming sessions</a></li> <li><a href="http://ziobrando.blogspot.be/search/label/EventStorming">Posts tagged with Event Storming on Alberto Brandolini’s blog</a></li> <li><a href="http://sch3lp.github.io/2014/07/12/event-storming-exercise/">Tim Schraepen’s Event Storming experience report</a></li> <li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li> </ul> Form, Command, and Model Validation 2015-02-16T00:00:00+00:00 https://verraes.net/2015/02/form-command-model-validation <p>Many of the frameworks I’ve worked with, promise to separate responsibilities with MVC. In practice, they end up coupling everything to everything. The forms are coupled to the models, and there’s a grand unified validation layer. This may be convenient at first, but it breaks down for larger systems, and creates headaches when having to support multiple clients. My approach is to clearly separate the validation for the form itself, from the Command validation and the model validation.</p> <h2 id="commands">Commands</h2> <p><img style="float:right;margin-left: 10px" src="/img/posts/2015-02-16-form-command-model-validation/smokesignals-small.jpg" alt="Smoke Signals" /></p> <p><a href="/2015/01/messaging-flavours/">A Command is a message that represents the intention of the user</a>. Command validation happens after the form is submitted and before the Command is passed to the model. The question to answer here is “Does this look like a valid Foo Command?” It guarantees that the “birthday” field contains an actual date, or that “zipcode” looks like a zipcode. It doesn’t guarantee set constraints, like username uniqueness. In other words, you validate the message, not whether you should execute the message.</p> <p>The Command object should use Value Objects. They guarantee their own consistency, so the Command delegates validation to them. Note that we’re not trying to inform the user of validation errors. We simply throw exceptions. The assumption here is that either the form prevents malformed values, or the user is trying to bypass form validation. We don’t return friendly error messages to attackers.</p> <h2 id="model">Model</h2> <p>The model is the “official” source of truth. It doesn’t worry whether the “birthday” field looks like a date and whether the “zipcode” looks like a zipcode, because it can trust the Command. The model is responsible for protecting its own invariants, business rules, and constraints. It can reliably<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> prevent duplicate usernames, account overdrafts, and all kinds of other business concerns. It fails hard when executing the Command would violate those concerns.</p> <p>The Commands, the Value Objects, and the objects that make up the model, are decoupled from the framework and the persistence. We can effectively transport them to a different context, such as a different framework. They are independently testable with unit tests.</p> <h2 id="forms">Forms</h2> <p>Form validation is a UX concern: you add it to help the user fill in the values quickly and correctly. Nowadays, users expect forms to give them instant visual feedback, and rightly so. Client side is the only option. Of course, this client side validation can call back to the server to validate things that the client cannot know, such as “is this username available?”.</p> <p>By the time the submit button is pressed, some of the data may already be stale, for example another user might have registered the same name. The form validation stops most of the cases, but it can never be relied on. Not only because it lags behind on the server, but also because it is easy to spoof requests and bypass the validation.</p> <p>I’m not aiming to make the form validation as complete as possible, at least not initially. Rather, I try to look at it from the point of view of the user. The things that they are most likely to get wrong, are the things I try to guide them into doing correctly. Ideally, I watch or monitor how a real life user fills in the form, keep track of the most frequent model errors they trigger, and focus on preventing those first.</p> <h2 id="conclusion">Conclusion</h2> <p><strong>The fundamental idea here is that “helping the user”, “validating the message”, and “protecting the model’s consistency”, are three completely independent concerns.</strong></p> <p>Yes, you will end up with something that looks like duplication. For example, both the form and the model will have a concept of “account overdraft” and what it means. These are however different concepts, because the goal is different: the model protects business rules, the form helps the end user. I feel that this separation of responsibilities <a href="/2014/08/dry-is-about-knowledge/">trumps the minor cost of duplication</a>.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li> <li><a href="/2014/08/dry-is-about-knowledge/">DRY is about Knowledge</a></li> <li><a href="/2013/04/decoupling-symfony2-forms-from-entities/">Decoupling (Symfony2) Forms from Entities</a></li> </ul> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:1" role="doc-endnote"> <p>In some cases, I might choose eventual consistency: Accept the small risk that usernames might clash, detect the conflicts after the events are persisted, and deal with them somehow. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div> Messaging Flavours 2015-01-09T00:00:00+00:00 https://verraes.net/2015/01/messaging-flavours <p>A message is a unit of communication between systems. There are three flavours of messages: <strong>informational, interrogatory, and imperative</strong>.</p> <h2 id="imperative-messages">Imperative Messages</h2> <p>Imperative messages represent the sender’s intention. They instruct the receiver to perform an action or make a change. Do this, change that. We can model these messages as Commands. More on that in a later post.</p> <h2 id="interrogatory-messages">Interrogatory Messages</h2> <p>Interrogatory messages ask the system about it’s current state, or usually a subset of that state. You know them as Queries. If you’ve done any web programming, you’ve written Queries as:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">GET</span> <span class="o">/</span><span class="n">pupils</span><span class="o">?</span><span class="n">bornBefore</span><span class="o">=</span><span class="mi">1997</span><span class="o">-</span><span class="mo">01</span><span class="o">-</span><span class="mi">09</span> </code></pre></div></div> <p>If you’ve ever talked to an SQL database, you’ve written Queries as:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">SELECT</span> <span class="o">*</span> <span class="no">FROM</span> <span class="n">pupils</span> <span class="no">WHERE</span> <span class="n">date_of_birth</span> <span class="o">&lt;</span> <span class="s2">"1997-01-09"</span> </code></pre></div></div> <p>If you’ve done any Object-Oriented Programming, you know that everything is an object, and so is a Query</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">UserQuery</span> <span class="p">{</span> <span class="n">where</span> <span class="o">=</span> <span class="p">[{</span><span class="n">field</span><span class="o">=</span><span class="s2">"date_of_birth"</span><span class="p">,</span> <span class="n">operator</span> <span class="o">=</span> <span class="s2">"&lt;"</span><span class="p">,</span> <span class="n">value</span> <span class="o">=</span> <span class="s2">"1997-01-09"</span><span class="p">}]</span> <span class="mf">...</span> <span class="p">}</span> </code></pre></div></div> <p>If you’re into Domain-Driven Design, you look for the Ubiquitous Language, you dig into the domain, and you make the implicit explicit. We can make a Query message that communicates its purpose (Intention-Revealing Interfaces)</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">LuckyPupilsWhoDontNeedTheirParentsToSignTheirReportsAnymore</span> <span class="p">{</span> <span class="n">asOf</span> <span class="o">=</span> <span class="nf">now</span><span class="p">()</span> <span class="n">age_of_consent</span> <span class="o">=</span> <span class="mi">18</span> <span class="p">}</span> </code></pre></div></div> <p>This is what is meant by <strong>Intention Revealing Interfaces</strong> in the <a href="http://www.amazon.com/gp/product/0321125215/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0321125215&amp;linkCode=as2&amp;tag=verraesnet-20&amp;linkId=JK4IHA2DQIYZB5ZR">blue book</a>. The name <code class="language-plaintext highlighter-rouge">LuckyPupilsWhoDontNeedTheirParentsToSignTheirReportsAnymore</code> is obviously exaggerated to make a point. You’ll have to admit that at least now, you’ve learned something about the domain that was tricky to infer from the previous examples.</p> <h2 id="informational-messages">Informational Messages</h2> <p><img style="float:right;margin-left: 10px" src="/img/posts/2015-01-09-messaging-flavours/icecream-small.jpg" alt="Ice Cream Flavours" /></p> <p>Informational messages are sent by a system that wants to <strong>communicate something about itself</strong> to the outside world. It does this not to achieve a goal (as with imperative messages) or to learn about another system’s state (interrogatory). It provides this service for the benefit of other systems. It’s the altruistic pattern of the distributed world! (It does however have some sort of contract with it’s receivers.) It can notify an interested party directly, or through publish/subscribe (the subscribers are not hardcoded, creating better decoupling), or through broadcasting (“This is my message and I don’t care who hears it!”).</p> <p>Events are one flavour of informational messages, where the sender tells others what has happened. <a href="/2014/11/domain-events/">I’ve written about Domain Events extensively</a>. Alternatively, instead of events, the sender might simply communicate its current state. Say we have a thermometer, that emits an informational message as an event:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">TemperatureHasChanged</span> <span class="p">{</span> <span class="n">temperature</span><span class="o">:</span> <span class="nc">Temperature</span> <span class="n">at</span><span class="o">:</span> <span class="nc">DateTime</span> <span class="p">}</span> </code></pre></div></div> <p>If the temperature changes at 08:00 and stays stable until 17:00, we have a bit of a problem. If I subscribe at 09:00, I have to wait 8 hours before I know the state. I can query the thermometer (interrogatory message), but this complicates both the sender and the receiver. The sender can resend the last event to any new subscribers, but that still means our thermometer needs to be stateful.</p> <p>More realistically (disclaimer: I’ve never actually interfaced with a thermometer, so I don’t know), the thermometer is pretty dumb and simply communicates its current measurement every second:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">CurrentTemperature</span> <span class="p">{</span> <span class="n">temperature</span><span class="o">:</span> <span class="nc">Temperature</span> <span class="n">at</span><span class="o">:</span> <span class="nc">DateTime</span> <span class="p">}</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">TemperatureHasChanged</code> is an event-flavoured informational message, <code class="language-plaintext highlighter-rouge">CurrentTemperature</code> is a state-flavoured informational message. My gut feeling is that, when faced with this, I would still model state-flavoured messages as event-flavoured, by tweaking the language a little bit:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">TemperatureWasMeasured</span> <span class="p">{</span> <span class="n">temperature</span><span class="o">:</span> <span class="nc">Temperature</span> <span class="n">at</span><span class="o">:</span> <span class="nc">DateTime</span> <span class="p">}</span> </code></pre></div></div> <h2 id="mixing-flavours">Mixing Flavours</h2> <p>Given this categorisation, it follows from the Single Responsibility Principle that:</p> <ul> <li>A Command should not return state. (Replying with an ACK/NACK is fine though). If you want to know the resulting state of the receiver after sending a Command, you can either ask for it (Query) or wait for a message saying what has happened (Event).</li> <li>An Event should not be coupled to the expectation that something will change in the receiver. The sender shouldn’t know who’s listening and what they’re doing as a result.</li> <li>A Query shouldn’t change the state of the receiver.</li> </ul> <h2 id="read-more">Read More</h2> <ul> <li><a href="/2014/11/domain-events/">Domain Events</a></li> <li><a href="/2015/02/form-command-model-validation/">Form, Command, and Model Validation</a></li> </ul> Economy of Tests 2015-01-05T00:00:00+00:00 https://verraes.net/2015/01/economy-of-tests <p><em>Joint post with <a href="https://twitter.com/everzet">Konstantin Kudryashov</a>.</em></p> <p>A common complaint with teams that first try their hand at automated testing, is that it is hard, costly, and not worth the effort. On the other hand, supporters say that <a href="https://twitter.com/mathiasverraes/status/234641929260908544">it saves them time</a>. In this post, we’ll try to add some nuance to the discussion. We’ll take a look at how different levels of automated testing affect the cost of development, both in the short and the long term, for both greenfield and brownfield projects. Finally, we’ll look at a simple strategy for introducing tests and migrating them across test levels, in order to reduce maintenance costs.</p> <h2 id="definitions-of-test-levels">Definitions of Test Levels</h2> <p>Coming up with universal definitions of types of tests, is hard. There are many styles, and different authors choose different categorizations. <strong>For the scope of this post</strong>, we will simplify to three levels.</p> <ul> <li>A <strong>unit test</strong> verifies a single behavior of a single unit – this can be a function, a method, or a class, but it can be something other as well. A logical unit is something that can be <strong>reasoned about in isolation</strong>. The unit must be <strong>completely under your control</strong>. The test should limit the number of other units it touches on, and should not touch any part outside of your own system. As such, the test should <strong>only fail for one reason</strong>.</li> <li><strong>Integration testing</strong> means we’re verifying that that different units are working together. It focuses on the <strong>interfaces between components</strong>. Some of these components may be from third parties. When they break, they signal that, although individual units may behave correctly, something changed in the API that caused failure in another place. Often, integration tests are slower than unit tests.</li> <li><strong>System testing</strong> uses a <strong>production-like environment</strong> to verify the system as a whole. This may include user interfaces, databases, web servers, logging, etc. Typically a call is made at the <strong>outside boundary of the system</strong>. Ideally, the outcome is verified at the same boundary.</li> </ul> <h2 id="learning">Learning</h2> <p>System and integration tests do not require you to adapt your system’s design in order to be testable. The difficulty with good unit tests, is that they require good software design to be useful. Because of this, unit tests are more costly than system and integration tests. To learn system testing, all you need to learn is the tools and the process. To learn unit testing, you need to learn design at the same time. This frustrates developers who are new to unit testing. The method appears not to work for them, because they’re trying to learn unit testing without learning better design.</p> <p>When discussing the cost of a method, we need to separate the cost of the learning from the cost of actually applying the method. Once you’ve learned something, you can transfer this knowledge and reuse it.</p> <h2 id="test-metrics">Test Metrics</h2> <p>A number of metrics affect the cost of a test.</p> <ul> <li><strong>Execution speed</strong>: the faster a test runs, the shorter the feedback loop. You will run the test more often during development. The lifespan of a mistake is reduced. The failing tests notifies you of the mistake before you get a chance to move on to other tasks. The reasoning is still fresh in your head. Nothing else is built on top of the mistake. You can quickly fix it and move on. People with little experience in TDD usually underestimate the <strong>cumulative effect of this feedback loop</strong>.</li> <li><strong>Fragility</strong>: the less stable your test is, the less you trust it. If your test breaks a lot, it’s harder to use as a meaningful measure of the system quality. <strong>Brittle tests often indicate a coupling problem</strong>. Either your test touches too many parts of the system, or the system under test itself is too coupled. For example, a small change in the GUI breaks a test in the domain model. Even worse is <em>flickering</em>: a test that breaks, even when you haven’t made any changes. This might happen because of things like race conditions or dependencies on services outside of your control.</li> <li><strong>Understandability</strong>: writing code is usually easier than reading it. A clear test is the best documentation for the system you could have. Textual documentation tends to rot. Tests on the other hand are executable. When the system changes, the test breaks, and therefore the documentation breaks. However, a misleading or confusing test can actually derail the learning process of a developer. If the effort of reading the test is too great, it won’t be read, it won’t be maintained, and it won’t be used.</li> </ul> <p>When do these metrics matter?</p> <p>When you are introducing a first test, we have a specific goal: we want to add a new feature, or we want to assert an existing behavior. We focus on one test at a time. The execution speed is usually not that important. Even if it takes ten seconds, that’s still short enough to get the benefit of fast feedback. We run the test in isolation, so it only breaks because of direct changes. Fragility won’t show up until much later. And finally, as we have just written the test, understandability is not a factor yet either.</p> <p>The impact of these metrics increases dramatically the more tests you have. The cost you pay in order to keep the tests fast, green, and understandable, is affecting overall development effort. You could ignore the tests, but then you lose all the benefits they bring.</p> <h2 id="project-lifecycle">Project Lifecycle</h2> <p>Is it a new project, or an existing codebase? That is our final factor for looking at the cost of tests. In our experience, in a greenfield project, the cost of introducing tests at any level (unit, integration, system), is equivalent.</p> <p>Being able to introduce tests cheaply at any level, is great, but there’s a hidden risk. People new to automated testing, might be inclined to put too much focus on system and integration tests. As we discussed earlier, those levels are easier to learn, and, by consequence, more popular. Over time, system and integration tests become costlier: they are slower, harder to read, and they are brittle. These forces accumulate, leading to technical debt in the tests.</p> <style> td { border: 1px solid white; } </style> <table cellpadding="5" cellspacing="5" style="width:100%; margin: 30px 5px"> <thead> <tr> <th>&nbsp;</th> <th colspan="2">Greenfield</th> <th colspan="2">Brownfield</th> </tr> <tr> <th>&nbsp;</th> <th>Introduction</th> <th>Maintenance</th> <th>Introduction</th> <th>Maintenance</th> </tr> </thead> <tbody> <tr> <th style="text-align: right;">System tests</th> <td style="text-align: center; background-color: #d9ead3">€</td> <td style="text-align: center; background-color: #f4cccc">€ € €</td> <td style="text-align: center; background-color: #d9ead3">€</td> <td style="text-align: center; background-color: #f4cccc">€ € €</td> </tr> <tr> <th style="text-align: right;">Integration tests</th> <td style="text-align: center; background-color: #d9ead3">€</td> <td style="text-align: center; background-color: #fff2cc">€ €</td> <td style="text-align: center; background-color: #fff2cc">€ €</td> <td style="text-align: center; background-color: #fff2cc">€ €</td> </tr> <tr> <th style="text-align: right;">Unit tests</th> <td style="text-align: center; background-color: #d9ead3">€</td> <td style="text-align: center; background-color: #d9ead3">€</td> <td style="text-align: center; background-color: #f4cccc">€ € €</td> <td style="text-align: center; background-color: #d9ead3">€</td> </tr> </tbody> </table> <p>To counter the growing costs, Mike Cohn proposed the <a href="http://www.amazon.com/gp/product/0321579364/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=0321579364&amp;linkCode=as2&amp;tag=verraesnet-20&amp;linkId=BPE4YKYDY3WHOFAB">Test Pyramid</a>. The idea is that the majority of your tests should be unit tests, with less focus on the higher levels.</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2015-01-05-economy-of-tests/test_pyramid-small.png" alt="Test Pyramid" /></p> <p>In greenfield, advice like “write more unit tests” is easy to follow. Ignoring that advice creates the sort of technical debt we usually associate with brownfield projects. Those have a very different kind of test economy. Unit tests are very difficult in legacy code: unless you’re extremely lucky, the project is big ball of mud, with huge blocks of code and nothing even remotely resembling a unit that you can test in isolation. You’ll naturally resort to high level testing. That is often the only option.</p> <h2 id="test-level-migration">Test Level Migration</h2> <p>The Test Pyramid is a static model of an ideal end goal. In the real world, when dealing with legacy, we advise to <strong>migrate tests from top to bottom</strong>.</p> <p>For brownfield, it’s perfectly fine to introduce tests at the level you are most comfortable with, and that suits the codebase at hand. But high level tests should be seen as a first step. An inverted pyramid is unbalanced, but you can use the system tests to <strong>put pressure on the test suite</strong>. We can call this <strong>test level migration</strong>: as the system tests give you more confidence to change things in your code base, you use this to add more low level tests.</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2015-01-05-economy-of-tests/Snow_Geese_Migration-small.jpg" alt="Snow geese migration" /> After a while, more behaviour is covered with low-level tests. Now you can remove the high level tests, especially those where the maintenance cost is greater than the value they provide. (<a href="https://verraes.net/2014/12/how-much-testing-is-too-much/">Yes, it’s ok to delete tests.</a>) The balance shifts to the base of the pyramid.</p> <p>Migrating tests can be done gradually. The first step is to drop your system tests to the level of the integration tests. Use the same integration points, such as databases, and other services. The tests still might break a lot, but at least they don’t break because of changes in the high level components such as the GUI or HTTP API. The next step would be to isolate integration points and cover the bulk of the system behaviour with actual unit tests.</p> <p>Our process for introducing and evolving tests in a brownfield project looks like this:</p> <ol> <li>Identify the part of system you want to change.</li> <li>Estimate how expensive adding a unit tests is.</li> <li>If it’s expensive, move up one layer. Can you isolate the test to touch only a small number of components?</li> <li>If that’s still too expensive, you move up again and write a test against the entire system, using the GUI or an API as the boundary.</li> <li>As the number of higher level tests grows and your technical debt starts showing, find ways to refactor the code base, and push tests to lower levels.</li> <li>Get rid of brittle, slow, and unreadable high level tests that test behaviours already covered by lower level tests.</li> </ol> <h2 id="opposing-forces">Opposing forces</h2> <p>Testing on brownfield projects always has two opposing forces:</p> <ul> <li>Adding new test tends to push you to the higher levels of testing.</li> <li>Carefully maintaining your test suite tends to push you to the lower levels of testing.</li> </ul> <p><img style="float:right;margin-left: 10px" src="/img/posts/2015-01-05-economy-of-tests/test_economy_forces-small.png" alt="Opposing forces in the economy of tests" /></p> <p>This is the essence of the Test Pyramid: when you write only high level tests, you take on technical debt. Sometimes this is necessary, but as soon as you have debt, the clock is ticking. Interest accumulates, and it becomes harder to remedy the situation. That’s why putting pressure on your test suite to migrate test levels, pays of in the long run.</p> <h2 id="what-to-migrate">What to migrate</h2> <p>The choice of tests to migrate, is important. You can use the metrics of execution speed, readability, and maintainability, to identify tests in dire need of migration. Another heuristic is asking whether the test is at its natural lowest possible level. For example, when a test is exercising the whole system in order to test a single unit of business logic, it is clear that this test is unnecessary costly, and should be migrated to the level of a unit test. Leave the system tests in place that verify if the system as a whole is working, and use unit tests to fix the most important permutations of the expected behavior.</p> <h2 id="conclusion">Conclusion</h2> <p>For projects with a long life expectancy, investing in your test suite is well worth it. Proper automated testing is a careful balance. Quickly introduce tests when you need them, and build a comprehensive, fast, and reliable test suite in the long run. We hope this post can give you some ammunition to win your team to this idea.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/12/how-much-testing-is-too-much/">How Much Testing is Too Much?</a></li> </ul> How Much Testing is Too Much? 2014-12-29T00:00:00+00:00 https://verraes.net/2014/12/how-much-testing-is-too-much <p>Figuring out how much unit tests you need to write, can be tricky, especially if you are new to Test-Driven Development. Some teams strive for 100% code coverage. Some open source projects even announce their test coverage on their GitHub profiles – as if coverage is an indicator of quality. Coverage only measures the lines of code that are executed by the test suite. It doesn’t tell you whether the outcome of the execution is actually tested, let alone how valuable that test is.</p> <p>Because of that, <strong>code coverage of your entire code base is a pretty lousy metric</strong>. If you use TDD as intended, you don’t need to measure it. The coverage will be good, as a direct result of the method. Some people claim they use coverage to enforce their team to write tests. The better way to solve that problem, is with a healthy dose of education, <a href="/2013/10/pre-merge-code-reviews/">pre-merge code reviews</a>, and pair programming.</p> <p>Code coverage has its uses though. When measured ad hoc, in small doses, it is <strong>a great tool to validate your assumptions</strong> about how a single unit test executes a complex block of code. For an example of how to use code coverage to your advantage, watch <a href="/2013/09/extract-till-you-drop/">Extract Til You Drop</a>.</p> <h2 id="over-test">Over-test</h2> <p>The Standard Consultant’s Answer to “How much tests do I need?” is “It depends”. That won’t do. My preferred approach is to ask <a href="/2014/03/small-controlled-experiments/">“What is the easiest way to find an answer?”</a>. Experiments beat opinions. <strong>When it comes to getting comfortable with TDD, be overzealous.</strong> Be religious about it. Don’t write a single line that isn’t covered by a test first. Do it with the entire team – no code goes to master without being fully tested. Accidentally wrote a code without a test? Delete the code, write the test, implement the code.</p> <p><img src="/img/posts/2014-12-29-how-much-testing-is-too-much/inception-mirrors-small.jpg" alt="Over-designing" /></p> <p>Timebox this experiment, say a couple of weeks. As with all experiments, we ask what’s the worst that could happen. Progress on features could stop entirely, you all learn nothing, and the whole thing was a failure. (Or you have learned that TDD is not for you. Learning happened!)</p> <p>More likely, you’ll get some interesting results:</p> <ul> <li>You’ll learn that unit testing isn’t so scary (provided you make the units small);</li> <li>You’ll make smaller units;</li> <li>You’ll learn to appreciate that <a href="http://www.jefclaes.be/2014/12/tdd-as-crack-cocaine-of-software.html">TDD helps you to get in the Zone</a>;</li> <li>You’ll discover that some things simply aren’t worth writing tests for (getters and setters, empty constructors, …);</li> <li>You’ll have fewer getters and setters anyway, and more methods that actually do something;</li> <li>YAGNI code.</li> </ul> <p>Add a couple more weeks of overzealous testing, and <strong>you’ll get a feel for how much testing is just right</strong>. Time to back down and bend the rules a little. Sometimes, you’ll want to do some exploratory coding, and add tests afterwards (<em>spike &amp; stabilize</em>). Sometimes, you can implement some code in a way that is so obviously correct, that it doesn’t need tests. Sometimes, if you pair program, having extra eyes on the code beats extensive testing. And some edge cases are so rare that worrying about them is a waste.</p> <p>I don’t think you can develop these instincts if you consistently under-test. <strong>When in doubt, test more</strong>, and keep balancing on the thin line between insufficient and too much.</p> <p>After the experiment, you’ll also end up with a bunch of low-value tests. Remove them, especially if they start bothering you or require maintenance. There’s <strong>no value in maintaining useless tests</strong>. A little bit of cleanup can make the tests so much more pleasant to work with. “Deleting tests” is the new “deleting code”.</p> <h2 id="over-design">Over-design</h2> <p>Coincidentally, I give the same advice when it comes to software design and modelling. There will always be bystanders, telling you that your solution is over-designed, especially if they don’t have intimate knowledge of the problem domain or the non-functional requirements. They may be right, but they have no way of knowing that – an neither do you. <a href="/2014/10/software-design-is-just-theory/">Software design has a slow feedback cycle</a>. Over-design just a little, get a feel for it, evaluate, and back down. That last bit is important: <strong>if you don’t refactor when gaining deeper insight, you’re simply creating technical debt</strong>. It’s at the other end of the spectrum now, where the grass is slightly greener, but an over-designed model is technical debt nonetheless.</p> <p>(Update, Dec 30, 2014, based on a comment:)</p> <h2 id="when-to-delete-tests">When to delete tests?</h2> <p>Tests become a problem when:</p> <ul> <li>they are slow;</li> <li>they need to be changed all the time;</li> <li>they break often;</li> <li>they are hard to read;</li> <li>… or they bother you in some other way.</li> </ul> <p>When any of those occur, the tests need to be inspected. Now is the time to decide whether you want to refactor the test itself, or refactor the code under test, or, in some cases, remove the tests. Low-value tests are usually harmless. There’s no urgent need to decide upfront whether they need to be deleted. Trust your instinct, or in this case, your annoyance level.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2015/01/economy-of-tests/">Economy of Tests</a></li> </ul> Let It Crash 2014-12-09T00:00:00+00:00 https://verraes.net/2014/12/erlang-let-it-crash <p>“<a href="http://www.amazon.com/gp/product/193778553X/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=193778553X&amp;linkCode=as2&amp;tag=verraesnet-20&amp;linkId=JM5GZYMS2A36UJTF">Programming Erlang: Software for a Concurrent World, 2nd ed</a>” by Joe Armstrong, is not just a great introduction to the language, but also offers a glimpse into the “Let It Crash” philosophy that drives it. It’s a great read, even if you have no intention of programming in Erlang. I’ve selected and reordered a number of passages that were scattered across the book, and added some comments, that I hope will do the book justice.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-12-09-erlang-let-it-crash/armstrong-erlang-small.jpg" alt="Programming Erlang" /></p> <blockquote>The Erlang view of the world is that everything is a process and that processes can interact only by exchanging messages. Having such a view of the world imposes conceptual integrity on our designs, making them easier to understand.</blockquote> <blockquote>The real world actually has independent things communicating through messages. I’m an ex-physicist — we perceive the world by receiving messages. Packets of light and sound carry information. All that we know about the world is what we’ve learned by receiving messages. We don’t have shared memory. I have my memory, you have yours, and I don’t know what you think about anything. If I want to know what you think about something, then I have to ask you a question and wait for you to reply.</blockquote> <p>(In recent years, the idea of messages as the first class citizens of a system, has been rediscovered by the CQRS/EventSourcing community, albeit with a focus on domain models. In fact, Event Sourced Aggregates are a way to model processes, not things – they are the poor man’s actor model. Building CQRS/ES systems is what led me to Erlang in the first place.)</p> <blockquote>Imagine a system with only one sequential process. If this process dies, we might be in deep trouble since no other process can help. For this reason, sequential languages have concentrated on the prevention of failure and an emphasis on defensive programming. In Erlang we have a large number of processes at our disposal, so the failure of any individual process is not so important. We usually write only a small amount of defensive code and instead concentrate on writing corrective code. We take measures to detect the errors and then correct them after they have occurred.</blockquote> <p>(A typical Erlang program might have hundreds, thousands, or even millions of processes. Letting processes crash is central to Erlang. It’s the equivalent of unplugging your router and plugging it back in – as long as you can get back to a known state, this turns out to be a very good strategy. To make that happen, you build supervision trees. A supervisor will decide how to deal with a crashed process. It will restart the process, or possibly kill some other processes, or crash and let someone else deal with it. Despite the lack of defensive programming in idiomatic Erlang, you spend more time thinking about errors then when building in other languages.)</p> <blockquote>This leads to a clean separation of issues. We write code that solves problems and code that fixes problems, but the two are not intertwined.</blockquote> <blockquote>How can we write programs that run faster on a multicore CPU? It’s all about mutable state and concurrency. Back in the old days (twenty-odd years ago), there were two models of concurrency: Shared state concurrency, and message passing concurrency. The programming world went one way (toward shared state). The Erlang community went the other way.</blockquote> <blockquote>(...) all languages such as C, Java, C++, and so on, have the notion that there is this stuff called state and that we can change it. This is fine as long as you have only one process doing the changing. If you have multiple processes sharing and modifying the same memory, you have a recipe for disaster — madness lies here.</blockquote> <blockquote>To protect against the simultaneous modification of shared memory, we use a locking mechanism. Call this a mutex, a synchronized method, or what you will, but it’s still a lock. If programs crash in the critical region (when they hold the lock), disaster results. All the other programs don’t know what to do. If programs corrupt the memory in the shared state, disaster will also happen.</blockquote> <blockquote>Erlang has no mutable data structures (that’s not quite true, but it’s true enough). No mutable data structures = No locks. No mutable data structures = Easy to parallelize.</blockquote> <p>(Erlang was originally written for networks of single core computers, back in the eighties. The fact that it turns out to be great for multicore CPU’s, is an interesting side-effect.)</p> <blockquote>Once started, Erlang/OTP applications are expected to run forever (...)</blockquote> <p>(This is no overstatement. Erlang allows you to upgrade the system while it is running. <a href="https://www.youtube.com/watch?v=96UzSHyp0F8">But don’t take my word for it</a>.)</p> Higher Order Programming 2014-11-24T00:00:00+00:00 https://verraes.net/2014/11/higher-order-programming <p>Let’s have some fun with higher order programming in PHP. I’ll start by showing how to program with Lambdalicious (or λlicious for friends) and introduce the real meat along the way. Don’t worry too much about the dark magic that may appear to power some of the features of Lambdalicious. It’s <a href="https://github.com/mathiasverraes/lambdalicious">on GitHub</a> if you’re curious. Just follow along and keep track of all the functions.</p> <p><strong>Note:</strong> The code in this blogpost works with <a href="https://github.com/mathiasverraes/lambdalicious/tree/v0.1.0">Lambdalicious v0.1.0</a>. If you want to adapt this blog post to the current version, please send a <a href="https://github.com/mathiasverraes/mathiasverraes.github.com/edit/master/_posts/2014-11-24-higher-order-programming.markdown">pull request</a>.</p> <p>But first, something smaller:</p> <h2 id="atoms">Atoms</h2> <p><code class="language-plaintext highlighter-rouge">@foobar</code> is an atom. You can think of an atom as a constant that has itself as a value. So the value of <code class="language-plaintext highlighter-rouge">@foobar</code> is <code class="language-plaintext highlighter-rouge">foobar</code>.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">require_once</span> <span class="k">__DIR__</span> <span class="mf">.</span> <span class="s1">'path/to/src/Verraes/Lambdalicious/load.php'</span><span class="p">;</span> <span class="nb">assert</span><span class="p">(</span> <span class="nf">isatom</span><span class="p">(</span><span class="o">@</span><span class="n">foobar</span><span class="p">)</span> <span class="p">);</span> </code></pre></div></div> <p>(Note that if you combine all the code in this post in a single file, it is executable and all assertions will pass.)</p> <p>Unlike constants in PHP, atoms don’t need to be defined before you use them. They are global and immutable.</p> <p>Atoms are very useful for many things. Later on, we’ll use them to refer to global functions, without needing to resort to quotes. In fact, the philosophy of λlicious is to remove syntactical noise, verbosity, and ceremony. To that effect, let’s get rid of the <code class="language-plaintext highlighter-rouge">@</code> prefix. We can do that by affixing an atom using the <code class="language-plaintext highlighter-rouge">atom</code> function:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nf">atom</span><span class="p">(</span><span class="o">@</span><span class="n">foobar</span><span class="p">);</span> <span class="nb">assert</span><span class="p">(</span> <span class="nf">isatom</span><span class="p">(</span><span class="n">foobar</span><span class="p">)</span> <span class="c1">// no @ prefix needed anymore</span> <span class="p">);</span> </code></pre></div></div> <p>Once an atom is affixed, you can use it everywhere without the prefix.</p> <h2 id="lists">Lists</h2> <p>Let’s affix some atoms and make a list.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nf">atom</span><span class="p">(</span><span class="o">@</span><span class="n">a</span><span class="p">,</span> <span class="o">@</span><span class="n">b</span><span class="p">,</span> <span class="o">@</span><span class="n">c</span><span class="p">,</span> <span class="o">@</span><span class="n">d</span><span class="p">);</span> <span class="nb">assert</span><span class="p">(</span> <span class="nf">islist</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">])</span> <span class="p">);</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nf">cons</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="p">[</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]),</span> <span class="p">[</span><span class="n">d</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]</span> <span class="p">));</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">cons($head, $tail)</code> constructs a list by adding <code class="language-plaintext highlighter-rouge">$head</code> to the front of the list <code class="language-plaintext highlighter-rouge">$tail</code>. In fact, the square brackets are nothing but a shortcut for a bunch of nested <code class="language-plaintext highlighter-rouge">cons</code>.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nf">cons</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="nf">cons</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="nf">cons</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="p">[]))),</span> <span class="p">[</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]</span> <span class="p">));</span> </code></pre></div></div> <p>We can deconstruct lists by taking the <code class="language-plaintext highlighter-rouge">head</code>, which is the first element, or the <code class="language-plaintext highlighter-rouge">tail</code>, which is all the elements without the head.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nf">head</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]),</span> <span class="n">a</span> <span class="p">));</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nf">tail</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]),</span> <span class="p">[</span><span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]</span> <span class="p">));</span> </code></pre></div></div> <p>Remember that <code class="language-plaintext highlighter-rouge">head</code> always returns an element, and <code class="language-plaintext highlighter-rouge">tail</code> returns a list.</p> <h2 id="functions">Functions</h2> <p>Now let’s make a global function. Global means that we can reach it from anywhere.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">function</span> <span class="n">half</span><span class="p">(</span><span class="nv">$x</span><span class="p">){</span> <span class="k">return</span> <span class="nf">divide</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="mi">2</span><span class="p">);}</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nf">half</span><span class="p">(</span><span class="mi">6</span><span class="p">),</span> <span class="mi">3</span> <span class="p">));</span> </code></pre></div></div> <p>If we affix the function’s name, we can refer to it by that atom.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nf">atom</span><span class="p">(</span><span class="o">@</span><span class="n">half</span><span class="p">);</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nf">call</span><span class="p">(</span><span class="n">half</span><span class="p">,</span> <span class="p">[</span><span class="mi">6</span><span class="p">]),</span> <span class="nf">half</span><span class="p">(</span><span class="mi">6</span><span class="p">)</span> <span class="p">));</span> </code></pre></div></div> <p>We can also make locally scoped functions. We assign them to a variable that we can use to pass the function to another function.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$half</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nf">divide</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="mi">2</span><span class="p">);};</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$half</span><span class="p">(</span><span class="mi">3</span><span class="p">),</span> <span class="nf">call</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">])</span> <span class="p">));</span> </code></pre></div></div> <h2 id="conditionals">Conditionals</h2> <p>We’ll need conditionals. In λlicious, they look like this:</p> <div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>return condition1 ? result1 : (condition2 ? result2 : (condition3 ? result3 : (finalResult))); </code></pre></div></div> <p>In other words, conditions are tried one after another until one of them evaluates to true. The value of the complete expression is the result that follows the first true condition. If none of the conditions evaluate to true, the finalResult is returned.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-11-24-higher-order-programming/recursive_percy_small.jpg" alt="Recursive Percy" /></p> <h2 id="loops--list-processing">Loops &amp; List Processing</h2> <p>Now we want to get the halves of a whole list of integers. Our test will look like this: <code class="language-plaintext highlighter-rouge">assert(isequal($halves([2, 4, 6]), [1, 2, 3]));</code></p> <p>In λlicious, we only ever assign a value to a variable once. PHP does not constrain this: you can override the value of a variable as much as you like. Worry not: a λlicious-programmer is disciplined and strong-minded, and does not require any hand-holding from the compiler. This also means that we never use loops like foreach, while, …, as they mutate variables. The alternative way to loop, is by using recursion.</p> <p><code class="language-plaintext highlighter-rouge">$halves</code> takes a list as an argument. We’ll calculate the half of the head of that list, and then recurse to <code class="language-plaintext highlighter-rouge">$halves</code> again with the tail of the list. We cons up the calculated half with the result of the recursive call to <code class="language-plaintext highlighter-rouge">$halves</code>. Here’s our first attempt:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$halves1</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="k">use</span><span class="p">(</span><span class="o">&amp;</span><span class="nv">$halves1</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nf">cons</span><span class="p">(</span> <span class="c1">// create a new list consisting of:</span> <span class="nf">divide</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="mi">2</span><span class="p">),</span> <span class="c1">// the half of the first element</span> <span class="nv">$halves1</span><span class="p">(</span><span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">))</span> <span class="c1">// the halves of the remaining elements</span> <span class="p">);</span> <span class="p">}</span> </code></pre></div></div> <p>(The <code class="language-plaintext highlighter-rouge">use(&amp;$halves1)</code> deserves some explanation. At this point in the code, <code class="language-plaintext highlighter-rouge">$halves1</code> is not yet defined. By closing it in our function by reference with <code class="language-plaintext highlighter-rouge">&amp;</code>, we can call it at the point in the code where we reach <code class="language-plaintext highlighter-rouge">$halves1(tail($list))</code>. It’s a little trick in PHP to recurse on locally scoped functions. Don’t worry about it. When you get a <code class="language-plaintext highlighter-rouge">Notice: Undefined variable: foo</code>, just remember to add <code class="language-plaintext highlighter-rouge">use(&amp;$foo)</code>.)</p> <p>If we try to use <code class="language-plaintext highlighter-rouge">$halves1</code>, we crash with a message saying <code class="language-plaintext highlighter-rouge">HeadIsDefinedOnlyForNonEmptyLists</code>. The problem is that we are recursing, but we have not thought about how the loop stops. The λlicious-programmer accepts that mistakes happen, and draws lessons from it. From now on, whenever we recurse, we will think about the termination first.</p> <p>As we are picking elements of our <code class="language-plaintext highlighter-rouge">$list</code>, it will become empty at some point. We’ll need some place to store elements that we halved, so we add an accumulator called <code class="language-plaintext highlighter-rouge">$acc</code>, which starts as an empty list. Whe <code class="language-plaintext highlighter-rouge">$list</code> becomes empty, we return the accumulated values in <code class="language-plaintext highlighter-rouge">$acc</code>.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$halves2</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&amp;</span><span class="nv">$halves2</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nf">isempty</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="o">?</span> <span class="nv">$acc</span> <span class="o">:</span> <span class="c1">// return $acc when we're done picking off items</span> <span class="nv">$halves2</span><span class="p">(</span> <span class="c1">// Recurse</span> <span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="nf">cons</span><span class="p">(</span> <span class="c1">// Our new $acc will be our newly calculated half, </span> <span class="c1">// followed by the old $acc</span> <span class="nf">divide</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="mi">2</span><span class="p">),</span> <span class="nv">$acc</span> <span class="p">)</span> <span class="p">);</span> <span class="p">};</span> </code></pre></div></div> <p>The assert still fails. Closer inspection learns that <code class="language-plaintext highlighter-rouge">$halves([2, 4, 6])</code> returns <code class="language-plaintext highlighter-rouge">[3, 2, 1]</code>. That makes sense: we keep consing up the halved head of the original list, to the accumulated list, so our <code class="language-plaintext highlighter-rouge">$acc</code> ends up backwards. That’s easy to fix. We simply call <code class="language-plaintext highlighter-rouge">reverse</code> on <code class="language-plaintext highlighter-rouge">$acc</code> before returning it.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$halves3</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&amp;</span><span class="nv">$halves3</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nf">isempty</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="o">?</span> <span class="nf">reverse</span><span class="p">(</span><span class="nv">$acc</span><span class="p">)</span> <span class="o">:</span> <span class="c1">// Reversing $acc at the last moment</span> <span class="nv">$halves3</span><span class="p">(</span> <span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="nf">cons</span><span class="p">(</span> <span class="c1">// This is where our halved heads end up in the wrong order</span> <span class="nf">divide</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="mi">2</span><span class="p">),</span> <span class="nv">$acc</span> <span class="p">)</span> <span class="p">);</span> <span class="p">};</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$halves3</span><span class="p">([</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">]),</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span> <span class="p">));</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">$halves3</code> passes our test. Recursion is great fun, and can be very rewarding. It’s especially useful for making obvious jokes. If you feel you don’t really get recursion, I advise you to read <a href="/2014/11/higher-order-programming/">this wonderful blog post</a>.</p> <h2 id="deduplication">Deduplication</h2> <p><code class="language-plaintext highlighter-rouge">$halves3</code> works, but we’re not quite happy with it. <code class="language-plaintext highlighter-rouge">divide(head($list), 2)</code> duplicates the logic of our <code class="language-plaintext highlighter-rouge">$half</code> function. Let’s inject that function into our 4th version.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$halves4</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&amp;</span><span class="nv">$halves4</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nf">isempty</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="o">?</span> <span class="nf">reverse</span><span class="p">(</span><span class="nv">$acc</span><span class="p">)</span> <span class="o">:</span> <span class="nv">$halves4</span><span class="p">(</span> <span class="nv">$half</span><span class="p">,</span> <span class="c1">// Don't forget to keep passing $half along to $halves4</span> <span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="nf">cons</span><span class="p">(</span> <span class="nv">$half</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">)),</span> <span class="c1">// Using our injected $half function</span> <span class="nv">$acc</span> <span class="p">)</span> <span class="p">);</span> <span class="p">};</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$halves4</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">]),</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span> <span class="p">));</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">$halves4</code> is a higher order function: it takes another function as an argument. In our case, it applies this function to each element of <code class="language-plaintext highlighter-rouge">$list</code>. This pattern is actually quite common. So common indeed, that it has a name: map. We can simply rename <code class="language-plaintext highlighter-rouge">$halves4</code> and <code class="language-plaintext highlighter-rouge">$half</code> to be more generic.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$map</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$function</span><span class="p">,</span> <span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&amp;</span><span class="nv">$map</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nf">isempty</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="o">?</span> <span class="nf">reverse</span><span class="p">(</span><span class="nv">$acc</span><span class="p">)</span> <span class="o">:</span> <span class="nv">$map</span><span class="p">(</span> <span class="c1">// Recurse over map</span> <span class="nv">$function</span><span class="p">,</span> <span class="c1">// Passing $function along to the next call of $map</span> <span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="nf">cons</span><span class="p">(</span> <span class="nv">$function</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">)),</span> <span class="c1">// Using our injected $function</span> <span class="nv">$acc</span> <span class="p">)</span> <span class="p">);</span> <span class="p">};</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$map</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">]),</span> <span class="c1">// Map $half over the list</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span> <span class="p">));</span> </code></pre></div></div> <p>(Later on we’ll use the built-in global function <code class="language-plaintext highlighter-rouge">map</code> instead of our own <code class="language-plaintext highlighter-rouge">$map</code>, as it’s a little smarter.)</p> <h2 id="filter-and-reduce">Filter and Reduce</h2> <p><code class="language-plaintext highlighter-rouge">filter</code> and <code class="language-plaintext highlighter-rouge">reduce</code> are two other very commonly used higher order functions. They are globally defined in λlicious. The implementation is similar to that of <code class="language-plaintext highlighter-rouge">map</code>, so I’ll leave them as an exercise to the λlicious student. We’ll use them later in this post. Here’s a skeleton to get you started:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$filter</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$function</span><span class="p">,</span> <span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&amp;</span><span class="nv">$filter</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="no">TERMINATING_CONDITION</span> <span class="o">?</span> <span class="no">SOME_VALUE</span> <span class="o">:</span> <span class="nv">$filter</span><span class="p">(</span> <span class="nv">$function</span><span class="p">,</span> <span class="no">A_LIST</span><span class="p">,</span> <span class="no">CALCULATE_THE_NEW_ACCUMULATOR</span> <span class="p">)</span> <span class="p">;</span> <span class="p">};</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$filter</span><span class="p">(</span><span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$x</span> <span class="o">&gt;=</span> <span class="mi">3</span><span class="p">;},</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]),</span> <span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span> <span class="p">));</span> <span class="nv">$reduce</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$function</span><span class="p">,</span> <span class="nv">$list</span><span class="p">,</span> <span class="nv">$init</span><span class="p">)</span> <span class="k">use</span><span class="p">(</span><span class="o">&amp;</span><span class="nv">$reduce</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="no">TERMINATING_CONDITION</span> <span class="o">?</span> <span class="no">SOME_VALUE</span> <span class="o">:</span> <span class="nv">$reduce</span><span class="p">(</span> <span class="nv">$function</span><span class="p">,</span> <span class="no">A_LIST</span><span class="p">,</span> <span class="no">CALCULATE_THE_NEW_INIT</span> <span class="p">)</span> <span class="p">;</span> <span class="p">};</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$reduce</span><span class="p">(</span><span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="nv">$y</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nf">add</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="nv">$y</span><span class="p">);},</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="mi">0</span><span class="p">),</span> <span class="mi">6</span> <span class="p">));</span> </code></pre></div></div> <h2 id="functions-returning-functions">Functions returning functions</h2> <p>Functions that take functions as arguments are just one type of higher order functions. We can also return functions. Let’s make a function that makes <code class="language-plaintext highlighter-rouge">$half</code>.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$halfMaker</span> <span class="o">=</span> <span class="k">function</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// make a new function and return it </span> <span class="k">return</span> <span class="nf">divide</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span> <span class="p">};</span> <span class="p">};</span> <span class="nv">$half2</span> <span class="o">=</span> <span class="nv">$halfMaker</span><span class="p">();</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$half2</span><span class="p">(</span><span class="mi">8</span><span class="p">),</span> <span class="mi">4</span> <span class="p">));</span> </code></pre></div></div> <p>This may seem a little pointless. But this is actually a pattern as well. We’ve hardcoded the division by 2, but what if we want to make that number dynamic? We rename <code class="language-plaintext highlighter-rouge">$halfMaker</code> into <code class="language-plaintext highlighter-rouge">$divisionMaker</code> and take a number as an argument:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$divisionMaker</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$y</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="k">function</span> <span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="k">use</span> <span class="p">(</span><span class="nv">$y</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// make a closure with $y</span> <span class="k">return</span> <span class="nf">divide</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="nv">$y</span><span class="p">);</span> <span class="p">};</span> <span class="p">};</span> <span class="nv">$third</span> <span class="o">=</span> <span class="nv">$divisionMaker</span><span class="p">(</span><span class="mi">3</span><span class="p">);</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$third</span><span class="p">(</span><span class="mi">9</span><span class="p">),</span> <span class="mi">3</span> <span class="p">));</span> </code></pre></div></div> <p>We can make it even more generic. Right now, <code class="language-plaintext highlighter-rouge">division</code> is hardcoded into <code class="language-plaintext highlighter-rouge">$divisionMaker</code>. We can actually pass that function in as well. We rename <code class="language-plaintext highlighter-rouge">$divisionMaker</code> to <code class="language-plaintext highlighter-rouge">$partial</code>.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$partial</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$f</span><span class="p">,</span> <span class="nv">$y</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="k">use</span><span class="p">(</span><span class="nv">$f</span><span class="p">,</span> <span class="nv">$y</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$f</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="nv">$y</span><span class="p">);</span> <span class="p">};</span> <span class="p">};</span> <span class="nv">$third2</span> <span class="o">=</span> <span class="nv">$partial</span><span class="p">(</span><span class="n">divide</span><span class="p">,</span> <span class="mi">3</span><span class="p">);</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$third2</span><span class="p">(</span><span class="mi">9</span><span class="p">),</span> <span class="mi">3</span> <span class="p">));</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">$partial</code> takes two arguments: a function <code class="language-plaintext highlighter-rouge">$f</code> and one argument <code class="language-plaintext highlighter-rouge">$y</code>. It returns a new function, which takes another argument <code class="language-plaintext highlighter-rouge">$x</code>, and return the result of <code class="language-plaintext highlighter-rouge">$f($x, $y)</code>. Partial function application is a nice example of higher order programming: you create new functions from existing functions.</p> <h2 id="partial-function-application">Partial Function Application</h2> <p>Wouldn’t it be nice if we had an elegant syntax to partially call functions? In λlicious, many functions already feature partial function application by default:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$half3</span> <span class="o">=</span> <span class="nf">divide</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span> <span class="c1">// returns a function $f($x){ return divide($x, 2); }</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$half3</span><span class="p">(</span><span class="mi">8</span><span class="p">),</span> <span class="mi">4</span> <span class="p">));</span> <span class="nv">$increment</span> <span class="o">=</span> <span class="nf">add</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">__</span><span class="p">);</span> <span class="c1">// returns a function $f($y){ return add(1, $y); }</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$increment</span><span class="p">(</span><span class="mi">5</span><span class="p">),</span> <span class="mi">6</span> <span class="p">));</span> </code></pre></div></div> <p>The double underscores are placeholders: You can think of them as arguments that you don’t know yet, and that you’ll fill in later. Partial function application allows you to define behaviour in one place, and then process data using that behaviour, in another place. It’s great way of writing compact, highly reusable, highly readable code. It might take some getting used to, but once you get it, you don’t want to code without it anymore. Here’s an example with <code class="language-plaintext highlighter-rouge">map</code>:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$halves5</span> <span class="o">=</span> <span class="nf">map</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="n">__</span><span class="p">);</span> <span class="c1">// elsewhere:</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$halves5</span><span class="p">([</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">]),</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span> <span class="p">));</span> </code></pre></div></div> <p>At the point where we use <code class="language-plaintext highlighter-rouge">$halves5</code>, we don’t need to know that internally, it’s actually a map of <code class="language-plaintext highlighter-rouge">$half</code>.</p> <h2 id="composition">Composition</h2> <p>Sometimes you need a function that is composed of other functions. If we want to half a value and then add one, we can make a function that does the work for us:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$halfAndIncrementMaker</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$half3</span><span class="p">,</span> <span class="nv">$increment</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="k">use</span> <span class="p">(</span><span class="nv">$half3</span><span class="p">,</span> <span class="nv">$increment</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$increment</span><span class="p">(</span><span class="nv">$half3</span><span class="p">(</span><span class="nv">$x</span><span class="p">));</span> <span class="p">};</span> <span class="p">};</span> <span class="nv">$halfAndIncrement</span> <span class="o">=</span> <span class="nv">$halfAndIncrementMaker</span><span class="p">(</span><span class="nv">$half3</span><span class="p">,</span> <span class="nv">$increment</span><span class="p">);</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$halfAndIncrement</span><span class="p">(</span><span class="mi">10</span><span class="p">),</span> <span class="mi">6</span> <span class="p">));</span> </code></pre></div></div> <p>Once again, we’ll turn our <code class="language-plaintext highlighter-rouge">$halfAndIncrementMaker</code> into a more generic function, called <code class="language-plaintext highlighter-rouge">compose</code>. It’s another example of a higher order function: it takes functions as arguments and returns a new function.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$compose</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$f</span><span class="p">,</span> <span class="nv">$g</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="k">use</span> <span class="p">(</span><span class="nv">$f</span><span class="p">,</span> <span class="nv">$g</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$g</span><span class="p">(</span><span class="nv">$f</span><span class="p">(</span><span class="nv">$x</span><span class="p">));</span> <span class="p">};</span> <span class="p">};</span> <span class="nv">$halfAndIncrement</span> <span class="o">=</span> <span class="nv">$compose</span><span class="p">(</span><span class="nv">$half3</span><span class="p">,</span> <span class="nv">$increment</span><span class="p">);</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$halfAndIncrement</span><span class="p">(</span><span class="mi">10</span><span class="p">),</span> <span class="mi">6</span> <span class="p">));</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">$compose</code> creates a function that calls <code class="language-plaintext highlighter-rouge">$f</code> on <code class="language-plaintext highlighter-rouge">$x</code>, and then calls <code class="language-plaintext highlighter-rouge">$g</code> on the result of that. The built–in <code class="language-plaintext highlighter-rouge">compose</code> is a little bit smarter and accepts more than two arguments.</p> <h2 id="piping">Piping</h2> <p><code class="language-plaintext highlighter-rouge">pipe</code> is the same as <code class="language-plaintext highlighter-rouge">compose</code>, but it applies the functions in reverse. In my opinion, it is more natural to read. Think of pipes and filters in Linux. All together now:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$half5</span> <span class="o">=</span> <span class="nf">divide</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span> <span class="nv">$greaterThanSix</span> <span class="o">=</span> <span class="nf">gt</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">6</span><span class="p">);</span> <span class="nv">$calculate</span> <span class="o">=</span> <span class="nf">pipe</span><span class="p">(</span> <span class="c1">// pipe returns a new function</span> <span class="nf">map</span><span class="p">(</span><span class="nv">$half5</span><span class="p">,</span> <span class="n">__</span><span class="p">),</span> <span class="c1">// Half all the elements of a list</span> <span class="nf">filter</span><span class="p">(</span><span class="nv">$greaterThanSix</span><span class="p">,</span> <span class="n">__</span><span class="p">),</span> <span class="c1">// Keep only the elements &gt; 6 of the new list</span> <span class="nf">reduce</span><span class="p">(</span><span class="n">add</span><span class="p">,</span> <span class="n">__</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="c1">// Add up all the filtered elements, starting with 0</span> <span class="p">);</span> <span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span> <span class="nv">$calculate</span><span class="p">([</span><span class="mi">10</span><span class="p">,</span> <span class="mi">20</span><span class="p">,</span> <span class="mi">30</span><span class="p">]),</span> <span class="mi">25</span> <span class="p">));</span> </code></pre></div></div> <p>If we call <code class="language-plaintext highlighter-rouge">$calculate</code> with a list <code class="language-plaintext highlighter-rouge">[10, 20, 30]</code>, the first placeholder is replaced with that list. The outcome is <code class="language-plaintext highlighter-rouge">[5, 10, 15]</code>. That result is fed into the <code class="language-plaintext highlighter-rouge">filter</code>, which removes the <code class="language-plaintext highlighter-rouge">5</code> from that list. Finally, <code class="language-plaintext highlighter-rouge">reduce</code> adds up <code class="language-plaintext highlighter-rouge">10</code> to <code class="language-plaintext highlighter-rouge">0</code>, and than the adds up <code class="language-plaintext highlighter-rouge">15</code> to that <code class="language-plaintext highlighter-rouge">10</code>, resulting in <code class="language-plaintext highlighter-rouge">25</code> which is the final outcome of <code class="language-plaintext highlighter-rouge">$calculate</code>.</p> <p>A great way to read that code, is to follow the pipe top the bottom. The first placeholder means “fill in a list that we don’t know about yet”. The second placeholder means: “pipe the result of the map to the filter”. The third placeholder means: “pipe the filtered list to reduce”. You can really follow the flow data through the placeholders, without being distracted by the actual data.</p> <p>We can get rid of <code class="language-plaintext highlighter-rouge">$half5</code> and <code class="language-plaintext highlighter-rouge">$greaterThanTen</code>, to make it even prettier.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$calculate2</span> <span class="o">=</span> <span class="nf">pipe</span><span class="p">(</span> <span class="nf">map</span><span class="p">(</span><span class="nf">divide</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="n">__</span><span class="p">),</span> <span class="nf">filter</span><span class="p">(</span><span class="nf">gt</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">6</span><span class="p">),</span> <span class="n">__</span><span class="p">),</span> <span class="nf">reduce</span><span class="p">(</span><span class="n">add</span><span class="p">,</span> <span class="n">__</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="p">);</span> </code></pre></div></div> <p>We’ve now expressed a calculation as a composition of partially applied functions. It’s compact, elegant, and to the point. If you have trouble understanding what’s going on, you can inject some <code class="language-plaintext highlighter-rouge">dump</code> functions along the way:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$calculate3</span> <span class="o">=</span> <span class="nf">pipe</span><span class="p">(</span> <span class="nf">map</span><span class="p">(</span><span class="nf">divide</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="n">__</span><span class="p">),</span> <span class="n">dump</span><span class="p">,</span> <span class="nf">filter</span><span class="p">(</span><span class="nf">gt</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">6</span><span class="p">),</span> <span class="n">__</span><span class="p">),</span> <span class="n">dump</span><span class="p">,</span> <span class="nf">reduce</span><span class="p">(</span><span class="n">add</span><span class="p">,</span> <span class="n">__</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="p">);</span> </code></pre></div></div> <p>Imagine you had to write all that in traditional procedural PHP. The beauty is that we separates the definition of how data flows through the functions, from having to deal with the actual data. From a small set of primitive functions, we can build ever more complex abstractions. And that is the point of higher order programming.</p> <h2 id="lambdalicious-and-php">Lambdalicious and PHP</h2> <p>Keep in mind that, at the time of writing, λlicious is not stable and might already have changed since you read this. Lambdalicious is not fit for any purpose. If you use it in production and everything breaks, your friends will laugh at you behind your back. I will provide a shoulder to cry on at twice my normal consultancy rates, while saying “I told you so.”</p> <p>The reason it can not work is that PHP is not suitable for recursion. Some languages have something called tail call optimization, which means that if the last expression is recursive, the compiler will make a jump. The call stack does not increase, and all is well.</p> <p>PHP will either run out of memory (<em>Fatal error: Allowed memory size of X bytes exhausted</em>) or, if you have XDebug installed, it will stop at a 100 calls by default. (<em>PHP Fatal error: Maximum function nesting level of ‘100’ reached, aborting</em>). You can increase <code class="language-plaintext highlighter-rouge">xdebug.max_nesting_level</code> in php.ini. HHVM overflows (<em>Fatal error: Stack overflow</em>). I promise to send some fine Belgian beer or chocolates to whoever fixes this in PHP or HHVM.</p> <h2 id="links">Links</h2> <ul> <li><a href="https://github.com/mathiasverraes/lambdalicious">Lambdalicious on GitHub</a></li> <li><a href="https://gist.github.com/mathiasverraes/4b76822c6be565a092f7">The full code for this blog post</a></li> </ul> Modelling Heuristics 2014-11-20T00:00:00+00:00 https://verraes.net/2014/11/modelling-heuristics <p>The slides for my talk at Build Stuff in Vilnius. (As often, the slides are not very interesting without the text.)</p> <script async="" class="speakerdeck-embed" data-id="047406a05332013247257e07207ad8af" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><a href="https://speakerdeck.com/mathiasverraes/modelling-heuristics"><small>See the slides on Speakerdeck</small></a></p> <h2 id="abstract">Abstract</h2> <p>“How do you approach software modelling problems?” When a client asked me about my mental process, I didn’t have a proper answer — yet. The question intrigued me. Do you need a propensity for modelling, or is it a learnable skill? I embarked on a mission to find modelling heuristics: the mental tools, that help us look at the problem from different angles; the little tricks to validate a model before we build it; the rules of thumb that lead us to iteratively refine our solutions. In this talk, I share some of the ideas I collected from books, from people I interviewed, and from my own experiences.</p> Domain Events 2014-11-11T00:00:00+00:00 https://verraes.net/2014/11/domain-events <h2 id="boundaries--messages">Boundaries &amp; Messages</h2> <p>A system’s design is defined by it’s boundaries, and by the way it communicates with other systems. This outward facing design is, all things considered, more important than the internals. It is important whether we are talking about small systems (such as an object) or large systems.</p> <p>The communication between systems happens with messages. We can say that our system is bounded when it does not need to know anything about the internal structure of systems it communicates with, and those systems do no need to know anything about our system. When the boundaries are chosen well, we can achieve greater decoupling between systems. This allows us to change the internals of the systems, or replace them altogether.</p> <p>In software, different systems agree on a protocol that describes the shape of the message. Without it, the message is meaningless. The protocol might be implicit, but typically, the more users a protocol has, the more it needs to be formally described. For example, look at the RFCs that describe the Internet. A protocol is not just an encoding, like JSON or XML. It also describes what properties you’ll find in the message, and how to interpret them. Messages can be broadcast, queued, logged, persisted, discarded, … Depending on the infrastructure, they can arrive late, out of order, more than once, or never at all. If we need high precision, we can’t even trust their timestamp. Understanding the infrastructure and taking the constraints into account, is critical here.</p> <h2 id="separate-models">Separate Models</h2> <p>The receiving end does not need to understand anything about the sender, as long as it understands the message. In other words, the model that the sender and the receiver must share, is constrained to the message. The receiver might conform to the model of the sender, but it is often better for the receiver to go its separate way. Having different models for different systems, is very powerful. A grand, unified model for all our systems can be convenient in the beginning, but becomes a burden as the systems grow. In separated models, each individual system is liberated from having to know a lot of information, or from having structures that are not suitable for that system’s purpose. Contrast this to typical system integrations using an API that transmits entire entities’ states, but leaves it to the client to figure out what has changed and why. In that scenario, the client knows more about the structure of the server, but has less valuable information. Introducing (or discovering) good boundaries, and the communication between them, is the idea behind Context Mapping and Bounded Contexts in Domain-Driven Design.</p> <h2 id="immutability">Immutability</h2> <p>Messages are immutable. Say A sends a message X to B and C. If B were to change the message before passing it on to D, that would not affect A and C. If you draw on a newspaper you’ve received, only your own copy is affected. Modeling messages as immutable reflects that idea. A better interpretation is that B reacts to X by sending a message Y to D. Even though X and Y may be mostly identical, it is in fact a new message.</p> <p>(Update Nov 14, 2014) Domain Events should not contain Entities (or Aggregates for that matter). Entities represent something that changes over time. At the point in time that we look at the Event, the Entity might already have changed. So now the Domain Event is no longer an accurate representation of state at that point in time.</p> <h2 id="envelopes">Envelopes</h2> <p>Sometimes we want to add metadata to a message. These are values that are not part of the actual message, but rather of the surrounding infrastructure: the originating machine, the destination, timestamps, IP’s, the browser, user information, … I like the analogy of an envelope here. You can write on it, without affecting the message. The message is wrapped before being sent, and unwrapped at its arrival. All other systems that are involved in the transport, queuing, or persistence of messages, are unaware of the contents of the envelope. This reflects how postal services have worked for centuries. (Censorship and redaction of messages can be seen as a violation of this contract between sender and receiver.)</p> <h2 id="events">Events</h2> <p>An Event is a type of message that describes something that has happened in the past. This is very useful, because using Events, systems can tell other interested parties something about themselves. The receiver can now react to the event. This is modeled after the way things work in the real world.</p> <p>Domain Events are a special type of Event messages. A Domain Event is something that has happened in the past, that is of interest to the business. This last distinction means we separate all technical concerns from the domain. Usually, the distinction is very clear. In other cases, we should validate with the business whether they are in fact interested that this Domain Event has happened. In rare cases, the line between technical and domain events is blurred.<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup></p> <p>By its nature, an Event has already happened by the time we know about it. We might cheat, and notify other systems of Events before they happen, but this is risky: if we crash before we can achieve our goal, other systems will believe the Event has happened, where in fact it never took place. For a higher level of security, we will persist the Event before we tell anybody about it.</p> <h2 id="event-naming">Event Naming</h2> <p>Precisely because we observe Domain Events from the outside after they have happened, we should name them in the past tense. We’ll want to stay as close to the language of the domain as possible. This reduces all kinds of translation errors when talking to the business. I recommend using natural language and make small sentences, such as “Stock was depleted”, instead of newspaper-style shortcuts such as “Depleted stock”. In my experience, this greatly improves the communication and understanding.</p> <h2 id="event-contents">Event Contents</h2> <p>A Domain Event can be as little as just a name. More often, it will contain values and identifiers to answer questions about who did what to whom, when, why, where, and how much. The values represent the state at the time the Event happened. Ideally, the Event only contains values that are directly relevant to that Event. A receiver that needs more information, should listen to other relevant Events, and keep its own state, in order to make decisions later. The receiver shouldn’t query the current state of the sender. Remember that the Event message is received some time after it has happened. The state of the sender might already be different from the state it had at the time it emitted the Event.</p> <h2 id="fat-events">Fat Events</h2> <p>In some cases, we can reduce the burden on the receiver to keep track of older state. We add redundant information to the Domain Event, such as (parts of) the previous state of an Entity. The receiver now knows the old value as well as the changed value. A Domain Event can also contain other values, that have not changed, but that are still useful to the receiver. A typical example are labels that accompany an identity (such as a customer’s name, when strictly the Event should only contain the customer’s id). We call these “Fat Events”. There is some risk involved: by ignoring certain types of Events, and getting all its information from redundant values in the Fat Events, it might miss some of the changes. In our example, the receiver might be working with an older version of the customer’s name, because it only listens to sales events and not to events that change the customer’s name.</p> <h2 id="representation">Representation</h2> <p>It is a mistake to design Domain Events as objects first, and then worry about serializing them. It should be the other way around. Domain Events are first and foremost about communication with other systems (or with yourself in the future, when you’ll want to understand what happened in the past – business intelligence comes to mind). Because of this, we’ll want to design them keeping our non-functional requirements in mind: efficiency (binary) versus readability (json), using namespacing to prevent clashes, nested or flat, consumability, … Here’s a simple example:</p> <div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span> <span class="dl">"</span><span class="s2">eventName</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">acme.my_app.StockWasDepleted</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">payload</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span> <span class="dl">"</span><span class="s2">productId</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">2028A516-04C0-4D42-81F4-98A42E2B08BC</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">depletedOn</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">2014-11-11</span><span class="dl">"</span><span class="p">,</span> <span class="dl">"</span><span class="s2">pricePerUnit</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span> <span class="dl">"</span><span class="s2">amount</span><span class="dl">"</span><span class="p">:</span> <span class="mi">500</span><span class="p">,</span> <span class="dl">"</span><span class="s2">currency</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">EUR</span><span class="dl">"</span><span class="p">}</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>The price could be expressed as “EUR 500” instead of a nested structure. The point here is to choose based on what is best for all involved systems, not just what is easiest to produce or consume in the system you are currently working on.</p> <h2 id="internal-representation">Internal representation</h2> <p>The different systems producing and consuming Events can each have their own representation of these events. They can share the code when it’s practical, but that’s not a necessity. They should pick representations that are idiomatic to the environment: tuples and records in Erlang, objects in Java/C#/PHP, … We use objects because they are easier to work with than plain strings. The type of the object represents the type of the Event. We make them immutable, to reflect that messages are immutable. The public getters on the objects should return Value Objects: they encapsulate calculations that you might want to do on the values, and they express the domain model better than primitives can.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li> <li><a href="/2014/07/event-storming-fake-domains-happy-paths/">Event Storming: Fake Domains and Happy Paths</a></li> <li><a href="/2014/05/functional-foundation-for-cqrs-event-sourcing/">A Functional Foundation for CQRS/Event Sourcing</a></li> <li><a href="/2014/02/buzzword-free-bounded-contexts/">Buzzword-free Bounded Contexts</a></li> <li><a href="/2014/01/domain-driven-design-is-linguistic/">Domain-Driven Design is Linguistic</a></li> </ul> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:1" role="doc-endnote"> <p>If your business is online advertising, then a click or even an impression on a website is a Domain Event. After all, that’s what your business is selling. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div> Object Reorientation 2014-11-07T00:00:00+00:00 https://verraes.net/2014/11/object-reorientation <p>Presentation delivered at DrupalCamp Ghent, November 2014</p> <script async="" class="speakerdeck-embed" data-id="776467504884013239ca36e5e480a335" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><small>No slides? <a href="https://speakerdeck.com/mathiasverraes/object-reorientation#">Try here</a></small></p> “Software design is just theory” 2014-10-07T00:00:00+00:00 https://verraes.net/2014/10/software-design-is-just-theory <!-- image http://www.deviantart.com/art/Animal-farm-234426211 --> <p><em>“Software design is just theory.”</em><br /> <em>“Design patterns are too academic.”</em><br /> <em>“Writing code is the only way to become a better programmer.”</em><br /> <em>“Just ship it.”</em><br /> <em>“That’s over-designed.”</em></p> <p>As a consultant, I visit different clients throughout Europe. Some of them hire me for help with greenfield projects. Some others want my help with legacy projects, that they have been working on for typically 4-6 years. They are in different domains, with completely different codebases, but they also share many of the same problems. Progress is slow, they are spending 80% of their time on bug fixing, and each fix seems to create two new regressions. Some of them use popular open source frameworks, others use their own frameworks.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-10-07-software-design-is-just-theory/animal_farm_small.jpg" alt="animal farm by amanda jackson" /></p> <p>Early on in my career, I hated working with legacy just as much as everyone else. We work in an industry that is getting better at shipping large amounts of legacy at an ever increasing pace. We have created outsourcing farms, that produce legacy at the speed of typing. It’s legacy as a service<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>. I felt I had to make a choice: get out, or get better. When you get better at working with legacy, you start enjoying it more. When you enjoy it more, you get better. You get the picture.</p> <p>The 4-6 years is no coincidence: that’s when they start feeling the pain of every design decision that they made or didn’t make. Everything is coupled; moving one part causes a collapse in everything else. The frameworks that once were convenient and fast to market, are now a hindrance that they cannot get rid of. The RAD features that once put stars in the eyes of the developers, are now cursed upon. The business wants to add new features and access new markets, but they can’t. Management doesn’t understand why the team, that was so fast at delivering anything they were asked, is now slow and reluctant. New recruits take six months or more before they are productive — if they even stay that long.</p> <p>Software design has a slow feedback cycle. It takes a long time before your design starts biting you in the butt. Over the years, smarter people than myself have seen the same patterns and anti-patterns, and have built designs that worked or didn’t work. They thought about them, looked at them from every angle, discussed them with colleagues. They investigated them with all the methods at our disposal: observation, logical reasoning, experimentation. They wrote blogs, papers, and books about software and design. Their ideas are not “just theory”. It’s years of accumulated practice and insights. They work — often in many different contexts.</p> <p>We’re lucky: we can build on top of their ideas. You may need to type some more lines of code, but typing is not the bottleneck. You may need to rebuild the same piece of code a couple of times, but surely that’s a better than debugging. Sometimes, you need to over-design it, then take a step back, to understand how much design is just enough. You’ll need to learn a lot. For the rest of your career. You can’t accumulate all that knowledge just by writing some code every day. Read books, discuss them with others, read other people’s code, experiment, draw on a whiteboard, or take a walk and just think.</p> <p>Learning is the bottleneck<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>, and always will be.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/01/build-for-failure/">Build for Failure</a></li> <li><a href="http://www.jefclaes.be/2011/10/gift-of-legacy.html">The Gift of Legacy</a> by <a href="https://twitter.com/JefClaes">@JefClaes</a></li> <li><a href="/2012/12/how-to-read-more-books/">How to Read More Books</a></li> <li><a href="/2011/04/beautiful-code/">Beautiful Code</a></li> </ul> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:1" role="doc-endnote"> <p>Quote by Pieter Hintjens <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:2" role="doc-endnote"> <p>I’m not sure who said this first, but it appears that Ashley Johnson was one of the first. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div> Small Uncontrolled Experiments (revisited) 2014-10-03T00:00:00+00:00 https://verraes.net/2014/10/small-controlled-experiments-revisited <p><a href="/2014/03/small-controlled-experiments/">My blog post on Small Uncontrolled Experiments</a> landed me an invitation to speak at Agile Cambridge. As the slides are probably not that useful if you haven’t seen the presentation, I advise you to read the blog post instead.</p> <p><strong>Update 2021-01-25: I used to call this “small controlled experiments”. It was a name that grew organically and it stuck. Nowadays, I believe they’re very much uncontrolled: a team is too small and a sprint is too short for anything remotely scientific. That doesn’t mean they’re not valuable. Emotions are data, and if people’s gut feeling after (after, not before!) doing an experiment doesn’t agree, the proposed change will not work well anyway. Impose time constraints on these experiments, but do allow them to be uncontrolled.</strong></p> <script async="" class="speakerdeck-embed" data-id="5a5733b02d0e01324430365ebb6a31b9" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><a href="https://speakerdeck.com/mathiasverraes/small-controlled-experiments#"><small>See the slides on Speakerdeck</small></a></p> <h2 id="abstract">Abstract</h2> <p>The project was of to a bad start: an inherited legacy codebase, a waterfall contract, and a projected loss. The promise of Kaizen or Continuous Improvement seemed very appealing. But when we tried to incorporate this into our process, it didn’t catch on. Biweekly retrospectives didn’t seem to expose any problems we could improve upon. The ceremonies we tried, like Deming’s Plan-Do-Check-Act cycles, added too much overhead. We were doing something wrong.</p> <p>Continuous Improvement implies that you know exactly where to focus your efforts. Like scientists, we started to experiment, without deciding upfront what we expected the outcome to be. The rules? Make every experiment as small as possible. No meetings, no consensus, no cumbersome evaluation process. We let the results speak for themselves. This talk explores the successes and failures of a team that went from survival mode to learning mode over the course of a year.</p> <p><a href="https://twitter.com/sgrame">Peter Decuyper</a> drew this while I gave the talk at PHPBenelux 2015:</p> <p><img src="/img/posts/2014-10-03-small-controlled-experiments-revisited/sketchnotes_by_@sgrame-small.jpg" alt="Small Uncontrolled Experiments, drawn by @sgrame" /></p> <h2 id="read-more">Read more</h2> <p><a href="/2014/03/small-controlled-experiments/">The original blog post on Small Uncontrolled Experiments</a></p> Objects as Contracts for Behaviour 2014-09-29T00:00:00+00:00 https://verraes.net/2014/09/objects-as-contracts-for-behaviour <!-- image source http://www.dexigner.com/news/20067 --> <p>A lot of so-called Object-Oriented Programming<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> is in fact nothing more than procedural code wrapped in objects. The funny thing is: many people actually defend this style of programming! This is one of the chief arguments:</p> <p>“<em>Objects should just be bags of state and nothing more. An invoice does not pay itself. An appointment does not reschedule itself.</em><sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup> <code class="language-plaintext highlighter-rouge">invoice.pay(anAmount)</code> <em>and</em> <code class="language-plaintext highlighter-rouge">appointment.reschedule(aDate)</code> <em>do not match the real world. Something on the outside drives the action. A service of some kind should do the paying and the rescheduling, and then update the state of those objects:</em> <code class="language-plaintext highlighter-rouge">invoice.setPaidAmount(anAmount)</code> <em>and</em> <code class="language-plaintext highlighter-rouge">appointment.setDate(aDate)</code>. <em>By consequence, not in the objects, but the services should contain the business logic.</em>”</p> <p>(<strong>Update</strong>: For the record, I do not agree with the view stated above in italics. What follows explains why encapsulating operations in objects, such as invoice.pay(…), is in fact the correct way to model objects.)</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-09-29-objects-as-contracts-for-behaviour/Action_Design_Over_Time_01_gallery_small.jpg" alt="Anaemic objects" /></p> <p>Of course invoices do not pay themselves, but that’s not what an object model is trying to do. An invoice <strong>exposes the behaviour of being able to be paid</strong>. That ability is in fact essential to what it means to be an invoice. The behaviour is an inherent property of an invoice. If an invoice doesn’t have the ability of being paid, there’s no point in issuing invoices at all. In other words, the contract of an invoice object declares that its interface includes payment as a feature. Its promise to the outside world is that it allows an outsider to pay for it.</p> <p>Encapsulation of state and behaviour is the core idea behind objects. Go forth and encapsulate!</p> <h2 id="language">Language</h2> <p>Is the paradigm of Object-Oriented to blame for this widespread misunderstanding, or is it merely a matter of education? Part of the confusion is caused by the language. In languages like English, in an imperative sentence, the verb comes before the object. In OOP languages, it’s the other way around. I’d love to see a programming language that puts method calls before the object, but leaves the method arguments at the end <sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>: <code class="language-plaintext highlighter-rouge">pay.invoice(anAmount)</code> and <code class="language-plaintext highlighter-rouge">reschedule.appointment(aDate)</code>.</p> <h2 id="footnotes">Footnotes</h2> <div class="footnotes" role="doc-endnotes"> <ol> <li id="fn:1" role="doc-endnote"> <p>The term Object-Oriented Programming is rather overloaded. There’s the style that is used in many popular languages like Java and PHP; and there’s the original Smalltalk-style or Alan Kay-style object-oriented programming, which is a misnomer because it puts the focus on messaging and not on objects. I’m talking about the former. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:2" role="doc-endnote"> <p>The traditional argument used to involve “a car doesn’t drive itself”. Obviously by now people have realised that a car should indeed drive itself. Technology has failed us for so long, but change is coming. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> <li id="fn:3" role="doc-endnote"> <p>Functional programming does that to some degree, because it’s all functions operating on data: <code class="language-plaintext highlighter-rouge">pay(invoice, anAmount)</code>. But the <code class="language-plaintext highlighter-rouge">pay</code> function and the invoice data are not strictly coupled to each other. That opens the possibility for another function to manipulate the data without respecting the business rules that pay() implements. Apart from programmer discipline, I haven’t figured out yet if FP allows to somehow combine the benefits of strict encapsulation, with the fluency of the language. <strong>Update</strong> October 8, 2014: Clojure, when working with Java objects. But I guess Clojure stops being Clojure when you start talking to Java :-) <a href="#fnref:3" class="reversefootnote" role="doc-backlink">&#8617;</a></p> </li> </ol> </div> Decoupling the Model from the Framework 2014-09-01T00:00:00+00:00 https://verraes.net/2014/09/decoupling-model-framework <h2 id="video">Video</h2> <iframe width="640" height="360" src="//www.youtube-nocookie.com/embed/QaIGN_cTcc8" frameborder="0" allowfullscreen=""></iframe> <p>The talk that I mention is <a href="https://www.youtube.com/watch?v=0L_9NutiJlc">The Framework as an Implementation Detail</a> by Marcello Duarte and Konstantin Kudryashov.</p> <h2 id="presentations">Presentations</h2> <ul> <li>Laracon Europe in Amsterdam, August 30, 2014 <ul> <li><a href="https://joind.in/talk/view/11693">Joind.in reviews</a></li> </ul> </li> <li>Symfony User Group Belgium, Ghent, January 29, 2014</li> <li>Your company or event? <a href="https://verraes.net/#contact">Contact me</a></li> </ul> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2013/04/crud-is-an-anti-pattern/">CRUD is an anti-pattern</a></li> <li><a href="/2013/04/decoupling-symfony2-forms-from-entities/">Decoupling (Symfony2) Forms from Entities</a></li> <li><a href="/2014/11/domain-events/">Domain Events</a></li> <li><a href="/2014/11/domain-events/">Messaging Flavours</a></li> </ul> Sterile Estimation 2014-08-26T00:00:00+00:00 https://verraes.net/2014/08/sterile-estimation <div style="border: 1px dotted gray; padding: 10px; width: 50%; margin-bottom: 10px;"> <strong>On Estimation</strong> <ol> <li> <a href="/2014/06/cost-of-estimation/">The Cost of Estimation</a> </li> <li> <a href="/2014/08/sterile-estimation/">Sterile Estimation</a> </li> </ol> </div> <p><em>To reduce the risk of incidents, the FAA imposed the so-called “sterile cockpit” regulation. Most air traffic accidents happen while taking off or landing, or in the vicinity of airports. The rule instructs pilots and cabin crew to refrain from non-essential activities and conversations while the plane is below 10.000 feet.</em></p> <h2 id="anchoring-bias">Anchoring Bias</h2> <p>Let’s try an experiment. Do you believe me when I tell you that Mahatma Ghandi died at the ripe old age of 140? Probably not, because that would have been rather spectacular. Just make a guess of what his actual age was, and write it down.</p> <p>Now go to a colleague (you are reading this at work, aren’t you?) and ask them whether Ghandi died at age 9. Then ask them to estimate his age was. My prediction is that their answer will be significantly lower than yours (assuming neither of you knows the real answer). The effect at play here is called the <strong>Anchoring Bias</strong>.</p> <p>Car salesmen have known about it for a long time of course. They’ll start by quoting a high price for a car, more than it’s worth. Because they’ve anchored this number in your mind, any lower price they will quote you next, will seem quite reasonable to you. You are not comparing the number to whatever you’re willing to spend, but to whatever they put in your head. (I bet you can see where I’m going with this.)</p> <p>Tversky and Kahneman (and others), have studied this phenomenon. The book <a href="http://www.amazon.com/gp/product/B00555X8OA/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=B00555X8OA&amp;linkCode=as2&amp;tag=verraesnet-20&amp;linkId=WRTRBQWNNPU4DUAC">“Thinking Fast and Slow”</a> describes some interesting discoveries.</p> <ul> <li>The anchoring bias really works, in all contexts, with all kinds of numbers and estimates. When your manager comes in and asks: “Could you finish this new feature in two weeks?”, your estimate will be somewhere around two weeks.</li> <li>The bias also works when the anchoring number is mentioned out of context. I could say something like “It’s almost five o’clock, I need to go pick up my kids.” Following that, I ask you how long it will take to fix a certain bug. You’re now very likely to answer between 4 and 6.</li> <li>It even works when the numbers are not time related: “I had 50 guests at my parties.” vs “How many M&amp;M’s go in a single pack?”</li> <li>And the strangest thing: it works, <strong>even when you first explain anchoring to somebody, and then ask them to make an estimate!</strong></li> </ul> <h2 id="planning-poker">Planning Poker</h2> <p>Planning Poker is an estimation method based around the idea of preventing anchoring. The team members first discuss a story, and then choose a card with the number of their estimate. They all turn the cards simultaneously. If the numbers vary, a discussion is needed to resolve the difference. You reach consensus, or defer the estimate and await more information.</p> <p>However, with an anchoring bias looming over our heads, and knowing it affects even the most hardened psychologists, we may need to introduce some extra care. It’s very easy to either accidentally influence the results, or manipulate them on purpose. (In fact, some managers believe that if they can trick you into estimating lower, the work will be done faster. And some customers believe that if they can get a project done cheaper, they will still get the same quality. Ah, the joys of our industry!)</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-08-26-sterile-estimation/clock-small.jpg" alt="Sterile Estimation" /></p> <h2 id="sterile-estimation">Sterile Estimation</h2> <p>If we must have estimates, whether time or story point based, we need a clean environment. <strong>Sterile Estimation</strong>, like sterile cockpits, prevent unfortunate accidents. Tell all the participants up front of the problem of anchoring bias, and ask them not to mention numbers. When someone does mention a number, consider whether this contaminates the planning session.</p> <p>After the first round, the consensus estimate is now the new anchor. You can use this to your advantage. Make sure to pick an average story for the first estimation of the session. Even better is to look at some completed stories first. That way, new stories are estimated in relation to older ones. The anchoring bias helps you to keep estimating on the same scale throughout a project.</p> <p>Of course, creating a truly sterile environment is hard. For me, explaining anchoring bias to clients, is just a tool: it illustrates the idea that estimates are not data, but just guesses. Costly, easily manipulated, and sometimes dangerous, guesses.</p> <div style="border: 1px dotted gray; padding: 10px; width: 50%; margin-bottom: 10px;"> <strong>On Estimation</strong> <ol> <li> <a href="/2014/06/cost-of-estimation/">The Cost of Estimation</a> </li> <li> <a href="/2014/08/sterile-estimation/">Sterile Estimation</a> </li> </ol> </div> Resolving Feature Envy in the Domain 2014-08-11T00:00:00+00:00 https://verraes.net/2014/08/resolving-feature-envy-in-the-domain <p><a href="https://twitter.com/beberlei">Benjamin Eberlei</a> did <a href="http://www.whitewashing.de/2014/08/11/spotting_featureenvy_and_refactoring.html">a really nice job of explaining refactoring the Feature Envy code smell</a> on his blog. I wrote a comment because I felt the example could be taken one step further. You should read the original post. Below are Benjamin’s code examples (for reference), followed by a repost of my comment.</p> <h2 id="original-examples">Original examples</h2> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">function</span> <span class="n">calculateReport</span><span class="p">(</span><span class="kt">DateTime</span> <span class="nv">$start</span><span class="p">,</span> <span class="nv">$days</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span> <span class="p">(</span><span class="nv">$days</span> <span class="o">&lt;</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="nc">InvalidArgumentException</span><span class="p">(</span><span class="nv">$days</span><span class="p">);</span> <span class="p">}</span> <span class="nv">$endDate</span> <span class="o">=</span> <span class="k">clone</span> <span class="nv">$start</span><span class="p">;</span> <span class="nv">$endDate</span><span class="o">-&gt;</span><span class="nf">modify</span><span class="p">(</span><span class="s1">'+'</span> <span class="mf">.</span> <span class="nv">$days</span> <span class="mf">.</span> <span class="s1">' day'</span><span class="p">);</span> <span class="c1">// calculation code here</span> <span class="p">}</span> </code></pre></div></div> <p>Benjamin then explains how the calculation of the end date can be moved into a custom DateTime class, thus freeing <code class="language-plaintext highlighter-rouge">calculateReport()</code> from the responsibility:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">DateTime</span> <span class="p">{</span> <span class="c1">// ... other methods</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getDateDaysInFuture</span><span class="p">(</span><span class="nv">$days</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span> <span class="p">(</span><span class="nv">$days</span> <span class="o">&lt;</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="nc">InvalidArgumentException</span><span class="p">(</span><span class="nv">$days</span><span class="p">);</span> <span class="p">}</span> <span class="nv">$endDate</span> <span class="o">=</span> <span class="k">clone</span> <span class="nv">$this</span><span class="p">;</span> <span class="nv">$endDate</span><span class="o">-&gt;</span><span class="nf">modify</span><span class="p">(</span><span class="s1">'+'</span> <span class="mf">.</span> <span class="nv">$days</span> <span class="mf">.</span> <span class="s1">' day'</span><span class="p">);</span> <span class="k">return</span> <span class="nv">$endDate</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="k">function</span> <span class="n">calculateReport</span><span class="p">(</span><span class="kt">DateTime</span> <span class="nv">$start</span><span class="p">,</span> <span class="nv">$days</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$endDate</span> <span class="o">=</span> <span class="nv">$start</span><span class="o">-&gt;</span><span class="nf">getDateDaysInFuture</span><span class="p">(</span><span class="nv">$days</span><span class="p">);</span> <span class="c1">// calculation code here.</span> <span class="p">}</span> </code></pre></div></div> <h2 id="whole-value">Whole Value</h2> <p><code class="language-plaintext highlighter-rouge">calculateReport()</code> is envious of an object that does not yet exist. <code class="language-plaintext highlighter-rouge">DateTime</code> and <code class="language-plaintext highlighter-rouge">$days</code> (an int) belong closely together, they are one concept. Grouping them together into one logical value is a pattern called Whole Value. The implementation of that pattern in OOP is a Value Object.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">function</span> <span class="n">calculateReportFor</span><span class="p">(</span><span class="kt">ReportingPeriod</span> <span class="nv">$reportingPeriod</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// no more period calculation here</span> <span class="c1">// calculation code here.</span> <span class="p">}</span> </code></pre></div></div> <p>By having a single value object that represents a period instead of two values, we have the added benefit that <code class="language-plaintext highlighter-rouge">calculateReportFor()</code> adheres to SRP now: it doesn’t calculate periods, it only calculates reports.</p> <p>So where does the <code class="language-plaintext highlighter-rouge">ReportingPeriod</code> come from? We could just instantiate it:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$startDate</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">DateTime</span><span class="p">(</span><span class="s1">'now'</span><span class="p">);</span> <span class="nv">$endDate</span> <span class="o">=</span> <span class="nv">$start</span><span class="o">-&gt;</span><span class="nf">getDateDaysInFuture</span><span class="p">(</span><span class="nv">$days</span><span class="p">);</span> <span class="nv">$reportingPeriod</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ReportingPeriod</span><span class="p">(</span><span class="nv">$startTime</span><span class="p">,</span> <span class="nv">$endDate</span><span class="p">);</span> </code></pre></div></div> <p>But now we moved to responsibility of calculating the end date to the client, which creates lots of boring duplication. Value objects have the property of attracting behavior. We can add a nice little static factory method to instantiate our <code class="language-plaintext highlighter-rouge">ReportingPeriod</code>:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$reportingPeriod</span> <span class="o">=</span> <span class="nc">ReportingPeriod</span><span class="o">::</span><span class="nf">forDuration</span><span class="p">(</span><span class="nc">DateTime</span> <span class="nv">$startDate</span><span class="p">,</span> <span class="nv">$days</span><span class="p">);</span> </code></pre></div></div> <h2 id="applying-domain-driven-design">Applying Domain-Driven Design</h2> <p>Now let’s take a closer look at the domain. (Of course there’s no real domain in Benjamin’s example code, so I’m going to assume we’re dealing with finance.) Reports in financial statements don’t just cover any random period, they cover months, quarters, or years. Let’s make that explicit:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nf">calculateReportFor</span><span class="p">(</span><span class="nc">ReportingPeriod</span><span class="o">::</span><span class="nf">month</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">2014</span><span class="p">));</span> <span class="nf">calculateReportFor</span><span class="p">(</span><span class="nc">ReportingPeriod</span><span class="o">::</span><span class="nf">quarter</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="mi">2014</span><span class="p">));</span> <span class="nf">calculateReportFor</span><span class="p">(</span><span class="nc">ReportingPeriod</span><span class="o">::</span><span class="nf">year</span><span class="p">(</span><span class="mi">2014</span><span class="p">));</span> </code></pre></div></div> <p>Because calculateReportFor() is decoupled from the calculation of periods, it’s reusable for any kind of period, so the option of passing in arbitrary ReportPeriods is still there if we need it. The important thing is that we are modelling the Ubiquitous Language, staying close to the domain, and have smaller, simpler, more reusable, SRP objects, that are also easier to test in isolation.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/06/named-constructors-in-php/">Named Constructors</a></li> <li><a href="/2014/06/when-to-use-static-methods-in-php/">When to Use Static Methods</a></li> <li><a href="/2014/08/dry-is-about-knowledge/">DRY is about Knowledge</a></li> </ul> DRY is about Knowledge 2014-08-02T00:00:00+00:00 https://verraes.net/2014/08/dry-is-about-knowledge <p>Have a look at these two classes:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="c1">// example 1</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Basket</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$products</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">addProduct</span><span class="p">(</span><span class="nv">$product</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span> <span class="p">(</span><span class="mi">3</span> <span class="o">==</span> <span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="n">products</span><span class="p">))</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="nc">Exception</span><span class="p">(</span><span class="s2">"Max 3 products allowed"</span><span class="p">);</span> <span class="p">}</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">products</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$product</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Shipment</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$products</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">addProduct</span><span class="p">(</span><span class="nv">$product</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span> <span class="p">(</span><span class="mi">3</span> <span class="o">==</span> <span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="n">products</span><span class="p">))</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="nc">Exception</span><span class="p">(</span><span class="s2">"Max 3 products allowed"</span><span class="p">);</span> <span class="p">}</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">products</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$product</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Would you say this is duplicate code? Does it violate the DRY principle (aka “Don’t Repeat Yourself”)?</p> <p>If it is, then the solution to get rid of it, could be something like this:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="c1">// example 2</span> <span class="k">abstract</span> <span class="kd">class</span> <span class="nc">ProductContainer</span> <span class="p">{</span> <span class="k">protected</span> <span class="nv">$products</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">addProduct</span><span class="p">(</span><span class="nv">$product</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span> <span class="p">(</span><span class="mi">3</span> <span class="o">==</span> <span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="n">products</span><span class="p">))</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="nc">Exception</span><span class="p">(</span><span class="s2">"Max 3 products allowed"</span><span class="p">);</span> <span class="p">}</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">products</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$product</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Basket</span> <span class="kd">extends</span> <span class="nc">ProductContainer</span> <span class="p">{}</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Shipment</span> <span class="kd">extends</span> <span class="nc">ProductContainer</span> <span class="p">{}</span> </code></pre></div></div> <p>The code is identical, but why? As good Domain-Driven Design practitioners, we’d have to check with the business. The product could be a flammable chemical, and for safety concerns, a shipment is not allowed to have more than three of them. As a consequence, we don’t allow customers to order more than three at a time.</p> <p>In another scenario, the similarity of the code might also be simple coincidence. Maybe supply of the product is limited, and we want to give our customers equal opportunity to buy it.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-08-02-dry-is-about-knowledge/find-the-differences-small.jpg" alt="Find the differences" /></p> <h2 id="reasons-to-change">Reasons to Change</h2> <p>Whatever the case, it doesn’t matter. The problem is that the rules might change independently. The business might realize that they can still sell more than three products at a time, and divide the products over multiple shipments. In example 2, we are now stuck with high coupling of both types of domain objects, to the same business rule, and indirectly to each other. Changing the rule for Basket, changes the rule for Shipment, but potentially has dangerous consequences in the real world. The example is of course blatantly simple to refactor, but legacy code is usually a lot harder.</p> <p>You might be tempted to solve the problem by abstracting the limit:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="c1">// example 3</span> <span class="k">abstract</span> <span class="kd">class</span> <span class="nc">ProductContainer</span> <span class="p">{</span> <span class="k">protected</span> <span class="nv">$products</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">addProduct</span><span class="p">(</span><span class="nv">$product</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span> <span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">getProductLimit</span><span class="p">()</span> <span class="o">==</span> <span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="n">products</span><span class="p">))</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="nc">Exception</span><span class="p">(</span><span class="nb">sprintf</span><span class="p">(</span><span class="s2">"Max %d products allowed"</span><span class="p">,</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">getProductLimit</span><span class="p">()));</span> <span class="p">}</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">products</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$product</span><span class="p">;</span> <span class="p">}</span> <span class="k">abstract</span> <span class="k">protected</span> <span class="k">function</span> <span class="n">getProductLimit</span><span class="p">();</span> <span class="p">}</span> </code></pre></div></div> <p>This works to certain extent, but the rules might change in unexpected ways. Example 3 assumes that the limit is always easy to determine with a single method call. New rules might take into account the customer, their history with our company, certain legal conditions, promotions, etc. The new rules can dynamically influence the number of products you can buy, in ways that can’t be modelled using our abstract <code class="language-plaintext highlighter-rouge">addProduct()</code> method.</p> <h2 id="knowledge">Knowledge</h2> <p>The business rule in my example is not “Max 3 products allowed”. There are in fact two business rules: “A basket is not allowed to have more than three products” and “A shipment is not allowed to have more than three products”. Two rules, no matter how similar, should have two individual representations in the model.</p> <p>“Don’t Repeat Yourself” was never about code. It’s about knowledge. It’s about cohesion. If two pieces of code represent the exact same knowledge, they will always change together. Having to change them both is risky: you might forget one of them. On the other hand, if two identical pieces of code represent different knowledge, they will change independently. De-duplicating them introduces risk, because changing the knowledge for one object, might accidentally change it for the other object.</p> <p>Looking at the reasons for change, is a very powerful modelling heuristic.</p> <h2 id="read-more">Read more</h2> <p>Follow up posts:</p> <ul> <li><a href="http://blog.ploeh.dk/2014/08/07/why-dry/">Why DRY?</a> by Mark Seemann (<a href="https://twitter.com/ploeh">@ploeh</a>)</li> <li><a href="http://www.devjoy.com/2014/09/aim-for-dry-but-be-willing-to-fall-short/">Aim for DRY, but be willing to fall short</a> by Richard Dalton (<a href="https://twitter.com/richardadalton">@richardadalton</a>]</li> </ul> <p>Some posts about Domain-Driven Design:</p> <ul> <li><a href="https://verraes.net/2014/01/domain-driven-design-is-linguistic/">Domain-Driven Design is linguistic</a></li> <li><a href="https://verraes.net/2014/05/why-domain-driven-design-matters/">Why Domain-Driven Design Matters</a></li> </ul> Event Storming: on Fake Domains and Happy Paths 2014-07-23T00:00:00+00:00 https://verraes.net/2014/07/event-storming-fake-domains-happy-paths <p>Inspired by <a href="http://sch3lp.github.io/2014/07/12/event-storming-exercise/">Tim Schraepen’s experience report</a>, I thought I’d list some recent experiences with facilitating Event Storming sessions. You might want to read my <a href="/2013/08/facilitating-event-storming/">older post with notes on facilitating Event Storming sessions</a>, and the <a href="http://ziobrando.blogspot.be/search/label/EventStorming">huge amount of content Alberto Brandolini has produced on the topic</a>.</p> <h2 id="pick-a-good-domain">Pick a good domain</h2> <p>If you’re doing it as an exercise, as opposed to a real project, pick a real(-ish) domain.</p> <p>I’ve been using a government/education project for a long time, because it’s complex, vague, messy, and in constant motion. It turned out to be great domain to teach participants about challenging your own assumptions. For example, report cards that do not show grades, but observations and conclusions, some of which are written by the actual pupils! The discovery phase is quite useful, but as soon as we tried working towards a more concrete model, the rather “fluffy” domain was too hard to explain. You just can’t learn anything useful about this domain in a few hours.</p> <p>So the last couple of times, when running Event Storming workshops with Jef Claes, Alberto Brandolini, and Stijn Vannieuwenhuyse, we picked a car rental domain. It’s a lot more strict — for example, a car cannot be in two places, driven by two people. That makes it a lot easier to evolve the model with business rules and aggregates. It’s easy to come up with these kinds of business rules, so the role of fake domain expert is easier. That leaves more focus for actually facilitating the exercise.</p> <p>(Last week I started consulting for an actual car rental company. Talk about having my own assumptions challenged!)</p> <p>If you get the opportunity though, apply Event Storming with real projects, and real domain experts. It will be easier than using fake domains and experts. And you’ll all have skin in the game, which will focus the discussions.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-07-23-event-storming-fake-domains-happy-paths/event-storming-wall.jpg" alt="Event Storming" /></p> <h2 id="actors">Actors</h2> <p>I only add roles or actors to an event when they are not obvious. But before I do, I try to find if there’s a concept I’m missing. There could be a hint in the Ubiquitous Language. Often it turns out that it shouldn’t be modelled as one event, caused by different actors. Instead, there are two similar, but slightly different events. If both the customer and the administrator deal with payments, maybe PaymentWasMade is the customer’s event, and PaymentWasRegistered is the administrator’s event. The different language makes it explicit that it’s not the admin making the payment; they’re only recording that it happened.</p> <h2 id="duplication">Duplication</h2> <p>You can discover some kind of cyclic behaviour. An event causes the flow to start over at an earlier point. Or flows branch off into different directions. Or a Process Manager listens to events from different places in the flow. Visualizing this turns into a spaghetti of arrows flying everywhere. And as soon as you draw an arrow, you’ve coupled the position of the sticky to the location of the arrow, making it hard to move them around.</p> <p>Our terrible developer brains, confused and screwed up by years of “Don’t Repeat Yourself”, fail to see the obvious solution: it’s ok to duplicate events. Relax, everything is going to be fine. It’s just stickies you’re copying, not code. Uncle Bob is not going to haunt your nightmares.</p> <p>Write down the same event on as many stickies as you need. Place them wherever they occur or influence something else. Use the same name, and perhaps some visual indicator, to make them easily recognisable. As a facilitator, as soon as I notice an event is central to the model and highly reusable, I make a bunch of copies. I put them on the wall or give them to the participants. They can keep focusing on modelling the flows.</p> <h2 id="happy-path">Happy Path</h2> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-07-23-event-storming-fake-domains-happy-paths/event-storming-with-branches-small.jpg" alt="Happy Path" /></p> <p>Now that we’ve conquered our irrational fear of writing the same event down twice, there’s lots of cool things we can do. Typically, without duplication, you’d show a branches starting from the same place (first picture). With duplication, you can more clearly visualise different flows. Some of them are Happy Paths, i.e. a flow desired to the business or the user. The others are exceptions, edge cases, race conditions, or any possible outcomes. In the second picture, I mark them with green and red stickies.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-07-23-event-storming-fake-domains-happy-paths/event-storming-happy-path-small.jpg" alt="Happy Path" /></p> <p>You can experiment with variations in flow. What happens if a customer pays the invoice before we sent it? What happens when the customer contests it before we sent it? Because it’s so cheap to move stickies around, you trigger discussions with non-technical business people, and gain new insights. It’s domain-driven design in its purest form!</p> <h2 id="visual-language">Visual Language</h2> <p>Model Storming is closer to the idea of brain storming: anything can happen, and there’s no standard notation. When done well, it leads to finding your own visual notation for various concepts. On the other hand, Event Storming uses a rather strict notation and ordered set of steps. The clear rules make it easier for people who are not used to be creative in an everything-is-permitted environment. Still, that shouldn’t stop us from adding some imaginative visuals into the mix. Draw simple icons like smiling or sad customers, lightning, time bombs, various traffic signs, objects from the actual domain,… They speak to the emotional side of the brain, and can convey a lot of information without explanation. You’ll find you will remember the bits with drawings a lot better as well.</p> <h2 id="read-more">Read More</h2> <ul> <li><a href="/2014/11/domain-events/">Domain Events</a></li> <li><a href="/2013/08/facilitating-event-storming/">Notes on facilitating Event Storming sessions</a></li> <li><a href="/2015/03/event-storming-storytelling-visualisations/">Event Storming, Storytelling, Visualisations</a></li> <li><a href="http://ziobrando.blogspot.be/search/label/EventStorming">Posts tagged with Event Storming on Alberto Brandolini’s blog</a></li> <li><a href="http://sch3lp.github.io/2014/07/12/event-storming-exercise/">Tim Schraepen’s Event Storming experience report</a></li> <li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li> </ul> Managed Technical Debt (revisited) 2014-06-27T00:00:00+00:00 https://verraes.net/2014/06/managed-technical-debt-revisited <p>Debt is good. Organizations use it all the time to invest. Technical debt can be good. Are you investing wisely? This talk is based on an <a href="/2013/07/managed-technical-debt/">older blog post of mine</a>.</p> <script async="" class="speakerdeck-embed" data-id="be237370e027013121922af1816f6724" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <h2 id="read-more">Read More</h2> <ul> <li><a href="/2013/07/managed-technical-debt/">Managed Technical Debt</a></li> <li><a href="/2016/04/repair-replace-heuristic-for-legacy-software/">The Repair/Replace Heuristic for Legacy Software</a></li> </ul> Information Overload 2014-06-27T00:00:00+00:00 https://verraes.net/2014/06/information-overload <div class="tags has-addons"> <span class="tag">Update</span> <span class="tag is-warning">2021-05-23</span> <span class="tag "> <a href="/2021/05/its-just-like-heuristic/"> I wrote about the “It's Just Like...” Heuristic which explains this all better. </a> </span> </div> <p>You are reading this on an Information Overload Device. Probably one that fits in your bag or even your pocket. You also have a large Information Overload Device in your living room, and there’s a bunch of others spread across your home, your office, and your environment.</p> <p><img style="margin-bottom: 10px" src="/img/posts/2014-06-27-information-overload/biografias-alicia-martin-2.jpeg" alt="Information Overload" /></p> <p>The way humans deals with information overload, is to scan quickly, and put everything in buckets. We find the similarities with ideas we already know, and then categorize it under the same name. It’s a database that stores entire documents? Oh, then it’s a document database, like the document database I already know about. It has small iterations? It’s an agile methodology like Scrum. It uses the language of the business? It’s like BDD. You get the picture. Categorizing information is often a necessary strategy.</p> <p>We do so much of the “scan, bucket, repeat”-cycles, that we forget to actually process interesting information. My method is simply this:</p> <ul> <li>Be alert;</li> <li>If you catch yourself looking for similarities, stop;</li> <li>Ask: What is different about this? What inspired the author of this information to make a different thing, give it a different name, choose this method over the other method?</li> </ul> <p>The divergence, the conflicts, the raison d’être’s, that is where the learning happens.</p> <p>(Post intentionally kept short to prevent information overload).</p> The Cost of Estimation 2014-06-17T00:00:00+00:00 https://verraes.net/2014/06/cost-of-estimation <div style="border: 1px dotted gray; padding: 10px; width: 50%; margin-bottom: 10px;"> <strong>On Estimation</strong> <ol> <li> <a href="/2014/06/cost-of-estimation/">The Cost of Estimation</a> </li> <li> <a href="/2014/08/sterile-estimation/">Sterile Estimation</a> </li> </ol> </div> <h2 id="ballpark-figures">Ballpark Figures</h2> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-06-17-cost-of-estimation/78704029_3e0e8cf027_z.jpg" alt="Risk" /></p> <p>When somebody asks you to estimate how long a feature or a project will take, you will estimate the <strong>shortest possible time</strong> in which you can complete it. Think about this. Observe your own thoughts while estimating, and you’ll know it’s true. Even if the request was done with the best of intentions, the implicit assumption is not “how long will it take to build it right” but “how fast can you build it”.</p> <p>If you have been burned by this, you move on to phase 2: padding your estimation. You do it secretly, and you feel like a cheat for doing it. For some reason, it’s <strong>a taboo to estimate the time to do it right, as opposed to doing it fast</strong>.</p> <p>At the worst end of the spectrum, there’s <strong>blame culture</strong>. Somebody storms into the office, and demands a “guesstimate” or a “ballpark figure”. Yet they still expect you to complete the task in that time, without even hesitating to dump more tasks in your lap. In the end, you are blamed for “not sticking to the plan” or “making wrong estimates”. Distrust between the team and the rest of business grows. A set of parallel universes spring into existence: reality; the plans of the managers; and the public secret that the plan is wrong anyway.</p> <h2 id="ternary-estimates">Ternary Estimates</h2> <p>Some people try to hack this, using <strong>ternary estimates</strong>: the combination of the expected duration, the optimistic duration, and the pessimistic duration. I’ve never seen this work in practice. Managing software projects is hard enough as it is, having to deal with three numbers all the time doesn’t make it easier. And the numbers are usually calculated (again implicitly) as <code class="language-plaintext highlighter-rouge">expected time +- 20%</code>, which is not the same as a proper optimistic or pessimistic estimate.</p> <h2 id="creative-problem-solving">Creative Problem Solving</h2> <p>All of this is normal. It’s intuitive. Many people ask for estimation without ever giving it any thought. Estimation feels like an effective way to control projects. We measure time and money, because time and money are the easiest to measure. We even fooled ourselves into believing that time <em>is</em> money, and that they can easily be converted back and forth.</p> <p>It’s been said many times before, but it bears repeating: software development is a creative profession. If you do software design right, you solve every problem only once. That means that <strong>every problem is a new problem</strong>. You’ve never solved it before, so you are in the dark as to what the best solution is and how long that will take.</p> <p>And it’s messy: there are many possible solutions, all with different strengths and weaknesses. Some are clearly better, but clearly more expensive, but most of the time, that distinction is not obvious. And not every problem needs the <em>best</em> solution; it needs a solution that is preferably cheaper than the total value. Imagine a multidimensional matrix, with many possible solutions scattered across.</p> <h2 id="the-cost-of-estimation">The Cost of Estimation</h2> <p>Time and creativity are enemies. You can’t solve problems faster. You can’t <a href="http://amzn.to/1iDPNQY">think faster</a>. There’s some wonderful chemistry going on in your brain when you think, and you can’t just speed that up by pushing harder.</p> <p>This is the real cost of estimation: When somebody watches over your shoulder, asking how much longer it will take; when it’s three in the morning and you’re patching a server; in short when you are under pressure, then you <strong>stop solving problems</strong>. Your brain is frantically looking for the quickest thing you can do to make the problem go away. You’re not considering the whole matrix of solutions and consequences, of costs and benefits. It’s like craving for junk food: you’re hungry right now, and you promise yourself you’ll diet later to compensate. But of course, you’re not <a href="https://verraes.net/2013/07/managed-technical-debt/">managing the technical debt you’re creating</a>, because you don’t have time for that either.</p> <p>In other words, <strong>time pressure is the source of much of our legacy</strong>. And boy, do we have legacy in industry! Following from my introduction, estimates often cause or imply time pressure. The interest that builds up on unmanaged technical debt, is accidental complexity: models that do not match the domain, incomprehensible user interfaces, lack of tests, … That, in turn, is costly: a complex system is harder to learn, harder to work with, harder to change. <strong>When the business depends on software that is hard to change, the business itself becomes hard to change.</strong> Before you know it, the next startup comes along, faster, smarter, more agile, and wipes your organization straight from the face of the Earth.</p> <h2 id="why-estimates-are-always-wrong">Why Estimates Are Always Wrong</h2> <p><img style="float:right;margin-left: 10px" src="/img/posts/2014-06-17-cost-of-estimation/1200521270Hine_Icarus_575.jpg" alt="Risk" /></p> <p>If we always estimate the shortest possible time to complete a task, then whenever we are wrong, we will fail upwards: the actual completion can only get longer for every possible bump in the road. There’s little to no happy accidents, causing the task to finish sooner.</p> <p>Why is that? In software, solutions are highly reusable. You automatically take this into account when estimating. “We’ve already built something similar, we can reuse that.” But even though building the first part took longer than you thought, you still assume that this time, there will be no setbacks. We are blind to our own inability to estimate. We never seem to learn that there will always be bumps: from spending half a day looking for a bug, caused by a missing comma somewhere, to losing half a month waiting for the client to deliver some critical piece of information that they said “will be in your mailbox by the end of the day”. As if <em>they</em> somehow magically do know how to estimate!</p> <p>In the interest of making the implicit explicit, let’s call this <strong>“risk”: the probability by which a time based estimate can be wrong</strong>.</p> <h2 id="better-estimation">Better Estimation</h2> <p>We’re getting somewhere now. There is a better way of estimating, by making the ingredients very visible. The formula for estimating in story points, is:</p> <p><del></del></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>story points = time x complexity x risk </code></pre></div></div> <p>&lt;/del&gt;</p> <p>How this works in practice, and how to run estimation sessions, is the topic for future blog post.</p> <p><strong>Update:</strong></p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>story points = (time, complexity, risk) </code></pre></div></div> <p>The first formula makes it look like something mathematically calculable. It isn’t.</p> <div style="border: 1px dotted gray; padding: 10px; width: 50%; margin-bottom: 10px;"> <strong>On Estimation</strong> <ol> <li> <a href="/2014/06/cost-of-estimation/">The Cost of Estimation</a> </li> <li> <a href="/2014/08/sterile-estimation/">Sterile Estimation</a> </li> </ol> </div> When to Use Static Methods 2014-06-13T00:00:00+00:00 https://verraes.net/2014/06/when-to-use-static-methods-in-php <p>Some of the reactions to my last blog post on <a href="/2014/06/named-constructors-in-php/">Named Constructors in PHP</a>, originate from the notion that static methods are inherently bad and should never be used. This is rather overgeneralized.</p> <p>Static methods are nothing more than namespaced global functions. Namespacing, I think we can all agree on, is great. As for global functions: We use those all the time. The native functions in PHP form our basic building blocks.</p> <p>The problem to consider is shared global state. The example that I gave in my previous post, is referentially transparent:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$time</span> <span class="o">=</span> <span class="nc">Time</span><span class="o">::</span><span class="nf">from</span><span class="p">(</span><span class="s2">"11:45"</span><span class="p">);</span> </code></pre></div></div> <p>In other words, it is stateless, it is free of side effects, and as such, it is entirely predictable. You can call the exact same function with the exact same argument as often as you like, and you will always get the exact same result back (a Time instance with value 11:45), no matter the history of the system, no matter the context from which you call it.</p> <p>Another example:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$sum</span> <span class="o">=</span> <span class="nc">Calculator</span><span class="o">::</span><span class="nf">sum</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span> </code></pre></div></div> <p>Again, the outcome is predictable. <code class="language-plaintext highlighter-rouge">Calculator::sum($x, $y)</code> provides a service that is stateless, that doesn’t remember anything, and whose outcome can’t be influenced by anything other than the arguments you put in there. Furthermore, this service will never have polymorphic, or have different implementations. Returning anything other 3 would break the contract. Of course, you could come up with a more efficient algorithm for adding to numbers, but that would have no effect on any of the clients of Calculator.</p> <p>For a counterexample, let’s look at a stateful service:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nc">Counter</span><span class="o">::</span><span class="nf">increment</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="nv">$count</span> <span class="o">=</span> <span class="nc">Counter</span><span class="o">::</span><span class="nf">getCount</span><span class="p">();</span> </code></pre></div></div> <p>This is of course a simple example, but in more complex situations, it can be rather opaque to understand this statefullness. Imagine that one developer use the Counter in one part of the code, and another developer uses it in another part. They both test their code in isolation, and it works fine. As soon as the parts are integrated, the count becomes erratic, because they both share the same global state, instead of each owning a separate Counter. The solution is to have objects here:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$myCounter</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Counter</span><span class="p">;</span> <span class="nv">$myCounter</span><span class="o">-&gt;</span><span class="nf">increment</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span> <span class="nv">$count</span> <span class="o">=</span> <span class="nv">$myCounter</span><span class="o">-&gt;</span><span class="nf">getCount</span><span class="p">();</span> </code></pre></div></div> <h2 id="abstractions">Abstractions</h2> <p>You might still feel resistance against something like <code class="language-plaintext highlighter-rouge">Calculator::sum($x, $y)</code>, because it can not be extended or mocked. Keep in mind however that we are at the lowest level of abstraction here. You can’t extend or mock the + operator either in PHP. Yet you’ve never felt the need for that. If you do need higher levels of abstraction, then composition is your friend. OOP composition is well known, but let’s look at FP really quickly. There is, in PHP, an interesting distinction between <code class="language-plaintext highlighter-rouge">+</code> and <code class="language-plaintext highlighter-rouge">Calculator::sum($x, $y)</code>: the former can not be injected, the latter can:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$result</span> <span class="o">=</span> <span class="nb">array_reduce</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">],</span> <span class="s1">'Calculator::sum'</span><span class="p">,</span> <span class="mi">0</span><span class="p">);</span> <span class="c1">// $result = 6</span> </code></pre></div></div> <p>I’m getting in the realm of higher-order programming now, which is rather out of scope, but let’s just agree that functions and static methods can be very useful, with the right mindset.</p> <h2 id="read-more">Read More</h2> <ul> <li><a href="/2014/06/named-constructors-in-php/">Named Constructors in PHP</a></li> </ul> Named Constructors in PHP 2014-06-12T00:00:00+00:00 https://verraes.net/2014/06/named-constructors-in-php <p>PHP allows only a single constructor per class. That’s rather annoying. We’ll probably never have proper constructor overloading in PHP, but we can at least enjoy some of the benefits. Let’s take a simple Time value object. Which is the best way of instantiating it?</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="s2">"11:45"</span><span class="p">);</span> <span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="mi">11</span><span class="p">,</span> <span class="mi">45</span><span class="p">);</span> </code></pre></div></div> <p>The only correct answer is “it depends”. Both are correct from the point of view of the domain. Supporting both is an option:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$timeOrHours</span><span class="p">,</span> <span class="nv">$minutes</span> <span class="o">=</span> <span class="kc">null</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span><span class="p">(</span><span class="nb">is_string</span><span class="p">(</span><span class="nv">$timeOrHours</span><span class="p">)</span> <span class="o">&amp;&amp;</span> <span class="nb">is_null</span><span class="p">(</span><span class="nv">$minutes</span><span class="p">))</span> <span class="p">{</span> <span class="k">list</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="n">hours</span><span class="p">,</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">minutes</span><span class="p">)</span> <span class="o">=</span> <span class="nb">explode</span><span class="p">(</span><span class="nv">$timeOrHours</span><span class="p">,</span> <span class="s1">':'</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span> <span class="p">}</span> <span class="k">else</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">hours</span> <span class="o">=</span> <span class="nv">$timeOrHours</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">minutes</span> <span class="o">=</span> <span class="nv">$minutes</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>This is terribly ugly. It makes using the Time class rather confusing. And what happens if we need to add more ways to instantiate Time?</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$minutesSinceMidnight</span> <span class="o">=</span> <span class="mi">705</span><span class="p">;</span> <span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$minutesSinceMidnight</span><span class="p">);</span> </code></pre></div></div> <p>Or if we want to support numeric strings as well as integers?</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="s2">"11"</span><span class="p">,</span> <span class="s2">"45"</span><span class="p">);</span> </code></pre></div></div> <p>(Note: in production code, I would make my Time class a lot more idiot-proof.)</p> <h2 id="refactor-to-named-constructors">Refactor to named constructors</h2> <p>Let’s add some public static methods to instantiate Time. This will allow us to get rid of the conditionals (which is always a good thing!).</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">hours</span> <span class="o">=</span> <span class="p">(</span><span class="n">int</span><span class="p">)</span> <span class="nv">$hours</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">minutes</span> <span class="o">=</span> <span class="p">(</span><span class="n">int</span><span class="p">)</span> <span class="nv">$minutes</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromString</span><span class="p">(</span><span class="nv">$time</span><span class="p">)</span> <span class="p">{</span> <span class="k">list</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span> <span class="o">=</span> <span class="nb">explode</span><span class="p">(</span><span class="nv">$time</span><span class="p">,</span> <span class="s1">':'</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span> <span class="k">return</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span> <span class="p">}</span> <span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromMinutesSinceMidnight</span><span class="p">(</span><span class="nv">$minutesSinceMidnight</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$hours</span> <span class="o">=</span> <span class="nb">floor</span><span class="p">(</span><span class="nv">$minutesSinceMidnight</span> <span class="o">/</span> <span class="mi">60</span><span class="p">);</span> <span class="nv">$minutes</span> <span class="o">=</span> <span class="nv">$minutesSinceMidnight</span> <span class="o">%</span> <span class="mi">60</span><span class="p">;</span> <span class="k">return</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Every method now satisfies the Single Responsibility Principle. The public interface is clear and understandable interface, and the implementations are straightforward. Are we done?</p> <p>Well, something is bothering me: <code class="language-plaintext highlighter-rouge">__construct($hours, $minutes)</code> kinda sucks: it exposes the internals of the Time value object, and we can’t change the interface because it is public. Imagine that for some reason, we want Time to store the string representation and not the individual values.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$time</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">time</span> <span class="o">=</span> <span class="s2">"</span><span class="nv">$hours</span><span class="s2">:</span><span class="nv">$minutes</span><span class="s2">"</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromString</span><span class="p">(</span><span class="nv">$time</span><span class="p">)</span> <span class="p">{</span> <span class="k">list</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span> <span class="o">=</span> <span class="nb">explode</span><span class="p">(</span><span class="nv">$time</span><span class="p">,</span> <span class="s1">':'</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span> <span class="k">return</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span> <span class="p">}</span> <span class="c1">// ...</span> <span class="p">}</span> </code></pre></div></div> <p>This is ugly: we go through all the trouble of splitting up the string, only to rebuild it in the constructor.</p> <p>Do we even need a constructor now that we have named constructors? Of course not! They are just an implementation detail, that we want to encapsulate behind meaningful interfaces. So we make it private:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">;</span> <span class="k">private</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">hours</span> <span class="o">=</span> <span class="p">(</span><span class="n">int</span><span class="p">)</span> <span class="nv">$hours</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">minutes</span> <span class="o">=</span> <span class="p">(</span><span class="n">int</span><span class="p">)</span> <span class="nv">$minutes</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromValues</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span> <span class="p">}</span> <span class="c1">// ...</span> <span class="p">}</span> </code></pre></div></div> <p>Now that the constructor is no longer public, we can choose to refactor all the internals of Time as much as we want. For example, sometimes you’ll want every named constructor to assign properties without passing them through a constructor:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">;</span> <span class="c1">// We don't remove the empty constructor because it still needs to be private</span> <span class="k">private</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(){}</span> <span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromValues</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">;</span> <span class="nv">$time</span><span class="o">-&gt;</span><span class="n">hours</span> <span class="o">=</span> <span class="nv">$hours</span><span class="p">;</span> <span class="nv">$time</span><span class="o">-&gt;</span><span class="n">minutes</span> <span class="o">=</span> <span class="nv">$minutes</span><span class="p">;</span> <span class="k">return</span> <span class="nv">$time</span><span class="p">;</span> <span class="p">}</span> <span class="c1">// ...</span> <span class="p">}</span> </code></pre></div></div> <h2 id="ubiquitous-language">Ubiquitous Language</h2> <p>Our code begins to clean up nicely, and our Time class now has some very useful ways of being instantiated. As it happens with better design, other, previously hidden design flaws, start to become visible. Look at the interface for Time:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$time</span> <span class="o">=</span> <span class="nc">Time</span><span class="o">::</span><span class="nf">fromValues</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span> <span class="nv">$time</span> <span class="o">=</span> <span class="nc">Time</span><span class="o">::</span><span class="nf">fromString</span><span class="p">(</span><span class="nv">$time</span><span class="p">);</span> <span class="nv">$time</span> <span class="o">=</span> <span class="nc">Time</span><span class="o">::</span><span class="nf">fromMinutesSinceMidnight</span><span class="p">(</span><span class="nv">$minutesSinceMidnight</span><span class="p">);</span> </code></pre></div></div> <p>Notice anything? We’re mixing no less than three languages:</p> <ul> <li><code class="language-plaintext highlighter-rouge">fromString</code> is a PHP implementation detail;</li> <li><code class="language-plaintext highlighter-rouge">fromValues</code> is a sort of generic programming term;</li> <li>and <code class="language-plaintext highlighter-rouge">fromMinutesSinceMidnight</code> is part of the domain language.</li> </ul> <p>Being a language geek and Domain-Driven Design aficionado, I can’t let this pass. As Time is part of our domain, my preferred style is to find inspiration in the Ubiquitous Language.</p> <ul> <li><code class="language-plaintext highlighter-rouge">fromString</code> =&gt; <code class="language-plaintext highlighter-rouge">fromTime</code></li> <li><code class="language-plaintext highlighter-rouge">fromValues</code> =&gt; <code class="language-plaintext highlighter-rouge">fromHoursAndMinutes</code></li> </ul> <p>(If you worry about the extra characters you’d need to type, get an editor with contextual code completion.)</p> <p>This focus on the domain, gives you some great options:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$customer</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Customer</span><span class="p">(</span><span class="nv">$name</span><span class="p">);</span> <span class="c1">// We can't "new a customer" or "instantiate a customer" in real life.</span> <span class="c1">// Better:</span> <span class="nv">$customer</span> <span class="o">=</span> <span class="nc">Customer</span><span class="o">::</span><span class="nf">fromRegistration</span><span class="p">(</span><span class="nv">$name</span><span class="p">);</span> <span class="nv">$customer</span> <span class="o">=</span> <span class="nc">Customer</span><span class="o">::</span><span class="nf">fromImport</span><span class="p">(</span><span class="nv">$name</span><span class="p">);</span> </code></pre></div></div> <p>Granted, that’s not always better. In the case of Time, I might stick to fromString, because maybe at this level of detail in our code, we want to serve the programmer more than the domain. I might even provide both options. But at least, thanks to named constructors, we now <em>have</em> options.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/06/when-to-use-static-methods-in-php/">When to Use Static Methods</a></li> <li><a href="/2011/03/accessing-private-properties-from-other-instances/">Accessing private properties from other instances</a></li> <li><a href="/2013/02/casting-value-objects/">Casting Value Objects to String</a></li> <li><a href="/2014/05/final-classes-in-php/">Final Classes: Open for Extension, Closed for Inheritance</a></li> </ul> Why Domain-Driven Design Matters 2014-05-21T00:00:00+00:00 https://verraes.net/2014/05/why-domain-driven-design-matters <script async="" class="speakerdeck-embed" data-id="318140a0c29a013152315aee9e57b660" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p>See the slides at <a href="https://speakerdeck.com/mathiasverraes/why-domain-driven-design-matters">Speakerdeck</a></p> <h2 id="abstract">Abstract</h2> <p>In the software industry, the life expectancy of ideas, methodologies, and technologies, is extremely short. And yet, after ten years, Domain-Driven Design is still growing bigger. From it’s original roots in OOP, it has now expanded into Functional Programming, Reactive Programming and Event Sourcing, and architectural styles such as Hexagonal and CQRS. Clearly something about Domain-Driven Design makes it such an appealing choice to build systems for complex domains.</p> <p>In this session, we’ll discuss what DDD is: from design patterns and modelling techniques, to the more philosophical ideas about how we deal with complexity. We explore why it has made such a profound impact, and how to decide whether it’s right for your project. We’ll have lots of room for open discussion, to make sure all your questions are answered.</p> <h2 id="presentations">Presentations</h2> <ul> <li><a href="http://akamon.com/">Akamon</a> in Barcelona, Spain May 20, 2014</li> </ul> <h2 id="read-more">Read More</h2> <ul> <li><a href="https://verraes.net/2013/08/john-gall-systemantics-the-systems-bible/">The Systems Bible</a></li> </ul> A Functional Foundation for CQRS/ES 2014-05-14T00:00:00+00:00 https://verraes.net/2014/05/functional-foundation-for-cqrs-event-sourcing <p>I’ve been dabbling with Functional Programming languages for a while now. Although I can’t say I’m at all proficient, it has affected my programming style. Even if my code is mostly OOP, my whiteboard drawings look more functional. I’m <a href="/2014/01/domain-driven-design-is-linguistic/">thinking about functions a lot</a>. So it made sense to describe the CQRS-architecture of my system as a set of functions.</p> <p>Greg Young describes Command Query Responsibility Segregation as “using two objects where previously there was only one”. The idea is as ingenious as it is simple, but I feel the quote doesn’t do it justice. Rinat Abdullin proposed in a <a href="https://twitter.com/abdullin/status/465747026953908225">Twitter conversation</a> with Eric Evans, the idea that CQRS enables you to denormalize your model. The value lies in the fact that, because you have independent read and write models, you can reason more clearly about your model. The separation opens up opportunities to capture deeper insights. You can now optimise the two sides separately.</p> <p>Even though you can use CQRS without Event Sourcing, this post assumes Domain Events all the way down. In a brainstorming session with <a href="https://twitter.com/stijnvnh">@stijnvnh</a>, we identified four (conceptual) functions.</p> <h2 id="protection">Protection</h2> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>f(history, command) -&gt; events (where history is the series of past events for a single Aggregate instance) </code></pre></div></div> <p>Commands and Domain Events are first class citizens of our model. Aggregates no longer expose state, but only events. Aggregates are at the core of the decision making process. They decide what the outcome of a Command is, and express that outcome with Domain Events. The history of that particular Aggregate instance, influences the decision, so we must inject it. If you want to split the reconstitution of the Aggregate instance from the handling of the command, you can use currying. This helps to keep infrastructural and model concerns separated:</p> <p><code class="language-plaintext highlighter-rouge">f(history)(command) -&gt; events</code></p> <p>‘Protection’ refers to the primary goal of the Aggregates: protecting its business rules, making sure that the user can not cause the system to go in an inconsistent or illegal state. When a Command violates the business rules, the operation should fail:</p> <p><code class="language-plaintext highlighter-rouge">f(history, command) -&gt; either&lt;exception|events&gt;</code></p> <p>(The error handling part of FP is still a bit terra incognita for me though.)</p> <p>The exact same idea, can be expressed in BDD-style scenario’s: Given this history, when this command is received, then these events should happen.</p> <h2 id="interpretation">Interpretation</h2> <p><code class="language-plaintext highlighter-rouge">f(events) -&gt; state</code></p> <p>As the Aggregates do not expose, something else must. The function above is a Projector. It takes (a stream of) events, and projects these into something that represents a subset of the perceived state of our system. That state makes up our read models for further consumption. I like the idea of “interpretation” here: Two different Projectors could have a completely different interpretation of what the meaning of a set of behaviours is. This is similar to how a journalist or politician interprets a series of events, and comes up with a different conclusion, or at least a different focus, than her colleagues.</p> <p>Of course these interpretations are based in the domain. The beauty of the decoupling from the write model, is that when the domain evolves, or we gain deeper insight, it’s now easy to shift the interpretation. Make a new projector and a new read model, and replay the history of the system into them.</p> <p>You can express the projection as a left fold:</p> <p><code class="language-plaintext highlighter-rouge">f(events) -&gt; foldLeft(apply, events, init)</code></p> <h2 id="intention">Intention</h2> <p><code class="language-plaintext highlighter-rouge">f(state) -&gt; command</code></p> <p>One of the major consumers of the read models, is usually an end user. Many applications need to somehow report state (projected in the previous section). The user looks at the state, and makes decisions. The user then makes his intention known to domain model, by sending commands. And it is an intention indeed, because, as we saw earlier, the Aggregate might refuse the command later.</p> <p>Note that we don’t present the user with the list of events, (unless that temporality is important). Humans are pretty bad at deducing state from events – so never base your opinion on a single journalist or politician! In the case of our application, we give the user ready-made, automated interpretations.</p> <p>It’s the only function presented here that is not pure. Human users are influenced by lots of global state, they have ideas, principles, mood swings, and <a href="/2014/01/domain-driven-design-is-linguistic/">biases</a>. It’s like they have a mind of their own! Modelling the end user is pretty much impossible…</p> <h2 id="automation">Automation</h2> <p><code class="language-plaintext highlighter-rouge">f(events) -&gt; command</code></p> <p>… but sometimes it is! Humans in organisations behave predictably in certain circumstances. Many tasks in an organisation are repeatable: “Whenever X and Y have happened, Z needs to happen.” Automating these business processes, making them faster and safer, is one of the appeals of enterprise software. Process Managers take on this job: they are event listeners, that react to certain events or combinations of events. They may hold intermediate state, so internally, they look like projectors. But the state is invisible. The only observable output are new commands.</p> <p>In some cases, a Process Manager can output events straight away. This makes sense when there are no invariants other than what the Process Manager can take care of on its own. But most of the time, you’ll want a proper Command that can be refused by an Aggregate.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-05-14-functional-foundation-for-cqrs/circle-diagram-small.png" alt="Functional Foundation for CQRS" /></p> <h2 id="conclusion">Conclusion</h2> <p>What is the point of all of this? Thinking about <a href="https://verraes.net/2013/08/john-gall-systemantics-the-systems-bible/">how systems work</a>, is one my hobbies. Having a good theoretical foundation, has often helped me when designing systems. I may not end up doing things according to the theory, but the theory forces me to think hard about what I’m doing.</p> <p>Pure functions are deterministic for the same inputs and outputs. Independent of shared mutable state, a pure function can isolated in a separate process, on a separate server, doin’ his thang. See where this is going?</p> <h2 id="update-june-8-2014">Update June 8, 2014</h2> <p>I wrote that user intention can not be represented as a pure function, but of course you simply need to pass in the state of the human, like this:</p> <p><code class="language-plaintext highlighter-rouge">f(brain, state) -&gt; command</code></p> <h2 id="update-dec-17-2016">Update Dec 17, 2016</h2> <p>My model completely lacked Queries. Having state without being able to query it is rather useless. Like Events and Commands, a <a href="/2015/01/messaging-flavours/">Query is just a message</a>. It fits in the Query Model (duh) aka Read Model like this: <code class="language-plaintext highlighter-rouge">f(history, query) -&gt; state</code> where state is a subset of the all the state that the projector writes. In CQRS, I think ideally, a projector should have <strong>private state</strong>, meaning that the projector is the only one that read and write its own (logical) private database. All clientss get state by sending one of the projector’s supported Query messages. Another type of query is a Subscription, as sometimes a client is interested in a stream of (filtered) events as opposed to state.</p> <p>So this represents my current thinking here:</p> <div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">handle</span> <span class="o">::</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="o">-&gt;</span> <span class="kt">Command</span> <span class="o">-&gt;</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="n">subscribe</span> <span class="o">::</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="o">-&gt;</span> <span class="kt">SubscriptionQuery</span> <span class="o">-&gt;</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="n">process</span> <span class="o">::</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="o">-&gt;</span> <span class="p">[</span><span class="kt">Command</span><span class="o">|</span><span class="kt">Event</span><span class="p">]</span> <span class="n">query</span> <span class="o">::</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="o">-&gt;</span> <span class="kt">Query</span> <span class="o">-&gt;</span> <span class="kt">State</span> </code></pre></div></div> <p>or with intermediate states made explicit:</p> <div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">project</span> <span class="o">::</span> <span class="kt">State</span> <span class="o">-&gt;</span> <span class="kt">Event</span> <span class="o">-&gt;</span> <span class="kt">State</span> <span class="n">query'</span> <span class="o">::</span> <span class="kt">State</span> <span class="o">-&gt;</span> <span class="kt">Query</span> <span class="o">-&gt;</span> <span class="kt">State</span> </code></pre></div></div> <p>And modelling the human:</p> <div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">learn</span> <span class="o">::</span> <span class="kt">State</span> <span class="o">-&gt;</span> <span class="kt">Brain</span> <span class="o">-&gt;</span> <span class="kt">Query</span> <span class="n">decide</span> <span class="o">::</span> <span class="kt">State</span> <span class="o">-&gt;</span> <span class="kt">Brain</span> <span class="o">-&gt;</span> <span class="p">[</span><span class="kt">Command</span><span class="p">]</span> </code></pre></div></div> <h2 id="read-more">Read More</h2> <ul> <li><a href="/2014/11/domain-events/">Domain Events</a></li> <li>If you like the circular representation of CQRS, I have a lot more of them in my slides for <a href="/2013/12/fighting-bottlenecks-with-cqrs/">Fighting Bottlenecks with CQRS</a></li> <li><a href="/2014/01/domain-driven-design-is-linguistic/">Domain-Driven Design is Linguistic</a></li> <li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li> </ul> Final Classes: Open for Extension, Closed for Inheritance 2014-05-12T00:00:00+00:00 https://verraes.net/2014/05/final-classes-in-php <p>I make all my classes final by default. I even configured the templates in my IDE prefix new classes with ‘final’. I’ve often explained my reasoning to people. A blog post is in order!</p> <h2 id="clarity-of-intent">Clarity of Intent</h2> <p>A guiding principle here is Clarity of Intent. Making a machine do what we want is not so hard. The reason we need clean code, is not for the compiler. It’s to help our fellow developers, third parties, and even ourselves in six months time, understand the purpose and the design of our system. Many design principles help us with that: picking good names for our elements, create low coupling and high cohesion, SOLID principles, …</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-05-12-final-classes-php/explicit-interface-small.png" alt="Explicit escalator interface" /></p> <p>Code is often part of something bigger. This can get really complex, really quickly. The constraints we impose, help us to limit the cognitive load for developers working with our code. We encapsulate the details, and expose only what outside code needs to talk to us.</p> <p>Another such constraint is the Open/Closed Principle. It states that software entities should be open for extension, but closed for modification. Closed for modification, in this context, means that when your code exposes some behavior to the outside world, that interface should be stable. Providing an API is a responsibility: by allowing other code to access features of your system, you need to give certain guarantees of stability or low change frequency. The behaviour should be deterministic. It does not mean your implementation can not change. The changes should not affect outside code.</p> <p>It’s important to understand that “Open for extension”, does not mean “Open for inheritance”. Composition, strategies, callbacks, plugins, event listeners, … are all valid ways to extend without inheritance. And usually, they are much preferred to inheritance – hence the adage “favour composition over inheritance”. The latter creates more coupling, that can be hard to get rid of, and that can make understanding the code quite tough.</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2014-05-12-final-classes-php/not_how_to_extend_a_car.jpg" alt="Not how to extend a car" /></p> <p>Extension points are part of your public API. By making a class inheritable, you are saying to the outside world: this class is meant to be extended. In some cases that may be the best option, but it is not the only option. Declaring explicitly what the extension points are, is part of the contract your code has with the rest of the system. Final classes help to enforce this contract.</p> <p>The fewer behaviours and extension points you expose, the more freedom you have to change system internals. This is the idea behind encapsulation.</p> <h2 id="php">PHP</h2> <p>Making a class final has some drawbacks. For example, you cannot mock it anymore with PHPUnit. This is a limitation of the infrastructure. One option is to remove the final keyword, but add an @final annotation. We loose the strict enforcement, but we still communicate intent. The class is still conceptually closed down for inheritance.</p> <p>Similarly, we can mark elements with @internal or @api. @internal indicates to the outside world that, even though you can technically access the element, you’re not supposed to use or consume it outside of the boundaries of the module or library. @api means it is part of the public contract and can be trusted not to break easily.</p> <h2 id="conclusion">Conclusion</h2> <p>Still in doubt? Consider this: when a class is closed for inheritance, or a method is private, or an element is @internal, you are free to change it or remove it. This freedom removes a lot of the burden of refactoring. Should a valid reason come up to open the class or method, it will be easy to do so, because nothing depends on it being closed. On the other hand, if you start by making everything open or inheritable, it will be very hard to close it later. Somewhere, perhaps invisible to you, something might depend on it. Closing it, or even making a small change, will break that code.</p> <p>Finally, I have a hypothesis. If, like some other languages, classes in php were final by default, and needed an “inheritable” keyword to open them up, many people who are now opposed to the final keyword, would have no problem with it at all.</p> <p><strong>Update June 24, 2014</strong> From <a href="http://programmers.stackexchange.com/a/92771/29370">Programmers Exchange</a></p> <blockquote>"(...) designing a class that can be meaningfully inherited from takes more than just removing a final specifier; it takes a lot of care." &mdash; Konrad Rudolph </blockquote> Small Uncontrolled Experiments 2014-03-22T00:00:00+00:00 https://verraes.net/2014/03/small-controlled-experiments <p><strong>Update 2021-01-25: I used to call this “small controlled experiments”. It was a name that grew organically and it stuck. Nowadays, I believe they’re very much uncontrolled: a team is too small and a sprint is too short for anything remotely scientific. That doesn’t mean they’re not valuable. Emotions are data, and if people’s gut feeling after (after, not before!) doing an experiment doesn’t agree, the proposed change will not work well anyway. Impose time constraints on these experiments, but do allow them to be uncontrolled.</strong></p> <p>You’re in a meeting. Maybe it’s a daily stand-up meeting, or an agile retrospective. Somebody has an idea. A heated debate follows. Everybody is stating opinions, trying to convince the others, explaining why their idea is going to work, and the alternative is stupid. Because everybody is focused on making their point, nobody is actually listening to the others. Eventually the decision is either postponed, or the person with the highest salary makes the final decision, or the one who pushes the hardest. The decision is final, and it may or may not be actually get implemented. And if it is, it is never properly evaluated, or adapted.</p> <p>I humbly admit to being guilty of having done all of these. And it’s all such a terrible waste.</p> <p>The cure is to set up <strong>small uncontrolled experiments</strong>.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-03-22-small-controlled-experiments/chemistry-show.jpg" alt="experiment" /></p> <h2 id="continuous-improvement">Continuous Improvement</h2> <p>I used to think that simply trying to do a good job, was sufficient to incrementally improve the way you do things in your organisation, your process, your culture. Having worked closely with <a href="http://twitter.com/VincentVanderh">@VincentVanderh</a> for a year, I’ve learned to appreciate that, even when you think you’re doing just fine and you work with the greatest people, a relentless focus on making small improvements every day, is going to pay off massively.</p> <p>The basic principle is to identify problems, find the simplest thing you could try to improve it, try it for a short time, and evaluate. After that, you either implement the change permanently, dismiss it, or try a new cycle. The idea is of course not new. Deming calls it the <em>PDCA-cycle (Plan-Do-Check-Act)</em>. The Japanese, spearheaded by Toyota, have made <em>Kaizen</em>, meaning “change for the best”, a central part of business culture.</p> <p>Our first attempts were clumsy at best. We had improved a lot since the start of the project, so we felt we didn’t need a formal process. We didn’t identify major problems, let alone major solutions. We were ready to dismiss the whole idea of PDCA as just another management buzzword.</p> <p>We were partly right. The process was too formal for our context, and the last thing you want to do is burden knowledge workers in a tight schedule with more processes to worry about. More importantly, we were trying too hard: we focused on finding major, critical problems, and grand, all-encompassing solutions. And that only works in the movies. We did keep trying however. We came up with experiments, wrote them on a sticky, put them in a corner of the whiteboard, and started doing them. Well, sometimes. More often than not, the stickies hung there for a while, until someone would feel they were pointless, and they were thrown out. Some ideas did get implemented – after all we had been improving somewhat steadily before we had a name for it – but it was all rather vague and unpredictable.</p> <h2 id="make-it-lightweight">Make it lightweight</h2> <p>Some changes had to be made. The biweekly retrospectives made the improvement process too slow. By this time, we had pretty much gotten rid of Scrum(ish) sprints, in favour of a more continuous Kanban-flow. Saving up all the improvement ideas and discussion for the retrospective, started feeling like a remnant from the biweekly sprint cadence, something that did no longer fit in our flow. So we needed the benefits of evaluating our process, without the slowness of retrospectives.</p> <p>We changed the rules of the experiments:</p> <ul> <li> <p>You can write down any idea for an experiment on a sticky and put it in the drop zone of the Kanban board.</p> </li> <li> <p>It will be discussed at the stand-up meeting, the following morning.</p> </li> <li> <p>The discussion is kept as short as possible. You’re not asking if it will work – the point is that the experiment will prove if the idea works. You’re answering the questions: “Will we try this? For how long? Can we do it in a cheaper, lower effort way, that doesn’t affect our velocity?”</p> </li> <li> <p>The sticky is posted on a timeline, usually about 10-14 days in the future.</p> </li> <li> <p>Every day during stand-up, we check whichever stickies are on today’s slot on the timeline.</p> </li> <li> <p>The experiment is evaluated, we discuss positive or negative side-effects, and decide on making the change permanent or trying more experiments.</p> </li> </ul> <p>A number of things turned out to be critical here:</p> <ol> <li> <p>It works best if the experiment takes as little effort as possible. Most of them do not take any time at all, as they only made a small change to something we were already doing. Others take a minute a day, or even a minute a week. The <strong>“small”</strong> in “small uncontrolled experiment”.</p> </li> <li> <p>Sometimes one person is assigned to an experiment, but usually it’s <strong>for the whole team</strong>.</p> </li> <li> <p>When starting an experiment, make sure everybody understands that the experiment is <strong>limited in time</strong>, and that they <strong>will be heard</strong> and allowed veto when evaluating it. This is especially important with more <strong>introvert team members</strong>, especially if you have one or two loudmouths like yours truly.</p> </li> <li> <p>Make sure the experiments are <strong>for everybody</strong>. I tended to focus on improving conditions for the developers, but of course, testers, analyst, the project manager, … should all benefit.</p> </li> <li> <p>Deciding, and <strong>visualising exactly when the evaluation will happen</strong>, is probably the best thing we did. It gives a sense of urgency and deadline to the experiment, preventing it from slowly fading into oblivion.</p> </li> <li> <p>We’d usually have a couple of experiments running <strong>simultaneously</strong> at any given time.</p> </li> </ol> <p>Here’s a sample of some of the things we experimented with, to give you some ideas. Some happened before we tried PDCA or Kaizen, before we settled on the small uncontrolled experiments above, but making it a core part of our standup has greatly increased the rate of improvement.</p> <h2 id="hacking-our-productivity">Hacking our productivity</h2> <ul> <li> <p>When a story was estimated as a big one, say an 8-pointer, more time would be spent on it to refactor it. When it was a 3-pointer, fewer time was spent. I’m all for <a href="/2013/09/extract-till-you-drop/">refactoring</a>, but not everything requires the <a href="/2013/07/managed-technical-debt/">same level of perfection</a>. We wanted the quality of a story to depend on value, not on the original estimate. In the experiment, we moved the estimated number from the front of the sticky, to the back. Because of this, the developers could no longer see the estimation, so they were no longer influenced by it. We made the experiment permanent.</p> </li> <li> <p>We noticed how at the end of a two-week sprint, the pace would speed up to reach the sprint target, but as soon as it was reached, the tempo slowed down. The fact that the last day was a Friday, didn’t help either. Kanban came to rescue here: We stopped setting sprint targets, set strict work-in-progress limits, and made sure we had a pull-based system. (More on Kanban in a future post.)</p> </li> <li> <p>The testers work for multiple teams. As our delivery of stories was very irregular, they couldn’t predict when they would need time to test our stories. This caused stories to be blocked in testing, creating too much of a gap between development and testing. We experimented with the commitment to deliver exactly one story a day. This worked great for predictability for the testers, but the unwanted side-effects were that stories were being prioritized to reach this goal, along with other subtle ways to cheat the system. The experiment was stopped.</p> </li> </ul> <h2 id="technical-debt">Technical Debt</h2> <ul> <li> <p>The <a href="https://verraes.net/2013/07/managed-technical-debt/">Wall of Technical Debt</a> started as an experiment. Cost: a bunch of stickies, and a few minutes. Gain: Having a complete map of everything in the system that needs improving, how to do it, and how important it is.</p> </li> <li> <p>To reduce technical debt, we’ve tried different approaches. We’re now on a mix of:</p> <ul> <li>having a limit of one technical debt ticket in progress at any time,</li> <li>yet being allowed to pull a technical ticket, if we judge it would be really beneficial to the functional story that we’re working on.</li> </ul> </li> </ul> <h2 id="visualisation">Visualisation</h2> <ul> <li>The Kanban board is regularly redrawn as we get more insight in our process. I think we are in the seventh iteration by now. I think it is beautiful :-) Many of the changes may seem minimal at first, but the accumulated effect is that this board is pure bliss to work with. One example is that the “expedite” swimming lane moved from an afterthought at the bottom of the board, to a prominent place at the top.</li> </ul> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-03-22-small-controlled-experiments/kanbanboard-small.jpg" alt="Kanban board" /></p> <ul> <li> <p>We fooled around with the idea from Core Protocols to make our emotions explicit during standup: “I’m glad-mad-sad-afraid that …”. We had no idea what we expected of this experiment, but we felt it was worth trying (it is, once again, dead cheap to try for few weeks). Current status: undecided. The PM will research it better and we’ll try it again.</p> </li> <li> <p>Hanging up a monitor to show build statuses, sending mails when builds fail, … nothing extraordinary there, as more and more teams do CI, but you need to make an effort to get it set up.</p> </li> <li> <p>We’ve experimented with different ways of visualising our metrics. It turns out that the best graphs are the ones that are extremely simple, and show you a key metric in a glance, without requiring you to understand how to interpret it first.</p> </li> <li> <p>The timeline that we used to visualise the small uncontrolled experiments, was itself a small uncontrolled experiment. It’s expanded into showing all important events, like people being on holiday, deadlines, …</p> </li> </ul> <h2 id="process">Process</h2> <ul> <li> <p>We made it dead easy for testers to deploy a pull request to testing environments, so they can do it whenever it suits them, without dependencies on the developers.</p> </li> <li> <p>We have a process for testing a pull request in isolation, then merging it, and having a different person testing it as part of the whole application. We have lowest defect rate I’ve ever experienced on a project.</p> </li> <li> <p>We spent time improving the test fixtures, so testers can have a good, realistic starting point, that is exactly the same for every test run.</p> </li> <li> <p>It happens more and more often that we work remotely. We’ve had digital boards before, but they never really worked as well as the physical board. We’re now trying Trello. Undecided.</p> </li> <li> <p>We moved from occasionally scheduled releases, to weekly releases, using feature branches. Deploying to production has gradually evolved into a non-event.</p> </li> <li> <p>(Added March 29 2014) We start every story with pair programming. No matter how hard or easy you think it will be. After inital work has begun, you can decide to continue pairing, split the work, or finish it alone. The benefit here is of course that potential pitfalls are found earlier. There’s less need for code reviews. The pairing partner can take over the story later if needed. Even without full-blown pair programming, starting a story in pair is one of those little magic tricks that create huge gains in productiveness and collaboration.</p> </li> </ul> <h2 id="analysis">Analysis</h2> <ul> <li>Stories evolved from vague, widely scoped features, to very specific, atomic descriptions of the smallest idea that provides business value. Because they are small, unnecessary ones can easily be identified and dropped. They can be tested and agreed upon in isolation. Different stories can be worked on in parallel. The predictability of the flow is a lot better. They come with a list of acceptance criteria that we copy into the Github pull request, so we can check off the ones we’ve implemented.</li> </ul> <h2 id="exposing-invisible-problems">Exposing invisible problems</h2> <p>I could go on for a bit, but I hope I made the point clear. According to the Toyota Way, any process, no matter how good you think it is, has waste. The waste is usually not obvious, because if it was, it would have been improved already. Many experiments are not focused on solving a known problem, but on trying things to expose unknown problems. Make sure they are small, low cost, low risk. The benefit may be anywhere between zero and enormous. You won’t know in advance, so opinions have little value. You are the scientist of your own culture.</p> <h2 id="optionality">Optionality</h2> <p>As an aside:</p> <p>Humans favour courses of action that have a well-known, limited set of potential gains, with an unknown, unlimited potential cost. We ignore this cost because it is invisible. In <strong>“Thinking, Fast and Slow”</strong>, Daniel Kahneman refers a lot to the brain’s habit of ignoring the unseen, and calls this “What you see is all there is”.</p> <p>The opposite is a course of action that has a well-known, limited potential cost, with an unknown, unlimited potential gain. In “<a href="/2013/08/antifragile-nassim-nicholas-taleb/">Antifragile</a>”, borrowing from the finance world, Nicholas N. Taleb calls this “optionality”. Keep this in mind: many many very small experiments with a very limited cost, will eventually produce totally unexpected outcomes.</p> <h2 id="read-more">Read More</h2> <p><a href="/2014/10/small-controlled-experiments-revisited/">My talk on Small Uncontrolled Experiments</a></p> Practical Event Sourcing 2014-03-18T00:00:00+00:00 https://verraes.net/2014/03/practical-event-sourcing <p><strong>Update September 18, 2014</strong> Added video</p> <iframe src="//player.vimeo.com/video/104095245" width="640" height="360" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe> <p><a href="http://vimeo.com/104095245">Watch the video on Vimeo</a>.</p> <script async="" class="speakerdeck-embed" data-id="78d43d2090a401318cc322b59c6a013f" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><a href="https://speakerdeck.com/mathiasverraes/practical-event-sourcing">See the slides on Speakerdeck</a></p> <h2 id="abstract">Abstract</h2> <p>Traditionally, we create structural models for our applications, and store the state of these models in our databases.</p> <p>But there are alternatives: Event Sourcing is the idea that you can store all the domain events that affect an entity, and replay these events to restore the object’s state. This may sound counterintuitive, because of all the years we’ve spent building relational, denormalized database schemas. But it is in fact quite simple, elegant, and powerful.</p> <p>In the past year, I’ve had the pleasure of building and shipping two event sourced systems. In this session, I will show practical code, to give you a feel of how you can build event sourced models using PHP.</p> Religiously RESTful 2014-03-03T00:00:00+00:00 https://verraes.net/2014/03/religiously-restful <p>Should you always speak and write perfect Oxford English? Most people will agree that this would be impractical, and take a lot of effort. It depends on your conversation partner, or your audience, and, quite importantly, on the expected lifespan of the message. If you are among friends, in the pub, you will most likely speak your local dialect. If you write a text message to your best mate, same. If, on the other hand, you were the King, and you were addressing the nation on a matter of the gravest importance, you’d want it to be in the best English you have at your disposal. If you write a document that is likely to survive you, such as legislature, you’d want to make sure it is still readable in a hundred years. Your local vernacular wouldn’t provide that guarantee.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-03-03-religiously-restful/colin-firth-as-king-george-vi-in-the-king.jpeg" alt="Colin Firth as King George VI" /></p> <p>REST is a protocol, a language. You can speak it perfectly, but this comes at a greater cost. You and your team need a deeper understanding of the protocol. There’s effort in figuring out the tools, making sure you follow the rules, and refactoring when you realised that you misinterpreted how the protocol should be implemented.</p> <p>The truth is, not all APIs need that level of REST maturity. Often, all you are building is a quick and dirty interface to allow two systems to talk to each other. If you build an API, and the only client is your own web user interface, and perhaps an mobile application, then there will be very little benefit in spending all the effort on perfection. After all, next year, you might throw away the REST API and replace it with some new protocol, or perhaps even something like a message based integration. Only if you are building an API that you expect to be maintained for a very long time, with lots of clients all over the planet, a strict protocol will serve you.</p> <p>REST is tool, and it solves the problem of standardisation and long time maintainability quite well. Consider carefully if you need to whole tool, or just an approximation. Use the time you gain to build a better language for the domain model at the heart of your system, because this is a lot more critical than the infrastructure around it. Speak the RESTish language you are comfortable with, and learn the official language when you need it.</p> Buzzword-free Bounded Contexts 2014-02-13T00:00:00+00:00 https://verraes.net/2014/02/buzzword-free-bounded-contexts <p>Context is everywhere. The human brain interprets context immediately, without effort. Try this:</p> <ul> <li>A product of my company</li> <li>A product of prime factors</li> <li>A product of my imagination</li> </ul> <p>Note how easy it was for you to derive context. For machines, this is hard, which is one reason we feed them models with unambiguous names.</p> <p>The difference is often even a lot subtler.</p> <ul> <li>A product in the sales department is a thing with a weight and dimensions, a description, a few pictures and some customer reviews.</li> <li>For the inventory, it’s a box that takes space on a shelf and that you always have too many or too few of.</li> <li>For the shipping department, it’s a package you need to deliver to a customer. It has a weight and dimensions, but this time they include the packaging.</li> </ul> <p>In each of these departments, people know the context perfectly well. They don’t bother finding a different name, because even though it’s all called product, there’s never any misunderstanding.</p> <h2 id="a-model-within-a-context">A Model within a Context</h2> <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-02-13-buzzword-free-bounded-contexts/m-and-ms-small.jpg" alt="Separate into Bounded Contexts" /></p> <p>And then the software developers come along, and we put all these totally different things that are all called product, in the same database table and model. We’re supposed to be the smart guys, but we’ve been fooled by the fact that they share the same name. We’ve totally missed the context. We end up with a huge model, that tries to be all these things at once, for every department, despite the fact that these departments are quite independent in real life anyway.</p> <p>Now imagine that instead of one team, building one application and one model, we have three teams, in three different companies, building three separate systems: Salesify, SaasInventory, and SpeedyShip. It’s up to the users to integrate them. Nobody is calling the other teams going “Hey, we should all share one database!”. That’s a recipe for disaster. They use different infrastructure, different storage, different models. What you would do, is agree on a narrow, strictly defined way to integrate the systems. You could use messaging, an API, import scripts, or dumping files on an FTP.</p> <p>Whether the three teams will easily reach agreement on those integration contracts, is <a href="/2014/01/bandwidth-and-context-mapping/">a whole different discussion</a>. But apart from that, it’s great not to share your database, or any other infrastructure. It gives you freedom. There is no complication from trying to put all the things in one model.</p> <p>So why do we make it so hard on ourselves when we are just one team? We can borrow the benefits of having multiple teams (clarity, clean separation), without the downside (communication overhead). That’s essentially what Bounded Contexts are about:</p> <ul> <li>Split up your system according to natural divisions in the domain, such as departments, userbases, lifecycles, goals, …</li> <li>Make the contexts explicit. (“A Product in the context of Shipping is …”)</li> <li>Make the boundaries explicit. (“Shipping uses database A, Inventory uses database B…”)</li> <li>Keep the models consistent within a Bounded Context. Well, keeping your models consistent is a good idea anyway.</li> </ul> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/01/bandwidth-and-context-mapping/">Bandwidth and Context Mapping</a></li> <li><a href="/2015/04/context-mapping-life-expectancy/">Context Mapping: Life Expectancy</a></li> </ul> Domain-Driven Design: Bounded Contexts, Modelling 2014-02-12T00:00:00+00:00 https://verraes.net/2014/02/domain-driven-design-basics <p>Yesterday, we did a DDD Basics session with the <a href="http://domaindriven.be">Domain-Driven Design Belgium</a> community. We only covered Strategic DDD, and split up the work amongst the five founders of the group:</p> <ul> <li><a href="http://twitter.com/yreynhout">@yreynhout</a> (Defining Domain-Driven Design),</li> <li><a href="http://twitter.com/JefClaes">@JefClaes</a> (Ubiquitous Language),</li> <li><a href="http://twitter.com/ONE75">@ONE75</a> (Context Mapping),</li> <li><a href="http://twitter.com/ToJans">@ToJans</a> (Selling Domain-Driven Design to the business),</li> <li><a href="http://twitter.com/mathiasverraes">myself</a> (Bounded Contexts and Modelling).</li> </ul> <p>Update June 10, 2014: new &amp; improved slides for the second edition of this session.</p> <script async="" class="speakerdeck-embed" data-id="a75cfd80d2e40131173366cd45aa8ef5" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><a href="https://speakerdeck.com/mathiasverraes/ddd-basics-bounded-contexts-modelling-kortrijk-edition">Watch the slides on Speakerdeck</a></p> <h2 id="read-more">Read more</h2> <ul> <li><a href="https://verraes.net/2014/02/buzzword-free-bounded-contexts/">Buzzword-free Bounded Contexts</a></li> </ul> Model Storming Workshop 2014-01-22T00:00:00+00:00 https://verraes.net/2014/01/model-storming-workshop <script async="" class="speakerdeck-embed" data-id="5d64816065aa0131a8e94ae38a915c91" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><a href="https://speakerdeck.com/mathiasverraes/model-storming-workshop">See the slides on Speakerdeck</a></p> <h2 id="abstract">Abstract</h2> <p>Complexity in software hits us late in the game: long after contracts were signed, budgets were approved, and code was written. Big Design Upfront consistently misses the mark, but Agile methodologies don’t answer to the challenge either. Model Storming is an emergent practice in the Domain-Driven Design community, that brings modelling back to all stakeholders.</p> <p>The workshop is similar to a hackathon, where developers tackle a problem as a team. But instead of writing code, you’ll be drawing models. We’ll start by explaining the domain, and then help you on your way to turn this into an expressive model. At regular intervals, we’ll break to discuss insights, and provide with you with new techniques and ideas to help you move forward.</p> <p>No previous knowledge required; all you need to bring is your creativity and an open mind!</p> <h2 id="acknowledgement">Acknowledgement</h2> <p>I couldn’t have done it without <a href="http://twitter.com/stijnvnh">Stijn Vannieuwenhuyse</a>, who played the part of Domain Expert and co-facilitator.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/01/domain-driven-design-is-linguistic/">Domain-Driven Design is Linguistic</a></li> <li><a href="/2013/08/facilitating-event-storming/">Facilitating Event Storming</a></li> <li><a href="/2013/09/dddbe-modellathon/">The DDDBE Modellathon</a></li> </ul> Bandwidth and Context Mapping 2014-01-15T00:00:00+00:00 https://verraes.net/2014/01/bandwidth-and-context-mapping <p>Context Mapping, first described by Eric Evans in Domain-Driven Design, is lightweight method of drawing out the relations between systems, and parts of systems. It’s not technical per se. It exposes the politics of the organisations and the teams building the systems. As you start mapping, every edge that connects two systems, defines an upstream, and a downstream point; the latter system being the one that is affected by changes in the former. Or, to put it more graphically: if the upstream people piss in the river, the downstream people are drinking it. Pardon my French. In the book, Eric defines a number of patterns, that characterise those relations: Anticorruption Layer, Custom/Supplier, Open Host Service, Published Language, Shared Kernel, etc.</p> <p>Mind that it’s not about the direction of the data. You might be sending data to another system, but still be downstream, because they decide what that data should look like. Eric, and <a href="https://twitter.com/ziobrando/status/423531883893252096">Alberto Brandolini</a>, have <a href="http://www.infoq.com/articles/ddd-contextmapping">explained it better</a> than I could, so I won’t go into detail.</p> <p>In the workshop today, and over dinner, <a href="https://twitter.com/ziobrando/status/423531883893252096">Alberto Brandolini</a> explained his recent idea about Bandwidth. For me, that ties the whole of Context Mapping together. I will try to do it justice here.</p> <p><img src="/img/posts/2014-01-15-bandwidth-and-context-mapping/the-deepest-explanation-of-ddd.png" alt="I think today I gave the deepest explanation ever of Context Mapping ...and I feel good!" /></p> <h2 id="whitespace-and-bandwidth">Whitespace and Bandwidth</h2> <p>Claude Debussy predicted funk music:</p> <blockquote>Music is the space between the notes.</blockquote> <p>When you draw a Context Map, you visualize the different Bounded Contexts of the system. This is of course useful information, especially when tackling a legacy system. Invisible to the untrained eye, it is in fact more than a static representation. In the whitespace between the parts, there is implicit information about the future success of the project. Say you are downstream from another team’s system. You’ve labeled the relationship as Conformist on your side. Now someone wants you to make a significant change, and this change touches on this relationship with the upstream system. Will this project succeed?</p> <p>You can not answer this from just looking at the Context Map. Some missing concept is hiding in the whitespace. The patterns describe the character of the communication, but not how much communication is going on between the two teams. Alberto names this quantity the Bandwidth.</p> <p><img src="/img/posts/2014-01-15-bandwidth-and-context-mapping/context-map-bandwidth.jpg" alt="Context Map with Bandwidth" /></p> <p>Bandwidth will affect you a lot. If the upstream team is in a different country, and you only have one monthly call with them, the Bandwidth is low. If they are in the office next door, and you have lunch with them daily, your Bandwidth is going to be higher. Even though in my example, you are mostly Conformist, you still have a chance of altering the relation slightly, perhaps pushing it more in the direction of a Customer/Supplier pattern. The Bandwidth allows more communication, more opportunity for shared understanding, more opportunity for having at least some say in the service they provide. Many things affect this Bandwidth: obvious things like physical availability, frequency of communications, responsiveness… But also things like seniority: people who have been in the organisation a long time, are usually the ones setting the rules. Sometimes it will be determined by which system is more essential to the business, although, in some case, a less important system might still call the shots. If one system is legacy, it will be upstream, simply because it is too hard to change. Or perhaps the upstream team simply has too much ego to want to empathise with other teams.</p> <p>Making this Bandwidth explicit, by drawing fatter and thinner edges, helps you make more informed decisions. Low Bandwidth means higher risk for projects that depend on communications between Bounded Contexts. With Bandwidth, the Context Map is more complete: it now shows the structure, the character of the relations, and the quantity of the communications.</p> <p><strong>Update 2014-01-16</strong> I missed a small nuance. It’s not about the amount of communication going on per se, but rather about the <em>potential</em>, or the <em>capacity</em>, for communication.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/02/buzzword-free-bounded-contexts/">Buzzword-free Bounded Contexts</a></li> <li><a href="/2015/04/context-mapping-life-expectancy/">Context Mapping: Life Expectancy</a></li> </ul> Build for Failure 2014-01-05T00:00:00+00:00 https://verraes.net/2014/01/build-for-failure <p>We can’t avoid failure. Systems fail all the time. The deep assumption of failure needs to be engraved in everything we do or build.</p> <p>This insight has been hitting me in the face for the past year, with the BuildStuff conference as the pinnacle. Failure in Systems is inevitable (<a href="/2013/08/john-gall-systemantics-the-systems-bible/">Gall</a>). Predictions fail, and anything that depends on randomness (<a href="http://www.amazon.com/Fooled-Randomness-Hidden-Chance-Markets/dp/0812975219">Taleb</a>). We’re not rational beings, even if we understand to what extent we are not rational (<a href="http://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman-ebook/dp/B005MJFA2W">Kahneman</a>). It’s the philosophy of Erlang, to build for failure. It’s the idea behind the Lean Startup: fail fast and fail often, until you start to suck at failing. Instead of aiming for small known potential gains (with unknown potential losses), aim for small known losses (with unknown potential gains) (<a href="http://localhost:4000/2013/08/antifragile-nassim-nicholas-taleb/">Taleb</a> again). Publish blogs first, fix later (<a href="http://twitter.com/ToJans">@ToJans</a>). Model it wrong, refactor towards deeper insight (<a href="http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215/">Evans</a>). Visualise divergence of interpretations (<a href="http://twitter.com/ziobrando">@ziobrando</a>). Cause frequent failures, to force yourself to build resilient systems (Netflix Chaos Monkey). And so on…</p> <p>I’ve had a sticky note on my team’s whiteboard for the past six months, saying “Find at least three solutions”. Problems do not have solutions. All they have, are a bunch of random approximations of solutions, each with a bunch of known and unknown drawbacks. The thing with the number three, is that after you’ve found three solutions, your collective brains are nicely warmed up, and you find three more, by combining parts from the first three. Challenge the solutions, break them down, and when you pick one, build the most minimal version of it. Evaluate. If it’s not what you thought it could be, you still have five more scenario’s to try.</p> <p>As if BuildStuff didn’t fry my brain as it was, I was lucky to <a href="http://hintjens.com/blog:73">spend three hours on a plane</a> with <a href="http://twitter.com/hintjens">Pieter Hintjens</a>, the founder of the ZeoMQ community and the author of the book. Here’s a quote:</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2014-01-05-build-for-failure/zeromq-book.jpg" alt="ZeroMQ" /></p> <blockquote> We pretend to ourselves and others that we can be (could be) perfect, when in fact we consistently make mistakes. Bugs in code are seen as "bad", rather than "inevitable", so psychologically we want to see fewer of them, not uncover more of them. "He writes perfect code" is a compliment rather than a euphemism for "he never takes risks so his code is as boring and heavily used as cold spaghetti". <br /> <br /> Some cultures teach us to aspire to perfection and punish mistakes in education and work, which makes this attitude worse. To accept that we're fallible, and then to learn how to turn that into profit rather than shame is one of the hardest intellectual exercises in any profession. We leverage our fallibilities by working with others and by challenging our own work sooner, not later. <br /> <br /> (...) <br /> <br /> The faster you can prove code incorrect, the faster and more accurately you can fix it. Believing that code works and proving that it behaves as expected is less science, more magical thinking. It's far better to be able to say, "libzmq has five hundred assertions and despite all my efforts, not one of them fails". </blockquote> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/10/software-design-is-just-theory/">“Software design is just theory”</a></li> <li><a href="/2013/08/john-gall-systemantics-the-systems-bible/">The Systems Bible</a></li> </ul> The Henry Ford Fallacy 2014-01-01T00:00:00+00:00 https://verraes.net/2014/01/henry-ford-fallacy <blockquote> If I had asked people what they wanted, they would have said faster horses. <small>Attributed to Henry Ford</small> </blockquote> <p>The line is often quoted as an excuse for deliberately ignoring users’ requests for features. It’s usually followed by some mention of the iPhone, Steve Jobs, or Apple in general. And, usually, it’s uttered by people who build a €20 app, not a hardware device or an entire car.</p> <p>It’s the wrong question. If you ask users what they want, you’ll get an answer that is unimaginative, and that is additive. They’ll seldom ask to rethink an existing feature, and they’ll never ask to remove a feature. If Henry Ford would have asked what problem people had, and kept asking it over and over again, and drilled down to the core of the problem, they would have said: we want to get places faster, cheaper, more comfortable, and with less fuss.</p> <p><img src="/img/posts/2014-01-01-henry-ford-fallacy/iron-dobbin.png" alt="The iron Dobbin is one creepy looking vehicle" /></p> Domain-Driven Design is Linguistic 2014-01-01T00:00:00+00:00 https://verraes.net/2014/01/domain-driven-design-is-linguistic <p><img style="float:left;margin-right: 10px" src="/img/posts/2014-01-01-domain-driven-design-is-linguistic/hoarding small.jpg" title="Hoarding is an extreme form of loss aversion. http://jamesgulliverhancock.com/projects/hoarding/" /></p> <p>If I give you half a million euros, you will be quite happy. If I give you a million euros, but on the way home, you get robbed and lose half a million, you will be unhappy, even though the end result is the same. Even if you only lose 200k, you will be unhappier than if you only had the half million to begin with. In psychology, this is called <a href="http://www.amazon.com/gp/product/B00555X8OA/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=B00555X8OA&amp;linkCode=as2&amp;tag=verraesnet-20&amp;linkId=WRTRBQWNNPU4DUAC">Loss Aversion</a>. That is, loss has a higher impact on us than gains. We are not purely rational economic beings. The behaviour that leads to a certain state is more important than the actual state.</p> <p>Businesses don’t make decisions based on the money they have in the bank. It’s the successes and the failures that lead that profit, that are going to determine what you’re going to do next.</p> <p>This is what, to me, is so appealing in Domain-Driven Design. Software development has traditionally been mostly about mathematics, about technical matters. DDD makes the linguistics the central concern. To express it in a formula:</p> <p><code class="language-plaintext highlighter-rouge">f(x) = y and g(x) = y =/=&gt; f = g</code></p> <p>Or: Just because two behaviours lead to the same outcome, does not mean they are replaceable. In Domain-Driven modelling, state is a side effect of behaviour. State is only interesting as far as it has the potential to influence new behaviour. An Aggregate is not just an composition of objects that belong together to represent a certain state. An Aggregate manages behaviours that belong together, and keeps state only where they are relevant to those behaviours. Event Sourcing takes the idea to the its logical conclusion: We forget about persisting state altogether, and instead, we only persist the Domain Events that lead up to that state. We preserve the meaning, instead of the outcome. When we need the state, to make a decision, we replay the events and interpret them. The beauty of this, is that if we change our minds about how to interpret the behaviour, we are not stuck with the state the we decided to persist in the past.</p> <p><strong>Update May 11, 2014</strong> I just had to add <a href="https://twitter.com/abdullin/status/465463982347026432">this quote from Rinat Abullin</a>:</p> <blockquote>Behavior is not data. To be more precise, data is often an approximation of observed behaviors under fixed circumstances.</blockquote> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/01/model-storming-workshop/">Model Storming Workshop</a></li> </ul> Related Entities vs Child Entities 2013-12-30T00:00:00+00:00 https://verraes.net/2013/12/related-entities-vs-child-entities <h2 id="entity">Entity</h2> <p><img style="float:right;margin-left: 10px" src="/img/posts/2013-12-30-related-entities-vs-child-entities/marshfrog_illus_cropped.jpg" alt="Frog lifecycle" /></p> <p>An <strong>Entity is an object in our Domain Model with an identity and a lifecycle</strong>. “Lifecycle” means that it is created, possibly changes over time, and eventually can cease to exist. Throughout this lifecycle, we need some way to identify this object, especially when our system talks to other systems about this Entity.</p> <p>Imagine a simple project management application. When we start a new Project, we give it a name, a due date, and a ‘started’ status. We assign an identifier to the Project. Later, at different times, we rename the Project, change the due date, and, at a certain point, and change the status to ‘ended’. After a while, we delete the Project from the system. Because we need to work with this Project at different times in its lifecycle, we need to persist it to something more trustworthy than memory. We use a database for this, and every time we want to access the Project, we use its identifier to talk to the database. Our system is not an island: we publish a list of Projects to client systems, and these clients can query our system for the current state of a Project, also by using the identifier.</p> <h2 id="entity-relationships">Entity relationships</h2> <p>Two entities, of the same or of different types, can have a relation. In our example, we could be interested in the Employee who is responsible for the Project, and the Employees who are assigned to the Project. Projects and Employees have their own lifecycles: Both can change independently, the link can be broken. If a Project is cancelled (deleted), the Employee Entities are not deleted, and when an Employee is fired, the Project can still continue to exist. This is a weak relationship.</p> <h2 id="child-entities">Child Entities</h2> <p>Sometimes, an <strong>Entity’s lifecycle is entirely dependent on another Entity. We call this a Child Entity</strong>. It’s a strong relationship: the Child Entity has only meaning in the context of its parent, and cannot exist outside of it. If the Parent Entity ceases to exist, the Child Entity is deleted as well. Ideally, all operations on the child are handled by the parent, even its creation.</p> <p>In our example, our Project has Tasks. A Task is part of one Project, and one Project alone. The Task can’t be created independently of the Project, it can’t exist outside of it, and when the Project is deleted, its Tasks are deleted. Tasks have a ‘completed’ boolean flag. Clearly a Task has a lifecycle: it gets created, at a certain point it is completed, and it can be removed. This lifecycle only exists in the context of Project.</p> <p>Why is all of this important? <strong>Modelling is the art of making the implicit explicit</strong>. By making the difference between related Entities and Child Entities explicit,we open up the possibility to treat them differently, make different design decisions, and optimise differently. It’s going to affect how we encapsulate behaviours, how we build repositories, REST APIs, and database schemas. It determines the boundaries of our models, which in turn determines the opportunities for partitioning our system. <strong>Thinking hard about this difference, is going to help us decide what to couple, and what to decouple</strong>.</p> <h2 id="coding-the-example">Coding the example</h2> <p>Let’s build the code for Project and Task. Keep in mind that this is not necessarily the best or the only way to do it. I just want to try to get you to think about how Child Entities differ conceptually from related Entities at the same hierarchical level of each other.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Task</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$description</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$description</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">description</span> <span class="o">=</span> <span class="nv">$description</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Project</span> <span class="p">{</span> <span class="cd">/** * @var string */</span> <span class="k">private</span> <span class="nv">$projectId</span><span class="p">;</span> <span class="cd">/** * @var string */</span> <span class="k">private</span> <span class="nv">$title</span><span class="p">;</span> <span class="cd">/** * @var Task[] */</span> <span class="k">private</span> <span class="nv">$tasks</span> <span class="o">=</span> <span class="p">[];</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$projectId</span><span class="p">,</span> <span class="nv">$title</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">projectId</span> <span class="o">=</span> <span class="nv">$projectId</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">title</span> <span class="o">=</span> <span class="nv">$title</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">addTask</span><span class="p">(</span><span class="nv">$description</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">tasks</span><span class="p">[]</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Task</span><span class="p">(</span><span class="nv">$description</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>In this code, Task is effectively a Value Object. That could definitely work, but in remember that we want to track whether a Task is completed. A Task that is completed, is still the same Task, even though one of its properties changed. Its identity remains, independent of its values. That makes it an Entity.</p> <p>Why is <code class="language-plaintext highlighter-rouge">Project#addTask()</code> taking only a description, and not a Task instance? Tasks are an essential part of a Project. We want our Project Parent Entity to be in control of the lifecycle of Task. That means we don’t want other code to start making Task objects. In PHP, there’s no concept of ‘friend classes’, so we can’t really prevent anyone from making Task instances – <a href="/2013/10/pre-merge-code-reviews/">except by the power of code reviews</a> :-)</p> <p>Moving on, we need a way to identify Tasks. We give the control over the Task identity to Project. A Task’s identity only has meaning in the context of Project, so the id does not have to be unique across all Tasks, only across Tasks within a Project. Here’s a simple solution:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Task</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$taskId</span><span class="p">;</span> <span class="k">private</span> <span class="nv">$description</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$taskId</span><span class="p">,</span> <span class="nv">$description</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">taskId</span> <span class="o">=</span> <span class="nv">$taskId</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">description</span> <span class="o">=</span> <span class="nv">$description</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Project</span> <span class="p">{</span> <span class="c1">// ...</span> <span class="k">public</span> <span class="k">function</span> <span class="n">addTask</span><span class="p">(</span><span class="nv">$description</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">tasks</span><span class="p">[]</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Task</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">getNextTaskId</span><span class="p">(),</span> <span class="nv">$description</span><span class="p">);</span> <span class="p">}</span> <span class="k">private</span> <span class="k">function</span> <span class="n">getNextTaskId</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="n">tasks</span><span class="p">)</span> <span class="o">+</span> <span class="mi">1</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Because we can now identify Tasks, we can complete individual Tasks.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Task</span> <span class="p">{</span> <span class="c1">// ...</span> <span class="cd">/** * @var bool */</span> <span class="k">private</span> <span class="nv">$completed</span> <span class="o">=</span> <span class="kc">false</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getTaskId</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">taskId</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">complete</span><span class="p">()</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">completed</span> <span class="o">=</span> <span class="kc">true</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Project</span> <span class="p">{</span> <span class="c1">// ...</span> <span class="k">public</span> <span class="k">function</span> <span class="n">completeTask</span><span class="p">(</span><span class="nv">$taskId</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">getTaskById</span><span class="p">(</span><span class="nv">$taskId</span><span class="p">)</span> <span class="o">-&gt;</span><span class="nf">complete</span><span class="p">();</span> <span class="p">}</span> <span class="k">private</span> <span class="k">function</span> <span class="n">getTaskById</span><span class="p">(</span><span class="nv">$taskId</span><span class="p">)</span> <span class="p">{</span> <span class="k">foreach</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="n">tasks</span> <span class="k">as</span> <span class="nv">$task</span><span class="p">)</span> <span class="p">{</span> <span class="k">if</span><span class="p">(</span><span class="nv">$task</span><span class="o">-&gt;</span><span class="nf">getTaskId</span><span class="p">()</span> <span class="o">==</span> <span class="nv">$taskId</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$task</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="c1">// throw</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>As you can see, Task is still entirely private to Project. Only Project knows how to complete a Task. If for some reason, the underlying code changes completely, the API of Project will remain intact. Hopefully, this illustrates how <strong>the Child Entity pattern fits nicely with object oriented ideas of information hiding and encapsulation</strong>. Imagine that every object only has access to a very limited set of information, a limited set of operations on a limited set of collaborators. Writing spaghetti code would be impossible. We can’t protect our code to that extent, but having a mental model of this, is definitely going to help you reason about your system.</p> <p>Another benefit of all this encapsulation, is that Project can guard invariants for its set of Tasks. One such invariant could be a business rule stating that “Tasks can only be completed if the Project has at least five Tasks.” This business rule should not be guarded inside Task, but Project is a natural fit for it.</p> <p>Other operations on Task, like changing the description and removing a Task, can be handled in the same manner. I leave that as an exercise for the reader.</p> <h2 id="persistence">Persistence</h2> <p>In the real world, we can’t keep all of this in memory, so we persist stuff in databases. If we want to map Project and Task to a relational database, we’d likely use a one-to-many relationship. We’ll need to make a slight change to our code.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Task</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$projectId</span><span class="p">;</span> <span class="k">private</span> <span class="nv">$taskId</span><span class="p">;</span> <span class="k">private</span> <span class="nv">$description</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$projectId</span><span class="p">,</span> <span class="nv">$taskId</span><span class="p">,</span> <span class="nv">$description</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">projectId</span> <span class="o">=</span> <span class="nv">$projectId</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">taskId</span> <span class="o">=</span> <span class="nv">$taskId</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">description</span> <span class="o">=</span> <span class="nv">$description</span><span class="p">;</span> <span class="p">}</span> <span class="c1">// ...</span> <span class="p">}</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Project</span> <span class="p">{</span> <span class="c1">// ...</span> <span class="k">public</span> <span class="k">function</span> <span class="n">addTask</span><span class="p">(</span><span class="nv">$description</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">tasks</span><span class="p">[]</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Task</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="n">projectId</span><span class="p">,</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">getNextTaskId</span><span class="p">(),</span> <span class="nv">$description</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Before, the Task didn’t know about the Project it belonged to. Now it does, which is a slightly annoying trade-off. Just be aware that we don’t really want a bidirectional relation here. We could give Task a reference to Project instead of just the projectId, and let the ORM work it out.</p> <p>In our database schema, Task’s primary key is a composite of projectId and taskId. If the ORM or the infrastructure doesn’t deal well with composite ids that are also foreign keys, we could make taskId the only primary key, and make it unique for all Tasks. Again, conceptually, this is a small trade-off, with benefits and drawbacks.</p> <h2 id="repository">Repository</h2> <p>Because Task is a Child Entity, it doesn’t have its own repository. All write operations can easily be done by fetching the Project instance from the ProjectRepository, and calling methods on Project that manipulate its Tasks, as shown above. For reading, we do the same: we see Project as a complete unit, including its children, so when we fetch a Project, we get the full object graph. This only includes Child Entities, and not related entities such as User. If we were to include all related entities as well, we risk getting into a big, highly coupled mess. We could get circular references (which your ORM should deal with, but still). And we would be very tempted to access methods on the User objects we get from Project, which would be a clear violation of the Law of Demeter.</p> <p>But again, in the real world, we might run into issues. We may need reports across Child Entities from different parents, for example a list of incomplete Tasks for the whole system. Fetching all Projects and looping over their Tasks might cause performance issues. A read model projection could fix this, but it’s out of scope for this post. We can introduce a TaskRepository to answer such queries. It blurs the line of a Child Entity’s encapsulation, but again, if we are aware that it is a trade-off, we are already better equipped than if we did it blindly, without consideration.</p> <h2 id="passing-references">Passing references</h2> <p>To render a Project with its Tasks, we’ll need to access Tasks in the view, so we need Project to expose a getTasks() method. The idea here is that what you get from this method, is a <strong>passing reference</strong> to Tasks. You are allowed to use it, but your use is temporary. Your client code is supposed to forget about the Task object, and always refer back to Project if it needs it again.</p> <h2 id="coupling">Coupling</h2> <p>To wrap it up, have a look at the graph below.</p> <!-- Source of this svg is in /graphs/2013-12-30-related-entities-vs-child-entities.dot --> <svg width="255pt" height="264pt" viewBox="0.00 0.00 254.94 264.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> <g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 260)"> <title>g</title> <polygon fill="white" stroke="white" points="-4,4 -4,-260 250.941,-260 250.941,4 -4,4" /> <g id="clust1" class="cluster"><title>cluster_0</title> <polygon fill="Lavender" stroke="black" points="8,-130 8,-248 209,-248 209,-130 8,-130" /> </g> <g id="clust2" class="cluster"><title>cluster_1</title> <polygon fill="palegreen" stroke="black" points="133,-8 133,-92 203,-92 203,-8 133,-8" /> </g> <!-- Project --> <g id="node1" class="node"><title>Project</title> <polygon fill="white" stroke="black" points="134.536,-139 134.536,-239 201.464,-239 201.464,-139 134.536,-139" /> <text text-anchor="middle" x="168" y="-221" font-family="Times,serif" font-size="14.00">Project</text> <polyline fill="none" stroke="black" points="134.536,-214.2 201.464,-214.2 " /> <text text-anchor="middle" x="168" y="-196.2" font-family="Times,serif" font-size="14.00">projectId</text> <text text-anchor="middle" x="168" y="-179.4" font-family="Times,serif" font-size="14.00"> title</text> <text text-anchor="middle" x="168" y="-162.6" font-family="Times,serif" font-size="14.00"> tasks[]</text> <text text-anchor="middle" x="168" y="-145.8" font-family="Times,serif" font-size="14.00"> assignee</text> </g> <!-- Task --> <g id="node2" class="node"><title>Task</title> <polygon fill="white" stroke="black" points="16.1465,-155.8 16.1465,-222.2 97.8535,-222.2 97.8535,-155.8 16.1465,-155.8" /> <text text-anchor="middle" x="57" y="-204.2" font-family="Times,serif" font-size="14.00">Task</text> <polyline fill="none" stroke="black" points="16.1465,-197.4 97.8535,-197.4 " /> <text text-anchor="middle" x="57" y="-179.4" font-family="Times,serif" font-size="14.00">taskId</text> <text text-anchor="middle" x="57" y="-162.6" font-family="Times,serif" font-size="14.00"> description</text> </g> <!-- Project&#45;&gt;Task --> <g id="edge1" class="edge"><title>Project&#45;&gt;Task</title> <path fill="none" stroke="black" d="M134.41,-189C126.171,-189 117.164,-189 108.301,-189" /> <polygon fill="black" stroke="black" points="108.017,-185.5 98.0169,-189 108.017,-192.5 108.017,-185.5" /> <text text-anchor="middle" x="115.768" y="-194.8" font-family="Times,serif" font-size="14.00">has</text> </g> <!-- User --> <g id="node3" class="node"><title>User</title> <polygon fill="white" stroke="black" points="141,-16.8 141,-83.2 195,-83.2 195,-16.8 141,-16.8" /> <text text-anchor="middle" x="167.993" y="-65.2" font-family="Times,serif" font-size="14.00">User</text> <polyline fill="none" stroke="black" points="141,-58.4 194.986,-58.4 " /> <text text-anchor="middle" x="167.993" y="-40.4" font-family="Times,serif" font-size="14.00">userId</text> <text text-anchor="middle" x="167.993" y="-23.6" font-family="Times,serif" font-size="14.00"> name</text> </g> <!-- Project&#45;&gt;User --> <g id="edge2" class="edge"><title>Project&#45;&gt;User</title> <path fill="none" stroke="black" d="M168,-138.927C168,-124.099 168,-107.947 168,-93.5798" /> <polygon fill="black" stroke="black" points="171.5,-93.4779 168,-83.4779 164.5,-93.478 171.5,-93.4779" /> <text text-anchor="middle" x="207.471" y="-105.4" font-family="Times,serif" font-size="14.00"> is assigned to</text> </g> </g> </svg> <p>It should be clear now that Project and Task are highly cohesive, and have strong coupling, because that’s what they are like in our simple domain. On the other hand, User is only very weakly coupled to Project. I find that most people, when building models, make no distinction between these relations, and simply use the same approach for everything. The result is that the connection between parents and children is too weak, and the relation between merely related entities, is too strong. This coupling creates messy models. The underlying business reasoning is not made explicit in the model.</p> <p>As is always the problem with examples, is that the one I used here is too simple and not production-ready. Consider it pseudo-code, to illustrate the concept, rather than a set of rules of how to model a domain. If nothing else, I hope it helps you to model more explicitly.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2013/04/crud-is-an-anti-pattern/">CRUD is an anti-pattern</a></li> <li><a href="https://verraes.net/2013/11/value-objects-and-user-interfaces/">Value Objects and User Interfaces</a> (and especially the comments)</li> <li>Elephant in the Room podcast, <a href="http://elephantintheroom.io/blog/2013/10/episode-2-heart-and-soul-of-oop/">Episode #002 on Value Objects</a></li> </ul> Fighting Bottlenecks with CQRS 2013-12-01T00:00:00+00:00 https://verraes.net/2013/12/fighting-bottlenecks-with-cqrs <p>I hadn’t done any presentations or blog posts about Command/Query Responsibility Segregation yet, even though I’ve been working with it for more than year (and have been wanting to use it for another year). <a href="https://twitter.com/__edorian">Volker Dusch</a> invited me to speak at the first <a href="http://news.researchgate.net/index.php?/archives/181-ResearchGate-Developer-Day.html">ResearchGate Developer Day</a> in Berlin, and offered me the luxury to choose any SE topic I wanted, I just had to do it.</p> <script async="" class="speakerdeck-embed" data-id="9954fc103bd40131db9c2eaec66c710f" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><a href="https://speakerdeck.com/mathiasverraes/fighting-bottlenecks-with-cqrs">See the slides on Speakerdeck</a></p> <p>Volker gave me a tour of the ResearchGate office. They’re all about sharing scientific research. It’s a fast growing startup, with an awesome culture, and <a href="http://www.researchgate.net/careers/">they’re hiring</a>, so check them out.</p> Value Objects and User Interfaces 2013-11-15T00:00:00+00:00 https://verraes.net/2013/11/value-objects-and-user-interfaces <p>I got this mail from a listener of the <a href="http://elephantintheroom.io/blog/2013/10/episode-2-heart-and-soul-of-oop/">podcast episode on Value Objects</a> I recorded with Konstantin:</p> <blockquote> I've been listening to your Elephant in the room podcast Episode #002, i was wondering if you have a coding example of where you use the Country as a value object and not as an entity. I've come across this problem too, and was hoping you may have an example that i can look at to give me a better understanding in terms of how to code this? My first question would really be how would you logically store a list of all countries outside of a database if you were using value objects for use in a Form lookup field? </blockquote> <p>There’s nothing intrinsically wrong with modelling countries as entities and storing them in the database. But in most cases, that overcomplicating things. Countries don’t change often. When a country’s name changes, it is in fact, for all practical purposes, a new country. If a country one day does not exist anymore, you can’t simply change all addresses, because possibly the country was split into two countries.</p> <p>Whenever you have this kind of friction, there’s usually some missing concept screaming to be discovered. In this case, two concerns are being mixed. One is a modelling concern: we want to treat <code class="language-plaintext highlighter-rouge">Country</code> as a value object with all the benefits that it brings. The other is a user experience concern: we want to help the user to pick a country from a list, so we don’t end up with twenty different spellings of the same country. And finally, we’d probably want to validate somehow that the submitted country indeed exists.</p> <p>So what’s the missing concept? The value object represents an actual country, and the UX and validation require a list of available countries. Even though both are closely related, the nuance is slightly different. So let’s make it explicit and call the new concept <code class="language-plaintext highlighter-rouge">AvailableCountries</code>. These available countries can be entities in a database, records in a JSON, or even simply a hardcoded list in your code. (That depends on whether the business wants easy access to them through a UI.)</p> <p>In code, it could look something like this:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">Country</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$countryCode</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$countryCode</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">countryCode</span> <span class="o">=</span> <span class="nv">$countryCode</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__toString</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">countryCode</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">AvailableCountry</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$country</span><span class="p">;</span> <span class="k">private</span> <span class="nv">$name</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">Country</span> <span class="nv">$country</span><span class="p">,</span> <span class="nv">$name</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">country</span> <span class="o">=</span> <span class="nv">$country</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">name</span> <span class="o">=</span> <span class="nv">$name</span><span class="p">;</span> <span class="p">}</span> <span class="cd">/** @return Country */</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getCountry</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">country</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getName</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">name</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="k">final</span> <span class="kd">class</span> <span class="nc">AvailableCountryRepository</span> <span class="p">{</span> <span class="cd">/** @return AvailableCountry[] */</span> <span class="k">public</span> <span class="k">function</span> <span class="n">findAll</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="p">[</span> <span class="s1">'BE'</span> <span class="o">=&gt;</span> <span class="k">new</span> <span class="nc">AvailableCountry</span><span class="p">(</span><span class="k">new</span> <span class="nc">Country</span><span class="p">(</span><span class="s1">'BE'</span><span class="p">),</span> <span class="s1">'Belgium'</span><span class="p">),</span> <span class="s1">'FR'</span> <span class="o">=&gt;</span> <span class="k">new</span> <span class="nc">AvailableCountry</span><span class="p">(</span><span class="k">new</span> <span class="nc">Country</span><span class="p">(</span><span class="s1">'FR'</span><span class="p">),</span> <span class="s1">'France'</span><span class="p">),</span> <span class="c1">//...</span> <span class="p">];</span> <span class="p">}</span> <span class="cd">/** @return AvailableCountry */</span> <span class="k">public</span> <span class="k">function</span> <span class="n">findByCountry</span><span class="p">(</span><span class="kt">Country</span> <span class="nv">$country</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">findAll</span><span class="p">()[(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$country</span><span class="p">];</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>So now you can query <code class="language-plaintext highlighter-rouge">AvailableCountryRepository</code> to populate select boxes, and use it to see if a Country exists.</p> <h2 id="when-to-use">When to use</h2> <p>This pattern should be easy to implement in many different ways, suited to whatever your use case is. Deciding when and when not to use this pattern, is probably the hardest part. Start by imagining you are building a ‘pure’ model. There’s no UI, no database, just code. Is the thing you’re modelling an entity or a value object? Does it change over time? If two instances have the same value, can they have a different identity? Or is it the value alone that defines what they are? Try listing the problems you are trying to solve (model, UX, validation, …) and try to come up with a separate solution for each of them. Take decoupling to the max, poke around for missing concepts, and make the implicit explicit. Don’t think of database normalisation, relational integrity, performance, … those are all optimisations that you can add on top of your solution, as opposed to allowing your logical solution to be defined by the infrastructural constraints. And above all, experiment, fail, try again, learn!</p> <h2 id="read-more">Read More</h2> <ul> <li>Elephant in the Room podcast, <a href="http://elephantintheroom.io/blog/2013/10/episode-2-heart-and-soul-of-oop/">Episode #002 on Value Objects</a></li> <li><a href="http://www.jefclaes.be/2013/05/accidental-entities-you-dont-need-that.html">Accidental Entities - You don’t need that identity</a> by <a href="http://twitter.com/JefClaes">@JefClaes</a></li> <li><a href="http://www.jefclaes.be/2013/05/accidental-entities-you-dont-need-that.html">Accidental Entities - What about the UI?</a> by <a href="http://twitter.com/JefClaes">@JefClaes</a></li> <li><a href="/2013/12/related-entities-vs-child-entities/">Related Entities vs Child Entities</a></li> </ul> Pre-merge Code Reviews 2013-10-26T00:00:00+00:00 https://verraes.net/2013/10/pre-merge-code-reviews <p>I’m a big proponent of pre-merge code reviews. From my experience consulting for teams in problematic projects, I can say that (along with daily standup meetings) pre-merge code reviews are one of the most effective and yet fairly easy changes a team can introduce to radically improve the condition of the project. And even if you consider your project to be healthy, there’s always room for improvement.</p> <h2 id="the-rules">The Rules</h2> <ol> <li> <p><strong>Never Push to Master</strong><br /> Always push to a separate branch per logical unit (story, feature, bug, optimisation, refactor, improvement). Branches are easy to make and easy to merge when you use git (and you apply some of tips further down in this post).</p> </li> <li> <p><strong>Never Merge Your Own Branch</strong><br /> This helps to ensures that code is in fact reviewed. If you are caught merging into master, you will order pizza for the whole team.</p> </li> <li> <p><strong>Review Work in Progress First</strong><br /> When you are finished with a task, you notify the other team members that your work is ready for final review. Then you review existing branches. Before picking up a new task, you look at all open pull requests (including unfinished ones) and review the changes since the last time you checked.</p> </li> <li> <p><strong>Merge responsibly</strong><br /> Merging a pull request is the responsibility of the whole team. A pull request can not be merged when someone in the team does not understand the code or the reasoning, or does not agree with the solution.</p> </li> </ol> <p>(Note that these rules are starting points. Figure out what works in your team, adapt continuously.)</p> <h2 id="the-benefits">The Benefits</h2> <p>Having multiple sets of eyes review a pull request before it gets merged to master or an integration branch, is a great way to catch defects early. At this time, they are usually still cheap to fix.</p> <p>There are however much more important benefits. Instead of individual developers, the team is responsible for the internal and external quality of the code. This is a great remedy against the blame culture that is still present in many organisations. Managers or team members can no longer point fingers to an individual for not delivering a feature within the expected quality range. You tend to become happier and more productive, knowing that the team has your back. You can afford to make a mistake; someone will find it quickly.</p> <p>Another effect is something called ‘swarming’ in Kanban. Because you are encouraged to help out on other branches before starting your own, you start to help others finishing work in progress. Stories are finished faster, and there’s a better flow throughout the system. Especially when stories are difficult, or when stories block other stories, it’s liberating to have people come and help you to get it done.</p> <p>And of course, there’s all the benefits from the clear sense of code co-ownership. It’s invaluable to have a team where everybody knows what the code does, why it’s designed that way, how everything fits together. It also reduces the Bus Factor: no single team member is a bottleneck. Best practices are shared, and the code is more consistent. Opportunities for reuse are spotted before lots of duplication happens.</p> <p>In short, pre-merge code reviews grow the team’s maturity.</p> <h2 id="making-it-easier">Making It Easier</h2> <p>Reading code is hard, much harder than writing it. Here are some ideas that I have found to make things easier.</p> <ul> <li> <p><strong>Atomic Stories</strong><br /> Break down stories in smaller, more atomic stories. “Manage products” is way to large. “Add products”, “Add an image to a product”, “Modify the description of a product”, “Remove a product”, “Set a new price”: these are atomic, and it will make branches much easier to review and merge.</p> </li> <li> <p><strong>Acceptance Criteria</strong><br /> List the acceptance criteria in the pull request. (In GitHub, you can use <a href="https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments">checkboxes in Markdown</a>, making it easy to tick them).</p> </li> <li> <p><strong>Atomic Commits</strong><br /> Commit atomically. For example, don’t do code formatting and code changes in one commit, it makes the diffs very hard to read. Each commit should preferably leave the code base in a consistent state, but apart from that, there’s no limit to how small the commits can be. I often commit single lines.</p> </li> <li> <p><strong>Pull Requests as Conversations</strong><br /> A pull request is a conversation. Open it when you push your first commit. Comment, ask for feedback, commit to other people’s branches if that helps them out. The pull request is the story of how the feature came into being.</p> </li> <li> <p><strong>Integrate Frequently</strong><br /> Pull master into your branch regularly, to avoid difficult merges.</p> </li> <li> <p><strong>Pair on Merges</strong><br /> Pair program on difficult merge conflicts, preferably with the person who wrote the code you are merging with. (Pair programming in general is highly recommended, but off topic here.)</p> </li> <li> <p><strong>Review Tests Only</strong><br /> If you use TDD, and you are short on time, review only the tests. If these are satisfactory, the implementation will probably be ok too.</p> </li> <li> <p><strong>Solve the Problem Before Coding</strong><br /> Catching problems early when the code is written is great, but catching them before the code is written, is better. Go to the whiteboard with a couple of team members and draw out a solution everybody agrees with. Reviewing the code afterwards will be easier because you already have a model on the whiteboard.</p> </li> </ul> <p>I’m assuming in this post that you use GitHub, but there are other solutions, such as ReviewBoard, BarKeep, Phabricator, and Gerrit. I did a little bit of research last year, and felt that GitHub was the best tool, but I have no hands-on experience with the others. YMMV. The important factor for me is that you can review stories or branches as a whole, and comment inline. (Update: @tvlooy suggested GitLab.)</p> <h2 id="pitfalls-and-anti-patterns">Pitfalls and anti-patterns</h2> <ul> <li> <p><strong>Merge Buddies</strong><br /> As soon as you impose rules, you’ll find people who try to game the system. Sometimes two team members consistently merge each other’s pull requests, without actually looking at them. “I’ll merge yours if you merge mine.” They appear to conform to the rules, but of course it’s just a variation on their old habit of pushing to master. Catch this early. Gatekeeper can be a good remedy.</p> </li> <li> <p><strong>GateKeeper</strong><br /> One person is responsible for doing all the reviews and the merges. It’s not necessarily an anti-pattern, as it can be useful to prevent people from cheating. But if you keep having a long term Gatekeeper, you have serious problems. People will not feel responsible for their code, let alone others. They will not learn from other people’s code, and they will adapt their code to the Gatekeeper’s comments, without bothering to understand why. Avoid when possible.</p> </li> <li> <p><strong>Total Lockdown</strong><br /> One step further than having a Gatekeeper, is taking away everybody’s permissions to touch the main repository, and forcing them to use their own forks and send pull requests from there. It an be a good measure if the situation is really problematic, but it’s a terrible downer for morale. Deserved or not, people will feel distrusted. (Update: To clarify, it’s not a problem if they never had pull permissions from the beginning.)</p> </li> <li> <p><strong>The Apologists</strong><br /> “We don’t have time for code reviews.” Or: “Code reviews do not apply in this project.” As with all discussions about software quality, there will always be people who feel it’s perfectly alright to save time writing junk first, and then waste huge amounts of it bugfixing and putting out fires. They probably don’t write tests either, or refactor code. This is a deeper issue that won’t be fixed by enforcing code reviews alone.</p> </li> <li> <p><strong>Cowboys and Hermits</strong><br /> Some people don’t want their code looked at. They write their share, and guard it from others, by making sure they are the only ones working on that area. That’s unacceptable, and you’ll need to figure out why they act this way.</p> </li> <li> <p><strong>Issue Trackers</strong><br /> Many teams discuss issues inside their issue trackers. The distance from the actual code is too great. GitHub’s inline commenting, on individual lines, allows developers to talk about actual code instead of abstract descriptions of code.</p> </li> <li> <p><strong>Focus on Junk Details</strong><br /> If your code reviews focus on having the correct amount of spaces in the indentation, or typos in comments, you are wasting the effort. Focus on the important issues: Are the tests sufficiently conclusive? Is the naming expressive? Is there coupling where none should be? Will we still understand the code 12 months from now? Is it refactorable? …</p> </li> </ul> <p>As usual, I will keep adding to this post as I find more patterns. Got tips?</p> <p>Update: Apparently Phil Haack <a href="http://haacked.com/archive/2013/10/28/code-review-like-you-mean-it.aspx">blogged about code reviews</a> almost at the exact same time. He has some great tips like keeping a checklist, focusing on the code and not the author, and stepping through the code. Worth reading!</p> <h2 id="read-next">Read next</h2> <ul> <li><a href="/2016/04/code-reviews-and-blame-culture/">Code Reviews and Blame Culture</a></li> <li><a href="/2014/12/how-much-testing-is-too-much/">How Much Testing is Too Much?</a></li> <li><a href="/2014/03/small-controlled-experiments/">Small Uncontrolled Experiments</a></li> </ul> Verbs in Class Names 2013-10-06T00:00:00+00:00 https://verraes.net/2013/10/verbs-in-class-names <p>When you first learned Object Oriented Programming, somebody probably told you that objects map to things. And that still holds true most of the time. And maybe somebody explained it using the simple heuristic to “look for the nouns”, which can indeed be a great discovery technique. But then somebody probably phrased that as “class names should not have verbs in them”. To me, that rule is severely limiting the possibilities for your models. So here are some cases where I prefer to use verbs.</p> <h2 id="messages">Messages</h2> <p>The king sends a message to the general: “Attack the enemy on the south flank with three divisions”. The general does as he’s told, and succeeds. He sends a message back to the king: “The enemy is defeated!”. Peace is restored in the <a href="http://steve-yegge.blogspot.co.uk/2006/03/execution-in-kingdom-of-nouns.html">Kingdom of Nouns</a>.</p> <p>The messages that are sent back and forth, are “things”, so they can be modeled as objects. Message objects have all the benefits of OOP: you can pass them around, have type hinting, encapsulation… They can have properties: <code class="language-plaintext highlighter-rouge">private $flank = 'south'</code>, <code class="language-plaintext highlighter-rouge">private $numberOfDivisions = 3</code>. You can serialize the messages, send them over the wire, persist them in an audit log, or even an event store. You can publish the messages, or put them in a message queue, using ZeroMQ or an alternative.</p> <p>If you know me at all, you know I’m obsessed with naming. So what should we name these two messages? We could use the nouns “Attack” and “Defeat”, as in <code class="language-plaintext highlighter-rouge">AttackMessage</code>, <code class="language-plaintext highlighter-rouge">DefeatMessage</code>. (English is a bit messy in this regard, as nouns and verbs are often exactly the same word – an attack, to attack.) To me, this naming is too vague, too implicit. Let’s try a bit harder. Both message objects are very much alike. Are we missing a concept? If we focus on the differences, then we discover that one of them is a (Domain-) Command: The king tells the general what to do. The other one is a (Domain) Event: it’s something that happened in the past, that is of interest to the king: the enemy was defeated. <code class="language-plaintext highlighter-rouge">AttackCommand</code>, <code class="language-plaintext highlighter-rouge">DefeatEvent</code>? Close.</p> <p>The most expressive option is to name it after its purpose. <code class="language-plaintext highlighter-rouge">AttackEnemy implements Command</code>, <code class="language-plaintext highlighter-rouge">EnemyWasDefeated implements Event</code>. Again, we have the problem with English here. But this time it’s either an imperative verb (“Do This”), or a past tense (“This happened”). More examples are <code class="language-plaintext highlighter-rouge">CheckoutBasket</code>, <code class="language-plaintext highlighter-rouge">PayForOrder</code>, <code class="language-plaintext highlighter-rouge">OrderWasPaid</code>, <code class="language-plaintext highlighter-rouge">OrderWasShipped</code>.</p> <p>There’s no ambiguity about what any of these objects represents. And you can use them to build expressive api’s: <code class="language-plaintext highlighter-rouge">$auditLog-&gt;recordThat(new EnemyWasDefeated($onDate));</code> Imagine the <code class="language-plaintext highlighter-rouge">new</code> keyword isn’t there, and what you end up with is a nice sentence, saying exactly what it does: the audit log records that the enemy was defeated on a certain date.</p> <h2 id="specifications">Specifications</h2> <p>The Specification pattern is a way to model business rules as individual objects – I talk about this in <a href="/2013/06/unbreakable-domain-models/">Unbreakable Domain Models</a>. The idea is that a question about an object, is answered by a <code class="language-plaintext highlighter-rouge">isSatisfiedBy()</code> method:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">CustomerIsPremium</span> <span class="kd">implements</span> <span class="nc">CustomerSpecification</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$someCollaborator</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* ... */</span> <span class="p">}</span> <span class="cd">/** @return bool */</span> <span class="k">public</span> <span class="k">function</span> <span class="n">isSatisfiedBy</span><span class="p">(</span><span class="kt">Customer</span> <span class="nv">$customer</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// figure out if the customer is indeed premium and return true or false.</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Other examples are <code class="language-plaintext highlighter-rouge">OrderIsReadyForShipment</code>, <code class="language-plaintext highlighter-rouge">BankAccountIsWithinCreditLimit</code>, … Again, having the verb in there makes it much more natural. They are small, composable objects, and you can build sentences from them.</p> <h2 id="exceptions">Exceptions</h2> <p>Exceptions can be similar to events, as they describe something that has happened in your system, with the nuance that it is an undesirable event, a warning that some operation was inconsistent with, for example, business rules that are in effect. For application- or infrastructure-level exceptions, I don’t really bother much: <code class="language-plaintext highlighter-rouge">DbException</code> or <code class="language-plaintext highlighter-rouge">FSException</code> are fine, the message string will usually tell you more, and it’s all very technical anyway. But for domain model exceptions, I want it, once more, in the language of the business: <code class="language-plaintext highlighter-rouge">OrderShipmentHasFailed</code>, <code class="language-plaintext highlighter-rouge">BackAccountWasOverdrawn</code>. I don’t generalize them either: I have an individual exception class for each specific occasion.</p> <h2 id="interfaces">Interfaces</h2> <p>Interfaces are not classes, but close enough for inclusion. I go on about them in <a href="/2013/09/sensible-interfaces/">Sensible Interfaces</a>, but let’s hammer it some more.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">FireEmployee</span> <span class="kd">implements</span> <span class="nc">DomainCommand</span><span class="p">,</span> <span class="nc">HasPermissions</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getPermissions</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="p">[</span><span class="s1">'ROLE_CEO'</span><span class="p">,</span> <span class="s1">'ROLE_HR_MANAGER'</span><span class="p">];</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>I feel like I’m ranting here, but once, again, it’s a sentence in natural language. You can talk to another developer and say “This domain command has permissions”. There is no additional translation in your brain that converts that sentence to code such as <code class="language-plaintext highlighter-rouge">implements Permissionable</code>.</p> <p>The sentence <em>is</em> the code.</p> <p>If you have more types of classes where you use verbs, let me know!</p> <h3 id="read-more">Read more</h3> <ul> <li><a href="/2013/09/sensible-interfaces/">Sensible Interfaces</a></li> <li><a href="http://steve-yegge.blogspot.co.uk/2006/03/execution-in-kingdom-of-nouns.html">Execution in the Kingdom of Nouns</a></li> <li><a href="/2013/06/unbreakable-domain-models/">Unbreakable Domain Models</a></li> </ul> Sensible Interfaces 2013-09-24T00:00:00+00:00 https://verraes.net/2013/09/sensible-interfaces <p>Here are some guidelines to help you live in friendly cohabitation with your fellow rockstar ninja code monkeys. I’ll update them as my ideas about the matter evolve.</p> <h2 id="naming">Naming</h2> <p>In my team, suffixing your interface with the word <code class="language-plaintext highlighter-rouge">Interface</code>, is a sackable offence. Well, maybe it’s not <em>that</em> bad, but don’t bother trying it. The same goes for prefixing it with <code class="language-plaintext highlighter-rouge">I</code> – looking at you Microsoft. Here’s the reasoning.</p> <p>If you have an interface, then you are suggesting that multiple implementations of that interface are possible. Usually, when you see something like <code class="language-plaintext highlighter-rouge">TranslatorInterface</code>, chances are there’s an implementation called <code class="language-plaintext highlighter-rouge">Translator implements TranslatorInterface</code>. It makes me wonder: what makes the <code class="language-plaintext highlighter-rouge">Translator</code> so special, that it has the unique right of being called <code class="language-plaintext highlighter-rouge">Translator</code>? Every other implementation needs a descriptive name, such as <code class="language-plaintext highlighter-rouge">XmlTranslator</code> or <code class="language-plaintext highlighter-rouge">CachedTranslator</code>, but that one is somehow “the default”, as suggested by it’s preferential treatment in being named <code class="language-plaintext highlighter-rouge">Translator</code> without a description.</p> <p>Is that bad? I believe it is. It confuses people, as they tend to misunderstand whether they should typehint for <code class="language-plaintext highlighter-rouge">Translator</code> or <code class="language-plaintext highlighter-rouge">TranslatorInterface</code>. So both are being used in the client code. Program sometimes to an interface, sometimes to an implementation?</p> <p>Just as bad, is that it suggests that the implementation is the real thing, and the interface is a label. It should be the other way around. Look at it from the point of view of the client code:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">KlingonDecoder</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">TranslatorInterface</span> <span class="nv">$translator</span><span class="p">)</span> </code></pre></div></div> <p>This constructor definition is saying: “I need a translator <strong>interface</strong> to operate”. But that would be silly. It needs an object that is a <code class="language-plaintext highlighter-rouge">Translator</code>. It does not need an interface. And that object has a certain role, a certain contract, namely that of a <code class="language-plaintext highlighter-rouge">Translator</code>. I hope I’m making this clear. The interface <code class="language-plaintext highlighter-rouge">Translator</code> is the essential concept, the thing that clients use. They don’t care whether <code class="language-plaintext highlighter-rouge">Translator</code> is a concrete class or an interface, and they don’t care how it’s implemented. The client wants to be decoupled from all those details. That’s the power of interfaces.</p> <h2 id="default-implementations">Default implementations</h2> <p>The burden of having a descriptive name then lies with the implementations. If we rename <code class="language-plaintext highlighter-rouge">TranslatorInterface</code> to <code class="language-plaintext highlighter-rouge">Translator</code>, our former <code class="language-plaintext highlighter-rouge">Translator</code> class needs a new name. People tend to solve this problem by calling it <code class="language-plaintext highlighter-rouge">DefaultTranslator</code>. Again, what makes it so special to be called <code class="language-plaintext highlighter-rouge">Default</code>? Don’t be lazy, think really hard about what it does, and why that’s different from other possible implementations. You might even discover a thing or two about that class, such as having too many responsibilities.</p> <h2 id="nameable">Nameable</h2> <p>Another bad habit, is using the <code class="language-plaintext highlighter-rouge">-able</code> suffix for interface names. I guess I can live with something like <code class="language-plaintext highlighter-rouge">Translatable</code>, or maybe <code class="language-plaintext highlighter-rouge">Serializable</code>. But <code class="language-plaintext highlighter-rouge">Timestampable</code>? <code class="language-plaintext highlighter-rouge">Jsonable</code>? Is that the world we want our children to inherit? English motherfucker, do you speak it? Try making a sentence, it’s so much nicer.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">Product</span> <span class="kd">implements</span> <span class="nc">CastsToJson</span><span class="p">,</span> <span class="nc">HasTimestamp</span> </code></pre></div></div> <p>Say it out loud: “Product casts to json and has a timestamp”. It’s beautiful, it’s – dare I say it? – Shakespearian.</p> <h2 id="respect-the-contract">Respect the contract</h2> <p>PHP, having grown organically (to put it politely), is rather permissive when it comes to interfaces. Look at this code:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">interface</span> <span class="nc">Animal</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">makeNoise</span><span class="p">();</span> <span class="p">}</span> <span class="kd">class</span> <span class="nc">Dog</span> <span class="kd">implements</span> <span class="nc">Animal</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">makeNoise</span><span class="p">()</span> <span class="p">{}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">fetchStick</span><span class="p">()</span> <span class="p">{}</span> <span class="p">}</span> <span class="c1">// elsewhere:</span> <span class="k">public</span> <span class="k">function</span> <span class="n">myClient</span><span class="p">(</span><span class="kt">Animal</span> <span class="nv">$animal</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$animal</span><span class="o">-&gt;</span><span class="nf">fetchStick</span><span class="p">();</span> <span class="p">}</span> </code></pre></div></div> <p>Even though <code class="language-plaintext highlighter-rouge">myClient()</code> accepts an <code class="language-plaintext highlighter-rouge">Animal</code>, and should have no knowledge whether the <code class="language-plaintext highlighter-rouge">$animal</code> is a <code class="language-plaintext highlighter-rouge">Dog</code>b, PHP allows you to call <code class="language-plaintext highlighter-rouge">fetchStick()</code> if <code class="language-plaintext highlighter-rouge">$animal</code> has a method by that name. That flexibility can be very useful, in a very limited set of cases. For all normal cases: never call a method on an object that is not part of the interface you are typehinting for.</p> <h2 id="interface-segregation">Interface segregation</h2> <p>If you find that your client code depends on an interface with many methods that your client doesn’t care about, your interface may be too big. The same is true if your classes implementing an interface, have a lot of unused stub methods:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">Fish</span> <span class="kd">implements</span> <span class="nc">Animal</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">makeNoise</span><span class="p">()</span> <span class="p">{</span> <span class="k">throw</span> <span class="k">new</span> <span class="nc">NotImplemented</span><span class="p">(</span><span class="s2">"Fish don't make noise"</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>This is a good sign that you need to split off the <code class="language-plaintext highlighter-rouge">makeNoise()</code> method into a separate interface. Perhaps <code class="language-plaintext highlighter-rouge">MakesNoise</code>, or <code class="language-plaintext highlighter-rouge">Noisy</code>?</p> <h2 id="roles">Roles</h2> <p>Interfaces can be a nice way to share code without the client knowing, by seeing them as roles. Say the product prices are in a database. You have some logic in <code class="language-plaintext highlighter-rouge">OrderBuilder</code>, but you don’t want <code class="language-plaintext highlighter-rouge">OrderBuilder</code> to know that the prices are in the database, because that might change in the future. You could solve this with composition.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">interface</span> <span class="nc">ProductRepository</span> <span class="p">{</span> <span class="cm">/* defines find(), add()... */</span> <span class="p">}</span> <span class="kd">interface</span> <span class="nc">ProductPricer</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">priceProduct</span><span class="p">(</span><span class="kt">Product</span> <span class="nv">$product</span><span class="p">);</span> <span class="p">}</span> <span class="kd">class</span> <span class="nc">DbProductPriceRepository</span> <span class="kd">implements</span> <span class="nc">ProductRepository</span> <span class="p">{</span> <span class="cm">/* implements find(), add()... */</span> <span class="p">}</span> <span class="kd">class</span> <span class="nc">DbProductPricer</span> <span class="kd">implements</span> <span class="nc">ProductPricer</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">ProductRepository</span> <span class="nv">$productRepository</span><span class="p">){</span> <span class="cm">/* ... */</span> <span class="p">}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">priceProduct</span><span class="p">(</span><span class="kt">Product</span> <span class="nv">$product</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* ... */</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>To save a bit of typing, you can give the <code class="language-plaintext highlighter-rouge">ProductPricer</code> role to <code class="language-plaintext highlighter-rouge">DbProductPricer</code>. Sure, it’s doing double duty, but the clients don’t know that. In this example, it’s probably not a good long term solution when your application grows, but if nothing else, it’s a great timesaver when prototyping.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">DbProductPriceRepository</span> <span class="kd">implements</span> <span class="nc">ProductRepository</span><span class="p">,</span> <span class="nc">ProductPricer</span> <span class="p">{</span> <span class="cm">/* implements find(), add()... */</span> <span class="k">public</span> <span class="k">function</span> <span class="n">priceProduct</span><span class="p">(</span><span class="kt">Product</span> <span class="nv">$product</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* ... */</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Roles of course work great with entities:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">Teacher</span> <span class="kd">implements</span> <span class="nc">User</span> <span class="p">{}</span> <span class="kd">class</span> <span class="nc">Pupil</span> <span class="kd">implements</span> <span class="nc">User</span> <span class="p">{}</span> <span class="kd">class</span> <span class="nc">Parent</span> <span class="kd">implements</span> <span class="nc">User</span> <span class="p">{}</span> </code></pre></div></div> <h2 id="one-implementation">One implementation</h2> <p><code class="language-plaintext highlighter-rouge">ProductPricer</code> is a great use case for an interface, because it’s easy to imagine different business rules applying in different circumstances: <code class="language-plaintext highlighter-rouge">GermanProductPricer</code>, <code class="language-plaintext highlighter-rouge">BelgianProductPricer</code>. There could be different technical implementations: <code class="language-plaintext highlighter-rouge">DbProductPricer</code>, <code class="language-plaintext highlighter-rouge">SoapProductPricer</code>, or a <code class="language-plaintext highlighter-rouge">CachedProductPricer</code> that wraps one of the other implementations.</p> <p>But often, it’s less clear. If your business has only One True Way to calculate prices, and One True Datasource to store them. My rule of thumb would be that if you can imagine that there could be more implementations than just the one, it’s good to have an interface. If you can’t imagine different implementations, don’t have an interface. An example could be <code class="language-plaintext highlighter-rouge">OrderTotalCalculator</code>: There’s only one valid way to sum the different prices, so an interface does not make sense.</p> <h2 id="aware">Aware</h2> <p>I’m sort of undecided about the <code class="language-plaintext highlighter-rouge">Aware</code> suffix. I don’t think it’s a problem per se. But of course <code class="language-plaintext highlighter-rouge">ContainerAware</code> is evil and should never have been in Symfony. But that is a Dependency Injection anti-pattern, not an interface naming problem. Thoughts?</p> <p>Read next: <a href="/2011/03/interface-discovery-with-phpunit-mock-objects/">Interface discovery with PHPUnit’s Mock objects</a> - March 21, 2011</p> Extract Till You Drop 2013-09-20T00:00:00+00:00 https://verraes.net/2013/09/extract-till-you-drop <p>“Extract Till You Drop” is a live refactoring demo that I used to do at conferences and as a webinar. There are no slides, it’s all coding.</p> <p><strong>Update Sep 6, 2014</strong> (Replaced my home made recording with the one from PHP Tour Lyon. The talk has evolved slightly during the months I presented it at conferences. Find <a href="https://www.youtube.com/playlist?list=PLKpYkFzrFTkGXp_AODGIyiNrm5wdySghs">other recordings of that talk on Youtube</a>.)</p> <iframe width="640" height="360" src="//www.youtube-nocookie.com/embed/1_dpOZmKXBw" frameborder="0" allowfullscreen=""></iframe> <p>The title refers to a quote from <a href="https://sites.google.com/site/unclebobconsultingllc/one-thing-extract-till-you-drop">Robert C. Martin</a> about applying Extract Method and other refactors until every element in your system has just one responsibility:</p> <blockquote>Perhaps you think this is taking things too far. I used to think so too. But after programming for over 40+ years, I’m beginning to come to the conclusion that this level of extraction is not taking things too far at all. In fact, to me, it looks just about right. So, my advice: Extract till you just can’t extract any more. Extract till you drop.</blockquote> <p>I owe a lot to Sandro Mancuso, who did a similar, much more advanced presentation in Java, which greatly inspired me to try this in PHP. Many of the ideas here are borrowed from him.</p> <h3 id="abstract">Abstract</h3> <p>We’ve all seen them: applications out of control. Under the pressure of deadlines and endless change requests, with the weight of years of legacy, the code has become unmaintainable. Adding features is a slow hit and miss process. You know something needs to be done, but nobody knows how. To change the code safely, you need tests, but to make it testable, you need to change it. Rebuilding the system from scratch is not an option.</p> <p>With the right tools, techniques, and mindset, any codebase can be brought under test, and be refactored towards a better architecture. All without affecting the behavior of the system, and allowing the business to continue.</p> <p>This presentation is not for the weak of heart. We’ll skip the theory and dive straight into the spaghetti code. We’ll wrestle in the mud, and we’ll tame the beast, whether it wants to or not. It’s live coding, and it may fail. But it will change how you approach legacy code forever.</p> <script async="" class="speakerdeck-embed" data-id="3ce44050041f0131954556f7ac4f018a" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> The DDDBE Modellathon 2013-09-05T00:00:00+00:00 https://verraes.net/2013/09/dddbe-modellathon <h3 id="goal">Goal</h3> <p>Many teams are not spending enough time on modeling. Often, they’re not particularly good at it, and often, the business considers time spent at the whiteboard “unproductive”. The goal of the Modellathon workshop, was to give people a safe environment to play around with different modeling techniques, without the pressure of producing something of immediate value. For more experienced attendees, it’s an opportunity to exercise their modeling skills on a new and unfamiliar domain.</p> <p>By collaborating with people, and by learning from the models of the other teams, the attendees get a glimpse of the myriad of techniques that exist (often not documented anywhere) to explore, grasp, and visualize, their understanding of the domain.</p> <h3 id="slides">Slides</h3> <script async="" class="speakerdeck-embed" data-id="105cfa00f86b0130e851227600bb7d26" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><a href="https://speakerdeck.com/mathiasverraes/dddbe-modellathon-2013">See the slides on Speakerdeck</a></p> <h3 id="what-we-did">What we did</h3> <p><img style="float:right;margin-left: 10px" src="/img/posts/2013-09-05-modellathon/mathias_verraes_at_dddbe.jpg" alt="Mathias at DDDBE" /></p> <p>We formed teams of four people. Each team was provided with plenty of <a href="http://www.ikea.com/be/nl/catalog/products/20152281/">paper</a>, markers, and stickies. Stijn and me played the role of domain experts. We gave a short introduction about “The United Schools of Kazachstan”. This domain was obviously made up, but most of what we talked about, comes from a real project we are working on. We gave the people a paper report card for <a href="https://twitter.com/DDD_Borat">a Kazach pupil</a>, and told them we wanted a better system for making the reports. Along the way we introduced more complexity: the different ways pupils can be evaluated, the laws and governmental regulations, the importance of the progress a pupil makes, and the confidentiality of the pupils.</p> <p>In two rounds of 25 minutes, the teams got to work to try and draw the problem space into a model. Meanwhile, Stijn and I walked around to answer questions. Just like real domain experts, we sometimes took things for granted, or casually mentioned some new piece of critical information.</p> <p>For the third round, we shifted the focus from understanding the domain, to modeling a somewhat workable solution. Yves and me now tried to give guidance on techniques, and give the teams some pointers. For example, I showed one team how to get started with <a href="https://verraes.net/2013/08/facilitating-event-storming/">Event Storming</a>. Another team had already produced a very detailed and complete model, so I told them that if this were a real application, the teachers would have to spend too much time entering data, and too little time actually teaching. This forced the team to re-evaluate some of their assumptions.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2013-09-05-modellathon/presenting_a_model_at_dddbe.jpg" alt="Presenting a model at DDDBE" /></p> <p>After each round, a couple of teams presented their progress. Some had UML-like drawings, others had mostly stickies with words, some had drawn UI mockups as a starting point. It was quite wonderful to see how distinct each team’s model was, despite the short time frame. Even after the first 25 minutes, it was surprising to see how much the teams had already captured of the problem space.</p> <p>At the end, we had an improvised round table discussion, about the challenges involved with modeling complex domains, where many people contributed their insights.</p> <h3 id="what-ive-learned-as-a-facilitator">What I’ve learned as a facilitator</h3> <h4 id="ask-the-right-question">Ask the right question</h4> <p>In attempt to distribute people of different levels evenly across the teams, I asked who was an experienced modeler. No reaction. (Belgians especially can be shy in this regard.) <del>Somebody (I think it was Jef)</del> Tom then asked “who had previously screwed up a model”. Some people raised their hands, so that proved a much more effective way to find the attendees with modeling experience!</p> <h4 id="struggling">Struggling</h4> <p>Somebody told me aftwerwards that he would have preferred to first get an overview of modeling techniques. On the other hand, struggling with a problem is a really good way to learn. It triggers your instinct to look for better ways. And, when somebody then shows you a new technique, you immediately recognize why it’s better. Perhaps, a good approach would be to have the teams try it for themselves during one or two rounds, and then have a short presentation on one or two techniques they can try. You’d probably need more time, 3-4 hours at least.</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2013-09-05-modellathon/tom_janssens_at_dddbe.jpg" alt="Presenting a model at DDDBE" /></p> <h4 id="be-very-attentive">Be very attentive</h4> <p>As a facilitator, it’s easy to get carried away with explaining the domain. It’s important to be aware of teams that have trouble moving forward. It’s better to step out of the domain expert role, and help them move forward. Perhaps a good approach would be to have different people in the expert and in the facilitator role (which is more or less what we did when Yves starting helping out in the third round.</p> <h4 id="throw-away-the-models">Throw away the models</h4> <p>We gave teams the choice of working on the same model round after round. In retrospect, it might have been better to explicitly ask them to throw everything away. This way, you give yourself the opportunity to start over with a different technique, borrowed from another team’s presentation. Some teams became quite attached to their models, which limited their progress. <a href="http://c2.com/cgi/wiki?PlanToThrowOneAway">Kill your darlings!</a></p> <h4 id="feedback-feedback-feedback">Feedback, Feedback, Feedback!</h4> <p>Using the Pomodoro’s and having a few minutes of feedback proved extremely useful. I tended to provide the options, and have the attendees choose, but soon I noticed that they came up with better ideas than me. So as a reminder to myself: the group is smarter than the individual!</p> <h3 id="conclusion">Conclusion</h3> <p>It was a miracle that we had so much attendees for our brand new <a href="http://domaindriven.be">Belgian Domain-Driven Design community</a>, and that all these people were prepared to run this experiment with us. In fact, I don’t think anything like a modellathon was ever attempted anywhere else. Let me know if you do.</p> <p>In any case, I can highly recommend running a modellathon of your own. And, incidentally, I’d love to come facilitate it :-)</p> <h3 id="more">More</h3> <ul> <li>Belgian Domain-Driven Design community <a href="http://domaindriven.be">domaindriven.be</a></li> <li><a href="http://twitter.com/DDDBE">@DDDBE</a> on Twitter</li> <li>Read next: <a href="/2013/08/facilitating-event-storming/">Facilitating Event Storming</a></li> </ul> Antifragile - Things That Gain from Disorder 2013-08-29T00:00:00+00:00 https://verraes.net/2013/08/antifragile-nassim-nicholas-taleb <p>What is the opposite of <strong>fragile</strong> (breaks when confronted with unexpected events)? Changes are people answer with <strong>robustness</strong> (stays the same under unexpected events). But there’s another way to look at it.</p> <table class="table table-bordered"> <thead> <tr> <th>&nbsp;</th> <th>Fragile</th> <th>Robust</th> <th>Antifragile</th> </tr> </thead> <tbody> <tr> <th>Best case scenario</th> <td>Remains unchanged</td> <td>Remains unchanged</td> <td>Improves</td> </tr> <tr> <th>Worst case scenario</th> <td>Breaks</td> <td>Remains unchanged</td> <td>Remains unchanged</td> </tr> </tbody> </table> <p>Taleb introduces the term <em>antifragile</em> for systems that don’t simply stay the same when subjected to stressors, volatility, and exceptions, but that actually change for the better.</p> <h2 id="wildfire">Wildfire</h2> <p>We tend to naively intervene in systems, without taking into account their natural antifragility. Take wildfire. If we attempt to stabilize - that is, prevent wildfires from happening - we believe we make the forest more robust. But over time, the combustible materials in the forest accumulate. When they finally do catch fire, the resulting damage is much larger. Even though the parts of the system, when left alone, are more fragile, the whole is antifragile. Just like bailing out banks can give us some temporary perceived stability, but can lead to bigger disasters in the long term.</p> <blockquote><p>Fire feeds on obstacles. <small>Marcus Aurelius</small></p></blockquote> <h2 id="taxi-driver">Taxi driver</h2> <p><img style="float:right;margin-left: 10px" src="/img/posts/2013-08-29/antifragile-taxidriver.png" alt="Antifragile taxidriver" /></p> <p>The book is full of great examples. Taleb tells of of a taxi driver and an office clerk. The clerk has a stable job: he gets paid at the end of the month, and he gets a promotion every five years. The taxi driver on the other hand, has good days and bad days. He’s constantly forced to be creative, experiment to find better spots to pick up customers, try honest as well as less honourable ways to increase his income. Then 2008 hits, the clerk suddenly loses all of his income at once, and finds himself on the street with no survival skills to speak of, and no experience with selling himself in interviews. The taxidriver might take a hit from the recession, but his years of being subjected to highs and low, made him antifragile.</p> <p>Even though small economic units may be fragile, the fragility of the parts can make the system as a whole more antifragile. (Compare to John Gall’s “An airplane is a system that consists of parts that tend to fall down.”)</p> <h2 id="the-book">The Book</h2> <p><a href="http://www.amazon.com/gp/product/B0083DJWGO/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=B0083DJWGO&amp;linkCode=as2&amp;tag=verraesnet-20&amp;linkId=Q2UYVYAWOKWT7F5V"> <img style="float:left;margin-right: 10px" src="/img/posts/2013-08-29/antifragile-book.png" alt="Antifragile - Things That Gain from Disorder" /> </a><img src="http://ir-na.amazon-adsystem.com/e/ir?t=verraesnet-20&amp;l=as2&amp;o=1&amp;a=B0083DJWGO" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /></p> <p>I feel Antifragile is one of the most important books I’ve read in ages. Neither my short explanation above, nor the quotes below do it justice. It’s hard to convey why it’s so gripping. Already I’m seeing everything in my life in the light of the triad fragile - robust - antifragile.</p> <h2 id="quotes">Quotes</h2> <blockquote>The chief ethical rule is the following: Thou shalt not have antifragility at the expense of the fragility of others.</blockquote> <blockquote>How do you innovate? First, try to get in trouble. I mean serious, but not terminal, trouble. I hold—it is beyond speculation, rather a conviction—that innovation and sophistication spark from initial situations of necessity, in ways that go far beyond the satisfaction of such necessity (...) The excess energy released from overreaction to setbacks is what innovates!</blockquote> <blockquote>we can estimate, even measure, fragility and antifragility, while we cannot calculate risks and probabilities of shocks and rare events, no matter how sophisticated we get. Risk management as practiced is the study of an event taking place in the future, and only some economists and other lunatics can claim—against experience—to “measure” the future incidence of these rare events, with suckers listening to them—against experience and the track record of such claims. But fragility and antifragility are part of the current property of an object, a coffee table, a company, an industry, a country, a political system. We can detect fragility, see it, even in many cases measure it, or at least measure comparative fragility with a small error while comparisons of risk have been (so far) unreliable.</blockquote> <blockquote>A complex system, contrary to what people believe, does not require complicated systems and regulations and intricate policies. The simpler, the better. Complications lead to multiplicative chains of unanticipated effects. Because of opacity, an intervention leads to unforeseen consequences, followed by apologies about the “unforeseen” aspect of the consequences, then to another intervention to correct the secondary effects, leading to an explosive series of branching “unforeseen” responses, each one worse than the preceding one.</blockquote> <blockquote>We hear about the more lurid post-traumatic disorder, not post-traumatic growth, in the intellectual and so-called learned vocabulary. But popular culture has an awareness of its equivalent, revealed in the expression “it builds character.” So do the ancient Mediterranean classics, along with grandmothers.</blockquote> <blockquote>In the complex world, the notion of “cause” itself is suspect; it is either nearly impossible to detect or not really defined—another reason to ignore newspapers, with their constant supply of causes for things.</blockquote> <blockquote>So some parts on the inside of a system may be required to be fragile in order to make the system antifragile as a result. Or the organism itself might be fragile, but the information encoded in the genes reproducing it will be antifragile. The point is not trivial, as it is behind the logic of evolution. </blockquote> <blockquote>This is the central illusion in life: that randomness is risky, that it is a bad thing—and that eliminating randomness is done by eliminating randomness.</blockquote> <p>Read next: <a href="/2013/08/john-gall-systemantics-the-systems-bible/">Systemantics - The Systems Bible</a> - August 10, 2013</p> Facilitating Event Storming 2013-08-28T00:00:00+00:00 https://verraes.net/2013/08/facilitating-event-storming <p>Event Storming is a technique where you get the developers and the business stakeholders in a room, and visualize the business processes. You do this using stickies with domain events and causality (such as commands), and end with drawing boundaries for aggregates, bounded contexts and subdomains. It’s developed by <a href="http://ziobrando.blogspot.be/">Alberto Brandolini</a> and is deeply rooted in Domain Driven Design, and CQRS/ES.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2013-08-28/event-storming.jpg" alt="Event Storming at Qandidate.com" /></p> <p>I facilitated an Event Storming session with the team of Qandidate.com in Rotterdam. I’ve read an early draft of Alberto’s paper on Event Storming, that inspired most of what I did. It was my first time doing this as a facilitator, so I’d thought I’d share my notes. It’s just the raw material – I was going to process them into something more readable first, but <a href="http://www.jefclaes.be/">Jef Claes</a> convinced me to just post them as is. So forgive me if they are vague; I promise I will blog about Event Storming more later.</p> <p>Despite my rookie mistakes (too much intervention, then too little) the whole thing went very well. The team was very excited, they want to do it again with their founders. We also did a bit of context mapping for the legacy systems of their parent company, but time ran out.</p> <h2 id="notes">Notes</h2> <p>I think there are three main reasons for Model Storming:</p> <ul> <li>Discovering complexity early on, finding missing concepts, understanding the business process;</li> <li>Modelling or solving a specific problem in detail;</li> <li>Learning how to model and think about modelling.</li> </ul> <p>Facilitator tips:</p> <ul> <li>Hang the first sticky yourself (a tip from Alberto, works really well)</li> <li>Know when to step back. Don’t do the modelling, guide the modelling</li> <li>Ask questions: <ul> <li>Is there something missing here? Why is there a gap?</li> <li>How does this make money?</li> <li>How does the business evaluate that this is working? What are the targets, how will we know we’ve reached them?</li> <li>For whom is this event of importance (end user, business, tenant,…) ?</li> <li>I can’t see this particular role, or type of user, in this model. Should they be on here somewhere?</li> </ul> </li> <li>Change the direction, e.g. start at the end of the flow, move back in time, then later start at the beginning and move forward.</li> <li>Interrupt long discussions. Visualise both opinions, and, very important: ask both parties if they feel their opinion is accurately represented.</li> <li>Timebox, using pomodoro’s (25 minutes). After each pomodoro, ask what is going well and what isn’t. It’s a good opportunity to move to the next phase (e.g. from adding events to adding causality, to drawing aggregate boundaries). You may want to move on even if you don’t feel the model is complete.</li> <li>Constantly move stickies to create room for hotspots.</li> <li>Hang red stickies with exclamation marks, question marks, or other notes, anywhere you feel there’s an issue.</li> <li>At the end, make a photo. Then tell them to throw the model away, and to do it over the next day. If possible in the presence of other stakeholders.</li> <li>I personally prefer sentence-style event names (“A product was added to a basket” vs “Product added to basket”). I believe this makes the business people feel more comfortable.</li> </ul> <h2 id="read-more">Read More</h2> <ul> <li><a href="/2014/07/event-storming-fake-domains-happy-paths/">Event Storming: on Fake Domains and Happy Paths</a></li> <li><a href="/2015/03/event-storming-storytelling-visualisations/">Event Storming, Storytelling, Visualisations</a></li> <li><a href="/2014/01/domain-driven-design-is-linguistic/">Domain-Driven Design is Linguistic</a></li> <li><a href="/2013/09/dddbe-modellathon/">The DDDBE Modellathon</a></li> <li><a href="/2014/01/model-storming-workshop/">Model Storming Workshop</a></li> </ul> Switch - How to Change Things When Change Is Hard 2013-08-18T00:00:00+00:00 https://verraes.net/2013/08/chip-dan-heath-switch <p><img style="float:left;margin-right: 10px" src="/img/books/switch.jpg" alt="Switch - How to Change Things When Change Is Hard" /></p> <p>As I <a href="/2013/08/john-gall-systemantics-the-systems-bible/">wrote earlier</a>, I want to post some quotes and a mini-review of the books I read. “<a href="http://www.amazon.com/Switch-Change-Things-When-Hard/dp/0385528752">Switch</a>” compares humans in change processes to Rider (our logical side), riding an Elephant (our emotional side) down a Path (the environment). By directing the Rider, motivating the Elephant, and shaping the Path, you can change yourself, a team, an organization, or even society. As with all frameworks, it’s limited (which the authors admit upfront). However, as a consultant who’s made it his professional ambition to help teams and organizations build better software, I definitely found a lot of useful advise. If nothing else, the book will help me see which areas of a change process I haven’t paid sufficient attention to. What most appealed to me, was the idea of finding the bright spots. We tend to focus on problems, but if we look at the things people are doing that are in fact working, we can imitate and multiply those.</p> <p><br /><br /><br /></p> <h3 id="environment">Environment</h3> <blockquote>What looks like a people problem is often a situation problem.</blockquote> <p>(Contrast this with Weinberg’s «It’s always a people problem.» I don’t think the statements are mutually exclusive.)</p> <blockquote>(...) people have a systematic tendency to ignore the situational forces that shape other people's behavior. He called this deep-rooted tendency the "Fundamental Attribution Error." The error lies in our inclination to attribute people's behavior to *the way they are* rather than to *the situation they are in*.</blockquote> <h3 id="identity">Identity</h3> <blockquote>Because identities are central to the way people make decisions, any change effort that violates someone's identity is likely doomed to failure. (That's why it's so clumsy when people instinctively reach for 'incentives' to change other people's behavior.) So the question is this: How can you make your change a matter of identity rather than a matter of consequences?</blockquote> <p>I can recall a number of occasions where a manager attempted to bribe a team into delivering faster, at lower quality, even though the team was priding itself on its craftsmanship.</p> <blockquote>If you want to change the culture of your organization, you've got to get the reformers together. (...) Counterintuitively, you've got to let your organization have an identity conflict. For a time, at least, you've got to permit an 'us versus them' struggle to take place. We know this violates our 'we're all on the same team,' Kumbaya-ish instincts. It's not desirable, but it's necessary.</blockquote> <blockquote>Also, *cognitive dissonance* works in your favor. People don't like to act in one way and think in another. So once a small step has been taken, and people have begun to act in a new way, it will be increasingly difficult for them to dislike the way they're acting. Similarly, as people begin to act differently, they'll start to think of themselves differently, and as their identity evolves, it will reinforce the new way of doing things.</blockquote> <h3 id="growth-mindset">Growth mindset</h3> <blockquote>In the business world, we implicitly reject the growth mindset. Businesspeople think in terms of two stages: You plan, and then you execute. There's no 'learning stage' or 'practice stage' in the middle. From the business perspective, practice looks like poor execution. Results are the thing: *We don't care how ya do it, just get it done.*</blockquote> <p>Learning is the bottleneck. Also see: «The management question, therefore, is not <em>whether</em> to build a pilot system and throw it away. You will do that. (…) Hence plan to throw one away; you will, anyhow.» - Fred Brooks</p> <blockquote>Everything can look like a failure in the middle.</blockquote> Systemantics - The Systems Bible 2013-08-10T00:00:00+00:00 https://verraes.net/2013/08/john-gall-systemantics-the-systems-bible <p><a href="http://jonjagger.blogspot.com/">Jon Jagger</a> regularly posts snippets from books he’s read on his blog. I think this is an excellent habit, that has helped me to discover some very interesting books, so I want to start doing the same.</p> <h3 id="the-book">The book</h3> <p><a href="http://www.amazon.com/dp/B00AK1BIDM/ref=r_soa_w_d">“Systemantics. The Systems Bible” by John Gall</a> is a classic work from 1975. It deals mostly with the idea that one cannot simply design a large complex system, and expect to predict its entire set of behaviours. It’s one of those books that help you see the world as it is. Although it mostly deals with failing systems and the dangers of complexity, it is, almost inadvertently, a book on problem solving – even though it claims that “problems are not the problem. Coping is the problem.”</p> <h3 id="quotes">Quotes</h3> <p><img style="float:left;margin-right: 10px" src="/img/books/systemantics.jpg" alt="Systemantics - The Systems Bible" /></p> <blockquote>In setting up a new System, tread softly. You may be disturbing another System that is actually working.</blockquote> <blockquote>The word "Solution" is only a fancy term for the Response of System A (ourselves) to System B (the Problem). And its a misleading word, because it implies something that can be done once and for all. But System B is sure to Kick Back in response to our Response, and then we must respond once again.</blockquote> <blockquote>As events get further and further ahead of the System's inappropriate responses, the Model of the Universe generated in the control room by such Wishful Feedback bears less and less resemblance to outside reality. The System *hallucinates* its way to Terminal Instability. In summary: Just calling it "Feedback" doesn't mean that is had actually fed back. To speak precisely: It hasn't fed back until the System changes course.</blockquote> <blockquote>And when Anthropologists began to study tribes in their native habitats, they increasingly began to notice that the typical New Guinea family consisted of father, mother, three children, and one or more Anthropologists. In brief, there can be no System without its Observer.</blockquote> <blockquote>Systems are seductive. They promise to do a hard job faster, better, and more easily than you could do it by yourself. But if you set up a System, you are likely to find your time and effort now being consumed in the care and feeding of the System itself. New Problems are created by its very presence. Once set up, it won't Go Away; it Grows and Encroaches. It begins to do Strange and Wonderful Things and Breaks Down in Ways You Never Thought Possible. It Kicks Back, Gets In The Way and Opposes Its Own Proper Function. Your own perspective becomes distorted by being In The System. You become anxious and Push On It To Make It Work. Eventually you come to believe that the misbegotten product it so grudgingly delivers is What You Really Wanted all the time. At that point, Encroachment has become complete. You have become absorbed. You are now a Systems-person.</blockquote> <blockquote>(..) persons or Systems in this unblessed state often adhere strongly to a bizarre Systems-delusion, the Inevitability-of-Reality fallacy, the belief that: Thing have to be the way they are and not otherwise because that's just the way they are.</blockquote> <blockquote>The problem is a Problem precisely because it is incorrectly conceptualized in the first place, and a large System for studying and attacking the Problem merely locks in the erroneous conceptualization into the minds of everyone concerned. What is required is not a large System, but a *different approach*. Trying to design a System in the hope that the System will somehow solve the Problem, rather than simply solving the Problem in the first place, is to present oneself with two problems in place of one.</blockquote> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/01/build-for-failure/">Build for Failure</a></li> <li><a href="/2013/08/antifragile-nassim-nicholas-taleb/">Antifragile - Things That Gain from Disorder</a></li> </ul> Managed Technical Debt 2013-07-21T00:00:00+00:00 https://verraes.net/2013/07/managed-technical-debt <p><strong>Update 2020-01-22 <a href="/2020/01/wall-of-technical-debt/">I wrote a new version of this</a>.</strong></p> <p><a href="https://twitter.com/mathiasverraes/status/326435664319111170">Alberto Brandolini</a> said at IDDD Belgium (quoting from memory):</p> <blockquote>We've always explained 'technical debt' badly to the business. If you have debt with a bank, you can talk to someone, negotiate, and agree on a payment plan. But technical debt is like debt with the mob: they come at night, pointing a gun to your head, and they want their money NOW.</blockquote> <p><a href="http://codebetter.com/gregyoung/2013/03/06/startups-and-tdd/">Greg Young</a> on the other hand, has talked on several occasions about how technical debt is not necessarily bad. Businesses take loans all the time. They are a powerful tool to make an investment now, and pay for it later. It’s a fundamental aspect of our economy.</p> <h3 id="shortcuts">Shortcuts</h3> <p>I get very irritated when a non-technical person tells me to take shortcuts to meet a deadline. Or – even worse – when a developer-turned-project-manager does so. Technical debt is often what made the project slow in the first place. And as a consultant being more and more specialized in “problem projects”, I’ve seen a lot of crappy code that grinds projects to a halt.</p> <p>And yet, I myself tell teams to take shortcuts all the time. But there’s a difference. I know the codebase intimately. I know where it’s problematic, and where it’s under control. I know what is covered with tests, which parts are hurting us, and which parts are simply not elegant. And most importantly: when I introduce technical debt, I know how to get rid of it, and I refactor as soon as I feel it’s needed. I add comments in the code suggesting how to change it, and often I hang a sticky note in the backlog. Whenever new code is affected by earlier debt, I bump the priority of the backlog item.</p> <p><img style="float:left;margin-right: 10px" src="/img/posts/2013-07-21-managed-technical-debt/map-of-technical-debt.jpg" alt="Mapped Technical Debt (bottom)" /></p> <h3 id="team-discipline">Team discipline</h3> <p>When opinions are at odds, like in the quotes above, there’s often a problem in the language. There’s one term, and two concepts. In the interest of making the implicit explicit, I’d like to propose that we make a clear distinction between <strong>Managed Technical Debt</strong>, and <strong>Unmanaged Technical Debt</strong>. The former is defined as technical debt where most of the following conditions are present:</p> <ul> <li>The team is disciplined: it applies techniques such as refactoring, pair programming, iterative development, CI, TDD (or other testing schemes) …</li> <li>The codebase is well structured, well tested, and stable.</li> <li>When introducing new technical debt, the team estimates the cost/benefit.</li> <li>Even though the introduced solution is not optimal, the team understands what the optimal solution would be, and how the current solution can later be evolved towards that goal.</li> <li>Technical debt is documented.</li> <li>Technical debt is not introduced on top of existing technical debt.</li> <li>Existing technical debt is paid off as soon as there’s a clear need.</li> </ul> <p>For Unmanaged Technical Debt on the other hand, most of the opposite conditions are present:</p> <ul> <li>The team lacks maturity.</li> <li>Hacks are built as workarounds for the effects of previous hacks.</li> <li>The team is unaware that a newly introduced solution is not optimal,</li> <li>or the team is aware, but does not understand what an optimal solution would be or how to achieve it.</li> </ul> <h3 id="how-we-manage-it">How we manage it</h3> <p>(Update September 12, 2013)</p> <p>The photo above shows a project’s story map, and the bottom part is the map of technical debt, in columns per categories (blue stickies). The green stickies are prioritized, and the priorities regularly change: whenever we find that something makes us slower, the stickies move up. Sooner or later they move to the kanban board to be addressed in the sprint. We informally use the “three strikes and you refactor” rule of thumb, but mostly it’s gut feeling, experience, and discussion that help us decide whether something needs fixing – for example when we believe that future stories will be affected.</p> <p>Visualizing the story map and the technical debt, is incidentally a great way to relieve stress: sure, there’s a lot of work ahead, but it never feels unmanageable.</p> <p>(Update July 22, 2014)</p> <p>Some people have started using the technique after seeing my lightning talk about it. Some early pictures:</p> <p><img style="width:340px; border: 1px solid gray" src="/img/posts/2013-07-21-managed-technical-debt/technical-debt-ramon.png" /> <img style="width:340px; border: 1px solid gray" src="/img/posts/2013-07-21-managed-technical-debt/technical-debt-pascal.png" /></p> <p>Be sure to let me know your experience with if you’ve tried it as well!</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="https://verraes.net/2011/04/beautiful-code/">Beautiful Code</a></li> <li><a href="/2014/07/managed-technical-debt-revisited/">Managed Technical Debt - Revisited</a></li> <li><a href="/2016/04/repair-replace-heuristic-for-legacy-software/">The Repair/Replace Heuristic for Legacy Software</a></li> </ul> Unbreakable Domain Models 2013-06-09T00:00:00+00:00 https://verraes.net/2013/06/unbreakable-domain-models <h3 id="slides">Slides</h3> <script async="" class="speakerdeck-embed" data-id="838642907d6c0131238f2adccd741a3a" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script> <p><a href="https://speakerdeck.com/mathiasverraes/unbreakable-domain-models-phpuk-2014-london#">See the slides on Speakerdeck</a></p> <h2 id="video">Video</h2> <iframe width="560" height="315" src="//www.youtube.com/embed/ZJ63ltuwMaE?list=PL_aPVo2HeGF-7o9SPO5arFrAaU8bcIjba" frameborder="0" allowfullscreen=""></iframe> <p><a href="http://www.youtube.com/watch?v=ZJ63ltuwMaE&amp;feature=share&amp;list=PL_aPVo2HeGF-7o9SPO5arFrAaU8bcIjba&amp;index=8">Watch the video on Youtube</a></p> <h3 id="abstract">Abstract</h3> <p>Data Mappers (like Doctrine2) help us a lot to persist data. Yet many projects are still struggling with tough questions:</p> <ul> <li>Where to put business logic?</li> <li>How to protect our code from abuse?</li> <li>Where to put queries, and how test them?</li> </ul> <p>Let’s look beyond the old Gang of Four design patterns, and take some clues from tactical Domain Driven Design. At the heart of our models, we can use Value Objects and Entities, with tightly defined consistency boundaries. Repositories abstract away the persistence. Encapsulated Operations helps us to protect invariants. And if we need to manage a lot of complexity, the Specification pattern helps us express business rules in the language of the business.</p> <p>These patterns help us evolve from structural data models, to rich behavioral models. They capture not just state and relationships, but true meaning.</p> <p>The presentation is a fast paced introduction to some patterns and ideas that will make your Domain Model expressive, unbreakable, and beautiful.</p> <h2 id="learn-more">Learn More</h2> <p>Check out <a href="http://elephantintheroom.io/blog/2013/10/episode-2-heart-and-soul-of-oop/">Elephant in the Room #002: The Heart and Soul of OOP</a> for more info about Value Objects.</p> Decoupling (Symfony2) Forms from Entities 2013-04-20T00:00:00+00:00 https://verraes.net/2013/04/decoupling-symfony2-forms-from-entities <p>I usually tell people to stay away from the Symfony2 Form component, or forms libraries in general. The problem is that all these libraries are designed in a very CRUD mindset. You’ve done all the work to keep your models, views, and controllers nicely separated, and then this component comes along and violates every boundary.</p> <p>The good news is that there is a solution, that allows you to cleanly separate your model from UI.</p> <h3 id="another-example-of-encapsulating-operations">Another example of Encapsulating Operations</h3> <p>In my post about the <a href="https://verraes.net/2013/04/crud-is-an-anti-pattern/">CRUD anti-pattern</a>, I showed how to get rid of setters, using the Encapsulate Operations pattern. Let’s now take the example of an Employee in an HR system.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$employee</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Employee</span><span class="p">;</span> <span class="nv">$employee</span><span class="o">-&gt;</span><span class="nf">setName</span><span class="p">(</span><span class="s1">'Sarah Jones'</span><span class="p">);</span> <span class="nv">$employee</span><span class="o">-&gt;</span><span class="nf">setPosition</span><span class="p">(</span><span class="s1">'Developer'</span><span class="p">);</span> <span class="nv">$employee</span><span class="o">-&gt;</span><span class="nf">setSalaryScale</span><span class="p">(</span><span class="mi">3</span><span class="p">);</span> <span class="c1">// a few years later</span> <span class="nv">$employee</span><span class="o">-&gt;</span><span class="nf">setPosition</span><span class="p">(</span><span class="s1">'Lead Developer'</span><span class="p">);</span> <span class="nv">$employee</span><span class="o">-&gt;</span><span class="nf">setSalaryScale</span><span class="p">(</span><span class="mi">4</span><span class="p">);</span> </code></pre></div></div> <p>Imagine we have a good talk with the client’s HR director, and we’ve learned two important things about the domain:</p> <ul> <li>Once an employee is entered into the system, the name never changes.</li> <li>Promotions and Salary increases always go hand in hand.</li> </ul> <p>We want to reflect these new insights in our domain model. First, we get rid of the <code class="language-plaintext highlighter-rouge">setName()</code> method, and make sure the name can never be changed, by passing it in the constructor:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$employee</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Employee</span><span class="p">(</span><span class="s1">'Sarah Jones'</span><span class="p">);</span> </code></pre></div></div> <p>Next we make sure that an employee always has a position and a salary scale, from the moment they are registered in the system.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$employee</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Employee</span><span class="p">(</span><span class="s1">'Sarah Jones'</span><span class="p">,</span> <span class="s1">'Lead Developer'</span><span class="p">,</span> <span class="nv">$salaryScale</span><span class="p">);</span> </code></pre></div></div> <p>We get rid of the other setters, and add a method that communicates that a promotion is always accompanied by a salary change:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$employee</span><span class="o">-&gt;</span><span class="nf">promote</span><span class="p">(</span><span class="nv">$newPosition</span><span class="p">,</span> <span class="nv">$newSalaryScale</span><span class="p">);</span> </code></pre></div></div> <p>Finally, we realize that in the real world, employees are not ‘instantiated’, they are hired. We make the constructor private, and add a static method to make that explicit. The resulting code looks something like this:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">Employee</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$name</span><span class="p">;</span> <span class="k">private</span> <span class="nv">$position</span><span class="p">;</span> <span class="k">private</span> <span class="nv">$salaryScale</span><span class="p">;</span> <span class="k">private</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$name</span><span class="p">,</span> <span class="nv">$position</span><span class="p">,</span> <span class="nv">$salaryScale</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">name</span> <span class="o">=</span> <span class="nv">$name</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">position</span> <span class="o">=</span> <span class="nv">$position</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">salaryScale</span> <span class="o">=</span> <span class="nv">$salaryScale</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">hire</span><span class="p">(</span><span class="nv">$name</span><span class="p">,</span> <span class="nv">$forPosition</span><span class="p">,</span> <span class="nv">$withSalaryScale</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="k">new</span> <span class="nc">self</span><span class="p">(</span><span class="nv">$name</span><span class="p">,</span> <span class="nv">$forPosition</span><span class="p">,</span> <span class="nv">$withSalaryScale</span><span class="p">);</span> <span class="p">}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">promote</span><span class="p">(</span><span class="nv">$toNewPosition</span><span class="p">,</span> <span class="nv">$withNewSalaryScale</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">position</span> <span class="o">=</span> <span class="nv">$toNewPosition</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">salaryScale</span> <span class="o">=</span> <span class="nv">$withNewSalaryScale</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="nv">$employee</span> <span class="o">=</span> <span class="nc">Employee</span><span class="o">::</span><span class="nf">hire</span><span class="p">(</span><span class="s1">'Sarah Jones'</span><span class="p">,</span> <span class="s1">'Developer'</span><span class="p">,</span> <span class="mi">3</span><span class="p">);</span> <span class="nv">$employee</span><span class="o">-&gt;</span><span class="nf">promote</span><span class="p">(</span><span class="s1">'Lead Developer'</span><span class="p">,</span> <span class="mi">4</span><span class="p">);</span> </code></pre></div></div> <h3 id="towards-a-task-based-ui">Towards a task based UI</h3> <p>Now that we have our shiny model, we want to add a create and an edit form for the employee. We try using the Symfony2 Form component, but very soon we get stuck. The component expects there to be a bunch of setters, and it doesn’t know how to deal with constructor arguments, let alone private constructors and static factory methods.</p> <p>We don’t want to reintroduce the setters, and screw up our domain model for the sake of the UI. But we don’t want to make forms by hand. How do we get out of this situation?</p> <p>Let’s think again about what the HR Director told us, about how they work. The department <strong>hires</strong> and employee, and later <strong>promotes</strong> them. At no point he mentioned anything about <strong>creating</strong> and <strong>editing</strong> employees. Our own old-fashioned CRUD-thinking added those words. So why are we still considering a “create” and an “edit” form? We really should be talking about a “hire” form and and a “promote” form. This is a central idea in Task Based UI’s.</p> <p>So we need a way to translate the result of our <code class="language-plaintext highlighter-rouge">HireEmployeeForm</code> and <code class="language-plaintext highlighter-rouge">PromoteEmployeeForm</code>, into our <code class="language-plaintext highlighter-rouge">Employee::hire()</code> and <code class="language-plaintext highlighter-rouge">$employee-&gt;promote()</code> methods. This is starting to sound a lot like the Gang of Four Command pattern. As you may recall, a Command is an object that represents all the information needed to call a method. To represent the <code class="language-plaintext highlighter-rouge">Employee::hire()</code> method, we use a <code class="language-plaintext highlighter-rouge">HireEmployeeCommand</code>:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">HireEmployeeCommand</span> <span class="p">{</span> <span class="k">public</span> <span class="nv">$name</span><span class="p">;</span> <span class="k">public</span> <span class="nv">$forPosition</span><span class="p">;</span> <span class="k">public</span> <span class="nv">$withSalaryScale</span><span class="p">;</span> <span class="p">}</span> </code></pre></div></div> <p>You can use public properties, or privates with getters and setters, that’s really a matter of taste here. Both are ok, because the Command is just a very simple DTO object, with no behaviours. (I personally prefer getters and setters.)</p> <p>Let’s add our <code class="language-plaintext highlighter-rouge">PromoteEmployeeCommand</code> as well:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">PromoteEmployeeCommand</span> <span class="p">{</span> <span class="k">public</span> <span class="nv">$employeeId</span><span class="p">;</span> <span class="k">public</span> <span class="nv">$toNewPosition</span><span class="p">;</span> <span class="k">public</span> <span class="nv">$withNewSalaryScale</span><span class="p">;</span> <span class="p">}</span> </code></pre></div></div> <p>You’ll notice that the Command nicely matches with the form fields we need.</p> <p>The next step is self-evident: Instead of making the Form component work off our Employee Entity, it now uses our Commands. The Form and the Domain Model have no knowledge of each other, apart from the Commands. The Commands function as a clear, explicit API that outsiders use to pass instructions to the Domain Model. You can use these Commands for your Symfony2 Form, but they can just as well be used for your REST API or other clients.</p> <p>That leaves us the matter of getting the data from the Command into the Employee instance. We could do this in our Controller:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$employee</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">employeeRepository</span><span class="o">-&gt;</span><span class="nf">find</span><span class="p">(</span> <span class="nv">$promoteEmployeeCommand</span><span class="o">-&gt;</span><span class="n">employeeId</span> <span class="p">);</span> <span class="nv">$employee</span><span class="o">-&gt;</span><span class="nf">promote</span><span class="p">(</span> <span class="nv">$promoteEmployeeCommand</span><span class="o">-&gt;</span><span class="n">toNewPosition</span><span class="p">,</span> <span class="nv">$promoteEmployeeCommand</span><span class="o">-&gt;</span><span class="n">withNewSalaryScale</span> <span class="p">);</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">entityManager</span><span class="o">-&gt;</span><span class="nb">flush</span><span class="p">();</span> </code></pre></div></div> <p>A better solution is to move that particular code into a CommandHandler. Maybe something for a later blog post.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2015/02/form-command-model-validation/">Form, Command, and Model Validation</a></li> <li><a href="https://verraes.net/2014/06/when-to-use-static-methods-in-php/">When to Use Static Methods</a></li> </ul> <h2 id="comments">Comments</h2> <h3 id="cryptocompress----20130420"><a href="https://twitter.com/CryptoCompress">CryptoCompress</a> - 2013/04/20</h3> <p>Very nice Blog! Thank You!</p> <p>off-topic:</p> <h4 id="static-function-hire-1">“static function hire()” (1):</h4> <p>Static methods considered harmful. I use static-method-in-object instead in-factory for stateful objects too. Is this a common approach? Maybe you have more information about it?</p> <h4 id="static-function-hire-2">“static function hire()” (2):</h4> <p>A company hire “instantiated” employees. Bonus points for <code class="language-plaintext highlighter-rouge">$company-&gt;hire($employee)</code> :D</p> <h4 id="last-code-snippet">last code snippet</h4> <p>There is no obvious connection between repository and entity manager. <code class="language-plaintext highlighter-rouge">$this-&gt;employeeRepository-&gt;persist($employee);</code> or <code class="language-plaintext highlighter-rouge">$this-&gt;entityManager-&gt;employeeRepository-&gt;find();</code></p> <h3 id="bernhard-schussek---20130421"><a href="https://twitter.com/webmozart">Bernhard Schussek</a> - 2013/04/21</h3> <p>FYI, the Form component is able to deal with constructors by using the (empty_data)[http://symfony.com/doc/current/cookbook/form/use_empty_data.html] option.</p> <p>Of course, if you don’t want to use any getters/setters, the automatic data mapping is pointless. It is there only as a RAD helper to save some lines of code (especially when the entities also define validation constraints). But if you don’t want that, your approach is a very good alternative. Another alternative would be to use an array-backed form.</p> CRUD is an antipattern 2013-04-19T00:00:00+00:00 https://verraes.net/2013/04/crud-is-an-anti-pattern <p>CRUD is, as Greg Young calls it, our industry’s <a href="http://herdingcode.com/?p=189">Grand Failure</a>. We have deluded our users, and ourselves, that applications are nothing more than a thin layer around our database. We present the user with grids and edit forms, and ask them to fill in fields, that we map more or less directly to our relational tables. We use terms like “data model”.</p> <p>Real processes and businesses are not CRUD though. Real people don’t say “I’ve set the paid amount of the order to ‘120’, the paid currency to ‘EUR’, and status to ‘paid’”. They say “I’ve paid €120 for the order”.</p> <p>A good domain model is not a data model. A domain model describes behaviour, and the data is an artefact of that. Let’s translate our user story to code.</p> <h3 id="crud-thinking">CRUD thinking:</h3> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$order</span><span class="o">-&gt;</span><span class="nf">setStatus</span><span class="p">(</span><span class="s1">'paid'</span><span class="p">);</span> <span class="nv">$order</span><span class="o">-&gt;</span><span class="nf">setPaidAmount</span><span class="p">(</span><span class="mi">120</span><span class="p">);</span> <span class="nv">$order</span><span class="o">-&gt;</span><span class="nf">setPaidCurrency</span><span class="p">(</span><span class="s1">'EUR'</span><span class="p">);</span> <span class="nv">$order</span><span class="o">-&gt;</span><span class="nf">setCustomer</span><span class="p">(</span><span class="nv">$customer</span><span class="p">);</span> </code></pre></div></div> <h3 id="step-1-replace-a-setter-with-an-expressive-method">Step 1: Replace a setter with an expressive method</h3> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$order</span><span class="o">-&gt;</span><span class="nf">pay</span><span class="p">();</span> </code></pre></div></div> <p>Note that internally, the Order object still stores the <code class="language-plaintext highlighter-rouge">status</code> field. That’s perfectly fine.</p> <h3 id="step-2-replace-values-that-belong-together-with-a-value-object">Step 2: Replace values that belong together with a Value Object:</h3> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$money</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="p">(</span><span class="mi">120</span><span class="p">,</span> <span class="k">new</span> <span class="nc">Currency</span><span class="p">(</span><span class="s1">'EUR'</span><span class="p">));</span> <span class="nv">$order</span><span class="o">-&gt;</span><span class="nf">setPaidMoney</span><span class="p">(</span><span class="nv">$money</span><span class="p">);</span> </code></pre></div></div> <p>I’ve written about Value Objects and Money elsewhere on this blog already.</p> <h3 id="step-3-encapsulate-operation">Step 3: Encapsulate Operation</h3> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$order</span><span class="o">-&gt;</span><span class="nf">pay</span><span class="p">(</span><span class="nv">$customer</span><span class="p">,</span> <span class="nv">$money</span><span class="p">);</span> </code></pre></div></div> <p>It should be clear that the code in step 3 is a lot closer to demonstrating the behaviour and the actual purpose of our application. We haven’t changed anything in the internal representation of our Order object. It still has fields like <code class="language-plaintext highlighter-rouge">paidAmount</code>, <code class="language-plaintext highlighter-rouge">paidCurrency</code>, <code class="language-plaintext highlighter-rouge">customerId</code>, and <code class="language-plaintext highlighter-rouge">status</code>, that are mapped to the database. Order no longer exposes it’s internals directly, but instead exposes a public api that is intention-revealing. It matches how users see our application: as a set of behaviours, instead of a bunch of relational tables.</p> <h3 id="bonus-points">Bonus points</h3> <p>Optionally, we can make it even more expressive by stating that it’s the Customer who’s paying:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$customer</span><span class="o">-&gt;</span><span class="nf">pay</span><span class="p">(</span><span class="nv">$order</span><span class="p">,</span> <span class="nv">$money</span><span class="p">);</span> </code></pre></div></div> <p>The downside of this, is that we are coupling the Customer to the Order. This may or may not be desirable, depending on context. In any case, it’s always worth considering.</p> <h2 id="read-more">Read more</h2> <ul> <li><a href="/2014/09/decoupling-model-framework/">Decoupling the Model from the Framework</a></li> <li><a href="/2013/04/decoupling-symfony2-forms-from-entities/">Decoupling Forms from Entities</a></li> </ul> <h2 id="comments">Comments</h2> <h3 id="daniel-lo-nigro---20130420"><a href="http://dan.cx/">Daniel Lo Nigro</a> - 2013/04/20</h3> <p>Hmm… I don’t know if CRUD is the right term here. Your <code class="language-plaintext highlighter-rouge">pay</code> method is only setting values and saving the entity, and thus is still a CRUD operation. I think you’re complaining about using “raw” getters and setters as opposed to methods that encapsulate the business or domain logic. There’s definite advantages to containing domain logic, but domain logic and CRUD aren’t mutually exclusive.</p> <h3 id="mathias-verraes---20130420"><a href="http://twitter.com/mathiasverraes">Mathias Verraes</a> - 2013/04/20</h3> <p>Care to elaborate why you believe <code class="language-plaintext highlighter-rouge">$order-&gt;pay(...)</code> is still CRUD?</p> <p>CRUD is the idea that everything in a model can be described using only four verbs: <code class="language-plaintext highlighter-rouge">create</code>, <code class="language-plaintext highlighter-rouge">read</code>, <code class="language-plaintext highlighter-rouge">update</code>, <code class="language-plaintext highlighter-rouge">delete</code>. In this post, and in <a href="https://verraes.net/2013/04/decoupling-symfony2-forms-from-entities/">the next one</a> I show some examples where I introduce new verbs: <code class="language-plaintext highlighter-rouge">pay</code>, <code class="language-plaintext highlighter-rouge">hire</code>, <code class="language-plaintext highlighter-rouge">promote</code>. Having domain logic in CRUD systems is possible, but it’s harder to express. My point here is mostly that we should build applications that express user intent using a rich language with verbs that users actually use. And that is in fact mutually exclusive with CRUD.</p> <h3 id="daniel-lo-nigro---20130420-1"><a href="http://dan.cx/">Daniel Lo Nigro</a> - 2013/04/20</h3> <p>I think that <code class="language-plaintext highlighter-rouge">$order-&gt;pay(...)</code> might still be CRUD because in the end, it’s just doing a single database update (like <code class="language-plaintext highlighter-rouge">UPDATE order SET status = 'paid', amount = 120 WHERE order_id = 123</code>) and no other logic, which is a CRUD operation. It’s a thin domain model abstraction of the CRUD operation, but it’s just a regular update statement with a different name, and hence it’s still a CRUD operation. I wouldn’t consider it CRUD if it had more logic. I could be totally wrong though!</p> <p>And in any case, getters and setters aren’t CRUD, and calling <code class="language-plaintext highlighter-rouge">$order-&gt;setStatus</code> doesn’t automatically imply CRUD.</p> <h3 id="mathias-verraes---20130420-1"><a href="http://twitter.com/mathiasverraes">Mathias Verraes</a> - 2013/04/20</h3> <p>You’re right that setters don’t imply CRUD, but they are mostly seen together. The fact that the backend could use an UPDATE SQL has nothing to do with the problem at hand though. Databases use primitive verbs, because they have no knowledge of the meaning of the data.</p> <p>Whether or not we have additional business logic in our methods, is a different matter. My example is oversimplified, and indeed doesn’t have much logic in the <code class="language-plaintext highlighter-rouge">pay()</code> method. It would however be a perfect place to put some invariants: What if the paid amount is incorrect? What if the currency is wrong? What if the order was already paid? This is all logic that is harder to find a place for in a CRUD system.</p> <h3 id="tobias-wooldridge---20130420"><a href="http://tobias.wooldridge.id.au/">Tobias Wooldridge</a> - 2013/04/20</h3> <p>One of the most useful things about design patterns and anti-patterns is that, outside of highlighting problems and solutions, they have a widely understood, unique name.</p> <p>In this case, my confusion came about because CRUD is an inappropriate term for what you’re describing. The anti-pattern definition unfortunately takes the word “CRUD” from the database implementation level of design and applies it to the domain level.</p> <p>Compared to names such as ‘God object’ and ‘Spaghetti code’, ‘CRUD’ is ambiguous. While I’m not a fan of <a href="http://en.wikipedia.org/wiki/Object_orgy">other names given to this anti-pattern</a>, I don’t feel CRUD is an improvement, as it fails to clearly identify the problem at hand.</p> <h3 id="paul-mitchum---20130420"><a href="http://github.com/paul-m">Paul Mitchum</a> - 2013/04/20</h3> <p>CRUD is a pattern that encapsulates certain behavior, related to persisting and managing data outside the script.</p> <p>If you need that pattern, then use it. If you don’t, then don’t. :-)</p> <p>Your customer doesn’t care whether you implement <code class="language-plaintext highlighter-rouge">$order-&gt;setStuff()-&gt;persist();</code> or <code class="language-plaintext highlighter-rouge">$order-&gt;pay(new SemtanticallyCleanObjectName($currency, $amt));</code> Your work is to abstract these details away for your customer, in exactly the same way that <code class="language-plaintext highlighter-rouge">setStuff()-&gt;persist();</code> abstracts CRUD away for you.</p> <p>The real question is what your customer needs, and how you can deliver it to them in an efficient and maintainable way. CRUD isn’t an anti-pattern working against this goal. Viewing it as the only required layer of abstraction is, however, most likely a design error.</p> <h3 id="rasmus-schultz---2013-04-22"><a href="http://twitter.com/mindplaydk">Rasmus Schultz</a> - 2013-04-22</h3> <p>Hi Mathias,</p> <p>Two things :-)</p> <p>First, your post explains how to “replace” the code that uses the accessors directly to perform the change - but “replace” is not the right term here, that’s not what’s happening. You’re not “replacing” anything - you’re introducing better encapsulation for an identified type of transaction, and your code is now self-documenting and clearly reflects the fact that the code that uses the accessors is in fact a transcation. Your Order::pay() implementation will still contain the “replaced” code - and possibly other aspects of the transaction, such as logging, statistics, e-mail notifications, etc.</p> <p>The code that uses the accessors of course should not go directly in a Controller - and I think that was your point? Although you never used the word Controller. But it has got to go somewhere, right? I think we agree on that much, and your post definitely points out something very important. So far so good :-)</p> <p>Now secondly, you give “bonus points” for implementing the transaction method in the Customer entity instead of the Order entity. But this particular transaction depends on two things equally: an Order and a Transaction - while implementing the method in the Customer entity makes the controller-code read out more like english (“Customer pay order”) there is no logical reason why one is better than the other, and the semantics remain essentially the same.</p> <p>You can find countless examples of this in the real world - for any transaction that involves more than one entity, you will have to make this decision. It gets particularly tricky when you have three or more entities involved in a transaction - which party is going to be responsible? And even worse, when one of those entities are optional in the same type of transaction. Of course, these are all questions you can answer, but your answers and thinking might not be the same as the next guy who has to work with your code.</p> <p>That’s why I prefer to avoid those questions and use a service-oriented approach instead. Introduce a static PaymentService class to act as a mediator for that transaction. Or introduce an encapsulated PaymentTransaction object, since encapsulation was what you were questing for in the first place. This eliminates all of the above questions - the responsible party is now the transaction service or object, not an arbitrary entity. If an entity is optional in that transaction, make it obvious by allowing a required null-argument for that entity in the transaction service or object.</p> <p>Adhering to this pattern has other advantages too - in particular, it enables composition: more than one type of service or transaction probably sends e-mail notifications or performs logging, so you can encapsulate those requirements in a base-class, and so on. Which furthermore helps with testing, since you can now inject a mock e-mail client or logger during tests.</p> <p>But perhaps most importantly, it helps with perception - because your transactions are no longer scattered across entities, but encapsulated in services or transaction objects, it’s easier for somebody else to gain an overview of all the possible transactions in an application. It’s also easier to look at an entity and expect to find only methods that operate on the entity itself - rather than methods that depend on other entities. It scales better in terms of complexity, because each entity and service will have a fixed scope - rather than growing each class to meet new requirements, you introduce more classes, each with an isolated responsibility and fixed scope.</p> <p>In the case of PHP specifically, it also marginally helps with performance, because you’re no longer loading transaction code that doesn’t get executed.</p> <p>Maybe this is just way beyond the scope of your post :-)</p> <p>And under any circumstances, having done this little write-up, I will probably end up posting that on my own blog. Feel free to post as comments on your blog though, if you wish. Though it’s almost longer than your blog-post at this point ;-)</p> <p>I know, I do go on. Sorry about that.</p> Casting Value Objects to strings 2013-02-16T00:00:00+00:00 https://verraes.net/2013/02/2013-02-16-casting-value_objects <p>I’m a big fan of Value Objects, as they are very helpful in encapsulating behavior, and communicating intent. In fact, as a friend remarked, Value Objects are the heart and soul of Object Oriented Programming. Discussing the uses of Value Objects is not what I want to talk about here though.</p> <p>Let’s set up two simple examples first:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">TwitterHandle</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$handle</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$handle</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Put some validation here, like a regex check</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">handle</span> <span class="o">=</span> <span class="nv">$handle</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="kd">class</span> <span class="nc">DateRange</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$start</span><span class="p">,</span> <span class="nv">$stop</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">DateTime</span> <span class="nv">$start</span><span class="p">,</span> <span class="kt">DateTime</span> <span class="nv">$end</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// Put some validation here, like checking if $end &gt; $start</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">start</span> <span class="o">=</span> <span class="nv">$start</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">stop</span> <span class="o">=</span> <span class="nv">$end</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <h3 id="simple-string-casting">Simple string casting</h3> <p>Value Objects are one of the few places where using <a href="http://www.php.net/manual/en/language.oop5.magic.php#object.tostring">PHP’s magic __toString()</a> makes perfect sense. For single-value Value Objects like the <code class="language-plaintext highlighter-rouge">TwitterHandle</code>, it’s obvious what should be in there. The _toString() method simply returns the string representation.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">TwitterHandleTest</span> <span class="kd">extends</span> <span class="nc">PHPUnit_Framework_TestCase</span> <span class="p">{</span> <span class="cd">/** @test */</span> <span class="k">public</span> <span class="k">function</span> <span class="n">ItShouldCastToString</span><span class="p">()</span> <span class="p">{</span> <span class="nv">$twitterHandle</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">TwitterHandle</span><span class="p">(</span><span class="s1">'@mathiasverraes'</span><span class="p">);</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">assertEquals</span><span class="p">(</span> <span class="s1">'@mathiasverraes'</span><span class="p">,</span> <span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$twitterHandle</span><span class="p">,</span> <span class="s2">"Casting a TwitterHandle to a string "</span> <span class="mf">.</span><span class="s2">"should return the string version of the handle"</span> <span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Implemented inside TwitterHandle, it looks like this:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">TwitterHandle</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$handle</span><span class="p">;</span> <span class="c1">// constructor ....</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__toString</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">handle</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>That’s quite self-evident, but what about other cases?</p> <h3 id="rules-for-casting-value-objects-to-string">Rules for casting Value Objects to string</h3> <p>I had a couple of basic rules, that I have been using for a while for Value Objects. They were always implicit, but talking about them in a pair programming session made them explicit.</p> <h3 id="1-dont-use-_tostring-for-presentation">1. Don’t use _toString() for presentation.</h3> <p>It’s tempting to use a __toString() method to render the Value Object in a View. More often than not, you would need to put presentation logic in your Value Object, and that’s a big no-no. It’s better to perform presentation rendering in a separate class or function, like a filter for your templating engine. It allows you to fine-tune the rendering for your use case.</p> <div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nt">&lt;span&gt;</span> Find me on Twitter: <span class="cp">&lt;?php renderAsTwitterLink($user-&gt;getTwitterAccount()) ?&gt;</span> <span class="nt">&lt;/span&gt;</span> <span class="c">&lt;!-- renders to: --&gt;</span> <span class="nt">&lt;span&gt;</span> Find me on Twitter: <span class="nt">&lt;a</span> <span class="na">href=</span><span class="s">"https://twitter.com/mathiasverraes"</span> <span class="na">title=</span><span class="s">"Visit Twitter"</span><span class="nt">&gt;</span> @mathiasverraes <span class="nt">&lt;/a&gt;</span> <span class="nt">&lt;/span&gt;</span> </code></pre></div></div> <p>Imagine we would have put the HTML rendering inside the TwitterHandle class; that would have been a sackable offense in my book! You’ll notice that this is especially important with Value Objects like dates and monetaries, where the rendering depends on the language.</p> <h3 id="2-always-return-a-complete-representation-of-the-value-object">2. Always return a complete representation of the Value Object.</h3> <p>Another temptation is to use string casting to get the numeric part of a measurement.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="cp">&lt;?php</span> <span class="nv">$aDistance</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Distance</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'meters'</span><span class="p">);</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">assertEquals</span><span class="p">(</span><span class="s1">'5'</span><span class="p">,</span> <span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$aDistance</span><span class="p">);</span> <span class="c1">// bad, don't do this</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">'5'</code> is not a valid representation of a distance. Is it meters, centimeters, or inches? That information is now lost. If you start doing calculations with that <code class="language-plaintext highlighter-rouge">'5'</code>, you may get nasty bugs. <a href="http://en.wikipedia.org/wiki/Mars_Climate_Orbiter">The kind of bugs that crash spacecrafts</a>. So make sure that the string represents all of the Value Object:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="cp">&lt;?php</span> <span class="nv">$aDistance</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Distance</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'meters'</span><span class="p">);</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">assertEquals</span><span class="p">(</span><span class="s1">'5 meters'</span><span class="p">,</span> <span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$aDistance</span><span class="p">);</span> <span class="c1">// much better</span> </code></pre></div></div> <h3 id="3-return-a-parsable-string">3. Return a parsable string</h3> <p>The most useful string representation you can come up with, is one that can be parsed back into the Value Object. For the TwitterHandle example, that’s again very easy, so let’s use DateRange as an example:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="cp">&lt;?php</span> <span class="nv">$aDateRange</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">DateRange</span><span class="p">(</span> <span class="k">new</span> <span class="nc">DateTime</span><span class="p">(</span><span class="s1">'2013-01-01'</span><span class="p">),</span> <span class="k">new</span> <span class="nc">DateTime</span><span class="p">(</span><span class="s1">'2013-02-07'</span><span class="p">)</span> <span class="p">);</span> <span class="nv">$stringVersion</span> <span class="o">=</span> <span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$aDateRange</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">assertEquals</span><span class="p">(</span><span class="s1">'2013-01-01 - 2013-02-07'</span><span class="p">,</span> <span class="nv">$stringVersion</span><span class="p">);</span> <span class="nv">$parsedDateRange</span> <span class="o">=</span> <span class="nc">DateRange</span><span class="o">::</span><span class="nf">parse</span><span class="p">(</span><span class="nv">$stringVersion</span><span class="p">);</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">assertEquals</span><span class="p">(</span><span class="nv">$aDateRange</span><span class="p">,</span> <span class="nv">$parsedDateRange</span><span class="p">);</span> </code></pre></div></div> <p>So as you can see, <code class="language-plaintext highlighter-rouge">'2013-01-01 - 2013-02-07'</code> is a string representation of a DateRange, that can be parsed back into a DateRange object. In pseudocode:</p> <p><code class="language-plaintext highlighter-rouge">DateRange == parse(string(DateRange))</code></p> <p>It works just as well for our other examples:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="cp">&lt;?php</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">assertEquals</span><span class="p">(</span> <span class="k">new</span> <span class="nc">Distance</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'meters'</span><span class="p">),</span> <span class="nc">Distance</span><span class="o">::</span><span class="nf">parse</span><span class="p">(</span> <span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="k">new</span> <span class="nc">Distance</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'meters'</span><span class="p">)</span> <span class="p">)</span> <span class="p">);</span> <span class="c1">// leaving the implementation as an exercise for the reader :-)</span> </code></pre></div></div> <p>This will turn out to be very handy, especially when integrating with other systems, like a REST api or a database. But that’s something we’ll cover in a future blog post.</p> <p>Related posts:</p> <ul> <li><a href="/2011/04/fowler-money-pattern-in-php/">Representing Money in PHP, Fowler-style</a></li> <li>Check out <a href="http://elephantintheroom.io/blog/2013/10/episode-2-heart-and-soul-of-oop/">Elephant in the Room #002: The Heart and Soul of OOP</a> for more info about Value Objects.</li> </ul> Using Contextual Code Formatting for Readability 2013-01-05T00:00:00+00:00 https://verraes.net/2013/01/2013-01-05-contextual-code-formatting <p>I used to care a great deal about having a coding standard. I often took the initiative to write a document for the team that described the do’s and don’ts, and I was often the one setting up automated formatting sniffers on every commit.</p> <p>I’ve come to see that we have been ignoring an opportunity to use formatting to our advantage. By arguing and voting over standards, and building automated tools to do the formatting for us, and refusing commits that failed the formatting validators, we lost track of why we needed a standard in the first place. It’s not to fulfil the desire of the pedants in our teams, or for the esthetic of neatly aligned spaces.</p> <p>The goal should be to write easily readable and maintainable code. What makes code more readable depends on context.</p> <h3 id="an-example">An example</h3> <p>Can you spot the difference between these to statements?</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$countries</span> <span class="o">=</span> <span class="p">[</span><span class="s2">"AD"</span><span class="o">=&gt;</span><span class="s2">"Andorra"</span><span class="p">,</span><span class="s2">"AE"</span><span class="o">=&gt;</span><span class="s2">"UnitedArabEmirates"</span><span class="p">,</span><span class="s2">"AF"</span><span class="o">=&gt;</span><span class="s2">"Afghanistan"</span><span class="p">,</span><span class="s2">"AG"</span><span class="o">=&gt;</span><span class="s2">"AntiguaAndBarbuda"</span><span class="p">,</span><span class="s2">"AI"</span><span class="o">=&gt;</span><span class="s2">"Anguilla"</span><span class="p">,</span><span class="s2">"AL"</span><span class="o">=&gt;</span><span class="s2">"Albania"</span><span class="p">,</span><span class="s2">"AM"</span><span class="o">=&gt;</span><span class="s2">"Armenia"</span><span class="p">,</span><span class="s2">"AN"</span><span class="o">=&gt;</span><span class="s2">"NetherlandsAntilles"</span><span class="p">,</span><span class="s2">"AO"</span><span class="o">=&gt;</span><span class="s2">"Angola"</span><span class="p">,</span><span class="s2">"AQ"</span><span class="o">=&gt;</span><span class="s2">"Antarctica"</span><span class="p">,</span><span class="s2">"AR"</span><span class="o">=&gt;</span><span class="s2">"Argentina"</span><span class="p">,</span><span class="s2">"AS"</span><span class="o">=&gt;</span><span class="s2">"AmericanSamoa"</span><span class="p">,</span><span class="s2">"AT"</span><span class="o">=&gt;</span><span class="s2">"Austria"</span><span class="p">,</span><span class="s2">"AU"</span><span class="o">=&gt;</span><span class="s2">"Australia"</span><span class="p">,</span><span class="s2">"AW"</span><span class="o">=&gt;</span><span class="s2">"Aruba"</span><span class="p">,</span><span class="n">etc</span> <span class="nv">$options</span> <span class="o">=</span> <span class="p">[</span> <span class="s2">"entryPoint"</span> <span class="o">=&gt;</span> <span class="s2">"http://example.com"</span><span class="p">,</span> <span class="s2">"alwaysUsePost"</span> <span class="o">=&gt;</span> <span class="kc">false</span><span class="p">,</span> <span class="s2">"followRedirects"</span> <span class="o">=&gt;</span> <span class="kc">false</span><span class="p">,</span> <span class="p">];</span> </code></pre></div></div> <p><code class="language-plaintext highlighter-rouge">$countries</code> is written very compact, no whitespace, in a single line that laughs in the face of 80 character margins. <code class="language-plaintext highlighter-rouge">$options</code>, despite having only three elements, uses ample whitespace and newlines. But why?</p> <p>The <code class="language-plaintext highlighter-rouge">$countries</code> is not very readable, because it doesn’t have to be. Chances are you’ll never need to change that list, let alone read it entirely. Just glancing at the beginning of the line tells you what you can expect the rest of the contents to be like. And if you do need to update the countries, you’ll probably copy/paste a list from somewhere else.</p> <p>By writing the line very compact, you give a visual suggestion to the reader: “The contents of this line is not important to you, don’t bother reading it.”</p> <p>The contents of the <code class="language-plaintext highlighter-rouge">$options</code> array on the other hand are important. If one day you need to investigate why redirects aren’t being followed, you will read this code and change it. You’ll be thankful you didn’t have to squint your eyes to find the relevant element in a huge unreadable compressed line of code.</p> <h3 id="drawing-attention">Drawing attention</h3> <p>That’s just a simple example of making our formatting dependend on what works for a specific situation. Make the boring implementation details, look like boring code. Compact the stuff the never changes, the generic boilerplate stuff, so that readers understand intuitively that it doesn’t matter. But make the important code stand out. Highlight it using whitespace, well-placed comments, and attractive formatting. Be creative and figure out where formatting can help you be more expressive and intentional.</p> <h2 id="comments">Comments</h2> <h3 id="tvlooy---20130105"><a href="http://ctors.net">tvlooy</a> - 2013/01/05</h3> <p>Nice post, thanks. Another advantage of the $option approach is that it is easier to read VCS diffs. For example, if you would change alwaysUsePost to true, this is very obvious in a diff. If you make a change to $countries, this is less obvious. I personally also format method calls this way, if the call is bigger than one line for example, and only if it improves readability. Like:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$myObject</span><span class="o">-&gt;</span><span class="nf">setOptions</span><span class="p">(</span><span class="k">array</span><span class="p">(</span><span class="s2">"entryPoint"</span> <span class="o">=&gt;</span> <span class="s2">"http://example.com"</span><span class="p">,</span> <span class="s2">"alwaysUsePost"</span> <span class="o">=&gt;</span> <span class="kc">false</span><span class="p">),</span> <span class="kc">false</span><span class="p">,</span> <span class="nc">LongClassName</span><span class="o">::</span><span class="no">SOME_CONSTANT</span><span class="p">);</span> <span class="nv">$myObject</span><span class="o">-&gt;</span><span class="nf">setOptions</span><span class="p">(</span> <span class="k">array</span><span class="p">(</span> <span class="s2">"entryPoint"</span> <span class="o">=&gt;</span> <span class="s2">"http://example.com"</span><span class="p">,</span> <span class="s2">"alwaysUsePost"</span> <span class="o">=&gt;</span> <span class="kc">false</span><span class="p">,</span> <span class="p">),</span> <span class="kc">false</span><span class="p">,</span> <span class="nc">LongClassName</span><span class="o">::</span><span class="no">SOME_CONSTANT</span> <span class="p">);</span> </code></pre></div></div> <p>Adding an option to the array will then also just show a clear 1 line add in VCS diffs.</p> How to read more books 2012-12-23T00:00:00+00:00 https://verraes.net/2012/12/2012-12-23-how-to-read-more-books <p><img src="/img/posts/2012-12-23/reading_improves_your_view_of_the_world.jpg" alt="Reading improves your view of the world" /></p> <p>“You can’t learn everything from books”, people say. It’s nothing but an excuse not to read books. It’s a straw man argument, because no-one has ever claimed that you can. Or, “knowledge from experience can’t be taught in a book”. Even though that’s true, there are many people out there with years of experiences that you will never have. Some of them are very good at sharing at least some of that gained knowledge through writing. Books broaden your horizon, and deepen your insight.</p> <p>With that out of the way, here are some of the tips that help me to read more books.</p> <h3 id="get-glasses">Get glasses</h3> <p>If reading makes you tired, gives you headaches, or is generally uncomfortable, you may need glasses.</p> <h3 id="stop-watching-tv">Stop watching tv</h3> <p>When people tell me they don’t have time to read books, I always ask if they have time to watch tv. You don’t have to stop entirely, but only watch things that you consciously choose to watch.</p> <ul> <li>Don’t watch live television. Most programs last either 22 or 44 minutes, but that’s 30 or 60 minutes with commercials. Watch recorded tv instead.</li> <li>Don’t zap. The ultimate time sink hole.</li> <li>Don’t watch the news. It’s always the same. The little information you can get from a 22 minute news program, can be read online in 5 minutes. <a href="http://www.guardian.co.uk/media/2013/apr/12/news-is-bad-rolf-dobelli">Giving up news will make you happier</a>.</li> </ul> <h3 id="stop-reading-newspapers">Stop reading newspapers</h3> <p>Newspapers and magazines compete by getting fatter year after year, with declining quality as the result. Everything you read will be outdated tomorrow anyway. And let’s be honest: first you read the funnies, then the headers, then some celebrity news. Do you really need a newspaper anymore?</p> <blockquote>"Never read anything that isn't worth reading." — Gerald M. Weinberg</blockquote> <h3 id="get-an-e-reader">Get an e-reader</h3> <p><a href="http://www.amazon.com/gp/product/B007HCCNJU/ref=as_li_tl?ie=UTF8&amp;camp=1789&amp;creative=390957&amp;creativeASIN=B007HCCNJU&amp;linkCode=as2&amp;tag=verraesnet-20&amp;linkId=PAXYR27D7TJUIIG3"> Or get two, or three. </a><img src="http://ir-na.amazon-adsystem.com/e/ir?t=verraesnet-20&amp;l=as2&amp;o=1&amp;a=B007HCCNJU" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /> They’re cheap, small, and light. They carry thousands of books. They’re a pleasure to read, because they don’t blast bright light at you like iPads, monitors, and TV’s. Rid yourself of the romantic idea that books must be touchable, sniffable, paper objects. They’re just dead trees collecting dust in your living room, and you’ll be really sorry when you move to a new house and have to carry them down four flights of stairs. The content is what matters.</p> <p><img style="" src="/img/posts/2012-12-23/matilda-roald-dahl.jpg" alt="Matilda" /></p> <h3 id="read-everywhere">Read everywhere</h3> <p>Kindle has apps for every device, and syncs your progress. When I’m standing in line in the supermarket, I read a couple of pages on my phone. I read in bed, in the bathroom, on lunch breaks, in transit, and when I’m early for a meeting.</p> <h3 id="make-it-a-habit">Make it a habit</h3> <p>I read when I’m too tired to read. Obviously, some of it gets lost, and I often need to reread the last couple of pages the next day. I do it because I feel that even reading only a little bit, keeps the habit alive. That’s also why I try to make sure I always have my next book lined up. Some e-book stores give you the first chapter for free, which is great way to find out if a book is worth reading.</p> <h3 id="train-your-muscle">Train your muscle</h3> <p>Like many things, the more you read, the easier it gets. Even if it’s hard in the beginning, if you read daily, you’ll find that it gets easier, that you learn new concepts faster, that your brain gets more adapted to obtaining and processing information.</p> <h3 id="keep-a-list">Keep a list</h3> <p>Keep track of what you’ve read, on sites like <a href="https://kindle.amazon.com/">Kindle Reading List</a>, or <a href="http://www.goodreads.com/">GoodReads</a>. You can share your list, follow others, turn it into a game. It’s very rewarding to watch your reading list grow, realizing that all that knowledge is now in your head as well and makes you better at what you do.</p> <p><img src="/img/posts/2012-12-23/adopt_books_small.jpg" alt="I don't read books, I adopt them" /></p> <p>(Update September 10, 2013:)</p> <h3 id="skip-to-a-better-book">Skip to a better book</h3> <p><a href="https://verraes.net/2013/08/antifragile-nassim-nicholas-taleb/">Nassim N. Taleb</a> spent years reading 30-60 hours a week:</p> <blockquote>"The minute I was bored with a book or a subject I moved to another one, instead of giving up on reading altogether (...) The trick is to be bored with a specific book, rather than with the act of reading."</blockquote> <h3 id="start-easy">Start easy</h3> <p>If you have a backlog of books you want to read, you can start with an easy or short one and build your way up. A finished book is a small victory that boosts your confidence, and prepares you for tougher tomes.</p> <h3 id="listen-to-audiobooks">Listen to audiobooks</h3> <p>I’d never given audiobooks much thought, but the last few months I’ve been driving a lot, so I wanted to spend the time well. I finished my first two audiobooks – easy and short ones to begin with. I find that I’m more relaxed and focused when driving with the gentle voice of a good narrator, than with blaring radio.</p> <p>(Update December 22, 2013:)</p> <h3 id="read-it-before-you-need-it">Read it before you need it</h3> <p>In an ideal world, you’d read (technical) books just in time, precisely before you need the information. But you can’t plan learning like that. You don’t know how much time you’ll need to read it, let alone to have it sink in. When you need information to finish a task you’re working on, a book is often not the right choice.</p> <p>A better approach is to read a book long before you might need it. Learning takes time, so giving your brain ample time to process the new ideas, will give you a much better understanding. When the time comes to use the knowledge, a quick refresher will be all you need to reactivate it in your brain.</p> <h3 id="make-room-and-make-options">Make room and make options</h3> <p><a href="http://www.noop.nl/2013/12/dont-make-goals-make-room-and-make-options.html">Jurgen Appelo</a> recommends not to set a target amount of books to read. Targets lead to sub-optimisation. And how would you know what amount is ‘just right’? Focus on removing waste (he cites watching TV and drinking in bars), and on making options (have a backlog and always bring reading material).</p> <p>(Update June 24, 2014:)</p> <h3 id="walk-n-read">Walk ‘n’ read</h3> <p>I’ve always felt walking was good for thinking and getting the creative juices flowing – back when I was a film music composer, that’s how I came up with my best melodies. For the past few weeks, I’ve been trying something new: every morning, I take the same 45 minute walk, while reading on my Kindle. I’m taking small farmers’ roads between the fields, so there’s no traffic and it’s safe. It’s good for your health, so it’s a double win. And if reading on the couch makes you drowsy, walking is the perfect remedy.</p> <h3 id="pomodoro">Pomodoro</h3> <p><a href="https://twitter.com/tvlooy/status/481500378362818560">@tvlooy</a> uses the <a href="http://pomodorotechnique.com/">Pomodoro technique</a>: set an alarm, read for 25 minutes, take a 5 minute break away from the book or screen, repeat ad lib, with the same or a different book. For the full Pomodoro experience, keep track of the finished (uninterrupted) reading sessions, and set goals.</p> <p>(Update June 30, 2014:)</p> <p><img style="float:right;margin-left: 10px" src="/img/posts/2012-12-23/install-software-in-brain.jpg" alt="It's called reading. It's how people install new software into their brains" /></p> <h3 id="dont-read-biographies">Don’t read biographies</h3> <p>Although I’m sure some are valuable, don’t read books written <em>about</em> interesting people. Read books written <em>by</em> interesting people.</p> <h3 id="dont-read-single-idea-books">Don’t read single-idea books</h3> <p>“This book was based on a popular article in the New York Times”? Dead give away: one idea, lots of air. “The Three Secrets to…”, “Five steps to…”: It’s pretty likely that the book can be explained in a just a few paragraphs. Sometimes, a helpful Amazon customer review will in fact summarize the whole book for you. Read that instead.</p> <p>(Update July 22, 2014:)</p> <h3 id="read-what-gives-you-delight">Read what gives you delight</h3> <p>From <a href="http://www.farnamstreetblog.com/2013/01/the-pleasures-of-reading-in-an-age-of-distraction/">The Pleasures of Reading in an Age of Distraction</a>:</p> <blockquote>"Read what gives you delight — at least most of the time — and do so without shame. And even if you are that rare sort of person who is delighted chiefly by what some people call Great Books, don’t make them your steady intellectual diet, any more than you would eat at the most elegant of restaurants every day."</blockquote> <p>(Update Feb 10, 2015:)</p> <h3 id="riffle-the-pages">Riffle the pages</h3> <p>I don’t know how much Gerald M. Weinberg <em>reads</em>, but he has <em>written</em> over 50 books. From “Weinberg on Writing”, I’m quoting a tip on reading nonfiction:</p> <blockquote>"My own way (...) is to riffle the pages, scanning from back to front for something interesting that catches my eye, usually a picture or diagram of some sort. I start reading from there and continue until I lose interest. The I riffle again and repeat the process." </blockquote> <h3 id="stop-means-stop">Stop means stop</h3> <p>Also from “Weinberg on Writing”, similar to Taleb’s advice:</p> <blockquote>"If you decide that it's not worth finishing, do you stop, or do you drag yourself through it? If you stop, do you *really* stop, or do you leave the book partly finished, as a nagging chore in the back of your mind?"</blockquote> <p>Just remember that finishing a bad book is a waste of time and motivation. On the other hand, a great book is a window to a world you could never have found on your own. You still need to get out the door and discover that world. But that, dear reader, is out of reach for any book, and for this blog post.</p> <p>(Update March 26, 2016:)</p> <h3 id="diversify-before-re-reading">Diversify before Re-reading</h3> <p>If you read a book you really want to (or need to) understand, but you’re struggling, it may seem tempting but drudging to re-read it. Find a different book on the same topic instead. Often the different point of view helps you grasp the original book.</p> <p>(Update February 12, 2021:)</p> <h3 id="the-sigert-method">The Sigert Method</h3> <p>My friend Sigert went to music school at age 15, and didn’t learn much math or science there. Ten years later he found his passion after designing a small radio studio for the university. He wanted to become a freelance radio &amp; music recording studio designer/advisor, but he knew he needed to get a lot better at acoustics than only relying on what his ears told him.</p> <p>Sigert started reading what was essentially the bible of acoustics theory. As soon as he didn’t understand something (which happened at page 2 or so), he stopped and found a book to explain it. He worked his way back, from acoustics to waves to general physics to basic math. Then he moved forward again. By the time he got to the acoustics bible, not only was it a fast and easy read, but he was already helping clients, which he attracted through his blog on designing studios.</p> <h3 id="what-are-your-reading-tips">What are your reading tips?</h3> <p>Comment below.</p> Javascript Dependency Injection with partial functions 2012-10-30T00:00:00+00:00 https://verraes.net/2012/10/javascript-dependency-injection-with-partial-functions <p>My gut reaction when writing Javascript, is to try and mimck patterns and concepts that I’m used to in class-based languages. Turns out Javascript allows you to do a whole bunch of things differently, and, dare I say it, more elegantly. All you have to do is let go of the luggage you bring from languages like Java or PHP.</p> <p>Say we want to handle some commands. In classical OOP, we’d start by making a <code class="language-plaintext highlighter-rouge">CommandHandler</code>, and give it one public method called <code class="language-plaintext highlighter-rouge">handle(command)</code>. Our method is probably going to have some services it depends on, like a Repository. We inject that into the class’ constructor. (Note that in javascript, passing <code class="language-plaintext highlighter-rouge">repository</code> as a parameter will automatically bring it into scope in <code class="language-plaintext highlighter-rouge">handle()</code>).</p> <p>Elsewhere, we create an instance of the <code class="language-plaintext highlighter-rouge">CommandHandler</code> – for example when we set up a dependency injection container. And again elsewhere, we call the <code class="language-plaintext highlighter-rouge">handle</code> method on our instance.</p> <div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// CommandHandler Class</span> <span class="kd">var</span> <span class="nx">CommandHandler</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">repository</span><span class="p">)</span> <span class="p">{</span> <span class="k">this</span><span class="p">.</span><span class="nx">handle</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">command</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// do stuff with repository and command</span> <span class="p">}</span> <span class="p">}</span> <span class="c1">// Elsewhere, create an instance of the class...</span> <span class="kd">var</span> <span class="nx">myCommandHandler</span> <span class="o">=</span> <span class="k">new</span> <span class="nx">CommandHandler</span><span class="p">(</span><span class="nx">repository</span><span class="p">);</span> <span class="c1">// ...and use it</span> <span class="nx">myCommandHandler</span><span class="p">.</span><span class="nx">handle</span><span class="p">(</span><span class="nx">command</span><span class="p">);</span> </code></pre></div></div> <h3 id="going-class-less">Going class-less</h3> <p>This sort of code is all too familiar, and it’s perfectly valid in Javascript. But a couple of things bug me. First of all there’s the Noun.verb() combo. We’re really only interested in one method: <code class="language-plaintext highlighter-rouge">handle</code>. Yet we have to have a class, with an artificial name like <code class="language-plaintext highlighter-rouge">CommandHandler</code>, which is just a there to bring us the <code class="language-plaintext highlighter-rouge">handle</code> method.</p> <p>A second quirk of class-based OOP, is that we need to have a class, as well as an instance of that class. I’m not saying there’s anything wrong with that, but after a couple of days diging into Javascript, it feels silly.</p> <p>So let’s get rid of the class, and keep only the verb, aka the <code class="language-plaintext highlighter-rouge">handle()</code> method. That means we now have to pass in the <code class="language-plaintext highlighter-rouge">Repository</code> in some other way. We can pass it in as a parameter, but that’s awkward, because then we’d have to have the <code class="language-plaintext highlighter-rouge">Repository</code> object available everywhere we want the <code class="language-plaintext highlighter-rouge">handle()</code>, but that’s too much internal information for client code to know.</p> <h3 id="partial-functions-to-the-rescue">Partial functions to the rescue</h3> <p>Functional programming has a neat little trick that can be applied here. A partial is function that takes another function, and ‘pre-fills’ a number of parameters. It returns a function that only needs the leftover parameters. Here’s a typical example:</p> <div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">sum</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">a</span><span class="p">,</span> <span class="nx">b</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nx">a</span> <span class="o">+</span> <span class="nx">b</span><span class="p">;</span> <span class="p">}</span> <span class="kd">var</span> <span class="nx">addFiveTo</span> <span class="o">=</span> <span class="nx">partial</span><span class="p">(</span><span class="nx">sum</span><span class="p">,</span> <span class="mi">5</span><span class="p">);</span> <span class="nx">sum</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="mi">10</span><span class="p">)</span> <span class="c1">// returns 15</span> <span class="nx">addFiveTo</span><span class="p">(</span><span class="mi">10</span><span class="p">)</span> <span class="c1">// returns 15</span> </code></pre></div></div> <p>Partials can easily be used to ‘configure’ a function in a DIC, and the use the pre-configured version of the function in the client code:</p> <div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">var</span> <span class="nx">_</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">lodash</span><span class="dl">'</span><span class="p">);</span> <span class="c1">// Lodash is an API-compatible fork of Underscore.js</span> <span class="c1">// The same as the first example, but as a class-less function</span> <span class="kd">var</span> <span class="nx">handle</span> <span class="o">=</span> <span class="kd">function</span><span class="p">(</span><span class="nx">repository</span><span class="p">,</span> <span class="nx">command</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// do stuff with repository and command</span> <span class="p">}</span> <span class="c1">// Elsewhere, inject the repository by creating a partial</span> <span class="kd">var</span> <span class="nx">handle</span> <span class="o">=</span> <span class="nx">_</span><span class="p">.</span><span class="nx">partial</span><span class="p">(</span><span class="nx">handle</span><span class="p">,</span> <span class="nx">repository</span><span class="p">);</span> <span class="c1">// ...and use it</span> <span class="nx">handle</span><span class="p">(</span><span class="nx">command</span><span class="p">);</span> </code></pre></div></div> <p>I’m still playing around with this kind of code, so I haven’t decided yet if this is the approach I’ll be using to replace all single-method classes. In any case, Javascript’s dynamic nature makes it easy to come up with many different ways of doing this.</p> <h2 id="comments">Comments</h2> <h3 id="marijn-huizendveld---20121030"><a href="http://twitter.com/huizenveld">Marijn Huizendveld</a> - 2012/10/30</h3> <p>This looks nice. However, what you loose in cruft you also loose in clarity. In the class-based implementation, a new developer can simply look up the class definition to see the dependencies of a certain command handler. It seems to me that with this partial-function approach a new developer has to look at the handler implementation <em>and</em> the DIC to validate what stuff gets injected. Not sure if that’s a trade-off I’d be willing to make.</p> <h3 id="mathias-verraes---20121030"><a href="http://twitter.com/mathiasverraes">Mathias Verraes</a> - 2012/10/30</h3> <p>If you’re just writing client code, I don’t think there’s any added complexity. In production code, you’d write something like <code class="language-plaintext highlighter-rouge">container.handleCommand(command)</code> instead of <code class="language-plaintext highlighter-rouge">container.commandHandler.handle(command)</code>. I suppose you could do type hinting in an IDE with annotations like JsDoc or Google Closure Compiler, so that should help reduce the clicking around. But as I said, I’m stilling fooling around, I’m sure I’ll get a better feel for what is practical and what is not.</p> Code Folder Structure 2011-10-22T00:00:00+00:00 https://verraes.net/2011/10/code-folder-structure <p>All code bases I ever read or worked with, share a similar folder structure:</p> <ul> <li>Controllers <ul> <li>BlogPostController</li> <li>CommentController</li> </ul> </li> <li>Models <ul> <li>BlogPostModel</li> <li>CommentModel</li> </ul> </li> <li>Views <ul> <li>BlogPostsView</li> <li>BlogPostDetailView</li> </ul> </li> <li>Helpers <ul> <li>…</li> </ul> </li> </ul> <p>The ones where the developer has read Domain Driven Design, or is using Doctrine2 or Hibernate, usually have a better focus on the domain model:</p> <ul> <li>Model <ul> <li>Entities <ul> <li>BlogPost</li> <li>Comment</li> <li>User</li> </ul> </li> <li>Repositories <ul> <li>BlogPostRepository</li> <li>CommentRepository</li> <li>UserRepository</li> </ul> </li> <li>Services <ul> <li>UserService</li> <li>…</li> </ul> </li> </ul> </li> </ul> <p>The philosophy of these folder structures is usually inspired by the frameworks they use. After all, if your framework is organized like this, it must be a best practice, right? It does make really good sense for a framework to be organized in packages [modules, components, …] like this.</p> <p>For your application, it’s a missed opportunity. You are not communicating the role each class has in relation to others, or the dependencies it has. A BlogPostRepository and a CommentRepository have no direct relation to each other, apart from the fact that they are both Repositories. However, a BlogPostRepository has a very tight dependency on BlogPost.</p> <ul> <li>BlogDomain <ul> <li>BlogPost <ul> <li>BlogPost</li> <li>BlogPostRepository</li> </ul> </li> <li>Comment <ul> <li>Comment</li> <li>CommentRepository</li> </ul> </li> </ul> </li> <li>CoreDomain <ul> <li>User <ul> <li>User</li> <li>UserRepository</li> </ul> </li> </ul> </li> </ul> <p>This makes it a lot easier to communicate bounded contexts, and to illustrate dependencies. For example, the BlogDomain depends on the CoreDomain. On a smaller scale, the BlogPost package depends on the Comment package. Zooming in even further, BlogPostRepository depends on BlogPost.</p> <p><img src="/img/posts/folderstructure1.png" alt="Folder structure" /></p> <p>In other words: A BlogPost and a Comment know about their author. A BlogPost has zero or more Comments, but the Comments are not aware that they belong to BlogPost. A BlogPostRepository manages BlogPost entities, but those entities have no idea that they are being managed.</p> <p>Obviously the whole example is too simple, as examples usually are. The point is that, to keep code clean, it’s important to think hard about coupling between elements. A folder structure aides to delineate depedencies. Close proximity in the tree suggests closer coupling. Documentation can further help to explain the direction of the coupling. We may just as well decide that Comments do know about the BlogPost they belong too, but that should be a conscious decision.</p> <p><strong>Update August 18, 2014:</strong> I still agree with the general idea of this post, which is to group files or classes according to their cohesion in the domain and the model, as opposed to their underlying patterns or infrastructure. However, I wouldn’t use names like BlogDomain and CoreDomain anymore. Whatever your core domain is, it might change in the future. Businesses evolve. Authentication might be a better name. I also wouldn’t use the suffix “Domain” anymore. And finally, I would attempt to reduce the coupling between blogs and authentication, for example by only sharing a small set of Domain Events.</p> Coping with Change in Software Development 2011-07-21T00:00:00+00:00 https://verraes.net/2011/07/coping-with-change-in-software-development <p>In a discussion about keeping developers motivated, someone said:</p> <p>“I also try as hard as I can to isolate our developers […] from out-of-spec change requests”</p> <p>Sticking to the plan sure allows developers and managers to stay in their comfort zone. The project keeps moving forward, the deadlines are met. But we’re not in the business of meeting deadlines. We’re in the business of building software that delivers value to customers. As the customer gains new insight in what will get the most value, the specs change. <a href="http://www.google.com/?q=change+is+the+only+constant">And change they will.</a></p> <p>The reaction of the software industry has traditionally been to avoid change. Get the customer to sign off on the plan, then stick to it. Make them understand that changes are going to cost them dearly. Shield the developers from change.</p> <p>The effect is backwards: customers, managers and developers are instilled with a holy fear of change. When, after much resistance, a change becomes inevitable, nobody knows how to deal with it.</p> <p>Knowing how to cope with change in a software project is a skill like any other, whether it’s atomic refactoring, or big architectural changes, or writing the automated tests that will help you keep your sanity in the process.</p> <p>Like any skill, you can master it with study and practice. Perhaps the best way to learn how to cope with change in software development, is to deliberately introduce change. It would make an interesting experiment: tell your developers to change things, just to keep them alert and well-trained in the art of change. I don’t know if they will declare you crazy, or if they will be motivated by it. All I know is that I would.</p> Lazy Loading in PHP with Closures 2011-05-17T00:00:00+00:00 https://verraes.net/2011/05/lazy-loading-with-closures <p>Closures are a great way to do all kinds of neat tricks in PHP, and they’re particularly useful for Lazy Loading. I’m currently involved in a +200k SLoC legacy project, and the challenge is moving it to a Domain Driven implementation (while improving the performance), with the ultimate goal of making it more testable.</p> <h3 id="the-problem">The problem</h3> <p>We want to find a Customer, and ask it for a list of Orders:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="c1">// client code</span> <span class="nv">$customer</span> <span class="o">=</span> <span class="nv">$customerRepository</span><span class="o">-&gt;</span><span class="nf">find</span><span class="p">(</span><span class="nv">$id</span><span class="p">);</span> <span class="nv">$orders</span> <span class="o">=</span> <span class="nv">$customer</span><span class="o">-&gt;</span><span class="nf">getOrders</span><span class="p">();</span> </code></pre></div></div> <p>With the ActiveRecord pattern, this is simple. The Customer object holds an instance of the database adapter, and queries it for related Orders:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">Customer</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getOrders</span><span class="p">()</span> <span class="p">{</span> <span class="nv">$ordersData</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">db</span><span class="o">-&gt;</span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select orders... */</span><span class="p">);</span> <span class="nv">$orders</span> <span class="o">=</span> <span class="k">array</span><span class="p">();)</span> <span class="k">foreach</span><span class="p">(</span><span class="nv">$ordersdata</span> <span class="k">as</span> <span class="nv">$orderdata</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$orders</span><span class="p">[]</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Order</span><span class="p">(</span><span class="nv">$orderdata</span><span class="p">);</span> <span class="p">}</span> <span class="k">return</span> <span class="nv">$orders</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>The downside of ActiveRecord, is that it violates the principle of Separation of Concerns. The Customer class contains domain knowledge (“What is a customer, how does it behave?”), as well as persistence knowledge (“How do we store a customer?”). The Customer class in our example even knows how to find it’s Orders in the database. The tight coupling between Customer and the database makes it less portable, and hard to test.</p> <p>In DDD, we solve this by keeping the Customer class pure, and move the logic for storing the object to a CustomerRepository. Clients of the Repository don’t know how or where it finds Customers, and the Customer class itself doesn’t know anything about the Repository or the database it is stored in. As for the Orders, they are pushed in the Customer at creation time.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">Customer</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getOrders</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">orders</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="kd">class</span> <span class="nc">CustomerRepository</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">find</span><span class="p">(</span><span class="nv">$id</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$customerdata</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">db</span><span class="o">-&gt;</span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select customer ...*/</span><span class="p">);</span> <span class="nv">$customer</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Customer</span><span class="p">(</span><span class="nv">$customerdata</span><span class="p">);</span> <span class="nv">$ordersdata</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">db</span><span class="o">-&gt;</span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select orders ... */</span><span class="p">);</span> <span class="k">foreach</span><span class="p">(</span><span class="nv">$ordersdata</span> <span class="k">as</span> <span class="nv">$orderdata</span><span class="p">){</span> <span class="nv">$customer</span><span class="o">-&gt;</span><span class="nf">addOrder</span><span class="p">(</span><span class="k">new</span> <span class="nc">Order</span><span class="p">(</span><span class="nv">$orderdata</span><span class="p">));</span> <span class="p">}</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Note that the client code for this example is still the same as in the first snippet.</p> <h3 id="adding-lazy-loading-to-the-mix">Adding Lazy Loading to the mix</h3> <p>The problem with the previous example is that we always query the database for the Orders, even when we don’t need them. We don’t want to move that query back to the Customer class, but we want to keep our client code intact. The trick is to move the logic for finding Orders into a Closure, push it into the Customer instance, and execute only when we actually need the Orders. In other words, Customer now holds a reference to the Orders, and only dereferences it at the very last moment – hence the term Lazy Loading.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">Customer</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">setOrdersReference</span><span class="p">(</span><span class="kt">Closure</span> <span class="nv">$ordersReference</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">ordersReference</span> <span class="o">=</span> <span class="nv">$ordersReference</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getOrders</span><span class="p">()</span> <span class="p">{</span> <span class="k">if</span><span class="p">(</span><span class="o">!</span><span class="k">isset</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="n">orders</span><span class="p">))</span> <span class="p">{</span> <span class="nv">$reference</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">ordersReference</span><span class="p">;</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">orders</span> <span class="o">=</span> <span class="nv">$reference</span><span class="p">();</span> <span class="p">}</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">orders</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="kd">class</span> <span class="nc">CustomerRepository</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">find</span><span class="p">(</span><span class="nv">$id</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$db</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">db</span><span class="p">;</span> <span class="nv">$customerdata</span> <span class="o">=</span> <span class="nv">$db</span><span class="o">-&gt;</span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select customer ...*/</span><span class="p">);</span> <span class="nv">$customer</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Customer</span><span class="p">(</span><span class="nv">$customerdata</span><span class="p">);</span> <span class="nv">$ordersReference</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$customer</span><span class="p">)</span> <span class="k">use</span><span class="p">(</span><span class="nv">$id</span><span class="p">,</span> <span class="nv">$db</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$ordersdata</span> <span class="o">=</span> <span class="nv">$db</span><span class="o">-&gt;</span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select orders ... */</span><span class="p">);</span> <span class="nv">$orders</span> <span class="o">=</span> <span class="k">array</span><span class="p">();</span> <span class="k">foreach</span><span class="p">(</span><span class="nv">$ordersdata</span> <span class="k">as</span> <span class="nv">$orderdata</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$orders</span><span class="p">[]</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Order</span><span class="p">(</span><span class="nv">$orderdata</span><span class="p">);</span> <span class="p">}</span> <span class="k">return</span> <span class="nv">$orders</span><span class="p">;</span> <span class="p">};</span> <span class="nv">$customer</span><span class="o">-&gt;</span><span class="nf">setOrderReference</span><span class="p">(</span><span class="nv">$ordersReference</span><span class="p">);</span> <span class="k">return</span> <span class="nv">$customer</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>The client code is still exactly as in the very first code snippet. But this time, when we call getOrders(), the Closure we prepared in find() is executed. It’s self-contained: it has the database instance and the Customer’s id, and it knows what to do. Customer on the other hand has no idea what goes on inside the Closure, but can perfectly deliver on getOrders() calls.</p> <h3 id="conclusion">Conclusion</h3> <p>Clearly this is a simplified example. My colleague, who has a lot more experience with the project, pointed out a whole bunch of real life use cases where this method would cause an abundance of queries. I’ll keep you posted when we figure out how to deal with those.</p> <h2 id="comments">Comments</h2> <h3 id="alessandro-nadalin---20110517">Alessandro Nadalin - 2011/05/17</h3> <p>One of the good things of doctrine 1 was that it used lazy loading by default. That meant if you didn’t want to do N bunch of separated queries you only needed to do a -&gt;leftJoin() when quering a Doctrine_Table (which, conceptually, is similar - but far away - to the repository).</p> <p>But it was active record… gosh :)</p> <p>BTW, i’d suggest not to inject the whole instance of the DB abstraction into the closure, but something like a query object ( Doctrine_Query, for instance )</p> <p>In order to avoid “a whole bunch of real life use cases where this method would cause an abundance of queries” you should not use a -&gt;find(), which should be relation-agnostic, but some other pre-defined methods like:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">UserRepository</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getUserWithFriends</span><span class="p">(</span><span class="nv">$userId</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">createQuery</span><span class="p">(</span><span class="s1">'UserTable'</span><span class="p">)</span><span class="o">-&gt;</span><span class="nf">leftJoin</span><span class="p">(</span><span class="s1">'friends'</span><span class="p">)</span><span class="o">-&gt;</span><span class="nf">blablabla</span><span class="p">();</span> <span class="p">}</span> </code></pre></div></div> <h3 id="herman-peeren---20110517">Herman Peeren - 2011/05/17</h3> <p>My idea about Lazy Loading:</p> <p>The Lazy Loading in this example only is more efficient when the total number of hits to getOrders() is less than the number of customer-objects. So, it depends on the situation when to use Lazy Loading or not.</p> <p>To be able to take this design decision, you need to know in what situations it will be used: the “art” of programming.</p> <h3 id="mathias-verraes---20110517">Mathias Verraes - 2011/05/17</h3> <p>Exactly. I failed to mentioned that normally, you’d use an ORM like Doctrine2 to do this kind of stuff. Unfortunately the project is too large, too complex, and too critical to refactor to Doctrine2 (or at least not in one go).</p> <h3 id="herman-peeren---20110517-1">Herman Peeren - 2011/05/17</h3> <p>But, as far as I know, Doctrine2 ALWAYS uses Lazy Loading. And my point is: there might be situations where that is not the best choice. So, we have a lot of ease of use when using an ORM-framework, but will sometimes get a lack of performance.</p> <p>An advantage when you need to program everything yourself, is that you can still take the decision yourself to use Eager or Lazy Loading. As Johan Cruijff (Dutch Ajax Champion) would say: “every disadvantage has it’s advantage”.</p> Ubiquitous Language 2011-05-12T00:00:00+00:00 https://verraes.net/2011/05/ubiquitous-language <p>Customers usually have never been forced to really think about their domain in a structured way – let alone explain it in detail to a developer, who doesn’t know anything about that domain. Often the customer doesn’t have a set of clear definitions for the concepts his business uses daily, or has multiple terms for the same concepts.</p> <p>Developers are even worse. They also tend to have vague concepts and lots of ways to name the same thing. Depending on where you look in the code or the documentation (if any), you might see: “bank account”, “account”, “bank_account”, “bankAccount”, “BankAccount”. And worse: “BA”, “bank_acc”, “acnt”. When storage is involved, you’ll find “item”, “row”, “record”, “model”. And another set of words for a collection of bank accounts: “list”, “array”, “collection”, “bankaccounts”, “acntset”, … you get the picture.</p> <p>Even on small projects, this hinders communication. Developers, project managers, customers, and others involved with the project, need to communicate about the domain, but everybody uses a different, but equally vague terminology.</p> <h3 id="a-common-language">A common language</h3> <p>The overcome this, you should start building a ubiquitous language, from the moment you start talking to the customer. Use a single term for a single concept, be it a thing, a relationship, or a behavior. Use the exact same term in the code, in the documentation, and in speech, and give it a strict spelling, down to the capitalization. Always “BankAccount”, never “bank account”. Don’t use abbreviations, as they make everything harder to read (except for really common ones, like “ISBN”).</p> <p>You need to be disciplined about it, and expect the same from everybody involved: correct the customer when he uses a different word, fix the spelling when someone writes it down incorrectly, and change other’s code. Some people might get annoyed at first at your insistence, but they will soon get comfortable with using the language.</p> <p>Maintain this ubiquitous language throughout the project. When concepts change, or your understanding of the domain changes, adjust the language, make sure it is adapted everywhere, and tell everyone.</p> <h3 id="benefits">Benefits</h3> <p>You’ll notice that soon, communication between everyone in the project will go much easier. When something is unclear, you can refer to the documentation. For new people on the project, it will be a lot easier to get up to speed. Code becomes easier to read, and easier to understand, as there are less concepts, and they are clearly defined. The application is much less vulnerable to regressions, and changing or refactoring is a breeze.</p> Beautiful code 2011-04-05T00:00:00+00:00 https://verraes.net/2011/04/beautiful-code <p>Code doesn’t have to be beautiful. Some of the most successful projects have the ugliest code. Marketing, features, vendor lock-in, shiny GUI’s and all kinds of real or imagined qualities are what make or break a software product. Of all the factors that’ll make you reach your audience, beautiful code dangles somewhere at the bottom. The business value of beautiful code is near zero. CEO’s, managers, customers, and users, they don’t give a shit about the code. So neither should we. Just keep pumping out those LOC’s.</p> <p>But we do care. All good programmers care about beautiful code. A well-crafted code base makes us happy to go to work every day. It makes us confident that anything they’ll throw at us, can be handled with grace.</p> <p>And even if the code isn’t there yet, just having the opportunity to improve it gives us energy. Making all kinds of small, hardly visible changes, and over time seeing how all the wrinkles and hiccups disappear, is like watching natural evolution produce the perfect flower.</p> <p>A last-minute change request from a customer is no problem for beautiful code. We welcome it as a chance to prove that the painstaking refactoring was worth it. We’re thrilled at how ridiculously easy it has become to respond to change.</p> <p>The world is littered with garbage. So let’s pick up the trash in our own neighbourhood. Leave every piece of code behind a little cleaner than you found it. Beautiful code does make a difference.</p> <p>Read next: <a href="https://verraes.net/2013/07/managed-technical-debt/">Managed Technical Debt</a> - July 21, 2013</p> Representing Money in PHP, Fowler-style 2011-04-04T00:00:00+00:00 https://verraes.net/2011/04/fowler-money-pattern-in-php <p>Whenever working with values in object oriented programming, it’s often a good idea to wrap them in a ValueObject. Money is a perfect candidate for a ValueObject: When talking about money, numbers are meaningless if they are not combined with a currency.</p> <p>I’ve been using a very simple version of the Money pattern as described in <a href="http://martinfowler.com/books.html">Martin Fowler’s PoEAA</a>. I couldn’t find a PHP implementation anywhere, so I decided to make my own little open source library for it. You can find it on <a href="https://github.com/mathiasverraes/money">my GitHub account</a> (where else?).</p> <h3 id="immutability">Immutability</h3> <p>An important aspect of ValueObjects is their immutability:</p> <p>Let’s say Jim and Hannah both want to buy a copy of book priced at EUR 25.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$jim_price</span> <span class="o">=</span> <span class="nv">$hannah_price</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="p">(</span><span class="mi">2500</span><span class="p">,</span> <span class="k">new</span> <span class="nc">Euro</span><span class="p">);</span> </code></pre></div></div> <p>Jim has a coupon for EUR 5.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$coupon</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="p">(</span><span class="mi">500</span><span class="p">,</span> <span class="k">new</span> <span class="nc">Euro</span><span class="p">);</span> <span class="nv">$jim_price</span><span class="o">-&gt;</span><span class="nf">subtract</span><span class="p">(</span><span class="nv">$coupon</span><span class="p">);</span> </code></pre></div></div> <p>Because <code class="language-plaintext highlighter-rouge">$jim_price</code> and <code class="language-plaintext highlighter-rouge">$hannah_price</code> are the same object, you’d expect Hannah to now have the reduced price as well. To prevent this problem, Money objects are immutable. With the code above, both <code class="language-plaintext highlighter-rouge">$jim_price</code> and <code class="language-plaintext highlighter-rouge">$hannah_price</code> are still EUR 25:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$jim_price</span><span class="o">-&gt;</span><span class="nf">equals</span><span class="p">(</span><span class="nv">$hannah_price</span><span class="p">);</span> <span class="c1">// true</span> </code></pre></div></div> <p>The correct way of doing operations is:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$jim_price</span> <span class="o">=</span> <span class="nv">$jim_price</span><span class="o">-&gt;</span><span class="nf">subtract</span><span class="p">(</span><span class="nv">$coupon</span><span class="p">);</span> <span class="nv">$jim_price</span><span class="o">-&gt;</span><span class="nf">lessThan</span><span class="p">(</span><span class="nv">$hannah_price</span><span class="p">);</span> <span class="c1">// true</span> <span class="nv">$jim_price</span><span class="o">-&gt;</span><span class="nf">equals</span><span class="p">(</span><span class="nc">Money</span><span class="o">::</span><span class="nf">euro</span><span class="p">(</span><span class="mi">2000</span><span class="p">));</span> <span class="c1">// true</span> </code></pre></div></div> <h3 id="allocation">Allocation</h3> <p>My company made a whopping profit of 5 cents, which has to be divided amongst myself (70%) and my investor (30%). Cents can’t be divided, so I can’t give 3.5 and 1.5 cents. If I round up, I get 4 cents, the investor gets 2, which means I need to conjure up an additional cent. Rounding down to 3 and 1 cent leaves me 1 cent. Apart from re-investing that cent in the company, the best solution is to keep handing out the remainder until all money is spent. In other words:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$profit</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="k">new</span> <span class="nc">Euro</span><span class="p">);</span> <span class="k">list</span><span class="p">(</span><span class="nv">$my_cut</span><span class="p">,</span> <span class="nv">$investors_cut</span><span class="p">)</span> <span class="o">=</span> <span class="nv">$profit</span><span class="o">-&gt;</span><span class="nf">allocate</span><span class="p">(</span><span class="mi">70</span><span class="p">,</span> <span class="mi">30</span><span class="p">);</span> </code></pre></div></div> <p>Now <code class="language-plaintext highlighter-rouge">$my_cut</code> is 4 cents, and <code class="language-plaintext highlighter-rouge">$investors_cut</code> is 1 cent. The order in which you allocate the the money is important:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="k">list</span><span class="p">(</span><span class="nv">$investors_cut</span><span class="p">,</span> <span class="nv">$my_cut</span><span class="p">)</span> <span class="o">=</span> <span class="nv">$profit</span><span class="o">-&gt;</span><span class="nf">allocate</span><span class="p">(</span><span class="mi">30</span><span class="p">,</span> <span class="mi">70</span><span class="p">);</span> </code></pre></div></div> <p>Now <code class="language-plaintext highlighter-rouge">$my_cut</code> is 3 cents, and <code class="language-plaintext highlighter-rouge">$investors_cut</code> is 2 cents.</p> <h3 id="conclusion">Conclusion</h3> <p>At the moment my Money class has all the features that Fowler lists. You can see some more example by looking at the <a href="https://github.com/mathiasverraes/money/tree/master/tests">unit tests</a>. I hope to add a lot stuff more if time permits: currency conversion, parsing of strings like ‘$2.00’ and ‘USD 2.00’, dealing with major units and subunits in currencies, etc. In any case, I hope it’s useful to somebody in it’s present form.</p> Random thoughts on using Git 2011-03-29T00:00:00+00:00 https://verraes.net/2011/03/git-vs-svn <p>A couple of weeks ago, I switched from Subversion to Git for a couple of smaller projects. Below are some of my findings.</p> <h3 id="switching-from-subversion">Switching from Subversion</h3> <p>Git is a little harder to understand than Subversion, because of it’s lack of a centralized repository, and because it has more concepts you need to understand. But if you already understand SVN well, it’s very manageable. I’ve no idea how the learning curve compares to SVN for a complete newbie to version control.</p> <p>If you are on the fence, there are two things that I wish I knew about sooner:</p> <ul> <li><a href="http://gitimmersion.com/">Git Immersion</a> only takes an hour or so, but you’ll understand so much more of git than when using conventional books or tutorials.</li> <li><a href="http://help.github.com/svn-importing/">Migrating an SVN repo to git</a> is dead easy.</li> </ul> <h3 id="open-source-community">Open source community</h3> <p>GitHub alone is plenty of reason to switch to Git. It seems like the whole PHP community (and the whole open source community) are moving to GitHub. And you can’t blame them. GitHub works really, really well. It makes it easy to maintain your own fork of a project, make changes and send them as pull requests. Symfony2 for example noticed a huge increase in contributions since the move.</p> <p>Even if you only consume open source code and don’t plan on contributing patches, you’ll find you need to understand at least how cloning and pulling work, so you can easily download code.</p> <h3 id="svnexternals">Svn:externals</h3> <p>I usually include third party libraries in my projects using svn:externals. At the moment, a lot of PHP libraries that switched to git, still sync to SVN. But I’m sure that people will stop bothering with that. That’s a problem if your SVN repository depends externals. You could set up your own syncing, or take the hint and move to git.</p> <h3 id="fixing-oops-moments">Fixing ‘oops’ moments</h3> <p>With SVN, commits are pretty much written in stone. Because of that, a convention has emerged:</p> <ul> <li>Faulty commits are followed by a commit labeled ‘reverted rev 123’</li> <li>Incomplete commits are followed by a commit with the same message</li> </ul> <p>Git has a lot more possibilities to fix errors, amend to commits, etc. That’s a blessing and a curse. I find that by trying to make my history look pretty, I seem to make a mess of things. It’s not a big deal, because in Git, nothing is ever lost. Perhaps it’s just my lack of experience with git. In any case, I think that for now, I will stick to the SVN convention and won’t try to hide my clumsiness from the world.</p> Accessing private properties from other instances 2011-03-24T00:00:00+00:00 https://verraes.net/2011/03/accessing-private-properties-from-other-instances <p>In PHP, when a property or method is marked private, it can only be accessed from within that class. That includes <strong>other instances of the same class</strong>. This may seem counter-intuitive at first, because we are used to dealing with instances of classes. The visibility operator however works not on object-level, but on class level.</p> <p>An example:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">Foo</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$private</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$value</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="k">private</span> <span class="o">=</span> <span class="nv">$value</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">getOther</span><span class="p">(</span><span class="kt">Foo</span> <span class="nv">$object</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$object</span><span class="o">-&gt;</span><span class="k">private</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="nv">$foo1</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Foo</span><span class="p">(</span><span class="s1">'foo1'</span><span class="p">);</span> <span class="nv">$foo2</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Foo</span><span class="p">(</span><span class="s1">'foo2'</span><span class="p">);</span> <span class="k">echo</span> <span class="nv">$foo1</span><span class="o">-&gt;</span><span class="nf">getOther</span><span class="p">(</span><span class="nv">$foo2</span><span class="p">);</span> <span class="c1">// outputs 'foo2'</span> </code></pre></div></div> <p>This should make it clear that both instances of Foo have access to each other’s private properties.</p> <p>What practical use does this have? A great candidate for this are <a href="http://domaindrivendesign.org/node/135">Value Objects</a>. If we want to make sure that to separate instances of Foo are actually equal, we can easily compare their private properties:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">Foo</span> <span class="p">{</span> <span class="c1">// ...</span> <span class="k">public</span> <span class="k">function</span> <span class="n">equals</span><span class="p">(</span><span class="kt">Foo</span> <span class="nv">$other</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="k">private</span> <span class="o">===</span> <span class="nv">$other</span><span class="o">-&gt;</span><span class="k">private</span><span class="p">;</span> <span class="p">}</span> <span class="p">}</span> <span class="c1">// ...</span> <span class="k">echo</span> <span class="nv">$foo1</span><span class="o">-&gt;</span><span class="nf">equals</span><span class="p">(</span><span class="nv">$foo2</span><span class="p">)</span> <span class="o">?</span> <span class="s1">'Equal'</span> <span class="o">:</span> <span class="s1">'Different'</span><span class="p">;</span> </code></pre></div></div> <h2 id="comments">Comments</h2> <h3 id="nicholas-k-dionysopoulos---20110324">Nicholas K. Dionysopoulos - 2011/03/24</h3> <p>Nice info, Mathias! It’s very counter-intuitive indeed. On the other OO languages I know of (Delphi, various .NET dialects) a private variable is inaccessible outside the context of the owner object instance.</p> <h3 id="jonathan-mayhak---20110324">Jonathan Mayhak - 2011/03/24</h3> <p>Good info. Had no idea php worked like this.</p> <p>Also, great use case….very practical!</p> <h3 id="lolol---20110324">Lolol - 2011/03/24</h3> <p>This is also the case in Java, which i found rather awkward when i found it.. but apparently it has some use. kind of takes away the principle behind private if you can extend a class to create functions to read another instance of that base class` privates :)</p> Keep your controllers thin with Doctrine2 2011-03-23T00:00:00+00:00 https://verraes.net/2011/03/keep-you-controllers-thin-with-doctrine2 <p>Doctrine2 does such a nice job abstracting everything related to the database, that you might be tempted to do everything else in your controllers. Say we have a Bug entity:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="cd">/** @Entity */</span> <span class="kd">class</span> <span class="nc">Bug</span> <span class="p">{</span> <span class="cd">/** @Column(type="integer") */</span> <span class="k">private</span> <span class="nv">$id</span><span class="p">;</span> <span class="cd">/** @Column(length=50) */</span> <span class="k">private</span> <span class="nv">$status</span><span class="p">;</span> <span class="c1">//...</span> <span class="p">}</span> </code></pre></div></div> <p>To get a list of fixed bugs, we get the Bug repository from the EntityManager and ask for a list of Bugs where status equals ‘fixed’.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="o">&lt;?</span><span class="n">php</span> <span class="c1">// $em instanceof Doctrine\ORM\EntityManager</span> <span class="nv">$fixedbugs</span> <span class="o">=</span> <span class="nv">$em</span><span class="o">-&gt;</span><span class="nf">getRepository</span><span class="p">(</span><span class="s1">'Bug'</span><span class="p">)</span> <span class="o">-&gt;</span><span class="nf">findBy</span><span class="p">(</span><span class="k">array</span><span class="p">(</span><span class="s1">'status'</span> <span class="o">=&gt;</span> <span class="s1">'fixed'</span><span class="p">));</span> </code></pre></div></div> <p>That’s easy enough. Surely there can be no harm in having this code inside a controller? Although this code doesn’t look like one, it is in a fact a database query. It’s a shortcut for this:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$fixedbugs</span> <span class="o">=</span> <span class="nv">$em</span> <span class="o">-&gt;</span><span class="nf">createQuery</span><span class="p">(</span><span class="s2">"SELECT b FROM Bug b WHERE b.status = 'fixed'"</span><span class="p">)</span> <span class="o">-&gt;</span><span class="nf">getResult</span><span class="p">();</span> </code></pre></div></div> <p>Having a query in our controller should ring some serious alarm bells. It means that despite all abstraction Doctrine2 provides, we are at this point still coupling the controller to the database. If one day we decide to change how bug status is represented in the database, we’d need to modify all our controllers.</p> <p>Let’s have a closer look at the Repository. <a href="http://domaindrivendesign.org/books/evans_2003">Evans</a> defines it as “an object that can provide the illusion of an in-memory collection of all objects of that type”, that clients talk to using the domain language. In other words, findBy(array(‘status’ =&gt; ‘fixed’)) is too generic: in domain language, we want to ask the repository to findAllFixedBugs(). If the database schema changes, we’ll only have to change that method. Luckily Doctrine2’s repositories can be extended:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="cd">/** * @Entity(repositoryClass="BugRepository") */</span> <span class="kd">class</span> <span class="nc">Bug</span> <span class="p">{</span> <span class="cm">/* ... */</span> <span class="p">}</span> <span class="kd">class</span> <span class="nc">BugRepository</span> <span class="kd">extends</span> <span class="nc">EntityRepository</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">findAllFixedBugs</span><span class="p">()</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">_em</span> <span class="o">-&gt;</span><span class="nf">createQuery</span><span class="p">(</span><span class="s2">"SELECT b FROM Bug b WHERE b.status = 'fixed'"</span><span class="p">)</span> <span class="o">-&gt;</span><span class="nf">getResult</span><span class="p">();</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Finally we can replace the code in our controller with this</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$fixedbugs</span> <span class="o">=</span> <span class="nv">$em</span><span class="o">-&gt;</span><span class="nf">getRepository</span><span class="p">(</span><span class="s1">'Bug'</span><span class="p">)</span><span class="o">-&gt;</span><span class="nf">findAllFixedBugs</span><span class="p">();</span> </code></pre></div></div> <p>Our controller is now decoupled from the database. As an added bonus, the code does a much better job of communicating it’s intent than our first version ever could.</p> <h2 id="comments">Comments</h2> <h3 id="torkil-johnsen---20120323">Torkil Johnsen - 2012/03/23</h3> <p>Looks extremely nifty, but left me with questions. I’m not familiar with Doctrine, and you probably just wrote this as a crude example, so forgive the silly question! :)</p> <p>I’m wondering why you write something as specific as findAllFixedBugs()? Seems to me like you’ll need a findAllFixedBugsFilteredByUser() and findAllFixedBugsWithStatusCritical() around the next bend?</p> <p>And why did you not use findBy() in your final version of the method, but instead write a regular SQL statement?</p> <p>Other than that: Step up the pace on the blog posts, we need more of these :)</p> <h3 id="mathias-verraes----20120323">Mathias Verraes - 2012/03/23</h3> <p>It’s indeed a crude example.</p> <p>You would indeed need more methods, which you would discover as you write tests (see my previous post). The point of having these methods is that your code is speaking the same language as you speak with your customer (called ‘ubiquitous language’ in DDD). If you decide to move your bugs from a relational database to, say, a remote service, your methods would still be there, but the implementation would be entirely different. And of course, you can easily replace BugRepository with a mock object during testing, so you wouldn’t need a database to test your controller.</p> <p>There’s no particular reason I didn’t use findBy() in the last snippet, except that I wanted to illustrate that the method can contain complex queries that can’t be done with find by.</p> <p>Off topic: the queries in the example are DQL, not SQL. http://docs.doctrine-project.org/projects/doctrine-orm/en/2.0.x/reference/dql-doctrine-query-language.html</p> <h3 id="alessandro-nadalin---20120323">Alessandro Nadalin - 2012/03/23</h3> <p>Hi Torkil,</p> <p>yes, the repository patterns was specifically born to incapsulate all the methods you need to retrieve objects from the DB. So you basically add there all the possible queries you need in your application…</p> <p>You are obviously able to abstract some methods using a custom class between your repositories and the EntityRepository, which inherits from the latter.</p> <p>To use OO api and not raw DQL statements you need a QueryBuilder instance:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="nv">$qb</span> <span class="o">=</span> <span class="nv">$em</span><span class="o">-&gt;</span><span class="nf">getRepository</span><span class="p">(</span><span class="s1">'Odino\BlogBundle\Entity\Content'</span><span class="p">)</span><span class="o">-&gt;</span><span class="nf">createQueryBuilder</span><span class="p">(</span><span class="s1">'u'</span><span class="p">)</span> <span class="o">-&gt;</span><span class="nf">where</span><span class="p">(</span><span class="s1">'u.isActive = 1'</span><span class="p">)</span> <span class="o">-&gt;</span><span class="nf">andWhere</span><span class="p">(</span><span class="s2">"u.keywords LIKE ?1"</span><span class="p">)</span> <span class="o">-&gt;</span><span class="nf">setParameter</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s2">"%</span><span class="nv">$tag</span><span class="s2">%"</span><span class="p">);</span> <span class="nv">$qb</span><span class="o">-&gt;</span><span class="nf">getQuery</span><span class="p">()</span><span class="o">-&gt;</span><span class="nf">execute</span><span class="p">();</span> </code></pre></div></div> <p>ciao!</p> <h3 id="torkil-johnsen---20120323-1">Torkil Johnsen - 2012/03/23</h3> <p>Thanks to you both, enlightening!</p> Interface discovery with PHPUnit’s Mock objects 2011-03-21T00:00:00+00:00 https://verraes.net/2011/03/interface-discovery-with-phpunit-mock-objects <p><a href="https://github.com/sebastianbergmann/phpunit/">PHPUnit</a> provides some great features to create <a href="http://en.wikipedia.org/wiki/Mock_object">mock objects</a>. The idea is that when you are testing code that depends on another class, you provide the object with a mock instance of that class, instead of a real object. That way, you are making sure that your test will only fail if the system under test is broken, and not if one of it’s dependencies is broken. You could simply write a mock class and instantiate it, but PHPUnit can generate them for you.</p> <p>The PHPUnit documentation doesn’t explicitly state this, but you can also create mock objects from interfaces. This makes a lot of sense if you think about it. In many cases, you should actually use mocked interfaces in your tests instead of mocked concrete classes. After all, the interface is the contract by which classes agree to talk to the outside world.</p> <h3 id="a-simple-example">A simple example</h3> <p>Let’s write some code, in true <a href="http://en.wikipedia.org/wiki/Test-driven_development">TDD</a> style. Let’s say we want to post to Twitter whenever someone deposits money in our bank account. We don’t want the test to actually send out tweets. In fact, we haven’t even thought about what our Twitter class will look like. This is our test:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">BankAccountTest</span> <span class="kd">extends</span> <span class="nc">PHPUnit_Framework_TestCase</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">testSendEmailWhenReceivingMoney</span><span class="p">()</span> <span class="p">{</span> <span class="nv">$twitter</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">getMock</span><span class="p">(</span><span class="s1">'Twitter'</span><span class="p">);</span> <span class="nv">$account</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">BankAccount</span><span class="p">(</span><span class="nv">$twitter</span><span class="p">);</span> <span class="nv">$account</span><span class="o">-&gt;</span><span class="nf">deposit</span><span class="p">(</span><span class="mi">10</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>Running this test fails, as we haven’t got a BankAccount class yet.</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PHP Fatal error: Class 'BankAccount' not found </code></pre></div></div> <p>Let’s add it:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">BankAccount</span> <span class="p">{</span> <span class="k">private</span> <span class="nv">$twitter</span><span class="p">;</span> <span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">Twitter</span> <span class="nv">$twitter</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">twitter</span> <span class="o">=</span> <span class="nv">$twitter</span><span class="p">;</span> <span class="p">}</span> <span class="k">public</span> <span class="k">function</span> <span class="n">deposit</span><span class="p">(</span><span class="nv">$amount</span><span class="p">){</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>The test now succeeds, for the simple reason we are not really testing anything. Let’s make sure that BankAccount::deposit() actually sends out a tweet. We do this by telling the mock to expect a call to it’s tweet() method.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">BankAccountTest</span> <span class="kd">extends</span> <span class="nc">PHPUnit_Framework_TestCase</span> <span class="p">{</span> <span class="k">public</span> <span class="k">function</span> <span class="n">testSendEmailWhenReceivingMoney</span><span class="p">()</span> <span class="p">{</span> <span class="nv">$twitter</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">getMock</span><span class="p">(</span><span class="s1">'Twitter'</span><span class="p">);</span> <span class="nv">$twitter</span><span class="o">-&gt;</span><span class="nf">expects</span><span class="p">(</span><span class="nv">$this</span><span class="o">-&gt;</span><span class="nf">once</span><span class="p">())</span> <span class="o">-&gt;</span><span class="nf">method</span><span class="p">(</span><span class="s1">'tweet'</span><span class="p">);</span> <span class="nv">$account</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">BankAccount</span><span class="p">(</span><span class="nv">$twitter</span><span class="p">);</span> <span class="nv">$account</span><span class="o">-&gt;</span><span class="nf">deposit</span><span class="p">(</span><span class="mi">10</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>The test fails with the following message:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1) BankAccountTest::testSendEmailWhenReceivingMoney Expectation failed for method name is equal to &lt;string:tweet&gt;  when invoked 1 time(s) Method was expected to be called 1 times, actually called 0 times. </code></pre></div></div> <p>Let’s add some code that calls tweet() to our deposit() method.</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">class</span> <span class="nc">BankAccount</span> <span class="p">{</span> <span class="c1">// ...</span> <span class="k">public</span> <span class="k">function</span> <span class="n">deposit</span><span class="p">(</span><span class="nv">$amount</span><span class="p">)</span> <span class="p">{</span> <span class="nv">$this</span><span class="o">-&gt;</span><span class="n">twitter</span><span class="o">-&gt;</span><span class="nf">tweet</span><span class="p">(</span><span class="s2">"Yay, someone deposited </span><span class="nv">$amount</span><span class="s2">"</span><span class="p">);</span> <span class="p">}</span> <span class="p">}</span> </code></pre></div></div> <p>We get a new error:</p> <div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PHP Fatal error: Call to undefined method Mock_Twitter_28053312::tweet() </code></pre></div></div> <p>Mock_Twitter_28053312 is the class that PHPUnit generated based on the Twitter interface, which we haven’t written yet. The good news is that by now, we have discovered what the interface should look like:</p> <div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp">&lt;?php</span> <span class="kd">interface</span> <span class="nc">Twitter</span> <span class="p">{</span> <span class="k">function</span> <span class="n">tweet</span><span class="p">(</span><span class="nv">$message</span><span class="p">);</span> <span class="p">}</span> </code></pre></div></div> <h3 id="conclusion">Conclusion</h3> <p>The whole point of this technique, is that we have now written BankAccount, without worrying how our Twitter implementation will work. In the process, we discovered that we will need an implementation of a Twitter interface, and we discovered what that interface looks like.</p> <p>Later on, we probably build a RESTfulTwitter implements Twitter, which would post messages using the Twitter REST API. If we had build the Twitter implementation first, we’d probably would have been tempted to add all kinds of features that we thought we might need. Instead of that, we discovered what we actually needed. Our test proves that our code adheres to <a href="http://en.wikipedia.org/wiki/You_ain't_gonna_need_it">YAGNI</a>.</p> <p>Read next: <a href="/2013/09/sensible-interfaces/">Sensible Interfaces</a> - September 24, 2013</p>