Mathias Verraes2024-03-10T16:04:00+00:00https://verraes.net/Mathias Verraesmathias@verraes.netHow to Fix a Bug: Tests, Hypotheses, Timeboxes2024-03-10T00:00:00+00:00https://verraes.net/2024/03/how-to-fix-a-bug-tests-hypotheses-timeboxes<p>Here’s roughly how I fixed bugs early on in my career:</p>
<ul>
<li>Browse around in the code.</li>
<li>Try stuff.</li>
<li>See if it works.</li>
</ul>
<p>Below is my preferred way of doing it since about 2012:</p>
<h2 id="step-1-pair--ensemble">Step 1. Pair / Ensemble</h2>
<p>Find one or more people to collaborate with. <a href="https://en.wikipedia.org/wiki/Linus%27s_law">Linus’ Law</a> “Given enough eyeballs, all bugs are shallow” was formulated at least 25 years ago, and yet we do so much work in isolation.</p>
<h2 id="step-2-test">Step 2. Test</h2>
<ul>
<li>Write a failing test (or multiple) that proves the existence of the bug. This test will make sure you don’t fix the wrong thing, or a perceived bug. The test will help to ensure that the bug doesn’t return later. Bugs are the result of some misunderstanding about the system, and fixing a bug doesn’t imply that everybody’s misunderstanding is fixed. But if the test fails again in the future, at least you’ll know something’s up.</li>
<li>Besides a descriptive name, add the bug’s ticket number in the test name (or use a custom annotation). This will help keep track of the discussions about the bug.</li>
<li>Commit and push the test. Yes, this will break the build. I think this is perfectly fine: the code was already broken, it was just invisible. I get that breaking the build will be controversial in some organisations. In that case, if your tooling allows it, make sure the failing test shows as a warning but don’t let keep it from deploying.</li>
</ul>
<p>If writing a test is hard or impossible in your environment:</p>
<ul>
<li>Push a test that simply fails with an explanation of the bug and why it’s hard to test.</li>
<li>Once you’re confident the bug is fixed, remove the fail command and (depending on your tooling) mark it as “not implemented” “or “skipped”.</li>
<li>Invest in making it easier to test.</li>
</ul>
<h2 id="step-3-form-hypotheses">Step 3. Form Hypotheses</h2>
<ul>
<li>Brainstorm as many hypotheses about what causes the bug as you can think of. Having more people in the group helps, as groups simply have more knowledge and ideas than individuals.</li>
<li>As with all brainstorming, avoid debating the hypotheses. Just try to list as many as you can, including the ones you’re skeptical about.</li>
<li>In fact, I even recommend to come up with some hypotheses that are very unlikely to be the correct one. They trigger more ideas, and occasionally, the bug’s cause is the most unexpected one.</li>
<li>Definitely resist the urge to jump in the code and start fixing things. If you do need to look at the code to form hypotheses, don’t try to fix it right away.</li>
</ul>
<h2 id="step-4-prioritise-and-timebox">Step 4. Prioritise and Timebox</h2>
<ul>
<li>Reorder your list starting with the most probable hypotheses.</li>
<li>Add an estimate to each hypothesis. The estimate is not the time you need to fix the bug, but the time you need to falsify the hypotheses.</li>
<li>Now move the hypotheses that are both highly probably and have a short estimate to the top of the list. If something is quite improbable but only takes a minute to verify, you may also want to put those at the top.</li>
</ul>
<h2 id="step-5-timebox-and-falsify">Step 5. Timebox and Falsify</h2>
<ul>
<li>Set a timer.</li>
<li>Try to proof as fast as possible that the first hypothesis is not the cause of the bug.</li>
<li>To falsify, you can either write another test, or do the minimum effort to the make the original test pass.</li>
<li>Stop as soon as you have your evidence.</li>
<li>If your timer runs out before you’re done, decide to add another timebox, or move on to the next hypothesis.</li>
<li>Keep in mind that bugs can have multiple interacting causes, so proving or disproving one hypothesis doesn’t guarantee you falsified the others.</li>
</ul>
<h2 id="step-6-fix">Step 6. Fix</h2>
<ul>
<li>Now that you’re confident about the cause of the bug, fix it properly.</li>
<li>Write more tests to help guide your fix.</li>
</ul>
<p>One of my programming adages is</p>
<blockquote>
<p>“When in doubt, write a test.”</p>
</blockquote>
<p>And of course there’s</p>
<blockquote>
<p>“Doubt is the origin of wisdom.”</p>
<p>— René Descartes</p>
</blockquote>
<h2 id="the-engineers-duty">The Engineer’s Duty</h2>
<p>Does this process take more time?</p>
<blockquote>
<p>“There is never enough time to do it right, but there is always enough time to do it over.”</p>
<p>— John W. Bergman</p>
</blockquote>
<p>That said, I find that for complicated bugs, this process avoids a lot of diving into the wrong rabbit holes, chasing wild geese, doggedly unnecessarily rewriting entire chunks of code with your head in the sand. Instead of these animal metaphors, you can be rational about bugs. The group mind identifies better hypotheses; priorities and timeboxes avoid getting into a flow state for the wrong hypothesis.</p>
<p>If people in your organisation don’t do this sort of process, or don’t write tests, I have a hypothesis on why that is: Most organisations implicitly belief that the engineer’s duty is to deliver features. Fixing bugs is seen as a distraction from delivery, so you need to get it over with as fast as possible.</p>
<p>However, there’s a healthier way of looking at it: The engineer’s duty is building a system that works, that keeps working over time, and that keeps being evolvable and understandable over time. Basically, don’t just deliver features, but make it easy to deliver high quality features. In the same vein, tests reduce bugs and regressions, more tests lead to more testability, and more testability makes it easier to avoid more bugs. Jumping onto this self-reinforcing feedback loop may be tricky, but I promise it speeds up as you go along.</p>
Critical Software Redesign: Creating the Environment for Large Scale Change2024-01-08T00:00:00+00:00https://verraes.net/2024/01/critical-software-redesign<p>There’s a moment where it’s too soon to invest in a radical new software design for your system. And then you reach a moment where it’s too late, too expensive to change it. Is there a sweet spot in between, a point where pivoting the software design is the right choice, at the right time?</p>
<p>We’ve updated our book, <a href="https://leanpub.com/design-and-reality/">“Design and Reality”</a> with a new essay, “Critical Software Redesign: Creating the Environment for Large Scale Change”. It explores how you as a technical leader can prepare for major design shifts. We use as a running example, the evolution of a product that calculates taxes on global shipments.</p>
<p>In a nutshell, ongoing design challenges, the solutions you pick for them, and the impacts of those solutions can be treated as signals, both positive and negative. These signals show where the external environment repeatedly puts stress on your system’s design. More importantly, by paying attention to these signals over time, you can figure out what changes your system’s design needs. And you can use the signals to trigger experiments. Lots of them. And what you learn from these experiments can help you build the culture and support for more significant design changes.</p>
<p>Right now, the new essay is <strong>only available as part of our book</strong>. Rebecca and I will publish it in its entirety on our blogs at some point in the future.</p>
<p>In the meantime, we hope you’ll support our writing and <a href="https://leanpub.com/design-and-reality/">buy our book</a>!</p>
<p><em>Written by <a href="https://twitter.com/mathiasverraes">Mathias Verraes</a> and <a href="https://twitter.com/rebeccawb">Rebecca Wirfs-Brock</a>.</em></p>
<div class="columns mb-6 ">
<div class="column is-6">
<figure class="media-left image " style="width: 75%; box-shadow: 2px 2px 7px darkgray">
<a href="https://leanpub.com/design-and-reality">
<img src="https://d2sofvawe08yqg.cloudfront.net/design-and-reality/s_hero2x?1664874345" />
</a>
</figure>
</div>
<div class="column is-6">
<h1 class="title">Design and Reality</h1>
<h2 class="subtitle">Essays on Software Design</h2>
<p>by Mathias Verraes & Rebecca Wirfs-Brock</p>
<br />
<a class="button is-info" href="https://leanpub.com/design-and-reality">
Get it on Leanpub
</a>
<br />
<br />
<blockquote>
“Happily purchased. Handy to have these in one place. Thank you!” <br />— Kent Beck<br />
</blockquote>
<br />
<blockquote>
“Excellent new thinking on Domain-Driven Design. It's full of real practical experience in getting the most
value from domain modelling. Just like the Eric Evans' DDD book, this gives more insight each time you read
it.”<br />— Ian Russell<br />
</blockquote>
</div>
</div>
Surfacing Worldviews in Design2023-07-03T00:00:00+00:00https://verraes.net/2023/07/surfacing-worldviews-in-design<p>Design choices carry along the worldviews of the designer. This often is not apparent, especially when design ideas are for obvious technical improvements. Let’s look into a design challenge faced by a fictional Maker Lab.</p>
<p>In this Maker Lab, people in the community can come in and make all kinds of things such as metal work, robotics, furniture, glasswork, 3D printing, laser cutting, lighting, electronics, … The Lab provides tools, machines, and inventory that makers can use in the maker space. The makers are supposed to use the hand scanners, and scan the barcodes on equipment and materials as they take them out for use. Sometimes makers may need something for a few days, so it’s normal that they don’t put everything back right away. Or they might make parts, that are then stored for later use or for sharing.</p>
<p>As it happens in a community space, things get lost or misplaced. You don’t always know if it’s really lost, or just being used somewhere for an extended period of time. In any case, it’s a problem, and management decides to do something about it.</p>
<h2 id="intervention">Intervention</h2>
<p>The Lab’s software engineer is asked to add a new feature to the inventory app: it should track who was the last person to scan an item. The idea is that if you can check who was the last person to touch it, you can ask them about the item’s whereabouts. The feature is quickly put in production, and for a while, things get a bit better. But after a while, it gets worse: items are rarely being scanned anymore, if at all. Things get lost more often, and turn up much later in unexpected places.</p>
<p>Eventually they figure out what happened: When makers couldn’t find the item or tool they were looking for, they would ask the person who was the last to scan it. This led to some heated discussions, and accusations of theft. But of course when something is lost, the last person to scan it, isn’t necessarily the last person to touch it. For example, someone leaves a tool at their station because they’ll need it again soon. But then someone else wants to use the station, and moves the tool to makes space. Or sometimes a maker grabs the tool from the station for a quick use, doesn’t scan it, and forgets to put it back. The makers who conscientiously scanned all their tools were getting all the blame. They felt they were punished for trying to do a good job. And after all, this is a hobby environment, not a NASA lab. Eventually they too stopped bothering, and the problem of lost items got even worse.</p>
<h2 id="consequences">Consequences</h2>
<p>What happened here, is that a technical solution was implemented to “help” deal with a social problem. Social problems tend to require more nuanced solutions than technical ones. Seemingly simple and straightforward interventions can have unintended consequences, and those consequences can have more consequences, and so on. It’s impossible to predict the chains of consequences. The result is that sometimes, the consequences have the opposite effect that you expect.</p>
<p>What happens next? Management discovers that the tracking isn’t sufficient, and imposes new, stricter rules on using items. A person is assigned to guard the inventory, and makers have to go through them to get the items they need. The guard makes sure the makers bring back the material before leaving, even if they intend to use it the next day. These “improvements” add more work for the makers, most of whom were already compliant, and with no clear extra benefit to them. They are made to feel they couldn’t be trusted. Gradually, the culture declines, and makers stop coming. Often in such situations, you never really find out why.</p>
<h2 id="another-scenario">Another scenario</h2>
<p>Now let’s imagine another scenario of how the Maker Lab inventory problem might have played out. Instead of rushing to a technical solution, management took stock of how people in the lab behave. What aspects and qualities of the larger system make the Maker Lab what it is? People enjoy coming, and most of them actually do try to put things back correctly. Some do so because it’s the rules, and they want to be good community members. Others intrinsically understand that by putting things back, they increase the likelihood of finding them later. They get value from finding items, and they create value by putting them back.</p>
<p>If 80% of people are doing things well, should they be punished for the other 20%? Perhaps we can find out how we can grow the 80% to an acceptable 95%. If you take that point of view, the question changes from “How can we force people into good behaviour?”, into “What bottlenecks can we remove to get the remaining people into this mindset?”. It’s not just about preserving what already works, it’s about expanding on it.</p>
<p>So management decides to do gradual redesign of how inventory is handled, through communicating expectations. Management makes some improvements in how they onboard new makers and welcome them to the lab. They add signs explaining the value of a well organised lab. They even assign a person to help locate things when they do get lost, and to thank people when they put stuff back. It’s subtle, but it’s focused on rewarding good behaviour rather than enforcing it.</p>
<p>A social system like the lab often requires a social approach instead of merely a technical intervention. You consider the whole system, and by finding and reinforcing positive actions instead of making big sweeping changes, you can gradually tease out better ways.</p>
<h2 id="worldviews">Worldviews</h2>
<p>At the heart of these scenarios are two conflicting worldviews.</p>
<p>In the first one, management views the makers as unmotivated community members who need to be coerced into behaving correctly. They look for rules and enforcing mechanisms to get a change in behaviour. In the second scenario, management assumes that the makers are intrinsically motivated to do good things and be part of a community. They look for ways to enable that behaviour, remove any barriers, and get out of the way.</p>
<p>There are contexts where coercion is in fact the right approach. For example, when it comes to security, the appropriate assumption is that some users are malevolently trying to abuse the system. It also applies to areas like safety and regulatory compliance.</p>
<p>But many cases aren’t as clearcut. When looking for solutions, or when a solution is given to us (as in a feature request), we need to surface which worldviews underlies that solution: Does it treat all problems as technical ones? Does it favour coercion over rewards? Does it assume that users are unwilling participants, or that they are intrinsically motivated?</p>
<p>By making these assumptions explicit, we can evaluate different solutions more thoroughly. Instead of debating the pros and cons of potential solutions based on only their visible aspects, we can debate what worldview is the most appropriate for the current context and what might be some potential consequences of promoting that worldview in our design.</p>
<p><em>Written by <a href="https://twitter.com/mathiasverraes">Mathias Verraes</a> and <a href="https://twitter.com/rebeccawb">Rebecca Wirfs-Brock</a>.</em></p>
<div class="columns mb-6 ">
<div class="column is-6">
<figure class="media-left image " style="width: 75%; box-shadow: 2px 2px 7px darkgray">
<a href="https://leanpub.com/design-and-reality">
<img src="https://d2sofvawe08yqg.cloudfront.net/design-and-reality/s_hero2x?1664874345" />
</a>
</figure>
</div>
<div class="column is-6">
<h1 class="title">Design and Reality</h1>
<h2 class="subtitle">Essays on Software Design</h2>
<p>by Mathias Verraes & Rebecca Wirfs-Brock</p>
<br />
<a class="button is-info" href="https://leanpub.com/design-and-reality">
Get it on Leanpub
</a>
<br />
<br />
<blockquote>
“Happily purchased. Handy to have these in one place. Thank you!” <br />— Kent Beck<br />
</blockquote>
<br />
<blockquote>
“Excellent new thinking on Domain-Driven Design. It's full of real practical experience in getting the most
value from domain modelling. Just like the Eric Evans' DDD book, this gives more insight each time you read
it.”<br />— Ian Russell<br />
</blockquote>
</div>
</div>
EventSourcing Testing Patterns2023-05-28T00:00:00+00:00https://verraes.net/2023/05/eventsourcing-testing-patterns<p>Below are the common patterns I use when writing tests for an EventSourced application. I’m documenting them here because I mention them regularly and I needed a place to point people to. I believe I talk about them in some of my <a href="https://verraes.net/2019/06/talk-temporal-modelling/">Temporal Modelling talks</a>.</p>
<p>Surely these patterns deserve more explanation. <a href="https://twitter.com/mathiasverraes">Tweet</a> me with your questions and I will amend this post.</p>
<h2 id="command-handlers">Command handlers</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Given Events (0 or more)
When Command
Then Events (1 or more)
(example)
Given ItemWasStocked
When PurchaseItem
Then ItemWasPurchased
Then StockWasReduced
</code></pre></div></div>
<p>Sometimes no Event is produced, for example for idempotency, because it’s a legacy Command that no longer requires a result, or any other reason. In those cases, I prefer to make this explicit in my tests:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Given Events (0 or more)
When Command
ThenNothing
</code></pre></div></div>
<p>Commands can fail:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Given Event(s)
When Command
ThenThrow Exception
(example)
Given ItemWasStocked
Given ItemWasPurchased
When PurchaseItem
ThenThrow ItemIsOutOfStock
</code></pre></div></div>
<h2 id="projections">Projections</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Given Events (0 or more)
When Query
Then Response
(example)
Given LocationWasTracked
Given LocationWasTracked
Given LocationWasTracked
WhenAsked FetchWalkingDistance
ThenRespondWith WalkingDistance
</code></pre></div></div>
<p>I like to make Queries explicit, which I’ve writte, about in my post on <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-domain-query/">Domain Queries</a>.</p>
<h2 id="process-managers">Process managers</h2>
<p>Process Manager are reactors, they listen to Event Streams and produce various outcomes.</p>
<p>Process managers can produce new Events. For example, a <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Event</a>, or an interpretation of other Events.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Given Events (1 or more)
Then Event
(example)
Given AddressWasChanged
Given AddressWasChanged
Given AddressWasChanged
Given AddressWasChanged
Given AddressWasChanged
Then PossibleFraudWasDetected
</code></pre></div></div>
<p>A common pattern is that a Process Managers reacts to a <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-passage-of-time-event/">Passage Of Time Event</a>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>(example)
Given Midnight(date=2018-05-28)
Then GDPRRegulationWasActivated
</code></pre></div></div>
<p>Process managers can produce new Commands:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Given Events (1 or more)
Then Command
(example)
Given AddressWasChanged
Given AddressWasChanged
Given AddressWasChanged
Given AddressWasChanged
Given AddressWasChanged
Then BlockAddressChangesForUser
</code></pre></div></div>
<p>This allows to test in one system if it produces the right Commands, and in the other system how it handles Commands.</p>
<p>Process Managers are the perfect place to deal with side effects such as sending emails, doing filesystem operations, dealing with third-party APIs, and any I/O you need. You may want to use mocks, and test whether the actual I/O happens elsewhere.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Given Events (1 or more)
Then I/O
(example)
Given PossibleFraudWasDetected
Then SendEmailToInfosecTeam
</code></pre></div></div>
<h2 id="notes">Notes</h2>
<ul>
<li>The tests are all at the abstraction level of the domain, and use the Ubiquitous Language.</li>
<li>The messages use the <a href="https://verraes.net/2019/06/messaging-patterns-natural-language-message-names/">Natural Language Message Names</a> pattern.</li>
<li>Command Handlers often defer the work to Aggregates, but I rarely test my Aggregates directly. That allows me to change the underlying implementation, without having to change the tests.</li>
<li>In fact, all these tests should change if and only if the domain changes (or if bugs are discovered).</li>
<li>I consider all of these unit tests: they test a single unit of behaviour, a single example in the language of the domain.</li>
</ul>
<h2 id="testing-projection-state-is-an-anti-pattern">Testing Projection State is an anti-pattern</h2>
<p>When testing Projections, the much more commonly used patterns involve testing the state of the Projection, or the changes to the that state:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Given Events
When Event
Then State
Given State
When Event
Then State
(example)
Given Invoice {status = unpaid}
When InvoiceWasPaid
Then Invoice {status = paid}
</code></pre></div></div>
<p>I consider this an anti-pattern when doing domain-level testing. The state of the system is an implementation detail. I conceptualise a system as a black box that receives and produces messages (mostly of the types Commands, Queries, Events). If I’m at this abstraction level, I should only test for the observable behaviour of the system. If you think about it, the client of the system (such as a UI or third-party system) is only interested in whether it gets the right Responses to its Queries, and not in what you do underneath.</p>
<p>It has practical benefits:</p>
<ul>
<li>You want your APIs not to change much. Query tests preserve the contract of the API, but allow you to change the underlying state easily. When I test the state, I couple my domain tests to my implementation choice, which makes the tests more brittle.</li>
<li>I can implement these Query tests using an in-memory eventstore and in-memory state. That means I can do lots of very fast and cheap tests. I can write these tests before writing any code or dealing with databases, so I can use them to explore my domain model and validate it before committing to it. I can also easily explore various edge cases that come up later.</li>
</ul>
<p>Patterns are only anti-patterns in a context: It does make sense to test for state changes when doing integration testing. After I’m happy with the model and the logic, I want to test if my database schema and queries are correct. I also have integration tests for non-functional requirements such as retry behaviour. These tests are slower and more cumbersome. I only want a few of these, to test the integration itself, but the real bulk of tests for the domain behaviour is going to be done with Query tests.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>, and my company <a href="https://aardling.eu">Aardling</a> can help you with this stuff as well.</li>
</ul>
From Music to Languages and Models (DevJourney Podcast)2023-05-16T00:00:00+00:00https://verraes.net/2023/05/devjourney-podcast<p>On the <a href="https://devjourney.info/Guests/253-MathiasVerraes">DevJourney podcast</a>, Tim Bourguignon asked me to talk about my career.</p>
<h2 id="abstract">Abstract</h2>
<blockquote>
<p>Mathias placed the start of his journey with both hands on a TRS-80 computer and no other games to play than a compiler and the BASIC language. But fast forward a few years, we spoke about his music studies, his work creating music for ads and movies, and how programming came back as a hobby, which finally took precedence. We then jumped with both feet into languages and modeling. We spent the rest of the interview talking about interactions and how models serve us programmers and help our domains innovate.</p>
</blockquote>
<p>There’s a <a href="https://devjourney.info/Guests/253-MathiasVerraes">transcript at the DevJourney site</a>.</p>
<iframe allow="autoplay" width="100%" height="300" src="https://www.iheart.com/podcast/256-software-developers-journe-31156226/episode/253-mathias-verraes-from-music-to-115204388/?embed=true" frameborder="0"></iframe>
<ul>
<li><a href="https://devjourney.info/Guests/253-MathiasVerraes">DevJourney</a></li>
<li><a href="https://podcasts.apple.com/us/podcast/253-mathias-verraes-from-music-to-languages-and-models/id1079113167?i=1000613173064">Apple Podcasts</a></li>
<li><a href="https://open.spotify.com/episode/2E5NGxz3rGVzcliuDJk1MM?si=vN_C59_KSNCVUCXK0yFHqQ">Spotify</a></li>
<li><a href="https://listen.stitcher.com/yvap/?af_dp=stitcher://episode/303280382&af_web_dp=https://www.stitcher.com/episode/303280382&deep_link_value=stitcher://episode/303280382">Stitcher</a></li>
<li><a href="https://www.iheart.com/podcast/256-software-developers-journe-31156226/episode/253-mathias-verraes-from-music-to-115204388/?cmp=web_share&embed=true">iHeart</a></li>
<li><a href="https://www.buzzsprout.com/190346/12804392-253-mathias-verraes-from-music-to-languages-and-models.mp3">Podcast Addict</a></li>
<li><a href="https://castbox.fm/channel/1364069?utm_source=website&utm_medium=dlink&utm_campaign=ex_share_ch&utm_content=Software%20Developer's%20Journey-CastBox_FM">CastBox</a></li>
<li><a href="https://www.podchaser.com/podcasts/software-developers-journey-200335">Podchaser</a></li>
<li><a href="https://pca.st/87TE">Pocket Casts</a></li>
<li><a href="https://www.linkedin.com/in/ahmedsamygaber/">PlayerFM</a></li>
<li><a href="https://web.podfriend.com/podcast/software-developers-journey/15496785728">PodFriend</a></li>
</ul>
<h2 id="references">References</h2>
<ul>
<li><a href="https://www.twitter.com/mathiasverraes">https://www.twitter.com/mathiasverraes</a></li>
<li><a href="https://www.linkedin.com/in/mathiasverraes/">https://www.linkedin.com/in/mathiasverraes/</a></li>
<li><a href="https://mastodon.social/@mathiasverraes">https://mastodon.social/@mathiasverraes</a></li>
<li><a href="https://aardling.eu/">https://aardling.eu/</a></li>
<li><a href="https://2023.dddeurope.com/">https://2023.dddeurope.com/</a></li>
</ul>
Parser Combinators (Full Stack Europe)2023-04-20T00:00:00+00:00https://verraes.net/2023/04/parser-combinators-full-stack-europe<p>I was at <a href="https://fullstackeurope.com/">Full Stack Europe 2022</a> in Antwerp, and one of the speakers had to cancel. So I offered to do an improvised talk on Parsers Combinators. (My Covid Lockdown hobby was creating an <a href="https://parsica-php.github.io/">open source library to help you build parsers</a>, so I have some experience.)</p>
<p>A parser is a function that takes some unstructured input and returns something more structured. One way to write them is using a lot of procedural code. But with parser combinators, you can create a parser declaratively by combining small composable parsers into larger ones.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">$parser</span> <span class="o">=</span> <span class="nf">between</span><span class="p">(</span><span class="nf">char</span><span class="p">(</span><span class="s1">'{'</span><span class="p">),</span> <span class="nf">char</span><span class="p">(</span><span class="s1">'}'</span><span class="p">),</span> <span class="nf">atLeastOne</span><span class="p">(</span><span class="nf">alphaChar</span><span class="p">()));</span>
</code></pre></div></div>
<p>In the talk I try to show how these work from first principles.</p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/pHsaf8qdxEE" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<p><a href="https://www.youtube.com/watch?v=pHsaf8qdxEE">Watch on YouTube</a></p>
Bounded Contexts: Manage the Understandability of Your Systems (DDD Australia)2023-03-29T00:00:00+00:00https://verraes.net/2023/03/bounded-contexts-manage-understandability-ddd-australia<p>The new DDD Australia user group just had its first meetup, and I was invited to speak. A major bottleneck for fast high quality software development is that people in the organisation don’t understand their systems and domains well enough. So I did a (mostly improvised) session on using Bounded Contexts to manage meaning and understandability.</p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/_iwKzfY7eFo" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<p><a href="https://youtu.be/_iwKzfY7eFo">Watch on YouTube</a></p>
<h2 id="abstract">Abstract</h2>
<p>From Parnas’ paper in the 1970’s to microservices in the 2010’s, we’ve always used modularisation as a way to manage complexity in software. And yet, we still end up with big balls of mud. Technical separation alone isn’t enough. We’ve also tried separating into business domains, but that’s not enough either: software wants to be deeply interconnected, spanning different domains, and doesn’t respect those boundaries.</p>
<p>Bounded Contexts provide an alternative to splitting on domains or on technical modules. We can separate by looking at the semantics. The domain models that underlie our systems, the language that is being used, and the meanings of the terms. We can draw “understandability boundaries”: separations that look at how concepts in our system are understood together (or can be understood autonomously). If we organise the teams along the same lines, then team members will need to understand fewer concepts to be productive. Teams will need less coordination with other teams. Having better semantic boundaries lowers the cognitive load.</p>
Software Design for Startups and Scaleups2023-02-15T00:00:00+00:00https://verraes.net/2023/02/software-design-scaleups-startups".md"<p>Software Captains is a organisation that helps startup CTOs through coaching, advice, and networking. In the interview, the founder Tom Klaassen and I discuss software design. We talk about challenges CTOs face, such making technical debt visible for non-technical colleagues, about how software quality is contextual, but still very important even for early stage startups.</p>
<p><em>The podcast was recorded in Dutch. On YouTube, you can turn on captions and set the autogenerated translation to your language of choice.</em></p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/1d2DNPqkRQg" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<iframe title="Libsyn Player" style="border: none" src="//html5-player.libsyn.com/embed/episode/id/25921914/height/90/theme/custom/thumbnail/yes/direction/forward/render-playlist/no/custom-color/000000/" height="90" width="100%" scrolling="no" allowfullscreen="" webkitallowfullscreen="" mozallowfullscreen="" oallowfullscreen="" msallowfullscreen=""></iframe>
<p>Or listen here:</p>
<ul>
<li><a href="https://open.spotify.com/episode/3OgdvEWj3CwSwgHHGPFsoS?si=Tcag56SsSR6LFyi6CxEaWw">Spotify</a></li>
<li><a href="https://podcasts.apple.com/be/podcast/mathias-verraes-software-design-voor-startups-en-scaleups/id1564631367?i=1000599476918">Apple</a></li>
<li><a href="https://amzn.eu/d/4L1kPQT">Amazon</a></li>
<li><a href="https://deezer.page.link/bfd8WZhfv6qEyYfy9">Deezer</a></li>
<li><a href="https://softwarecaptains.libsyn.com/website/mathias-verraes-software-design-voor-startups-en-scaleups">Libsyn</a></li>
</ul>
<p>References:</p>
<ul>
<li><a href="https://leanpub.com/design-and-reality">Design & Reality book</a></li>
<li><a href="https://www.softwarecaptains.be/blog/podcast-mathias-verraes-software-design-voor-startups-en-scaleups">Source</a></li>
</ul>
Critically Engaging With Models (DDD Europe Keynote)2023-01-24T00:00:00+00:00https://verraes.net/2023/01/critically-engaging-with-models<p>Here’s the video of Rebecca Wirfs-Brock & my keynote at Domain-Driven Design Europe 2022, where we talk about the ideas presented in our essay <a href="https://verraes.net/2022/09/critically-engaging-with-models/">“Critically Engaging with Models”</a>, also available as an <a href="https://leanpub.com/design-and-reality">e-book on Leanpub</a>.</p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/ecW1jMhIt2k" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<p><a href="https://www.youtube.com/watch?v=ecW1jMhIt2k">Watch on YouTube</a></p>
<h2 id="abstract">Abstract</h2>
<p>Models, whether for a software system, a development process, diseases, political systems, or otherwise, are a way to look at (a part of) the world. They make a choice about what is important, what categories we classify things in, what we see, what’s invisible, what’s valued, or even what’s valid. They are reductionist, that is, they only show a selection of the subject they’re describing. And they are biased: They implicitly reflect the assumptions, constraints, and values of the model’s author.</p>
<p>Most of the time, when you adopt a model created by someone else, you assimilate it into your worldview without much thought. You acquire a new way of seeing something. But when you do that, you may not understand the model’s limitations.</p>
<p>You can choose to look at someone’s model more intentionally. You can critically assess whether this model fits your needs. If you’re looking at a model for the first time, you can use that fresh perspective to see what the model includes and what it leaves out. Models are a powerful lens for perceiving a subject, and we should be deliberate when wielding them. This talk gives you tools for critically evaluating any models that come your way.</p>
<div class="columns mb-6 ">
<div class="column is-6">
<figure class="media-left image " style="width: 75%; box-shadow: 2px 2px 7px darkgray">
<a href="https://leanpub.com/design-and-reality">
<img src="https://d2sofvawe08yqg.cloudfront.net/design-and-reality/s_hero2x?1664874345" />
</a>
</figure>
</div>
<div class="column is-6">
<h1 class="title">Design and Reality</h1>
<h2 class="subtitle">Essays on Software Design</h2>
<p>by Mathias Verraes & Rebecca Wirfs-Brock</p>
<br />
<a class="button is-info" href="https://leanpub.com/design-and-reality">
Get it on Leanpub
</a>
<br />
<br />
<blockquote>
“Happily purchased. Handy to have these in one place. Thank you!” <br />— Kent Beck<br />
</blockquote>
<br />
<blockquote>
“Excellent new thinking on Domain-Driven Design. It's full of real practical experience in getting the most
value from domain modelling. Just like the Eric Evans' DDD book, this gives more insight each time you read
it.”<br />— Ian Russell<br />
</blockquote>
</div>
</div>
Design and Reality (Leanpub Podcast)2023-01-11T00:00:00+00:00https://verraes.net/2023/01/leanpub-podcast-design-reality<p>Rebecca Wirfs-Brock and I <a href="https://leanpub.com/design-and-reality">published a book last year called “Design & Reality”</a> with essays on software design. It’s available on <a href="https://leanpub.com/design-and-reality">Leanpub</a>.</p>
<p>Leanpub co-founder <a href="https://twitter.com/lenepp">Len Epp</a> interviewed me about my background, Domain-Driven Design, the book, the notion that your solution can influence your understanding of the problem, and at the end, we talk a bit about my experience as a self-published author.</p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/KFfljAI5mvc" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<p>Or listen on:</p>
<ul>
<li><a href="https://itunes.apple.com/ca/podcast/leanpub-podcast/id517117137">iTunes</a></li>
<li><a href="https://open.spotify.com/show/00DiOFL9aJPIx8c2ALxUdz?si=oUl7lMp_RtGfXpr8bFGESA">Spotify</a></li>
<li><a href="https://www.stitcher.com/podcast/frontmatter">Stitcher</a></li>
<li><a href="https://podcasts.google.com/?feed=aHR0cHM6Ly9sZWFucHViLmNvbS9wb2RjYXN0LnhtbA&ved=0CAAQ4aUDahcKEwiYiJ275arnAhUAAAAAHQAAAAAQAg&hl=en">Google Podcasts</a></li>
<li><a href="https://player.fm/series/frontmatter">Player FM</a></li>
<li><a href="https://tunein.com/podcasts/Technology-Podcasts/Frontmatter-The-Leanpub-Author-Stories-Podcast-p1138203/">TuneIn</a></li>
<li><a href="https://castbox.fm/channel/Frontmatter%3A-A-Leanpub-Podcast-id966476?country=us">Castbox</a></li>
<li><a href="https://podbay.fm/podcast/517117137">Podbay</a></li>
</ul>
<p>References:</p>
<ul>
<li><a href="https://leanpub.com/design-and-reality">Design & Reality book</a></li>
<li><a href="https://leanpub.com/podcasts/frontmatter/mathias-verraes-10-01-23">Source</a></li>
</ul>
Critically Engaging With Models2022-09-15T00:00:00+00:00https://verraes.net/2022/09/critically-engaging-with-models<div class="has-background-light box">
<p><strong>Contents</strong></p>
<div class="columns">
<div class="column is-half">
<ul>
<li><a href="#models-are-worldviews">Models Are Worldviews</a></li>
<li><a href="#hierarchies-and-networks">Hierarchies and Networks</a></li>
<li><a href="#heuristics">Heuristics</a></li>
<li><a href="#models-distract">Models Distract</a></li>
<li><a href="#the-spotify-model">The Spotify Model</a></li>
<li><a href="#agile-fluency">Agile Fluency</a></li>
<li><a href="#team-topologies">Team Topologies</a></li>
<li><a href="#explicit-building-blocks">Explicit Building Blocks</a></li>
<li><a href="#when-the-team-gets-too-big--a-case-study">When the Team Gets Too Big: a Case Study</a></li>
</ul>
</div>
<div class="column is-half ">
<ul>
<li><a href="#contrasting-spotify-and-team-topologies">Contrasting Spotify and Team Topologies</a></li>
<li><a href="#comparing-models-yourself">Comparing Models Yourself</a></li>
<li><a href="#bossy-models">Bossy Models</a></li>
<li><a href="#getting-too-cosy-with-the-model">Getting Too Cosy with the Model</a></li>
<li><a href="#prescriptive--descriptive--aspirational">Prescriptive, Descriptive, Aspirational</a></li>
<li><a href="#putting-the-model-to-work">Putting the Model to Work</a></li>
<li><a href="#conclusions">Conclusions</a></li>
<li><a href="#footnotes">Footnotes</a></li>
</ul>
</div>
</div>
</div>
<p><em>Note: There’s a <a href="https://verraes.net/2023/01/critically-engaging-with-models/">video of the DDD Europe keynote</a> that we did about this essay.</em></p>
<p>You might be familiar with The Five Stages of Grief aka the Kübler-Ross model for processing grief (denial, anger, bargaining, depression, and acceptance). Independent of what the authors intended, this model gives us a framing right off the bat:</p>
<ul>
<li>The word “Five” suggests a fixed set.<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup></li>
<li>The word “Stages” suggests that they are discrete phases.</li>
<li>It also suggests that these stages come in a fixed order.</li>
<li>The word “The” suggests that you always go through these five stages when grieving.</li>
<li>And it is our impression that people interpret this model as prescriptive: in order to process grief, you <em>must</em> go through these stages in order.</li>
</ul>
<p>We’re not interested here in whether these statements are accurate or not. Perhaps the authors’ wording was accidental, and it could just as well have been named “Various Feelings of Grief”. We’re interested in how the language suggests ordered stages, and that’s how most people will perceive it.</p>
<p>When a model puts us in a certain framing, how can we tell? How do we understand what framing a model imposes on us? How do we engage with any model? How can we evaluate it? Should we break out of that framing? Do we accept the model as is? Or do we use it as scaffolding for finding better models, that are better suited to our situation?</p>
<h2 id="models-are-worldviews">Models Are Worldviews</h2>
<p>Models, whether for a software system, a development process, diseases, political systems, or otherwise, are a way to look at (a part of) the world. They explain how something behaves and how to modify that behaviour. Every model makes a choice about what is important, what categories we classify things in, what we see, what’s invisible, what’s valued, or even what’s valid. Models are reductionist; that is, they only show a selection of the subject they’re describing, and lose something in the process. And models are biased: they implicitly reflect the assumptions, constraints, and values of the model’s author.</p>
<p>Most of the time, when you adopt a model created by someone else, you assimilate it into your worldview without much thought. You acquire a new way of seeing something and accept it. It’s how learning works. However, when you do that, you may not understand the model’s limitations.</p>
<p>Models mess with you. They impose a distinct perspective, and a set of rules for operating within them. They frame how you look at your problem. And you’re usually not aware of this perspective and framing.</p>
<p>But you can choose to look at a model more intentionally. If you’re looking at a model for the first time, you can use your fresh perspective to see what it includes and what it leaves out. You can critically assess whether that model fits your view and your needs. Models are a powerful lens for perceiving a subject, and you should be deliberate when wielding them.</p>
<p>We’d like to share some tools for critically evaluating any models that come your way. We’ll do this by example. We have picked a number of organisational models to illustrate how to examine them and see whether they offer what you need to solve your problems. First, we’ll discuss the hierarchical, social network, and the value creation models. Then, we’ll examine three organisational models that are specific to software development: the Spotify Model, the Agile Fluency Model, and Team Topologies.</p>
<p>Our goal is not to voice an opinion one way or the other about these different models, or to suggest which ones to use for your organisation. Our interpretation of these models is by necessity brief. If you are familiar with any of them, you might feel we do them a great injustice by our summarizations. We ask that you look past the specifics of the organisational models we examine, and instead focus on the methods we demonstrate for evaluating any model, in any context.</p>
<p>Here we use well-known published models to illustrate approaches for understanding and comparing them. But these techniques apply equally well to various models that you may encounter or create, such as domain models, working agreements, business processes, or politics.</p>
<h2 id="hierarchies-and-networks">Hierarchies and Networks</h2>
<p>A traditional way of looking at organisations is the hierarchical model. In this model, power is concentrated at the “top” and authority is structured into levels. People talk about “going up the chain of command” in order to reach the “right person” in the hierarchy who has the authority to make a decision. People have often mistaken this model for the whole: “an organisation <em>is</em> a hierarchy”. In reality, the model only describes one aspect of an organisation.</p>
<p>It’s often by looking at alternative models, that we see the limitations of this hierarchical model. For example, a competing model says: “an organisation consists of a hierarchical control structure <em>and</em> a social network”. The social network is who you talk to, and how you informally acquire information you learn at the coffee machine or over lunch. This model is an improvement over the hierarchical model: it shows us that human interactions can affect the functions of the organisation. A manager might want to encourage their organisation’s social network, or change the dynamics to improve opportunities for informal interactions. They’ll try to improve the functions of the organisation, and make sure the social network doesn’t hinder it.</p>
<p>In our role of model evaluators, this teaches us that an organisational model can have two networks operating simultaneously, each representing a different function of the organisation. That’s significant, because if our organisation has two networks, it is possible to have more.</p>
<blockquote>
<p><strong>Heuristic: If there are two ways of looking at something, look for a third.</strong></p>
</blockquote>
<p>Another model augments these two organisational networks with a third network: the value creation network. This is the network you turn to to produce meaningful outcomes. It’s knowing which people to ask when you need something done, who the experts are, who can execute it. By modelling our organisation now as a hierarchy, a social network, <em>and</em> a value creation network, we’re challenging the assumptions made by the previous two models. The new model says: “value creation doesn’t happen through the same channels as control or gossip”.</p>
<p>Before we adopted this third organisational model, there were two possibilities: either value creation was not on our radar at all, or value creation was a responsibility of one of the other two networks (most likely the hierarchy).</p>
<p>When we accept this third model, we believe that value creation:</p>
<ul>
<li>is important enough to be recognized as an organisational responsibility;</li>
<li>doesn’t happen in a hierarchy or social network;</li>
<li>it happens in its own network.</li>
</ul>
<p>This new model doesn’t simply add a third network. The new model supplants the prior belief that value is created in the hierarchy, with the belief that it is created through a value creation network. Through the addition of the third network, the other two networks in the model have acquired a new purpose. The third network adjusts your perception of the other networks.</p>
<p>Say a manager has been mandating that all communications and decision-making happens along the lines of the hierarchy. Now, as they accept the new model, they change how they operate, and instead allow and encourage social and value creation networks to thrive. The manager will no longer demand that all communications take a certain shape, and will choose not the exert control over all aspects of communication.</p>
<p>None of these three models capture reality. Instead, they capture beliefs about reality that, in this case, drives business decision-making. Each model alters the way we perceive reality, and how we take action based on that perception. By accepting any model, we accept the belief system that comes along with it, and we change our behaviour accordingly.</p>
<h2 id="heuristics">Heuristics</h2>
<p>From our short analysis of three organisational models, we can derive several heuristics for helping to evaluate any organisational model:</p>
<blockquote>
<p><strong>Heuristic: Compare different models to figure out what one model adds or omits, emphasizes, or downplays.</strong></p>
</blockquote>
<p>That is, use one model to discover if there are any gaps in the other. Look for ways one model might give a lot of attention to one aspect and not to another.</p>
<blockquote>
<p><strong>Heuristic: Understand the underlying belief system that comes with the model.</strong></p>
</blockquote>
<p>If I accept this model, how does it change my belief about part of the world that this model addresses?</p>
<blockquote>
<p><strong>Heuristic: Determine whether the model confirms my existing belief system and values, or conflicts with them.</strong></p>
</blockquote>
<p>Am I choosing models based on how well they conform to something I already believe?</p>
<blockquote>
<p><strong>Heuristic: Understand whether the model addresses or solves a problem that I’m interested in solving.</strong></p>
</blockquote>
<p>It may be an appealing model, but do I really need it?</p>
<blockquote>
<p><strong>Heuristic: Ascertain whether the model points me to problems I might have but haven’t yet considered.</strong></p>
</blockquote>
<p>We find these heuristics are useful for critically evaluating any model, not just organisational models. In this text, however, we’ll stick to organisational models.</p>
<h2 id="models-distract">Models Distract</h2>
<p>As we have shown, a model creates a lens, a way of looking at the problem. Additionally, models often come with a richer set of guidelines or instruction sets, that tell you how to use them.</p>
<p>The organisational hierarchy model, for example, comes with advice on how to achieve a more desirable organisational structure, such as “Don’t have more than 6 levels”, or “Have between 5-30 direct reports per manager”, or “When your organisation gets too big, split it along these functions”. These guidelines are congruent with the building blocks of the hierarchical model and with the belief system that underlies it.</p>
<p>Guidelines are useful for implementing this model. Whenever you embrace a model, its instructions lead you to considerations that it views as important. It focuses you on the changes that it wants you to make. It’s saying “this is an important thing to worry about”. The things it finds important have names: the levels, the functions, the reports, … Later, we’ll look at models that don’t include names for levels and reports, but do include names for teams, value streams, fluency, or evolution. The choices of things a model names are what puts us in a frame.</p>
<p>But by discussing “How many levels do we need?” we’re distracted from asking the deeper question of “Do we need levels at all?”. Because the hierarchy model comes with a name for levels, it imposes the need for levels.</p>
<p>The methods and tools that come with any model are insidious in that sense. The presuppositions that a model makes are:</p>
<ul>
<li>The model is adequate (an organisation is indeed a hierarchy);</li>
<li>The tools and techniques are adequate (choosing the right levels is the right solution for organisational problems);</li>
<li>If you fail, you haven’t applied them well (choosing not enough or too many levels).</li>
</ul>
<p>Models rarely include a technique for determining whether the model fits your environment.</p>
<p>If we look at the flat organisational model, we see that it also makes some presuppositions (which conflict with the hierarchy model):</p>
<ul>
<li>The model is adequate (an organisation should indeed be flatter);</li>
<li>The tools and techniques are adequate (removing middle management is the right solution to improve your organisation);</li>
<li>If you fail, you haven’t applied them well (you’ve allowed too much of the old hierarchy to survive).</li>
</ul>
<p>But, again, if we only discuss techniques for removing middle management, we can forget to ask if any functions of middle management are not being handled with a flat organization model. The flat model doesn’t tell us to look for valid functions of middle management, and it doesn’t provide a pattern language for understanding these functions.</p>
<p>Models help you ask questions, but not necessarily the right ones. If we compare different models however, we can take the questions from one model, and ask them in the context of another. By looking at their difference answers, we can see where they diverge. And in the case of the hierarchy and flat organisational models, we can see they have diametrically opposing world views. Let’s look at some software development specific organisational models next.</p>
<h2 id="the-spotify-model">The Spotify Model</h2>
<p>A popular model for structuring organisations is the Spotify Model.<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup> Again, this model introduces elements not found in previous models we examined. Roughly, the model consists of Squads (cross-functional autonomous teams that are responsible for a discrete part of the product, led by a triumvirate of a Product Owner, Tribe Lead, and Agile Coach), Tribes (for coordinating multiple Squads), a Tribe lead that coordinates with other Tribes, and Alliances (for collaboration across Tribes). Then you have Chapters (to deal with shared interests, such as standardisation and preferred practices in a field). Finally, a Guild is an ad hoc grouping of people interested in learning a specific topic.</p>
<p>Although the language is different from our previous models, there’s still a clear sense of hierarchy in this model, but it’s fairly shallow. This probably explains why it’s a popular model: it appeals to managers who believe some form of hierarchy is critical to delivering anything, but it doesn’t emphasize hierarchy at all.</p>
<p>The interesting thing about this model is that it introduces two styles of learning into organisational models: one for broad institutional learning (Chapters grow the skills of the organisation as a whole and capture that in standards and common practices), and another for individual upskilling (Guilds). Both are knowledge creation networks. All organisations have a need for learning, of course. But by explicitly naming Chapters and Guilds, the Spotify Model is saying that learning is important enough that you should deliberately structure your organisation to support it. It says that creating the knowledge network should be on people’s radar as a distinct responsibility, not just an accidental by-product. Underlying this model is the belief that knowledge work requires structural support.</p>
<p>By adopting the Spotify Model, you’re accepting that you need some hierarchy, but that it should be limited; and that besides value creation, individual and institutional learning are critical problems to organise for.</p>
<p>When you first learn about the Spotify Model, you accept implicitly that learning is an important concern. But if instead of merely accepting this model, you actively compare it to other models, you can ask: “Why is it that value creation and knowledge are considered important? Which other aspects does this model exclude, and why?” Comparing models makes it easier to see what each model brings to the table.</p>
<p>How do other models solve for growing organisational knowledge? For example, some organisations introduce Centers of Excellence. These are standalone teams with dedicated skilled members. This is different from Chapters in the Spotify Model, where the members of a Chapter are also members of different Tribes. Because we are aware now that we can organise for knowledge, we can actively seek out other models, and compare how they address it.</p>
<h2 id="agile-fluency">Agile Fluency</h2>
<p>Let’s take a quick look at another model which is focused on learning. Agile Fluency is an organisational model that represents teams and organizations as being in different stages of their progress in becoming agile. These stages are defined by the activities they do to improve their capabilities and value to the business.</p>
<p><img src="/img/posts/2022-09-15-critically-engaging-with-models/agile-fluency.png" alt="Agile Fluency" title="Agile Fluency" /></p>
<div class="table-container">
<table class="table">
<thead>
<tr>
<th>Zone</th>
<th>Benefit</th>
<th>Investment</th>
<th>Learn From</th>
<th>Time to Fluency</th>
</tr>
</thead>
<tbody>
<tr>
<th>Focusing</th>
<td>Greater visibility into teams’ work; ability to redirect.</td>
<td>Team development and work process design.</td>
<td>"Scrum, Kanban, non-technical XP"</td>
<td>2-6 months</td>
</tr>
<tr>
<th>Delivering</th>
<td>Low defects and high productivity.</td>
<td>Lowered productivity during technical skill development.</td>
<td>"Extreme Programming, DevOps movement"</td>
<td>+3-24 months</td>
</tr>
<tr>
<th>Optimizing</th>
<td>Higher-value deliveries and better product decisions.</td>
<td>Social capital expended on moving business decisions and expertise into team.</td>
<td>"Lean Software Development, Lean Startup, Beyond Budgeting"</td>
<td>+1-5 years</td>
</tr>
<tr>
<th>Strengthening</th>
<td>Cross-team learning and better organizational decisions.</td>
<td>Time and risk in developing new approaches to managing the organization.</td>
<td>Organization design and complexity theories</td>
<td>unknown</td>
</tr>
</tbody>
</table>
</div>
<p><small>Image and table: Diana Larsen and James Shore, agilefluency.org</small></p>
<p>The picture alone doesn’t do Agile Fluency justice of course. The underlying assumption of Agile Fluency is that when your organisation isn’t producing enough value, you can solve this problem by first improving individual teams’ capabilities and ways of working, and then restructuring the organisation itself.</p>
<p>Like the Spotify Model, this model addresses learning as an explicit organisational need (as opposed to something you tack on). Unlike the Spotify Model, however, Agile Fluency is not a model for structuring your organisation. It does recommend that you address your organisational structure, but leaves that to other models. The scope of the Agile Fluency Model is about agile culture and skills.</p>
<p>What this model does introduce is the idea that teams, and therefore the organisation, progress over time. This notion of progression is not present (or in any case it is not very obvious) in the previous models we have discussed. It’s not that those models force you into stasis; you can always change things around. But the previous models don’t have the building blocks for representing progressive change. You’re just supposed to change when needed, with no support from the model, and no language for explicitly reasoning about change.</p>
<p>In Agile Fluency, you can impact progress by choosing the appropriate interventions (focusing, delivering, optimizing, strengthening) at the right time, in the right order. The model’s worldview is that doing these activities out of order is not going to yield the results you’re looking for, because capabilities depend on previously acquired skills.</p>
<p>Now that we understand how the building blocks of Agile Fluency represent progress (instead of structure), we can use this view to compare it to other models.</p>
<p>The Spotify Model allows for evolution (you can split up teams and reorganize them), it allows for learning and encourages it by introducing structure for learning. But it stops short at explaining how to use that learning to progress. What should progress or evolution in the hierarchy model, or in the Spotify Model, or any other look like? The answer you come up with, leads to the next question: do I care about progress in my context? Maybe your situation doesn’t call for progression. If it does, you’ve learned something about the gaps and assumptions of the model you use.</p>
<blockquote>
<p><strong>Heuristic: Compare different models to understand what you actually care about.</strong></p>
</blockquote>
<blockquote>
<p><strong>Heuristic: If a model doesn’t explicitly address something you value highly, try fitting in something from another model and see if the model still hangs together.</strong></p>
</blockquote>
<p>Another interesting aspect of the Agile Fluency model is that it has an aspirational aspect, and admits this openly. True fluency playing a musical instrument is achieved over a very long time, and after a lifetime you can still learn more. Similarly, with the Agile Fluency model there is no achievable end state, but you should still strive to get in the strengthening zone. It is worth increasing your skills and capabilities, even if you never obtain all its benefits.</p>
<h2 id="team-topologies">Team Topologies</h2>
<p><img src="/img/posts/2022-09-15-critically-engaging-with-models/Team_Topologies.png" alt="Team Topologies" /></p>
<p><small>Image: Henny Portman</small></p>
<p>Team Topologies is a software organisational model that focuses on fast flow and value creation. It advocates composing your engineering organisation from four types of teams and three types of team interactions. You should aligned teams on a common purpose, and reduce their cognitive load, so they can be efficient and focused. This model explicitly values reducing unnecessary cognitive load. The patterns (or topologies as they call them) are Stream-Aligned Teams, Enabling Teams, Complicated Subsystem Teams, and Platform Teams. These teams interact using three communication “modes”: Collaboration, Facilitating, and X-as-a-Service.</p>
<p>When you have teams that don’t map directly into these patterns, they “should either be dissolved, with the work going into stream-aligned teams or converted into another team type … [and] the team should adopt the corresponding team behaviours and interaction modes.”<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup> Of course there’s more to Team Topologies than our summary, but team types and communication styles are the building blocks of the model.</p>
<p>With Team Topologies, there isn’t a static structure with fixed hierarchies as in the Spotify Model. Instead, you combine the team types and interactions to suit your business goals in order to create value. Another difference is that the Spotify Model values team autonomy so much that it doesn’t directly address cross-team collaboration. Team Topologies advocates autonomy as well, but includes collaboration building blocks. However, it advises to keep interactions between teams tightly constrained.</p>
<p>What’s particularly interesting about Team Topologies in our opinion, is that it defines a set of composable patterns. As an implementer of Team Topologies, you are expected to combine these patterns in ways that optimise for flow in your context. Not only that, but you should expect this structure to evolve. When you sense that your context has evolved, you are expected reshape your organisation’s team structure by reapplying these patterns. If this sounds like the design pattern catalogs that we have in software design, it’s because it is very much inspired by that idea.</p>
<p><img src="/img/posts/2022-09-15-critically-engaging-with-models/Evolution_of_Team_Topologies.png" alt="Evolution of Team Topologies" /></p>
<p><small>Image: Henny Portman</small></p>
<p>Like software design patterns, Team Topologies even comes with some anti-patterns, such as team structures and communication styles to avoid. Unlike pattern catalogs, which typically are open-ended (or at least admit that they are incomplete), Team Topologies doesn’t encourage us to extend the model with new team types and interaction modes.</p>
<p>In our context of evaluating organisational models, using patterns and a catalog of patterns that can be assembled to structure an organisation is a powerful idea worth exploring. Perhaps the other models would be better served with such a pattern language. Future models could take this approach even further and create pattern languages for other kinds of organisations that aren’t doing software engineering.</p>
<p>Team Topologies is rooted in the DevOps-philosophy. Although it seems to claim to be suitable for all software organisations, it only addresses product-centric organisations. We don’t feel it is suitable for systems like life-critical medical devices or railway infrastructure, because it doesn’t have building blocks that address reliability or safety. It is biased towards this premise: create value and enable flow in product-oriented software organisations. It proposes a small, specific set of building blocks as the means to achieve that. If we wanted to, say, organise for human safety over fast flow, we’d need to come up with a different set of patterns for doing so. If we organised for software quality, correctness, worker wellbeing, learning, or domain understanding, our choices will be different again. Optimising for flow may ignore other important aspects of systems that you may need to consider.</p>
<blockquote>
<p><strong>Heuristic: All models are created within a specific context. Does the original context match yours? Can it be adapted?</strong></p>
</blockquote>
<p>You could fit a need for considering human safety into Team Topologies, by moving the responsibility for overall safety into a stream-aligned team, having an enabling team to support them, and a platform team to build the necessary infrastructure. But that is not the same as the model being explicitly designed to enable safety. None of Team Topologies’ defined building blocks suggest that a specific team type or interaction mode serves to enable safety. Likewise, nothing in Team Topologies explicitly addresses learning: you can fit learning in there (the book suggests that some enabling teams are really like chapters and guilds from the Spotify Model), but nothing in the model explicitly supports learning. Team Topologies doesn’t have a rich model and language for how teams learn, as exists in the Agile Fluency model.</p>
<p>These omissions are not a problem with Team Topologies per se, but rather a consequence of its focus. Team Topologies has value creation and flow as its scope. By presenting only building blocks for managing flow, it frames flow as the central concern worth organising for. Subsequently, it does not include explicit building blocks for learning or safety or programmer happiness. These omissions should be no surprise.</p>
<p>A problem arises, however, if we adopt the Team Topologies model, with its building blocks and patterns and language —thinking it will solve our organisational problem— without first asking whether improving flow is indeed our most pressing need, and whether we have other needs. If we ask such questions, we can avoid the Golden Hammer Syndrome. Furthermore, by comparing models we can figure out what we need from a model and what its benefits and limitations are.</p>
<h2 id="explicit-building-blocks">Explicit Building Blocks</h2>
<p>Building blocks are explicit, named concepts in a model. They’re not the focus point of the model (like “control” in the hierarchy), but they’re the elements the model uses to create that focus (like the “levels” in the hierarchy).</p>
<p>A building block is a lightning rod. It predictably focuses attention and energy. The things you find most important should be first class building blocks in the model you’re introducing.</p>
<blockquote>
<p><strong>Heuristic: If you care about something, have a building block for it.</strong></p>
</blockquote>
<p>When you have a building block for something important, it draws people’s attention. Simply by existing, that building block tells people that this is something that requires their consideration. If, on the other hand, there is no building block for it, people might still give it attention, but there is no home for it. Any attention given could be spotty, brittle, temporary or even deemed unwelcome.</p>
<p>Most of the time, when you introduce a new model, you’ve spent the time understand it and buy into all it has to offer. But others will not understand it as deeply as you do, nor will they apply it as rigourously as you do. You may understand where the model can be extended or adapted, but others will apply the model as is. So even if you can make the model work for your context by combining its building blocks with your adaptations, it doesn’t mean others will. They are more likely to do it by the book (or by following your instructions). Adapting the models and their building blocks to your needs, gives you influence over people’s attention, and therefore over the outcomes they produce. If you don’t adapt these models, bringing focus to what you value, will be much harder.</p>
<p>If, say, you’re reorganising and you want to improve how institutional learning happens in your organisation, you can either pick a model that explicitly addresses it with its building blocks (for example, Chapters and Guilds in the Spotify Model), or you can adapt another model to suit your needs. It’s not that this learning won’t happen in, say, a hierarchy or Team Topologies. But without the attention-grabbing power of a good, explicit, and well-explained set of building blocks, there’s not much that tells others what to do and how to do it.</p>
<p>In all knowledge organisations, individual and institutional learning is important. If you’re looking to apply an organisational model, you can ask how they deal with learning, and whether that’s important to you. If learning is happening smoothly, you don’t need to touch it. But if you feel it requires change, you need to have learning explicit in the model.</p>
<p>If instead of improving learning or flow, you want to improve the quality of delivery, or the safety of life-critical systems, your organisational building blocks would need to express that, whether they’re blocks for types of teams, interactions, activities, meetings, or reports.</p>
<h2 id="when-the-team-gets-too-big-a-case-study">When the Team Gets Too Big: a Case Study</h2>
<p>Models guide our choices for appropriate actions to take. As an example, let’s ask what we should do when a team gets too big. We’ll compare what different models suggest we do, to see what they teach us about their approach, and their underlying value systems.</p>
<blockquote>
<p><strong>Heuristic: When you’re evaluating models, find questions that are relevant in your context, and use them to compare the models.</strong></p>
</blockquote>
<p>In a traditional hierarchy, when a branch has too many people, you add more subbranches and a corresponding layer of management to manage the people who report to them. At some point, when a hierarchy gets too deep, you make it wider: you split it apart by adding departments or business units with their own management, and allow these to each have their own hierarchies. The value system behind hierarchy is control. It assumes that to scale control, you need to delegate control and control the controllers; so the idea of growing deeper or wider is embedded in the model.</p>
<p>From Jason Yip<sup id="fnref:4" role="doc-noteref"><a href="#fn:4" class="footnote" rel="footnote">4</a></sup>, a Senior Agile Coach at Spotify, we get some clues on how Spotify deals with splitting squads or tribes.</p>
<p>“You wait for seams to appear: clunkiness in communication flow and interaction patterns. (…) Then you nudge things apart. Maybe a subgroup of people has a slightly different rhythm and events. Nothing formal, just nudging things apart. Over time you notice, hey, I’m not interacting with the other people in the team anyway. So you just formalise that [split]. If done correctly, this is mostly an acknowledgment and a non-event.” Jason calls this “organic organisational design”. In that model, structure should always follow strategy.</p>
<p>In Team Topologies, when the flow of a Stream-Aligned Team slows, or the team is experiencing increased cognitive load, then it’s grown too big. To fix this you would look for multiple value streams hiding in there, and split the team according to those lines. You could also look for people working on a highly specialised aspect of the value stream, and move them to a separate Complicated Subsystem Team.</p>
<p>When a Platform Team gets too big, Team Topologies suggests that the topologies are fractal. The Platform Team acquires internal topologies, consisting of internal Stream-Aligned Teams, Complicated Subsystem Teams, and Enabling Teams.</p>
<p>The Spotify Model makes no mention of Platform teams, but how would it solve the problem of a Platform Team getting too big? We think that you would create a Chapter that oversees and supports multiple Platform Teams, helping them with standardisation and tooling, but you wouldn’t have them impose too much. This solution is different from a Team Topologies’ Enabling Team, because the Chapter is composed of people from across the different Platform Teams.</p>
<h2 id="contrasting-spotify-and-team-topologies">Contrasting Spotify and Team Topologies</h2>
<p>Interestingly, Team Topologies has no building blocks for any kind of group that spans people from different teams. Of course, a Team Topologies organisation could have such groups, but as a model it doesn’t tell us to have them, doesn’t tell us how to organise them, or even to recognize that such groups exist.</p>
<p>In the same sense, Tribes are another key difference between the model at Spotify and Team Topologies. Tribes are larger structures than squads. A Tribe manages strategy and goal-setting at a higher level. According to Jason Yip, Squads are less important to preserve than Tribes. Tribes can evolve fast and react organically to change. Team Topologies puts teams first, suggesting to make them long-living and seeing teams as the central unit of organisation. Teams are the only tool for looking at organisations, so there’s no natural place for organising in terms of business strategy. You organise for flow.</p>
<p>Another fundamental difference between these two models is that the Spotify Model doesn’t mention cognitive load. Instead, it talks about clunkiness, friction, long meetings, … The term “cognitive load” doesn’t show up in its vocabulary. By naming cognitive load, Team Topologies calls attention to it, telling us explicitly to look for and reduce it.</p>
<p>Team Topologies and the Spotify Model also perceive interaction between teams differently. The Spotify Model assumes interactions happen organically inside Squads, between Squads in Tribes, and cross-cutting though Chapters and Guilds.</p>
<p>Team Topologies sees too much interaction as an inhibitor to fast flow. Consequently, interactions need to be designed explicitly. Teams are supposed to be limited to as a few as possible interactions and interaction styles. Ideally, these interactions are expressed in a “team API”: a set of well-defined places where communication happens, such as pull requests, issue trackers, or chat channels.</p>
<p>Finally, when an interaction doesn’t work well in the Spotify Model, an Agile Coach will intervene. That Agile Coach is an explicit building block in the Spotify Model. In Team Topologies, there’s no building blocks for coaches, managers, or other roles that can intervene. (Possibly, Team Topologies wants teams to self-organise, but there’s also no mention of that option.) Again, lack of an explicit building block doesn’t preclude you from having manager roles in Team Topologies, but you won’t get any support for them from the model. Team Topologies doesn’t tell you to focus on such a role or the value it might bring to the organisation.</p>
<h2 id="comparing-models-yourself">Comparing Models Yourself</h2>
<p>We encourage you to compare models to evaluate them. As we have shown, one way to compare models is to ask questions about how those different models might handle a particular problem you are interested in solving. To get started, pick situations (real or imagined) relevant to your environment and test them against the models. Then, try to imagine how each model would handle them.</p>
<p>This is speculative: during this evaluation you’re taking a theoretical view of how the models work and how they stack up against each other. That will never beat the insights you can get from practical experience applying the models. But it’s much cheaper to test-drive a model through realistic scenarios, than it is to put it into practice only to uncover big surprises a year down the road. Besides the measurable costs, reorganisations lead to “reorg fatigue”. Likewise, models that address aspects other than organisations might have varying costs. Running thought experiments with scenarios this way teaches you something about the value system of each model early on. It also exposes whether the model cares about the same things you find important.</p>
<p>As you run various scenarios, you’ll notice that:</p>
<ul>
<li>some models may give very detailed advice (eg “split a team when it exceeds 10 members”);</li>
<li>others offer only vague guidance (eg. “teams shouldn’t be too big”, without quantifying what is too big or what should be done);</li>
<li>some models have nothing to say about that specific situation.</li>
</ul>
<p>None of these are bad things. All models include and omit different things, and give more practical or more broad advice. But when applying model you need to understand who you’re getting married to.</p>
<p>Now that you have some ideas on how to compare models, we’d like to give you some more clues. First, we’ll discuss how some models (or their authors) are authoritative. They provide one way, and aren’t open to being extended. Then, we’ll discuss how our own cognitive biases can influence our perception of models. Finally, we’ll look into how models are intended to be used: as observational tools, or as prescriptions of how to act, or even as aspirational guidelines for a future state.</p>
<h2 id="bossy-models">Bossy Models</h2>
<p>You might be familiar with SOLID. It’s a set of five object-oriented software design principles. The first letter of each principle forms the word SOLID. For example the Single Responsibility Principle is the “S” in SOLID. This model gives us a framing right off the bat:</p>
<ul>
<li>There are only five design principles worth caring about.</li>
<li>It’s these five that matter, and not any others.</li>
<li>They apply to all software design, independent of context.</li>
<li>You’re not just supposed to apply these principles in an ad hoc fashion, but you’re supposed to apply them at all times.</li>
<li>The name implies that by applying them, your design will in fact be solid, which is a good thing.</li>
</ul>
<p>As before, we want to break out of the framing of this model, so that we can understand it better. Here are some questions we ask: Why five? Why these five and not others? What if a sixth principle is discovered? In what context were these principles formulated? Do they apply to the design of all kinds of software in all contexts? Are they as true now as they were 25 years ago? Has software design changed since then?</p>
<p>As evidenced by their framing, it’s clear that the authors intended for this model to be authoritative. As an adopter of this model, you’re not supposed to extend it, or challenge it. (And many of its adopters seem to buy into this framing.)</p>
<p>In reality, there are dozens (if not hundreds) of design principles that have been discovered, long before and long after SOLID. And yet, the SOLID model didn’t adapt. Even if you wanted to evolve it, SOLID doesn’t offer any hooks for extensibility.</p>
<p>The Hierarchical organisation model is another example of a definitive model. The building blocks are the hierarchy, where decisions made at the correct level are pushed down, and the reporting goes up the hierarchy chain. There’s no room in the scope of the model to extend it. Extend the hierarchy differently, altering decision-making, or changing the reporting structure, doesn’t fit in the belief system that underlies this model.</p>
<p>Some models are authoritative and definitive, while others are open to being extended. But models rarely make this distinction explicit. They don’t tell you whether to extend them and in what ways. It’s hidden in the language they use, the names and the structures they propose, and in the communications of the authors about the model.</p>
<p>Often, all you need is a straightforward model to tell you how to do something. In this case, the model’s simple building blocks are an <em>enabling constraint</em>: they help you to see something clearly, and make progress faster. In other cases, an authoritative model that seems suitable at first sight, can become a <em>limiting constraint</em> over time. In this case, on top of dealing with your original problem, you’re now also faced with problems caused by the limitations imposed by the model. You start force-fitting things into the building blocks, instead of using the building blocks to achieve your goals. Force-fitting causes you to lose essential components of your context because they don’t fit the model. Inversely, it also causes you to overemphasize unimportant components of your context because the model gives them focus. The metaphors that the model uses, can distort your perception of your context.</p>
<p>Finding alternative models, again, can help us out of this problem. In the case of the Hierarchical model, it’s the Social Network and the Value Creation Network models that break us out of constraints. They tell us to stop accepting the framing of the Hierarchy.</p>
<blockquote>
<p><strong>Heuristic: Are you fitting the model to your context, or are you force-fitting your context to the model?</strong></p>
</blockquote>
<h2 id="getting-too-cosy-with-the-model">Getting Too Cosy with the Model</h2>
<p>When we are faced with something new, like a model, we are initially inclined to question it. But we want to believe. We like certainty. So as soon as we’re comfortable with the model, we stop questioning. Being in a state of cognitive ease makes us more accepting of new information or ideas.</p>
<p>Successful models tend to have some characteristics that contribute to our cognitive ease. They have a small number of building blocks clearly presented (perhaps even simplistically so), usually 7 or less. There’s symmetry, balance, and the structures are often repeating. Strangely, these characteristics make us believe the model is more sound, more rigorous, better thought out. It may well be, however, that the model’s structure has been simplified to be symmetrical, balanced, repetitive, and slim. We associate feeling at ease with goodness. Visualisations that fit our sense of “good” structure also help sell a model: quadrants, pyramids, funnels, radars, Venn diagrams with three neatly intersecting sets.<sup id="fnref:5" role="doc-noteref"><a href="#fn:5" class="footnote" rel="footnote">5</a></sup></p>
<p>Anecdotes are another tool for putting us at ease. If something is explained to us through stories, we’re more likely to accept it. We trust the story that a familiar face tells us more than when someone gives us critical facts. When a model comes to us with anecdotes of how it contributed to a successful outcome, we don’t question the story.</p>
<p>When we’re at cognitive ease, we trust our intuitions. We narrowly frame our problems to fit the models we’re comfortable with. We become overconfident.</p>
<p>The remedy to cognitive ease is a healthy dose of skepticism, aimed not just at new models, but at our existing models as well. Skeptics examine things, even if this makes them uncomfortable. They let go of certainty in order to see more of what the problem really is and what the model has to offer. We use “getting out of your comfort zone” for social situations or challenging tasks, but it applies to our inner world as well.</p>
<blockquote>
<p><strong>Heuristic: Be more skeptical of nice models, be more open minded to jarring models.</strong></p>
</blockquote>
<p>Be on the lookout for things that don’t fit, that are slightly awkward, that break the nice structure of the model you’re using. Allow yourself to entertain multiple competing models at the same time, and to context-switch between them liberally. This gives you a certain freedom: no longer boxed in by a single model, you can come to see the models for what they are: potentially useful worldviews, not “truths”.</p>
<blockquote>
<p><strong>Heuristic: Models are potentially useful worldviews, not truths.</strong></p>
</blockquote>
<h2 id="prescriptive-descriptive-aspirational">Prescriptive, Descriptive, Aspirational</h2>
<p>In the 16th century, the word network referred to wires or threads arranged in a fishing net like construct. In the 18th century, the term network was adopted to first represent railroads and canals, by the 1940’s it meant radio broadcasting, then linked computers in the 1970’s, and finally, in the 1980’s, it acquired the meaning of building human relationships. Our modern use of the word is a metaphor, with extra steps.</p>
<p>The model that introduced the social network inside organisations is descriptive. Someone observed their environment, and noticed a pattern of informal communications. They then mapped this pattern to the metaphor of the network. Using the network metaphor, they found a shortcut to describe these real-world patterns. Perhaps you have seen these informal communications as well, but you had never truly noticed them. But then, when someone points out to you that it is a social network, a lightbulb goes on. You now have a language for it.</p>
<p>A descriptive model doesn’t have an intention, it only means to clarify, and help you see something in a new light. This change of perspective does have consequences: you might change your behaviour, for example, by creating opportunities for people in your organisation to mingle socially.</p>
<p>A prescriptive model, on the other hand, deliberately aims to change your behaviour. The Spotify Model clearly has this intention. It’s telling us that our situation will improve if we restructure our organisation according to the prescribed building blocks. The Spotify Model originated as a descriptive model of how people organised, before it was turned into a prescriptive model for organisations by others later.<sup id="fnref:6" role="doc-noteref"><a href="#fn:6" class="footnote" rel="footnote">6</a></sup></p>
<blockquote>
<p><strong>Heuristic: Does this model help me see something clearly, or does it intend to create a change in behaviour?</strong></p>
</blockquote>
<p>The distinction between descriptive and prescriptive is not as a clear cut as we might like. Only in science do we find purely descriptive models. While the social network model intends to be descriptive, by introducing the term into your organisation, you make a value choice. You’re saying that organisations should care about the social network. Usually this is followed by a prescription of how to do that. You may want to enable this network, leave it to evolve organically, or prohibit it. When you adopt a model that you believe to be descriptive, but that actually comes with an underlying value system that you unconsciously adopt, then the model controls you. You are inadvertently allowing it to affect your behaviour. By critically evaluating whether a model is mostly descriptive or prescriptive, you can choose to adopt its values and use its prescriptions more intentionally.</p>
<blockquote>
<p><strong>Heuristic: Find the prescriptions hidden in innocent looking models, and use them to your advantage.</strong></p>
</blockquote>
<p>Some models go even further: they are aspirational. They describe an ideal situation that has not been observed before. The author has created a model of how they believe the world could work, but has not actually applied it, or only partially. Aspirational models serve as a call to action. They speculate that, if we were to adopt all its prescriptions, we could get close to that ideal state, but perhaps never reach it. If we understand that a model is aspirational, then our goal is not to apply it precisely to the letter, but to get as close to it as is useful and practical. Interestingly, some models start as aspirational, but as people start getting better at explaining and implementing them, they observe that the model does indeed give them its stated benefits. If used wisely, aspirational models can drive positive change by getting people to see a new possibility.</p>
<blockquote>
<p><strong>Heuristic: Consider that some models may not actually have been tried by the authors.</strong></p>
</blockquote>
<h2 id="putting-the-model-to-work">Putting the Model to Work</h2>
<p>With many of the models we acquire throughout our lives, we only engage superficially, and that’s fine. We absorb and assimilate a lot of things without ever deeply engaging with them. This happens when, for example, you’re being onboarded at a new employer, and you simply learn their operating models for doing things in certain ways. The same goes when you’re given some design specifications and implement them as stated. And as a student, you’re usually expected to learn and adopt the models you are taught.</p>
<p>But when we’re trying to achieve something important, we need to take a more critical view. After learning of a new model, we need to decide whether we want to put it to work to help solve our problem. We need to engage more deeply with that model before we decide to commit to it. What if we could make this engagement an intentional process?</p>
<p>What we’ve been trying here, is to identify competing models and to compare them to understand their benefits and limitations, their framing, their values, and the contexts in which they apply. This comparison breaks us out of thinking there is only one option to solve our problem.</p>
<p>We’re making an upfront analysis of a model, before we invest in the model too heavily. This comes with risks: Because we haven’t lived with this model yet, we might be overanalysing things. We may discount some of benefits because they only surface after spending some effort implementing the model. Still, if we do this analysis upfront, it saves us effort that we would otherwise waste on implementing an unsuitable model. As we’re not invested deeply yet, it’s much cheaper to identify faults in a model and how address them. To avoid the risk of “analysis paralysis”, we remain conscious that no model is perfect, but may still be useful.</p>
<blockquote>
<p><strong>Heuristic: There’s always another possible model.</strong></p>
</blockquote>
<p>Then, when we find a new model that has a reasonable representation of the problem we’re trying to solve, we can adopt it. We apply its building blocks, and execute the activities it prescribes. As we do, we need to be observant: there is a thin line between learning to adopt a model, and overfitting a model to our situation. This friction is valuable data. Maybe the model has problems, lacks detail, has too much detail, focuses on the wrong aspects, has unintended consequences, or doesn’t directly tackle the problem we’re actually trying to solve. Maybe the model doesn’t fit that well. Or perhaps we don’t want to use the model in the way it’s prescribed. Perhaps we have different values than those supported by the model. If we collect this feedback early, we can use it to reexamine the model.</p>
<blockquote>
<p><strong>Heuristic: Following a model as prescribed is not the same as achieving success.</strong></p>
</blockquote>
<p>In other words, no matter how beautiful your diagram of hierarchies, Squads and Tribes, or Stream-Aligned and Enabling Teams is, the proof is in the pudding.</p>
<p>As we gain more experience with a model, and learn from feedback, we’ll feel more comfortable reshaping the model, bringing in elements from other models we previously dismissed, adjusting it to our context, and making it truly our own.</p>
<p>In summary, these are the five steps for engaging with a model:</p>
<ul>
<li>Intentionally study models</li>
<li>Analyse and compare them critically</li>
<li>Adopt a model in your context</li>
<li>Gather feedback about the impact</li>
<li>Reshape it to your needs</li>
</ul>
<p>Did we just present to you a 5-step model for evaluating models? Are there other models for evaluating models you can compare this one to? Should you adopt our model or create your own? Does our model put you in a certain framing that isn’t relevant to your context? We leave all these question as an exercise for the reader.</p>
<h2 id="conclusions">Conclusions</h2>
<p>Our worldviews and behaviours are mostly the result of other people’s models that we’ve assimilated, along with their biases. Models give us a framing, they highlight some aspects of the problem and obscure others. This is useful, assuming they focus on aspects that are important to us.</p>
<p>When we introduce an unsuitable model into our environment, it distracts us. It makes us ask questions in the framing of the model, and we don’t see that we should be asking different questions. It does this through its building blocks. These building blocks act like lightning rods. They attract the attention and energy of the people who are engaging with them.</p>
<p>Understanding a model’s framing is hard, precisely because we’ve been put into that framing. In that sense, we’re like the fish who can’t understand the world outside of the fishbowl, because nothing in their environment offers any building blocks to explain and structure it.</p>
<p>Acknowledging that you have limited agency is liberating. Comparing a model to other models gives us an edge: we can see more clearly which aspects we could highlight or obscure. By comparing models, we can break out of their constraints, and engage critically with the models.</p>
<p>You can choose which models to let in.</p>
<p><em>Written by <a href="https://twitter.com/rebeccawb">Rebecca Wirfs-Brock</a> and <a href="https://twitter.com/mathiasverraes">Mathias Verraes</a>.</em></p>
<div class="columns mb-6 ">
<div class="column is-6">
<figure class="media-left image " style="width: 75%; box-shadow: 2px 2px 7px darkgray">
<a href="https://leanpub.com/design-and-reality">
<img src="https://d2sofvawe08yqg.cloudfront.net/design-and-reality/s_hero2x?1664874345" />
</a>
</figure>
</div>
<div class="column is-6">
<h1 class="title">Design and Reality</h1>
<h2 class="subtitle">Essays on Software Design</h2>
<p>by Mathias Verraes & Rebecca Wirfs-Brock</p>
<br />
<a class="button is-info" href="https://leanpub.com/design-and-reality">
Get it on Leanpub
</a>
<br />
<br />
<blockquote>
“Happily purchased. Handy to have these in one place. Thank you!” <br />— Kent Beck<br />
</blockquote>
<br />
<blockquote>
“Excellent new thinking on Domain-Driven Design. It's full of real practical experience in getting the most
value from domain modelling. Just like the Eric Evans' DDD book, this gives more insight each time you read
it.”<br />— Ian Russell<br />
</blockquote>
</div>
</div>
<h2 id="footnotes">Footnotes</h2>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>One of the authors published a book about a sixth stage in 2020. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>The company Spotify doesn’t use the Spotify Model. We’ll use Spotify’s model and the Spotify Model interchangeably here. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>“Team Topologies”, Matthew Skelton & Manuel Pais, IT Revolution Press, 2019 <a href="#fnref:3" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:4" role="doc-endnote">
<p>“3 insights from 4 years at Spotify”, Jason Yip, presentation at YOW! 2019. <a href="https://www.slideshare.net/jchyip/yow-2019-3-insights-from-4-years-at-spotify-205092986">Slides</a> and <a href="https://www.youtube.com/watch?v=c6Jl88cImxc">video</a>. <a href="#fnref:4" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:5" role="doc-endnote">
<p>Outside of mathematics, when is the last time you saw three sets in a Venn diagram intersecting, but one of the intersections is empty? <a href="#fnref:5" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:6" role="doc-endnote">
<p><a href="https://www.youtube.com/watch?v=92Pq4-e0QyI">”Why Do Hotel Bathrooms Lack Toothpaste? Prescriptivists vs Descriptivists and You”</a>, Romeu Moura, presentation at Domain-Driven Design Europe 2019. <a href="#fnref:6" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Conway’s Law Doesn’t Apply to Rigid Designs2022-05-23T00:00:00+00:00https://verraes.net/2022/05/conways-law-vs-rigid-designs<p>The Inverse Conway Manoeuvre is an organisational engineering device. It tells us to change our team and organisational structure to achieve the system design we want. More and more organisations reorganise based on this idea, and yet they don’t end up with the system they’d like. What are we missing?</p>
<p>There’s evidence of the Manoeuvre being applied. GitHub for example is a distributed company that built a tool for distributed collaboration using GitHub to build it. While systems do mimic their organisations’ structure, <em>changing</em> the organisation doesn’t seem to have the same effect. That’s because we’re not accounting for the properties of the system under design itself.</p>
<h2 id="theories-are-systems">Theories are Systems</h2>
<p>In his paper, Dr. Conway describes a “structure-preserving relationship [between] the linear graph of a system to the linear graph of its design organisation”.<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> He uses a much a broader definition of “system” than just “software system”. His is more in line with Systems Theory, where (loosely defined) everything is a system and everything is part of a system. For example, Conway writes: “It may be less obvious that a theory is a system in the same sense.” His example involves investigators who form a theory of how a plane crashed. The investigators create sub-theories of different events leading up to the crash, based on their areas of expertise. The structure of the theory will mimic the communication structure of the investigators’ organisation.</p>
<p>A software system doesn’t live in a vacuum. It’s embedded in the organisation that produces this system. And it’s governed by the theories we have about the software: how it works, what it does, why it does that, how it’s all connected. We call these theories “models” and “architectures”. In some organisations, these theories exist only in people’s heads. Others express these in diagrams and documentation. It’s even better when they give continuous attention to collaborating and evolving these models, (for example, using Domain-Driven Design). But whether you have diagrams or not, everybody who uses or designs the system has some theory of that system. Without modelling these theories explicitly, we not only risk having a badly designed system, we risk having wrong beliefs about the system.</p>
<h2 id="examples">Examples</h2>
<p>Over a decade ago I was consulting for a SaaS company. I discovered that the belief of how the system worked varied wildly between the CEO, the sales people, the customer success managers and trainers, the customers, and the engineers. If you immediately assumed that the engineers had the best theory: nope. They understood so little about the design (which was largely created by their predecessors), that they wiped and rebuilt the customer dashboard database nightly. It was the only way they could get some consistency in the results.</p>
<p>Another example is a client of Aardling, a B2B company operating in a domain with complex regulatory requirements. When founding the company, the CEO (a domain expert) and the CTO agreed that the CTO would not get engaged in the domain’s complexity. Instead, the technology would be built in a way that allowed the tech to remain agnostic of the domain. To do this, the engineers built a form builder, a rule engine, and a templating language. They hired (non-technical) domain experts and taught them how to use these tools to build customer-facing features.</p>
<p>Fast-forward to a few years later. This theory about how the company should work now permeated everything, from the system design to the product to the organisational control structures. But there’s only so much you can do with forms and rule engines. Features that could easily be built with those tools, had fast turnaround times, but everything else had to be done by the engineers. This required budgets and prioritisation, and was often done through shortcuts, which in turn made subsequent changes even harder. The whole setup was making it impossible to achieve certain growth goals. The teams had no autonomy, and required a lot of alignment to achieve anything. Could a reorganisation solve this problem? Even though the system design had mimicked the communication structures that were put in place in the early days of the company, the system was now too rigid to be affected by a mere shuffling of teams.</p>
<h2 id="rigid-systems">Rigid Systems</h2>
<p>Advocates of the Inverse Conway Manoeuvre tell us that all we have to do is to reorganise teams to get a better system design. But, if the software itself, the sociotechnical environment that creates it, and the beliefs the people hold about the system, are all systems in their own right, and they’re all impacted by the organisation’s communication structure, then applying the Manoeuvre becomes a lot more messy. To intervene effectively in my two examples above, you’d have to address the organisation, the systems, and the theories the people hold about the organisation and the system.</p>
<p>Software is supposed to be soft, that is, malleable, easy to change. Often, it’s not, because these systems are usually deeply coupled and interconnected. Changing one component has ripple effects, not just in the software components, but in the entire ecosystem of the organisation and its customers. If this rigidness of our software is one of our main bottlenecks, you’d think we’d be eager to address it. And yet, a disappointingly low amount of software organisations give enough attention to deep structural improvement of their software.</p>
<p>Instead, they turn to reorganisations. And it is a tempting promise: change your organisation, get the system design you want. It even comes with a law, so it can’t fail! However, as a CTO at a client once told me, “We reorganised, but the system didn’t get the memo.” (There is a redeeming factor: managers can only impact the software system indirectly, and their only tools are meetings and reorganisations.)</p>
<h2 id="extending-conways-law-to-account-for-rigidity">Extending Conway’s Law to Account for Rigidity</h2>
<p>Is Conway’s Law wrong? No, I don’t think so. It holds true when designing a system from scratch, which is what the paper describes. It holds true when working with a flexible system design, flexible enough to adapt to mimic the new communication structures. But it doesn’t transfer well to existing systems with rigid designs. I propose this:</p>
<blockquote>
<p>The system design will mimic the communication structures of the organisation, but <em>only as much as the flexibility of the design allows.</em></p>
</blockquote>
<p>Dr. Conway commented: “I think of the relationship [between communication structure and system design] as a <em>constraint</em> and no more. What you can change by messing with one side or the other depends on their specific characteristics”<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>.</p>
<p>By consequence, the Inverse Conway Manoeuvre needs to be extended as well:</p>
<blockquote>
<p>If the system is new or flexible, change the organisational structure to get the system design you want. If the system is rigid, emphasize design work.</p>
</blockquote>
<p>Or, more aphoristically:</p>
<blockquote>
<p>A reorganisation won’t fix a broken design.</p>
</blockquote>
<p>Conway’s Law is undervalued when it comes to the design of new systems, but overstated when it comes to the <em>redesign</em> of existing systems. A greenfield project has unlimited flexibility. But as the design cements in more coupling and interconnectedness, it loses flexibility. We need to think about our organisational structure early and continuously, not only as an afterthought when the design has ossified. Preventing our systems from becoming rigid requires constant attention, from the thousands of small local design decisions to large scale architecture. To support and enable decision-making, we need good models or theories about the system, the domain, the product, the ecosystem it serves.</p>
<p>And when we do find ourselves with a rigid system, the Inverse Conway Manoeuvre is just one of the engineering devices we can use, not a magic wand.</p>
<p>In the end, good design is what leads to well-designed systems.</p>
<h3 id="acknowledgements">Acknowledgements</h3>
<p>Thanks to Dr. Conway for encouraging me to publish this.</p>
<h3 id="footnotes">Footnotes</h3>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p><a href="https://www.melconway.com/Home/pdf/committees.pdf">“How Do Committees Invent?”, Dr. Melvin Conway, April 1968</a> <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>Dr. Melvin Conway, quoted with permission from private correspondence, May 2022 <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Domain-Driven Design Applied2022-03-24T00:00:00+00:00https://verraes.net/2022/03/domain-driven-design-applied<p>I joined Alex Bolboacă on his podcast ‘Think. Design. Work Smart.’ We discussed DDD, the value of models, misconceptions about DDD (it’s not a Big Design Upfront method!) and more.</p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/RoyYR8KTowc" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
Eventsourcing Patterns: Multi-temporal Events2022-03-22T00:00:00+00:00https://verraes.net/2022/03/multi-temporal-events<h2 id="multi-temporal-events">Multi-temporal Events</h2>
<p>Within a Domain Event, use separate timestamps to distinguish when the event occurred and when it was captured.</p>
<h3 id="problem">Problem</h3>
<p>A Domain Event typically has a timestamp. A common pattern is to have the eventstore add the timestamp at the moment when the event is written. For example, there could be a database field named <code class="language-plaintext highlighter-rouge">recorded_at</code>, with a value that defaults to <code class="language-plaintext highlighter-rouge">now()</code>. The field is considered part of the metadata of the event (as opposed to the domain-specific payload). The field is accessible in the userland code that consumes the event, and can be used for infrastructural tasks (such as ordering the events chronologically) or domain-specific operations, projections, analytics…</p>
<p>This is fine in many situations. For most purposes, the recorded time coincides with the time the event happened. Many events are generated by the system, or are a direct consequence of a user-generated action. Even when there is a small delay between, say, a user clicking a button and the resulting event being persisted, the difference is often negligible. After all, a lot of business processes operate at a granularity of minutes, hours, days, and even months and quarters.</p>
<p>However, sometimes we actually care about the difference. Say that every midnight we receive a statement with the day’s bank transactions. We record each transaction as a Domain Event, and its <code class="language-plaintext highlighter-rouge">recorded_at</code> timestamp is somewhere after midnight. However, the transaction happened somewhere during the day before. This matters if the payment date has an impact on, say, the calculation of interests, fiscal benefits, legal implications, or other time-sensitive aspects.</p>
<p>Another example is tracking car crashes in a fleet management system. When we persist the event, we have the time when we recorded it. But when did the crash occur, and when was it reported?</p>
<h3 id="solution">Solution</h3>
<p>Identify Domain Event types where the time they occurred is different from the time they are recorded, and where the difference matters. In the Domain Events’ schema, add a domain-specific property that reflects the time the event occurred. Name the property after its purpose in the domain. Do not have a default <code class="language-plaintext highlighter-rouge">now()</code> value, and instead rely on the Event Producer to fill in the property.</p>
<p>Consumers of the event can now use that property to make relevant decisions.</p>
<h3 id="example">Example</h3>
<p>In our bank statements example, the schema could look like this:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># (pseudocode, details omitted)</span>
<span class="pi">-</span> <span class="s">Event</span>
<span class="s">- eventId</span><span class="err">:</span> <span class="s">UUID</span>
<span class="s">- recorded_at</span><span class="err">:</span> <span class="s">timestamp</span>
<span class="s">- type</span><span class="err">:</span> <span class="s">BankAccountWasCredited</span>
<span class="s">- payload</span><span class="err">:</span>
<span class="pi">-</span> <span class="na">amount</span><span class="pi">:</span> <span class="s">number</span>
<span class="pi">-</span> <span class="na">currency</span><span class="pi">:</span> <span class="s">string</span>
<span class="pi">-</span> <span class="na">deposited_at</span><span class="pi">:</span> <span class="s">timestamp</span>
</code></pre></div></div>
<h2 id="discussion">Discussion</h2>
<p>The <code class="language-plaintext highlighter-rouge">deposited_at</code> property represents the actual transaction time, which happened before the <code class="language-plaintext highlighter-rouge">recorded_at</code> persistence time. The event is multi-temporal, as it represents two moments in time.</p>
<p>Why don’t we simply inject the event somewhere in the eventstore at the point where the transaction happened? It would simplify the design of the event itself. But it’s actually a bad idea. Eventstores are ordered chronologically, so we’d have to insert the event and shift all the events that come after. More importantly, in eventsourcing, the events are published after they have been persisted. If we inject the event somewhere within the history, we’d have to tell all consumers that this particularly event is out of order. It creates additional complexity for the consumers.</p>
<h3 id="good-habits">Good habits</h3>
<p>I recommend this technique even for events with no special timestamping requirements. That means that every event type’s payload will have at least 1 timestamp.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">-</span> <span class="s">Event</span>
<span class="s">- eventId</span><span class="err">:</span> <span class="s">UUID</span>
<span class="s">- recorded_at</span><span class="err">:</span> <span class="s">timestamp</span>
<span class="s">- type</span><span class="err">:</span> <span class="s">BankAccountWasCredited</span>
<span class="s">- payload</span><span class="err">:</span>
<span class="pi">-</span> <span class="na">amount</span><span class="pi">:</span> <span class="s">number</span>
<span class="pi">-</span> <span class="na">currency</span><span class="pi">:</span> <span class="s">string</span>
<span class="pi">-</span> <span class="na">deposited_at</span><span class="pi">:</span> <span class="s">timestamp</span>
<span class="pi">-</span> <span class="na">statement_received_at</span><span class="pi">:</span> <span class="s">timestamp</span>
</code></pre></div></div>
<p>Note that the new <code class="language-plaintext highlighter-rouge">statement_received_at</code> contains duplicate information, as the timestamp will be equal to the <code class="language-plaintext highlighter-rouge">recorded_at</code> one. This is, in my opinion, a small price to pay, compared to the benefits:</p>
<ul>
<li>We now have a domain-specific name for that property, which communicates the intent of the design better. Another developer can understand the meaning of the property within the domain, and doesn’t have to guess whether they can use the <code class="language-plaintext highlighter-rouge">recorded_at</code> field for their needs.</li>
<li>We have better decoupling: consumers only need to worry about the payload, and don’t need to rely on the metadata. This makes sense, as it decouples the infrastructural needs from the domain-specific needs. With no dependence on the metadata, you could (in theory) replace the eventstore with a different vendor, without affecting the domain model.</li>
<li>It is easier to evolve the model itself as well. If we depend on <code class="language-plaintext highlighter-rouge">recorded_at</code>, and later we decide that we need to track occurrence, we’ll need to adapt all the consumers’ code. If we already had the domain-specific properties, we only need to change the producer. Let’s look at an example.</li>
</ul>
<h3 id="example-2">Example 2</h3>
<p>In the first version of the model, we track car crashes:</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">-</span> <span class="s">Event</span>
<span class="s">- eventId</span><span class="err">:</span> <span class="s">UUID</span>
<span class="s">- recorded_at</span><span class="err">:</span> <span class="s">timestamp</span>
<span class="s">- type</span><span class="err">:</span> <span class="s">CarHasCrashed</span>
<span class="s">- payload</span><span class="err">:</span>
<span class="pi">-</span> <span class="na">carId</span><span class="pi">:</span> <span class="s">UUID</span>
<span class="pi">-</span> <span class="na">driverId</span><span class="pi">:</span> <span class="s">UUID</span>
<span class="pi">-</span> <span class="na">crashed_at</span><span class="pi">:</span> <span class="s">timestamp</span> <span class="c1"># using now()</span>
</code></pre></div></div>
<p>Later, we realise that the crash time can’t possibly coincide with the time the crash is reported. Perhaps the insurance company requires us to have both the crash time and the reported time. We can simply adjust the schema and the producer, but existing consumers can keep using <code class="language-plaintext highlighter-rouge">crashed_at</code>.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">-</span> <span class="s">Event</span>
<span class="s">- eventId</span><span class="err">:</span> <span class="s">UUID</span>
<span class="s">- recorded_at</span><span class="err">:</span> <span class="s">timestamp</span>
<span class="s">- type</span><span class="err">:</span> <span class="s">CarHasCrashed_V2</span>
<span class="s">- payload</span><span class="err">:</span>
<span class="pi">-</span> <span class="na">carId</span><span class="pi">:</span> <span class="s">UUID</span>
<span class="pi">-</span> <span class="na">driverId</span><span class="pi">:</span> <span class="s">UUID</span>
<span class="pi">-</span> <span class="na">crashed_at</span><span class="pi">:</span> <span class="s">timestamp</span> <span class="c1"># no longer using now(), the user has to fill this in</span>
<span class="pi">-</span> <span class="na">reported_at</span><span class="pi">:</span> <span class="s">timestamp</span> <span class="c1"># using now()</span>
</code></pre></div></div>
<p>Then, after observing the phone operators, we realise another discrepancy: After the conversation, the operators first need to do a number of actions, before they have time to enter the crash report in the system. We can now have the phone operators provide the <code class="language-plaintext highlighter-rouge">reported_at</code> time, and introduce a new field `report_entered_at’.</p>
<div class="language-yaml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="pi">-</span> <span class="s">Event</span>
<span class="s">- eventId</span><span class="err">:</span> <span class="s">UUID</span>
<span class="s">- recorded_at</span><span class="err">:</span> <span class="s">timestamp</span>
<span class="s">- type</span><span class="err">:</span> <span class="s">CarHasCrashed_V3</span>
<span class="s">- payload</span><span class="err">:</span>
<span class="pi">-</span> <span class="na">carId</span><span class="pi">:</span> <span class="s">UUID</span>
<span class="pi">-</span> <span class="na">driverId</span><span class="pi">:</span> <span class="s">UUID</span>
<span class="pi">-</span> <span class="na">crashed_at</span><span class="pi">:</span> <span class="s">timestamp</span> <span class="c1"># no longer using now(), the user has to fill this in</span>
<span class="pi">-</span> <span class="na">reported_at</span><span class="pi">:</span> <span class="s">timestamp</span> <span class="c1"># no longer using now(), the user has to fill this in</span>
<span class="pi">-</span> <span class="na">report_entered_at</span><span class="pi">:</span> <span class="s">timestamp</span> <span class="c1"># using now()</span>
</code></pre></div></div>
<p>At every step, the domain model underlying the schema remains consistent with our updated understanding of the domain, and impact on consumers is limited.</p>
<h3 id="timestamp-precision">Timestamp Precision</h3>
<p>As we have domain-specific timestamps now, we can choose to constrain them to the precision that is a relevant to our context. We don’t need car crash or bank transaction events with a precision in milliseconds, when we only care about the hours and minutes, or the day, or even only the month in which something happened. That said, the problem can also be easily solved in the presentation layer, by rounding the timestamps to the needed precision.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Loss Aversion Heuristics2022-02-25T00:00:00+00:00https://verraes.net/2022/02/loss-aversion-heuristics<p>When we make decisions, we often do not explore options that have a risk of loss, even if that loss is balanced out by potential gains. I joined João Rosa’s “Software Crafts” podcast to talk about 3 heuristics that help you overcome that bias.</p>
<h3 id="podcast-player">Podcast player</h3>
<iframe src="https://anchor.fm/software-crafts-podcast/embed/episodes/Interview-with-Mathias-Verraes-e1hmc3e/a-a7qjr8n" height="102px" width="400px" frameborder="0" scrolling="no"></iframe>
<h2 id="resources">Resources</h2>
<ul>
<li><a href="https://www.softwarecraftspodcast.com/episodes/20220216-ep53-mathias-verraes/">Software Crafts Podcast</a></li>
<li><a href="https://amzn.to/3IpHpra">Thinking, Fast and Slow</a> by Daniel Kahneman</li>
<li><a href="https://amzn.to/33TlppG">The Scout Mindset</a> by Julia Galef</li>
<li><a href="https://youarenotsosmart.com/">You Are Not So Smart</a></li>
<li><a href="https://verraes.net/2021/05/its-just-like-heuristic/">The “It’s Just Like…” Heuristic</a></li>
<li><a href="https://thecynefin.co/exaptation-managed-serendipity-part-i/">Exaptation & managed serendipity</a></li>
<li><a href="https://dddeurope.com/">Domain-Driven Design Europe conference</a></li>
</ul>
Design & Reality (VirtualDDD presentation)2022-02-25T00:00:00+00:00https://verraes.net/2022/02/design-and-reality-virtualddd<p>In this talk, I discuss the ideas previously presented in the articles <a href="https://verraes.net/2021/09/design-and-reality/">“Design and Reality”</a> and <a href="https://verraes.net/2021/12/models-and-metaphors/">“Models and Metaphors”</a></p>
<h2 id="abstract">Abstract:</h2>
<p>Our models should be driven by the domain, but not constrained by what domain experts tell us. After all, the domain language is messy, organic, ambiguous, social, incomplete, and if it has any intentional design to it at all, it’s not designed to be turned into software. Modelling is more than capturing requirements, it’s the opportunity to create novel concepts.</p>
<h2 id="video">Video</h2>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/s_ihqdfcHDY" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<h2 id="podcast">Podcast</h2>
<iframe title="Libsyn Player" style="border: none" src="//html5-player.libsyn.com/embed/episode/id/21773594/height/90/theme/custom/thumbnail/yes/direction/forward/render-playlist/no/custom-color/88AA3C/" height="90" width="100%" scrolling="no" allowfullscreen="" webkitallowfullscreen="" mozallowfullscreen="" oallowfullscreen="" msallowfullscreen=""></iframe>
<h2 id="resources">Resources</h2>
<ul>
<li><a href="https://virtualddd.com/sessions/77">VirtualDDD</a></li>
</ul>
Models and Metaphors2021-12-20T00:00:00+00:00https://verraes.net/2021/12/models-and-metaphors<p>One of us (Mathias) consulted for a client that acted as a broker for paying copyright holders for the use of their content. To do this, they figured out who the copyright holders of a work were. Then they tracked usage claims, calculated the amounts owed, collected the money, and made the payments. Understanding who owned what was one of the trickier parts of their business.</p>
<p>— “It’s just a technical problem.”</p>
<p>— “But nobody really understands how it works!”</p>
<p>— “Some of us understand most of it. It just happens to be a complicated problem.”</p>
<p>— “Let’s do a little bit of modelling anyway.”</p>
<p>A few weeks later, we had a rich model. It had crisp concepts, and was easier to understand. Business stakeholders started to pay attention, and became actively involved in modelling. They saw the big idea, and could describe what they wanted. Here’s how that happened.</p>
<h2 id="case-study">Case Study</h2>
<p>Determining ownership was a complicated data matching process which pulled data from a number of data sources:</p>
<ul>
<li>Research done by the company itself</li>
<li>Offshore data cleaning</li>
<li>Publicly available data from a wiki-style source</li>
<li>Publicly available, curated data</li>
<li>Private sources, for which the company paid a licence fee</li>
<li>Direct submissions from individuals</li>
<li>Agencies representing copyright holders</li>
</ul>
<p>The company had a data quality problem. Because of the variety of data sources, there wasn’t a single source of truth for any claim. The data was often incomplete and inconsistent. On top of that, there was a possibility for fraud: bad actors claimed ownership of authors’ work. Most people acted in good faith. Even then, the data was always going to be messy, and it took considerable effort to sort things out. The data was in constant flux: even though the ownership of a work rarely changes, the data did.</p>
<h2 id="data-matching">Data Matching</h2>
<p>The engineers were always improving the “data matching”. That’s what they called the process of reconciling the inconsistencies, and providing a clear view on who owned what and who had to pay whom. They used EventSourcing, and they could easily replay new matching algorithms on historic data. The data matching algorithms matched similar claims on the same works in the different data sources. When multiple data sources concurred, the match succeeded.</p>
<p>Initially, when most sources concurred on a claim, the algorithm ignored a lone exception. When there was more contention about a claim, it was less obvious what to do. The code reflected this lack of clarity. Later the team realised that a conflicting claim could tell them more: It was an indicator of the messiness of the data. If they used their records of noise in the data, they could learn about how often different data sources, parties, and individuals agreed on successful claims, and improve their algorithm.</p>
<p>For example, say a match was poor: 50% of sources point to one owner and 50% point to another owner. Based on that information alone, it’s impossible to decide who the owner is. But by using historical data, the algorithm could figure out which sources had been part of successful matches more often. They could give more weight to these sources, and tip the scales in one direction or the other. This way, even if 50% of sources claim A as the owner and 50% claim B, an answer can be found.</p>
<h2 id="domain-modelling">Domain Modelling</h2>
<p>The code mixed responsibilities: pulling data, filtering, reformatting, interpreting, and applying matching rules. All the cases and rules made the data matching very complicated. Only a few engineers knew how it worked. Mathias noticed that the engineers couldn’t explain how it worked very well. And the business people he talked to were unable to explain anything at all about how the system worked. They simply referred to it as the “data matching”. The team wasn’t concerned about this. In their eyes, the complexity was just something they had to deal with.</p>
<p>Mathias proposed a whiteboard modelling session. Initially, the engineers resisted. After all, they didn’t feel this was a business domain, just a purely technical problem. However, Mathias argued, the quality of the results determined who got paid what, and mistakes meant customers would eventually move to a competitor. So even if the data matching was technical, it performed an essential function in the Core Domain. The knowledge about it was sketchy, engineering couldn’t explain it, business didn’t understand it. Because of that, they rarely discussed it, and when they did, it was in purely technical terms. If communication is hard, if conversations are cumbersome, you lack a good shared model.</p>
<p>Through modelling, the matching process became less opaque to the engineers. We made clearer distinctions between different steps to pull data, process it, identifying a match, and coming to a decision. The model included sources, claims, reconciliations, exceptions. We drew the matching rules on the whiteboard as well, making those rules explicit first class concepts in the model. As the matching process became clearer, the underlying ideas that led to the system design started surfacing. From the “what”, we moved to the “why”. This put us in a good position to start discovering abstractions.</p>
<h2 id="trust">Trust</h2>
<p>Gradually, the assumptions that they built the algorithm on, surfaced in the conversations. We stated those assumptions, wrote them on stickies and put them on the whiteboard. One accepted assumption was that when a data source is frequently in agreement with other sources, it is less likely to be wrong in the future. If a source is more reliable, it should be trusted more, therefore claims from that source pulled more weight in the decision of who has a claim to what. When doing domain discovery and modelling, it’s good to be observant, and listen to subtleties in the language. Words like “reliable, “trust”, “pull more weight”, and “decision” were being used informally in these conversations. What works in these situations, is to have a healthy obsession with language. Add this language to the whiteboard. Ask questions: What does this word mean, in what context do you use it?</p>
<p>Through these discussions, the concept of “trust” grew in importance. It became explicit in the whiteboard models. It was tangible: you could see it, point to it, move it around. You could start telling stories about trust. Why would one source be more trusted? What would damage that trust? What edge cases could we find that would affect trust in different ways?</p>
<h2 id="trust-as-an-object">Trust as an Object</h2>
<p>During the next modelling session, we talked about trust a lot. From a random word that people threw into the conversation, it had morphed into a meaningful term. Mathias suggested a little thought experiment: What if <em>Trust</em> was an actual object in the code? What would that look like? Quickly, a simple model of Trust emerged. Trust is a Value Object, and its value represents the “amount” of trust we have in a data source, or the trust we have in a claim on a work or usage, or the trust we have in the person making the claim. Trust is measured on a scale of -5 to 5. That number determines whether a claim is granted or not, whether it needs additional sources to confirm it, or whether the company needs to do further research.</p>
<p>It was a major mindshift.</p>
<p>The old code dynamically computed similar values to determine “matches”. These computations were spread and duplicated across the code, hiding in many branches. The team didn’t see that all these values and computations were really aspects of the same underlying concept. They didn’t see that the computations could be shared, whether you’re matching sources, people, or claims. There was no shared abstraction.</p>
<p>But now, in the new code, those values are encapsulated in a first class concept called Trust objects. This is where the magic happens: we move from a whiteboard concept, to making Trust an essential element in the design. The team cleaned up the ad hoc logic spread across the data matching code and replaced it with a single Trust concept.</p>
<p>Trust entered the Ubiquitous Language. The idea that degrees of Trust are ranked on a scale from -5 to 5, also became part of the language. And it gave us a new way to think about our Core Domain: We pay owners based on who earns our Trust.</p>
<h2 id="trust-as-a-process">Trust as a Process</h2>
<p>The team was designing an EventSourced system, so naturally, the conversation moved to what events could affect Trust. How does Trust evolve over time? What used to be matching claims in the old model, now became events that positively or negatively affected our Trust in a claim. Earning Trust (or losing it) was now thought of as a process. A new claim was an event in that process. Trust was now seen as a snapshot of the Trust earning process. If a claim was denied, but new evidence emerged, Trust increased and the claim was granted. Certain sources, like the private databases that the company bought a licence for, were highly trusted and stable. For others, like the wiki-style sources where people could submit claims, Trust was more volatile.</p>
<h2 id="business-involvement">Business Involvement</h2>
<p>During the discussions about the new Trust and Trust-building concepts, the team went back to the business regularly to make sure the concepts worked. They asked for their insights into how they should assign Trust, and what criteria they should use. We saw an interesting effect: people in the business became invested in these conversations and joined in modelling sessions. Data matching faded from the conversations, and Trust took over. There was a general excitement about being able to assign and evolve Trust. The engineers’ new model became a shared model within the business.</p>
<h2 id="trust-as-an-arithmetic">Trust as an Arithmetic</h2>
<p>The copyright brokerage domain experts started throwing scenarios at the team: What if a Source A with a Trust of 0 made a claim that was corroborated by a Source B with a Trust of 5? The claim itself was now highly trusted, but what was the impact on Source A? One swallow doesn’t make Spring, so surely Source A shouldn’t be granted the same level of Trust as Source B. A repeated pattern of corroborated Trust on the other hand, should reflect in higher Trust for Source A.</p>
<p>During these continued explorations, people from the business and engineering listed the rules for how different events impacted Trust, and coding them. By seeing the rules in code, a new idea emerged. Trust could have its own arithmetic: a set of rules that defined how Trust was accumulated. For example, a claim with a Trust of 3, that was corroborated by a claim with a Trust of 5, would now be assigned a new Trust of 4. The larger set of arithmetics addressed various permutations of claims corroborating claims, sources corroborating sources, and patterns of corroboration over time. The Trust object encapsulated this arithmetic, and managed the properties and behaviours for it.</p>
<p>From an anaemic Trust object, we had now arrived at a richer model of Trust that was responsible for all these operations. The team came up with polymorphic Strategy objects. These allowed them to swap out different mechanisms for assigning and evolving Trust. The old data matching code had mixed fetching and storing information with the sprawling logic. Now, the team found it easy to separate it into a layer that dealt with the plumbing separate from the clean Trust model.</p>
<h2 id="the-evolution-of-the-model">The Evolution of the Model</h2>
<p>In summary, this was the evolution:</p>
<ol>
<li>Ad hoc code that computes values for matches.</li>
<li>Using Trust in conversations that explained how the current system worked.</li>
<li>Trust as a Value Object in the code.</li>
<li>Evolving Trust as a process, with events (such as finding a matching claim) that assigned new values of Trust.</li>
<li>Trust as a shared term between business and engineering, that replaced the old language of technical data matching.</li>
<li>Exploring how to assign Trust using more real-world scenarios.</li>
<li>Building an arithmetic that controls the computation of Trust.</li>
<li>Polymorphic Strategies for assigning Trust.</li>
</ol>
<p>When you find a better, more meaningful abstraction, it becomes a catalyst: it enables other modelling constructs, allowing other ideas to form around that concept. It takes exploration, coding, conversations, trying scenarios, … There’s no golden recipe for making this happen. You need to be open to possibility, and take the time for it.</p>
<h2 id="discussion">Discussion</h2>
<p>The engineers originally introduced the concept of “matching”, but that was an anaemic description of the algorithm itself, not the purpose. “If this value equals that value, do this”. Data matching was devoid of meaning. That’s what Trust introduces: conceptual scaffolding for the meaning of the system. Trust is a magnet, an attractor for a way of thinking about and organising the design.</p>
<p>Initially, the technical details of the problem were so complicated, and provided such interesting challenges to the engineers, that that was all they talked about with the business stakeholders. Those details got in the way of designing a useful Ubiquitous Language. The engineers had assumed that their code looked the way it needed to look. In their eyes, the code was complex because the problem of matching was complex. The code simply manifested that complexity. They didn’t see the complexity of that code as a problem in its own right. The belief that there wasn’t a better model to be found, obscured the Core Domain for both business and engineering.</p>
<p>The domain experts were indeed experts in the copyrights domain, and had crisp concepts for ownership, claims, intellectual property, the laws, and the industry practices. But that was not their Core Domain. The real Core was the efficient, automated business they’re trying to build out of it. That was their new domain. That explains why knowledge of copyright concepts alone wasn’t sufficient to make a great model.</p>
<p>Before they developed an understanding of Trust, business stakeholders could tell you detailed stories about how the system should behave in specific situations. But they had lacked the language to talk about these stories in terms of the bigger idea that governs them. They were missing crisp concepts for them.</p>
<h2 id="good-metaphors">Good Metaphors</h2>
<p>We moved from raw code, to a model based on the new concept of Trust. But what kind of thing is this Trust concept? Trust is a metaphor.<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> Actual trust is a human emotion, and partly irrational. You trust someone instinctively, and for entirely subjective reasons that might change. Machines don’t have these emotions. We have an artificial metric in our system, with algorithms to manipulate it, and we named it Trust. It’s a proxy term.</p>
<p>This metaphor enables a more compact conversation, as evidenced by the fact that engineers and domain experts alike can discuss Trust without losing each other in technical details. A sentence like “The claims from this source were repeatedly confirmed by other sources” was replaced by “This source has built up trust”, and all knew what that entailed.</p>
<p>The metaphor allows us to handle the same degree of complexity, but we can reason about determining Trust without having to understand every detail at the point where it’s used. For those of us without Einstein brains, it’s now a lot easier to work on the code, it lowers the cognitive load.</p>
<p>A good metaphor in the right context, such as Trust, enables us to achieve things we couldn’t easily do before. The team reconsidered a feature that would allow them to swap out different strategies for matching claims. Originally they had dismissed the idea, because, in the old code, it would have been prohibitively expensive to build. It would have resulted in huge condition trees and sprawling dependencies on shared state. They’d have to be very careful, and it would be difficult to test that logic. With the new model, swapping out polymorphic Strategy objects is trivial. The new model allows testing low level units like the Trust object, higher level logic like the Trust-building process, and individual Claims Strategies, with each test remaining at a single level of abstraction.</p>
<p>Our Trust model not only organises the details better, but it is also concise. We can go to a single point in the code and know how something is determined. A Trust object computes its own value, in a single place in the code. We don’t have to look at twenty different conditionals across the code to understand the behaviour; instead we can look at a single strategy. It’s much easier to spot bugs, which in turn helps us make the code more correct.</p>
<p>A good model helps you reason about the behaviour of a system. A good metaphor helps you reason about the <em>desired</em> behaviour of a system.</p>
<p>The Trust metaphor unlocked a path to tackle complexity. We discovered it by listening closely to the language used to describe the solution, using that language in examples, and trying thought experiments. We’re not matching data anymore, we’re determining Trust and using it to resolve claims. Instead of coding the rules, we’re now <em>encoding</em> them. We’re better copyright brokers because of this.</p>
<h2 id="bad-metaphors">Bad Metaphors</h2>
<p>Be wary of bad, ill-fitting metaphors. Imagine the team had come with Star Ratings as the metaphor. Sure, it also works as a quantification, but it’s based on popularity, and calculates the average. We could still have built all the same behaviour of the Trust model, but with a lot of bizarre rules, like “Our own sources get 20 five-star ratings”. When you notice that you have to force-fit elements of your problem space into a metaphor, and there’s friction between what you want to say and what that metaphor allows you to say, you need to get rid of it. No metaphor will make a perfect fit, but a bad metaphor leads you into awkward conversations without buying you clarity.</p>
<p>To make things trickier, whenever you introduce a new metaphor, it can be awkward at first. In our case study, Trust didn’t instantly become a fully explored and accepted metaphor. There’s a delicate line between the early struggles of adopting a new good metaphor, and one that is simply bad. Keep trying, work on using your new metaphor, see if it buys you explanatory power, and don’t be afraid to drop it if it does not.</p>
<p>And sometimes, there simply isn’t any good metaphor, or even a simpler model to be found. In those cases, you just have to crunch it. There’s no simplification to be found. You just have to work out all the rules, list all the cases, and deal with the complexity as is.</p>
<h2 id="conclusion">Conclusion</h2>
<p>To find good metaphors, put yourself in a position where you’ll notice them in conversation. Invite diverse roles into your design discussions. Have a healthy obsession with language: what does this mean? Is this the best way to say it? Be observant about this language, listen for terms that people say off the cuff. Capture any metaphors that people use. Reinforce them in conversations, but be ready to drop them if you feel you have to force-fit them. Is a metaphor bringing clarity? Does it help you express the problem better? Try scenarios and edge cases, even if they’re highly unlikely. They’ll teach you about the limits of your metaphor. Then distil the metaphor, agree on a precise meaning. Use it in your model, and then translate it to your code and tests. Metaphors are how language works, how our brains attach meaning, and we’re using that to our advantage.</p>
<p><em>Written by <a href="https://twitter.com/mathiasverraes">Mathias Verraes</a> and <a href="https://twitter.com/rebeccawb">Rebecca Wirfs-Brock</a>.</em></p>
<h2 id="update">Update</h2>
<ul>
<li>Mathias discusses this essay in a <a href="https://verraes.net/2022/02/design-and-reality-virtualddd/">VirtualDDD session</a>.</li>
</ul>
<div class="columns mb-6 ">
<div class="column is-6">
<figure class="media-left image " style="width: 75%; box-shadow: 2px 2px 7px darkgray">
<a href="https://leanpub.com/design-and-reality">
<img src="https://d2sofvawe08yqg.cloudfront.net/design-and-reality/s_hero2x?1664874345" />
</a>
</figure>
</div>
<div class="column is-6">
<h1 class="title">Design and Reality</h1>
<h2 class="subtitle">Essays on Software Design</h2>
<p>by Mathias Verraes & Rebecca Wirfs-Brock</p>
<br />
<a class="button is-info" href="https://leanpub.com/design-and-reality">
Get it on Leanpub
</a>
<br />
<br />
<blockquote>
“Happily purchased. Handy to have these in one place. Thank you!” <br />— Kent Beck<br />
</blockquote>
<br />
<blockquote>
“Excellent new thinking on Domain-Driven Design. It's full of real practical experience in getting the most
value from domain modelling. Just like the Eric Evans' DDD book, this gives more insight each time you read
it.”<br />— Ian Russell<br />
</blockquote>
</div>
</div>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>If you want to learn more about metaphors and how they shape language, read <a href="https://amzn.to/3ecGEUN">“Metaphors we Live By”</a> (George Lakoff and Mark Johnson, University of Chicago Press, 2003). <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Bounded Context Podcast2021-09-30T00:00:00+00:00https://verraes.net/2021/09/bounded-context-podcast<p>I had a conversation with Ryan Shriver of Singlestone Consulting — well, mostly a monologue, I didn’t warn Ryan that he should interrupt me more :-S</p>
<p>We talked about about DDD, boundaries, legacy, and a little bit about Shostakovich’ 15 string quartets.</p>
<h3 id="video">Video</h3>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://player.vimeo.com/video/600504936?h=c8bc8f8b29" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture fullscreen" allowfullscreen=""></iframe>
</figure>
<h3 id="podcast">Podcast</h3>
<iframe src="https://open.spotify.com/embed/episode/4gRhjFeMAw7mz0fTRi5fBF" width="100%" height="152" frameborder="0" allowfullscreen="" allow="autoplay; clipboard-write; encrypted-media; fullscreen; picture-in-picture"></iframe>
<h3 id="other-links">Other links</h3>
<ul>
<li><a href="https://podcasts.apple.com/us/podcast/mathias-verraes-founder-principal-consultant-at-aardling/id1538714502?i=1000534810736">Apple Podcasts</a></li>
<li><a href="https://www.breaker.audio/bounded-context/e/93771194">Breaker</a></li>
<li><a href="https://www.ivoox.com/en/mathias-verraes-8212-founder-amp-principal-consultant-at-audios-mp3_rf_75235517_1.html">iVoox</a></li>
<li><a href="https://www.listennotes.com/podcasts/bounded-context/mathias-verraes-founder-wuUMqIFKZ2D/">Listen Notes</a></li>
<li><a href="https://pca.st/1hnw6ywu">Pocket Casts</a></li>
<li><a href="https://podtail.com/podcast/bounded-context/mathias-verraes-founder-principal-consultant-at-aa/">Podtail</a></li>
<li><a href="https://open.spotify.com/episode/4gRhjFeMAw7mz0fTRi5fBF?si=_zP57km5RG-BUsePCuATZg&dl_branch=1">Spotify</a></li>
<li><a href="https://vimeo.com/600504936">Vimeo</a></li>
</ul>
<h2 id="references">References</h2>
<ul>
<li><a href="https://dddeurope.com/">Domain-Driven Design Europe</a></li>
<li><a href="https://martinfowler.com/books/refactoring.html">Refactoring,</a> Martin Fowler</li>
<li><a href="https://www.youtube.com/watch?v=pMuiVlnGqjk">DDD Conference Europe</a>, Eric Evans</li>
<li><a href="http://wirfs-brock.com/blog/">Rebecca Wirfs-Brock: Blog,</a> <a href="https://twitter.com/rebeccawb">Twitter</a></li>
<li><a href="https://www.amazon.com/gp/product/0136298257/ref=dbs_a_def_rwt_bibl_vppi_i1">Designing Object-Oriented Software</a>, Rebecca Wirfs-Brock</li>
<li><a href="https://verraes.net/2021/06/split-domain-across-bounded-contexts/">Splitting a Domain Across Multiple Bounded Contexts</a>, by Mathias Verraes and Rebecca Wirfs-Brock</li>
<li><a href="https://www.amazon.com/Data-Reality-Perspective-Perceiving-Information/dp/1935504215">Data and Reality</a>, William Kent</li>
<li><a href="http://davewest.us/product/object-thinking/">Object Thinking</a>, Dave West</li>
<li><a href="https://www.cognitive-edge.com/blog/">Cognitive Edge</a>, Dave Snowden</li>
<li><a href="https://www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/0131177052">Working Effectively with Legacy Code</a>, Michael Feathers</li>
</ul>
What is Domain-Driven Design (DDD)2021-09-22T00:00:00+00:00https://verraes.net/2021/09/what-is-domain-driven-design-ddd<p>Domain-Driven Design is a software design discipline centred on the principles that:</p>
<ul>
<li>Software for a complex domain requires all designers (engineers, testers, analysts, …) to have a deep, <strong>shared understanding of the domain, guided by domain experts</strong></li>
<li>That understanding is <strong>rooted in language</strong>: the domain language should be formalised into a Ubiquitous Language (shared, agreed upon, unambiguous)</li>
<li>The understanding is <strong>expressed in a model</strong>, shared between experts and designers, which express the problem space (as opposed to the solution space)</li>
<li>The model must not shy away from <strong>explicitly expressing the essential complexity</strong> of the domain</li>
<li>A complex domain can not be efficiently expressed as a single universal model and language, and must therefore be <strong>separated into Bounded Contexts</strong> (ie. an internally consistent language and model) by the system designers</li>
<li>The model and language should be <strong>in sync</strong> with our understanding of, and changes in the domain, through <strong>continuous refactoring towards deeper insight</strong></li>
</ul>
<p>As a mnemonic:</p>
<p><strong>DDD is a design discipline where you</strong></p>
<ul>
<li><strong>Grasp the domain</strong></li>
<li><strong>Agree on a language</strong></li>
<li><strong>Express it in shared models</strong></li>
<li><strong>Embrace complexity</strong></li>
<li><strong>Separate models in contexts</strong></li>
<li><strong>… and evolve them continuously</strong></li>
</ul>
<h2 id="notes">Notes</h2>
<p>Domain-Driven Design presents an all-encompassing view on software design. It considers design from the micro-level of code and design patterns, to models and their language, to communication and relationships between models, to the large scale reasoning about systems of systems. On top of that, it aims to be pragmatic. You don’t apply DDD everywhere, you do it where it will have the most impact.</p>
<p>DDD is not prescriptive. It doesn’t have rules of how to do it, and is open to new interpretation. It doesn’t prescribe methods, or practices, and even the patterns in the book<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> are meant to be illustrative rather than a final set. Many methods that people now consider core to DDD, such as EventStorming, didn’t exist when the book was written.</p>
<p>That makes DDD notoriously hard to define. This definition is specifically focused on DDD as a discipline, that is, design as something you <em>do</em>, activities with a set of guiding principles. I’ve been using and improving this definition in my workshops for probably 6 or 7 years now, so I hope that makes it hardened enough to be useful for both newcomers to DDD and seasoned practitioners. I hope that keep it up to date with new insights.</p>
<blockquote>
<div style="font-size: 1em"> <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/"><img alt="Creative Commons Licence" style="border-width:0" src="https://i.creativecommons.org/l/by-sa/4.0/88x31.png" /></a><br /><span xmlns:dct="http://purl.org/dc/terms/" href="http://purl.org/dc/dcmitype/Text" property="dct:title" rel="dct:type">What is Domain-Driven Design (DDD)</span> by <a xmlns:cc="http://creativecommons.org/ns#" href="https://verraes.net/2021/09/what-is-domain-driven-design-ddd/" property="cc:attributionName" rel="cc:attributionURL">Mathias Verraes</a> is licensed under a <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/">Creative Commons Attribution-ShareAlike 4.0 International License</a>.</div>
</blockquote>
<p><a href="https://verraes.net/What_is_Domain_Driven_Design.pdf">Download as PDF</a></p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p><a href="https://amzn.to/3lQOFlH">“Domain-Driven Design - Tackling Complexity in the Heart of Software”</a> — Eric Evans, Addison-Wesley Professional, 2003 <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Design and Reality2021-09-13T00:00:00+00:00https://verraes.net/2021/09/design-and-reality<blockquote>
<p>“The transition to a really deep model is a profound shift in your thinking and demands a major change to the design.” — <a href="https://amzn.to/3ljYdp5">Domain-Driven Design</a>, Eric Evans</p>
</blockquote>
<p>There is a fallacy about how domain modelling works. The misconception is that we can design software by discovering all the relevant concepts in the domain, turn them into concepts in our design, add some behaviours, and voilà, we’ve solved our problem. It’s a simplistic perception of how design works: a linear path from A to B:</p>
<ol>
<li>understand the problem,</li>
<li>apply design,</li>
<li>end up with a solution.</li>
</ol>
<p>That idea was so central to early Object-Oriented Design, that one of us (Rebecca) thought to refute it in her book:</p>
<blockquote>
<p>“Early object design books including [my own] Designing Object-Oriented Software [Wirfs-Brock 1990], speak of finding objects by identifying things (noun phrases) written in a design specification. In hindsight, this approach seems naive. Today, we don’t advocate underlining nouns and simplistically modeling things in the real world. It’s much more complicated than that. Finding good objects means identifying abstractions that are part of your application’s domain and its execution machinery. Their correspondence to real-world things may be tenuous, at best. Even when modeling domain concepts, you need to look carefully at how those objects fit into your overall application design.” — <a href="https://www.informit.com/promotions/object-design-142314">Object Design</a>, Rebecca Wirfs-Brock</p>
</blockquote>
<h2 id="domain-language-vs-ubiquitous-language">Domain language vs Ubiquitous Language</h2>
<p>The idea has persisted in many naive interpretations of Domain-Driven Design as well. Domain language and Ubiquitous Language are often conflated. They’re not the same.</p>
<p>Domain language is what is used by people working in the domain. It’s a natural language, and therefore messy. It’s organic: concepts are introduced out of necessity, without deliberation, without agreement, without precision. Terminology spreads across the organisation or fades out. Meaning shifts. People adapt old terms into new meanings, or terms acquire multiple, ambiguous meanings. It exists because it works, at least well enough for human-to-human communication. A domain language (like all language) only works in the specific context it evolved in.</p>
<p>For us system designers, messy language is not good enough. We need precise language with well understood concepts, and explicit context. This is what a Ubiquitous Language is: a constructed, formalised language, agreed upon by stakeholders and designers, to serve the needs of our design. We need more control over this language than we have over the domain language. The Ubiquitous Language has to be deeply connected to the domain language, or there will be discord. The level of formality and precision in any Ubiquitous Language depends on its environment: a meme sharing app and an oil rig control system have different needs.</p>
<h2 id="drilling-mud">Drilling Mud</h2>
<p>Talking of oil rigs:</p>
<p>Rebecca was invited to consult for a company that makes hardware and software for oil rigs. She was asked to help with object design and modelling, working on redesigning the control system that monitors and manages sensors and equipment on the oil rig. Drilling causes a lot of friction, and “drilling mud” (a proprietary chemical substance) is used as a lubricant. It’s also used as a carrier for the rocks and debris you get from drilling, lifting it all up and out of the hole. Equipment monitors the drilling mud pressure, and by changing the composition of the mud during drilling, you can control that pressure. Too much pressure is a really bad thing.</p>
<p>And then an oil rig in the gulf exploded.</p>
<p>As the news stories were coming out, the team found out that the rig was using a competitor’s equipment. Whew! The team started speculating about what could have happened, and were thinking about how something like that could happen with their own systems. Was it faulty equipment, sensors, the telemetry, communications between various components, the software?</p>
<h2 id="scenarios">Scenarios</h2>
<p>When in doubt, look for examples. The team ran through scenarios. What happens when a catastrophic condition occurs? How do people react? When something fails, it’s a noisy environment for the oil rig engineers: sirens blaring, alarms going off, … We discovered that when a problem couldn’t be fixed immediately, the engineers, in order to concentrate, would turn off the alarms after a while. When a failure is easy to fix, the control system logs reflect that the alarm went on and was turned off a few minutes later.</p>
<p>But for more consequential failures, even though these problems take much longer to resolve, it still shows up on the logs as being resolved within minutes. Then, when people study the logs, it looks like the failure was resolved quickly. But that’s totally inaccurate. This may look like a software bug, but it’s really a flaw in the model. And we should use it as an opportunity to improve that model.</p>
<p>The initial modelling assumption is that alarms are directly connected to the emergency conditions in the world. However, the system’s perception of the world is distorted: when the engineers turn off the alarm, the system believes the emergency is over. But it’s not, turning an alarm off doesn’t change the emergency condition in the world. The alarms are only indirectly connected to the emergency. If it’s indirectly connected, there’s something else in between, that doesn’t exist in our model. The model is an incomplete representation of a fact of the world, and that could be catastrophic.</p>
<h2 id="a-breakthrough">A Breakthrough</h2>
<p>The team explored scenarios, specifically the weird ones, the awkward edge cases where nobody really knows how the system behaves, or even how it should behave. One such scenario is when two separate sensor measurements raise alarms at the same time. The alarm sounds, an engineer turns it off, but what happens to the second alarm? Should the alarm still sound or not? Should turning off one turn off the other? If it didn’t turn off, would the engineers think the off switch didn’t work and just push it again?</p>
<p>By working through these scenarios, the team figured out there was a distinction between the alarm sounding, and the state of alertness. Now, in this new model, when measurements from the sensors exceed certain thresholds or exhibit certain patterns, the system doesn’t sound the alarm directly anymore. Instead, it raises an alert condition, which is also logged. It’s this alert condition that is associated with the actual problem. The new alert concept is now responsible for sounding the alarm (or not). The alarm can still be turned off, but the alert condition remains. Two alert conditions with different causes can coexist without being confused by the single alarm. This model decouples the emergency from the sounding of the alarm.</p>
<p>The old model didn’t make that distinction, and therefore it couldn’t handle edge cases very well. When at last the team understood the need for separating alert conditions from the alarms, they couldn’t unsee it. It’s one of those aha-moments that seem obvious in retrospect. Such distinctions are not easily unearthed. It’s what Eric Evans calls a Breakthrough.</p>
<h2 id="an-act-of-creation">An Act of Creation</h2>
<p>There was a missing concept, and at the first the team didn’t know something was missing. It wasn’t obvious at first, because there wasn’t a name for “alert condition” in the domain language. The oil rig engineers’ job isn’t designing software or creating a precise language, they just want to be able to respond to alarms and fix problems in peace. Alert conditions didn’t turn up in a specification document, or in any communication between the oil rig engineers. The concept was not used implicitly by the engineers or the software; no, the whole concept did not exist.</p>
<p>Then where did the concept come from?</p>
<p>People in the domain experienced the problem, but without explicit terminology, they couldn’t express the problem to the system designers. So it’s us, the designers, who created it. It’s an act of creative modelling. The concept is invented. In our oil rig monitoring domain, it was a novel way to perceive reality.</p>
<p>Of course, in English, alert and alarm exist. They are almost synonymous. But in our Ubiquitous Language, we agreed to make them distinct. We designed our Ubiquitous Language to fit our purpose, and it’s different from the domain language. After we introduced “alert conditions”, the oil rig engineers incorporated it in their language. This change in the domain is driven by the design. This is a break with the linear, unidirectional understanding of moving from problem to solution through design. Instead, through design, we reframed the problem.</p>
<h2 id="is-it-a-better-model">Is it a better model?</h2>
<p>How do we know that this newly invented model is in fact better (specifically, more fit for purpose)? We find realistic scenarios and test them against the alert condition model, as well as other candidate models. In our case, with the new model, the logs will be more accurate, which was the original problem.</p>
<p>But in addition to helping with the original problem, a deeper model often opens new possibilities. This alert conditions model suggests several:</p>
<ul>
<li>Different measurements can be associated with the same alert.</li>
<li>Alert conditions can be qualified.</li>
<li>We can define alarm behaviours for simultaneous alert conditions, for example by spacing the alarms, or picking different sound patterns.</li>
<li>Critical alerts could block less critical ones from hogging the alarm.</li>
<li>Alert conditions can be lowered as the situation improves, without resolving them.</li>
<li>…</li>
</ul>
<p>These new options are relevant, and likely to bring value. Yet another sign we’d hit on a better model is that we had new conversations with the domain experts. A lot of failure scenarios became easier to detect and respond to. We started asking, what other alert conditions could exist? What risks aren’t we mitigating yet? How should we react?</p>
<h2 id="design-creates-new-realities">Design Creates New Realities</h2>
<p>In a world-centric view of design, only the sensors and the alarms existed in the real world, and the old software model reflected that accurately. Therefore it was an accurate model. The new model that includes alerts isn’t more “accurate” than the old one, it doesn’t come from the real world,it’s not more realistic, and it isn’t more “domain-ish”. But it is more useful. Sensors and alarms are objective, compared to alert conditions. Something is an alert condition because in this environment, we believe it should be an alert condition, and that’s subjective.</p>
<p>The model works for the domain and is connected to it, but it is not purely a model <em>of</em> the problem domain. It addresses the problems in the contexts we envision better. The solution clarified the problem. Having only a real world focus for modelling blinds us to better options and innovations.</p>
<p>These creative introductions of novel concepts into the model are rarely discussed in literature about modelling. Software design books talk about turning concepts into types and data structures, but what if the concept isn’t there yet? Forming distinctions, not just abstractions, however, can help clarify a model. These distinctions create opportunities.</p>
<p>The model must be radically about its utility in solving the problem.</p>
<blockquote>
<p>“Our measure of success lies in how clearly we invent a software reality that satisfies our application’s requirements—and not in how closely it resembles the real world.” — <a href="https://www.informit.com/promotions/object-design-142314">Object Design</a>, Rebecca Wirfs-Brock</p>
</blockquote>
<p><em>Written by <a href="https://twitter.com/mathiasverraes">Mathias Verraes</a> and <a href="https://twitter.com/rebeccawb">Rebecca Wirfs-Brock</a>. Special thanks to <a href="https://twitter.com/ericevans0">Eric Evans</a> for the spot on feedback and constructive advice.</em></p>
<h2 id="updates">Updates</h2>
<ul>
<li>Mathias discusses this essay in a <a href="https://verraes.net/2022/02/design-and-reality-virtualddd/">VirtualDDD session</a>.</li>
<li>Alex & Adrian Bolboacă made a <a href="https://www.youtube.com/watch?v=-bWrZYtKNuE">reaction video about this article titled “DDD vs the Real World”</a></li>
</ul>
<div class="columns mb-6 ">
<div class="column is-6">
<figure class="media-left image " style="width: 75%; box-shadow: 2px 2px 7px darkgray">
<a href="https://leanpub.com/design-and-reality">
<img src="https://d2sofvawe08yqg.cloudfront.net/design-and-reality/s_hero2x?1664874345" />
</a>
</figure>
</div>
<div class="column is-6">
<h1 class="title">Design and Reality</h1>
<h2 class="subtitle">Essays on Software Design</h2>
<p>by Mathias Verraes & Rebecca Wirfs-Brock</p>
<br />
<a class="button is-info" href="https://leanpub.com/design-and-reality">
Get it on Leanpub
</a>
<br />
<br />
<blockquote>
“Happily purchased. Handy to have these in one place. Thank you!” <br />— Kent Beck<br />
</blockquote>
<br />
<blockquote>
“Excellent new thinking on Domain-Driven Design. It's full of real practical experience in getting the most
value from domain modelling. Just like the Eric Evans' DDD book, this gives more insight each time you read
it.”<br />— Ian Russell<br />
</blockquote>
</div>
</div>
Splitting a Domain Across Multiple Bounded Contexts2021-06-14T00:00:00+00:00https://verraes.net/2021/06/split-domain-across-bounded-contexts<p>Imagine a wholesaler of parts for agricultural machines. They’ve built a B2B webshop that resellers and machine servicing companies use to order. In their Ubiquitous Language, an Order represents this automated process. It enables customers to pick products, apply the right discounts, and push it to Shipment.</p>
<p>Our wholesaler merges with a competitor: They’re an older player with a solid customer base and a huge catalog. They also have an ordering system, but it’s much more traditional: customers call, and an account manager enters the order, applies an arbitrary discount, and pushes it to Shipment.</p>
<p>The merged company still has a single Sales Subdomain, but it now has two Sales Bounded Contexts. They both have concepts like Order and Discount in their models, and these concepts have fundamentally the same meaning. The employees from both wholesalers agree on what an order or a discount is. But they have different processes for using them, they enter different information in the forms, and there are different business rules.</p>
<p>In the technical sense that difference is expressed in the object design, the methods, the workflows, the logic for applying discounts, the database representation, and some of the language. It runs deeper though: for a software designer to be productive in either Bounded Context, they’d have to understand the many distinctions between the two models. Bounded Contexts represent Understandability Boundaries.</p>
<p>In a perfectly designed system, our ideal Bounded Contexts would usually align nicely with the boundaries of the subdomains. In reality though, Bounded Contexts follow the contours of the evolution of the system. Systems evolve along with the needs and opportunities of organisations. And unfortunately, needs and opportunities don’t often arise in ways that match our design sensibilities. We’re uncomfortable with code that could be made more consistent, if only we had the time to do so. We want to unify concepts and craft clean abstractions, because we think that is what we should do to create a well-designed system. But that might not be the better option.</p>
<h2 id="deliberate-design-choices">Deliberate Design Choices</h2>
<p>The example above trivially shows that a single subdomain may be represented by multiple Bounded Contexts.</p>
<p>Bounded Contexts may or may not align with app or service boundaries. Similarly, they may or may not align with domain boundaries. Domains live in the problem space. They are how an organisation perceives its areas of activity and expertise. Bounded Contexts are part of the solution space; they are deliberate design choices. As a systems designer, you choose these boundaries to manage the understandability of the system, by using different models to solve different aspects of the domain.</p>
<p>You might argue that in the wholesaler merger, the designers didn’t have a choice. It’s true that the engineers didn’t choose to merge the companies. And there will always be external triggers that put constraints on our designs. At this point however, the systems designers can make a case for:</p>
<ul>
<li>merging the two Sales Contexts,</li>
<li>migrating one to the other,</li>
<li>building a new Sales Context to replace both,</li>
<li>postponing this effort,</li>
<li>or not doing anything and keeping the existing two Contexts.</li>
</ul>
<p>These are design choices, even if ultimately the CEO picks from those options (because of the expected ROI for example). Perhaps, after considering the trade-offs, keeping the two Sales Contexts side by side is the best strategic design choice for now, as it allows the merged company to focus on new opportunities. After all, the existing systems do serve their purpose. The takeaway here is that having two Bounded Contexts for a single Subdomain can be a perfectly valid choice.</p>
<h2 id="twenty-commodities-traders">Twenty Commodities Traders</h2>
<p>When there is no external trigger (as in the wholesaler merger), would you ever choose to split a single domain over multiple Contexts, deliberately?</p>
<p>One of us was brought in to consult for a small commodities trader. They were focused on a few specialty commodities, and consisted of 20 traders, some operational support roles, and around 10 developers. The lead engineer gave a tour of the system, which consisted of 20 Bounded Contexts for the single Trading Domain. There was one Context for each trader.</p>
<p>This seemed odd, and our instinct was to identify similarities, abstract them, and make them reusable. The developers were doing none of that. At best they were copy-pasting chunks of each other’s code. The lead engineer’s main concern was <em>“Are we doing the optimal design for our situation?”</em> They were worried they were in a big mess.</p>
<p>Were they?</p>
<p>Every trader had their own representation of a trade. There were even <a href="https://verraes.net/2019/06/emergent-contexts-through-refinement/">multiple representations of money</a> throughout the company. The traders’ algorithms were different, although many were doing somewhat similar things. Each trader had a different dashboard. The developers used the same third party libraries, but when they shared their own code between each other, they made no attempt at unifying it. Instead, they copied code, and modified it over time as they saw fit. A lot of the work involved mathematical algorithms, more than typical business oriented IT.</p>
<p>It turned out that every trader had unique needs. They needed to move fast: they experimented with different algorithms, projections, and ways of looking at the market. The developers were serving the traders, and worked in close collaboration with them, constantly turning their ideas into new code. The traders were highly valued, they were the prima donnas in a high stress, highly competitive environment. You couldn’t be a slow coder or a math slacker if you wanted to be part of this. There were no (Jira) tickets, no feature backlogs. It was the ultimate fast feedback loop between domain experts and programmers.</p>
<p>Things were changing very rapidly, every day. Finding the right abstractions would have taken a lot of coordination, it would slow development down drastically. An attempt at unifying the code would have destroyed the company.</p>
<p>This design was not full of technical debt. It also wasn’t a legacy problem, where the design had accidentally grown this way over the years. The code was working. This lack of unifying abstractions was a deliberate design choice, fit for purpose, even if it seems like a radical choice at first. And all the developers and traders were happy with it.</p>
<p>These weren’t merely separate programs with a lot of repeated code either. This was a single domain, split over 20 Bounded Contexts, each with their own domain model, their own Ubiquitous Language, and their own rate of change. Coordinating the language and concepts in the models, would have increased the friction in this high speed environment. By deliberately choosing to design individual Contexts, they eliminated that friction.</p>
<h2 id="trade-offs">Trade-offs</h2>
<p>There are consequences of this design choice: When a developer wanted help with a problem, they had to bring that other developer up to speed. Each developer, when working in another bounded context, expected that they’d have to make a context switch. After all, their terms and concepts were different from each other, even though they shared similar terminology. Context-switching has a cost, which you’ve probably experienced if you’ve work on different projects throughout a day. But here, because the Contexts were clearly well-bounded, this didn’t cause many problems. And sometimes, by explaining a problem to another developer with a similar background (but a different Bounded Context), solutions became obvious.</p>
<h2 id="multiple-bounded-contexts-in-ordinary-it-systems">Multiple Bounded Contexts in Ordinary IT Systems</h2>
<p>The trading system is an extreme example, and you won’t come across many environments where a single Subdomain with 20 Bounded Contexts would make sense. But there are many common situations where you should consider splitting a domain. If in your company, the rules about pricing for individual and corporate customers are different, perhaps efforts to unify these rules in a single domain model will cost more than it is worth. Or in a payroll system, where the rules and processes for salaried and hourly employees are different, you might be better off splitting this domain.</p>
<h2 id="conclusion">Conclusion</h2>
<p>The question is not: Can I unify this? Of course you can. But should you? Perhaps not. The right Context boundaries follow the contours of the business. Different areas change at different times and at different speeds. And over time what appears similar may diverge in surprising and unexpectedly productive ways (if given the opportunity). Squeezing concepts into a single model is constraining. You’re complicating your model by making it serve two distinct business needs, and taking a continued coordination cost. It’s a hidden dependency debt.</p>
<p>There’s two heuristics we can derive here:</p>
<ol>
<li>Bounded Contexts shouldn’t serve the designer’s sensibilities and need for perfection, but enable business opportunities.</li>
<li>The Rate of Change Heuristic: Consider organising Bounded Contexts so they manage related concepts that change at the same pace.</li>
</ol>
<p><em>Written by Mathias Verraes <a href="https://twitter.com/mathiasverraes">@mathiasverraes</a> and Rebecca Wirfs-Brock <a href="https://twitter.com/rebeccawb">@rebeccawb</a>.</em></p>
<div class="columns mb-6 ">
<div class="column is-6">
<figure class="media-left image " style="width: 75%; box-shadow: 2px 2px 7px darkgray">
<a href="https://leanpub.com/design-and-reality">
<img src="https://d2sofvawe08yqg.cloudfront.net/design-and-reality/s_hero2x?1664874345" />
</a>
</figure>
</div>
<div class="column is-6">
<h1 class="title">Design and Reality</h1>
<h2 class="subtitle">Essays on Software Design</h2>
<p>by Mathias Verraes & Rebecca Wirfs-Brock</p>
<br />
<a class="button is-info" href="https://leanpub.com/design-and-reality">
Get it on Leanpub
</a>
<br />
<br />
<blockquote>
“Happily purchased. Handy to have these in one place. Thank you!” <br />— Kent Beck<br />
</blockquote>
<br />
<blockquote>
“Excellent new thinking on Domain-Driven Design. It's full of real practical experience in getting the most
value from domain modelling. Just like the Eric Evans' DDD book, this gives more insight each time you read
it.”<br />— Ian Russell<br />
</blockquote>
</div>
</div>
The “It's Just Like...” Heuristic2021-05-22T00:00:00+00:00https://verraes.net/2021/05/its-just-like-heuristic<p>There’s a common bias in the tech industry, and I imagine everywhere. When there’s a new idea, and some very smart people will say that the new idea X is really just the old idea Y. The idea could be a model for looking at things, a way of doing things, a technology, a pattern. People will make the case that X is nothing but Y in new packaging.</p>
<h2 id="brain-rewards">Brain Rewards</h2>
<p>Sometimes it really is just new packaging. But:</p>
<p>Your brain rewards you for classifying new information into existing buckets<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>. Looking for similarities costs less energy than understanding new differences. When you say “Oh I get it, it’s just like …”, then it probably is <em>not</em> “just like”. That’s very counterintuitive, because that <strong>“oh I get it” feeling is intuition, not reason</strong><sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>. You feel good about the insight, so it’s very tricky to then go “Oh I get it, so I must be wrong”.</p>
<p>Proclaiming that the new idea is the same as the old idea, <strong>gives you permission not to investigate it deeply</strong><sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>. You can stick to your existing worldview, in which you are an expert in the old idea. New ideas are merely repackaged old ones, not worthy of our consideration. Everything was already invented, nothing to see here, move along. Balance has been restored.</p>
<p>Sometimes this habit is intellectual laziness, sometimes it’s intellectual dishonesty. Sometimes there’s the undertone of superiority (“I’m still the smartest person in the room”). Sometimes it’s an unconscious attempt to preserve the power structures that are threatened by new ideas. In the worst cases, it’s a <em>deliberate</em> attempt.</p>
<p>But mostly, there’s such a firehose of information, that staying fully up to date on even a single topic is impossible. We should be forgiven for not knowing and understanding and keeping up with everything, even if our job is knowing and understanding and keeping up with everything.</p>
<h2 id="applying-the-its-just-like-heuristic">Applying the “It’s Just Like” Heuristic</h2>
<p>Still, there’s a remedy. We can use a low energy heuristic<sup id="fnref:3:1" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>, to catch our brain in the act of using a low energy classification. It works like this:</p>
<p>Be aware when you are presented with new information. <strong>When you feel “Oh I get it, it’s just like X”, then ask: “So what makes the new thing different from X?”</strong> You ask it to yourself, as a starting point for deeper analysis, and to keep you intellectually honest. And you ask it to others, because by giving them your frame of reference, they can explain the value of the idea better. And you can ask it when others dismiss an idea too eagerly.
This question has been a powerful learning tool for me.</p>
<h2 id="the-packaging-heuristic">The Packaging Heuristic</h2>
<p>A bonus heuristic: <strong>Sometimes the packaging <em>is</em> the new idea</strong>. New packaging gives old ideas new wings. The packaging in itself can be a valuable contribution. It can re-frame the problem, make the solution more appealing, highlight the benefits better, and get it in front of the right people. By dismissing it as mere packaging, we rob others of discovering it. We overvalue originality, but not everything needs to be original to be valuable. It takes a village to raise an idea.</p>
<h2 id="more-heuristics">More Heuristics</h2>
<ul>
<li><a href="/2021/03/heuristic-legacy-mirror/">The Legacy Mirror Heuristic</a> helps you evaluate new ideas by swapping their chronology.</li>
<li>I wrote about the “It’s Just Like…” Heuristic <a href="/2014/06/information-overload/">in 2014</a> as well, but the 2021 one here is much better.</li>
</ul>
<p><strong>Footnotes</strong></p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>Added 2022-09-02: A <a href="https://twitter.com/viktorcessan/status/1565641864748965892?s=20&t=29ceQi3jTDGI-RlsWiY2qw">commenter</a> informed me that this is called “assimilation” in Jean Piaget’s Theory of Cognitive Development. <a href="https://www.massey.ac.nz/~wwpapajl/evolution/assign2/DD/theory.html">Assimilation is defined</a> as “the cognitive process of fitting new information into existing cognitive schemas, perceptions, and understanding. Overall beliefs and understanding of the world do not change as a result of the new information.” <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>The low energy classification of information and the higher energy thinking fit in Daniel Kahneman’s two modes of thoughts. He calls them System 1 (fast, instinctive and emotional) and System 2 (slow, deliberative, logical). Coherence bias is at play here, and Reductionism, as well as Substitution bias, and “What You See Is All There Is”. See <a href="https://amzn.to/3vc9An1">“Thinking Fast & Slow” by Kahneman</a> <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>Fast and Frugal Heuristics limit the information search, and do not involve much computation. See <a href="https://amzn.to/3bINCQE">“Simple Heuristics That Make Us Smart” Gigerenzer, Todd et al.</a> <a href="#fnref:3" class="reversefootnote" role="doc-backlink">↩</a> <a href="#fnref:3:1" class="reversefootnote" role="doc-backlink">↩<sup>2</sup></a></p>
</li>
</ol>
</div>
Simple Models, Scaffolding, Enabling Constraints2021-04-23T00:00:00+00:00https://verraes.net/2021/04/simple-models-scaffolding-enabling-constraints<p>All models are wrong but simple models are more wrong than complex ones. Simple models are more appealing, easier to teach and spread and apply. Because of that, they can bring value faster, and they can cause harm faster. “Everything should be made as simple as possible, but no simpler.” (Einstein) is great advice, but the risk is that we forget the last bit. It’s not a joke, it’s a serious warning against reductionism.</p>
<h2 id="spotting-reductionism">Spotting reductionism</h2>
<p>Reductionism is looking at a problem space as a simple combination of parts, and ignoring hidden, but essential, complexities (eg in the relations between the parts). You end up with what looks like simple model but is actually a lossy model.</p>
<p>Luckily, reductionism is often easy to spot. It starts with “[Complex thing] is just a [simple thing]”. The “is just” gives it away. Reductionist models have “5 easy steps” or “3 secrets” or a convenient mnemonic acronym like E.A.S.Y.</p>
<p>Simple models that are not (overly) reductionist can also be spotted: they accept to be changed over time. They accept exceptions as inputs to refine the model. When an exception doesn’t fit, it isn’t discarded but embraced.</p>
<h2 id="how-to-use-a-simple-model">How to use a simple model</h2>
<p>However (and this is where I’m changing my opinion) <em>even</em> reductionist models have their use<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>, and not just for the teaching and spreading and applying part. There are some important ingredients here, which are not part of the reductionist model itself, but are about how <em>we</em> use the model: scaffolding and enabling constraints.</p>
<h2 id="enabling-constraints">Enabling Constraints</h2>
<p>Say a simple, reductionist model is expected to be good enough to cover 90% of cases. To evaluate our case, we try to fit it into the model. If it works, great, if not, we are either in a 10% case; or we are in a 90% case but we’re doing something wrong. In the 10% case, we need to put in the work to find better, richer, more suitable models. But because we’re more likely to be in the 90%, first we must look for problems in our information and in how we use the model.</p>
<p>This is how it becomes an enabling constraint then: The model offers a constraint on how to look at the problem. That enables you to see if you’re in a standard or exceptional situation, raises red flags if something is missing on your side, and helps you pick the right approach. Reductionist models aren’t usually presented to you that way. They should come with a sticker on the box: “Only applicable in 90% of cases. Use as enabling constraint only.”</p>
<h2 id="scaffolding">Scaffolding</h2>
<p>Scaffolding is the other ingredient. Scaffolding means that use the model to make progress fast in the beginning, use it as the fire starter for finding your own better models, and then get rid of the simple model.</p>
<p>This is basically a superpower. You start by not re-inventing the wheel, but you end up discovering novel solutions that address novel problems that the simple model’s author couldn’t possible foresee. Again, most of the time simple models aren’t sold to us explicitly as scaffolding. Perhaps the author, being a great problem solver, uses the model as scaffolding themselves, but it’s not labeled as such.</p>
<p>It is up to us to use them that way. Model authors can help though: they can include ways to get rid of the model as part of the model. “Here’s how you apply it, and here’s how you look for opportunities to evolve it, here’s how you get rid of it.”</p>
<h2 id="our-relation-to-simple-models">Our relation to simple models</h2>
<p>Ironically, by adding these ingredients to the model itself, the model becomes more complex (that is, it addresses more of the inherent complexity in the problem space). And complex models are, although less wrong, harder to teach, spread, and apply!</p>
<p>I used to blame the authors, but (and this is also something where I’m changing my mind) I think all these effects are inevitable. Simple models are more likely to become popular.</p>
<p><strong>The work of using simple models as enabling constraints and scaffolding is up to us.</strong></p>
<p>Sometimes you’ll want to follow the recipe. A recipe can be a great tool to understand why the soufflé doesn’t rise, and a kickstarter to create your own dishes. And if you follow the recipe, remember to taste the food before you serve it.</p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>This blog post was triggered by a conversation with <a href="https://twitter.com/matthewpskelton">Matthew Skelton</a> last night, and a conversation with <a href="https://twitter.com/conways_law">Dr. Mel Conway</a> years ago that took a bit longer to digest :-) <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Tensions when Designing Evolvable Bounded Contexts2021-04-09T00:00:00+00:00https://verraes.net/2021/04/tensions-when-designing-evolvable-bounded-contexts<p>(This is a small snapshot from a longer conversation I’m having with <a href="https://twitter.com/rebeccawb">Rebecca Wirfs-Brock</a> about Bounded Contexts. It’s very incomplete but I’m posting it anybody who get some value from it in its present form.)</p>
<h3 id="context">Context</h3>
<ul>
<li>A Bounded Context is an “understandability boundary”, a boundary around a model and its language. You can understand the model and the language in isolation, without having to understand other Bounded Contexts.</li>
<li>An Interface is the set of contracts or message types or APIs between Bounded Contexts. They translate from one model and language to another.</li>
<li>A Bounded Context or Interface is evolvable if we are able to make frequent, safe, and cheap changes to it.</li>
</ul>
<h3 id="tensions">Tensions</h3>
<p>When designing evolvable Bounded Contexts (BCs), there are fundamental tensions that affect our design.</p>
<ol>
<li>A BC is evolvable on the inside if it is small and understandable and has a small stable Interface.</li>
<li>A system of BCs is evolvable (adding, removing, reconfiguring BCs) if the Interfaces are small and understandable.</li>
<li>The tension between 1 and 2 is that choosing many small BCs implies having larger Interfaces, but choosing many small Interfaces implies having larger BCs.</li>
<li>The Interfaces themselves are more evolvable if they integrate fewer BCs, and less evolvable if they integrate more BCs.</li>
<li>The consequence of 2 and 4 is that a small evolvable Interface makes adding BCs easier, but becomes less evolvable in the process.</li>
<li>It’s easier for a BC to offer a single generic shared Interface, than many specialised Interfaces for each consumer. Having many specialised Interfaces is harder to maintain when the BC changes.</li>
<li>It’s easier for a BC to consume a specialised Interface, adapted to that BC. Generic interfaces are bloated because they support other consumers, and therefore require more integration work.</li>
<li>Trading off 6 and 7 involves finding a small set of semi-specialised Interfaces, that are shared between some consumers with similar needs.</li>
<li>Coordination between teams (which is in fact coordination of understanding, aka sharing models and language) is expensive, which is one of the reasons we have BCs to begin with.</li>
<li>Achieving 8. requires more coordination between more teams than either a single generic Interface or individual specialised Interface.</li>
</ol>
<p>There are no solutions or hard rules to make these trade-offs. Instead, you need to use heuristics for making the trade-offs, and heuristics for knowing when to choose different trade-offs than the ones you originally made.</p>
The Legacy Mirror Heuristic2021-03-14T00:00:00+00:00https://verraes.net/2021/03/heuristic-legacy-mirror<blockquote>
<p><em>Decision-making heuristics are patterns and ideas that can help us find another angle, generate new options, evaluate them, or otherwise address the problem. They don’t guarantee a good answer, they only give you some possible progress towards an answers.</em></p>
</blockquote>
<p>The Legacy Mirror Heuristic<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> helps you to evaluate whether to adopt a new idea, method, or technology. It works by imagining the new idea has actually been the mainstream choice for the past 30 years, and you just learned about the old idea for the first time.</p>
<h2 id="example-eventsourcing">Example: EventSourcing</h2>
<p>(EventSourcing is a relatively niche way to store and work with data, based on a very old<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup> idea.)</p>
<p>A recurring question is whether to adopt EventSourcing in favour of more mainstream data storage. The common hot take is some form of “Use a relational database unless you have a very specific reason for EventSourcing”. The lack of literature, the slow adoption curve, the learning cost, and the availability of tooling, are cited as reasons to avoid it. That specific reason, then, is that the problem involves some temporal aspect.</p>
<p>Using the Legacy Mirror, we swap EventSourcing and relational databases in time. It’s speculative of course, but I think we’d be getting hot takes like “Use EventSourcing everywhere, unless you have a very specific use case, involving relations between data, that can only be solved using a relational database. Is the extra complexity that relational brings worth the efforts, such as the need for ORMs with their impedance mismatch and leaky abstractions, the costly migrations, the lack of tooling and literature, and the learning curve?” There would of course be plenty of EventSourcing tooling and literature, students would learn it in school, the <a href="https://verraes.net/2019/05/ddd-msg-arch/">patterns and anti-patterns would be widely documented</a>.</p>
<p>I’m not saying the answer is now automatically to choose EventSourcing. Debates on technology choices are often blinded: one side sees the shiny new silver bullet, the other side is so invested in and well adapted to the downsides of the old method that they forgot that it, well, sucks.</p>
<h2 id="discussion">Discussion</h2>
<p>I first used this to help clients decide on a technical choice. Like many of the heuristics I use, it started organically. Later I saw that it was a reusable pattern, and can be applied to all kinds of decision-making.</p>
<p>Don’t use the Legacy Mirror as a rhetoric device though. As a heuristic, a tool to introduce new lines of thought into the debate. But it evens the playing field: both the new and the old idea can be evaluated on merit, instead of purely the benefits and downsides of age.</p>
<p>And, often, the fact that something is new, is precisely the reason not to do it: you may not want to be a trailblazer on something that is not core to your business. (That’s another heuristic for you.)</p>
<p>As always, identify competing heuristics: “X is not our core business, but doing X will attract engineering talent, which will help us achieve our core business goals” is also valid.</p>
<h2 id="example-work-from-home">Example: Work from home</h2>
<p>“Should we go back to our offices post-COVID?” What if working from home was the norm, and the hot new trend was this new-fangled idea called “offices”?</p>
<p>People would think it’s silly to commute all the time to a different location, when you have a nice office set up at home. They’d cite the waste of time, the environmental impact, the fact that offices eat up real estate, the risks of transmitting diseases. Others would report on the benefits on in-person collaboration, the social contact, buying furniture in bulk, getting your laptop repaired by the IT person on the next floor. The outcome would not be “offices with special accommodation for working from home”, but “work from home with occasional in-person meetings”.</p>
<h2 id="example-crypto">Example: Crypto</h2>
<ul>
<li>Decision: Should I trade fiat currency for cryptocurrencies?</li>
<li>Legacy Mirror: What if all money had been crypto for a long time, and fiat money was new on the scene?</li>
<li>Scenario: Massive amounts of fossil fuel would be burnt, and causing irreparable damage to the ecosystem and life on earth. (I don’t accept the notion that we would have evolved and massively adopted ecological alternatives, for the simple reason that there is no evidence so far that humanity is capable of massively adopting ecological alternatives for anything. That’s the Historical Perspective Heuristic.)
Conclusion: Don’t buy cryptocurrencies.</li>
</ul>
<h2 id="more-heuristics">More Heuristics</h2>
<ul>
<li><a href="/2021/05/its-just-like-heuristic/">The “It’s Just Like…” Heuristic</a> for figuring out if you understood a new idea well enough.</li>
</ul>
<h2 id="footnotes">Footnotes</h2>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>Thanks to <a href="https://twitter.com/CiaranMcNulty/status/1371083860323278849">Ciaran McNulty</a> for naming the heuristic. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>The oldest known recorded ledgers are 7000 years old and were found in Mesopotamia (today’s Iraq). <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
The Wall of Technical Debt2020-01-22T00:00:00+00:00https://verraes.net/2020/01/wall-of-technical-debt<p>“Technical debt” is a metaphor for all software design choices that turn out to be suboptimal, no longer valid, or just plain wrong. These choices incur a cost on future development, and the shortcuts taken today will later slow you down until you “pay back” the debt by fixing the problems. And it’s not only code: Artifacts like architecture, documentation, tests, and domain models can all suffer from technical debt.</p>
<p>Technical debt can severely drag down development. And paying back all debt by replacing all software isn’t usually very feasible either. But, like financial debt, technical debt is not always a bad thing. <strong>Debt allows you to invest in something before it makes money.</strong> Taking shortcuts when bringing a product to market allows you to test a business model cheaply, and it makes it easier to throw away the code if it turns out to be a bad idea. And often, the debt exists because the business has grown and evolved, and the design choices that were valid early on no longer are.</p>
<p>All this to say that the problem isn’t technical debt; it’s <em>unmanaged</em> technical debt. In any company, the CFO knows exactly how much financial debt there is. There are spreadsheets, quarterly reports, payment plans, and options to refinance or sell debt. But ask your CTO how much technical debt your organization has, and you’ll get an awkward “uh… a lot?” as an answer.</p>
<p>So how do you manage debt in a fast-moving agile project?</p>
<p>The <strong>Wall of Technical Debt</strong> is a surface in your office where you visualize issues on sticky notes. It’s easy to start and maintain, and yet it has a profound impact on how you choose to add, reduce, pay back, or ignore debt. It’s by no means intended as a complete solution to scale the management of debt, but it works precisely because it requires no buy-in.</p>
<h3 id="creating-your-own-wall">Creating Your Own Wall</h3>
<p>Creating your own Wall of Technical Debt is simple: Make the debt visible, build a habit, negotiate regularly, and give away control.</p>
<p><img style="margin-right: 10px" src="/img/posts/2020-01-22-wall-of-technical-debt/pelshoff2-edit.jpeg" alt="Example of a Wall of Technical Debt, from @Pelshoff" />
<br /><br />
<small>Image by <a href="https://twitter.com/Pelshoff/status/1220291781188866049?s=20">Pim Elshoff</a>. Licensed under a <a href="http://creativecommons.org/licenses/by/4.0/">Creative Commons Attribution 4.0 International License</a>.</small></p>
<p><br /><br /></p>
<h4 id="make-it-visible">Make It Visible</h4>
<p>Pick a wall in the space where your team works. Make sure it’s publicly visible. Label it the “Wall of Technical Debt.” Draw a nice dramatic logo. Make sure there are sticky notes and markers around so people can easily add to it.</p>
<h4 id="build-a-habit">Build a Habit</h4>
<p>Whenever you work on code and encounter technical debt:</p>
<ul>
<li>Ask yourself: <strong>What made you slow? What made the code difficult to understand? What caused a specific bug to be hard to track down? What should have been better documented or tested?</strong> Write down any technical debt you encountered on a sticky note. Keep it short, but make sure you’ll be able to understand it later.</li>
<li>Estimate the opportunity cost, i.e. the time you would have spent on something else if the issue at hand didn’t exist. Write it on the sticky note. Agree on a notation for time, such as tally marks or dots per half day. Being visual is more important than being precise.</li>
<li>Estimate how long it would take to fix it, and write that down as well.</li>
<li>Optionally, write down how to fix the technical debt. You’ll probably want to do this part in an issue tracker, so if you do, write the issue ID on the sticky note.</li>
<li>Put the sticky note on the Wall of Technical Debt.</li>
<li>Whenever someone runs into the same issue, add more marks to represent the time they lost.</li>
<li>Over time, you’ll want to add some categorisation and structure. Let these emerge, resist organising it prematurely.</li>
</ul>
<p>Don’t forget to also add sticky notes whenever you introduce new technical debt yourself. And be honest. There’s no shame in this, because <em>you are not your code</em>. Remember technical debt is an investment, not a crime — unless you’re hiding debt, in which case you’re cooking the books!</p>
<p>When this becomes a habit, the wall becomes an information radar of the state of your system.</p>
<h4 id="negotiate-regularly">Negotiate Regularly</h4>
<p>As the Wall of Technical Debt grows, people in your organization might get a little nervous. That’s OK! The debt was always there; you only made it visible. Hopefully some of the habits start changing.</p>
<p>Whenever someone gets up to add another dot to an existing sticky note, discuss it with the team. Compare the cost of the measured wasted time to the estimated time to fix the debt. Sure, it might take a day to fix the code, but if you keep losing two hours over it every week, the time spent fixing it will pay for itself soon enough.</p>
<p>Or, perhaps fixing it isn’t worth it. That’s no longer a matter of aesthetics or opinion. Rather, it’s a matter of empirically collected metrics. Congratulations, you’re negotiating tradeoffs as a team.</p>
<p>Meanwhile, when someone adds a sticky for newly discovered existing debt, consider if it’s something the team could fix right away. Of course, it’s perfectly fine to put it on the wall; that’s what it’s there for. However, when you’re introducing new debt, it’s different. Consider if the team could choose not to take the shortcut and instead fix the code while it’s still cheap. The question shouldn’t be “Do we have time now?” It should be “Is it worth paying interest on this later? How much?” The answer could range anywhere from “It doesn’t matter, just commit it,” to “Lots of things will depend on this, so let’s get it right before moving on.”</p>
<p>The point is not to fix all technical debt. <strong>The point is to learn to negotiate when to add it, when to fix it, and when to avoid it.</strong> Before, this happened mostly invisibly: Individual team members added or fixed debt without knowing all the facts. Now, the collective brain power of the team can be applied, using real data.</p>
<h4 id="give-away-control">Give Away Control</h4>
<p>Now we get to the real magic. Because the Wall of Technical Debt is so visible, managers should start to take notice. (If they still don’t, use a fat red marker to add some € or $ signs to the Wall. Money is the universal language, and <em>lost money</em> stings twice as hard.)</p>
<p>Normally, when a team asks for time to spend on technical debt, there’s no direct tangible benefit to the users or the business. A manager can’t decide whether those fixes are of critical importance, an attempt to play with shiny new tech, or a desire to have prettier code. Even for a manager with a deep technical background, if they’re not directly involved with the code, it’s impossible for them to judge the importance of a fix.</p>
<p>But with the Wall of Technical Debt, it’s different. Now it’s not about opinions. It’s not about the desire to be a craftsperson who writes elegant code. It’s not about solving challenging puzzles. This time, it’s about facts, numbers, debt and interest payments, and returns on investment. You’re speaking the manager’s language. Good code as an asset, bad code as a liability. They’re trained to look at the numbers and make decisions. They’ve got the methods for it: CBA, OODA, PCDA, SWOT, …</p>
<p>So now, give away control. The manager knows more about the company strategy, the budgets, the risks. Let them decide, with your help, what to fix and when to plan it. The numbers enable that.</p>
<h3 id="tips-and-traps">Tips and Traps</h3>
<ul>
<li>Don’t start with a complete audit of all your debt. Sure, you could spend a few days identifying all debt. But then you need to get buy-in, and set up dates, and you’re making it a big thing. “We’ll do it during Summer” or “after the big project is finished” or whatever the local euphemism for “never” is. Instead, a single sticky note on the wall today brings value today. You can always do your big audit later; just don’t make it a bottleneck.</li>
<li>Without tally marks or dots that represent real incurred costs, you’re back at opinion, and some of the things that bother you may not actually impact future development. This is why real-time tracking is important. By only adding problems when you run into them, your Wall of Technical Debt will represent actual time wasted and not aesthetic problems with no impact on development. This is important because you want to build a habit of negotiating debt, and not simply make a one-off effort and then go back to the old ways.</li>
<li>It works the other way around as well. If you encounter ugly or strange code but you didn’t lose any time over it, don’t mark it. We’re not measuring whether you like it, only whether it costs time.</li>
<li>Don’t skip the physical wall. Resist the urge to put everything in a digital tool such as a bug tracker. Logs and metrics only have an impact when people look at them. Digital tools make it easy to hide things, whereas the wall is visible — confrontational even.</li>
</ul>
<h3 id="conclusion">Conclusion</h3>
<p>I started writing and speaking about the Wall of Technical Debt in 2013, after my team and I had developed a first version. One of the first companies that adapted our approach, was a small startup I was consulting for. The little wall space they had was already used by business plans and app mockups. So the team put the sticky notes on the office’s only window. The joke became that whenever the room got too dark, they knew it was time to refactor!</p>
<p>More importantly, they had been stuck alternating between perfectionism and hacking things to get to market faster. The Wall of Technical Debt helped them break out of that cycle, and they launched soon after. I’ve since heard stories of teams in all sorts of companies using it successfully. Perhaps you could be next.</p>
<h3 id="credits">Credits</h3>
<p>I wrote this article for <a href="https://leanpub.com/visualcollaborationtools"><em>Visual Collaboration Tools</em></a>, a book that Kenny Baas-Schwegler and João Rosa are editing. It’s loosely based on <a href="https://verraes.net/2013/07/managed-technical-debt/">“Managed Technical Debt”, Verraes 2013</a>. Thanks to Indu Alagarsamy, Rebecca Wirfs-Brock, and Natalye Childress, for reviewing the drafts.</p>
Eventsourcing: State from Events or Events as State?2019-08-24T00:00:00+00:00https://verraes.net/2019/08/eventsourcing-state-from-events-vs-events-as-state<p>Trying to get people to agree on a single unambiguous definition of a software concept is usually impossible. In the very least, I can point out that such ambiguity exists when it comes to eventsourcing.</p>
<h2 id="state-from-events">State from Events</h2>
<p>For a given stream of events, I can process them to derive some state. This is a very common practice, because it makes a lot of sense to do this. You’ll encounter this basic concept referenced with different names, such as stream processing, event processing, real-time analytics, complex event processing, streaming analytics, real-time streaming analytics, projections, all depending on the environment you’re in. That’s normal, communities evolve language to fit their needs, and we can’t expect the DDD crowd (mostly people developing backend for line of business applications) to synchronize language with, say, data scientists.</p>
<p>An example:</p>
<p>Events:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>// (some fields omitted for clarity)
ItemWasAddedToCart {cartid = 1, itemid = 5, timestamp = 2018-01-01}
ItemWasAddedToCart {cartid = 2, itemid = 6, timestamp = 2019-01-01}
CartWasCheckedOut {cartid = 2, timestamp = 2019-01-02}
</code></pre></div></div>
<p>Queries<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>IsCartCheckedOut {cartid = 2}
HowManyCartsAreAbandoned {}
</code></pre></div></div>
<p>To answer the queries, two consumers can listen to the same stream of events. The first consumer projects a map of <code class="language-plaintext highlighter-rouge">cartid => boolean</code> resulting in <code class="language-plaintext highlighter-rouge">[ 1 => false, 2 => true ]</code>. The second consumer is a bit more involved. It has a business rule that says “A cart is abandoned if it hasn’t been checked out and the last operations is more than 7 days ago”. The projection is a map of <code class="language-plaintext highlighter-rouge">cartid => lastModified</code>, and every time a cart is checked out, we remove it from the map. Our stream projects as <code class="language-plaintext highlighter-rouge">[ 1 => 2018-01-01 ]</code> To answer the query, we count all records with <code class="language-plaintext highlighter-rouge">lastModified > 7 days</code>.</p>
<p>Should we call this Eventsourcing? It makes sense, as the <strong>state of the system is sourced from events.</strong></p>
<p>Let’s look at the other side.</p>
<h2 id="events-as-state">Events as State</h2>
<p>The previous example does not make any assumptions about where events come from. In all kinds of analytics and data processing contexts, this is all there is: data or events are provided as is, and the question of where they came from is irrelevant. I’m pointing this out because it explains why, in that context, you would call the previous example eventsourcing.</p>
<p>So where do the events come from? The producer could be modelled as a state based system. Every cart is represented by a document (or an equivalent set of records) that has a status, line items, timestamps… Changing the state of such a document is usually governed by constraints, such as <code class="language-plaintext highlighter-rouge">A cart with 0 items cannot be checked out</code> and <code class="language-plaintext highlighter-rouge">Items can not be added by a cart that was checked out</code>. Whenever we update the state, we can emit an event that represents what we did. But it’s important to know that often these events can only be emitted after the constraints are imposed. We might even store the events in a log, but the source of truth is the state in the database.</p>
<p>Instead of storing state as documents or records with an optional log, we can get rid of the database and promote the event log to be our single source of truth. I wouldn’t call it a log anymore, but an eventstore. So now, all updates are only expressed as new events, and no longer as state changes. Before storing a new event, all relevant constraints still need to be enforced. To enforce <code class="language-plaintext highlighter-rouge">Items can not be added by a cart that was checked out</code>, we project the state of the cart from the event history.</p>
<p>Should we call this Eventsourcing? Now <strong>the event store is the single source of truth for the system</strong>.</p>
<h2 id="state-from-events-vs-events-as-state">State from Events vs Events as State</h2>
<p>The distinction is important. “State from Events” assumes an existing event stream, regardless of how it was produced, and projects state from it. No new events are added to the stream<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>. “Events as State” is about events as the single source of truth. In other words, new events are added to the stream, but they’re constrained by business rules, and these rules depend on previous events as their input (as opposed to state as the input).</p>
<h2 id="defining-eventsourcing">Defining Eventsourcing</h2>
<p>Now how should we define Eventsourcing? Both make sense. But there are already many terms for “State from Events”. If we choose to call that Eventsourcing, what should we call the second type?</p>
<p>That’s why I use the term <strong>Eventsourcing</strong> only when I specifically talk about a system where state is stored as events and used for decision-making (aka enforcing constraints). We can now try to define Eventsourcing unambiguously<sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>:</p>
<blockquote>
<p><strong>A system is eventsourced when</strong></p>
<ul>
<li><strong>the single source of truth is a persisted history of the system’s events;</strong></li>
<li><strong>and that history is taken into account for enforcing constraints on new events.</strong></li>
</ul>
</blockquote>
<p>I’m pretty sure this definition can improved. In the meantime, that’s the definition I’m using for <a href="https://verraes.net/2019/05/ddd-msg-arch/">my articles on Eventsourcing and Messaging patterns</a>.</p>
<p><strong>Footnotes</strong></p>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-domain-query">Queries can be modelled as domain messages.</a> <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>New events <em>can</em> be added to the eventstore, such as <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Events</a>, but these are derived. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>Greg Young is often quoted that “Eventsourcing is just a fold over a list of data”, which is correct, but reductionist. (I’m not sure the quote is how he actually phrased it, or what the context was.) In any case, a more accurate phrasing is that “a projection can be implemented as a fold over a stream of events”. <a href="#fnref:3" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Patterns Are Not Defined by Their Implementation2019-07-02T00:00:00+00:00https://verraes.net/2019/07/patterns-are-not-defined-by-their-implementation<p>I wrote about something that I called the <em><a href="https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads/">Forgettable Payload</a></em> pattern. The abstract reads “Store the sensitive payload of an event in a separate store to control access and removal.” Please read the full pattern first. In short, it’s about being able to remove sensitive information from an eventstore, for example upon receiving a deletion request under the GDPR.</p>
<p>Some readers have remarked that <em>Forgettable Payloads</em> is the same the <em>Claim Check</em> pattern. <a href="http://amzn.to/1dFVRMd">Enterprise Integration Patterns</a> describes the problem that <em>Claim Check</em> solves as:</p>
<blockquote>
<p>“How can we reduce the data volume of a message sent across the system without sacrificing information content?”</p>
</blockquote>
<p>The solution of both patterns is pretty much identical:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>UserUploadedImage { userId:12, image: "...huge binary blob goes here" }
// Refactored using Claim Check pattern:
UserUploadedImage { userId:12, imageUrl: "http://...photo12.png" }
</code></pre></div></div>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>UserHasRegistered { userId:12, name: "Mathias Verraes", address: "..." }
// Refactored using Forgettable Payloads
UserHasRegistered { userId:12, payload: "b167a587-1f1f-45bf-bd41-af3e7a1aabd1" }
</code></pre></div></div>
<p>In both cases, the actual payload of the message is replaced by a reference to a separate store that holds the payload.</p>
<p>Does this mean that <em>Claim Check</em> and <em>Forgettable Payloads</em> are the exact same pattern? I’d argue they’re not. A pattern consists of a problem and a reusable solution. If the problem is different, the pattern is a different one.</p>
<p><em>Forgettable Payloads</em> is specifically about being able to delete information in an eventsourced system, without touching the eventstore. The same problem can be solved in different ways, for example using <em><a href="https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key/">Crypto-Shredding</a></em>.</p>
<p><em>Claim Check</em> is specifically about reducing message sizes (and is not limited to eventsourced systems). We can also reduce messages sizes by splitting them into smaller ones, called <em>Message Sequence</em> in Enterprise Integration Patterns.</p>
<h3 id="caveat">Caveat</h3>
<p>(<a href="http://amzn.to/1dFVRMd">Enterprise Integration Patterns</a> does mention that <em>Claim Check</em> could be used to hide information. It makes no mention of deleting the information. The book was written many years before GDPR, so the authors can be forgiven for not recognising <em>Forgettable Payloads</em> as a separate pattern. In all fairness, using <em>Claim Check</em> for sensitive information is only mentioned as an option in passing, and does not seem to be part of the actual pattern.)</p>
<h3 id="conclusion">Conclusion</h3>
<p>We must remember to think of a design pattern as “a problem in a context with a reusable solution”, not merely “a reusable solution”. If two patterns have identical solutions, we should not think of them as the same pattern. A solution that is great for one problem, could be a terrible idea for another problem. As our understanding of the problems evolve, their solutions could diverge in entirely separate directions. The problems themselves could evolve separately as well: the technical restrictions to large message sizes may disappear, while the need for robust privacy solutions may increase.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Temporal Modelling2019-06-04T00:00:00+00:00https://verraes.net/2019/06/talk-temporal-modelling<p>The versions of this talk are all part overlapping, part different. You should probably pick one at random to watch, and if you want to find out more, skip around a bit in the others.</p>
<ul>
<li><a href="https://vimeo.com/339925511">NewCrafts 2019, Paris</a></li>
<li><a href="https://skillsmatter.com/skillscasts/13787-keynote-temporal-modelling#video">MuCon 2019, London</a></li>
<li><a href="https://gotoams.nl/2019/sessions/817">GOTO 2019, Amsterdam</a></li>
<li><a href="https://www.youtube.com/watch?v=hh5lskgATCk&feature=youtu.be">ExploreDDD 2019, Denver</a></li>
</ul>
Emergent Contexts through Refinement2019-06-04T00:00:00+00:00https://verraes.net/2019/06/emergent-contexts-through-refinement<p><em>This article was first published in the book <a href="https://dddeurope.com/15years">Domain-Driven Design: The First 15 Years. Essays from the DDD Community</a>. The e-book is available for free in English and Traditional Chinese.</em></p>
<p><a href="https://dddeurope.com/15years">
<img src="https://verraes.net/img/posts/DDD_the_first_15_years_small.png" alt="DDD The First 15 Years" width="200" />
</a></p>
<p>Which Bounded Context owns a particular concept? One way to find out is by evolving your model until everything finds a natural place. All models are wrong, especially the early ones. Let’s look at some simple requirements, and explore how we can evolve the model over time. As we learn more about the problem we’re solving, we can bring that clarity into new iterations of the model.</p>
<h2 id="the-problem">The problem</h2>
<p>Imagine working on a business application, that deals with sales, accounting, reporting, that sort of thing. The existing software has some serious issues. For example, monetary values are represented as scalars. In many places, values are calculated at a high precision, and then rounded down to 2 decimals, and later used again for high precision calculations. These rounding errors are all over the code. It doesn’t make a huge difference on a single amount and a single rounding error, but eventually it could add up, and cost the business millions. The monetary values can represent different currencies, but the financial reporting is always in EUR. It is unclear if the code always correctly converts to EUR when needed, or accidentally adds up amounts from different currencies.</p>
<p>To solve these problems, the first thing to do is to have conversations with the domain experts from sales and accounting. As a result, we can come to a common agreement on some requirements.</p>
<h2 id="requirements">Requirements</h2>
<ol>
<li>Business needs to support about 10 currencies, possibly more in the future. When business wants to support a new currency, say Japanese Yen, then it is assumed that the developers will add support in the code. There’s no need for a UI for adding new currencies.</li>
<li>All price calculations need to be done with a precision of 8 decimals. This is a business decision.</li>
<li>When showing amounts to users, or passing them along over an API, the software will always stick to the currency’s official division. For example, in the case of EUR or USD, that’s 2 decimals. In the case of Bitcoin, a precision of 1 satoshi aka BTC 10^-8^.</li>
<li>In some markets, the software will need to do specific compliance reporting.</li>
<li>All internal reporting needs to be in EUR, no matter what the original currency was.</li>
<li>There will be some legacy and third-party systems, that also publish revenues to the internal reporting tool. Most of them only support the currency’s official division, and can’t deal with higher precision.</li>
</ol>
<blockquote>
<p>Some programming languages don’t deal well with highly precise calculations. There are workarounds which won’t be discussed in this article. Just assume that here “number” means a suitable datatype, such as float or Decimal in C# or BigDecimal in Java.</p>
</blockquote>
<h2 id="the-first-step">The first step</h2>
<p>Are there existing design patterns that can help here? A good pattern to apply when dealing with monetary values is the Value Object pattern from <a href="http://amzn.to/1CdXXP9">Domain-Driven Design</a>.
In fact, a couple of years before Eric Evans published his book, Martin Fowler described an implementation for <code class="language-plaintext highlighter-rouge">Money</code> in <a href="http://amzn.to/1TN7Tq4">Patterns of Enterprise Application Architecture</a>.
The pattern describes an object consisting of two properties, a number for the amount, and another property for the associated currency (which could be a Value Object as well). The <code class="language-plaintext highlighter-rouge">Money</code> Value Object is immutable, so all operations will return a new instance.</p>
<p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/Money.png" alt="Figure 1" /></p>
<p>The <code class="language-plaintext highlighter-rouge">Currency</code> type supports the 10 different currencies that the business is interested in. Enums can be used, but a simple assertion will do if the language doesn’t have support for enums. The following constraints can be added:</p>
<ul>
<li>The type can’t be initialised with a value different from any of the supported 10 currencies.</li>
<li>It only uses the 3-letter ISO symbols, anything else is an error. That satisfies requirement 1.</li>
</ul>
<p><code class="language-plaintext highlighter-rouge">Money</code>’s constructor rounds the numbers to 8 decimals. This gives it enough precision to deal with any currency. Operations like <code class="language-plaintext highlighter-rouge">Money.add(Money other) : Money</code> and <code class="language-plaintext highlighter-rouge">Money.multiply(Number operand) : Money</code> etc can be added to the <code class="language-plaintext highlighter-rouge">Money</code> class. The operations automatically round to 8 decimals. In addition, there’s also a <code class="language-plaintext highlighter-rouge">Money.round() : Money</code> method that returns a new Money object, rounded to 2 decimals.</p>
<pre><code class="language-`java">Money {
// ...
round() : Money {
return new Money(round(this.amount, 2), this.currency)
}
}
</code></pre>
<p>Now that we have modeled the types based on the requirements, the next step is to refactor all the places in the old code that do things with money to use the new <code class="language-plaintext highlighter-rouge">Money</code> object.</p>
<h2 id="composition">Composition</h2>
<p>An interesting aspect of Value Objects, is that they are the ultimate composable objects. That helps to make implicit concepts more explicit. For example, we could naively use the money type to represent prices. But what if a price is not so simple? In Europe, prices have a Value Added Tax component. You can now naturally compose a new <code class="language-plaintext highlighter-rouge">Price</code> Value Object from a <code class="language-plaintext highlighter-rouge">Money</code> and a <code class="language-plaintext highlighter-rouge">VAT</code> object, the latter representing a percentage.</p>
<p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/Price.png" alt="Figure 2" /></p>
<p>A Product could have different prices in different markets, and we can make this concept explicit in a another Value Object.</p>
<p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/ProductPrice.png" alt="Figure 3" /></p>
<p>… and so on. We can push a lot of logic that we’d traditionally put in services, down into these Value Objects.</p>
<p>Value Objects make it easy to build abstractions that can handle lots of complexity, at a low cognitive cost to the developer using it. Finding the abstractions can be hard, but spending the effort here usually impacts code quality and maintainability so much in the long run, that it’s very often worth it.</p>
<h2 id="look-for-weaknesses">Look for weaknesses</h2>
<p>Even if the current model feels intuitively good, there is always room for improvement. Eric often suggests to look for aspects that are awkward or painful to use. These are smells, pointing to opportunities for further refinement.</p>
<p>Looking closely at the model, there are two potential weaknesses:</p>
<ol>
<li>While the scalars in the code have been replaced by the <code class="language-plaintext highlighter-rouge">Money</code> type, the software is still at risk that money is being rounded and then reused in a higher precision calculation. This problem of rounding errors still has not been fully addressed.</li>
<li>The software supports 8 decimal precision only, and the model assumes this is fine. However, when <code class="language-plaintext highlighter-rouge">Money.round()</code> gets invoked, the model doesn’t really deal well with the fact that some currencies don’t have two but three decimals by default (such as Bahraini and Kuwaiti Dinar) or more (like Bitcoin), and some have none (like the Japanese Yen).</li>
</ol>
<p>This is where we should start feeling an itch to stop and rethink our model.</p>
<p>Have a close look at: <code class="language-plaintext highlighter-rouge">Money.round() : Money</code></p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">a</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="o">(</span><span class="mf">1.987654321</span><span class="o">,</span> <span class="n">eur</span><span class="o">)</span>
<span class="c1">// The constructor rounds it down to 1.98765432</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">a</span><span class="o">.</span><span class="na">round</span><span class="o">()</span>
<span class="c1">// round() rounds it up to 1.99 and instantiates a new Money</span>
<span class="c1">// Money’s constructor rounds it to 1.99000000</span>
</code></pre></div></div>
<p>Technically, most programming languages don’t distinguish between 1.99 and 1.99000000, but logically, there is an important nuance here. <code class="language-plaintext highlighter-rouge">b</code> is not just any <code class="language-plaintext highlighter-rouge">Money</code> , <strong>it is a fundamentally different type of money</strong>. The current design doesn’t make that distinction, and just mixes up money, whether it was rounded or not.</p>
<h2 id="make-the-implicit-explicit">Make the implicit explicit.</h2>
<p>A good heuristic when modelling, is to consider if we can be more explicit in the naming, and tease out subtly different concepts. We can rename <code class="language-plaintext highlighter-rouge">Money</code> to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>, and add a new type called <code class="language-plaintext highlighter-rouge">RoundedMoney</code>. The latter always rounds to whatever the currency’s default division is.</p>
<p>The <code class="language-plaintext highlighter-rouge">Money.round()</code> method now becomes very simple:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">PreciseMoney</span> <span class="o">{</span>
<span class="n">round</span><span class="o">()</span> <span class="o">:</span> <span class="nc">RoundedMoney</span> <span class="o">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">RoundedMoney</span><span class="o">(</span><span class="k">this</span><span class="o">.</span><span class="na">amount</span><span class="o">,</span> <span class="k">this</span><span class="o">.</span><span class="na">currency</span><span class="o">)</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/PreciseMoney.png" alt="Figure 4" /></p>
<p>The chief benefit is strong guarantees. We can now typehint against <code class="language-plaintext highlighter-rouge">PreciseMoney</code> in most of our domain model, and typehint against <code class="language-plaintext highlighter-rouge">RoundedMoney</code> where we explicitly want or need it.</p>
<p>It’s easy to underestimate how valuable this style of granular types can be.</p>
<ul>
<li><strong>It’s defensive coding, against a whole category of bugs</strong>. Methods and their callers now have an explicit contract, about what kind of money they are talking about. Good design communicates intent.</li>
<li><strong>Contracts beat tests</strong>. Obviously correct code doesn’t need tests. If <code class="language-plaintext highlighter-rouge">RoundedMoney</code> is well-tested, and some client code typehints for that type, we don’t need a test to verify that this money is in fact rounded. This is why proponents of strong static type systems like to talk about <em>Type Driven Development</em> as an alternative to <em>Test Driven Development</em>: Declare your intent through types, and have the type checker do the work.</li>
<li><strong>It communicates to different people working on the code, that we care specifically about the difference.</strong> A developer who is unfamiliar with this problem space, might look for a <code class="language-plaintext highlighter-rouge">Money</code> type. But the IDE tells them no such type exists, and suggests <code class="language-plaintext highlighter-rouge">RoundedMoney</code> and <code class="language-plaintext highlighter-rouge">PreciseMoney</code>. This forces the developer to consider which type to use, and learn something about how the domain deals with precision.</li>
<li><strong>It introduces a concept from the domain into the Ubiquitous Language and the model</strong>. Precision and Rounding was fundamental to the domain but clearly ignored in the original model. Co-evolving the language, the models, and the implementation, is central to Domain-Driven Design. These model refinements can have exponential payoffs in the long run.</li>
<li>This design also <strong>helps to apply the Interface Segregation Principle</strong>. Client code will not depend on an large set of APIs, only on the ones relevant to the type.</li>
</ul>
<h2 id="dealing-with-different-precision">Dealing with different precision</h2>
<p>We can add a <code class="language-plaintext highlighter-rouge">round()</code> method to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>, but we wouldn’t add a <code class="language-plaintext highlighter-rouge">toPrecise()</code> method to <code class="language-plaintext highlighter-rouge">RoundedMoney</code>. Just like in physics, we can’t create more precision out of nothing. In other words, you can cast <code class="language-plaintext highlighter-rouge">PreciseMoney</code> to <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, but you can’t cast <code class="language-plaintext highlighter-rouge">RoundedMoney</code> to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>. It’s a one way operation.</p>
<p>There’s an elegance to that constraint. Once you round something, the precision is lost forever. The lack of <code class="language-plaintext highlighter-rouge">RoundedMoney.toPrecise()</code> fits the understanding of the domain.</p>
<h2 id="common-interfaces">Common Interfaces</h2>
<p>You may have noticed that in the current design, there is no <code class="language-plaintext highlighter-rouge">Money</code> interface at the top of the object graph. Aren’t <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code> both a kind of <code class="language-plaintext highlighter-rouge">Money</code> or <code class="language-plaintext highlighter-rouge">AbstractMoney</code>? Don’t they share a lot of methods?</p>
<p>If the goal is to try and build a model inspired by the real-world, this would make sense. However, don’t judge your models by how well they fit into hierarchical categorisations. A Domain Model is not an taxonomy — in contrast to the OOP books that teach you that a <code class="language-plaintext highlighter-rouge">Cat extends Animal</code>. Judge your models based on usefulness instead. A top-level <code class="language-plaintext highlighter-rouge">Money</code> interface adds no value at all; in fact it takes away value.</p>
<p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/BadMoney.png" alt="Figure 5" /></p>
<p>This may be a bit counterintuitive. <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, although somewhat related, are fundamentally different types. The model is designed for clarity, for the guarantee that rounded and precise values are not mixed up. By allowing client code the typehint for the generic <code class="language-plaintext highlighter-rouge">Money</code>, you’ve taken away that clarity. There’s now no way of knowing which <code class="language-plaintext highlighter-rouge">Money</code> you’re getting. All responsibility for passing the correct type is now back in the hands of the caller. The caller could do <code class="language-plaintext highlighter-rouge">money instanceof PreciseMoney</code>, but that’s a serious code smell.</p>
<h2 id="dealing-with-conversions">Dealing with Conversions</h2>
<p>Converting between different currencies depends on today’s exchange rates. The exchange rates probably come from some third party API or a database. To avoid leaking these technical details into the model, we can have an <code class="language-plaintext highlighter-rouge">CurrencyService</code> interface, with a <code class="language-plaintext highlighter-rouge">convert</code> method. It takes a <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and a target <code class="language-plaintext highlighter-rouge">Currency</code>, and does the conversion.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">interface</span> <span class="nc">CurrencyService</span> <span class="o">{</span>
<span class="n">convert</span><span class="o">(</span><span class="nc">PreciseMoney</span> <span class="n">source</span><span class="o">,</span> <span class="nc">Currency</span> <span class="n">target</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">CurrencyService</code> implementations might deal with concerns such as caching today’s rates, to avoid unnecessary traffic on each call.</p>
<p>If fetching + caching + converting sounds like a lot of responsibility for one service, that’s because it is. Service classes like are <code class="language-plaintext highlighter-rouge">CurrencyService</code> fundamentally procedural code wrapped in an object. Even though leakage is reduced, thanks to the <code class="language-plaintext highlighter-rouge">CurrencyService</code> interface, the rest of the code still needs to depend on this interface. This makes testing harder, as all those tests will need to mock <code class="language-plaintext highlighter-rouge">CurrencyService</code>. And finally, all implementations of <code class="language-plaintext highlighter-rouge">CurrencyService</code> will need to duplicate the actual conversion, or somehow share that logic. A heuristic that helps, is to figure out if we can somehow isolate the actual domain logic code from the part of the code that is just trying to prepare the data for the domain logic to be applied. For example, differentiate the domain logic (conversion) from the infrastructure code (fetching, caching).</p>
<p>There’s a missing concept here. Instead of having the <code class="language-plaintext highlighter-rouge">CurrencyService</code> do the conversion, we can make it return a <code class="language-plaintext highlighter-rouge">ConversionRate</code> instead. This is a Value Object that represents a source <code class="language-plaintext highlighter-rouge">Currency</code>, a target <code class="language-plaintext highlighter-rouge">Currency</code>, and a factor (a float). Value Objects attract behaviour, and in this case, it’s the <code class="language-plaintext highlighter-rouge">ConversionRate</code> object that becomes the natural place for doing the actual calculation.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">interface</span> <span class="nc">CurrencyService</span> <span class="o">{</span>
<span class="n">getRate</span><span class="o">(</span><span class="nc">Currency</span> <span class="n">source</span><span class="o">,</span> <span class="nc">Currency</span> <span class="n">target</span><span class="o">)</span> <span class="o">::</span> <span class="nc">ConversionRate</span>
<span class="o">}</span>
<span class="nc">ConversionRate</span> <span class="o">{</span>
<span class="o">-</span> <span class="n">sourceCurrency</span> <span class="o">:</span> <span class="nc">Currency</span>
<span class="o">-</span> <span class="n">targetCurrency</span> <span class="o">:</span> <span class="nc">Currency</span>
<span class="o">-</span> <span class="n">factor</span> <span class="o">:</span> <span class="nc">Float</span>
<span class="nf">convert</span><span class="o">(</span><span class="nc">PreciseMoney</span> <span class="n">source</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">convert</code> method makes sure that we never accidentally convert USD to EUR using the rate that was actually meant to convert GBP to JPY. It can throw an exception if the arguments have the wrong currency.</p>
<p>The other cool thing here is that there’s no need to pass around <code class="language-plaintext highlighter-rouge">CurrencyService</code> interface. Instead, we pass around the much smaller, simpler, <code class="language-plaintext highlighter-rouge">ConversionRate</code> objects. They are, once again, more composable. Already the possibilities for reuse become obvious: for example, a transaction log can store a copy of the <code class="language-plaintext highlighter-rouge">ConversionRate</code> instance that was used for a conversion, so we get accountability.</p>
<h2 id="simpler-elements">Simpler Elements</h2>
<p>An <code class="language-plaintext highlighter-rouge">CurrencyService</code> is now something that represents the collection of <code class="language-plaintext highlighter-rouge">ConversionRate</code> objects, and provides access (and filters) on that collection. Sounds familiar? This is really just the <em>Repository</em> pattern! Repositories are not just for Entities or Aggregates, but for all domain objects, including Value Objects.</p>
<p>We can now rename <code class="language-plaintext highlighter-rouge">CurrencyService</code> to <code class="language-plaintext highlighter-rouge">ConversionRateRepository</code>. It has the benefit of being more explicit, and more narrowly defined. Having the pattern name <code class="language-plaintext highlighter-rouge">Repository</code> in the class name is a bit of an annoyance. As Repositories in DDD really represent collections of domain objects, calling it <code class="language-plaintext highlighter-rouge">ConversionRates</code> is a good alternative. But we should take the opportunity to look for terms in the Ubiquitous Language. A place where you can get today’s conversion rates, is a foreign exchange, so we can rename <code class="language-plaintext highlighter-rouge">ConversionRateRepository</code> to <code class="language-plaintext highlighter-rouge">ForeignExchange</code> (or <code class="language-plaintext highlighter-rouge">Forex</code>). Rather than put the word Repository in the classname, we can annotate <code class="language-plaintext highlighter-rouge">ForeignExchange</code> as a @Repository, or have it implement a marker interface called <code class="language-plaintext highlighter-rouge">Repository</code>.</p>
<p>The original procedural <code class="language-plaintext highlighter-rouge">CurrencyService</code> is now split into the two simpler patterns, Repository and Value Object. Notice how we have at no point removed any essential complexity, and yet each element, each object, is very simple in its own right. To understand this code, the only pattern knowledge a junior developer would need, is in a few pages of chapters 5 & 6 of <a href="http://amzn.to/1LrjmZF">Domain-Driven Design</a>.</p>
<h2 id="currency-type-safety">Currency Type Safety</h2>
<p>There are still have some issues left. Remember that some currencies have a division of 1/00, 1/1000, or 1/100000000. <code class="language-plaintext highlighter-rouge">RoundedMoney</code> needs to support this, and in this case, for 10 different currencies. The constructor can start to look somewhat ugly:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">switch</span><span class="o">(</span><span class="n">currency</span><span class="o">)</span>
<span class="k">case</span> <span class="nl">EUR:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span>
<span class="k">case</span> <span class="nl">USD:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span>
<span class="k">case</span> <span class="nl">BTC:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">8</span><span class="o">)</span>
<span class="n">etc</span>
</code></pre></div></div>
<p>Also, every time business needs to support a new currency, new code needs to get added here, and possibly in other places in the system. While not a serious issue, it’s not exactly ideal either. A switch statement (or a key/value pairs) can be a smell for missing types.</p>
<p>One way to achieve this is to make <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code> abstracts or interfaces, and factor the variation out into subtypes for each currency.</p>
<p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/FactorOutCurrency.png" alt="Figure 6" /></p>
<p>Each of the <code class="language-plaintext highlighter-rouge">PreciseEUR</code>, <code class="language-plaintext highlighter-rouge">PreciseBTC</code>, <code class="language-plaintext highlighter-rouge">RoundedEUR</code>, <code class="language-plaintext highlighter-rouge">RoundedBTC</code> etc classes have local knowledge about how they go about their business, such as the rounding switch example above.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">RoundedEUR</span> <span class="o">{</span>
<span class="nc">RoundedEUR</span> <span class="o">(</span><span class="n">amount</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Again, we can put the type system to work here. Remember the requirement that the reporting needs to be in EUR? We can now typehint for that, making it impossible to pass any other currency into our reporting. Similarly, the different compliance reporting strategies for different markets can each be limited to the currencies they support.</p>
<p>So, when should you <em>not</em> want to lift values to types?</p>
<p>Let’s say you’re not dealing with 5 or 10 values, but there is an infinite or very large set of potential values. Or when you’re expecting each of these implementations to change often. In this case, having to support 10 currencies is somewhat on the edge. But again, it’s very unlikely that you’re having to add support for all 180 currencies. You’re probably going to only support the 10 or 15 most relevant ones.</p>
<h2 id="minimalist-interfaces">Minimalist Interfaces</h2>
<p>At this point, we can start seeing how concepts from our requirements, and concepts from our refined Money model, start clustering. We can start breaking up this model, moving and copying parts to different Bounded Contexts. The name Money fooled us: Money means different things to different Contexts.</p>
<p>A benefit of having lots of small classes, is that we can get rid of a lot of code. Perhaps the Sales Bounded Context deals with the 10 <code class="language-plaintext highlighter-rouge">PreciseXYZ</code> types, but the Internal Reporting Bounded Context only needs <code class="language-plaintext highlighter-rouge">RoundedEUR</code>. That means there’s no need to support <code class="language-plaintext highlighter-rouge">RoundedUSD</code> etc, as there’s no need for it. This also implies that we don’t need <code class="language-plaintext highlighter-rouge">round()</code> methods on any of the <code class="language-plaintext highlighter-rouge">PreciseXYZ</code> classes, apart from EUR. Less code means less boilerplate, less bugs, less tests, and less maintenance.</p>
<p>Not supporting a way back from <code class="language-plaintext highlighter-rouge">RoundedEUR</code> to <code class="language-plaintext highlighter-rouge">PreciseEUR</code> is another example of a minimalist interface. Don’t build behaviours that you don’t need or want to discourage.</p>
<h2 id="single-responsibility">Single Responsibility</h2>
<p>Another benefit of these small, ultra-single-purpose classes, is that they very rarely need to change, and only when something in the domain changes. Currencies are very stable concepts, and our model mimics this slow rate of change. A good design allows you to easily add or remove elements, or change the composition of the elements, but rarely requires you to actually change existing code. This in turn leads to fewer bugs and less work and more maintainable code.</p>
<h2 id="ledger">Ledger</h2>
<p>One advantage of expressing our code in terms of rich, well-adapted domain models, is that sometimes this can lead to finding opportunities for other features. It also makes the implementation of those features very easy.</p>
<p>In this case, a goal of this modelling exercise was to solve the precision problem. Every time a precise value is rounded, what happens to the fractions of cents that the business gains or loses? Is this important to the domain?</p>
<p>If so, we can keep a separate ledger for rounding. Every time a fraction of the value is gained or lost by rounding, we record it in the ledger. When the fractions add up to more than a cent, we can add it to the next payment.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1">// rounding now separates the rounded part from the leftover fraction</span>
<span class="nc">PreciseMoney</span><span class="o">.</span><span class="na">round</span><span class="o">()</span> <span class="o">:</span> <span class="o">(</span><span class="nc">RoundedMoney</span><span class="o">,</span> <span class="nc">PreciseMoney</span><span class="o">)</span>
<span class="nc">Ledger</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="nc">PreciseMoney</span><span class="o">)</span> <span class="o">:</span> <span class="kt">void</span>
</code></pre></div></div>
<p>You might not have to deal with this in most domains, but when you have high volumes of small transactions, it could make a significant difference. Consider this a pattern for your toolbox.</p>
<p>The point here is that our model is easy to extend. In the original code, where rounded and precise amounts were not clear, keeping track of lost fractions of cents would have been a nightmare. Because the change in requirements is reflected in the types, we can rely on the type checker to find all places in the code that need to be adapted.</p>
<h2 id="refining-bounded-contexts">Refining Bounded Contexts</h2>
<p>Different capabilities of the system have different requirements. This is what Bounded Contexts are good for: they allow us to reason about the system as a number of cooperating models, as opposed to one unified model. Perhaps the Product Catalog needs a really simple model for money, because it doesn’t really do anything other than displaying prices. Sales may need precise price calculations. Our internal Reporting might be fine with reporting with rounded numbers, or even with reporting with no decimals. Company financials are sometimes expressed in thousands or millions. The Compliance Reporting again has different needs.</p>
<p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/Contexts.png" alt="Figure 7" /></p>
<p>Cleaning up the language:</p>
<p><img src="/img/posts/2019-06-04-emergent-contexts-through-refinement/Contexts2.png" alt="Figure 8" /></p>
<p>If the Sales context always deals with high precision, does it then make sense to call the type <code class="language-plaintext highlighter-rouge">PreciseMoney</code>? Why not just <code class="language-plaintext highlighter-rouge">Money</code>? It should then be clearly understood that in the Sales Ubiquitous Language, there is only one <code class="language-plaintext highlighter-rouge">Money</code> and it doesn’t tolerate rounding. In the Reporting Context, money is always rounded and in EUR. Again the type doesn’t have to be <code class="language-plaintext highlighter-rouge">RoundedMoney</code> or <code class="language-plaintext highlighter-rouge">RoundedEUR</code>, it can just be <code class="language-plaintext highlighter-rouge">Money</code>.</p>
<p>Every Bounded Context now gets its own domain models for Money. Some are simple, some have more complexity. Some have more features, others have less. We’ve already written all that code, we’re just separating it into the Contexts where we need it. Each Bounded Context has a small, unique model for Money, highly adapted to its specific needs. A new team member working on a Bounded Context can now learn this model quickly, because it’s unburdened by the needs of other Contexts. And we can rest assured that they won’t make rounding errors or swap EUR and USD. Different teams don’t need to coordinate on the features of Money, and don’t need to worry about breaking changes.</p>
<p>In some environments, a single Generic Subdomain for money would have been perfectly fine. In others, we want highly specialized models. Determining which is which is your job as a domain modeller.</p>
<p><strong>We didn’t do upfront analysis of what goes into which Bounded Context. If we had tried that, early on in the project, we would have naively built a shared Money library, assuming that the concept of Money is universal between all Contexts. Instead, we kept on refining our model based on our needs and new insights, and saw how these explicit concepts naturally found a place. Cultivate a healthy obsession with language, and keep refactoring towards deeper insight, one step at a time. All good design is redesign.</strong></p>
Messaging Patterns: Natural Language Message Names2019-06-02T00:00:00+00:00https://verraes.net/2019/06/messaging-patterns-natural-language-message-names<h3 id="problem">Problem</h3>
<p>Our industry has a tradition of unnatural, compact, lossy naming. This may save on typing, but costs in understandability.</p>
<h3 id="solution">Solution</h3>
<p>Naming is hard, because we <em>name</em> things. If we understand the domain and have discussed with domain experts, we don’t have to invent names, we can use the names from that domain. And no domain experts says <code class="language-plaintext highlighter-rouge">Payment Event</code> or <code class="language-plaintext highlighter-rouge">Invoice Paid</code> or <code class="language-plaintext highlighter-rouge">OnAfterPayment</code>, they say <code class="language-plaintext highlighter-rouge">The invoice was paid</code>.</p>
<p><strong>Use descriptive, natural language names. Do not remove the verbs. Use past tense sentences for events, imperative sentences for commands, questions for queries, negative statements for exceptions.</strong> (In English, I have a slight preference to leave of the article (“the”, “a”), but I might be convinced otherwise.)</p>
<h3 id="examples">Examples</h3>
<p>Events: <code class="language-plaintext highlighter-rouge">CustomerWasInvoiced</code>, <code class="language-plaintext highlighter-rouge">InvoiceWasPaid</code>
Commands: <code class="language-plaintext highlighter-rouge">InvoiceCustomer</code>, <code class="language-plaintext highlighter-rouge">FulfilOrder</code>
Queries: <code class="language-plaintext highlighter-rouge">WhichInvoicesAreUnpaid</code>, <code class="language-plaintext highlighter-rouge">HowManyInvoicesAreUnpaid</code>
Responses: <code class="language-plaintext highlighter-rouge">UnpaidInvoices</code>, <code class="language-plaintext highlighter-rouge">NumberOfUnpaidInvoices</code>
Exceptions: <code class="language-plaintext highlighter-rouge">UnpaidOrdersMayNotBeFulfilled</code></p>
<h3 id="discussion">Discussion</h3>
<p>Using natural language names helps us to see communication in our systems as mimicking human communication. This is how people tell stories: This has happened, that has happened, I did this, I expected that to happen. Embedding that language in our code and artefacts, is a core idea in Domain-Driven Design, because it enables two-way conversations. We may not expect a domain expert to actually read this code, but we can write it in a way that they would understand. Later, when more work needs to be done on the code, the intent is still very clear, and we can easily go abck and forth between code and domain.</p>
<h3 id="test-scenarios">Test scenarios</h3>
<p>We can test scenarios by structuring example stories:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scenario
.given(new OrderWasPlaced(orderId=5, ...))
.when(new FulfilOrder(orderId=5))
.then(new UnpaidOrdersMayNotBeFulfilled()) // exception
scenario
.given(new OrderWasPlaced(orderId=5, ...))
.and(new InvoiceWasPaid(invoiceId=6, orderId=5, ...))
.when(new FulfilOrder(orderId=5))
.then(new OrderWasFulfilled(orderId=5))
</code></pre></div></div>
<p>This style of writing tests as example stories is common in Behaviour-Driven Development, often using a DSL called Gherkin. But in a messaging architecture, it can be written in plain code as illustrated above.</p>
<p>If we see a persona as a history, we can write reusable scenarios:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>angryCustomerJim = scenario
.given(new OrderWasPlaced(...))
.given(new InvoiceWasPaid(...))
.given(new ShipmentWasDelayed(...))
.given(new OrderWasPlaced(...))
.given(new InvoiceWasPaid(...))
.given(new ShipmentWasDelayed(...))
</code></pre></div></div>
<p>The actual test can now reuse this story:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scenario
.involving(angryCustomerJim)
.when(new ExpediteShipment(...))
.then(...)
</code></pre></div></div>
<p>The pattern for scenarios that change the state of the system is “given events, when command, then events”. The pattern for queries is “given events, when query, then response”.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scenario
.given(new CustomerWasInvoiced(...))
.given(new CustomerWasInvoiced(...))
.given(new InvoiceWasPaid(...))
.given(new CustomerWasInvoiced(...))
.when(new HowManyInvoicesAreUnpaid())
.then(new NumberOfUnpaidInvoices(5))
</code></pre></div></div>
<h3 id="code">Code</h3>
<p>The benefits are not limited to tests. By using well-chosen class and method names, intent becomes even more explicit.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class KeepingTrackOfUnpaidInvoices implements Projector
when(CustomerWasInvoiced e)
unpaidInvoices++
when(InvoiceWasPaid e)
unpaidInvoices--
ask(HowManyInvoicesAreUnpaid q)
return new NumberOfUnpaidInvoices(unpaidInvoices)
</code></pre></div></div>
<h3 id="going-further">Going further</h3>
<p>In F#, identifiers wrapped in three backticks can contain spaces and other characters. <code class="language-plaintext highlighter-rouge">'''Invoice was paid'''</code> (This blog can’t print backticks, just imagine them.) This makes it even more natural. A problem arises when integrating with other systems, where those characters might not be supported.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Eventsourcing Patterns: Migration Events in a Ghost Context2019-06-01T00:00:00+00:00https://verraes.net/2019/06/eventsourcing-patterns-migration-events-ghost-context<p>Explicitly conform to the legacy component’s model in an eventsourcing migration.</p>
<h3 id="problem">Problem</h3>
<p>We are replacing a Legacy component with a new Eventsourced component. Besides being fully Eventsourced, the new component has a clear domain model with a well-defined <a href="http://amzn.to/1CdXXP9">Ubiquitous Language</a>. The domain events are granular, and named in this language, and track the entire history of the component.</p>
<p>The new component needs to retain and use data produced by the old component. The old component stores state, but no history. There is not enough information in that state to deduce all the domain events reproduce the real history. In other words, we know the state but not how we got there. There is no way for new component to have a real complete history from the old component.</p>
<h3 id="common-approaches">Common approaches</h3>
<p>One solution is to keep the database of the Legacy component, mark it as read-only, and query it from the Eventsourced system. The downside is that the Eventsourced component is now a hybrid, and relies on two entirely different persistence strategies.</p>
<p>Another common solution is to make a best effort to force the data from the legacy component, into the domain events of the Eventsourced component. The appeal of this approach is that we have a single, straightforward way of doing things: one canonical model, that fits everything, and therefore easy to work with and reason about. The downside is that the domain events are lying. They didn’t happen in that way, we forced them into the new model. We’re hiding essential complexity, which often causes accidental complexity as a side-effect.</p>
<p>That said, both solutions may be perfectly acceptable, especially if the underlying conceptual models for the legacy and new component are very close.</p>
<h3 id="migration-events">Migration Events</h3>
<blockquote>
<p><em>“The old system is now the new problem.”</em></p>
<p>— <a href="http://amzn.to/1LMl99l">John Gall; Systemantics</a></p>
</blockquote>
<p>The mindshift is to see the old system as a domain. The new system is not only solving for the business domain, but for the old system’s domain as well. If the old system is az domain, we can apply domain modelling to it, draw a <em>Bounded Context</em> around it, agree on a <em>Ubiquitous Language</em>, and reason about its relation to other <em>Bounded Contexts</em>.</p>
<p><em>Migration Events</em> are now domain events in the new component, but they use the <em>Ubiquitous Language</em> of the old component. They’re not shy about the fact that they are <em>Migration Events</em>, and they don’t try to pretend they happened in the new component. Typical <em>Migration Events</em> names are <code class="language-plaintext highlighter-rouge">LegacyCustomerWasImported</code>, <code class="language-plaintext highlighter-rouge">LegacyOrderWasPlaced</code>. If concepts existed in the Legacy component, that do not exist in the new component, the <em>Migration Events</em> reflect those legacy concepts. No attempt is made to deduce more history than what is available in the Legacy component.</p>
<p>This may sound more complex than the alternatives: we have more event types, that we are stuck with forever. All our code, such as projections, will need to deal with both the new event types and with the <em>Migration Events</em>. But by hiding the fact the we did have a Legacy component, we are hiding complexity and end up with a system that is harder to understand for people who where not involved in the old component.</p>
<h3 id="ghost-context">Ghost Context</h3>
<p>In DDD, we express the domain in the design, it’s just not always obvious what the domain is. <em>Bounded Contexts</em> are essentially about lifting a domain model and its language into a first-class concept in our designs. By treating a model+language as a unit, we can do higher-order reasoning about them: replace models, protect against models, find patterns for relationships between models. In our case, the new Bounded Context is <em><a href="http://amzn.to/1CdXXP9">Conformist</a></em> to the Legacy Bounded Context. The <em>Conformist</em> pattern has an unfortunate negative connotation, but is actually a good strategy when applied judiciously.</p>
<p>Even after we delete the Legacy component, the Legacy Bounded Context stays alive. It just isn’t tied to a physical component anymore. The language and model exist, but the implementation has disappeared. But it is still a Bounded Context: there’s language in there, like <code class="language-plaintext highlighter-rouge">LegacyCustomerWasImported</code>, and it is internally consistent to a model, even if most of that model has no implementation anymore. (Note that the <code class="language-plaintext highlighter-rouge">LegacyCustomerWasImported</code> did not exist in the old model, but it represents a <code class="language-plaintext highlighter-rouge">Customer</code> as it was defined in that model.) This approach of seeing Bounded Contexts as first-class concepts, autonomous from implementations, is one of the superpowers of Domain-Driven Design in my opinion, and one of its most unique innovations.</p>
<p>I propose <em>Ghost Context</em> as the pattern name for a legacy Bounded Context with no implementation, but that still has traces in other Bounded Contexts.</p>
<blockquote>
<p><em>“The ghost of the old system continues to haunt the new.”</em></p>
<p>— <a href="http://amzn.to/1LMl99l">John Gall; Systemantics</a></p>
</blockquote>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Eventsourcing Patterns: Decision Tracking2019-05-29T00:00:00+00:00https://verraes.net/2019/05/eventsourcing-patterns-decision-tracking<h2 id="decision-tracking">Decision Tracking</h2>
<p>Store the outcome of a decision to guard against rule changes.</p>
<h3 id="problem">Problem</h3>
<p>In an eventsourced system, a consumer listens to a stream of events. IT has some business logic that makes decisions, and acts upon those, for example in the form of a side effect or a new event. Because we’ve already stored the input events in the event store, we can recalculate that decision. Should our event store persist an event that represents that decision?</p>
<h3 id="solution">Solution</h3>
<p>Often, the reasoning is that the event that represents the decision, is redundant. We could store it, but we can recalculate it, so we don’t <em>have to</em> store it.</p>
<p>The problem with this line of reasoning is that it doesn’t take into account model changes. The business rule that is applied by the consumer, could change over time, perhaps because of a new requirement, or a bug fix.</p>
<p>If we now recalculate the decisions, using the stored events, we get a different outcome than the one that was decided the first time. The old outcome may have been wrong, but it was the one that was calculated at the time, and that caused the side effect to happen. Our present system is therefore lying, as it shows us “facts” about the past that didn’t actually happen. This goes against the idea that an eventstore serves as a historical record of the system.</p>
<p>One solution is to track decisions as events, and store these just like other events. We can now inspect the event store, and not only see domain events for the things that happened, but also for the decisions our system has made, whether these are correct or not.</p>
<p>This is much more explicit. It can, for example, help us to mitigate the consequences of a bug, because we know exactly what happened. In fact, many problems arise from attempts to make a system look like a bug never happened.</p>
<p><em>Decision Tracking</em> isn’t going to be worth the effort in all cases, for example if the we don’t expect anything to be affected by that decision long after it was made.</p>
<h3 id="example">Example</h3>
<p>A producer emits a stream of <code class="language-plaintext highlighter-rouge">CustomerHasPurchasedTeddyBear</code> events. A business rule states that “At every 1000th purchase, we give away a free teddy bear hat”. A consumer increments a counter for each event. Alice is the 1000th buyer, so the consumer stores <code class="language-plaintext highlighter-rouge">FreeHatWasAwarded{customer=Alice}</code> in the event store.</p>
<p>Later, after seeing how successful the campaign was, marketing wants every 750th customer to get a hat now. On top of that, they want to prevent people from getting more than one hat. Because we have a historical record of customers who received the hat, we are certain not to award hats twice.</p>
<h3 id="alternative">Alternative</h3>
<p>A different approach is <em>Model Change Tracking</em>. Here changes in the model are tracked in the event store, as regular events. When we want to recalculate decisions, we know exactly which version of the model was active at the time. We then need to recalculate using that specific version of the code, so that the outcome is identical with the original outcome (even if that outcome was wrong).</p>
<p>These could be generic model change events, such as <code class="language-plaintext highlighter-rouge">ModelWasDeployed{version=7}</code>, or domain-specific such as <code class="language-plaintext highlighter-rouge">NewFreeHatRuleWasDeployed{version=7}</code>, or, if possible, they could be configuration changes such as <code class="language-plaintext highlighter-rouge">FreeHatTresholdWasChanged{treshold=750}</code>.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Messaging Patterns: Throttling2019-05-23T00:00:00+00:00https://verraes.net/2019/05/messaging-patterns-throttling<h2 id="throttling">Throttling</h2>
<p>Only keep one event per time unit from a high frequency stream.</p>
<h3 id="problem">Problem</h3>
<p>A stream of <em><a href="https://verraes.net/2019/05/messaging-patterns-ephemeral-events">Ephemeral Events</a></em> is too much to handle for a consumer.</p>
<h3 id="solution">Solution</h3>
<p>An intermediary consumer is placed in front of the original consumer. It drops all events, except once per time unit, where it passes the event to the original consumer.</p>
<h3 id="example">Example</h3>
<p>This pattern is very common in user interfaces. An input device, such as a mouse, emits it’s position at 500 Hz. Instead of rendering at the same frequency, the stream is <em>throttled</em> at 5Hz, and therefore the consumer only needs to rerender 5 times per second.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Messaging Patterns: Ephemeral Events2019-05-23T00:00:00+00:00https://verraes.net/2019/05/messaging-patterns-ephemeral-events<h2 id="ephemeral-events">Ephemeral Events</h2>
<p>An event’s lifetime lasts until the next event.</p>
<h3 id="problem">Problem</h3>
<p>Some producers, such as sensors, emit realtime or near-realtime measurements, expressed as events. A new measurement makes the previous measurement obsolete. These events can be sent at regular or unpredictable intervals, but they are often high frequency. Consumers aren’t capable of keeping up with all events, or aren’t interested in processing all of them.</p>
<h3 id="solution">Solution</h3>
<p>Mark the events as being <em>Ephemeral Events</em>. The infrastructure knows that only the latest event matters. When a consumer can’t keep up with the event stream coming in, the infrastructure prefers to drop older events and skip to the most recent one. The consumer can still produce valuable outcomes with a subset of all events.</p>
<p>In an eventsourced system, the <em>Ephemeral Events</em> are not persisted. Partly this is because there are too many of them. More importantly, they have no lasting business value, or at least not at their original level of granularity. An <code class="language-plaintext highlighter-rouge">@ephemeral</code> annotation in the code can indicate to the eventsourcing infrastructure that this event must not be persisted.</p>
<h3 id="example">Example</h3>
<p>A thermometer emits <code class="language-plaintext highlighter-rouge">TemperatureWasMeasured { temperature, timestamp }</code> events multiple times per second. A consumer updates a graph on a TV monitor once a minute, and ignores the other events. The consumer is turned off during the night. In the morning, it doesn’t catch up with the nighttime events, but picks up from the latest one.</p>
<h3 id="patterns">Patterns</h3>
<p><em>Ephemeral Events</em> are often combined with other patterns:</p>
<ol>
<li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Event</a>. For example, consumer receives a continuous stream of stock prices. At <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-passage-of-time-event/">the close of trading</a>, the consumer emits an event containing statistics such as min/max/avg.</li>
<li><a href="https://verraes.net/2019/05/messaging-patterns-throttling/">Throttling</a></li>
<li><a href="https://verraes.net/2019/05/messaging-patterns-change-detection-events">Change Detection Events</a></li>
</ol>
<p><em>Ephemeral Events</em> should not be confused with <em><a href="http://amzn.to/1dFVRMd">Message Expiration</a></em>, as described in <a href="http://amzn.to/1dFVRMd">Enterprise Integration Patterns</a>, where the message itself contains an expiration date by which it is no longer relevant.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Messaging Patterns: Change Detection Events2019-05-23T00:00:00+00:00https://verraes.net/2019/05/messaging-patterns-change-detection-events<h2 id="change-detection-events">Change Detection Events</h2>
<p>Listen to a stream of events and produce a new event when a value changes.</p>
<h3 id="problem">Problem</h3>
<p>A consumer is interested in an event stream, but the consumer only wants to know when a particular value changes, and doesn’t care about the events where that value didn’t change. Optionally, the consumer only cares when the value changes more than a given interval.</p>
<h3 id="solution">Solution</h3>
<p>A second intermediary consumer is placed in front of the original consumer. It acts as a filter, emitting <em>Change Detection Events</em> when the observed value is different from the previous event. It may also ignore changes that are too small for the original consumer.</p>
<p><em>Change Detection Events</em> are often used in combination with <em><a href="https://verraes.net/2019/05/messaging-patterns-ephemeral-events">Ephemeral Events</a></em>.</p>
<h3 id="example">Example</h3>
<p>The producer emits <code class="language-plaintext highlighter-rouge">TemperatureWasMeasured</code> multiple times per second. A consumer is only interested in changes of 1°C or more. An intermediate consumer listens to the <code class="language-plaintext highlighter-rouge">TemperatureWasMeasured</code> events. When an event arrives, it compares the temperature with the temperature from the previous event. If it differs by more than 1°C, it emits a <code class="language-plaintext highlighter-rouge">TemperatureHasChanged</code> event.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
DDD and Messaging Architectures2019-05-21T00:00:00+00:00https://verraes.net/2019/05/ddd-msg-arch<p>My aim is to identify, name, and document patterns, that I’ve used or encountered in my consulting practice. All patterns are anti-patterns when used in the wrong context, so I try to add heuristics, trade-offs, and caveats.</p>
<p>You can help: tell me your experiences, point me to other people’s writings on patterns, distill your own heuristics, …</p>
<h3 id="meta">Meta</h3>
<ol>
<li><a href="https://verraes.net/2019/08/eventsourcing-state-from-events-vs-events-as-state">Eventsourcing: State from Events or Events as State?</a></li>
<li><a href="https://verraes.net/2019/07/patterns-are-not-defined-by-their-implementation/">Patterns Are Not Defined by Their Implementation</a></li>
</ol>
<h3 id="patterns-for-decoupling-in-distributed-systems">Patterns for Decoupling in Distributed Systems</h3>
<ol>
<li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-domain-query">Domain Queries</a></li>
<li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Event</a></li>
<li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee/">Completeness Guarantee</a></li>
<li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-passage-of-time-event/">Passage of Time Event</a></li>
<li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event/">Fat Event</a></li>
<li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-explicit-public-events/">Explicit Public Events</a></li>
<li><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-segregated-event-layers/">Segregated Event Layers</a></li>
</ol>
<h3 id="eventsourcing-patterns">Eventsourcing Patterns</h3>
<ol>
<li><a href="https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads">Forgettable Payloads</a></li>
<li><a href="https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key">Crypto-Shredding</a></li>
<li><a href="https://verraes.net/2019/05/eventsourcing-patterns-decision-tracking/">Decision Tracking</a></li>
<li><a href="https://verraes.net/2019/06/eventsourcing-patterns-migration-events-ghost-context/">Migration Events in a Ghost Context</a></li>
<li><a href="https://verraes.net/2022/03/multi-temporal-events/">Multi-temporal Events</a></li>
</ol>
<h3 id="general-messaging-patterns">General Messaging Patterns</h3>
<ol>
<li><a href="https://verraes.net/2019/05/messaging-patterns-ephemeral-events/">Ephemeral Events</a></li>
<li><a href="https://verraes.net/2019/05/messaging-patterns-throttling/">Throttling</a></li>
<li><a href="https://verraes.net/2019/05/messaging-patterns-change-detection-events">Change Detection Events</a></li>
<li><a href="https://verraes.net/2019/06/messaging-patterns-natural-language-message-names/">Natural Language Message Names</a></li>
</ol>
<h3 id="testing">Testing</h3>
<ol>
<li><a href="https://verraes.net/2023/05/eventsourcing-testing-patterns/">EventSourcing Testing Patterns</a></li>
</ol>
<p>Note: I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</p>
Eventsourcing Patterns: Crypto-Shredding2019-05-13T00:00:00+00:00https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key<h2 id="crypto-shredding">Crypto-Shredding</h2>
<p>Encrypt sensitive information in an event and delete the key.</p>
<h3 id="problem">Problem</h3>
<p>The problem is the same as for <a href="https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads">Forgettable Payloads</a>: some attributes of an event should not be read by all consumers, and we should be able to delete them, without touching the event store.</p>
<h3 id="solution">Solution</h3>
<p>Encrypt the sensitive attributes, with a different encryption key for each resource (such as a customer). Only give the key to consumers that require it. When the sensitive information needs to be erased, delete the encryption key instead, to ensure the information can never be accessed again. This effectively makes all copies and backups of the sensitive data unusable.</p>
<h3 id="discussion">Discussion</h3>
<p>The <em>Crypto-Shredding</em> pattern is of course only as good as your encryption and your key management practices. Today’s unbreakable encryption could be tomorrow’s infosec disaster.</p>
<p>If the consumers (such as projections) consistently store all sensitive information using encryption, it makes deletions extremely cheap, as only the key must be deleted, as opposed to deleting all copies. But this benefit can be achieved using <em><a href="https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads">Forgettable Payloads</a></em>. Both patterns do not solve the problem of consumer storing encrypted data, or using the encrypted data to compute some new value (that may still be sensitive).</p>
<p>There is also a concern of legality: is <em>Crypto-Shredding</em> a sufficient implementation of delete requests under GDPR, or does the law consider the data not truly deleted?</p>
<p><strong>UPDATE:</strong> Harrison J. Brown wrote the following answer to that question:</p>
<blockquote>
<p>The legal question is important and the GDPR does actually state that encrypted personal information is still personal information. I spoke to my lawyer about this and they said that in the event of a breach you’d still have to notify the authority (the ICO, in the UK) though you might not have to tell the data subjects (since the breach wouldn’t affect them). This document from the European Commission states this clearly in section1, paragraph 5:</p>
<p>“A confidentiality breach on personal data that were encrypted with a state of the art algorithm is still a personal data breach, and has to be notified to the authority. Nevertheless, if the confidentiality of the key is intact, the data are in principle unintelligible to any person who is not authorised, thus the breach is unlikely to adversely affect the data subject and therefore doesn’t need to be notified to the data subject.”
As for the deletion request scenario, the law does not consider deleting the encryption key equal to actually deleting the data itself. Encrypted personal data is still personal data, regardless of whether anyone has the key. So, if were asked by a data subject to delete their personal data and all you did was delete the encryption key you would not be complying with the removal request, at least according to the GDPR.</p>
<p>So, whilst this crypto-shredding pattern is, I think, really useful for certain types of data (business sensitive information, for example) I think your Forgettable Payloads pattern (where one stores the sensitive payload of an event in a separate store to control access and removal) is more appropriate for personal data.</p>
</blockquote>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
<li></li>
</ul>
Eventsourcing Patterns: Forgettable Payloads2019-05-13T00:00:00+00:00https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads<h2 id="forgettable-payloads">Forgettable Payloads</h2>
<p>Store the sensitive payload of an event in a separate store to control access and removal.</p>
<h3 id="problem">Problem</h3>
<p>An <em>Event Store</em> is an append-only chronological database of <em>Domain Events</em>, so ideally we never remove any event or event data. This poses a problem when data needs to be deleted, for example after a customer invokes their right to be forgotten under the GDPR.</p>
<p>A different problem (with a similar solution) exists when the events in the <em>Event Store</em> are emitted to outside consumers, but some events contain some information that must remain private to the producer.</p>
<h3 id="solution">Solution</h3>
<p>(Note: the second problem can also be solved using <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-explicit-public-events/">Explicit Public Events</a> or <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-segregated-event-layers/">Segregated Event Layers</a>.)</p>
<p>Remove the sensitive information from the definition of the event. Instead, store that information in a separate (logical) database. In the event, store a reference or URL that points to the payload’s location in the database.</p>
<p>Consumers still receive the events, but to read the sensitive data, they need to query the payload database. Tightly control the access to that database, so that only consumers with the right permissions can read the sensitive information. Consumers without access, can still depend on the rest of the event’s non-sensitive payload.</p>
<p>When a deletion request is received, remove the payload from the payload database, and leave the events in the Event Store untouched. Consumers that query the payload, get no result for a deleted payload, and show a blank instead.</p>
<p>Instead of deleting the payloads, they can also be replaced with a John Doe; that is, a set of fake data to substitute the real data.</p>
<h3 id="example">Example</h3>
<p>Original event:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerHasSignedUp {
customerId: 123,
name: "Isaac Newton",
organisation: "Trinity College, Cambridge",
birthday: 1643-01-04,
hobbies: "Physics",
signedUpOn: 2019-05-13
status: "Premium member"
}
</code></pre></div></div>
<p>Redesigned event:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerHasSignedUp {
customerId: 123,
personalData: "example.com/personalData/789"
signedUpOn: 2019-05-13
status: "Premium member"
}
Body of example.com/personalData/789:
{
personalDataId: 789,
customerId: 123,
name: "Isaac Newton",
organisation: "Trinity College, Cambridge",
birthday: 1643-01-04,
hobbies: "Physics",
}
After a removal request:
{
personalDataId: 789,
customerId: 123,
name: "John Doe",
organisation: "(removed under GDPR removal request #456)",
birthday: null,
hobbies: "(removed under GDPR removal request #456)"
}
</code></pre></div></div>
<h3 id="weaknesses">Weaknesses</h3>
<p>The <code class="language-plaintext highlighter-rouge">Forgettable Payloads</code> pattern requires to maintain a separate storage, so it breaks the concept of an <em>Event Store</em> as the Single Source of Truth in an eventsourced system. Consumers of the events can now no longer depend on the events alone. Instead, after receiving an event, they need to query the payload database. This increases coupling and complexity.</p>
<p>A second weakness is that a consumer might receive the event, query the sensitive payload, and then store that information locally When a delete request arrives, the consumer also needs to remove its local copy. This problem is not unique to eventsourcing, but there’s an elegant solution: the producer emits a <code class="language-plaintext highlighter-rouge">GDPRDeletionWasRequested {customerId}</code>, and the consumers react to it. A good habit is to never store the sensitive payloads, and make sure consumers always query the payload database.</p>
<p>Finally, sometimes a person can be identified not only through personal information, but through the associations and relations with other information. Say Alice exchanges messages with both Bob and Charlie. Alice requests a deletion and becomes Jane Doe in our records. We don’t know who Jane Doe is, but we now that Jane Doe is a friend of both Bob and Charlie. The <code class="language-plaintext highlighter-rouge">Forgettable Payloads</code> pattern offers no solution here.</p>
<h3 id="alternatives">Alternatives</h3>
<p>The <a href="https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key">Crypto-Shredding</a> pattern is an alternative solution to the <code class="language-plaintext highlighter-rouge">Forgettable Payloads</code> pattern.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Patterns for Decoupling in Distributed Systems: Segregated Event Layers2019-05-11T00:00:00+00:00https://verraes.net/2019/05/patterns-for-decoupling-distsys-segregated-event-layers<h2 id="segregated-event-layers">Segregated Event Layers</h2>
<p>Explicitly segregate a Bounded Context’s events in visibility layers, with their own language.</p>
<h3 id="problem">Problem</h3>
<p>The problem is the same as described in <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-explicit-public-events/">Explicit Public Events</a></em>. Sometimes marking some events as public is not enough. You really need to be able to evolve the internal events separately, while designing the public events differently to suit the clients. The clients may need <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Events</a></em> or <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event/">Fat Events</a></em>, but you don’t want to muddy your <em>Bounded Context</em> with all these consumers’ needs.</p>
<p>There’s also the problem that if too many consumers impose their needs on a producer, and all depend on the same events, the consumers are implicitly logically coupled to each other.</p>
<h3 id="solution">Solution</h3>
<p>Keep all internal events strictly private. Set up an adapter that listens to internal events, and emits a new stream of different events. This may be on stream for all public events, but we can go as far as setting up a separate adapter and stream for each consumer, highly adapted to their need.</p>
<p>The new event stream is effectively a different <em>Bounded Context</em> from the one we’re in. The stream has its own language, event types, and names. This makes the <em>Segregated Event Layers</em> effectively an implementation pattern to build an <em>Anti-Corruption Layer</em>. Some names may overlap or conflict with the names in the original <em>Bounded Context</em>, but this is fine: Having the freedom of defining concepts locally without trying to define them globally, is precisely why we use <em>Bounded Contexts</em>.</p>
<h3 id="example">Example</h3>
<p>In the private layer, we emit:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>OrderWasInitiated {orderId}
LineItemWasAdded {orderId, productId, quantity}
OrderWasPlaced {orderId, customerId}
StockWasReserved {orderId, stockId}
TransportWasBooked {transportId, orderId}
OrderWasPacked {orderId}
OrderWasShipped {orderId}
</code></pre></div></div>
<p>The adapter is a type of <em>Projector</em> that listens for these events, and emits a new stream for public consumption:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>OrderWasPlaced {orderId, customerName, lineItems}
OrderWasConfirmed {orderId, customerName, lineItems}
OrderWasShipped {orderId, customerName, lineItems}
</code></pre></div></div>
<p>The public stream is clearly designed to suit the clients: <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> is a <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Event</a>, the others are <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event/">Fat Events</a>, and there’s no <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee/">Completeness Guarantee</a> because <code class="language-plaintext highlighter-rouge">stockId</code> and <code class="language-plaintext highlighter-rouge">transportId</code> are not in that stream.</p>
<p>The public stream is a different <em>Bounded Context</em>: Some of the language in the public stream does not exist in the private stream, like <code class="language-plaintext highlighter-rouge">OrderWasConfirmed</code>; and <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> and <code class="language-plaintext highlighter-rouge">OrderWasShipped</code> have a slightly different meaning from their equivalents in the private stream.</p>
<p>We could take this example further by making a separate adapter, stream, and language, for each consumer, such as a customer notification system, the sales department, the shipping company, etc.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Patterns for Decoupling in Distributed Systems: Fat Event2019-05-11T00:00:00+00:00https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event<h2 id="fat-event">Fat Event</h2>
<p>Add redundant information to a Domain Event to reduce complexity in the consumer.</p>
<h3 id="problem">Problem</h3>
<p>A consumer is interested in one event type from a producer, to react to it or report information to a user. The producer’s events are designed with a <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee/">Completeness Guarantee</a>. The event only contains the attributes that have changed, and none of the other state of the resources that the consumer is interested in. So the consumer has to listen to a number of other events as well, that each contain some of the state changes of the resource. Most of the work the consumer is now doing, is building state based on events, and doing lookups to associate things with each other. There’s a lot of complexity in the consumer, just to achieve a small task.</p>
<h3 id="solution">Solution</h3>
<p>A <em>Fat Event</em> is an event that not only contains the attributes that have changed, but also a number of attributes that have not changed at the time the event is emitted. These are specifically chosen to lighten the work that the consumer has to do: it doesn’t have to persist intermediate states from other event types, as everything it needs to do its core job is the one (or few) <em>Fat Events</em>.</p>
<p>The consumer now needs to be aware of fewer event types, and is therefore better isolated from changes in the producer.</p>
<h3 id="example">Example</h3>
<p>A consumer wants to display a list of overdue invoices. It listens to <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue {invoiceId}</code>. This event only contains the <code class="language-plaintext highlighter-rouge">invoiceId</code>, but the user needs to see the customer’s name as well (and other informationbut let’s keep this example simple). So the consumer listens to <code class="language-plaintext highlighter-rouge">CustomerWasInvoiced {invoiceId, customerId, amount, lineItems...}</code>. Now the consumer knows which customer the invoice belongs to, but to get the name, it needs to listen to <code class="language-plaintext highlighter-rouge">CustomerHasSignedUp {customerId, customerName, ...}</code> and <code class="language-plaintext highlighter-rouge">CustomerWasRenamed {customerId, newCustomerName}</code>. The consumer keeps lookup tables:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class OverdueInvoices implements Projector
when(CustomerHasSignedUp e)
INSERT INTO CustomerNames (e.customerId, e.customerName)
when(CustomerWasRenamed e)
UPDATE CustomerNames (e.customerId, e.customerName)
when(CustomerWasInvoiced e)
INSERT INTO Invoices (e.invoiceId, e.customerId)
when(InvoiceBecameOverdue e)
INSERT INTO OverdueInvoices (
SELECT invoiceId, customerName FROM Invoices i
LEFT JOIN CustomerNames cn ON cn.customerId = i.customerId
WHERE invoiceId = e.invoiceId
)
when(WhichInvoicesAreOverdue q)
SELECT * FROM OverdueInvoices
</code></pre></div></div>
<p>In the new design, we adapt the producer first. <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> will now carry the attribute <code class="language-plaintext highlighter-rouge">customerName</code> as well. The consumer becomes very simple:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>class OverdueInvoices implements Projector
when(InvoiceBecameOverdue e)
INSERT INTO OverdueInvoices (e.invoiceId, e.customerName)
when(WhichInvoicesAreOverdue q)
SELECT * FROM OverdueInvoices
</code></pre></div></div>
<h3 id="anti-pattern">Anti-pattern</h3>
<p>All patterns become anti-patterns when applied incorrectly, but <em>Fat Events</em> certainly have their share of risks.</p>
<p>Adapting the API of a producer to fit the needs of the consumer, should always be done with care. With lots of consumers that each have their own specific demands, it could quickly become mess. Events become huge, and as consumers come and go, it becomes unclear wich attributes are still relevant. Changing the producer becomes harder. This is exactly the kind of situation that the <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee/">Completeness Guarantee</a></em> pattern was supposed to avoid. A useful heuristic is to consider the number of consumers and their ownership. If a single team owns both the producer and consumer, <em>Fat Events</em> pose less of a risk than if a large number of teams each want different redundant attributes in the messages.</p>
<h3 id="eventual-consistency-and-immutability">Eventual consistency and immutability</h3>
<p>In many cases, there’s a risk that the consumer’s state becomes inconsistent with the producer’s state. The invoice example above is chosen to illustrate this. In the second design, where we use the fat <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> event, suppose the customer is renamed <em>after</em> the invoice became overdue. Now the consumer has a list of overdue invoices with some outdated customer names. In the original design, this problem did not exist, because there it doesn’t matter if the customer name changes before or after the invoice became overdue, the state is always updated.</p>
<p>This inconsistency may not be a problem. If, say, only a very small number of customers do in fact change their name, and the invoies only stay unpaid for a short while, the chances of actual inconsistency become slim. And in the rare case it happens, the user might see the old name in one place and the new name in another place, and work out easily that it’s the same invoice. As always, it helps to carefully consider the domain, the users, their expectations and behaviours.</p>
<p>A very powerful heuristic is that immutable values are always safe to copy. If in the producer, a value can never change after it was initially set, there is no risk that the copy in the consumer becomes inconsistent. In our example, if we don’t allow customers to change their name, the <em>Fat Event</em> that carries the name is a fine solution to simplify the consumer.</p>
<h3 id="identifiers">Identifiers</h3>
<p>A special case of immutability are identifiers. In a well-designed system, IDs never change. So assuming that the customers can change their name, but that all IDs are stable, the <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> event can be designed to contain only <code class="language-plaintext highlighter-rouge">invoiceId</code> and <code class="language-plaintext highlighter-rouge">customerId</code>, but no <code class="language-plaintext highlighter-rouge">customerName</code>. The consumer now needs to track customer names again, but doesn’t have to deal with a separate <code class="language-plaintext highlighter-rouge">Invoices</code> lookup table.</p>
<p>Because of the safety of copying IDs, as a rule of thumb I recommend to always include redundant IDs in events. The cost is extremely low, and it simplifies consumers. It becomes especially important when dealing with multiple relations between the entities described by the events.</p>
<p>Consider:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerHasSignedUp {customerId, customerName}
OrderWasPlaced {orderId, customerId}
InvoiceWasMadeForOrder {invoiceId, orderId}
PaymentWasReceived {paymentId, invoiceId}
</code></pre></div></div>
<p>In this example, a consumer interested in payment, needs three lookup tables to associate a customer name with a payment. With <em>Fat Events</em>, each event has the redundant, but immutable, IDs, and only a single lookup table is required:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerHasSignedUp {customerId, customerName}
OrderWasPlaced {orderId, customerId}
InvoiceWasMadeForOrder {invoiceId, orderId, customerId}
PaymentWasReceived {paymentId, invoiceId, orderId, customerId}
</code></pre></div></div>
<h3 id="separated-payload">Separated payload</h3>
<p>If you’re concerned about putting redundant attributes in a an event, a nice trick is to separate the payload into the actual state changes, and the “fat”. This makes it obvious to other developers that these attributes are not part of the core event.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>InvoiceBecameOverdue { core: {invoiceId}, fat: {customerName} }
</code></pre></div></div>
<h3 id="summary-event">Summary Event</h3>
<p>The <em>Fat Event</em> pattern is distinct from the <a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event/">Summary Event</a> in a few ways:</p>
<ol>
<li>A <em>Summary Event</em> usually happens at the end of a business process, whereas all events can be <em>Fat Events</em>.</li>
<li>A <em>Summary Event</em> is often (but not necessarily) a new event type, whereas <em>Fat Events</em> are existing event types with additional attributes.</li>
<li>A <em>Summary Event</em> contains often a complete representation of the resource or artefact that resulted from the business process, whereas <em>Fat Events</em> usually only contain a few specific extra attributes the consumer cares about. <em>Summary Events</em> are more likely to be useful for a range of different consumers.</li>
</ol>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Patterns for Decoupling in Distributed Systems: Explicit Public Events2019-05-11T00:00:00+00:00https://verraes.net/2019/05/patterns-for-decoupling-distsys-explicit-public-events<h2 id="explicit-public-events">Explicit Public Events</h2>
<p>Mark a small subset of events as public, keep the rest private by default.</p>
<h3 id="problem">Problem</h3>
<p><em>Domain Events</em> are not only useful for communicating to other <em>Bounded Contexts</em>, but also for organising and decoupling code inside the <em>Bounded Context</em>. In CQRS/Eventsourcing architectures, this is even the primary way of organising the code. It’s very convenient to publish all these events using a single bus, effectively making them all public. This is a problem when some of these events contain sensitive data.</p>
<p>More importantly, the outside API is tightly coupled to the internal structure of the Bounded Context. Changing the internals would force an API change.</p>
<h3 id="solution">Solution</h3>
<p>Set up separate messaging channels for inside the Bounded Context and outside. Keep all events private by default, and indicate the ones you want to make public with an explicit <code class="language-plaintext highlighter-rouge">@Public</code> annotation, a marker interface, or an <code class="language-plaintext highlighter-rouge">isPublic():bool</code> method. When emitting events, the event publishing mechanism knows to read the annotation and either send the event on the private channel only, or on both the private and the public channel.</p>
<h3 id="discussion">Discussion</h3>
<p>In general, my feeling is that the problem seldom occurs when the <em>Domain Events</em> have been chosen carefully to reflect the business domain, using <em>Ubiquitous Language</em>, and at the right granularity. These events then tend to become stable very quickly during development, and rarely need to be altered. When there are significant changes in the domain, the events may need to change, but in these cases you’ll need an API change anyway.</p>
<p>That said, it’s a universally useful heuristic in software design to keep everything as closed off as possible, and only open up things where there’s a good case for it.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Patterns for Decoupling in Distributed Systems: Passage of Time Event2019-05-10T00:00:00+00:00https://verraes.net/2019/05/patterns-for-decoupling-distsys-passage-of-time-event<h2 id="passage-of-time-event">Passage of Time Event</h2>
<p>Replace cron jobs and scheduled commands, with an agnostic event to indicate the passage of time.</p>
<h3 id="problem">Problem</h3>
<p>Many business processes involve some action or job or workload that needs to be performed at a future date. It can be a one-off action or a repeated action, it can be scheduled for a specific date (eg Christmas), a recurring date (the last weekday of the month) or a timeout (thirty days from now).</p>
<p>We want to keep the domain logic for the entire process nicely isolated in a single (micro)service, so that’s also where the logic for this action goes. But there is a small amount of logic that is not inside this service: the fact itself that the action needs to happen, and when it needs to happen.</p>
<p>Cron is the most troublesome: it only works for repeated actions, and offers little control to outside tools. In large systems, the cron file can be a huge mess. More modern solutions allow us to schedule using by calling an API from within our code, so that logic moves to our own service. But fundamentally, the scheduler is still a separate service that knows more than it should. In DDD lingo, the Ubiquitous Language of our business process is leaking. We’ve got more types of elements, and more moving parts in our system.</p>
<h3 id="solution">Solution</h3>
<p>The mind switch is to think of the passage of time as just another <em>Domain Event</em>, exactly like all the other events. After all, if we define a <em>Domain Event</em> as a granular point in time where something happened that is relevant to the business, then certainly the next business day, month, or quarter, is extremely relevant.</p>
<p>In the new design, a cron or scheduler emits generic <em>Passage of Time Events</em> at regular intervals, such as a <code class="language-plaintext highlighter-rouge">DayHasPassed {date}</code> event at midnight, or a <code class="language-plaintext highlighter-rouge">QuarterHasPassed {year, quarter}</code>. All interested services listen for this event. They can react to it, by performing an action, by increasing a counter, or by querying some database and filter by date, to find items that have some work to be done.</p>
<h3 id="example">Example</h3>
<p>Time governs everything. It’s hard to look at a business and not find examples where the <code class="language-plaintext highlighter-rouge">Passage of Time</code> pattern is useful: billable hours, subscriptions, resource usage, rentals, accruing interest, payments, reporting, salaries, maintenance schedules, and everything that is cyclic.</p>
<p>On an invoice’s due date, we need to send a reminder to the client. In the old design, we can set up a cron job that calls <code class="language-plaintext highlighter-rouge">CheckForOverdueInvoices</code>. In the new design, the cron generates <code class="language-plaintext highlighter-rouge">DayHasPassed</code> every midnight. The <code class="language-plaintext highlighter-rouge">InvoiceDebtCollection</code> listens to <code class="language-plaintext highlighter-rouge">DayHasPassed</code>. Whenever this event arrives, <code class="language-plaintext highlighter-rouge">InvoiceDebtCollection</code> queries <code class="language-plaintext highlighter-rouge">SELECT * FROM Invoice AS i WHERE DATEDIFF(i.dueDate, NOW()) >= 30</code>. It can either send the reminder directly, but even better would be to emit a new <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> event. Now, the service can listen to its own event and send the reminder. Other services can react to <code class="language-plaintext highlighter-rouge">InvoiceBecameOverdue</code> as well, for example to adjust a revenue projection or suspend an account.</p>
<p><em>Passage of Time Events</em> can also be a little more domain-specific. The NASDAQ’s pre-market trading hours are from 04:00 to 09:30, followed by normal trading hours until 16:00, and after-hours trading until 20:00. On holidays there’s no trading. A service can produce events for each start and close, that can be consumed by many interested services.</p>
<h3 id="benefits">Benefits</h3>
<p>It’s very explicit. In the above example, an event log would show:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>CustomerWasInvoiced
DayHasPassed
DayHasPassed
...
InvoiceBecameOverdue
ReminderWasSent
AccountWasSuspended
</code></pre></div></div>
<p>It makes writing tests for business processes with a time element very elegant:</p>
<div class="language-gherkin highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kn">Scenario</span><span class="p">:</span> If no payment is received after 15 days, we suspend the account
<span class="nf">Given </span>CustomerWasInvoiced
<span class="nf">And </span>DayHasPassed
<span class="nf">And </span>DayHasPassed
<span class="nf">And </span>(...)
<span class="nf">When </span>DayHasPassed
<span class="nf">Then </span>InvoiceBecameOverdue
<span class="nf">And </span>AccountWasSuspended
<span class="kn">Scenario</span><span class="p">:</span> If payment is received in time, we do nothing
<span class="nf">Given </span>CustomerWasInvoiced
<span class="nf">And </span>DayHasPassed
<span class="nf">And </span>DayHasPassed
<span class="nf">When </span>PaymentWasMade
<span class="nf">Then </span>Nothing
</code></pre></div></div>
<p>(You’d probably want to add a shortcut so you don’t need to write the <code class="language-plaintext highlighter-rouge">DayHasPassed</code> line 15 times.)</p>
<p>More importantly, it’s a nicely reactive approach. When a service sends a <em>Command</em> to another service, it needs to know that this other service accepts that <em>Command</em>. When all we do is send generic <em>Passage of Time Events</em>, the scheduler doesn’t need to know who’s listening, how the consumers should react, or if there’s even any service left that listens. All decisions and domain knowledge lie with the reciever. That’s great decoupling.</p>
<p>It’s also temporal decoupling: The scheduler can put this <em>Passage of Time Event</em> on a queue, and it doesn’t matter if the consumers are available at this time to handle the event. For all it cares, a consumer could be down for a few days, and then simply catch up and process all the <code class="language-plaintext highlighter-rouge">DayHasPassed</code> events on the queue.</p>
<p>In Event Sourcing, you simply store the <code class="language-plaintext highlighter-rouge">DayHasPassed</code> events in the event store, so you can play back entire histories exactly as they happened over time, with no dependency on an external source.</p>
<h3 id="implementation">Implementation</h3>
<p>The <em>Passage of Time Events</em> use the exact same format and protocol as your other <em>Domain Events</em>. In code, you can make classes for them, which are exactly like the classes for other <em>Domain Events</em>. And you send them over the same messaging infrastructure as everything else. This makes this pattern very cheap in terms of implementation.</p>
<h3 id="weaknesses">Weaknesses</h3>
<p>There is some domain knowledge in the other direction: The domain logic to calculate when something needs to happen, such as “every 10th of the month”, has now moved effectively from the cron or scheduler, to the service. In practice, it’s not a big deal, as you can find time libraries that can do the work for you. On the upside, “every 10th of the month except in weekends, during full moons, or during the yearly office party” is something no scheduler will do for you anyway.</p>
<p>It’s worth noting that you don’t want to use the <em>Passage of Time Event</em> pattern for realtime systems. We can easily spare the 365 events per year for <code class="language-plaintext highlighter-rouge">DayHasPassed</code> but for systems that deal with minutes or seconds or less, it’s not feasible. Luckily for programmers, in most businesses, the word “immediately” means by “by the end of business day”, or “by the end of the week”, or “before the end of the quarter following the month where the thing happened”.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Patterns for Decoupling in Distributed Systems: Summary Event2019-05-08T00:00:00+00:00https://verraes.net/2019/05/patterns-for-decoupling-distsys-summary-event<h2 id="summary-event">Summary Event</h2>
<p>Instead of emitting a stream of Domain Events, emit a single Summary.</p>
<h3 id="problem">Problem</h3>
<p>A business process involves a number of steps that each produce domain events. A consumer depends on information in those events, and listens to all of them to make meaningful decisions. This in itself is perfectly fine, especially when it’s only a small number of event types. When the number of event types is large, we can get concerned about the amount of knowledge the consumer now has of the producer’s business process. The consumer is now almost like a micromanager, looking over the shoulder of the producer and getting immediate reports of everything the producer does.</p>
<p>This has a risk: when the original business process evolves, event types might be added, removed, or altered. This requires a change in the consumer, which needs to be adapted to the new types, and either be deployed at the same time or have a good versioning strategy.</p>
<p>And it gets worse if there are multiple consumers, that all need to be aware of every type of event the producer emits. There’s often also quite some duplication: each consumer projects similar or identical state off of the events.</p>
<h3 id="solution">Solution</h3>
<p>Instead of emitting each event, the original business process can keep all the events private. At the end of the process, the process emits a single <em>Summary Event</em>. This event is redundant, in the sense that it contains only information that was already available in the preceding events. Consumers, instead of being aware of every event, are now only listening to this <em>Summary Event</em>, which tells them everything they need to know, with little to no irrelevant (to them) information. Consumers do not need to track state changes during the process, because they get everything at the end, and can then act on them.</p>
<p>You can recognize opportunities to apply the <em>Summary Event</em> pattern, by looking at the consumer. If the consumer listens to multiple event types, but there’s only one event type that causes the consumer to do some meaningful domain action (besides keeping state), it’s a good sign. The producer offers clues as well: the business process involves creating some kind of artefact, and the business value is not necessarily in the artefact itself, but in the usage of the artefact after it has been created. (@TODO Cfr Three Archetypes by Cyrille)</p>
<h3 id="example">Example</h3>
<p>Many business processes involve creating an artefact for later productive use. During an <code class="language-plaintext highlighter-rouge">Ordering</code> process, different users might add and remove line items of items they wish to order. They might change the quantities of the items they want. And an account manager might change the prices, change the quantities based on available stock, and so on. In the original design, the consumer listens to <code class="language-plaintext highlighter-rouge">LineItemWasAdded</code>, <code class="language-plaintext highlighter-rouge">LineItemWasRemoved</code>, <code class="language-plaintext highlighter-rouge">QuantityWasChanged</code>, <code class="language-plaintext highlighter-rouge">PriceWasChanged</code>, as well as events such as <code class="language-plaintext highlighter-rouge">ShippingAddressWasAdded</code>. The consumer, say an <code class="language-plaintext highlighter-rouge">OrderFulfilment</code> process, listens to most of these events, and projects a list of of items that need to be packed and shipped.</p>
<p>But <code class="language-plaintext highlighter-rouge">OrderFulfilmentProcess</code> doesn’t do anything until the <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> event arrives. That’s when it kicks into action. The <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> doesn’t have redundant information, so it mostly consists of an <code class="language-plaintext highlighter-rouge">OrderId</code>, and not much else.</p>
<p>We can now refactor the <code class="language-plaintext highlighter-rouge">Ordering</code> process, to keep all its events private. Instead, we make a single public event for <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code>, but this one contains the complete information about the order. In fact, one way to look at it is that <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> is the actual order document. (@TODO I believe this is the Document Message pattern in Enterprise Integration Patterns, I need to look it up).</p>
<p>This is of course very familiar, precisely because this is how information travels in businesses, and has traveled for ages before computers: Someone working in the Order Fulfilment department is usually not aware of all the negotiations and changes that happen to a draft order. They only get a message with the resulting order, and act on that.</p>
<p>The point of the <em>Summary Event</em> pattern is to look for opportunities to apply it in domains that are not as mature as ordering, or in situations where it is not obvious that the two highly coupled systems could reduce the communication between them using this pattern.</p>
<h3 id="implementation">Implementation</h3>
<p>You can use a <em>Projection</em> to generate the <em>Summary Event</em>. A <em>Projection</em> is an event listener that persists state from events it listens to, and either exposes that state by responding to queries, or by emitting new events.</p>
<h3 id="weaknesses">Weaknesses</h3>
<ol>
<li>
<p>Sometimes the consumer does need to know a lot of internals of the producer to do its job. In that case, the <em>Summary Event</em> pattern is not helpful, as it would not only contain the resulting information, but also metadata about each step in the process.</p>
</li>
<li>
<p>In the ordering example above, <code class="language-plaintext highlighter-rouge">OrderFulfilment</code> doesn’t care about the price of the order. Another consumer, say <code class="language-plaintext highlighter-rouge">RevenueReporting</code>, cares about the prices but not the exact items ordered. Yet they both receive the same <code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> event with everything in there. There might be cases where this is a problem, for example if some of the information is sensitive.</p>
</li>
<li>
<p>The new <code class="language-plaintext highlighter-rouge">Summary Event</code> adds yet another event type, a “new standard to replace all standards”.</p>
</li>
<li>
<p>The <code class="language-plaintext highlighter-rouge">Summary Event</code> might not naturally belong to the domain of the producer. In that case, we can add an intermediate service, that consumes the producer’s events, summarizes them, and then emits the <code class="language-plaintext highlighter-rouge">Summary Event</code> for consumption.</p>
</li>
</ol>
<h3 id="diagram">Diagram</h3>
<p>Original design:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Producer: OrderWasInitiated LineItemWasAdded PriceWasChanged ... OrderWasPlaced
| | | |
v v v v
Consumer: (update state) (update state) (update state) ...-> (start fulfilment)
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">OrderWasPlaced</code> is refactored to a <em>Summary Event</em>, the other events become private:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Producer: OrderWasInitiated LineItemWasAdded PriceWasChanged ... OrderWasPlaced
|
v
Consumer: (start fulfilment)
</code></pre></div></div>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Patterns for Decoupling in Distributed Systems: Domain Query2019-05-08T00:00:00+00:00https://verraes.net/2019/05/patterns-for-decoupling-distsys-domain-query<h2 id="domain-query">Domain Query</h2>
<p>Replace Free Queries with Domain Queries to decouple from knowledge of the server’s internals.</p>
<h3 id="problem">Problem</h3>
<p>The word query is usually associated with database queries. There are however other ways we can query a system that we don’t perceive as a database. REST and GraphQL come to mind. I propose the term <em>Free Query</em> for every type of message that knows about the structure, schema, endpoints, … and has a great freedom in accessing and combining data from that system, typically using a rich query language.</p>
<p>In a very simple setup with a single client and server, owned by a single team, this isn’t a great problem, as changes to the server and client can be applied simultaneously.</p>
<p>In a more complex environments, where multiple clients are outside the control of the server’s team, it is. The database schema or service endpoints are essentially a public API. The clients have intimate knowledge of the schema. We don’t know what queries they will send us, so the only way to guarantee backwards compatibility of that API is to keep the schema intact. Sure, we could log the queries, but that doesn’t guarantee there are yet-to-be sent queries hiding in the client.</p>
<h3 id="solution">Solution</h3>
<p>Instead of exposing the schema or endpoints and allowing <em>Free Queries</em>, define a set of domain-specific messages and responses. These <em>Domain Queries</em> consist of a name and a small number of arguments. Both the <em>Domain Queries</em> and their responses are highly tailored to the use case and only use the <em>Ubiquitous Language</em>. I recommend the name to be a question expressed in natural language.</p>
<p>The server provides a single endpoint for all <code class="language-plaintext highlighter-rouge">Domain Queries</code>, and hides any implementation details. Knowledge about where and how to fetch and compute the relevant data, is encapsulated in the server.</p>
<h3 id="example">Example</h3>
<p>The client sends the following Free Query:</p>
<pre><code class="language-SQL">SELECT *
FROM Cars AS c
INNER JOIN LeasingContracts AS lc ON c.CarId = lc.CarId
WHERE c.RegistrationYear = 2014;
</code></pre>
<p>It is unclear from the query alone what the purpose is. Presumably not all of the fields from the two tables are used by the client, but we can’t change our schema without risking to break things.</p>
<p>Some investigation learns that the client wants to know which cars are up for replacement, and uses somes fields from <code class="language-plaintext highlighter-rouge">LeasingContracts</code> to compute the status. A better design is to close off all access to the database, and provide a <em>Domain Query</em> using natural language. The server fetches and computes the results, and the <em>Response</em> message contains only the data that the client actually needs.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>WhichCarsAreUpForReplacement : Query {
registrationYear: 2014
}
CarsThatAreUpForReplacement : Response {
cars: [CarId]
}
</code></pre></div></div>
<h3 id="discussion">Discussion</h3>
<p>It may feel strange to use names like <code class="language-plaintext highlighter-rouge">WhichCarsAreUpForReplacement</code> instead of programmer-language like <code class="language-plaintext highlighter-rouge">CarReplacementQuery</code>. I assert that this is only a matter of familiarity. The benefits are that we stay much closer to the language of the business, and that it puts us in the mindset of making very deliberate expressions of use cases.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Patterns for Decoupling in Distributed Systems: Completeness Guarantee2019-05-08T00:00:00+00:00https://verraes.net/2019/05/patterns-for-decoupling-distsys-completeness-guarantee<h2 id="completeness-guarantee">Completeness Guarantee</h2>
<p>Design the set of Domain Events from a producer so that they can be used to rebuild the producer’s state.</p>
<h3 id="problem">Problem</h3>
<p>Often, the events emitted by a producer, are designed haphazardly. New event types are added whenever a new feature requires them. A consumer needs to be aware of an event, so we come up with an event in the producer, give it a name, and add some attributes the consumer needs. Then perhaps later another consumer is added, and it reuses some existing events, but it needs just a little bit of additional information. This is added as new attributes in an existing event. Later, a consumer is removed, but the events still exist.</p>
<p>After a while it becomes hard to understand which events to use where, what they contain, and what they mean exactly. Consumers become indirectly coupled to each other, and the producer needs to support backwards compatibility to a growing sets of messy events.</p>
<h3 id="solution">Solution</h3>
<p>We need an organizing principle that answers the question “What goes in an event?”, looking holistically at the set of events from a producer.</p>
<p>A <em>Completeness Guarantee</em> is the idea that if a consumer listens to the entire stream of <em>Domain Events</em>, it must be able to faithfully reproduce the entire state of the producer. Every action in the producer that changes state, causes an event to be emitted, and each event contains every unit of information that is affected by the state change. On top of that, ideally the events contain no redundant information: the event only has the attributes that have changed, and not a single one more. (Note: <em><a href="https://verraes.net/2019/05/patterns-for-decoupling-distsys-fat-event/">Fat Events</a></em> discusses cases where you’d want to add redundant attributes.)</p>
<p>The effect of a <em>Completeness Guarantee</em> is that it becomes clear and obvious which events are relevant, what they mean, what attributes they contain, etc. There is the certainty that any feature that could be implemented in the producer, can be implemented in a consumer, with no missing information. This enables a system designer to choose where to move certain responsibilities.</p>
<p>The <em>Completeness Guarantee</em> gives a great degree of decoupling to consumers: they never need to query the producer, as they already have access to all information.</p>
<h3 id="implementation">Implementation</h3>
<p>In systems that use Event Sourcing, you get the <em>Completeness Guarantee</em> for free. I define Event Sourcing as a persistence style where a store of domain events is the single source of truth for the system’s state. State may stored in other ways, but this state is discardable, and can be rebuilt from the event store. If the events are the single source of truth, it follows that all state can be rebuild, so the guarantee is sound.</p>
<p>For system designers used to Event Sourcing, the <code class="language-plaintext highlighter-rouge">Completeness Guarantee</code> comes naturally. The point of the pattern is to show that it is useful outside of Event Sourcing as well.</p>
<p>How do you actually achieve that guarantee, when the single source of truth of a system is a mode traditional database, which stores state instead of events? One approach is to write a test for it. This test listens to all events from production, and projects them into an identical schema as the producer’s real schema. The test compares to projected state to the production state, and alerts when there are differences. However, instead of going through the effort of building and running this test, you might be better off just using Event Sourcing in the first place.</p>
<ul>
<li>Read my <a href="https://verraes.net/2019/05/ddd-msg-arch/">Messaging Patterns series</a></li>
<li>I teach workshops about <a href="https://dddeurope.academy/mathias-verraes/">DDD for Messaging Architectures</a>.</li>
</ul>
Design Heuristics2018-04-26T00:00:00+00:00https://verraes.net/2018/04/design-heuristics<p><a href="https://skillsmatter.com/skillscasts/11685-we-are-delighted-to-have-mathias-verraes-at-ddd-exchange-2018#video">Video for my DDD eXchange 2018 keynote in London</a></p>
<p><a href="https://skillsmatter.com/skillscasts/11685-we-are-delighted-to-have-mathias-verraes-at-ddd-exchange-2018#video">
<img style="float:left;margin-right: 10px" src="/img/posts/2018-04-26-design-heuristics/27918963728_6a70e4c83d_edit.jpg" alt="" />
</a></p>
<h2 id="abstract">Abstract</h2>
<p>Software design principles aspire to be universal. And yet, when you create software, you sometimes intentionally violate principles.</p>
<p>You might not be able to explain why this “wrong” design somehow “feels” better. You’re applying your own, unspoken design heuristics. Bringing them out in the open, improves design discussions, and helps you build a larger toolkit for making better tradeoffs.</p>
<p>In this session, Mathias will explore the benefits of distilling your design heuristics using practical examples.</p>
Emergent Boundaries2017-04-27T00:00:00+00:00https://verraes.net/2017/04/emergent-boundaries<ul>
<li><a href="https://skillsmatter.com/skillscasts/9847-emergent-boundaries#video">Video of my DDD eXchange 2017 talk in London</a></li>
<li><a href="https://www.youtube.com/watch?v=ECM1rPYxvD4">Video of my ExploreDDD 2017 talk in Denver</a></li>
</ul>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/ECM1rPYxvD4" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<h2 id="abstract">Abstract</h2>
<p>Modelling is more than knowledge management. It’s complexity management. To reduce cognitive load, you split things up, whether at small scale, in code, or in the large, such as Bounded Contexts. But if you choose large boundaries upfront, you risk getting it wrong, and being stuck to them for a long time. Join Mathias in a discussion of better ways of empirically discovering boundaries instead.</p>
Messages Over Structure2017-04-03T00:00:00+00:00https://verraes.net/2017/04/messages-over-structure<p><a href="https://www.youtube.com/watch?v=2nDNMB_wXcE">Video of my keynote at Neos Conference 2017 in Hamburg</a></p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/2nDNMB_wXcE" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
On Being Explicit2016-11-09T00:00:00+00:00https://verraes.net/2016/11/on-being-explicit<p><a href="https://skillsmatter.com/skillscasts/8915-on-being-explicit">Video of my talk at Agile Testing & BDD eXchange 2016 in London</a>
<a href="https://skillsmatter.com/skillscasts/8806-ddd-meetup">Video of my talk at the DDD London meetup</a></p>
<p>Slides:</p>
<script async="" class="speakerdeck-embed" data-id="70c97069d81543deb57b8cf974f5b58b" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script>
<h2 id="abstract">Abstract</h2>
<p>“Make the implicit explicit” must be one of the most valuable advices I ever got about software modelling and design. Gather around for some tales from the trenches: stories from software projects where identifying a missing concept, and bringing it front and centre, turned the model inside out. Our tools: metaphors, pedantry, type systems, the age old heuristic of “Follow the money”, visual models, and a healthy obsession with language.</p>
Designed Stickiness2016-05-24T00:00:00+00:00https://verraes.net/2016/05/designed-stickiness<p>Video & slides for my NCrafts talk on software metaphors.</p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://player.vimeo.com/video/167722762" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture fullscreen" allowfullscreen=""></iframe>
</figure>
<script async="" class="speakerdeck-embed" data-id="ebac02f6dafb48baa086edb30fdd3a49" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script>
<h2 id="abstract">Abstract</h2>
<p>When we model, we tend to do it for ourselves: gain understanding, capture the business language, and turn it into running code. But are we missing opportunities to do something more with our models? What if, instead of mirroring the business, our models could transform it? Let’s find out how a healthy obsession with language and metaphor can step up from “domain-driven” to “driving the domain”.</p>
Lightning Talks2016-05-05T00:00:00+00:00https://verraes.net/2016/05/lightning-talks<p>I submitted proposals to conferences for almost two years before getting accepted. When it finally happened, I found myself in front of a 250 head audience, realising <em>I didn’t actually know how to do proper presentations</em>. I wish I knew then that there are better ways.</p>
<p><strong>Lightning talks at meetups are ideal to practice public speaking</strong>, and experiment with explaining or pitching something in a short time. I’m trying to write one each month for the Software Craftsmanship Belgium meetups. It’s only 30 attendees, so it’s an opportunity to <strong>try things that I wouldn’t do as a proper conference talk yet</strong>. And I can recycle the ones that do work for other meetups or conferences.</p>
<p>My self-inflicted constraints:</p>
<ul>
<li><strong>10 minutes max</strong>, or 5 minutes + Q&A.</li>
<li>Write the slides in a <strong>single evening</strong>.</li>
<li>Take a topic that I haven’t talked or written about before. Trying to explaining a topic you don’t know well, is a great way to learn.</li>
<li><strong>Assume the audience doesn’t know the topic</strong> either, so it needs to cover the basics.</li>
<li>When showing code, it needs to be <strong>understandable for people unfamiliar with the language</strong>, and within the 30 seconds or so that the code is on the screen. I might write unidiomatic Haskell, or leave out anything complicated, or just show pseudo-code.</li>
</ul>
<p>Not constrained by:</p>
<ul>
<li><strong>Being original</strong>. It doesn’t have to be my own inventions, my own examples, etc. Presenting a blog post written by somebody else is perfectly fine.</li>
<li><strong>Being precise, perfect, or nuanced</strong>. First get it out there, then perhaps iterate later.</li>
</ul>
<p>I’ll post the slides here (reverse chronology). They’re probably useless without the explanation, but perhaps someone gets something out of them.</p>
<h2 id="how-to-find-the-bar">How to Find the Bar</h2>
<p>August 2016, SoCraTes Belgium
<script async="" class="speakerdeck-embed" data-id="b72a0afd9eff4a5db78be607bbdfdb01" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script></p>
<h2 id="the-monty-hall-problem">The Monty Hall Problem</h2>
<p>May 2016, SoCraTes Belgium
<script async="" class="speakerdeck-embed" data-id="b0c29a5d8501415ca9b4edb66e397e02" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script></p>
<h2 id="property-based-testing">Property Based Testing</h2>
<p>April 2016, SoCraTes Belgium and CukeUp London
<script async="" class="speakerdeck-embed" data-id="eedd62be375f4802b61e3a6afc683c84" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script></p>
<h2 id="the-worlds-shortest-and-most-chaotic-introduction-to-event-storming">The World’s Shortest and Most Chaotic Introduction to Event Storming</h2>
<p>April 2016, CukeUp London
This was a hands-on Event Storming session with a short introduction.)
<script async="" class="speakerdeck-embed" data-id="dc2dac4a332f44d6bc338f3a58660b8e" data-ratio="1.77777777777778" src="//speakerdeck.com/assets/embed.js"></script></p>
The Repair/Replace Heuristic for Legacy Software2016-04-26T00:00:00+00:00https://verraes.net/2016/04/repair-replace-heuristic-for-legacy-software<p><img style="float:right;margin-left: 10px" src="/img/posts/2016-04-27-repair-replace-heuristic-for-legacy-software/david-d-angers-small.jpg" alt="Galerie David d'Angers" /></p>
<p>Technical Debt is a great metaphor. It shares many analogous properties with financial debt: loans, accrued interest, token payments, bankrupty… There is a key difference however. <strong>We take financial debt with another party.</strong> This party desires to get their money back eventually, and then some. Over centuries, a system was developed to measure debt fairly unambiguously. We called it money, and we’re still iterating and experimenting with new designs, like Blockchain. A second system was developed, to create strong incentives for parties to honour debt agreements. We called this system Property Laws.</p>
<p>Technical Debt has no measure like money, and no ruleset like Property law, and, more importantly, <strong>with Technical Debt there is no other party</strong>. The organisation is <strong>both the creditor and debtor</strong>. This takes away all incentive to control how Technical Debt is taken out and paid back. If you ask your CFO how much financial debt your organisation has, she’ll show you spreadsheets, graphs, payment plans, and projections. She makes quarterly reports for the shareholders. Ask your CTO about the Technical Debt in your company, and you’ll get a vague “um… a lot”, or “lol idk, u can’t measure that bro”.</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2016-04-27-repair-replace-heuristic-for-legacy-software/tied_by_its_shoelaces.png" alt="Technical Debt by Ruth Malan" /></p>
<p>In “<a href="/2013/07/managed-technical-debt/">Managed Technical Debt</a>”, I propose a cheap, imprecise, but surprisingly effective method for mapping and measuring debt. In short, it involves posting stickies whenever progress is impeded by debt, and keep marking the stickies for every incident. Over time, we build a map that is not based on perceived quality, but on the actual cost of debt. The <em>Wall of Technical Debt</em> serves as a decision tool for focusing our repayment efforts.</p>
<h2 id="heuristics">Heuristics</h2>
<p>So now we have identified our worst Technical Debt, and made estimations on its relative size. What should we do, replace it or repair it?</p>
<p>The Wall of Technical Debt is a <strong>structural model</strong>, but I like to build <strong>temporal models</strong>. Move beyond “<em>What debt do we have?</em>” towards “<em>How did we get here? What happened before? What was it like back then?</em>”. Call it archaeology, or historical thinking if you will. The point is, you learn a lot from temporal models that you don’t get from a mere structural model of the here and now.</p>
<p>When evaluating our legacy systems, these questions become “<em>What was this system like before? Did people like building it? Did users like using it? Did it meet goals, did it balance the trade-offs well? Has the purpose of the system changed, or have the constraints changed?</em>” Perhaps the system was well-designed, but not designed for scale, or for mobile, or for multi-tenancy. Businesses change, markets change, regulation changes, technology changes.</p>
<blockquote><p>A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system.<small>Gall's Law, from <a href="http://amzn.to/1Qz1Yhg">Systemantics, John Gall, 1975</a></small></p></blockquote>
<p>Based on this, I propose:</p>
<p><strong>The Repair/Replace Heuristic for Legacy Software:</strong></p>
<ul>
<li><strong>If the system had good design, repair it;</strong></li>
<li><strong>If the system had bad design, replace it.</strong></li>
</ul>
<p>If the system used to work well, and it was well-designed, meaning its structure was congruent with the purpose and the trade-offs, there might still be a lot of value in it. There’s a lot of <strong>learning encapsulated in its design</strong>, and we don’t want to throw out the (middle-aged) baby with the bathwater. We compare the purpose and the constraints the system had back then with the current ones. From that, we might learn which adaptations can give it a second life. With luck, the system was designed with flexibility in mind, and can be changed cheaply.</p>
<p>On the other hand, a system that never worked really well, is likely beyond saving. Replacement is a more attractive alternative.</p>
<p>That said, it’s a <strong>heuristic, not a rule</strong>. There are many other forces that affect the outcome of a Replacement project, and you’ll need to understand them to succeed. <a href="http://amzn.to/1VQSNRW">Second-System Syndrome</a> is real, and this industry has spent insane amounts of money on failed attempts.</p>
<p>Even when you’re not trying to decide on <strong>Repair/Replace</strong> — perhaps the decision was already made by others — the <em>process</em> of mapping its history will teach you more about the system and and its design. And one deep insight you learn from temporal modelling, is what is captured in <em>Boulding’s Backward Basis</em>:</p>
<blockquote><p>Things are the way they are because they got that way.<small>
<a href="http://amzn.to/234iIUW">The Secrets of Consulting, Gerald M. Weinberg, 1985</a></small></p></blockquote>
<p>Or, an ounce of prevention is better than a pound of cure.</p>
<p>(Update May 5 2016:)</p>
<p>When I say system, I don’t necessarily mean “the entire system”. Any large system, no matter if it is monolithic, can be thought of as a composition of a systems. They may be tightly coupled, and hard to replace or repair individually, but they have their own purpose, history, forces, and authors. And, just to drive home the point: what I propose above is a heuristic. Good decision making uses a multitude of heuristics and balances all forces.</p>
<p><small><a href="http://www.ruthmalan.com/Journal/2013/2013JournalMarch.htm#Technical_Debt">Shoelaces cartoon by Ruth Malan</a></small></p>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="/2013/07/managed-technical-debt/">Managed Technical Debt</a></li>
</ul>
Property-based testing2016-04-14T00:00:00+00:00https://verraes.net/2016/04/property-based-testing<p><a href="https://skillsmatter.com/skillscasts/8088-property-based-testing">Video for my CukeUp 2016 lightning talk in London</a></p>
Code Reviews and Blame Culture2016-04-09T00:00:00+00:00https://verraes.net/2016/04/code-reviews-and-blame-culture<p><em>Gated pre-merge code reviews are bad. Always refactor on master. Always do Pair Programming. Don’t use branches. Apply Continuous Integration.</em></p>
<p>Best practices likes these are great. They <strong>encode experiences of many individuals and teams into memorable slogans, and help others to make decisions on how to run their teams processes</strong>. Unfortunately, the not-so-great practices also get their share of the slogans. Phrases like <em>“TDD is dead”</em> and <em>“Test-induced design damage”</em> are just as catchy.</p>
<p>The human brain likes to rationalise. We like to <strong>mix and match the advice that fits our narrative</strong>. When for example, our code makes writing unit tests hard, adopting <em>“TDD is dead”</em> as a slogan is much more comfortable than learning how to do safe refactors and introduce a test harness. Combine a lack of automated tests with the advice to refactor on master, and you’ll get a recipe for disaster.</p>
<p>As a consultant, sometimes I get hired to help out in high performing teams that deploy great quality software daily. More often though, the environment is less than ideal. If the idea of bringing in external help was a tough sell to management, I’m rarely there for more than a few days. I could come in, recite all the best practices, and be gone (and I’m afraid I’ve made that mistake on occasion). More valuable is to <strong><a href="/2014/03/small-controlled-experiments/">look for the small improvements that can have a lasting impact</a></strong> and put the team on the path to improvement. If that’s just enough wind in the sails to show that progress is possible, that might be enough to get them to take control of their own situation.</p>
<p><img style="float:right;margin-left: 10px; margin-bottom: 20px" src="/img/posts/2016-04-09-code-reviews-and-blame-culture/treasure-medium.jpg" alt="Gradual improvement" /></p>
<p>I advocated <a href="/2013/10/pre-merge-code-reviews/">Pre-merge Code Reviews</a> on this blog a couple of years ago, and I still stand by it. A common believe is that this kind of gated reviews lead to blame culture. If you work in a high performing team, and do a lot of Pair Programming (or even Mob Programming!), then shifting to reviews is indeed a step down, that removes trust. But simply shouting on Twitter that peer reviews are bad, displays <strong>a lack of appreciation for the contexts of others</strong>.</p>
<p>Many teams have no tests, no refactoring skills, and a high degree of blame culture, where individuals get blamed for breaking the code. Not coincidentally, managers in these teams tend to believe Pair Programming will cut the productivity in half. Refactoring is high risk, so nobody does it out of fear of getting blamed.</p>
<p>In my experience, reviews are often sellable here. And, counter-intuitively, <strong>Pre-merge Code Reviews can actually reduce blame culture!</strong> You need some simple rules:</p>
<ol>
<li><strong>Open pull requests early;</strong></li>
<li><strong>Never merge your own code;</strong></li>
<li><strong>Never merge code you don’t understand or disagree with;</strong></li>
<li><strong>Always review others’ code before starting on new tasks;</strong></li>
<li><strong>Commenting on others’ pull requests is good, adding commits is better.</strong></li>
</ol>
<p>The interesting effect is that bugs, regressions, bad design, and lack of tests, are now no longer the fault of the person who wrote the code. <strong>It is the team that has failed to catch the issues. If you fail collectively, you’ll be induced to find solutions collectively.</strong> To reduce the effort of reviews, perhaps you’ll start adding some automated smoke tests. The senior team members might put more effort into teaching the juniors. Maybe two people will spend a Friday afternoon figuring out how to get that test framework up and running.</p>
<p>I believe this is a valuable lesson for all of us: When we advocate a best practice, or take advice from someone else, we need to assume there’s always a context. There’s a history of gradual improvement, a path taken from where we were to where we are. Others haven’t taken this path yet, or might end up on a different one. Best practices show a destination, but we can all put some more effort into putting up signposts along the way.</p>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="/2013/10/pre-merge-code-reviews/">Pre-merge Code Reviews</a></li>
<li><a href="/2014/03/small-controlled-experiments/">Small Uncontrolled Experiments</a></li>
</ul>
Type Safety and Money2016-02-29T00:00:00+00:00https://verraes.net/2016/02/type-safety-and-money<!-- photo credit http://s1153.photobucket.com/user/campbellchevelle/media/MoneyTree5_zpszzjjrpdw.jpg.html -->
<p>Below is an attempt at illustrating a design/redesign process I went through at a client, who’s started refactoring the core systems their business depends on. Design is the part of software development that is the most messy, the hardest to fit into rules or well-defined processes. In fact, while writing this post, I tweeted:</p>
<blockquote>
<p><a href="https://twitter.com/mathiasverraes/status/704250689107709952">“There are surprisingly few software design books that recommend taking a walk, a shower, or a nap, as an important step.”</a></p>
</blockquote>
<p>None of the solutions offered below should be taken as truth. I may have already changed my mind on some of them by the time you read them.</p>
<h2 id="history">History</h2>
<p>In 2010, I wrote a small library with some Value Objects to represent money, and in 2011, I published a first version. It was a typical open source scratch-your-own-itch thing, that solved a problem with incorrect rounding and conversion in an e-commerce project I was working on. It seems to have scratched many itches, as it got quite popular.</p>
<p>I’ve been a bad parent to the project lately. It’s in a <a href="https://github.com/moneyphp/money">new GitHub organisation</a> now, with @sagikazarmark as the new maintainer, who’s blowing new life into it.</p>
<p>Over the years, a bunch of forks and variations appeared. I welcomed this, as many of them serve a need that my library didn’t. It freed me from having to support all possible use cases, and avoided feature bloat.</p>
<h2 id="design-flaws">Design Flaws</h2>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/MoneyTree-small.jpg" alt="MoneyTree" /></p>
<p>Apart from being incomplete, the library suffers from fundamental design flaws, some of which can’t be fixed without breaking it.</p>
<ul>
<li>It doesn’t support higher precision than the currency’s default precision.</li>
<li>As a result of this, it always rounds, so if you chain operations, you end up with rounding errors.</li>
<li>It doesn’t support blockchain currencies, or virtual, vendor-specific currencies, like Facebook Credits (now discontinued), casino chips, game credit…</li>
<li>Having to enter values in integers was often impractical (eg €1 had to be written as 100.)</li>
<li>Some currencies have mills (1/1000<sup>th</sup> of a unit) as their official division (although they are becoming rare). Bitcoin supports 10<sup>−3</sup> (a millibitcoin), 10<sup>−6</sup> (a microbitcoin or a bit), and 10<sup>−8</sup> (a satoshi). Reportedly, the community is considering to introduce even smaller divisions. The library doesn’t support any of this.</li>
</ul>
<h2 id="requirements">Requirements</h2>
<p>This is what we expect from our model:</p>
<ol>
<li>We need to support about 10 currencies, possibly more in the future. When we do add new currencies, it is assumed that the in-house developers will add support in the code. There’s no UI for adding new currencies.</li>
<li>All operations need to done with a precision of 8 decimals. This is a business decision.</li>
<li>When showing amounts to users, or pass them along over an API, we always stick to the currency’s official division. In the case of Bitcoin, that does indeed mean we’re keeping a precision of 1 satoshi aka BTC 10<sup>−8</sup>.</li>
<li>In some markets, our software will need to do specific compliance reporting.</li>
<li>All internal reporting needs to be in EUR, no matter what the original currency was.</li>
<li>We’re using some legacy and third-party systems, that also publish revenues to our internal reporting tool. Most of them only support the currency’s official division, and can’t deal with higher precision.</li>
</ol>
<h2 id="assumptions">Assumptions</h2>
<p>To understand this post, I’m assuming you know:</p>
<ul>
<li>what a Value Object is, and why you’d want some;</li>
<li>how you can make your objects immutable;</li>
<li>why they <em>should</em> be immutable;</li>
<li>how you can make operations safer, eg by disallowing $1+€1 without explicit conversion;</li>
<li>why splitting €0.05 between two parties is not €0.025+€0.025 but €0.03+€0.02.</li>
</ul>
<h2 id="fowler-style">Fowler-style</h2>
<p>Let’s design!</p>
<p>Our first instinct is to have the <code class="language-plaintext highlighter-rouge">Money</code> object from <a href="http://amzn.to/1TN7Tq4">Fowler’s PoEAA</a>, consisting of a float<sup>*</sup> and a currency, which in turn could be an object as well. Both are immutable.</p>
<p>(<sup> * </sup> You may not want to use the actual float type in your programming language for dealing with money, as you may run into issues with precision. Do your research.)</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/Money.png" alt="Money" /></p>
<p><code class="language-plaintext highlighter-rouge">Currency</code> is an enum type supporting our 10 currencies (A simple assertion will do if your language doesn’t have enums). We can’t initialise the <code class="language-plaintext highlighter-rouge">Currency</code> object with any other value than our 10 currencies. It only uses the 3-letter ISO symbols, anything else is an error. That satisfies requirement 1.</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/MoneyFormatter.png" alt="MoneyFormatter" /></p>
<p><code class="language-plaintext highlighter-rouge">Money</code>’s constructor can round floats with more than 8 decimals. We can add some operations, like <code class="language-plaintext highlighter-rouge">add(Money other) : Money</code> and <code class="language-plaintext highlighter-rouge">multiply(float operand) : Money</code>. They also round to 8 decimals if needed.</p>
<p>Requirement 3 (showing pretty-printed, rounded values to the users), is clearly presentation logic. We should avoid muddying up our domain model for this. So we add a <code class="language-plaintext highlighter-rouge">MoneyFormatter</code> in the presentation layer, which takes a <code class="language-plaintext highlighter-rouge">Money</code> argument and returns a string, such as €5.00 or 5,00€, depending on local standards. Perhaps we even make a HTML widget or helper of sorts, so that our templates don’t need to worry about it.</p>
<h2 id="itch">Itch</h2>
<p>Our current design doesn’t satisfy the problem of exposing the rounded values over an API. Of course, we could <code class="language-plaintext highlighter-rouge">round(money.getAmount())</code>, but <code class="language-plaintext highlighter-rouge">money.round() : Money</code> seems more in line with our existing methods.</p>
<p>This is where you should start feeling the itch. Have a close look at the type of this method. <code class="language-plaintext highlighter-rouge">money.round() : Money</code></p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">a</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="o">(</span><span class="mf">1.987654321</span><span class="o">,</span> <span class="n">eur</span><span class="o">)</span>
<span class="c1">// The constructor rounds it own to 1.98765432</span>
<span class="n">b</span> <span class="o">=</span> <span class="n">a</span><span class="o">.</span><span class="na">round</span><span class="o">()</span>
<span class="c1">// round() rounds it up to 1. 99 and instantiates a new Money</span>
<span class="c1">// Money’s constructor rounds it to 1.99000000</span>
</code></pre></div></div>
<p>Technically, most languages don’t distinguish between 1.99 and 1.99000000, but logically, there is an important nuance here. <code class="language-plaintext highlighter-rouge">b</code> is not just any <code class="language-plaintext highlighter-rouge">Money</code> , it is a fundamentally different type of money. Our design doesn’t make that distinction, and just mixes up money, whether it was rounded or not.</p>
<p>Let’s make the implicit explicit. We can rename <code class="language-plaintext highlighter-rouge">Money</code> to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>, and add a new type called <code class="language-plaintext highlighter-rouge">RoundedMoney</code>. The latter always rounds to whatever the currency’s default division is.</p>
<p>The <code class="language-plaintext highlighter-rouge">round()</code> method now becomes very simple:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">PreciseMoney</span> <span class="o">{</span>
<span class="n">round</span><span class="o">()</span> <span class="o">:</span> <span class="nc">RoundedMoney</span> <span class="o">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nf">RoundedMoney</span><span class="o">(</span><span class="k">this</span><span class="o">.</span><span class="na">amount</span><span class="o">,</span> <span class="k">this</span><span class="o">.</span><span class="na">currency</span><span class="o">)</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The chief benefit is strong guarantees. We can now typehint against <code class="language-plaintext highlighter-rouge">PreciseMoney</code> in most of our domain model, and typehint against <code class="language-plaintext highlighter-rouge">RoundedMoney</code> where we explicitly want or need it.</p>
<p>It’s easy to underestimate how valuable this style of granular types can be.</p>
<ul>
<li><strong>It’s defensive coding, against a whole category of bugs</strong>. Methods and their callers now have an explicit contract, about what kind of money they are talking about.</li>
<li><strong>It’s declarative coding</strong>. Declarative style require a lot less tests. There’s no point in testing something that is obviously correct. Code with an explicit contract like this, is obviously correct. This is why proponents of strong static type systems like to talk about <em>Type Driven Development</em> as an alternative to <em>Test Driven Development</em>.</li>
<li><strong>It communicates to different people working on the code, that we care specifically about the difference.</strong></li>
<li><strong>It introduces a concept from the domain into the Ubiquitous Language and the model</strong>. Before we did that, it would likely go unnoticed that terms like “precision” were in fact part of our language. Co-evolving the language, the models, and the implementation, is central to Domain-Driven Design.</li>
<li>This design also <strong>helps to apply the Interface Segregation Principle</strong>.</li>
</ul>
<h2 id="type-juggling">Type Juggling</h2>
<p>We’re not building this as a generic library, so we don’t need to supply a “complete” API. We can (and we should) only add the methods that we are actually going to use. In our case, <code class="language-plaintext highlighter-rouge">PreciseMoney</code> has a <code class="language-plaintext highlighter-rouge">round()</code>, but <code class="language-plaintext highlighter-rouge">RoundedMoney</code> has no <code class="language-plaintext highlighter-rouge">toPrecise()</code>. In other words, we can cast <code class="language-plaintext highlighter-rouge">PreciseMoney</code> to <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, but we can’t cast <code class="language-plaintext highlighter-rouge">RoundedMoney</code> to <code class="language-plaintext highlighter-rouge">PreciseMoney</code>. It’s a one way operation.</p>
<p>There’s an elegance to that constraint. Once you round something, the precision is lost forever. The lack of <code class="language-plaintext highlighter-rouge">RoundedMoney.toPrecise()</code> fits our understanding of our domain.</p>
<p>Perhaps we’ll eventually need some operations on <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, (although I’d rather avoid having them). We can choose some smart return types:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">RoundedMoney</span><span class="o">.</span><span class="na">add</span><span class="o">(</span><span class="nc">RoundedMoney</span> <span class="n">other</span><span class="o">)</span> <span class="o">:</span> <span class="nc">RoundedMoney</span>
<span class="nc">RoundedMoney</span><span class="o">.</span><span class="na">multiply</span><span class="o">(</span><span class="nc">Float</span> <span class="n">operand</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span>
</code></pre></div></div>
<p>Notice the different return types.</p>
<p>Perhaps you’re writing some client code that wants to do multiplication, but doesn’t care about high precision. Chaining methods would suffice here:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">someCash</span><span class="o">.</span><span class="na">multiply</span><span class="o">(</span><span class="mf">0.3333</span><span class="o">).</span><span class="na">round</span><span class="o">()</span>
</code></pre></div></div>
<p>Again, this style is very explicit. The compiler or type checker can protect us when we make mistakes against the types, such as passing <code class="language-plaintext highlighter-rouge">PreciseMoney</code> when <code class="language-plaintext highlighter-rouge">RoundedMoney</code> is required.</p>
<h2 id="conversions">Conversions</h2>
<p>Converting between different currencies depends on today’s exchange rates. We’ll need to fetch those from some third party web API. Or perhaps, another process is putting that information in a file or database somewhere. We don’t want to leak that kind of detail into our model, so we can have an Exchange interface, and have one or more implementations of our choice. The method on that interface that we’re interested in, takes a <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and a target <code class="language-plaintext highlighter-rouge">Currency</code>, and does the conversion.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">interface</span> <span class="nc">Exchange</span> <span class="o">{</span>
<span class="n">convert</span><span class="o">(</span><span class="nc">PreciseMoney</span> <span class="n">source</span><span class="o">,</span> <span class="nc">Currency</span> <span class="n">target</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">Exchange</code> implementations will also need to deal with concerns such as caching today’s rates, to avoid unnecessary traffic on each call.</p>
<p>If fetching + caching + converting sounds like a lot of responsibility for one service, that’s because it is (and this one is actually still very clean as compared to most service classes I see in actual code). These kinds of service classes are fundamentally procedural code wrapped in an object. Even though leakage is reduced thanks to the Exchange interface, the rest of our code still needs to depend on that interface. This makes testing harder, as all those tests will need to mock <code class="language-plaintext highlighter-rouge">Exchange</code>. And finally, (although I agree it’s subtle here), all implementations of <code class="language-plaintext highlighter-rouge">Exchange</code> will need to duplicate the actual conversion, or somehow share code. A heuristic that helps, is to figure out if we can somehow isolate the side-effectful code (fetching, caching) from the domain logic (conversion).</p>
<p>You guessed it, we need to identify a missing concept. Instead of having the <code class="language-plaintext highlighter-rouge">Exchange</code> do the conversion, we can make it return a <code class="language-plaintext highlighter-rouge">ConversionRate</code> instead. This is a Value Object that represents a source <code class="language-plaintext highlighter-rouge">Currency</code>, a target <code class="language-plaintext highlighter-rouge">Currency</code>, and a factor (a float). Value Objects attract behaviour, and in this case, it’s the <code class="language-plaintext highlighter-rouge">ConversionRate</code> object that becomes the natural place for doing the actual calculation.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">interface</span> <span class="nc">Exchange</span> <span class="o">{</span>
<span class="n">getRate</span><span class="o">(</span><span class="nc">Currency</span> <span class="n">source</span><span class="o">,</span> <span class="nc">Currency</span> <span class="n">target</span><span class="o">)</span> <span class="o">::</span> <span class="nc">ConversionRate</span>
<span class="o">}</span>
<span class="nc">ConversionRate</span> <span class="o">{</span>
<span class="n">convert</span><span class="o">(</span><span class="nc">PreciseMoney</span> <span class="n">source</span><span class="o">)</span> <span class="o">:</span> <span class="nc">PreciseMoney</span>
<span class="o">}</span>
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">convert</code> method makes sure that we never accidentally convert USD to EUR using the rate that was actually meant for GBP to BTC. All it needs to do is throw an exception if the arguments have the wrong currency.</p>
<p>The other cool thing here is that we don’t need to pass around <code class="language-plaintext highlighter-rouge">Exchange</code>. Instead, we pass around the much smaller, simpler, <code class="language-plaintext highlighter-rouge">ConversionRate</code> objects. They are, once again, more composable. Already the possibilities for reuse become obvious: for example, a transaction log can store a copy of the <code class="language-plaintext highlighter-rouge">ConversionRate</code> instance that was used for a conversion, so you get accountability.</p>
<h2 id="simpler-elements">Simpler Elements</h2>
<p>An <code class="language-plaintext highlighter-rouge">Exchange</code> is now something that represents the collection of <code class="language-plaintext highlighter-rouge">ConversionRate</code> objects, and provides access (and filters) on that collection. Sounds familiar? This is really just the <em>Repository</em> pattern! Repositories are not just for Entities or Aggregates, but for all domain objects, including Value Objects.</p>
<p>(We could rename <code class="language-plaintext highlighter-rouge">Exchange</code> to <code class="language-plaintext highlighter-rouge">ConversionRateRepository</code>, which would make the pattern more explicit, but remove a name from our Ubiquitous Language. It’s helpful in some contexts. Personally, I’d be more in favour of keeping the name. We can annotate <code class="language-plaintext highlighter-rouge">Exchange</code> as a @Repository, or have it implement a marker interface called <code class="language-plaintext highlighter-rouge">Repository</code>.)</p>
<p>We split up our procedural original <code class="language-plaintext highlighter-rouge">Exchange</code> service into the two simpler patterns, Repository and Value Object. Notice how we have at no point removed any essential complexity, and yet each element, each object, is very simple in its own right. To understand this code, the only pattern knowledge a junior developer would need, is in a few pages of chapters 5 & 6 of <a href="http://amzn.to/1LrjmZF">Domain-Driven Design</a>.</p>
<h2 id="currency-type-safety">Currency Type Safety</h2>
<p>We still have some issues. Remember that some currencies have a division of 1/00, 1/1000, or 1/100000000. <code class="language-plaintext highlighter-rouge">RoundedMoney</code> needs to support this, in our case for 10 currencies. The constructor is starting to look pretty ugly:</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">switch</span><span class="o">(</span><span class="n">currency</span><span class="o">)</span>
<span class="k">case</span> <span class="nl">EUR:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span>
<span class="k">case</span> <span class="nl">USD:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span>
<span class="k">case</span> <span class="nl">BTC:</span> <span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">8</span><span class="o">)</span>
<span class="n">etc</span>
</code></pre></div></div>
<p>Every time we need to support a new currency, we need to add code here, and possibly in other places in our system. Not a serious issue, but not ideal. A switch statement is often a smell for missing types. (Yes, we could replace this with a map, but it’s really the same thing. And a map wouldn’t fit if we’d be dealing with more interesting behavioural differences between currencies than rounding.)</p>
<p>What happens if we make <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code> abstracts or interfaces, and factor the variation out into subtypes for each currency?</p>
<p><img src="/img/posts/2016-02-29-type-safety-and-money/FactorOutCurrency.png" alt="Factoring out currency" /></p>
<p>Each of the <code class="language-plaintext highlighter-rouge">PreciseEUR</code>, <code class="language-plaintext highlighter-rouge">PreciseBTC</code>, <code class="language-plaintext highlighter-rouge">RoundedEUR</code>, <code class="language-plaintext highlighter-rouge">RoundedBTC</code> etc classes have very local knowledge about how they go about their business, such as the rounding switch example above.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">RoundedEUR</span> <span class="o">{</span>
<span class="nc">RoundedEUR</span> <span class="o">(</span><span class="n">amount</span><span class="o">)</span> <span class="o">{</span>
<span class="k">this</span><span class="o">.</span><span class="na">amount</span> <span class="o">=</span> <span class="n">round</span><span class="o">(</span><span class="n">amount</span><span class="o">,</span> <span class="mi">2</span><span class="o">)</span>
<span class="o">}</span>
<span class="o">}</span>
</code></pre></div></div>
<p>Again, we can put the type system to work here. Remember that, per requirement 5, our reporting needs to be in EUR? We can now typehint for that, making it impossible to pass any other currency into our reporting. Similarly, the different compliance reporting strategies we need for requirement 4 can each be limited to the currencies they support.</p>
<p>Sure, we get a bit of class explosion. But so what? It’s much preferable to long methods.</p>
<h2 id="minimalist-interfaces">Minimalist Interfaces</h2>
<p>A benefit of having lots of small classes, is that we can get rid of a lot of code. Perhaps our main Bounded Context deals with the 10 <code class="language-plaintext highlighter-rouge">PreciseXYZ</code> types, but our Reporting Bounded Context only supports <code class="language-plaintext highlighter-rouge">RoundedEUR</code>. That means there’s no need to support <code class="language-plaintext highlighter-rouge">RoundedUSD</code> etc, as we are not using it. This also implies that we don’t need <code class="language-plaintext highlighter-rouge">round()</code> methods on any of the <code class="language-plaintext highlighter-rouge">PreciseXYZ</code> classes, apart from EUR. Less code means less boilerplate, less bugs, less tests, and less maintenance.</p>
<p>Not supporting a way back from <code class="language-plaintext highlighter-rouge">RoundedEUR</code> to <code class="language-plaintext highlighter-rouge">PreciseEUR</code> is another example of a minimalist interface. Don’t build behaviours that you don’t need or want to discourage.</p>
<h2 id="single-responsibility">Single Responsibility</h2>
<p>Another benefit of these small, ultra-single-purpose classes, is that they very rarely need to change. This is Robert C. Martin’s heuristic for the Single Responsibility Principle:</p>
<blockquote>
<p>“A class should have only one reason to change.”</p>
</blockquote>
<p>A good design allows you to easily add or remove elements, or change the composition of the elements, but rarely requires you to actually change existing code. This in turn leads to fewer bugs and less work.</p>
<h2 id="parent-interface">Parent interface?</h2>
<p>You may have noticed that in the current design, I have no <code class="language-plaintext highlighter-rouge">Money</code> interface at the top of the object graph. Aren’t <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code> both a kind of <code class="language-plaintext highlighter-rouge">Money</code>? Don’t they share a lot of methods?</p>
<p>If we are trying to build a model inspired by the real-world, this would make sense. However, we shouldn’t judge our models by how well they fit our hierarchical categorisations. We judge our models by usefulness. A top-level <code class="language-plaintext highlighter-rouge">Money</code> interface adds no value at all; in fact it takes away value.</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/BadMoney.png" alt="Bad Money" /></p>
<p>This may be a bit counterintuitive. <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and <code class="language-plaintext highlighter-rouge">RoundedMoney</code>, although somewhat related, are fundamentally different types. We’ve designed our model for clarity, for the guarantee that we don’t mix up rounded and precise values. By allowing client code the typehint for the generic Money, we’ve taken away that clarity. There’s now no way of knowing which we’re getting. All responsibility for passing the correct type is now back in the hands of the caller.</p>
<h2 id="overdesign">Overdesign</h2>
<p>Is this overdesigned? Perhaps, depending on your context. When you only want to display some prices, it’s likely overkill. With code that decides about huge amounts of money, a design like this helps me sleep at night.</p>
<p>There’s a hint in the language. We don’t say “this thing is designed”, we say “this thing is designed for <strong>that purpose</strong>”. Overdesign (and underdesign) simply mean “not designed for purpose”, or “designed for a different purpose than the current one”. The model presented here is designed for the purpose of high precision and confidence in our operations with money.</p>
<p>Often, different aspects of our system have different requirements. This is precisely what Bounded Contexts are for: allow us to reason about our system as a a number of cooperating models, as opposed to one unified model. Perhaps our Product Catalog needs a really simple model for money, because it doesn’t really do anything other than displaying prices. Sales and Reporting on the other hand might benefit from our more intricate design.</p>
<p><img src="/img/posts/2016-02-29-type-safety-and-money/Hexagonal.png" alt="Hexagonal" /></p>
<h2 id="practicality">Practicality</h2>
<p>It’s important to distinguish between “overdesigned” and “impractical”. The solution to overdesign is to remove elements, make it simpler until it just fits. The solution to impracticality, can be to add more abstractions and shortcuts.</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">new</span> <span class="nc">PreciseMoney</span><span class="o">(</span><span class="mf">5.00</span><span class="o">,</span> <span class="k">new</span> <span class="nc">Currency</span><span class="o">(</span><span class="no">EUR</span><span class="o">))</span>
</code></pre></div></div>
<p>is a handful to write. There’s no reason why you shouldn’t add a static factory method, like</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">PreciseMoney</span><span class="o">.</span><span class="na">EUR</span><span class="o">(</span><span class="mf">5.00</span><span class="o">)</span>
</code></pre></div></div>
<p>Or just a naked function if your language supports it</p>
<div class="language-java highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">EUR</span><span class="o">(</span><span class="mf">5.00</span><span class="o">)</span>
</code></pre></div></div>
<p>Or a unicode function</p>
<p><code class="language-plaintext highlighter-rouge">€(5.00)</code></p>
<p>(Unfortunately, $ is used in many languages to mean all kinds of things, none of which are USD. Be creative!)</p>
<h2 id="defensive-model">Defensive model</h2>
<p>There is of course no way to defend your code against bad coders. Unless you’re doing <a href="/2013/10/pre-merge-code-reviews/">code reviews</a>, someone could change the Value Objects to be mutable, and delete the tests. (Funny story: I was once told that the new developer hired to replace me after I left a job, replaced all private methods by public ones, “because it’s more convenient”.)</p>
<p>I do believe though that good design communicates intent. No matter the level of the developers using your code, if they see two types for rounded and precise money, chances are they are at least going to consider that perhaps we can’t just mix the two in operations without thinking about it.</p>
<h2 id="going-further">Going Further</h2>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2016-02-29-type-safety-and-money/Price.png" alt="Price" /></p>
<p>Value Objects are the ultimate composable objects, so we can keep looking for implicit concepts that we can make explicit. For example, we can naively use our money type to represent prices. But what if a price is not so simple? Maybe <code class="language-plaintext highlighter-rouge">Price</code> is composed of a <code class="language-plaintext highlighter-rouge">PreciseMoney</code> and a <code class="language-plaintext highlighter-rouge">VATRate</code> or <code class="language-plaintext highlighter-rouge">Tax</code> object of sorts. Maybe a <code class="language-plaintext highlighter-rouge">ProductPrice</code> is composed multiple <code class="language-plaintext highlighter-rouge">Price</code> objects for different amounts, for example if you offer a discount when buying in bulk.</p>
<p>Value Objects make it easy to build abstractions that can handle lots of complexity at a very small cognitive cost to the developer using it. Finding the abstractions can be costly and hard, but spending the effort here usually impacts code quality and maintainability so much in the long run, that it’s very often worth it.</p>
<h2 id="timezone-type-safety">Timezone Type Safety</h2>
<p>The concept of using the type system to reduce a whole classes of errors in code without resorting to excessive tests, can be applied to many domains. Recently, I was late for a talk I was doing at a meetup in London, because a certain website’s “Export to Google Calendar” feature didn’t take timezones into account. Timezone bugs are not only a nuisance to a regular traveler like myself, but can have dire consequences.</p>
<p>Haskell’s Time library has separate types for <code class="language-plaintext highlighter-rouge">UTCTime</code> and <code class="language-plaintext highlighter-rouge">ZonedTime</code>, and offers functions to convert between them. This gives guarantees from the compiler that a programmer can not easily mix them up, or implicitly converts between them. Ideally, your code should use <code class="language-plaintext highlighter-rouge">UTCTime</code> everywhere, and only convert to and from <code class="language-plaintext highlighter-rouge">ZonedTime</code> when human users are involved. It should be easy enough to build something like this in object oriented languages.</p>
<h2 id="conclusion">Conclusion</h2>
<p>In many environments, dealing with money is too critical to be regarded as a Generic Subdomain. Different projects have different needs and expectations of how money will be handled. If money matters, you need to build a model that fits your specific problem space like a glove. Some of the tools in our belt are small composable building blocks, making the implicit explicit, and using the type system to our advantage. You might end up with something entirely different.</p>
<p><em>All good design is redesign.</em></p>
Towards Modelling Processes2015-05-22T00:00:00+00:00https://verraes.net/2015/05/towards-modelling-processes<p>The video & slides for my keynote at NCrafts 2015 in Paris.</p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://player.vimeo.com/video/131658140" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture fullscreen" allowfullscreen=""></iframe>
</figure>
<script async="" class="speakerdeck-embed" data-id="ff4ed4d875c246f3996807942a09df4d" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<h2 id="abstract">Abstract</h2>
<p>Processes are at the heart of how many businesses operate. The processes are governed by policies, agreed upon with contracts, and guided by documents. In software systems on the other hand, we often bury the processes. There’s a domain model for sure, but it’s all about “things” and “actions”. Let’s explore how Event Sourcing allows us to explicitly model the effects of changes over time.</p>
Context Mapping: Life Expectancy2015-04-26T00:00:00+00:00https://verraes.net/2015/04/context-mapping-life-expectancy<p>I was at a client recently, a large insurance and banking company. The organization is over 200 years old. Some of the software is only slightly younger: they first started automating some 40 years ago. Some of the people I spoke with, have been around almost as long. They have, depending on who you’re asking, 600 applications in production. The majority is written in Java and COBOL. For them, dealing with legacy is not just about removing some code, it’s about phasing out entire systems.</p>
<p>We did some Context Mapping, but due to time constraints, it was mostly as an exercise, rather than a serious attempt to make sense of it all. Yet already some interesting ideas arose, so I thought it might be valuable to describe them here. If there’s any prior art about these patterns, please let me know.</p>
<h2 id="reuse---buy---rebuild">Reuse - Buy - Rebuild</h2>
<p>Many systems serve a common need, so a lot of other systems depend on them. When making strategic decisions about adding a new system, the company has a mantra of “Reuse - Buy - Build”. It means of course that reusing an existing system is preferred to buying a new one, which in turn is preferred to building a system. A Context Map is good place to make these forces explicit, for example through some color code or symbol.</p>
<h2 id="life-expectancy">Life Expectancy</h2>
<p>When reusing an existing system, there is a risk of creating a lot of rework: when we phase out an old system, we need to rewrite the integration to work with the newer system. We started indicating the life expectancy of each Bounded Context on the map. It helped us to spot whether making a new integration with an old system still makes sense. Knowing that in reality, large organizations can be slow, we even went as far as to write down both the “official” life expectancy of a system (as decided by the powers that be), and the “realistic” expectation (often 2-4 times as long).</p>
<h2 id="sponsorship">Sponsorship</h2>
<p>You apply Context Mapping in part to figure out how the company’s communication structures can influence the success or failure of a project. A system is usually sponsored by one specific department, and in some cases by a number of different departments. It makes a lot of sense to bring the sponsors or owners of each system into the picture. Knowing who’s paying for what (or not) can make all the difference, when you want to determine whether a project is feasible.</p>
<h2 id="tragedy-of-the-commons">Tragedy of the Commons</h2>
<p>Different departments often come together to solve a common problem. With the best of intentions, they jointly build a system to address that. (This pattern is called a “Partnership” in the <a href="http://www.amazon.com/gp/product/0321125215/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0321125215&linkCode=as2&tag=verraesnet-20&linkId=DW5WV5IVDVK7SMFN">Domain-Driven Design book</a>.) Over time, the effect of the Tragedy of the Commons kicks in. More parties join the Partnership, they all adapt the system to their own needs, but no one invests in keeping the system maintainable. Quality declines. By indicating which systems suffer from this effect, we can make decisions about phasing those systems out, or forming a standalone team to take ownership. The Partnership dissolves and is replaced by a Customer/Supplier Relationship (another pattern from the book).</p>
<h2 id="visualization">Visualization</h2>
<p>I’m moving away from using whiteboards for Context Mapping. They are generally too small, and quickly become unrefactorable anyway. Find a large wall or glass surface, or move some tables together. Use stickies, agree on a color code, and buy some sticky shapes. <a href="http://www.post-it.com/wps/portal/3M/en_US/PostItNA/Home/Products/~/Post-it-Arrow-Flags-Bright-Colors-1-2-in-Wide-100-On-the-Go-Dispenser?N=4327+3294647258&rt=rud">Arrows</a> are especially useful of course.</p>
<p>Trying to get everything on a single Context Map is impossible. A Context Map is a model, and, as any model, it’s a problem solving tool. We judge a model by its usefulness. Of course, when you’re new to the organization, or you’ve never visualized it, your problem is a lack of insight. When the goal is discovery, I don’t mind if we end up with an unreadable model. Throw the map away afterwards, because the new shared understanding is the greatest artifact here. Over time, you might grow Context Maps that are stable. Only then you should treat the map as a document that you keep around.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/01/bandwidth-and-context-mapping/">Bandwidth and Context Mapping</a></li>
<li><a href="/2014/02/buzzword-free-bounded-contexts/">Buzzword-free Bounded Contexts</a></li>
</ul>
Event Storming, Storytelling, Visualisations2015-03-14T00:00:00+00:00https://verraes.net/2015/03/event-storming-storytelling-visualisations<h2 id="visual-storytelling-through-event-storming">Visual storytelling through Event Storming</h2>
<p>Last week, my client’s Chief Product Owner announced to the team they would introduce a set of new requirements using Event Storming, “as Mathias has taught us”. (I’ve visited there regularly for the past half year or so). The session took about ten minutes, and included no brainstorming or collaboration of any kind. The only thing it had in common with Event Storming, was that the PO used colorful stickies.</p>
<p>I was of course sceptical at first, but it turned out to be quite interesting in its own right, as a presentation technique. The Chief PO and his team were well prepared. They had researched the regulations they had to comply with to enter a new market, and third party services they’d need to integrate with. They had done Event Storming (or something similar) amongst themselves, cleaned up the resulting artifact, and ordered and collected the stickies. During the short session with the development team, the Chief PO told the story, while the assistants supported it by adding stickies to wall as the story evolved. It was not a linear story. You could easily understand were a new branch in the story began, because the PO’s attached new stickies at that location.</p>
<p>Surely, for the developers, it didn’t create the collaborative deep learning experience that Event Storming usually does. They were only spectators. But this style of visually supported presentation, was definitely much better than some of the alternatives, like purely oral storytelling, or a written set of requirements.</p>
<p><img src="/img/posts/2015-03-14-event-storming-storytelling-visualisations/storytelling.jpg" alt="Storytelling" /></p>
<h2 id="model-storming-visualisations">Model Storming visualisations</h2>
<p>Event Storming as described by Alberto Brandolini is rather prescriptive, and that’s a good thing. A fixed set of steps allows you to get started on exploring and modelling from the first minute. There’s no long introductions, no discussions on what the right approach is, and no analysis paralysis.</p>
<p>The first few steps go something like this:</p>
<ul>
<li>Write down a Domain Event on a sticky (preferably something that happens at the end, like “Bill was paid”).</li>
<li>Ask what happened before that (“Bill was sent”) and before that, until you get to the beginning.</li>
<li>Ask what caused the events to happen. Those are usually Commands (aka user intentions), other events, or simply the passage of time.</li>
<li>Ask for what reason a Command can fail, and find the rules that impact the outcome of the Commands. This pushes you to discover alternative scenarios, missing events, connections between distant events…</li>
</ul>
<p>The list goes on for a bit (I should probably do a proper writeup of how I run Event Storming sessions). The point I want to make here though, is that it’s ok to deviate. When a team has some maturity with the domain, more complex scenarios might emerge that are not easily captured with basic Event Storming. At this point, you should invest some energy in coming with new visualisations, new elements that capture the essence of what you’re trying to achieve. This brainstorming about the modelling itself, is crucial to evolving deeper insight.</p>
<p>Describing the experience of reaching a breakthrough, is hard. Good models are always obvious in retrospect. As an example, at this client, we were cutting our teeth on a complex customer accreditation process that varies highly in different markets. To add to the complexity, the process varies depending on whether the customer is using the site, talking to customer support by phone, or visiting an office. And finally, a process that started through one medium, could be finished through another.</p>
<p>For our first model, we focused too much on the design we thought we wanted: Reusable Specification and Policy objects, Adapters and Strategies, with the intent of injecting the behaviour for the different markets into the process. The next morning, while attempting a PoC implementation, we realized we had painted ourselves into a corner. Event Storming to the rescue. Based on nothing but a gut feeling, I proposed we started with Commands instead of Events though. Then we evolved that in to some sort of matrix, with one swim lane per Market x Medium combo. Finally it hit us that the implementation shouldn’t start from a single accreditation process where we injected strategies. Our visual model suggested that we needed multiple processes, implemented independently, and only share elements between those processes if it absolutely made sense. I don’t have a good picture of the resulting model, but it wouldn’t have made sense anyway. As with all brainstorming, you had to be there.</p>
<p>I don’t believe we would have come to this conclusion so fast in any other way. As developers, we are so often blinded by the way the requirements are described, and our own propensity to design according to preconceived notions of what a good design must look like. Find the elements that complicate a problem, find a visualisation, put it front and center, and brainstorm from there. Without visualisation, you spend too much energy synchronizing each other’s mental models, usually unsuccessfully.</p>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="/2014/11/domain-events/">Domain Events</a></li>
<li><a href="/2013/08/facilitating-event-storming/">Notes on facilitating Event Storming sessions</a></li>
<li><a href="http://ziobrando.blogspot.be/search/label/EventStorming">Posts tagged with Event Storming on Alberto Brandolini’s blog</a></li>
<li><a href="http://sch3lp.github.io/2014/07/12/event-storming-exercise/">Tim Schraepen’s Event Storming experience report</a></li>
<li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li>
</ul>
Form, Command, and Model Validation2015-02-16T00:00:00+00:00https://verraes.net/2015/02/form-command-model-validation<p>Many of the frameworks I’ve worked with, promise to separate responsibilities with MVC. In practice, they end up coupling everything to everything. The forms are coupled to the models, and there’s a grand unified validation layer. This may be convenient at first, but it breaks down for larger systems, and creates headaches when having to support multiple clients. My approach is to clearly separate the validation for the form itself, from the Command validation and the model validation.</p>
<h2 id="commands">Commands</h2>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2015-02-16-form-command-model-validation/smokesignals-small.jpg" alt="Smoke Signals" /></p>
<p><a href="/2015/01/messaging-flavours/">A Command is a message that represents the intention of the user</a>. Command validation happens after the form is submitted and before the Command is passed to the model. The question to answer here is “Does this look like a valid Foo Command?” It guarantees that the “birthday” field contains an actual date, or that “zipcode” looks like a zipcode. It doesn’t guarantee set constraints, like username uniqueness. In other words, you validate the message, not whether you should execute the message.</p>
<p>The Command object should use Value Objects. They guarantee their own consistency, so the Command delegates validation to them. Note that we’re not trying to inform the user of validation errors. We simply throw exceptions. The assumption here is that either the form prevents malformed values, or the user is trying to bypass form validation. We don’t return friendly error messages to attackers.</p>
<h2 id="model">Model</h2>
<p>The model is the “official” source of truth. It doesn’t worry whether the “birthday” field looks like a date and whether the “zipcode” looks like a zipcode, because it can trust the Command. The model is responsible for protecting its own invariants, business rules, and constraints. It can reliably<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> prevent duplicate usernames, account overdrafts, and all kinds of other business concerns. It fails hard when executing the Command would violate those concerns.</p>
<p>The Commands, the Value Objects, and the objects that make up the model, are decoupled from the framework and the persistence. We can effectively transport them to a different context, such as a different framework. They are independently testable with unit tests.</p>
<h2 id="forms">Forms</h2>
<p>Form validation is a UX concern: you add it to help the user fill in the values quickly and correctly. Nowadays, users expect forms to give them instant visual feedback, and rightly so. Client side is the only option. Of course, this client side validation can call back to the server to validate things that the client cannot know, such as “is this username available?”.</p>
<p>By the time the submit button is pressed, some of the data may already be stale, for example another user might have registered the same name. The form validation stops most of the cases, but it can never be relied on. Not only because it lags behind on the server, but also because it is easy to spoof requests and bypass the validation.</p>
<p>I’m not aiming to make the form validation as complete as possible, at least not initially. Rather, I try to look at it from the point of view of the user. The things that they are most likely to get wrong, are the things I try to guide them into doing correctly. Ideally, I watch or monitor how a real life user fills in the form, keep track of the most frequent model errors they trigger, and focus on preventing those first.</p>
<h2 id="conclusion">Conclusion</h2>
<p><strong>The fundamental idea here is that “helping the user”, “validating the message”, and “protecting the model’s consistency”, are three completely independent concerns.</strong></p>
<p>Yes, you will end up with something that looks like duplication. For example, both the form and the model will have a concept of “account overdraft” and what it means. These are however different concepts, because the goal is different: the model protects business rules, the form helps the end user. I feel that this separation of responsibilities <a href="/2014/08/dry-is-about-knowledge/">trumps the minor cost of duplication</a>.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li>
<li><a href="/2014/08/dry-is-about-knowledge/">DRY is about Knowledge</a></li>
<li><a href="/2013/04/decoupling-symfony2-forms-from-entities/">Decoupling (Symfony2) Forms from Entities</a></li>
</ul>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>In some cases, I might choose eventual consistency: Accept the small risk that usernames might clash, detect the conflicts after the events are persisted, and deal with them somehow. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Messaging Flavours2015-01-09T00:00:00+00:00https://verraes.net/2015/01/messaging-flavours<p>A message is a unit of communication between systems. There are three flavours of messages: <strong>informational, interrogatory, and imperative</strong>.</p>
<h2 id="imperative-messages">Imperative Messages</h2>
<p>Imperative messages represent the sender’s intention. They instruct the receiver to perform an action or make a change. Do this, change that. We can model these messages as Commands. More on that in a later post.</p>
<h2 id="interrogatory-messages">Interrogatory Messages</h2>
<p>Interrogatory messages ask the system about it’s current state, or usually a subset of that state. You know them as Queries. If you’ve done any web programming, you’ve written Queries as:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">GET</span> <span class="o">/</span><span class="n">pupils</span><span class="o">?</span><span class="n">bornBefore</span><span class="o">=</span><span class="mi">1997</span><span class="o">-</span><span class="mo">01</span><span class="o">-</span><span class="mi">09</span>
</code></pre></div></div>
<p>If you’ve ever talked to an SQL database, you’ve written Queries as:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">SELECT</span> <span class="o">*</span> <span class="no">FROM</span> <span class="n">pupils</span> <span class="no">WHERE</span> <span class="n">date_of_birth</span> <span class="o"><</span> <span class="s2">"1997-01-09"</span>
</code></pre></div></div>
<p>If you’ve done any Object-Oriented Programming, you know that everything is an object, and so is a Query</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">UserQuery</span> <span class="p">{</span>
<span class="n">where</span> <span class="o">=</span> <span class="p">[{</span><span class="n">field</span><span class="o">=</span><span class="s2">"date_of_birth"</span><span class="p">,</span> <span class="n">operator</span> <span class="o">=</span> <span class="s2">"<"</span><span class="p">,</span> <span class="n">value</span> <span class="o">=</span> <span class="s2">"1997-01-09"</span><span class="p">}]</span>
<span class="mf">...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>If you’re into Domain-Driven Design, you look for the Ubiquitous Language, you dig into the domain, and you make the implicit explicit. We can make a Query message that communicates its purpose (Intention-Revealing Interfaces)</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">LuckyPupilsWhoDontNeedTheirParentsToSignTheirReportsAnymore</span> <span class="p">{</span>
<span class="n">asOf</span> <span class="o">=</span> <span class="nf">now</span><span class="p">()</span>
<span class="n">age_of_consent</span> <span class="o">=</span> <span class="mi">18</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is what is meant by <strong>Intention Revealing Interfaces</strong> in the <a href="http://www.amazon.com/gp/product/0321125215/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0321125215&linkCode=as2&tag=verraesnet-20&linkId=JK4IHA2DQIYZB5ZR">blue book</a>. The name <code class="language-plaintext highlighter-rouge">LuckyPupilsWhoDontNeedTheirParentsToSignTheirReportsAnymore</code> is obviously exaggerated to make a point. You’ll have to admit that at least now, you’ve learned something about the domain that was tricky to infer from the previous examples.</p>
<h2 id="informational-messages">Informational Messages</h2>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2015-01-09-messaging-flavours/icecream-small.jpg" alt="Ice Cream Flavours" /></p>
<p>Informational messages are sent by a system that wants to <strong>communicate something about itself</strong> to the outside world. It does this not to achieve a goal (as with imperative messages) or to learn about another system’s state (interrogatory). It provides this service for the benefit of other systems. It’s the altruistic pattern of the distributed world! (It does however have some sort of contract with it’s receivers.) It can notify an interested party directly, or through publish/subscribe (the subscribers are not hardcoded, creating better decoupling), or through broadcasting (“This is my message and I don’t care who hears it!”).</p>
<p>Events are one flavour of informational messages, where the sender tells others what has happened. <a href="/2014/11/domain-events/">I’ve written about Domain Events extensively</a>. Alternatively, instead of events, the sender might simply communicate its current state. Say we have a thermometer, that emits an informational message as an event:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">TemperatureHasChanged</span> <span class="p">{</span>
<span class="n">temperature</span><span class="o">:</span> <span class="nc">Temperature</span>
<span class="n">at</span><span class="o">:</span> <span class="nc">DateTime</span>
<span class="p">}</span>
</code></pre></div></div>
<p>If the temperature changes at 08:00 and stays stable until 17:00, we have a bit of a problem. If I subscribe at 09:00, I have to wait 8 hours before I know the state. I can query the thermometer (interrogatory message), but this complicates both the sender and the receiver. The sender can resend the last event to any new subscribers, but that still means our thermometer needs to be stateful.</p>
<p>More realistically (disclaimer: I’ve never actually interfaced with a thermometer, so I don’t know), the thermometer is pretty dumb and simply communicates its current measurement every second:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">CurrentTemperature</span> <span class="p">{</span>
<span class="n">temperature</span><span class="o">:</span> <span class="nc">Temperature</span>
<span class="n">at</span><span class="o">:</span> <span class="nc">DateTime</span>
<span class="p">}</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">TemperatureHasChanged</code> is an event-flavoured informational message, <code class="language-plaintext highlighter-rouge">CurrentTemperature</code> is a state-flavoured informational message. My gut feeling is that, when faced with this, I would still model state-flavoured messages as event-flavoured, by tweaking the language a little bit:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nc">TemperatureWasMeasured</span> <span class="p">{</span>
<span class="n">temperature</span><span class="o">:</span> <span class="nc">Temperature</span>
<span class="n">at</span><span class="o">:</span> <span class="nc">DateTime</span>
<span class="p">}</span>
</code></pre></div></div>
<h2 id="mixing-flavours">Mixing Flavours</h2>
<p>Given this categorisation, it follows from the Single Responsibility Principle that:</p>
<ul>
<li>A Command should not return state. (Replying with an ACK/NACK is fine though). If you want to know the resulting state of the receiver after sending a Command, you can either ask for it (Query) or wait for a message saying what has happened (Event).</li>
<li>An Event should not be coupled to the expectation that something will change in the receiver. The sender shouldn’t know who’s listening and what they’re doing as a result.</li>
<li>A Query shouldn’t change the state of the receiver.</li>
</ul>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="/2014/11/domain-events/">Domain Events</a></li>
<li><a href="/2015/02/form-command-model-validation/">Form, Command, and Model Validation</a></li>
</ul>
Economy of Tests2015-01-05T00:00:00+00:00https://verraes.net/2015/01/economy-of-tests<p><em>Joint post with <a href="https://twitter.com/everzet">Konstantin Kudryashov</a>.</em></p>
<p>A common complaint with teams that first try their hand at automated testing, is that it is hard, costly, and not worth the effort. On the other hand, supporters say that <a href="https://twitter.com/mathiasverraes/status/234641929260908544">it saves them time</a>. In this post, we’ll try to add some nuance to the discussion. We’ll take a look at how different levels of automated testing affect the cost of development, both in the short and the long term, for both greenfield and brownfield projects. Finally, we’ll look at a simple strategy for introducing tests and migrating them across test levels, in order to reduce maintenance costs.</p>
<h2 id="definitions-of-test-levels">Definitions of Test Levels</h2>
<p>Coming up with universal definitions of types of tests, is hard. There are many styles, and different authors choose different categorizations. <strong>For the scope of this post</strong>, we will simplify to three levels.</p>
<ul>
<li>A <strong>unit test</strong> verifies a single behavior of a single unit – this can be a function, a method, or a class, but it can be something other as well. A logical unit is something that can be <strong>reasoned about in isolation</strong>. The unit must be <strong>completely under your control</strong>. The test should limit the number of other units it touches on, and should not touch any part outside of your own system. As such, the test should <strong>only fail for one reason</strong>.</li>
<li><strong>Integration testing</strong> means we’re verifying that that different units are working together. It focuses on the <strong>interfaces between components</strong>. Some of these components may be from third parties. When they break, they signal that, although individual units may behave correctly, something changed in the API that caused failure in another place. Often, integration tests are slower than unit tests.</li>
<li><strong>System testing</strong> uses a <strong>production-like environment</strong> to verify the system as a whole. This may include user interfaces, databases, web servers, logging, etc. Typically a call is made at the <strong>outside boundary of the system</strong>. Ideally, the outcome is verified at the same boundary.</li>
</ul>
<h2 id="learning">Learning</h2>
<p>System and integration tests do not require you to adapt your system’s design in order to be testable. The difficulty with good unit tests, is that they require good software design to be useful. Because of this, unit tests are more costly than system and integration tests. To learn system testing, all you need to learn is the tools and the process. To learn unit testing, you need to learn design at the same time. This frustrates developers who are new to unit testing. The method appears not to work for them, because they’re trying to learn unit testing without learning better design.</p>
<p>When discussing the cost of a method, we need to separate the cost of the learning from the cost of actually applying the method. Once you’ve learned something, you can transfer this knowledge and reuse it.</p>
<h2 id="test-metrics">Test Metrics</h2>
<p>A number of metrics affect the cost of a test.</p>
<ul>
<li><strong>Execution speed</strong>: the faster a test runs, the shorter the feedback loop. You will run the test more often during development. The lifespan of a mistake is reduced. The failing tests notifies you of the mistake before you get a chance to move on to other tasks. The reasoning is still fresh in your head. Nothing else is built on top of the mistake. You can quickly fix it and move on. People with little experience in TDD usually underestimate the <strong>cumulative effect of this feedback loop</strong>.</li>
<li><strong>Fragility</strong>: the less stable your test is, the less you trust it. If your test breaks a lot, it’s harder to use as a meaningful measure of the system quality. <strong>Brittle tests often indicate a coupling problem</strong>. Either your test touches too many parts of the system, or the system under test itself is too coupled. For example, a small change in the GUI breaks a test in the domain model. Even worse is <em>flickering</em>: a test that breaks, even when you haven’t made any changes. This might happen because of things like race conditions or dependencies on services outside of your control.</li>
<li><strong>Understandability</strong>: writing code is usually easier than reading it. A clear test is the best documentation for the system you could have. Textual documentation tends to rot. Tests on the other hand are executable. When the system changes, the test breaks, and therefore the documentation breaks. However, a misleading or confusing test can actually derail the learning process of a developer. If the effort of reading the test is too great, it won’t be read, it won’t be maintained, and it won’t be used.</li>
</ul>
<p>When do these metrics matter?</p>
<p>When you are introducing a first test, we have a specific goal: we want to add a new feature, or we want to assert an existing behavior. We focus on one test at a time. The execution speed is usually not that important. Even if it takes ten seconds, that’s still short enough to get the benefit of fast feedback. We run the test in isolation, so it only breaks because of direct changes. Fragility won’t show up until much later. And finally, as we have just written the test, understandability is not a factor yet either.</p>
<p>The impact of these metrics increases dramatically the more tests you have. The cost you pay in order to keep the tests fast, green, and understandable, is affecting overall development effort. You could ignore the tests, but then you lose all the benefits they bring.</p>
<h2 id="project-lifecycle">Project Lifecycle</h2>
<p>Is it a new project, or an existing codebase? That is our final factor for looking at the cost of tests. In our experience, in a greenfield project, the cost of introducing tests at any level (unit, integration, system), is equivalent.</p>
<p>Being able to introduce tests cheaply at any level, is great, but there’s a hidden risk. People new to automated testing, might be inclined to put too much focus on system and integration tests. As we discussed earlier, those levels are easier to learn, and, by consequence, more popular. Over time, system and integration tests become costlier: they are slower, harder to read, and they are brittle. These forces accumulate, leading to technical debt in the tests.</p>
<style>
td {
border: 1px solid white;
}
</style>
<table cellpadding="5" cellspacing="5" style="width:100%; margin: 30px 5px">
<thead>
<tr>
<th> </th>
<th colspan="2">Greenfield</th>
<th colspan="2">Brownfield</th>
</tr>
<tr>
<th> </th>
<th>Introduction</th>
<th>Maintenance</th>
<th>Introduction</th>
<th>Maintenance</th>
</tr>
</thead>
<tbody>
<tr>
<th style="text-align: right;">System tests</th>
<td style="text-align: center; background-color: #d9ead3">€</td>
<td style="text-align: center; background-color: #f4cccc">€ € €</td>
<td style="text-align: center; background-color: #d9ead3">€</td>
<td style="text-align: center; background-color: #f4cccc">€ € €</td>
</tr>
<tr>
<th style="text-align: right;">Integration tests</th>
<td style="text-align: center; background-color: #d9ead3">€</td>
<td style="text-align: center; background-color: #fff2cc">€ €</td>
<td style="text-align: center; background-color: #fff2cc">€ €</td>
<td style="text-align: center; background-color: #fff2cc">€ €</td>
</tr>
<tr>
<th style="text-align: right;">Unit tests</th>
<td style="text-align: center; background-color: #d9ead3">€</td>
<td style="text-align: center; background-color: #d9ead3">€</td>
<td style="text-align: center; background-color: #f4cccc">€ € €</td>
<td style="text-align: center; background-color: #d9ead3">€</td>
</tr>
</tbody>
</table>
<p>To counter the growing costs, Mike Cohn proposed the <a href="http://www.amazon.com/gp/product/0321579364/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=0321579364&linkCode=as2&tag=verraesnet-20&linkId=BPE4YKYDY3WHOFAB">Test Pyramid</a>. The idea is that the majority of your tests should be unit tests, with less focus on the higher levels.</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2015-01-05-economy-of-tests/test_pyramid-small.png" alt="Test Pyramid" /></p>
<p>In greenfield, advice like “write more unit tests” is easy to follow. Ignoring that advice creates the sort of technical debt we usually associate with brownfield projects. Those have a very different kind of test economy. Unit tests are very difficult in legacy code: unless you’re extremely lucky, the project is big ball of mud, with huge blocks of code and nothing even remotely resembling a unit that you can test in isolation. You’ll naturally resort to high level testing. That is often the only option.</p>
<h2 id="test-level-migration">Test Level Migration</h2>
<p>The Test Pyramid is a static model of an ideal end goal. In the real world, when dealing with legacy, we advise to <strong>migrate tests from top to bottom</strong>.</p>
<p>For brownfield, it’s perfectly fine to introduce tests at the level you are most comfortable with, and that suits the codebase at hand. But high level tests should be seen as a first step. An inverted pyramid is unbalanced, but you can use the system tests to <strong>put pressure on the test suite</strong>. We can call this <strong>test level migration</strong>: as the system tests give you more confidence to change things in your code base, you use this to add more low level tests.</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2015-01-05-economy-of-tests/Snow_Geese_Migration-small.jpg" alt="Snow geese migration" />
After a while, more behaviour is covered with low-level tests. Now you can remove the high level tests, especially those where the maintenance cost is greater than the value they provide. (<a href="https://verraes.net/2014/12/how-much-testing-is-too-much/">Yes, it’s ok to delete tests.</a>) The balance shifts to the base of the pyramid.</p>
<p>Migrating tests can be done gradually. The first step is to drop your system tests to the level of the integration tests. Use the same integration points, such as databases, and other services. The tests still might break a lot, but at least they don’t break because of changes in the high level components such as the GUI or HTTP API. The next step would be to isolate integration points and cover the bulk of the system behaviour with actual unit tests.</p>
<p>Our process for introducing and evolving tests in a brownfield project looks like this:</p>
<ol>
<li>Identify the part of system you want to change.</li>
<li>Estimate how expensive adding a unit tests is.</li>
<li>If it’s expensive, move up one layer. Can you isolate the test to touch only a small number of components?</li>
<li>If that’s still too expensive, you move up again and write a test against the entire system, using the GUI or an API as the boundary.</li>
<li>As the number of higher level tests grows and your technical debt starts showing, find ways to refactor the code base, and push tests to lower levels.</li>
<li>Get rid of brittle, slow, and unreadable high level tests that test behaviours already covered by lower level tests.</li>
</ol>
<h2 id="opposing-forces">Opposing forces</h2>
<p>Testing on brownfield projects always has two opposing forces:</p>
<ul>
<li>Adding new test tends to push you to the higher levels of testing.</li>
<li>Carefully maintaining your test suite tends to push you to the lower levels of testing.</li>
</ul>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2015-01-05-economy-of-tests/test_economy_forces-small.png" alt="Opposing forces in the economy of tests" /></p>
<p>This is the essence of the Test Pyramid: when you write only high level tests, you take on technical debt. Sometimes this is necessary, but as soon as you have debt, the clock is ticking. Interest accumulates, and it becomes harder to remedy the situation. That’s why putting pressure on your test suite to migrate test levels, pays of in the long run.</p>
<h2 id="what-to-migrate">What to migrate</h2>
<p>The choice of tests to migrate, is important. You can use the metrics of execution speed, readability, and maintainability, to identify tests in dire need of migration. Another heuristic is asking whether the test is at its natural lowest possible level. For example, when a test is exercising the whole system in order to test a single unit of business logic, it is clear that this test is unnecessary costly, and should be migrated to the level of a unit test. Leave the system tests in place that verify if the system as a whole is working, and use unit tests to fix the most important permutations of the expected behavior.</p>
<h2 id="conclusion">Conclusion</h2>
<p>For projects with a long life expectancy, investing in your test suite is well worth it. Proper automated testing is a careful balance. Quickly introduce tests when you need them, and build a comprehensive, fast, and reliable test suite in the long run. We hope this post can give you some ammunition to win your team to this idea.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/12/how-much-testing-is-too-much/">How Much Testing is Too Much?</a></li>
</ul>
How Much Testing is Too Much?2014-12-29T00:00:00+00:00https://verraes.net/2014/12/how-much-testing-is-too-much<p>Figuring out how much unit tests you need to write, can be tricky, especially if you are new to Test-Driven Development. Some teams strive for 100% code coverage. Some open source projects even announce their test coverage on their GitHub profiles – as if coverage is an indicator of quality. Coverage only measures the lines of code that are executed by the test suite. It doesn’t tell you whether the outcome of the execution is actually tested, let alone how valuable that test is.</p>
<p>Because of that, <strong>code coverage of your entire code base is a pretty lousy metric</strong>. If you use TDD as intended, you don’t need to measure it. The coverage will be good, as a direct result of the method. Some people claim they use coverage to enforce their team to write tests. The better way to solve that problem, is with a healthy dose of education, <a href="/2013/10/pre-merge-code-reviews/">pre-merge code reviews</a>, and pair programming.</p>
<p>Code coverage has its uses though. When measured ad hoc, in small doses, it is <strong>a great tool to validate your assumptions</strong> about how a single unit test executes a complex block of code. For an example of how to use code coverage to your advantage, watch <a href="/2013/09/extract-till-you-drop/">Extract Til You Drop</a>.</p>
<h2 id="over-test">Over-test</h2>
<p>The Standard Consultant’s Answer to “How much tests do I need?” is “It depends”. That won’t do. My preferred approach is to ask <a href="/2014/03/small-controlled-experiments/">“What is the easiest way to find an answer?”</a>. Experiments beat opinions. <strong>When it comes to getting comfortable with TDD, be overzealous.</strong> Be religious about it. Don’t write a single line that isn’t covered by a test first. Do it with the entire team – no code goes to master without being fully tested. Accidentally wrote a code without a test? Delete the code, write the test, implement the code.</p>
<p><img src="/img/posts/2014-12-29-how-much-testing-is-too-much/inception-mirrors-small.jpg" alt="Over-designing" /></p>
<p>Timebox this experiment, say a couple of weeks. As with all experiments, we ask what’s the worst that could happen. Progress on features could stop entirely, you all learn nothing, and the whole thing was a failure. (Or you have learned that TDD is not for you. Learning happened!)</p>
<p>More likely, you’ll get some interesting results:</p>
<ul>
<li>You’ll learn that unit testing isn’t so scary (provided you make the units small);</li>
<li>You’ll make smaller units;</li>
<li>You’ll learn to appreciate that <a href="http://www.jefclaes.be/2014/12/tdd-as-crack-cocaine-of-software.html">TDD helps you to get in the Zone</a>;</li>
<li>You’ll discover that some things simply aren’t worth writing tests for (getters and setters, empty constructors, …);</li>
<li>You’ll have fewer getters and setters anyway, and more methods that actually do something;</li>
<li>YAGNI code.</li>
</ul>
<p>Add a couple more weeks of overzealous testing, and <strong>you’ll get a feel for how much testing is just right</strong>. Time to back down and bend the rules a little. Sometimes, you’ll want to do some exploratory coding, and add tests afterwards (<em>spike & stabilize</em>). Sometimes, you can implement some code in a way that is so obviously correct, that it doesn’t need tests. Sometimes, if you pair program, having extra eyes on the code beats extensive testing. And some edge cases are so rare that worrying about them is a waste.</p>
<p>I don’t think you can develop these instincts if you consistently under-test. <strong>When in doubt, test more</strong>, and keep balancing on the thin line between insufficient and too much.</p>
<p>After the experiment, you’ll also end up with a bunch of low-value tests. Remove them, especially if they start bothering you or require maintenance. There’s <strong>no value in maintaining useless tests</strong>. A little bit of cleanup can make the tests so much more pleasant to work with. “Deleting tests” is the new “deleting code”.</p>
<h2 id="over-design">Over-design</h2>
<p>Coincidentally, I give the same advice when it comes to software design and modelling. There will always be bystanders, telling you that your solution is over-designed, especially if they don’t have intimate knowledge of the problem domain or the non-functional requirements. They may be right, but they have no way of knowing that – an neither do you. <a href="/2014/10/software-design-is-just-theory/">Software design has a slow feedback cycle</a>. Over-design just a little, get a feel for it, evaluate, and back down. That last bit is important: <strong>if you don’t refactor when gaining deeper insight, you’re simply creating technical debt</strong>. It’s at the other end of the spectrum now, where the grass is slightly greener, but an over-designed model is technical debt nonetheless.</p>
<p>(Update, Dec 30, 2014, based on a comment:)</p>
<h2 id="when-to-delete-tests">When to delete tests?</h2>
<p>Tests become a problem when:</p>
<ul>
<li>they are slow;</li>
<li>they need to be changed all the time;</li>
<li>they break often;</li>
<li>they are hard to read;</li>
<li>… or they bother you in some other way.</li>
</ul>
<p>When any of those occur, the tests need to be inspected. Now is the time to decide whether you want to refactor the test itself, or refactor the code under test, or, in some cases, remove the tests. Low-value tests are usually harmless. There’s no urgent need to decide upfront whether they need to be deleted. Trust your instinct, or in this case, your annoyance level.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2015/01/economy-of-tests/">Economy of Tests</a></li>
</ul>
Let It Crash2014-12-09T00:00:00+00:00https://verraes.net/2014/12/erlang-let-it-crash<p>“<a href="http://www.amazon.com/gp/product/193778553X/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=193778553X&linkCode=as2&tag=verraesnet-20&linkId=JM5GZYMS2A36UJTF">Programming Erlang: Software for a Concurrent World, 2nd ed</a>” by Joe Armstrong, is not just a great introduction to the language, but also offers a glimpse into the “Let It Crash” philosophy that drives it. It’s a great read, even if you have no intention of programming in Erlang. I’ve selected and reordered a number of passages that were scattered across the book, and added some comments, that I hope will do the book justice.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-12-09-erlang-let-it-crash/armstrong-erlang-small.jpg" alt="Programming Erlang" /></p>
<blockquote>The Erlang view of the world is that everything is a process and that processes can interact only by exchanging messages. Having such a view of the world imposes conceptual integrity on our designs, making them easier to understand.</blockquote>
<blockquote>The real world actually has independent things communicating through messages. I’m an ex-physicist — we perceive the world by receiving messages. Packets of light and sound carry information. All that we know about the world is what we’ve learned by receiving messages. We don’t have shared memory. I have my memory, you have yours, and I don’t know what you think about anything. If I want to know what you think about something, then I have to ask you a question and wait for you to reply.</blockquote>
<p>(In recent years, the idea of messages as the first class citizens of a system, has been rediscovered by the CQRS/EventSourcing community, albeit with a focus on domain models. In fact, Event Sourced Aggregates are a way to model processes, not things – they are the poor man’s actor model. Building CQRS/ES systems is what led me to Erlang in the first place.)</p>
<blockquote>Imagine a system with only one sequential process. If this process dies, we might be in deep trouble since no other process can help. For this reason, sequential languages have concentrated on the prevention of failure and an emphasis on defensive programming. In Erlang we have a large number of processes at our disposal, so the failure of any individual process is not so important. We usually write only a small amount of defensive code and instead concentrate on writing corrective code. We take measures to detect the errors and then correct them after they have occurred.</blockquote>
<p>(A typical Erlang program might have hundreds, thousands, or even millions of processes. Letting processes crash is central to Erlang. It’s the equivalent of unplugging your router and plugging it back in – as long as you can get back to a known state, this turns out to be a very good strategy. To make that happen, you build supervision trees. A supervisor will decide how to deal with a crashed process. It will restart the process, or possibly kill some other processes, or crash and let someone else deal with it. Despite the lack of defensive programming in idiomatic Erlang, you spend more time thinking about errors then when building in other languages.)</p>
<blockquote>This leads to a clean separation of issues. We write code that solves problems and code that fixes problems, but the two are not intertwined.</blockquote>
<blockquote>How can we write programs that run faster on a multicore CPU? It’s all about mutable state and concurrency. Back in the old days (twenty-odd years ago), there were two models of concurrency: Shared state concurrency, and message passing concurrency. The programming world went one way (toward shared state). The Erlang community went the other way.</blockquote>
<blockquote>(...) all languages such as C, Java, C++, and so on, have the notion that there is this stuff called state and that we can change it. This is fine as long as you have only one process doing the changing. If you have multiple processes sharing and modifying the same memory, you have a recipe for disaster — madness lies here.</blockquote>
<blockquote>To protect against the simultaneous modification of shared memory, we use a locking mechanism. Call this a mutex, a synchronized method, or what you will, but it’s still a lock. If programs crash in the critical region (when they hold the lock), disaster results. All the other programs don’t know what to do. If programs corrupt the memory in the shared state, disaster will also happen.</blockquote>
<blockquote>Erlang has no mutable data structures (that’s not quite true, but it’s true enough). No mutable data structures = No locks. No mutable data structures = Easy to parallelize.</blockquote>
<p>(Erlang was originally written for networks of single core computers, back in the eighties. The fact that it turns out to be great for multicore CPU’s, is an interesting side-effect.)</p>
<blockquote>Once started, Erlang/OTP applications are expected to run forever (...)</blockquote>
<p>(This is no overstatement. Erlang allows you to upgrade the system while it is running. <a href="https://www.youtube.com/watch?v=96UzSHyp0F8">But don’t take my word for it</a>.)</p>
Higher Order Programming2014-11-24T00:00:00+00:00https://verraes.net/2014/11/higher-order-programming<p>Let’s have some fun with higher order programming in PHP. I’ll start by showing how to program with Lambdalicious (or λlicious for friends) and introduce the real meat along the way. Don’t worry too much about the dark magic that may appear to power some of the features of Lambdalicious. It’s <a href="https://github.com/mathiasverraes/lambdalicious">on GitHub</a> if you’re curious. Just follow along and keep track of all the functions.</p>
<p><strong>Note:</strong> The code in this blogpost works with <a href="https://github.com/mathiasverraes/lambdalicious/tree/v0.1.0">Lambdalicious v0.1.0</a>. If you want to adapt this blog post to the current version, please send a <a href="https://github.com/mathiasverraes/mathiasverraes.github.com/edit/master/_posts/2014-11-24-higher-order-programming.markdown">pull request</a>.</p>
<p>But first, something smaller:</p>
<h2 id="atoms">Atoms</h2>
<p><code class="language-plaintext highlighter-rouge">@foobar</code> is an atom. You can think of an atom as a constant that has itself as a value. So the value of <code class="language-plaintext highlighter-rouge">@foobar</code> is <code class="language-plaintext highlighter-rouge">foobar</code>.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">require_once</span> <span class="k">__DIR__</span> <span class="mf">.</span> <span class="s1">'path/to/src/Verraes/Lambdalicious/load.php'</span><span class="p">;</span>
<span class="nb">assert</span><span class="p">(</span>
<span class="nf">isatom</span><span class="p">(</span><span class="o">@</span><span class="n">foobar</span><span class="p">)</span>
<span class="p">);</span>
</code></pre></div></div>
<p>(Note that if you combine all the code in this post in a single file, it is executable and all assertions will pass.)</p>
<p>Unlike constants in PHP, atoms don’t need to be defined before you use them. They are global and immutable.</p>
<p>Atoms are very useful for many things. Later on, we’ll use them to refer to global functions, without needing to resort to quotes. In fact, the philosophy of λlicious is to remove syntactical noise, verbosity, and ceremony. To that effect, let’s get rid of the <code class="language-plaintext highlighter-rouge">@</code> prefix. We can do that by affixing an atom using the <code class="language-plaintext highlighter-rouge">atom</code> function:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nf">atom</span><span class="p">(</span><span class="o">@</span><span class="n">foobar</span><span class="p">);</span>
<span class="nb">assert</span><span class="p">(</span>
<span class="nf">isatom</span><span class="p">(</span><span class="n">foobar</span><span class="p">)</span> <span class="c1">// no @ prefix needed anymore</span>
<span class="p">);</span>
</code></pre></div></div>
<p>Once an atom is affixed, you can use it everywhere without the prefix.</p>
<h2 id="lists">Lists</h2>
<p>Let’s affix some atoms and make a list.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nf">atom</span><span class="p">(</span><span class="o">@</span><span class="n">a</span><span class="p">,</span> <span class="o">@</span><span class="n">b</span><span class="p">,</span> <span class="o">@</span><span class="n">c</span><span class="p">,</span> <span class="o">@</span><span class="n">d</span><span class="p">);</span>
<span class="nb">assert</span><span class="p">(</span>
<span class="nf">islist</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">])</span>
<span class="p">);</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nf">cons</span><span class="p">(</span><span class="n">d</span><span class="p">,</span> <span class="p">[</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]),</span>
<span class="p">[</span><span class="n">d</span><span class="p">,</span> <span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]</span>
<span class="p">));</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">cons($head, $tail)</code> constructs a list by adding <code class="language-plaintext highlighter-rouge">$head</code> to the front of the list <code class="language-plaintext highlighter-rouge">$tail</code>. In fact, the square brackets are nothing but a shortcut for a bunch of nested <code class="language-plaintext highlighter-rouge">cons</code>.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nf">cons</span><span class="p">(</span><span class="n">a</span><span class="p">,</span> <span class="nf">cons</span><span class="p">(</span><span class="n">b</span><span class="p">,</span> <span class="nf">cons</span><span class="p">(</span><span class="n">c</span><span class="p">,</span> <span class="p">[]))),</span>
<span class="p">[</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]</span>
<span class="p">));</span>
</code></pre></div></div>
<p>We can deconstruct lists by taking the <code class="language-plaintext highlighter-rouge">head</code>, which is the first element, or the <code class="language-plaintext highlighter-rouge">tail</code>, which is all the elements without the head.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nf">head</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]),</span>
<span class="n">a</span>
<span class="p">));</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nf">tail</span><span class="p">([</span><span class="n">a</span><span class="p">,</span> <span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]),</span>
<span class="p">[</span><span class="n">b</span><span class="p">,</span> <span class="n">c</span><span class="p">]</span>
<span class="p">));</span>
</code></pre></div></div>
<p>Remember that <code class="language-plaintext highlighter-rouge">head</code> always returns an element, and <code class="language-plaintext highlighter-rouge">tail</code> returns a list.</p>
<h2 id="functions">Functions</h2>
<p>Now let’s make a global function. Global means that we can reach it from anywhere.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">function</span> <span class="n">half</span><span class="p">(</span><span class="nv">$x</span><span class="p">){</span> <span class="k">return</span> <span class="nf">divide</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="mi">2</span><span class="p">);}</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nf">half</span><span class="p">(</span><span class="mi">6</span><span class="p">),</span>
<span class="mi">3</span>
<span class="p">));</span>
</code></pre></div></div>
<p>If we affix the function’s name, we can refer to it by that atom.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nf">atom</span><span class="p">(</span><span class="o">@</span><span class="n">half</span><span class="p">);</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nf">call</span><span class="p">(</span><span class="n">half</span><span class="p">,</span> <span class="p">[</span><span class="mi">6</span><span class="p">]),</span>
<span class="nf">half</span><span class="p">(</span><span class="mi">6</span><span class="p">)</span>
<span class="p">));</span>
</code></pre></div></div>
<p>We can also make locally scoped functions. We assign them to a variable that we can use to pass the function to another function.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$half</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nf">divide</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="mi">2</span><span class="p">);};</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$half</span><span class="p">(</span><span class="mi">3</span><span class="p">),</span>
<span class="nf">call</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="p">[</span><span class="mi">3</span><span class="p">])</span>
<span class="p">));</span>
</code></pre></div></div>
<h2 id="conditionals">Conditionals</h2>
<p>We’ll need conditionals. In λlicious, they look like this:</p>
<div class="language-text highlighter-rouge"><div class="highlight"><pre class="highlight"><code>return
condition1 ? result1 :
(condition2 ? result2 :
(condition3 ? result3 :
(finalResult)));
</code></pre></div></div>
<p>In other words, conditions are tried one after another until one of them evaluates to true. The value of the complete expression is the result that follows the first true condition. If none of the conditions evaluate to true, the finalResult is returned.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-11-24-higher-order-programming/recursive_percy_small.jpg" alt="Recursive Percy" /></p>
<h2 id="loops--list-processing">Loops & List Processing</h2>
<p>Now we want to get the halves of a whole list of integers. Our test will look like this:
<code class="language-plaintext highlighter-rouge">assert(isequal($halves([2, 4, 6]), [1, 2, 3]));</code></p>
<p>In λlicious, we only ever assign a value to a variable once. PHP does not constrain this: you can override the value of a variable as much as you like. Worry not: a λlicious-programmer is disciplined and strong-minded, and does not require any hand-holding from the compiler. This also means that we never use loops like foreach, while, …, as they mutate variables. The alternative way to loop, is by using recursion.</p>
<p><code class="language-plaintext highlighter-rouge">$halves</code> takes a list as an argument. We’ll calculate the half of the head of that list, and then recurse to <code class="language-plaintext highlighter-rouge">$halves</code> again with the tail of the list. We cons up the calculated half with the result of the recursive call to <code class="language-plaintext highlighter-rouge">$halves</code>. Here’s our first attempt:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$halves1</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="k">use</span><span class="p">(</span><span class="o">&</span><span class="nv">$halves1</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span>
<span class="nf">cons</span><span class="p">(</span> <span class="c1">// create a new list consisting of:</span>
<span class="nf">divide</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="mi">2</span><span class="p">),</span> <span class="c1">// the half of the first element</span>
<span class="nv">$halves1</span><span class="p">(</span><span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">))</span> <span class="c1">// the halves of the remaining elements</span>
<span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<p>(The <code class="language-plaintext highlighter-rouge">use(&$halves1)</code> deserves some explanation. At this point in the code, <code class="language-plaintext highlighter-rouge">$halves1</code> is not yet defined. By closing it in our function by reference with <code class="language-plaintext highlighter-rouge">&</code>, we can call it at the point in the code where we reach <code class="language-plaintext highlighter-rouge">$halves1(tail($list))</code>. It’s a little trick in PHP to recurse on locally scoped functions. Don’t worry about it. When you get a <code class="language-plaintext highlighter-rouge">Notice: Undefined variable: foo</code>, just remember to add <code class="language-plaintext highlighter-rouge">use(&$foo)</code>.)</p>
<p>If we try to use <code class="language-plaintext highlighter-rouge">$halves1</code>, we crash with a message saying <code class="language-plaintext highlighter-rouge">HeadIsDefinedOnlyForNonEmptyLists</code>. The problem is that we are recursing, but we have not thought about how the loop stops. The λlicious-programmer accepts that mistakes happen, and draws lessons from it. From now on, whenever we recurse, we will think about the termination first.</p>
<p>As we are picking elements of our <code class="language-plaintext highlighter-rouge">$list</code>, it will become empty at some point. We’ll need some place to store elements that we halved, so we add an accumulator called <code class="language-plaintext highlighter-rouge">$acc</code>, which starts as an empty list. Whe <code class="language-plaintext highlighter-rouge">$list</code> becomes empty, we return the accumulated values in <code class="language-plaintext highlighter-rouge">$acc</code>.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$halves2</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&</span><span class="nv">$halves2</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span>
<span class="nf">isempty</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="o">?</span> <span class="nv">$acc</span> <span class="o">:</span> <span class="c1">// return $acc when we're done picking off items</span>
<span class="nv">$halves2</span><span class="p">(</span> <span class="c1">// Recurse</span>
<span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span>
<span class="nf">cons</span><span class="p">(</span> <span class="c1">// Our new $acc will be our newly calculated half, </span>
<span class="c1">// followed by the old $acc</span>
<span class="nf">divide</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="mi">2</span><span class="p">),</span>
<span class="nv">$acc</span>
<span class="p">)</span>
<span class="p">);</span>
<span class="p">};</span>
</code></pre></div></div>
<p>The assert still fails. Closer inspection learns that <code class="language-plaintext highlighter-rouge">$halves([2, 4, 6])</code> returns <code class="language-plaintext highlighter-rouge">[3, 2, 1]</code>. That makes sense: we keep consing up the halved head of the original list, to the accumulated list, so our <code class="language-plaintext highlighter-rouge">$acc</code> ends up backwards. That’s easy to fix. We simply call <code class="language-plaintext highlighter-rouge">reverse</code> on <code class="language-plaintext highlighter-rouge">$acc</code> before returning it.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$halves3</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&</span><span class="nv">$halves3</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span>
<span class="nf">isempty</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="o">?</span> <span class="nf">reverse</span><span class="p">(</span><span class="nv">$acc</span><span class="p">)</span> <span class="o">:</span> <span class="c1">// Reversing $acc at the last moment</span>
<span class="nv">$halves3</span><span class="p">(</span>
<span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span>
<span class="nf">cons</span><span class="p">(</span> <span class="c1">// This is where our halved heads end up in the wrong order</span>
<span class="nf">divide</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span> <span class="mi">2</span><span class="p">),</span>
<span class="nv">$acc</span>
<span class="p">)</span>
<span class="p">);</span>
<span class="p">};</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$halves3</span><span class="p">([</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">]),</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="p">));</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">$halves3</code> passes our test. Recursion is great fun, and can be very rewarding. It’s especially useful for making obvious jokes. If you feel you don’t really get recursion, I advise you to read <a href="/2014/11/higher-order-programming/">this wonderful blog post</a>.</p>
<h2 id="deduplication">Deduplication</h2>
<p><code class="language-plaintext highlighter-rouge">$halves3</code> works, but we’re not quite happy with it. <code class="language-plaintext highlighter-rouge">divide(head($list), 2)</code> duplicates the logic of our <code class="language-plaintext highlighter-rouge">$half</code> function. Let’s inject that function into our 4th version.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$halves4</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&</span><span class="nv">$halves4</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span>
<span class="nf">isempty</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="o">?</span> <span class="nf">reverse</span><span class="p">(</span><span class="nv">$acc</span><span class="p">)</span> <span class="o">:</span>
<span class="nv">$halves4</span><span class="p">(</span>
<span class="nv">$half</span><span class="p">,</span> <span class="c1">// Don't forget to keep passing $half along to $halves4</span>
<span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span>
<span class="nf">cons</span><span class="p">(</span>
<span class="nv">$half</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">)),</span> <span class="c1">// Using our injected $half function</span>
<span class="nv">$acc</span>
<span class="p">)</span>
<span class="p">);</span>
<span class="p">};</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$halves4</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">]),</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="p">));</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">$halves4</code> is a higher order function: it takes another function as an argument. In our case, it applies this function to each element of <code class="language-plaintext highlighter-rouge">$list</code>. This pattern is actually quite common. So common indeed, that it has a name: map. We can simply rename <code class="language-plaintext highlighter-rouge">$halves4</code> and <code class="language-plaintext highlighter-rouge">$half</code> to be more generic.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$map</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$function</span><span class="p">,</span> <span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&</span><span class="nv">$map</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span>
<span class="nf">isempty</span><span class="p">(</span><span class="nv">$list</span><span class="p">)</span> <span class="o">?</span> <span class="nf">reverse</span><span class="p">(</span><span class="nv">$acc</span><span class="p">)</span> <span class="o">:</span>
<span class="nv">$map</span><span class="p">(</span> <span class="c1">// Recurse over map</span>
<span class="nv">$function</span><span class="p">,</span> <span class="c1">// Passing $function along to the next call of $map</span>
<span class="nf">tail</span><span class="p">(</span><span class="nv">$list</span><span class="p">),</span>
<span class="nf">cons</span><span class="p">(</span>
<span class="nv">$function</span><span class="p">(</span><span class="nf">head</span><span class="p">(</span><span class="nv">$list</span><span class="p">)),</span> <span class="c1">// Using our injected $function</span>
<span class="nv">$acc</span>
<span class="p">)</span>
<span class="p">);</span>
<span class="p">};</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$map</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">]),</span> <span class="c1">// Map $half over the list</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="p">));</span>
</code></pre></div></div>
<p>(Later on we’ll use the built-in global function <code class="language-plaintext highlighter-rouge">map</code> instead of our own <code class="language-plaintext highlighter-rouge">$map</code>, as it’s a little smarter.)</p>
<h2 id="filter-and-reduce">Filter and Reduce</h2>
<p><code class="language-plaintext highlighter-rouge">filter</code> and <code class="language-plaintext highlighter-rouge">reduce</code> are two other very commonly used higher order functions. They are globally defined in λlicious. The implementation is similar to that of <code class="language-plaintext highlighter-rouge">map</code>, so I’ll leave them as an exercise to the λlicious student. We’ll use them later in this post. Here’s a skeleton to get you started:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$filter</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$function</span><span class="p">,</span> <span class="nv">$list</span><span class="p">,</span> <span class="nv">$acc</span> <span class="o">=</span> <span class="p">[])</span> <span class="k">use</span><span class="p">(</span><span class="o">&</span><span class="nv">$filter</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span>
<span class="no">TERMINATING_CONDITION</span> <span class="o">?</span> <span class="no">SOME_VALUE</span> <span class="o">:</span>
<span class="nv">$filter</span><span class="p">(</span>
<span class="nv">$function</span><span class="p">,</span>
<span class="no">A_LIST</span><span class="p">,</span>
<span class="no">CALCULATE_THE_NEW_ACCUMULATOR</span>
<span class="p">)</span>
<span class="p">;</span>
<span class="p">};</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$filter</span><span class="p">(</span><span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nv">$x</span> <span class="o">>=</span> <span class="mi">3</span><span class="p">;},</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]),</span>
<span class="p">[</span><span class="mi">3</span><span class="p">,</span> <span class="mi">4</span><span class="p">]</span>
<span class="p">));</span>
<span class="nv">$reduce</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$function</span><span class="p">,</span> <span class="nv">$list</span><span class="p">,</span> <span class="nv">$init</span><span class="p">)</span> <span class="k">use</span><span class="p">(</span><span class="o">&</span><span class="nv">$reduce</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span>
<span class="no">TERMINATING_CONDITION</span> <span class="o">?</span> <span class="no">SOME_VALUE</span> <span class="o">:</span>
<span class="nv">$reduce</span><span class="p">(</span>
<span class="nv">$function</span><span class="p">,</span>
<span class="no">A_LIST</span><span class="p">,</span>
<span class="no">CALCULATE_THE_NEW_INIT</span>
<span class="p">)</span>
<span class="p">;</span>
<span class="p">};</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$reduce</span><span class="p">(</span><span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="nv">$y</span><span class="p">)</span> <span class="p">{</span> <span class="k">return</span> <span class="nf">add</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="nv">$y</span><span class="p">);},</span> <span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">],</span> <span class="mi">0</span><span class="p">),</span>
<span class="mi">6</span>
<span class="p">));</span>
</code></pre></div></div>
<h2 id="functions-returning-functions">Functions returning functions</h2>
<p>Functions that take functions as arguments are just one type of higher order functions. We can also return functions. Let’s make a function that makes <code class="language-plaintext highlighter-rouge">$half</code>.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$halfMaker</span> <span class="o">=</span> <span class="k">function</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// make a new function and return it </span>
<span class="k">return</span> <span class="nf">divide</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span>
<span class="p">};</span>
<span class="p">};</span>
<span class="nv">$half2</span> <span class="o">=</span> <span class="nv">$halfMaker</span><span class="p">();</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$half2</span><span class="p">(</span><span class="mi">8</span><span class="p">),</span>
<span class="mi">4</span>
<span class="p">));</span>
</code></pre></div></div>
<p>This may seem a little pointless. But this is actually a pattern as well. We’ve hardcoded the division by 2, but what if we want to make that number dynamic? We rename <code class="language-plaintext highlighter-rouge">$halfMaker</code> into <code class="language-plaintext highlighter-rouge">$divisionMaker</code> and take a number as an argument:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$divisionMaker</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$y</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="k">function</span> <span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="k">use</span> <span class="p">(</span><span class="nv">$y</span><span class="p">)</span> <span class="p">{</span> <span class="c1">// make a closure with $y</span>
<span class="k">return</span> <span class="nf">divide</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="nv">$y</span><span class="p">);</span>
<span class="p">};</span>
<span class="p">};</span>
<span class="nv">$third</span> <span class="o">=</span> <span class="nv">$divisionMaker</span><span class="p">(</span><span class="mi">3</span><span class="p">);</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$third</span><span class="p">(</span><span class="mi">9</span><span class="p">),</span>
<span class="mi">3</span>
<span class="p">));</span>
</code></pre></div></div>
<p>We can make it even more generic. Right now, <code class="language-plaintext highlighter-rouge">division</code> is hardcoded into <code class="language-plaintext highlighter-rouge">$divisionMaker</code>. We can actually pass that function in as well. We rename <code class="language-plaintext highlighter-rouge">$divisionMaker</code> to <code class="language-plaintext highlighter-rouge">$partial</code>.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$partial</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$f</span><span class="p">,</span> <span class="nv">$y</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="k">use</span><span class="p">(</span><span class="nv">$f</span><span class="p">,</span> <span class="nv">$y</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nv">$f</span><span class="p">(</span><span class="nv">$x</span><span class="p">,</span> <span class="nv">$y</span><span class="p">);</span>
<span class="p">};</span>
<span class="p">};</span>
<span class="nv">$third2</span> <span class="o">=</span> <span class="nv">$partial</span><span class="p">(</span><span class="n">divide</span><span class="p">,</span> <span class="mi">3</span><span class="p">);</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$third2</span><span class="p">(</span><span class="mi">9</span><span class="p">),</span>
<span class="mi">3</span>
<span class="p">));</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">$partial</code> takes two arguments: a function <code class="language-plaintext highlighter-rouge">$f</code> and one argument <code class="language-plaintext highlighter-rouge">$y</code>. It returns a new function, which takes another argument <code class="language-plaintext highlighter-rouge">$x</code>, and return the result of <code class="language-plaintext highlighter-rouge">$f($x, $y)</code>. Partial function application is a nice example of higher order programming: you create new functions from existing functions.</p>
<h2 id="partial-function-application">Partial Function Application</h2>
<p>Wouldn’t it be nice if we had an elegant syntax to partially call functions? In λlicious, many functions already feature partial function application by default:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$half3</span> <span class="o">=</span> <span class="nf">divide</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span> <span class="c1">// returns a function $f($x){ return divide($x, 2); }</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$half3</span><span class="p">(</span><span class="mi">8</span><span class="p">),</span> <span class="mi">4</span>
<span class="p">));</span>
<span class="nv">$increment</span> <span class="o">=</span> <span class="nf">add</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">__</span><span class="p">);</span> <span class="c1">// returns a function $f($y){ return add(1, $y); }</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$increment</span><span class="p">(</span><span class="mi">5</span><span class="p">),</span> <span class="mi">6</span>
<span class="p">));</span>
</code></pre></div></div>
<p>The double underscores are placeholders: You can think of them as arguments that you don’t know yet, and that you’ll fill in later. Partial function application allows you to define behaviour in one place, and then process data using that behaviour, in another place. It’s great way of writing compact, highly reusable, highly readable code. It might take some getting used to, but once you get it, you don’t want to code without it anymore. Here’s an example with <code class="language-plaintext highlighter-rouge">map</code>:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$halves5</span> <span class="o">=</span> <span class="nf">map</span><span class="p">(</span><span class="nv">$half</span><span class="p">,</span> <span class="n">__</span><span class="p">);</span>
<span class="c1">// elsewhere:</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$halves5</span><span class="p">([</span><span class="mi">2</span><span class="p">,</span> <span class="mi">4</span><span class="p">,</span> <span class="mi">6</span><span class="p">]),</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">]</span>
<span class="p">));</span>
</code></pre></div></div>
<p>At the point where we use <code class="language-plaintext highlighter-rouge">$halves5</code>, we don’t need to know that internally, it’s actually a map of <code class="language-plaintext highlighter-rouge">$half</code>.</p>
<h2 id="composition">Composition</h2>
<p>Sometimes you need a function that is composed of other functions. If we want to half a value and then add one, we can make a function that does the work for us:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$halfAndIncrementMaker</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$half3</span><span class="p">,</span> <span class="nv">$increment</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="k">use</span> <span class="p">(</span><span class="nv">$half3</span><span class="p">,</span> <span class="nv">$increment</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nv">$increment</span><span class="p">(</span><span class="nv">$half3</span><span class="p">(</span><span class="nv">$x</span><span class="p">));</span>
<span class="p">};</span>
<span class="p">};</span>
<span class="nv">$halfAndIncrement</span> <span class="o">=</span> <span class="nv">$halfAndIncrementMaker</span><span class="p">(</span><span class="nv">$half3</span><span class="p">,</span> <span class="nv">$increment</span><span class="p">);</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$halfAndIncrement</span><span class="p">(</span><span class="mi">10</span><span class="p">),</span> <span class="mi">6</span>
<span class="p">));</span>
</code></pre></div></div>
<p>Once again, we’ll turn our <code class="language-plaintext highlighter-rouge">$halfAndIncrementMaker</code> into a more generic function, called <code class="language-plaintext highlighter-rouge">compose</code>. It’s another example of a higher order function: it takes functions as arguments and returns a new function.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$compose</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$f</span><span class="p">,</span> <span class="nv">$g</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="k">function</span><span class="p">(</span><span class="nv">$x</span><span class="p">)</span> <span class="k">use</span> <span class="p">(</span><span class="nv">$f</span><span class="p">,</span> <span class="nv">$g</span><span class="p">)</span> <span class="p">{</span>
<span class="k">return</span> <span class="nv">$g</span><span class="p">(</span><span class="nv">$f</span><span class="p">(</span><span class="nv">$x</span><span class="p">));</span>
<span class="p">};</span>
<span class="p">};</span>
<span class="nv">$halfAndIncrement</span> <span class="o">=</span> <span class="nv">$compose</span><span class="p">(</span><span class="nv">$half3</span><span class="p">,</span> <span class="nv">$increment</span><span class="p">);</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$halfAndIncrement</span><span class="p">(</span><span class="mi">10</span><span class="p">),</span> <span class="mi">6</span>
<span class="p">));</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">$compose</code> creates a function that calls <code class="language-plaintext highlighter-rouge">$f</code> on <code class="language-plaintext highlighter-rouge">$x</code>, and then calls <code class="language-plaintext highlighter-rouge">$g</code> on the result of that. The built–in <code class="language-plaintext highlighter-rouge">compose</code> is a little bit smarter and accepts more than two arguments.</p>
<h2 id="piping">Piping</h2>
<p><code class="language-plaintext highlighter-rouge">pipe</code> is the same as <code class="language-plaintext highlighter-rouge">compose</code>, but it applies the functions in reverse. In my opinion, it is more natural to read. Think of pipes and filters in Linux. All together now:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$half5</span> <span class="o">=</span> <span class="nf">divide</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span>
<span class="nv">$greaterThanSix</span> <span class="o">=</span> <span class="nf">gt</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">6</span><span class="p">);</span>
<span class="nv">$calculate</span> <span class="o">=</span> <span class="nf">pipe</span><span class="p">(</span> <span class="c1">// pipe returns a new function</span>
<span class="nf">map</span><span class="p">(</span><span class="nv">$half5</span><span class="p">,</span> <span class="n">__</span><span class="p">),</span> <span class="c1">// Half all the elements of a list</span>
<span class="nf">filter</span><span class="p">(</span><span class="nv">$greaterThanSix</span><span class="p">,</span> <span class="n">__</span><span class="p">),</span> <span class="c1">// Keep only the elements > 6 of the new list</span>
<span class="nf">reduce</span><span class="p">(</span><span class="n">add</span><span class="p">,</span> <span class="n">__</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="c1">// Add up all the filtered elements, starting with 0</span>
<span class="p">);</span>
<span class="nb">assert</span><span class="p">(</span><span class="nf">isequal</span><span class="p">(</span>
<span class="nv">$calculate</span><span class="p">([</span><span class="mi">10</span><span class="p">,</span> <span class="mi">20</span><span class="p">,</span> <span class="mi">30</span><span class="p">]),</span>
<span class="mi">25</span>
<span class="p">));</span>
</code></pre></div></div>
<p>If we call <code class="language-plaintext highlighter-rouge">$calculate</code> with a list <code class="language-plaintext highlighter-rouge">[10, 20, 30]</code>, the first placeholder is replaced with that list. The outcome is <code class="language-plaintext highlighter-rouge">[5, 10, 15]</code>. That result is fed into the <code class="language-plaintext highlighter-rouge">filter</code>, which removes the <code class="language-plaintext highlighter-rouge">5</code> from that list. Finally, <code class="language-plaintext highlighter-rouge">reduce</code> adds up <code class="language-plaintext highlighter-rouge">10</code> to <code class="language-plaintext highlighter-rouge">0</code>, and than the adds up <code class="language-plaintext highlighter-rouge">15</code> to that <code class="language-plaintext highlighter-rouge">10</code>, resulting in <code class="language-plaintext highlighter-rouge">25</code> which is the final outcome of <code class="language-plaintext highlighter-rouge">$calculate</code>.</p>
<p>A great way to read that code, is to follow the pipe top the bottom. The first placeholder means “fill in a list that we don’t know about yet”. The second placeholder means: “pipe the result of the map to the filter”. The third placeholder means: “pipe the filtered list to reduce”. You can really follow the flow data through the placeholders, without being distracted by the actual data.</p>
<p>We can get rid of <code class="language-plaintext highlighter-rouge">$half5</code> and <code class="language-plaintext highlighter-rouge">$greaterThanTen</code>, to make it even prettier.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$calculate2</span> <span class="o">=</span> <span class="nf">pipe</span><span class="p">(</span>
<span class="nf">map</span><span class="p">(</span><span class="nf">divide</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="n">__</span><span class="p">),</span>
<span class="nf">filter</span><span class="p">(</span><span class="nf">gt</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">6</span><span class="p">),</span> <span class="n">__</span><span class="p">),</span>
<span class="nf">reduce</span><span class="p">(</span><span class="n">add</span><span class="p">,</span> <span class="n">__</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
<span class="p">);</span>
</code></pre></div></div>
<p>We’ve now expressed a calculation as a composition of partially applied functions. It’s compact, elegant, and to the point. If you have trouble understanding what’s going on, you can inject some <code class="language-plaintext highlighter-rouge">dump</code> functions along the way:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$calculate3</span> <span class="o">=</span> <span class="nf">pipe</span><span class="p">(</span>
<span class="nf">map</span><span class="p">(</span><span class="nf">divide</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">2</span><span class="p">),</span> <span class="n">__</span><span class="p">),</span>
<span class="n">dump</span><span class="p">,</span>
<span class="nf">filter</span><span class="p">(</span><span class="nf">gt</span><span class="p">(</span><span class="n">__</span><span class="p">,</span> <span class="mi">6</span><span class="p">),</span> <span class="n">__</span><span class="p">),</span>
<span class="n">dump</span><span class="p">,</span>
<span class="nf">reduce</span><span class="p">(</span><span class="n">add</span><span class="p">,</span> <span class="n">__</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span>
<span class="p">);</span>
</code></pre></div></div>
<p>Imagine you had to write all that in traditional procedural PHP. The beauty is that we separates the definition of how data flows through the functions, from having to deal with the actual data. From a small set of primitive functions, we can build ever more complex abstractions. And that is the point of higher order programming.</p>
<h2 id="lambdalicious-and-php">Lambdalicious and PHP</h2>
<p>Keep in mind that, at the time of writing, λlicious is not stable and might already have changed since you read this. Lambdalicious is not fit for any purpose. If you use it in production and everything breaks, your friends will laugh at you behind your back. I will provide a shoulder to cry on at twice my normal consultancy rates, while saying “I told you so.”</p>
<p>The reason it can not work is that PHP is not suitable for recursion. Some languages have something called tail call optimization, which means that if the last expression is recursive, the compiler will make a jump. The call stack does not increase, and all is well.</p>
<p>PHP will either run out of memory (<em>Fatal error: Allowed memory size of X bytes exhausted</em>) or, if you have XDebug installed, it will stop at a 100 calls by default. (<em>PHP Fatal error: Maximum function nesting level of ‘100’ reached, aborting</em>). You can increase <code class="language-plaintext highlighter-rouge">xdebug.max_nesting_level</code> in php.ini. HHVM overflows (<em>Fatal error: Stack overflow</em>). I promise to send some fine Belgian beer or chocolates to whoever fixes this in PHP or HHVM.</p>
<h2 id="links">Links</h2>
<ul>
<li><a href="https://github.com/mathiasverraes/lambdalicious">Lambdalicious on GitHub</a></li>
<li><a href="https://gist.github.com/mathiasverraes/4b76822c6be565a092f7">The full code for this blog post</a></li>
</ul>
Modelling Heuristics2014-11-20T00:00:00+00:00https://verraes.net/2014/11/modelling-heuristics<p>The slides for my talk at Build Stuff in Vilnius. (As often, the slides are not very interesting without the text.)</p>
<script async="" class="speakerdeck-embed" data-id="047406a05332013247257e07207ad8af" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<p><a href="https://speakerdeck.com/mathiasverraes/modelling-heuristics"><small>See the slides on Speakerdeck</small></a></p>
<h2 id="abstract">Abstract</h2>
<p>“How do you approach software modelling problems?”
When a client asked me about my mental process, I didn’t have a proper answer — yet. The question intrigued me. Do you need a propensity for modelling, or is it a learnable skill? I embarked on a mission to find modelling heuristics: the mental tools, that help us look at the problem from different angles; the little tricks to validate a model before we build it; the rules of thumb that lead us to iteratively refine our solutions. In this talk, I share some of the ideas I collected from books, from people I interviewed, and from my own experiences.</p>
Domain Events2014-11-11T00:00:00+00:00https://verraes.net/2014/11/domain-events<h2 id="boundaries--messages">Boundaries & Messages</h2>
<p>A system’s design is defined by it’s boundaries, and by the way it communicates with other systems. This outward facing design is, all things considered, more important than the internals. It is important whether we are talking about small systems (such as an object) or large systems.</p>
<p>The communication between systems happens with messages. We can say that our system is bounded when it does not need to know anything about the internal structure of systems it communicates with, and those systems do no need to know anything about our system. When the boundaries are chosen well, we can achieve greater decoupling between systems. This allows us to change the internals of the systems, or replace them altogether.</p>
<p>In software, different systems agree on a protocol that describes the shape of the message. Without it, the message is meaningless. The protocol might be implicit, but typically, the more users a protocol has, the more it needs to be formally described. For example, look at the RFCs that describe the Internet. A protocol is not just an encoding, like JSON or XML. It also describes what properties you’ll find in the message, and how to interpret them. Messages can be broadcast, queued, logged, persisted, discarded, … Depending on the infrastructure, they can arrive late, out of order, more than once, or never at all. If we need high precision, we can’t even trust their timestamp. Understanding the infrastructure and taking the constraints into account, is critical here.</p>
<h2 id="separate-models">Separate Models</h2>
<p>The receiving end does not need to understand anything about the sender, as long as it understands the message. In other words, the model that the sender and the receiver must share, is constrained to the message. The receiver might conform to the model of the sender, but it is often better for the receiver to go its separate way. Having different models for different systems, is very powerful. A grand, unified model for all our systems can be convenient in the beginning, but becomes a burden as the systems grow. In separated models, each individual system is liberated from having to know a lot of information, or from having structures that are not suitable for that system’s purpose. Contrast this to typical system integrations using an API that transmits entire entities’ states, but leaves it to the client to figure out what has changed and why. In that scenario, the client knows more about the structure of the server, but has less valuable information. Introducing (or discovering) good boundaries, and the communication between them, is the idea behind Context Mapping and Bounded Contexts in Domain-Driven Design.</p>
<h2 id="immutability">Immutability</h2>
<p>Messages are immutable. Say A sends a message X to B and C. If B were to change the message before passing it on to D, that would not affect A and C. If you draw on a newspaper you’ve received, only your own copy is affected. Modeling messages as immutable reflects that idea. A better interpretation is that B reacts to X by sending a message Y to D. Even though X and Y may be mostly identical, it is in fact a new message.</p>
<p>(Update Nov 14, 2014) Domain Events should not contain Entities (or Aggregates for that matter). Entities represent something that changes over time. At the point in time that we look at the Event, the Entity might already have changed. So now the Domain Event is no longer an accurate representation of state at that point in time.</p>
<h2 id="envelopes">Envelopes</h2>
<p>Sometimes we want to add metadata to a message. These are values that are not part of the actual message, but rather of the surrounding infrastructure: the originating machine, the destination, timestamps, IP’s, the browser, user information, … I like the analogy of an envelope here. You can write on it, without affecting the message. The message is wrapped before being sent, and unwrapped at its arrival. All other systems that are involved in the transport, queuing, or persistence of messages, are unaware of the contents of the envelope. This reflects how postal services have worked for centuries. (Censorship and redaction of messages can be seen as a violation of this contract between sender and receiver.)</p>
<h2 id="events">Events</h2>
<p>An Event is a type of message that describes something that has happened in the past. This is very useful, because using Events, systems can tell other interested parties something about themselves. The receiver can now react to the event. This is modeled after the way things work in the real world.</p>
<p>Domain Events are a special type of Event messages. A Domain Event is something that has happened in the past, that is of interest to the business. This last distinction means we separate all technical concerns from the domain. Usually, the distinction is very clear. In other cases, we should validate with the business whether they are in fact interested that this Domain Event has happened. In rare cases, the line between technical and domain events is blurred.<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup></p>
<p>By its nature, an Event has already happened by the time we know about it. We might cheat, and notify other systems of Events before they happen, but this is risky: if we crash before we can achieve our goal, other systems will believe the Event has happened, where in fact it never took place. For a higher level of security, we will persist the Event before we tell anybody about it.</p>
<h2 id="event-naming">Event Naming</h2>
<p>Precisely because we observe Domain Events from the outside after they have happened, we should name them in the past tense. We’ll want to stay as close to the language of the domain as possible. This reduces all kinds of translation errors when talking to the business. I recommend using natural language and make small sentences, such as “Stock was depleted”, instead of newspaper-style shortcuts such as “Depleted stock”. In my experience, this greatly improves the communication and understanding.</p>
<h2 id="event-contents">Event Contents</h2>
<p>A Domain Event can be as little as just a name. More often, it will contain values and identifiers to answer questions about who did what to whom, when, why, where, and how much. The values represent the state at the time the Event happened. Ideally, the Event only contains values that are directly relevant to that Event. A receiver that needs more information, should listen to other relevant Events, and keep its own state, in order to make decisions later. The receiver shouldn’t query the current state of the sender. Remember that the Event message is received some time after it has happened. The state of the sender might already be different from the state it had at the time it emitted the Event.</p>
<h2 id="fat-events">Fat Events</h2>
<p>In some cases, we can reduce the burden on the receiver to keep track of older state. We add redundant information to the Domain Event, such as (parts of) the previous state of an Entity. The receiver now knows the old value as well as the changed value. A Domain Event can also contain other values, that have not changed, but that are still useful to the receiver. A typical example are labels that accompany an identity (such as a customer’s name, when strictly the Event should only contain the customer’s id). We call these “Fat Events”. There is some risk involved: by ignoring certain types of Events, and getting all its information from redundant values in the Fat Events, it might miss some of the changes. In our example, the receiver might be working with an older version of the customer’s name, because it only listens to sales events and not to events that change the customer’s name.</p>
<h2 id="representation">Representation</h2>
<p>It is a mistake to design Domain Events as objects first, and then worry about serializing them. It should be the other way around. Domain Events are first and foremost about communication with other systems (or with yourself in the future, when you’ll want to understand what happened in the past – business intelligence comes to mind). Because of this, we’ll want to design them keeping our non-functional requirements in mind: efficiency (binary) versus readability (json), using namespacing to prevent clashes, nested or flat, consumability, … Here’s a simple example:</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span> <span class="dl">"</span><span class="s2">eventName</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">acme.my_app.StockWasDepleted</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">payload</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span>
<span class="dl">"</span><span class="s2">productId</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">2028A516-04C0-4D42-81F4-98A42E2B08BC</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">depletedOn</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">2014-11-11</span><span class="dl">"</span><span class="p">,</span>
<span class="dl">"</span><span class="s2">pricePerUnit</span><span class="dl">"</span><span class="p">:</span> <span class="p">{</span> <span class="dl">"</span><span class="s2">amount</span><span class="dl">"</span><span class="p">:</span> <span class="mi">500</span><span class="p">,</span> <span class="dl">"</span><span class="s2">currency</span><span class="dl">"</span><span class="p">:</span> <span class="dl">"</span><span class="s2">EUR</span><span class="dl">"</span><span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The price could be expressed as “EUR 500” instead of a nested structure. The point here is to choose based on what is best for all involved systems, not just what is easiest to produce or consume in the system you are currently working on.</p>
<h2 id="internal-representation">Internal representation</h2>
<p>The different systems producing and consuming Events can each have their own representation of these events. They can share the code when it’s practical, but that’s not a necessity. They should pick representations that are idiomatic to the environment: tuples and records in Erlang, objects in Java/C#/PHP, … We use objects because they are easier to work with than plain strings. The type of the object represents the type of the Event. We make them immutable, to reflect that messages are immutable. The public getters on the objects should return Value Objects: they encapsulate calculations that you might want to do on the values, and they express the domain model better than primitives can.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li>
<li><a href="/2014/07/event-storming-fake-domains-happy-paths/">Event Storming: Fake Domains and Happy Paths</a></li>
<li><a href="/2014/05/functional-foundation-for-cqrs-event-sourcing/">A Functional Foundation for CQRS/Event Sourcing</a></li>
<li><a href="/2014/02/buzzword-free-bounded-contexts/">Buzzword-free Bounded Contexts</a></li>
<li><a href="/2014/01/domain-driven-design-is-linguistic/">Domain-Driven Design is Linguistic</a></li>
</ul>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>If your business is online advertising, then a click or even an impression on a website is a Domain Event. After all, that’s what your business is selling. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Object Reorientation2014-11-07T00:00:00+00:00https://verraes.net/2014/11/object-reorientation<p>Presentation delivered at DrupalCamp Ghent, November 2014</p>
<script async="" class="speakerdeck-embed" data-id="776467504884013239ca36e5e480a335" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<p><small>No slides? <a href="https://speakerdeck.com/mathiasverraes/object-reorientation#">Try here</a></small></p>
“Software design is just theory”2014-10-07T00:00:00+00:00https://verraes.net/2014/10/software-design-is-just-theory<!-- image http://www.deviantart.com/art/Animal-farm-234426211 -->
<p><em>“Software design is just theory.”</em><br />
<em>“Design patterns are too academic.”</em><br />
<em>“Writing code is the only way to become a better programmer.”</em><br />
<em>“Just ship it.”</em><br />
<em>“That’s over-designed.”</em></p>
<p>As a consultant, I visit different clients throughout Europe. Some of them hire me for help with greenfield projects. Some others want my help with legacy projects, that they have been working on for typically 4-6 years. They are in different domains, with completely different codebases, but they also share many of the same problems. Progress is slow, they are spending 80% of their time on bug fixing, and each fix seems to create two new regressions. Some of them use popular open source frameworks, others use their own frameworks.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-10-07-software-design-is-just-theory/animal_farm_small.jpg" alt="animal farm by amanda jackson" /></p>
<p>Early on in my career, I hated working with legacy just as much as everyone else. We work in an industry that is getting better at shipping large amounts of legacy at an ever increasing pace. We have created outsourcing farms, that produce legacy at the speed of typing. It’s legacy as a service<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup>. I felt I had to make a choice: get out, or get better. When you get better at working with legacy, you start enjoying it more. When you enjoy it more, you get better. You get the picture.</p>
<p>The 4-6 years is no coincidence: that’s when they start feeling the pain of every design decision that they made or didn’t make. Everything is coupled; moving one part causes a collapse in everything else. The frameworks that once were convenient and fast to market, are now a hindrance that they cannot get rid of. The RAD features that once put stars in the eyes of the developers, are now cursed upon. The business wants to add new features and access new markets, but they can’t. Management doesn’t understand why the team, that was so fast at delivering anything they were asked, is now slow and reluctant. New recruits take six months or more before they are productive — if they even stay that long.</p>
<p>Software design has a slow feedback cycle. It takes a long time before your design starts biting you in the butt. Over the years, smarter people than myself have seen the same patterns and anti-patterns, and have built designs that worked or didn’t work. They thought about them, looked at them from every angle, discussed them with colleagues. They investigated them with all the methods at our disposal: observation, logical reasoning, experimentation. They wrote blogs, papers, and books about software and design. Their ideas are not “just theory”. It’s years of accumulated practice and insights. They work — often in many different contexts.</p>
<p>We’re lucky: we can build on top of their ideas. You may need to type some more lines of code, but typing is not the bottleneck. You may need to rebuild the same piece of code a couple of times, but surely that’s a better than debugging. Sometimes, you need to over-design it, then take a step back, to understand how much design is just enough. You’ll need to learn a lot. For the rest of your career. You can’t accumulate all that knowledge just by writing some code every day. Read books, discuss them with others, read other people’s code, experiment, draw on a whiteboard, or take a walk and just think.</p>
<p>Learning is the bottleneck<sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup>, and always will be.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/01/build-for-failure/">Build for Failure</a></li>
<li><a href="http://www.jefclaes.be/2011/10/gift-of-legacy.html">The Gift of Legacy</a> by <a href="https://twitter.com/JefClaes">@JefClaes</a></li>
<li><a href="/2012/12/how-to-read-more-books/">How to Read More Books</a></li>
<li><a href="/2011/04/beautiful-code/">Beautiful Code</a></li>
</ul>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>Quote by Pieter Hintjens <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>I’m not sure who said this first, but it appears that Ashley Johnson was one of the first. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Small Uncontrolled Experiments (revisited)2014-10-03T00:00:00+00:00https://verraes.net/2014/10/small-controlled-experiments-revisited<p><a href="/2014/03/small-controlled-experiments/">My blog post on Small Uncontrolled Experiments</a> landed me an invitation to speak at Agile Cambridge. As the slides are probably not that useful if you haven’t seen the presentation, I advise you to read the blog post instead.</p>
<p><strong>Update 2021-01-25: I used to call this “small controlled experiments”. It was a name that grew organically and it stuck. Nowadays, I believe they’re very much uncontrolled: a team is too small and a sprint is too short for anything remotely scientific. That doesn’t mean they’re not valuable. Emotions are data, and if people’s gut feeling after (after, not before!) doing an experiment doesn’t agree, the proposed change will not work well anyway. Impose time constraints on these experiments, but do allow them to be uncontrolled.</strong></p>
<script async="" class="speakerdeck-embed" data-id="5a5733b02d0e01324430365ebb6a31b9" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<p><a href="https://speakerdeck.com/mathiasverraes/small-controlled-experiments#"><small>See the slides on Speakerdeck</small></a></p>
<h2 id="abstract">Abstract</h2>
<p>The project was of to a bad start: an inherited legacy codebase, a waterfall contract, and a projected loss. The promise of Kaizen or Continuous Improvement seemed very appealing. But when we tried to incorporate this into our process, it didn’t catch on. Biweekly retrospectives didn’t seem to expose any problems we could improve upon. The ceremonies we tried, like Deming’s Plan-Do-Check-Act cycles, added too much overhead. We were doing something wrong.</p>
<p>Continuous Improvement implies that you know exactly where to focus your efforts. Like scientists, we started to experiment, without deciding upfront what we expected the outcome to be. The rules? Make every experiment as small as possible. No meetings, no consensus, no cumbersome evaluation process. We let the results speak for themselves. This talk explores the successes and failures of a team that went from survival mode to learning mode over the course of a year.</p>
<p><a href="https://twitter.com/sgrame">Peter Decuyper</a> drew this while I gave the talk at PHPBenelux 2015:</p>
<p><img src="/img/posts/2014-10-03-small-controlled-experiments-revisited/sketchnotes_by_@sgrame-small.jpg" alt="Small Uncontrolled Experiments, drawn by @sgrame" /></p>
<h2 id="read-more">Read more</h2>
<p><a href="/2014/03/small-controlled-experiments/">The original blog post on Small Uncontrolled Experiments</a></p>
Objects as Contracts for Behaviour2014-09-29T00:00:00+00:00https://verraes.net/2014/09/objects-as-contracts-for-behaviour<!-- image source http://www.dexigner.com/news/20067 -->
<p>A lot of so-called Object-Oriented Programming<sup id="fnref:1" role="doc-noteref"><a href="#fn:1" class="footnote" rel="footnote">1</a></sup> is in fact nothing more than procedural code wrapped in objects. The funny thing is: many people actually defend this style of programming! This is one of the chief arguments:</p>
<p>“<em>Objects should just be bags of state and nothing more. An invoice does not pay itself. An appointment does not reschedule itself.</em><sup id="fnref:2" role="doc-noteref"><a href="#fn:2" class="footnote" rel="footnote">2</a></sup> <code class="language-plaintext highlighter-rouge">invoice.pay(anAmount)</code> <em>and</em> <code class="language-plaintext highlighter-rouge">appointment.reschedule(aDate)</code> <em>do not match the real world. Something on the outside drives the action. A service of some kind should do the paying and the rescheduling, and then update the state of those objects:</em> <code class="language-plaintext highlighter-rouge">invoice.setPaidAmount(anAmount)</code> <em>and</em> <code class="language-plaintext highlighter-rouge">appointment.setDate(aDate)</code>. <em>By consequence, not in the objects, but the services should contain the business logic.</em>”</p>
<p>(<strong>Update</strong>: For the record, I do not agree with the view stated above in italics. What follows explains why encapsulating operations in objects, such as invoice.pay(…), is in fact the correct way to model objects.)</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-09-29-objects-as-contracts-for-behaviour/Action_Design_Over_Time_01_gallery_small.jpg" alt="Anaemic objects" /></p>
<p>Of course invoices do not pay themselves, but that’s not what an object model is trying to do. An invoice <strong>exposes the behaviour of being able to be paid</strong>. That ability is in fact essential to what it means to be an invoice. The behaviour is an inherent property of an invoice. If an invoice doesn’t have the ability of being paid, there’s no point in issuing invoices at all. In other words, the contract of an invoice object declares that its interface includes payment as a feature. Its promise to the outside world is that it allows an outsider to pay for it.</p>
<p>Encapsulation of state and behaviour is the core idea behind objects. Go forth and encapsulate!</p>
<h2 id="language">Language</h2>
<p>Is the paradigm of Object-Oriented to blame for this widespread misunderstanding, or is it merely a matter of education? Part of the confusion is caused by the language. In languages like English, in an imperative sentence, the verb comes before the object. In OOP languages, it’s the other way around. I’d love to see a programming language that puts method calls before the object, but leaves the method arguments at the end <sup id="fnref:3" role="doc-noteref"><a href="#fn:3" class="footnote" rel="footnote">3</a></sup>: <code class="language-plaintext highlighter-rouge">pay.invoice(anAmount)</code> and <code class="language-plaintext highlighter-rouge">reschedule.appointment(aDate)</code>.</p>
<h2 id="footnotes">Footnotes</h2>
<div class="footnotes" role="doc-endnotes">
<ol>
<li id="fn:1" role="doc-endnote">
<p>The term Object-Oriented Programming is rather overloaded. There’s the style that is used in many popular languages like Java and PHP; and there’s the original Smalltalk-style or Alan Kay-style object-oriented programming, which is a misnomer because it puts the focus on messaging and not on objects. I’m talking about the former. <a href="#fnref:1" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:2" role="doc-endnote">
<p>The traditional argument used to involve “a car doesn’t drive itself”. Obviously by now people have realised that a car should indeed drive itself. Technology has failed us for so long, but change is coming. <a href="#fnref:2" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
<li id="fn:3" role="doc-endnote">
<p>Functional programming does that to some degree, because it’s all functions operating on data: <code class="language-plaintext highlighter-rouge">pay(invoice, anAmount)</code>. But the <code class="language-plaintext highlighter-rouge">pay</code> function and the invoice data are not strictly coupled to each other. That opens the possibility for another function to manipulate the data without respecting the business rules that pay() implements. Apart from programmer discipline, I haven’t figured out yet if FP allows to somehow combine the benefits of strict encapsulation, with the fluency of the language. <strong>Update</strong> October 8, 2014: Clojure, when working with Java objects. But I guess Clojure stops being Clojure when you start talking to Java :-) <a href="#fnref:3" class="reversefootnote" role="doc-backlink">↩</a></p>
</li>
</ol>
</div>
Decoupling the Model from the Framework2014-09-01T00:00:00+00:00https://verraes.net/2014/09/decoupling-model-framework<h2 id="video">Video</h2>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="//www.youtube-nocookie.com/embed/QaIGN_cTcc8" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<p>The talk that I mention is <a href="https://www.youtube.com/watch?v=0L_9NutiJlc">The Framework as an Implementation Detail</a> by Marcello Duarte and Konstantin Kudryashov.</p>
<h2 id="presentations">Presentations</h2>
<ul>
<li>Laracon Europe in Amsterdam, August 30, 2014</li>
<li>Symfony User Group Belgium, Ghent, January 29, 2014</li>
</ul>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2013/04/crud-is-an-anti-pattern/">CRUD is an anti-pattern</a></li>
<li><a href="/2013/04/decoupling-symfony2-forms-from-entities/">Decoupling (Symfony2) Forms from Entities</a></li>
<li><a href="/2014/11/domain-events/">Domain Events</a></li>
<li><a href="/2014/11/domain-events/">Messaging Flavours</a></li>
</ul>
Sterile Estimation2014-08-26T00:00:00+00:00https://verraes.net/2014/08/sterile-estimation<div style="border: 1px dotted gray; padding: 10px; width: 50%; margin-bottom: 10px;">
<strong>On Estimation</strong>
<ol>
<li>
<a href="/2014/06/cost-of-estimation/">The Cost of Estimation</a>
</li>
<li>
<a href="/2014/08/sterile-estimation/">Sterile Estimation</a>
</li>
</ol>
</div>
<p><em>To reduce the risk of incidents, the FAA imposed the so-called “sterile cockpit” regulation. Most air traffic accidents happen while taking off or landing, or in the vicinity of airports. The rule instructs pilots and cabin crew to refrain from non-essential activities and conversations while the plane is below 10.000 feet.</em></p>
<h2 id="anchoring-bias">Anchoring Bias</h2>
<p>Let’s try an experiment. Do you believe me when I tell you that Mahatma Ghandi died at the ripe old age of 140? Probably not, because that would have been rather spectacular. Just make a guess of what his actual age was, and write it down.</p>
<p>Now go to a colleague (you are reading this at work, aren’t you?) and ask them whether Ghandi died at age 9. Then ask them to estimate his age was. My prediction is that their answer will be significantly lower than yours (assuming neither of you knows the real answer). The effect at play here is called the <strong>Anchoring Bias</strong>.</p>
<p>Car salesmen have known about it for a long time of course. They’ll start by quoting a high price for a car, more than it’s worth. Because they’ve anchored this number in your mind, any lower price they will quote you next, will seem quite reasonable to you. You are not comparing the number to whatever you’re willing to spend, but to whatever they put in your head. (I bet you can see where I’m going with this.)</p>
<p>Tversky and Kahneman (and others), have studied this phenomenon. The book <a href="http://www.amazon.com/gp/product/B00555X8OA/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B00555X8OA&linkCode=as2&tag=verraesnet-20&linkId=WRTRBQWNNPU4DUAC">“Thinking Fast and Slow”</a> describes some interesting discoveries.</p>
<ul>
<li>The anchoring bias really works, in all contexts, with all kinds of numbers and estimates. When your manager comes in and asks: “Could you finish this new feature in two weeks?”, your estimate will be somewhere around two weeks.</li>
<li>The bias also works when the anchoring number is mentioned out of context. I could say something like “It’s almost five o’clock, I need to go pick up my kids.” Following that, I ask you how long it will take to fix a certain bug. You’re now very likely to answer between 4 and 6.</li>
<li>It even works when the numbers are not time related: “I had 50 guests at my parties.” vs “How many M&M’s go in a single pack?”</li>
<li>And the strangest thing: it works, <strong>even when you first explain anchoring to somebody, and then ask them to make an estimate!</strong></li>
</ul>
<h2 id="planning-poker">Planning Poker</h2>
<p>Planning Poker is an estimation method based around the idea of preventing anchoring. The team members first discuss a story, and then choose a card with the number of their estimate. They all turn the cards simultaneously. If the numbers vary, a discussion is needed to resolve the difference. You reach consensus, or defer the estimate and await more information.</p>
<p>However, with an anchoring bias looming over our heads, and knowing it affects even the most hardened psychologists, we may need to introduce some extra care. It’s very easy to either accidentally influence the results, or manipulate them on purpose. (In fact, some managers believe that if they can trick you into estimating lower, the work will be done faster. And some customers believe that if they can get a project done cheaper, they will still get the same quality. Ah, the joys of our industry!)</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-08-26-sterile-estimation/clock-small.jpg" alt="Sterile Estimation" /></p>
<h2 id="sterile-estimation">Sterile Estimation</h2>
<p>If we must have estimates, whether time or story point based, we need a clean environment. <strong>Sterile Estimation</strong>, like sterile cockpits, prevent unfortunate accidents. Tell all the participants up front of the problem of anchoring bias, and ask them not to mention numbers. When someone does mention a number, consider whether this contaminates the planning session.</p>
<p>After the first round, the consensus estimate is now the new anchor. You can use this to your advantage. Make sure to pick an average story for the first estimation of the session. Even better is to look at some completed stories first. That way, new stories are estimated in relation to older ones. The anchoring bias helps you to keep estimating on the same scale throughout a project.</p>
<p>Of course, creating a truly sterile environment is hard. For me, explaining anchoring bias to clients, is just a tool: it illustrates the idea that estimates are not data, but just guesses. Costly, easily manipulated, and sometimes dangerous, guesses.</p>
<div style="border: 1px dotted gray; padding: 10px; width: 50%; margin-bottom: 10px;">
<strong>On Estimation</strong>
<ol>
<li>
<a href="/2014/06/cost-of-estimation/">The Cost of Estimation</a>
</li>
<li>
<a href="/2014/08/sterile-estimation/">Sterile Estimation</a>
</li>
</ol>
</div>
Resolving Feature Envy in the Domain2014-08-11T00:00:00+00:00https://verraes.net/2014/08/resolving-feature-envy-in-the-domain<p><a href="https://twitter.com/beberlei">Benjamin Eberlei</a> did <a href="http://www.whitewashing.de/2014/08/11/spotting_featureenvy_and_refactoring.html">a really nice job of explaining refactoring the Feature Envy code smell</a> on his blog. I wrote a comment because I felt the example could be taken one step further. You should read the original post. Below are Benjamin’s code examples (for reference), followed by a repost of my comment.</p>
<h2 id="original-examples">Original examples</h2>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">function</span> <span class="n">calculateReport</span><span class="p">(</span><span class="kt">DateTime</span> <span class="nv">$start</span><span class="p">,</span> <span class="nv">$days</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nv">$days</span> <span class="o"><</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">InvalidArgumentException</span><span class="p">(</span><span class="nv">$days</span><span class="p">);</span>
<span class="p">}</span>
<span class="nv">$endDate</span> <span class="o">=</span> <span class="k">clone</span> <span class="nv">$start</span><span class="p">;</span>
<span class="nv">$endDate</span><span class="o">-></span><span class="nf">modify</span><span class="p">(</span><span class="s1">'+'</span> <span class="mf">.</span> <span class="nv">$days</span> <span class="mf">.</span> <span class="s1">' day'</span><span class="p">);</span>
<span class="c1">// calculation code here</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Benjamin then explains how the calculation of the end date can be moved into a custom DateTime class, thus freeing <code class="language-plaintext highlighter-rouge">calculateReport()</code> from the responsibility:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">DateTime</span>
<span class="p">{</span>
<span class="c1">// ... other methods</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">getDateDaysInFuture</span><span class="p">(</span><span class="nv">$days</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nv">$days</span> <span class="o"><</span> <span class="mi">0</span><span class="p">)</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">InvalidArgumentException</span><span class="p">(</span><span class="nv">$days</span><span class="p">);</span>
<span class="p">}</span>
<span class="nv">$endDate</span> <span class="o">=</span> <span class="k">clone</span> <span class="nv">$this</span><span class="p">;</span>
<span class="nv">$endDate</span><span class="o">-></span><span class="nf">modify</span><span class="p">(</span><span class="s1">'+'</span> <span class="mf">.</span> <span class="nv">$days</span> <span class="mf">.</span> <span class="s1">' day'</span><span class="p">);</span>
<span class="k">return</span> <span class="nv">$endDate</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">function</span> <span class="n">calculateReport</span><span class="p">(</span><span class="kt">DateTime</span> <span class="nv">$start</span><span class="p">,</span> <span class="nv">$days</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$endDate</span> <span class="o">=</span> <span class="nv">$start</span><span class="o">-></span><span class="nf">getDateDaysInFuture</span><span class="p">(</span><span class="nv">$days</span><span class="p">);</span>
<span class="c1">// calculation code here.</span>
<span class="p">}</span>
</code></pre></div></div>
<h2 id="whole-value">Whole Value</h2>
<p><code class="language-plaintext highlighter-rouge">calculateReport()</code> is envious of an object that does not yet exist. <code class="language-plaintext highlighter-rouge">DateTime</code> and <code class="language-plaintext highlighter-rouge">$days</code> (an int) belong closely together, they are one concept. Grouping them together into one logical value is a pattern called Whole Value. The implementation of that pattern in OOP is a Value Object.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">function</span> <span class="n">calculateReportFor</span><span class="p">(</span><span class="kt">ReportingPeriod</span> <span class="nv">$reportingPeriod</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// no more period calculation here</span>
<span class="c1">// calculation code here.</span>
<span class="p">}</span>
</code></pre></div></div>
<p>By having a single value object that represents a period instead of two values, we have the added benefit that <code class="language-plaintext highlighter-rouge">calculateReportFor()</code> adheres to SRP now: it doesn’t calculate periods, it only calculates reports.</p>
<p>So where does the <code class="language-plaintext highlighter-rouge">ReportingPeriod</code> come from? We could just instantiate it:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$startDate</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">DateTime</span><span class="p">(</span><span class="s1">'now'</span><span class="p">);</span>
<span class="nv">$endDate</span> <span class="o">=</span> <span class="nv">$start</span><span class="o">-></span><span class="nf">getDateDaysInFuture</span><span class="p">(</span><span class="nv">$days</span><span class="p">);</span>
<span class="nv">$reportingPeriod</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">ReportingPeriod</span><span class="p">(</span><span class="nv">$startTime</span><span class="p">,</span> <span class="nv">$endDate</span><span class="p">);</span>
</code></pre></div></div>
<p>But now we moved to responsibility of calculating the end date to the client, which creates lots of boring duplication. Value objects have the property of attracting behavior. We can add a nice little static factory method to instantiate our <code class="language-plaintext highlighter-rouge">ReportingPeriod</code>:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$reportingPeriod</span> <span class="o">=</span> <span class="nc">ReportingPeriod</span><span class="o">::</span><span class="nf">forDuration</span><span class="p">(</span><span class="nc">DateTime</span> <span class="nv">$startDate</span><span class="p">,</span> <span class="nv">$days</span><span class="p">);</span>
</code></pre></div></div>
<h2 id="applying-domain-driven-design">Applying Domain-Driven Design</h2>
<p>Now let’s take a closer look at the domain. (Of course there’s no real domain in Benjamin’s example code, so I’m going to assume we’re dealing with finance.) Reports in financial statements don’t just cover any random period, they cover months, quarters, or years. Let’s make that explicit:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nf">calculateReportFor</span><span class="p">(</span><span class="nc">ReportingPeriod</span><span class="o">::</span><span class="nf">month</span><span class="p">(</span><span class="mi">12</span><span class="p">,</span> <span class="mi">2014</span><span class="p">));</span>
<span class="nf">calculateReportFor</span><span class="p">(</span><span class="nc">ReportingPeriod</span><span class="o">::</span><span class="nf">quarter</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="mi">2014</span><span class="p">));</span>
<span class="nf">calculateReportFor</span><span class="p">(</span><span class="nc">ReportingPeriod</span><span class="o">::</span><span class="nf">year</span><span class="p">(</span><span class="mi">2014</span><span class="p">));</span>
</code></pre></div></div>
<p>Because calculateReportFor() is decoupled from the calculation of periods, it’s reusable for any kind of period, so the option of passing in arbitrary ReportPeriods is still there if we need it. The important thing is that we are modelling the Ubiquitous Language, staying close to the domain, and have smaller, simpler, more reusable, SRP objects, that are also easier to test in isolation.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/06/named-constructors-in-php/">Named Constructors</a></li>
<li><a href="/2014/06/when-to-use-static-methods-in-php/">When to Use Static Methods</a></li>
<li><a href="/2014/08/dry-is-about-knowledge/">DRY is about Knowledge</a></li>
</ul>
DRY is about Knowledge2014-08-02T00:00:00+00:00https://verraes.net/2014/08/dry-is-about-knowledge<p>Have a look at these two classes:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span> <span class="c1">// example 1</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Basket</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$products</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">addProduct</span><span class="p">(</span><span class="nv">$product</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="mi">3</span> <span class="o">==</span> <span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-></span><span class="n">products</span><span class="p">))</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">Exception</span><span class="p">(</span><span class="s2">"Max 3 products allowed"</span><span class="p">);</span>
<span class="p">}</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">products</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$product</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Shipment</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$products</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">addProduct</span><span class="p">(</span><span class="nv">$product</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="mi">3</span> <span class="o">==</span> <span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-></span><span class="n">products</span><span class="p">))</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">Exception</span><span class="p">(</span><span class="s2">"Max 3 products allowed"</span><span class="p">);</span>
<span class="p">}</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">products</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$product</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Would you say this is duplicate code? Does it violate the DRY principle (aka “Don’t Repeat Yourself”)?</p>
<p>If it is, then the solution to get rid of it, could be something like this:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span> <span class="c1">// example 2</span>
<span class="k">abstract</span> <span class="kd">class</span> <span class="nc">ProductContainer</span>
<span class="p">{</span>
<span class="k">protected</span> <span class="nv">$products</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">addProduct</span><span class="p">(</span><span class="nv">$product</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="mi">3</span> <span class="o">==</span> <span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-></span><span class="n">products</span><span class="p">))</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">Exception</span><span class="p">(</span><span class="s2">"Max 3 products allowed"</span><span class="p">);</span>
<span class="p">}</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">products</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$product</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Basket</span> <span class="kd">extends</span> <span class="nc">ProductContainer</span> <span class="p">{}</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Shipment</span> <span class="kd">extends</span> <span class="nc">ProductContainer</span> <span class="p">{}</span>
</code></pre></div></div>
<p>The code is identical, but why? As good Domain-Driven Design practitioners, we’d have to check with the business. The product could be a flammable chemical, and for safety concerns, a shipment is not allowed to have more than three of them. As a consequence, we don’t allow customers to order more than three at a time.</p>
<p>In another scenario, the similarity of the code might also be simple coincidence. Maybe supply of the product is limited, and we want to give our customers equal opportunity to buy it.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-08-02-dry-is-about-knowledge/find-the-differences-small.jpg" alt="Find the differences" /></p>
<h2 id="reasons-to-change">Reasons to Change</h2>
<p>Whatever the case, it doesn’t matter. The problem is that the rules might change independently. The business might realize that they can still sell more than three products at a time, and divide the products over multiple shipments. In example 2, we are now stuck with high coupling of both types of domain objects, to the same business rule, and indirectly to each other. Changing the rule for Basket, changes the rule for Shipment, but potentially has dangerous consequences in the real world. The example is of course blatantly simple to refactor, but legacy code is usually a lot harder.</p>
<p>You might be tempted to solve the problem by abstracting the limit:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span> <span class="c1">// example 3</span>
<span class="k">abstract</span> <span class="kd">class</span> <span class="nc">ProductContainer</span>
<span class="p">{</span>
<span class="k">protected</span> <span class="nv">$products</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">addProduct</span><span class="p">(</span><span class="nv">$product</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="nv">$this</span><span class="o">-></span><span class="nf">getProductLimit</span><span class="p">()</span> <span class="o">==</span> <span class="nb">count</span><span class="p">(</span><span class="nv">$this</span><span class="o">-></span><span class="n">products</span><span class="p">))</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">Exception</span><span class="p">(</span><span class="nb">sprintf</span><span class="p">(</span><span class="s2">"Max %d products allowed"</span><span class="p">,</span> <span class="nv">$this</span><span class="o">-></span><span class="nf">getProductLimit</span><span class="p">()));</span>
<span class="p">}</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">products</span><span class="p">[]</span> <span class="o">=</span> <span class="nv">$product</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">abstract</span> <span class="k">protected</span> <span class="k">function</span> <span class="n">getProductLimit</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This works to certain extent, but the rules might change in unexpected ways. Example 3 assumes that the limit is always easy to determine with a single method call. New rules might take into account the customer, their history with our company, certain legal conditions, promotions, etc. The new rules can dynamically influence the number of products you can buy, in ways that can’t be modelled using our abstract <code class="language-plaintext highlighter-rouge">addProduct()</code> method.</p>
<h2 id="knowledge">Knowledge</h2>
<p>The business rule in my example is not “Max 3 products allowed”. There are in fact two business rules: “A basket is not allowed to have more than three products” and “A shipment is not allowed to have more than three products”. Two rules, no matter how similar, should have two individual representations in the model.</p>
<p>“Don’t Repeat Yourself” was never about code. It’s about knowledge. It’s about cohesion. If two pieces of code represent the exact same knowledge, they will always change together. Having to change them both is risky: you might forget one of them. On the other hand, if two identical pieces of code represent different knowledge, they will change independently. De-duplicating them introduces risk, because changing the knowledge for one object, might accidentally change it for the other object.</p>
<p>Looking at the reasons for change, is a very powerful modelling heuristic.</p>
<h2 id="read-more">Read more</h2>
<p>Follow up posts:</p>
<ul>
<li><a href="http://blog.ploeh.dk/2014/08/07/why-dry/">Why DRY?</a> by Mark Seemann (<a href="https://twitter.com/ploeh">@ploeh</a>)</li>
<li><a href="http://www.devjoy.com/2014/09/aim-for-dry-but-be-willing-to-fall-short/">Aim for DRY, but be willing to fall short</a> by Richard Dalton (<a href="https://twitter.com/richardadalton">@richardadalton</a>]</li>
</ul>
<p>Some posts about Domain-Driven Design:</p>
<ul>
<li><a href="https://verraes.net/2014/01/domain-driven-design-is-linguistic/">Domain-Driven Design is linguistic</a></li>
<li><a href="https://verraes.net/2014/05/why-domain-driven-design-matters/">Why Domain-Driven Design Matters</a></li>
</ul>
Event Storming: on Fake Domains and Happy Paths2014-07-23T00:00:00+00:00https://verraes.net/2014/07/event-storming-fake-domains-happy-paths<p>Inspired by <a href="http://sch3lp.github.io/2014/07/12/event-storming-exercise/">Tim Schraepen’s experience report</a>, I thought I’d list some recent experiences with facilitating Event Storming sessions. You might want to read my <a href="/2013/08/facilitating-event-storming/">older post with notes on facilitating Event Storming sessions</a>, and the <a href="http://ziobrando.blogspot.be/search/label/EventStorming">huge amount of content Alberto Brandolini has produced on the topic</a>.</p>
<h2 id="pick-a-good-domain">Pick a good domain</h2>
<p>If you’re doing it as an exercise, as opposed to a real project, pick a real(-ish) domain.</p>
<p>I’ve been using a government/education project for a long time, because it’s complex, vague, messy, and in constant motion. It turned out to be great domain to teach participants about challenging your own assumptions. For example, report cards that do not show grades, but observations and conclusions, some of which are written by the actual pupils! The discovery phase is quite useful, but as soon as we tried working towards a more concrete model, the rather “fluffy” domain was too hard to explain. You just can’t learn anything useful about this domain in a few hours.</p>
<p>So the last couple of times, when running Event Storming workshops with Jef Claes, Alberto Brandolini, and Stijn Vannieuwenhuyse, we picked a car rental domain. It’s a lot more strict — for example, a car cannot be in two places, driven by two people. That makes it a lot easier to evolve the model with business rules and aggregates. It’s easy to come up with these kinds of business rules, so the role of fake domain expert is easier. That leaves more focus for actually facilitating the exercise.</p>
<p>(Last week I started consulting for an actual car rental company. Talk about having my own assumptions challenged!)</p>
<p>If you get the opportunity though, apply Event Storming with real projects, and real domain experts. It will be easier than using fake domains and experts. And you’ll all have skin in the game, which will focus the discussions.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-07-23-event-storming-fake-domains-happy-paths/event-storming-wall.jpg" alt="Event Storming" /></p>
<h2 id="actors">Actors</h2>
<p>I only add roles or actors to an event when they are not obvious. But before I do, I try to find if there’s a concept I’m missing. There could be a hint in the Ubiquitous Language. Often it turns out that it shouldn’t be modelled as one event, caused by different actors. Instead, there are two similar, but slightly different events. If both the customer and the administrator deal with payments, maybe PaymentWasMade is the customer’s event, and PaymentWasRegistered is the administrator’s event. The different language makes it explicit that it’s not the admin making the payment; they’re only recording that it happened.</p>
<h2 id="duplication">Duplication</h2>
<p>You can discover some kind of cyclic behaviour. An event causes the flow to start over at an earlier point. Or flows branch off into different directions. Or a Process Manager listens to events from different places in the flow. Visualizing this turns into a spaghetti of arrows flying everywhere. And as soon as you draw an arrow, you’ve coupled the position of the sticky to the location of the arrow, making it hard to move them around.</p>
<p>Our terrible developer brains, confused and screwed up by years of “Don’t Repeat Yourself”, fail to see the obvious solution: it’s ok to duplicate events. Relax, everything is going to be fine. It’s just stickies you’re copying, not code. Uncle Bob is not going to haunt your nightmares.</p>
<p>Write down the same event on as many stickies as you need. Place them wherever they occur or influence something else. Use the same name, and perhaps some visual indicator, to make them easily recognisable. As a facilitator, as soon as I notice an event is central to the model and highly reusable, I make a bunch of copies. I put them on the wall or give them to the participants. They can keep focusing on modelling the flows.</p>
<h2 id="happy-path">Happy Path</h2>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-07-23-event-storming-fake-domains-happy-paths/event-storming-with-branches-small.jpg" alt="Happy Path" /></p>
<p>Now that we’ve conquered our irrational fear of writing the same event down twice, there’s lots of cool things we can do. Typically, without duplication, you’d show a branches starting from the same place (first picture). With duplication, you can more clearly visualise different flows. Some of them are Happy Paths, i.e. a flow desired to the business or the user. The others are exceptions, edge cases, race conditions, or any possible outcomes. In the second picture, I mark them with green and red stickies.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-07-23-event-storming-fake-domains-happy-paths/event-storming-happy-path-small.jpg" alt="Happy Path" /></p>
<p>You can experiment with variations in flow. What happens if a customer pays the invoice before we sent it? What happens when the customer contests it before we sent it? Because it’s so cheap to move stickies around, you trigger discussions with non-technical business people, and gain new insights. It’s domain-driven design in its purest form!</p>
<h2 id="visual-language">Visual Language</h2>
<p>Model Storming is closer to the idea of brain storming: anything can happen, and there’s no standard notation. When done well, it leads to finding your own visual notation for various concepts. On the other hand, Event Storming uses a rather strict notation and ordered set of steps. The clear rules make it easier for people who are not used to be creative in an everything-is-permitted environment. Still, that shouldn’t stop us from adding some imaginative visuals into the mix. Draw simple icons like smiling or sad customers, lightning, time bombs, various traffic signs, objects from the actual domain,… They speak to the emotional side of the brain, and can convey a lot of information without explanation. You’ll find you will remember the bits with drawings a lot better as well.</p>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="/2014/11/domain-events/">Domain Events</a></li>
<li><a href="/2013/08/facilitating-event-storming/">Notes on facilitating Event Storming sessions</a></li>
<li><a href="/2015/03/event-storming-storytelling-visualisations/">Event Storming, Storytelling, Visualisations</a></li>
<li><a href="http://ziobrando.blogspot.be/search/label/EventStorming">Posts tagged with Event Storming on Alberto Brandolini’s blog</a></li>
<li><a href="http://sch3lp.github.io/2014/07/12/event-storming-exercise/">Tim Schraepen’s Event Storming experience report</a></li>
<li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li>
</ul>
Managed Technical Debt (revisited)2014-06-27T00:00:00+00:00https://verraes.net/2014/06/managed-technical-debt-revisited<p>Debt is good. Organizations use it all the time to invest. Technical debt can be good. Are you investing wisely?
This talk is based on an <a href="/2013/07/managed-technical-debt/">older blog post of mine</a>.</p>
<script async="" class="speakerdeck-embed" data-id="be237370e027013121922af1816f6724" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="/2013/07/managed-technical-debt/">Managed Technical Debt</a></li>
<li><a href="/2016/04/repair-replace-heuristic-for-legacy-software/">The Repair/Replace Heuristic for Legacy Software</a></li>
</ul>
Information Overload2014-06-27T00:00:00+00:00https://verraes.net/2014/06/information-overload<div class="tags has-addons">
<span class="tag">Update</span>
<span class="tag is-warning">2021-05-23</span>
<span class="tag ">
<a href="/2021/05/its-just-like-heuristic/">
I wrote about the “It's Just Like...” Heuristic which explains this all better.
</a>
</span>
</div>
<p>You are reading this on an Information Overload Device. Probably one that fits in your bag or even your pocket. You also have a large Information Overload Device in your living room, and there’s a bunch of others spread across your home, your office, and your environment.</p>
<p><img style="margin-bottom: 10px" src="/img/posts/2014-06-27-information-overload/biografias-alicia-martin-2.jpeg" alt="Information Overload" /></p>
<p>The way humans deals with information overload, is to scan quickly, and put everything in buckets. We find the similarities with ideas we already know, and then categorize it under the same name. It’s a database that stores entire documents? Oh, then it’s a document database, like the document database I already know about. It has small iterations? It’s an agile methodology like Scrum. It uses the language of the business? It’s like BDD. You get the picture. Categorizing information is often a necessary strategy.</p>
<p>We do so much of the “scan, bucket, repeat”-cycles, that we forget to actually process interesting information. My method is simply this:</p>
<ul>
<li>Be alert;</li>
<li>If you catch yourself looking for similarities, stop;</li>
<li>Ask: What is different about this? What inspired the author of this information to make a different thing, give it a different name, choose this method over the other method?</li>
</ul>
<p>The divergence, the conflicts, the raison d’être’s, that is where the learning happens.</p>
<p>(Post intentionally kept short to prevent information overload).</p>
The Cost of Estimation2014-06-17T00:00:00+00:00https://verraes.net/2014/06/cost-of-estimation<div style="border: 1px dotted gray; padding: 10px; width: 50%; margin-bottom: 10px;">
<strong>On Estimation</strong>
<ol>
<li>
<a href="/2014/06/cost-of-estimation/">The Cost of Estimation</a>
</li>
<li>
<a href="/2014/08/sterile-estimation/">Sterile Estimation</a>
</li>
</ol>
</div>
<h2 id="ballpark-figures">Ballpark Figures</h2>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-06-17-cost-of-estimation/78704029_3e0e8cf027_z.jpg" alt="Risk" /></p>
<p>When somebody asks you to estimate how long a feature or a project will take, you will estimate the <strong>shortest possible time</strong> in which you can complete it. Think about this. Observe your own thoughts while estimating, and you’ll know it’s true. Even if the request was done with the best of intentions, the implicit assumption is not “how long will it take to build it right” but “how fast can you build it”.</p>
<p>If you have been burned by this, you move on to phase 2: padding your estimation. You do it secretly, and you feel like a cheat for doing it. For some reason, it’s <strong>a taboo to estimate the time to do it right, as opposed to doing it fast</strong>.</p>
<p>At the worst end of the spectrum, there’s <strong>blame culture</strong>. Somebody storms into the office, and demands a “guesstimate” or a “ballpark figure”. Yet they still expect you to complete the task in that time, without even hesitating to dump more tasks in your lap. In the end, you are blamed for “not sticking to the plan” or “making wrong estimates”. Distrust between the team and the rest of business grows. A set of parallel universes spring into existence: reality; the plans of the managers; and the public secret that the plan is wrong anyway.</p>
<h2 id="ternary-estimates">Ternary Estimates</h2>
<p>Some people try to hack this, using <strong>ternary estimates</strong>: the combination of the expected duration, the optimistic duration, and the pessimistic duration. I’ve never seen this work in practice. Managing software projects is hard enough as it is, having to deal with three numbers all the time doesn’t make it easier. And the numbers are usually calculated (again implicitly) as <code class="language-plaintext highlighter-rouge">expected time +- 20%</code>, which is not the same as a proper optimistic or pessimistic estimate.</p>
<h2 id="creative-problem-solving">Creative Problem Solving</h2>
<p>All of this is normal. It’s intuitive. Many people ask for estimation without ever giving it any thought. Estimation feels like an effective way to control projects. We measure time and money, because time and money are the easiest to measure. We even fooled ourselves into believing that time <em>is</em> money, and that they can easily be converted back and forth.</p>
<p>It’s been said many times before, but it bears repeating: software development is a creative profession. If you do software design right, you solve every problem only once. That means that <strong>every problem is a new problem</strong>. You’ve never solved it before, so you are in the dark as to what the best solution is and how long that will take.</p>
<p>And it’s messy: there are many possible solutions, all with different strengths and weaknesses. Some are clearly better, but clearly more expensive, but most of the time, that distinction is not obvious. And not every problem needs the <em>best</em> solution; it needs a solution that is preferably cheaper than the total value. Imagine a multidimensional matrix, with many possible solutions scattered across.</p>
<h2 id="the-cost-of-estimation">The Cost of Estimation</h2>
<p>Time and creativity are enemies. You can’t solve problems faster. You can’t <a href="http://amzn.to/1iDPNQY">think faster</a>. There’s some wonderful chemistry going on in your brain when you think, and you can’t just speed that up by pushing harder.</p>
<p>This is the real cost of estimation: When somebody watches over your shoulder, asking how much longer it will take; when it’s three in the morning and you’re patching a server; in short when you are under pressure, then you <strong>stop solving problems</strong>. Your brain is frantically looking for the quickest thing you can do to make the problem go away. You’re not considering the whole matrix of solutions and consequences, of costs and benefits. It’s like craving for junk food: you’re hungry right now, and you promise yourself you’ll diet later to compensate. But of course, you’re not <a href="https://verraes.net/2013/07/managed-technical-debt/">managing the technical debt you’re creating</a>, because you don’t have time for that either.</p>
<p>In other words, <strong>time pressure is the source of much of our legacy</strong>. And boy, do we have legacy in industry! Following from my introduction, estimates often cause or imply time pressure. The interest that builds up on unmanaged technical debt, is accidental complexity: models that do not match the domain, incomprehensible user interfaces, lack of tests, … That, in turn, is costly: a complex system is harder to learn, harder to work with, harder to change. <strong>When the business depends on software that is hard to change, the business itself becomes hard to change.</strong> Before you know it, the next startup comes along, faster, smarter, more agile, and wipes your organization straight from the face of the Earth.</p>
<h2 id="why-estimates-are-always-wrong">Why Estimates Are Always Wrong</h2>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2014-06-17-cost-of-estimation/1200521270Hine_Icarus_575.jpg" alt="Risk" /></p>
<p>If we always estimate the shortest possible time to complete a task, then whenever we are wrong, we will fail upwards: the actual completion can only get longer for every possible bump in the road. There’s little to no happy accidents, causing the task to finish sooner.</p>
<p>Why is that? In software, solutions are highly reusable. You automatically take this into account when estimating. “We’ve already built something similar, we can reuse that.” But even though building the first part took longer than you thought, you still assume that this time, there will be no setbacks. We are blind to our own inability to estimate. We never seem to learn that there will always be bumps: from spending half a day looking for a bug, caused by a missing comma somewhere, to losing half a month waiting for the client to deliver some critical piece of information that they said “will be in your mailbox by the end of the day”. As if <em>they</em> somehow magically do know how to estimate!</p>
<p>In the interest of making the implicit explicit, let’s call this <strong>“risk”: the probability by which a time based estimate can be wrong</strong>.</p>
<h2 id="better-estimation">Better Estimation</h2>
<p>We’re getting somewhere now. There is a better way of estimating, by making the ingredients very visible. The formula for estimating in story points, is:</p>
<p><del></del></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>story points = time x complexity x risk
</code></pre></div></div>
<p></del></p>
<p>How this works in practice, and how to run estimation sessions, is the topic for future blog post.</p>
<p><strong>Update:</strong></p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>story points = (time, complexity, risk)
</code></pre></div></div>
<p>The first formula makes it look like something mathematically calculable. It isn’t.</p>
<div style="border: 1px dotted gray; padding: 10px; width: 50%; margin-bottom: 10px;">
<strong>On Estimation</strong>
<ol>
<li>
<a href="/2014/06/cost-of-estimation/">The Cost of Estimation</a>
</li>
<li>
<a href="/2014/08/sterile-estimation/">Sterile Estimation</a>
</li>
</ol>
</div>
When to Use Static Methods2014-06-13T00:00:00+00:00https://verraes.net/2014/06/when-to-use-static-methods-in-php<p>Some of the reactions to my last blog post on <a href="/2014/06/named-constructors-in-php/">Named Constructors in PHP</a>, originate from the notion that static methods are inherently bad and should never be used. This is rather overgeneralized.</p>
<p>Static methods are nothing more than namespaced global functions. Namespacing, I think we can all agree on, is great. As for global functions: We use those all the time. The native functions in PHP form our basic building blocks.</p>
<p>The problem to consider is shared global state. The example that I gave in my previous post, is referentially transparent:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$time</span> <span class="o">=</span> <span class="nc">Time</span><span class="o">::</span><span class="nf">from</span><span class="p">(</span><span class="s2">"11:45"</span><span class="p">);</span>
</code></pre></div></div>
<p>In other words, it is stateless, it is free of side effects, and as such, it is entirely predictable. You can call the exact same function with the exact same argument as often as you like, and you will always get the exact same result back (a Time instance with value 11:45), no matter the history of the system, no matter the context from which you call it.</p>
<p>Another example:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$sum</span> <span class="o">=</span> <span class="nc">Calculator</span><span class="o">::</span><span class="nf">sum</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span>
</code></pre></div></div>
<p>Again, the outcome is predictable. <code class="language-plaintext highlighter-rouge">Calculator::sum($x, $y)</code> provides a service that is stateless, that doesn’t remember anything, and whose outcome can’t be influenced by anything other than the arguments you put in there. Furthermore, this service will never have polymorphic, or have different implementations. Returning anything other 3 would break the contract. Of course, you could come up with a more efficient algorithm for adding to numbers, but that would have no effect on any of the clients of Calculator.</p>
<p>For a counterexample, let’s look at a stateful service:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nc">Counter</span><span class="o">::</span><span class="nf">increment</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="nv">$count</span> <span class="o">=</span> <span class="nc">Counter</span><span class="o">::</span><span class="nf">getCount</span><span class="p">();</span>
</code></pre></div></div>
<p>This is of course a simple example, but in more complex situations, it can be rather opaque to understand this statefullness. Imagine that one developer use the Counter in one part of the code, and another developer uses it in another part. They both test their code in isolation, and it works fine. As soon as the parts are integrated, the count becomes erratic, because they both share the same global state, instead of each owning a separate Counter. The solution is to have objects here:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$myCounter</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Counter</span><span class="p">;</span>
<span class="nv">$myCounter</span><span class="o">-></span><span class="nf">increment</span><span class="p">(</span><span class="mi">1</span><span class="p">);</span>
<span class="nv">$count</span> <span class="o">=</span> <span class="nv">$myCounter</span><span class="o">-></span><span class="nf">getCount</span><span class="p">();</span>
</code></pre></div></div>
<h2 id="abstractions">Abstractions</h2>
<p>You might still feel resistance against something like <code class="language-plaintext highlighter-rouge">Calculator::sum($x, $y)</code>, because it can not be extended or mocked. Keep in mind however that we are at the lowest level of abstraction here. You can’t extend or mock the + operator either in PHP. Yet you’ve never felt the need for that. If you do need higher levels of abstraction, then composition is your friend. OOP composition is well known, but let’s look at FP really quickly. There is, in PHP, an interesting distinction between <code class="language-plaintext highlighter-rouge">+</code> and <code class="language-plaintext highlighter-rouge">Calculator::sum($x, $y)</code>: the former can not be injected, the latter can:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$result</span> <span class="o">=</span> <span class="nb">array_reduce</span><span class="p">([</span><span class="mi">1</span><span class="p">,</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">],</span> <span class="s1">'Calculator::sum'</span><span class="p">,</span> <span class="mi">0</span><span class="p">);</span>
<span class="c1">// $result = 6</span>
</code></pre></div></div>
<p>I’m getting in the realm of higher-order programming now, which is rather out of scope, but let’s just agree that functions and static methods can be very useful, with the right mindset.</p>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="/2014/06/named-constructors-in-php/">Named Constructors in PHP</a></li>
</ul>
Named Constructors in PHP2014-06-12T00:00:00+00:00https://verraes.net/2014/06/named-constructors-in-php<p>PHP allows only a single constructor per class. That’s rather annoying. We’ll probably never have proper constructor overloading in PHP, but we can at least enjoy some of the benefits. Let’s take a simple Time value object. Which is the best way of instantiating it?</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="s2">"11:45"</span><span class="p">);</span>
<span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="mi">11</span><span class="p">,</span> <span class="mi">45</span><span class="p">);</span>
</code></pre></div></div>
<p>The only correct answer is “it depends”. Both are correct from the point of view of the domain. Supporting both is an option:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$timeOrHours</span><span class="p">,</span> <span class="nv">$minutes</span> <span class="o">=</span> <span class="kc">null</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="nb">is_string</span><span class="p">(</span><span class="nv">$timeOrHours</span><span class="p">)</span> <span class="o">&&</span> <span class="nb">is_null</span><span class="p">(</span><span class="nv">$minutes</span><span class="p">))</span> <span class="p">{</span>
<span class="k">list</span><span class="p">(</span><span class="nv">$this</span><span class="o">-></span><span class="n">hours</span><span class="p">,</span> <span class="nv">$this</span><span class="o">-></span><span class="n">minutes</span><span class="p">)</span> <span class="o">=</span> <span class="nb">explode</span><span class="p">(</span><span class="nv">$timeOrHours</span><span class="p">,</span> <span class="s1">':'</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span>
<span class="p">}</span> <span class="k">else</span> <span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">hours</span> <span class="o">=</span> <span class="nv">$timeOrHours</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">minutes</span> <span class="o">=</span> <span class="nv">$minutes</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is terribly ugly. It makes using the Time class rather confusing. And what happens if we need to add more ways to instantiate Time?</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$minutesSinceMidnight</span> <span class="o">=</span> <span class="mi">705</span><span class="p">;</span>
<span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$minutesSinceMidnight</span><span class="p">);</span>
</code></pre></div></div>
<p>Or if we want to support numeric strings as well as integers?</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="s2">"11"</span><span class="p">,</span> <span class="s2">"45"</span><span class="p">);</span>
</code></pre></div></div>
<p>(Note: in production code, I would make my Time class a lot more idiot-proof.)</p>
<h2 id="refactor-to-named-constructors">Refactor to named constructors</h2>
<p>Let’s add some public static methods to instantiate Time. This will allow us to get rid of the conditionals (which is always a good thing!).</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">hours</span> <span class="o">=</span> <span class="p">(</span><span class="n">int</span><span class="p">)</span> <span class="nv">$hours</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">minutes</span> <span class="o">=</span> <span class="p">(</span><span class="n">int</span><span class="p">)</span> <span class="nv">$minutes</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromString</span><span class="p">(</span><span class="nv">$time</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">list</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span> <span class="o">=</span> <span class="nb">explode</span><span class="p">(</span><span class="nv">$time</span><span class="p">,</span> <span class="s1">':'</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span>
<span class="k">return</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromMinutesSinceMidnight</span><span class="p">(</span><span class="nv">$minutesSinceMidnight</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$hours</span> <span class="o">=</span> <span class="nb">floor</span><span class="p">(</span><span class="nv">$minutesSinceMidnight</span> <span class="o">/</span> <span class="mi">60</span><span class="p">);</span>
<span class="nv">$minutes</span> <span class="o">=</span> <span class="nv">$minutesSinceMidnight</span> <span class="o">%</span> <span class="mi">60</span><span class="p">;</span>
<span class="k">return</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Every method now satisfies the Single Responsibility Principle. The public interface is clear and understandable interface, and the implementations are straightforward. Are we done?</p>
<p>Well, something is bothering me: <code class="language-plaintext highlighter-rouge">__construct($hours, $minutes)</code> kinda sucks: it exposes the internals of the Time value object, and we can’t change the interface because it is public. Imagine that for some reason, we want Time to store the string representation and not the individual values.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$time</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">time</span> <span class="o">=</span> <span class="s2">"</span><span class="nv">$hours</span><span class="s2">:</span><span class="nv">$minutes</span><span class="s2">"</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromString</span><span class="p">(</span><span class="nv">$time</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">list</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span> <span class="o">=</span> <span class="nb">explode</span><span class="p">(</span><span class="nv">$time</span><span class="p">,</span> <span class="s1">':'</span><span class="p">,</span> <span class="mi">2</span><span class="p">);</span>
<span class="k">return</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// ...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is ugly: we go through all the trouble of splitting up the string, only to rebuild it in the constructor.</p>
<p>Do we even need a constructor now that we have named constructors? Of course not! They are just an implementation detail, that we want to encapsulate behind meaningful interfaces. So we make it private:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">;</span>
<span class="k">private</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">hours</span> <span class="o">=</span> <span class="p">(</span><span class="n">int</span><span class="p">)</span> <span class="nv">$hours</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">minutes</span> <span class="o">=</span> <span class="p">(</span><span class="n">int</span><span class="p">)</span> <span class="nv">$minutes</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromValues</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span>
<span class="p">}</span>
<span class="c1">// ...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Now that the constructor is no longer public, we can choose to refactor all the internals of Time as much as we want. For example, sometimes you’ll want every named constructor to assign properties without passing them through a constructor:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Time</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">;</span>
<span class="c1">// We don't remove the empty constructor because it still needs to be private</span>
<span class="k">private</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(){}</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">fromValues</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$time</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Time</span><span class="p">;</span>
<span class="nv">$time</span><span class="o">-></span><span class="n">hours</span> <span class="o">=</span> <span class="nv">$hours</span><span class="p">;</span>
<span class="nv">$time</span><span class="o">-></span><span class="n">minutes</span> <span class="o">=</span> <span class="nv">$minutes</span><span class="p">;</span>
<span class="k">return</span> <span class="nv">$time</span><span class="p">;</span>
<span class="p">}</span>
<span class="c1">// ...</span>
<span class="p">}</span>
</code></pre></div></div>
<h2 id="ubiquitous-language">Ubiquitous Language</h2>
<p>Our code begins to clean up nicely, and our Time class now has some very useful ways of being instantiated. As it happens with better design, other, previously hidden design flaws, start to become visible. Look at the interface for Time:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$time</span> <span class="o">=</span> <span class="nc">Time</span><span class="o">::</span><span class="nf">fromValues</span><span class="p">(</span><span class="nv">$hours</span><span class="p">,</span> <span class="nv">$minutes</span><span class="p">);</span>
<span class="nv">$time</span> <span class="o">=</span> <span class="nc">Time</span><span class="o">::</span><span class="nf">fromString</span><span class="p">(</span><span class="nv">$time</span><span class="p">);</span>
<span class="nv">$time</span> <span class="o">=</span> <span class="nc">Time</span><span class="o">::</span><span class="nf">fromMinutesSinceMidnight</span><span class="p">(</span><span class="nv">$minutesSinceMidnight</span><span class="p">);</span>
</code></pre></div></div>
<p>Notice anything? We’re mixing no less than three languages:</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">fromString</code> is a PHP implementation detail;</li>
<li><code class="language-plaintext highlighter-rouge">fromValues</code> is a sort of generic programming term;</li>
<li>and <code class="language-plaintext highlighter-rouge">fromMinutesSinceMidnight</code> is part of the domain language.</li>
</ul>
<p>Being a language geek and Domain-Driven Design aficionado, I can’t let this pass. As Time is part of our domain, my preferred style is to find inspiration in the Ubiquitous Language.</p>
<ul>
<li><code class="language-plaintext highlighter-rouge">fromString</code> => <code class="language-plaintext highlighter-rouge">fromTime</code></li>
<li><code class="language-plaintext highlighter-rouge">fromValues</code> => <code class="language-plaintext highlighter-rouge">fromHoursAndMinutes</code></li>
</ul>
<p>(If you worry about the extra characters you’d need to type, get an editor with contextual code completion.)</p>
<p>This focus on the domain, gives you some great options:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$customer</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Customer</span><span class="p">(</span><span class="nv">$name</span><span class="p">);</span>
<span class="c1">// We can't "new a customer" or "instantiate a customer" in real life.</span>
<span class="c1">// Better:</span>
<span class="nv">$customer</span> <span class="o">=</span> <span class="nc">Customer</span><span class="o">::</span><span class="nf">fromRegistration</span><span class="p">(</span><span class="nv">$name</span><span class="p">);</span>
<span class="nv">$customer</span> <span class="o">=</span> <span class="nc">Customer</span><span class="o">::</span><span class="nf">fromImport</span><span class="p">(</span><span class="nv">$name</span><span class="p">);</span>
</code></pre></div></div>
<p>Granted, that’s not always better. In the case of Time, I might stick to fromString, because maybe at this level of detail in our code, we want to serve the programmer more than the domain. I might even provide both options. But at least, thanks to named constructors, we now <em>have</em> options.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/06/when-to-use-static-methods-in-php/">When to Use Static Methods</a></li>
<li><a href="/2011/03/accessing-private-properties-from-other-instances/">Accessing private properties from other instances</a></li>
<li><a href="/2013/02/2013-02-16-casting-value_objects/">Casting Value Objects to Strings</a></li>
<li><a href="/2014/05/final-classes-in-php/">Final Classes: Open for Extension, Closed for Inheritance</a></li>
</ul>
Why Domain-Driven Design Matters2014-05-21T00:00:00+00:00https://verraes.net/2014/05/why-domain-driven-design-matters<script async="" class="speakerdeck-embed" data-id="318140a0c29a013152315aee9e57b660" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<p>See the slides at <a href="https://speakerdeck.com/mathiasverraes/why-domain-driven-design-matters">Speakerdeck</a></p>
<h2 id="abstract">Abstract</h2>
<p>In the software industry, the life expectancy of ideas, methodologies, and technologies, is extremely short. And yet, after ten years, Domain-Driven Design is still growing bigger. From it’s original roots in OOP, it has now expanded into Functional Programming, Reactive Programming and Event Sourcing, and architectural styles such as Hexagonal and CQRS. Clearly something about Domain-Driven Design makes it such an appealing choice to build systems for complex domains.</p>
<p>In this session, we’ll discuss what DDD is: from design patterns and modelling techniques, to the more philosophical ideas about how we deal with complexity. We explore why it has made such a profound impact, and how to decide whether it’s right for your project. We’ll have lots of room for open discussion, to make sure all your questions are answered.</p>
<h2 id="presentations">Presentations</h2>
<ul>
<li><a href="http://akamon.com/">Akamon</a> in Barcelona, Spain May 20, 2014</li>
</ul>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="https://verraes.net/2013/08/john-gall-systemantics-the-systems-bible/">The Systems Bible</a></li>
</ul>
A Functional Foundation for CQRS/ES2014-05-14T00:00:00+00:00https://verraes.net/2014/05/functional-foundation-for-cqrs-event-sourcing<p>I’ve been dabbling with Functional Programming languages for a while now. Although I can’t say I’m at all proficient, it has affected my programming style. Even if my code is mostly OOP, my whiteboard drawings look more functional. I’m <a href="/2014/01/domain-driven-design-is-linguistic/">thinking about functions a lot</a>. So it made sense to describe the CQRS-architecture of my system as a set of functions.</p>
<p>Greg Young describes Command Query Responsibility Segregation as “using two objects where previously there was only one”. The idea is as ingenious as it is simple, but I feel the quote doesn’t do it justice. Rinat Abdullin proposed in a <a href="https://twitter.com/abdullin/status/465747026953908225">Twitter conversation</a> with Eric Evans, the idea that CQRS enables you to denormalize your model. The value lies in the fact that, because you have independent read and write models, you can reason more clearly about your model. The separation opens up opportunities to capture deeper insights. You can now optimise the two sides separately.</p>
<p>Even though you can use CQRS without Event Sourcing, this post assumes Domain Events all the way down. In a brainstorming session with <a href="https://twitter.com/stijnvnh">@stijnvnh</a>, we identified four (conceptual) functions.</p>
<h2 id="protection">Protection</h2>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>f(history, command) -> events
(where history is the series of past events for a single Aggregate instance)
</code></pre></div></div>
<p>Commands and Domain Events are first class citizens of our model. Aggregates no longer expose state, but only events. Aggregates are at the core of the decision making process. They decide what the outcome of a Command is, and express that outcome with Domain Events. The history of that particular Aggregate instance, influences the decision, so we must inject it. If you want to split the reconstitution of the Aggregate instance from the handling of the command, you can use currying. This helps to keep infrastructural and model concerns separated:</p>
<p><code class="language-plaintext highlighter-rouge">f(history)(command) -> events</code></p>
<p>‘Protection’ refers to the primary goal of the Aggregates: protecting its business rules, making sure that the user can not cause the system to go in an inconsistent or illegal state. When a Command violates the business rules, the operation should fail:</p>
<p><code class="language-plaintext highlighter-rouge">f(history, command) -> either<exception|events></code></p>
<p>(The error handling part of FP is still a bit terra incognita for me though.)</p>
<p>The exact same idea, can be expressed in BDD-style scenario’s: Given this history, when this command is received, then these events should happen.</p>
<h2 id="interpretation">Interpretation</h2>
<p><code class="language-plaintext highlighter-rouge">f(events) -> state</code></p>
<p>As the Aggregates do not expose, something else must. The function above is a Projector. It takes (a stream of) events, and projects these into something that represents a subset of the perceived state of our system. That state makes up our read models for further consumption. I like the idea of “interpretation” here: Two different Projectors could have a completely different interpretation of what the meaning of a set of behaviours is. This is similar to how a journalist or politician interprets a series of events, and comes up with a different conclusion, or at least a different focus, than her colleagues.</p>
<p>Of course these interpretations are based in the domain. The beauty of the decoupling from the write model, is that when the domain evolves, or we gain deeper insight, it’s now easy to shift the interpretation. Make a new projector and a new read model, and replay the history of the system into them.</p>
<p>You can express the projection as a left fold:</p>
<p><code class="language-plaintext highlighter-rouge">f(events) -> foldLeft(apply, events, init)</code></p>
<h2 id="intention">Intention</h2>
<p><code class="language-plaintext highlighter-rouge">f(state) -> command</code></p>
<p>One of the major consumers of the read models, is usually an end user. Many applications need to somehow report state (projected in the previous section). The user looks at the state, and makes decisions. The user then makes his intention known to domain model, by sending commands. And it is an intention indeed, because, as we saw earlier, the Aggregate might refuse the command later.</p>
<p>Note that we don’t present the user with the list of events, (unless that temporality is important). Humans are pretty bad at deducing state from events – so never base your opinion on a single journalist or politician! In the case of our application, we give the user ready-made, automated interpretations.</p>
<p>It’s the only function presented here that is not pure. Human users are influenced by lots of global state, they have ideas, principles, mood swings, and <a href="/2014/01/domain-driven-design-is-linguistic/">biases</a>. It’s like they have a mind of their own! Modelling the end user is pretty much impossible…</p>
<h2 id="automation">Automation</h2>
<p><code class="language-plaintext highlighter-rouge">f(events) -> command</code></p>
<p>… but sometimes it is! Humans in organisations behave predictably in certain circumstances. Many tasks in an organisation are repeatable: “Whenever X and Y have happened, Z needs to happen.” Automating these business processes, making them faster and safer, is one of the appeals of enterprise software. Process Managers take on this job: they are event listeners, that react to certain events or combinations of events. They may hold intermediate state, so internally, they look like projectors. But the state is invisible. The only observable output are new commands.</p>
<p>In some cases, a Process Manager can output events straight away. This makes sense when there are no invariants other than what the Process Manager can take care of on its own. But most of the time, you’ll want a proper Command that can be refused by an Aggregate.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-05-14-functional-foundation-for-cqrs/circle-diagram-small.png" alt="Functional Foundation for CQRS" /></p>
<h2 id="conclusion">Conclusion</h2>
<p>What is the point of all of this? Thinking about <a href="https://verraes.net/2013/08/john-gall-systemantics-the-systems-bible/">how systems work</a>, is one my hobbies. Having a good theoretical foundation, has often helped me when designing systems. I may not end up doing things according to the theory, but the theory forces me to think hard about what I’m doing.</p>
<p>Pure functions are deterministic for the same inputs and outputs. Independent of shared mutable state, a pure function can isolated in a separate process, on a separate server, doin’ his thang. See where this is going?</p>
<h2 id="update-june-8-2014">Update June 8, 2014</h2>
<p>I wrote that user intention can not be represented as a pure function, but of course you simply need to pass in the state of the human, like this:</p>
<p><code class="language-plaintext highlighter-rouge">f(brain, state) -> command</code></p>
<h2 id="update-dec-17-2016">Update Dec 17, 2016</h2>
<p>My model completely lacked Queries. Having state without being able to query it is rather useless. Like Events and Commands, a <a href="/2015/01/messaging-flavours/">Query is just a message</a>. It fits in the Query Model (duh) aka Read Model like this: <code class="language-plaintext highlighter-rouge">f(history, query) -> state</code> where state is a subset of the all the state that the projector writes. In CQRS, I think ideally, a projector should have <strong>private state</strong>, meaning that the projector is the only one that read and write its own (logical) private database. All clientss get state by sending one of the projector’s supported Query messages. Another type of query is a Subscription, as sometimes a client is interested in a stream of (filtered) events as opposed to state.</p>
<p>So this represents my current thinking here:</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">handle</span> <span class="o">::</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="o">-></span> <span class="kt">Command</span> <span class="o">-></span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span>
<span class="n">subscribe</span> <span class="o">::</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="o">-></span> <span class="kt">SubscriptionQuery</span> <span class="o">-></span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span>
<span class="n">process</span> <span class="o">::</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="o">-></span> <span class="p">[</span><span class="kt">Command</span><span class="o">|</span><span class="kt">Event</span><span class="p">]</span>
<span class="n">query</span> <span class="o">::</span> <span class="p">[</span><span class="kt">Event</span><span class="p">]</span> <span class="o">-></span> <span class="kt">Query</span> <span class="o">-></span> <span class="kt">State</span>
</code></pre></div></div>
<p>or with intermediate states made explicit:</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">project</span> <span class="o">::</span> <span class="kt">State</span> <span class="o">-></span> <span class="kt">Event</span> <span class="o">-></span> <span class="kt">State</span>
<span class="n">query'</span> <span class="o">::</span> <span class="kt">State</span> <span class="o">-></span> <span class="kt">Query</span> <span class="o">-></span> <span class="kt">State</span>
</code></pre></div></div>
<p>And modelling the human:</p>
<div class="language-haskell highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">learn</span> <span class="o">::</span> <span class="kt">State</span> <span class="o">-></span> <span class="kt">Brain</span> <span class="o">-></span> <span class="kt">Query</span>
<span class="n">decide</span> <span class="o">::</span> <span class="kt">State</span> <span class="o">-></span> <span class="kt">Brain</span> <span class="o">-></span> <span class="p">[</span><span class="kt">Command</span><span class="p">]</span>
</code></pre></div></div>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="/2014/11/domain-events/">Domain Events</a></li>
<li>If you like the circular representation of CQRS, I have a lot more of them in my slides for <a href="/2013/12/fighting-bottlenecks-with-cqrs/">Fighting Bottlenecks with CQRS</a></li>
<li><a href="/2014/01/domain-driven-design-is-linguistic/">Domain-Driven Design is Linguistic</a></li>
<li><a href="/2015/01/messaging-flavours/">Messaging Flavours</a></li>
</ul>
Final Classes: Open for Extension, Closed for Inheritance2014-05-12T00:00:00+00:00https://verraes.net/2014/05/final-classes-in-php<p>I make all my classes final by default. I even configured the templates in my IDE prefix new classes with ‘final’. I’ve often explained my reasoning to people. A blog post is in order!</p>
<h2 id="clarity-of-intent">Clarity of Intent</h2>
<p>A guiding principle here is Clarity of Intent. Making a machine do what we want is not so hard. The reason we need clean code, is not for the compiler. It’s to help our fellow developers, third parties, and even ourselves in six months time, understand the purpose and the design of our system. Many design principles help us with that: picking good names for our elements, create low coupling and high cohesion, SOLID principles, …</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-05-12-final-classes-php/explicit-interface-small.png" alt="Explicit escalator interface" /></p>
<p>Code is often part of something bigger. This can get really complex, really quickly. The constraints we impose, help us to limit the cognitive load for developers working with our code. We encapsulate the details, and expose only what outside code needs to talk to us.</p>
<p>Another such constraint is the Open/Closed Principle. It states that software entities should be open for extension, but closed for modification.
Closed for modification, in this context, means that when your code exposes some behavior to the outside world, that interface should be stable. Providing an API is a responsibility: by allowing other code to access features of your system, you need to give certain guarantees of stability or low change frequency. The behaviour should be deterministic. It does not mean your implementation can not change. The changes should not affect outside code.</p>
<p>It’s important to understand that “Open for extension”, does not mean “Open for inheritance”. Composition, strategies, callbacks, plugins, event listeners, … are all valid ways to extend without inheritance. And usually, they are much preferred to inheritance – hence the adage “favour composition over inheritance”. The latter creates more coupling, that can be hard to get rid of, and that can make understanding the code quite tough.</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2014-05-12-final-classes-php/not_how_to_extend_a_car.jpg" alt="Not how to extend a car" /></p>
<p>Extension points are part of your public API. By making a class inheritable, you are saying to the outside world: this class is meant to be extended. In some cases that may be the best option, but it is not the only option. Declaring explicitly what the extension points are, is part of the contract your code has with the rest of the system. Final classes help to enforce this contract.</p>
<p>The fewer behaviours and extension points you expose, the more freedom you have to change system internals. This is the idea behind encapsulation.</p>
<h2 id="php">PHP</h2>
<p>Making a class final has some drawbacks. For example, you cannot mock it anymore with PHPUnit. This is a limitation of the infrastructure. One option is to remove the final keyword, but add an @final annotation. We loose the strict enforcement, but we still communicate intent. The class is still conceptually closed down for inheritance.</p>
<p>Similarly, we can mark elements with @internal or @api. @internal indicates to the outside world that, even though you can technically access the element, you’re not supposed to use or consume it outside of the boundaries of the module or library. @api means it is part of the public contract and can be trusted not to break easily.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Still in doubt? Consider this: when a class is closed for inheritance, or a method is private, or an element is @internal, you are free to change it or remove it. This freedom removes a lot of the burden of refactoring. Should a valid reason come up to open the class or method, it will be easy to do so, because nothing depends on it being closed. On the other hand, if you start by making everything open or inheritable, it will be very hard to close it later. Somewhere, perhaps invisible to you, something might depend on it. Closing it, or even making a small change, will break that code.</p>
<p>Finally, I have a hypothesis. If, like some other languages, classes in php were final by default, and needed an “inheritable” keyword to open them up, many people who are now opposed to the final keyword, would have no problem with it at all.</p>
<p><strong>Update June 24, 2014</strong> From <a href="http://programmers.stackexchange.com/a/92771/29370">Programmers Exchange</a></p>
<blockquote>"(...) designing a class that can be meaningfully inherited from takes more than just removing a final specifier; it takes a lot of care." — Konrad Rudolph </blockquote>
Small Uncontrolled Experiments2014-03-22T00:00:00+00:00https://verraes.net/2014/03/small-controlled-experiments<p><strong>Update 2021-01-25: I used to call this “small controlled experiments”. It was a name that grew organically and it stuck. Nowadays, I believe they’re very much uncontrolled: a team is too small and a sprint is too short for anything remotely scientific. That doesn’t mean they’re not valuable. Emotions are data, and if people’s gut feeling after (after, not before!) doing an experiment doesn’t agree, the proposed change will not work well anyway. Impose time constraints on these experiments, but do allow them to be uncontrolled.</strong></p>
<p>You’re in a meeting. Maybe it’s a daily stand-up meeting, or an agile retrospective. Somebody has an idea. A heated debate follows. Everybody is stating opinions, trying to convince the others, explaining why their idea is going to work, and the alternative is stupid. Because everybody is focused on making their point, nobody is actually listening to the others. Eventually the decision is either postponed, or the person with the highest salary makes the final decision, or the one who pushes the hardest. The decision is final, and it may or may not be actually get implemented. And if it is, it is never properly evaluated, or adapted.</p>
<p>I humbly admit to being guilty of having done all of these. And it’s all such a terrible waste.</p>
<p>The cure is to set up <strong>small uncontrolled experiments</strong>.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-03-22-small-controlled-experiments/chemistry-show.jpg" alt="experiment" /></p>
<h2 id="continuous-improvement">Continuous Improvement</h2>
<p>I used to think that simply trying to do a good job, was sufficient to incrementally improve the way you do things in your organisation, your process, your culture. Having worked closely with <a href="http://twitter.com/VincentVanderh">@VincentVanderh</a> for a year, I’ve learned to appreciate that, even when you think you’re doing just fine and you work with the greatest people, a relentless focus on making small improvements every day, is going to pay off massively.</p>
<p>The basic principle is to identify problems, find the simplest thing you could try to improve it, try it for a short time, and evaluate. After that, you either implement the change permanently, dismiss it, or try a new cycle. The idea is of course not new. Deming calls it the <em>PDCA-cycle (Plan-Do-Check-Act)</em>. The Japanese, spearheaded by Toyota, have made <em>Kaizen</em>, meaning “change for the best”, a central part of business culture.</p>
<p>Our first attempts were clumsy at best. We had improved a lot since the start of the project, so we felt we didn’t need a formal process. We didn’t identify major problems, let alone major solutions. We were ready to dismiss the whole idea of PDCA as just another management buzzword.</p>
<p>We were partly right. The process was too formal for our context, and the last thing you want to do is burden knowledge workers in a tight schedule with more processes to worry about. More importantly, we were trying too hard: we focused on finding major, critical problems, and grand, all-encompassing solutions. And that only works in the movies. We did keep trying however. We came up with experiments, wrote them on a sticky, put them in a corner of the whiteboard, and started doing them. Well, sometimes. More often than not, the stickies hung there for a while, until someone would feel they were pointless, and they were thrown out. Some ideas did get implemented – after all we had been improving somewhat steadily before we had a name for it – but it was all rather vague and unpredictable.</p>
<h2 id="make-it-lightweight">Make it lightweight</h2>
<p>Some changes had to be made. The biweekly retrospectives made the improvement process too slow. By this time, we had pretty much gotten rid of Scrum(ish) sprints, in favour of a more continuous Kanban-flow. Saving up all the improvement ideas and discussion for the retrospective, started feeling like a remnant from the biweekly sprint cadence, something that did no longer fit in our flow. So we needed the benefits of evaluating our process, without the slowness of retrospectives.</p>
<p>We changed the rules of the experiments:</p>
<ul>
<li>
<p>You can write down any idea for an experiment on a sticky and put it in the drop zone of the Kanban board.</p>
</li>
<li>
<p>It will be discussed at the stand-up meeting, the following morning.</p>
</li>
<li>
<p>The discussion is kept as short as possible. You’re not asking if it will work – the point is that the experiment will prove if the idea works. You’re answering the questions: “Will we try this? For how long? Can we do it in a cheaper, lower effort way, that doesn’t affect our velocity?”</p>
</li>
<li>
<p>The sticky is posted on a timeline, usually about 10-14 days in the future.</p>
</li>
<li>
<p>Every day during stand-up, we check whichever stickies are on today’s slot on the timeline.</p>
</li>
<li>
<p>The experiment is evaluated, we discuss positive or negative side-effects, and decide on making the change permanent or trying more experiments.</p>
</li>
</ul>
<p>A number of things turned out to be critical here:</p>
<ol>
<li>
<p>It works best if the experiment takes as little effort as possible. Most of them do not take any time at all, as they only made a small change to something we were already doing. Others take a minute a day, or even a minute a week. The <strong>“small”</strong> in “small uncontrolled experiment”.</p>
</li>
<li>
<p>Sometimes one person is assigned to an experiment, but usually it’s <strong>for the whole team</strong>.</p>
</li>
<li>
<p>When starting an experiment, make sure everybody understands that the experiment is <strong>limited in time</strong>, and that they <strong>will be heard</strong> and allowed veto when evaluating it. This is especially important with more <strong>introvert team members</strong>, especially if you have one or two loudmouths like yours truly.</p>
</li>
<li>
<p>Make sure the experiments are <strong>for everybody</strong>. I tended to focus on improving conditions for the developers, but of course, testers, analyst, the project manager, … should all benefit.</p>
</li>
<li>
<p>Deciding, and <strong>visualising exactly when the evaluation will happen</strong>, is probably the best thing we did. It gives a sense of urgency and deadline to the experiment, preventing it from slowly fading into oblivion.</p>
</li>
<li>
<p>We’d usually have a couple of experiments running <strong>simultaneously</strong> at any given time.</p>
</li>
</ol>
<p>Here’s a sample of some of the things we experimented with, to give you some ideas. Some happened before we tried PDCA or Kaizen, before we settled on the small uncontrolled experiments above, but making it a core part of our standup has greatly increased the rate of improvement.</p>
<h2 id="hacking-our-productivity">Hacking our productivity</h2>
<ul>
<li>
<p>When a story was estimated as a big one, say an 8-pointer, more time would be spent on it to refactor it. When it was a 3-pointer, fewer time was spent. I’m all for <a href="/2013/09/extract-till-you-drop/">refactoring</a>, but not everything requires the <a href="/2013/07/managed-technical-debt/">same level of perfection</a>. We wanted the quality of a story to depend on value, not on the original estimate. In the experiment, we moved the estimated number from the front of the sticky, to the back. Because of this, the developers could no longer see the estimation, so they were no longer influenced by it. We made the experiment permanent.</p>
</li>
<li>
<p>We noticed how at the end of a two-week sprint, the pace would speed up to reach the sprint target, but as soon as it was reached, the tempo slowed down. The fact that the last day was a Friday, didn’t help either. Kanban came to rescue here: We stopped setting sprint targets, set strict work-in-progress limits, and made sure we had a pull-based system. (More on Kanban in a future post.)</p>
</li>
<li>
<p>The testers work for multiple teams. As our delivery of stories was very irregular, they couldn’t predict when they would need time to test our stories. This caused stories to be blocked in testing, creating too much of a gap between development and testing. We experimented with the commitment to deliver exactly one story a day. This worked great for predictability for the testers, but the unwanted side-effects were that stories were being prioritized to reach this goal, along with other subtle ways to cheat the system. The experiment was stopped.</p>
</li>
</ul>
<h2 id="technical-debt">Technical Debt</h2>
<ul>
<li>
<p>The <a href="https://verraes.net/2013/07/managed-technical-debt/">Wall of Technical Debt</a> started as an experiment. Cost: a bunch of stickies, and a few minutes. Gain: Having a complete map of everything in the system that needs improving, how to do it, and how important it is.</p>
</li>
<li>
<p>To reduce technical debt, we’ve tried different approaches. We’re now on a mix of:</p>
<ul>
<li>having a limit of one technical debt ticket in progress at any time,</li>
<li>yet being allowed to pull a technical ticket, if we judge it would be really beneficial to the functional story that we’re working on.</li>
</ul>
</li>
</ul>
<h2 id="visualisation">Visualisation</h2>
<ul>
<li>The Kanban board is regularly redrawn as we get more insight in our process. I think we are in the seventh iteration by now. I think it is beautiful :-) Many of the changes may seem minimal at first, but the accumulated effect is that this board is pure bliss to work with. One example is that the “expedite” swimming lane moved from an afterthought at the bottom of the board, to a prominent place at the top.</li>
</ul>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-03-22-small-controlled-experiments/kanbanboard-small.jpg" alt="Kanban board" /></p>
<ul>
<li>
<p>We fooled around with the idea from Core Protocols to make our emotions explicit during standup: “I’m glad-mad-sad-afraid that …”. We had no idea what we expected of this experiment, but we felt it was worth trying (it is, once again, dead cheap to try for few weeks). Current status: undecided. The PM will research it better and we’ll try it again.</p>
</li>
<li>
<p>Hanging up a monitor to show build statuses, sending mails when builds fail, … nothing extraordinary there, as more and more teams do CI, but you need to make an effort to get it set up.</p>
</li>
<li>
<p>We’ve experimented with different ways of visualising our metrics. It turns out that the best graphs are the ones that are extremely simple, and show you a key metric in a glance, without requiring you to understand how to interpret it first.</p>
</li>
<li>
<p>The timeline that we used to visualise the small uncontrolled experiments, was itself a small uncontrolled experiment. It’s expanded into showing all important events, like people being on holiday, deadlines, …</p>
</li>
</ul>
<h2 id="process">Process</h2>
<ul>
<li>
<p>We made it dead easy for testers to deploy a pull request to testing environments, so they can do it whenever it suits them, without dependencies on the developers.</p>
</li>
<li>
<p>We have a process for testing a pull request in isolation, then merging it, and having a different person testing it as part of the whole application. We have lowest defect rate I’ve ever experienced on a project.</p>
</li>
<li>
<p>We spent time improving the test fixtures, so testers can have a good, realistic starting point, that is exactly the same for every test run.</p>
</li>
<li>
<p>It happens more and more often that we work remotely. We’ve had digital boards before, but they never really worked as well as the physical board. We’re now trying Trello. Undecided.</p>
</li>
<li>
<p>We moved from occasionally scheduled releases, to weekly releases, using feature branches. Deploying to production has gradually evolved into a non-event.</p>
</li>
<li>
<p>(Added March 29 2014) We start every story with pair programming. No matter how hard or easy you think it will be. After inital work has begun, you can decide to continue pairing, split the work, or finish it alone. The benefit here is of course that potential pitfalls are found earlier. There’s less need for code reviews. The pairing partner can take over the story later if needed. Even without full-blown pair programming, starting a story in pair is one of those little magic tricks that create huge gains in productiveness and collaboration.</p>
</li>
</ul>
<h2 id="analysis">Analysis</h2>
<ul>
<li>Stories evolved from vague, widely scoped features, to very specific, atomic descriptions of the smallest idea that provides business value. Because they are small, unnecessary ones can easily be identified and dropped. They can be tested and agreed upon in isolation. Different stories can be worked on in parallel. The predictability of the flow is a lot better. They come with a list of acceptance criteria that we copy into the Github pull request, so we can check off the ones we’ve implemented.</li>
</ul>
<h2 id="exposing-invisible-problems">Exposing invisible problems</h2>
<p>I could go on for a bit, but I hope I made the point clear. According to the Toyota Way, any process, no matter how good you think it is, has waste. The waste is usually not obvious, because if it was, it would have been improved already. Many experiments are not focused on solving a known problem, but on trying things to expose unknown problems. Make sure they are small, low cost, low risk. The benefit may be anywhere between zero and enormous. You won’t know in advance, so opinions have little value. You are the scientist of your own culture.</p>
<h2 id="optionality">Optionality</h2>
<p>As an aside:</p>
<p>Humans favour courses of action that have a well-known, limited set of potential gains, with an unknown, unlimited potential cost. We ignore this cost because it is invisible. In <strong>“Thinking, Fast and Slow”</strong>, Daniel Kahneman refers a lot to the brain’s habit of ignoring the unseen, and calls this “What you see is all there is”.</p>
<p>The opposite is a course of action that has a well-known, limited potential cost, with an unknown, unlimited potential gain. In “<a href="/2013/08/antifragile-nassim-nicholas-taleb/">Antifragile</a>”, borrowing from the finance world, Nicholas N. Taleb calls this “optionality”. Keep this in mind: many many very small experiments with a very limited cost, will eventually produce totally unexpected outcomes.</p>
<h2 id="read-more">Read More</h2>
<p><a href="/2014/10/small-controlled-experiments-revisited/">My talk on Small Uncontrolled Experiments</a></p>
Practical Event Sourcing2014-03-18T00:00:00+00:00https://verraes.net/2014/03/practical-event-sourcing<h3 id="video">Video</h3>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://player.vimeo.com/video/104095245" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture fullscreen" allowfullscreen=""></iframe>
</figure>
<h3 id="slides">Slides</h3>
<script async="" class="speakerdeck-embed" data-id="78d43d2090a401318cc322b59c6a013f" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<h3 id="abstract">Abstract</h3>
<p>Traditionally, we create structural models for our applications, and store the state of these models in our databases.</p>
<p>But there are alternatives: Event Sourcing is the idea that you can store all the domain events that affect an entity, and replay these events to restore the object’s state. This may sound counterintuitive, because of all the years we’ve spent building relational, denormalized database schemas. But it is in fact quite simple, elegant, and powerful.</p>
<p>In the past year, I’ve had the pleasure of building and shipping two event sourced systems. In this session, I will show practical code, to give you a feel of how you can build event sourced models using PHP.</p>
Religiously RESTful2014-03-03T00:00:00+00:00https://verraes.net/2014/03/religiously-restful<p>Should you always speak and write perfect Oxford English? Most people will agree that this would be impractical, and take a lot of effort. It depends on your conversation partner, or your audience, and, quite importantly, on the expected lifespan of the message. If you are among friends, in the pub, you will most likely speak your local dialect. If you write a text message to your best mate, same. If, on the other hand, you were the King, and you were addressing the nation on a matter of the gravest importance, you’d want it to be in the best English you have at your disposal. If you write a document that is likely to survive you, such as legislature, you’d want to make sure it is still readable in a hundred years. Your local vernacular wouldn’t provide that guarantee.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-03-03-religiously-restful/colin-firth-as-king-george-vi-in-the-king.jpeg" alt="Colin Firth as King George VI" /></p>
<p>REST is a protocol, a language. You can speak it perfectly, but this comes at a greater cost. You and your team need a deeper understanding of the protocol. There’s effort in figuring out the tools, making sure you follow the rules, and refactoring when you realised that you misinterpreted how the protocol should be implemented.</p>
<p>The truth is, not all APIs need that level of REST maturity. Often, all you are building is a quick and dirty interface to allow two systems to talk to each other. If you build an API, and the only client is your own web user interface, and perhaps an mobile application, then there will be very little benefit in spending all the effort on perfection. After all, next year, you might throw away the REST API and replace it with some new protocol, or perhaps even something like a message based integration. Only if you are building an API that you expect to be maintained for a very long time, with lots of clients all over the planet, a strict protocol will serve you.</p>
<p>REST is tool, and it solves the problem of standardisation and long time maintainability quite well. Consider carefully if you need to whole tool, or just an approximation. Use the time you gain to build a better language for the domain model at the heart of your system, because this is a lot more critical than the infrastructure around it. Speak the RESTish language you are comfortable with, and learn the official language when you need it.</p>
Buzzword-free Bounded Contexts2014-02-13T00:00:00+00:00https://verraes.net/2014/02/buzzword-free-bounded-contexts<p>Context is everywhere. The human brain interprets context immediately, without effort. Try this:</p>
<ul>
<li>A product of my company</li>
<li>A product of prime factors</li>
<li>A product of my imagination</li>
</ul>
<p>Note how easy it was for you to derive context. For machines, this is hard, which is one reason we feed them models with unambiguous names.</p>
<p>The difference is often even a lot subtler.</p>
<ul>
<li>A product in the sales department is a thing with a weight and dimensions, a description, a few pictures and some customer reviews.</li>
<li>For the inventory, it’s a box that takes space on a shelf and that you always have too many or too few of.</li>
<li>For the shipping department, it’s a package you need to deliver to a customer. It has a weight and dimensions, but this time they include the packaging.</li>
</ul>
<p>In each of these departments, people know the context perfectly well. They don’t bother finding a different name, because even though it’s all called product, there’s never any misunderstanding.</p>
<h2 id="a-model-within-a-context">A Model within a Context</h2>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-02-13-buzzword-free-bounded-contexts/m-and-ms-small.jpg" alt="Separate into Bounded Contexts" /></p>
<p>And then the software developers come along, and we put all these totally different things that are all called product, in the same database table and model. We’re supposed to be the smart guys, but we’ve been fooled by the fact that they share the same name. We’ve totally missed the context. We end up with a huge model, that tries to be all these things at once, for every department, despite the fact that these departments are quite independent in real life anyway.</p>
<p>Now imagine that instead of one team, building one application and one model, we have three teams, in three different companies, building three separate systems: Salesify, SaasInventory, and SpeedyShip. It’s up to the users to integrate them. Nobody is calling the other teams going “Hey, we should all share one database!”. That’s a recipe for disaster. They use different infrastructure, different storage, different models. What you would do, is agree on a narrow, strictly defined way to integrate the systems. You could use messaging, an API, import scripts, or dumping files on an FTP.</p>
<p>Whether the three teams will easily reach agreement on those integration contracts, is <a href="/2014/01/bandwidth-and-context-mapping/">a whole different discussion</a>. But apart from that, it’s great not to share your database, or any other infrastructure. It gives you freedom. There is no complication from trying to put all the things in one model.</p>
<p>So why do we make it so hard on ourselves when we are just one team? We can borrow the benefits of having multiple teams (clarity, clean separation), without the downside (communication overhead). That’s essentially what Bounded Contexts are about:</p>
<ul>
<li>Split up your system according to natural divisions in the domain, such as departments, userbases, lifecycles, goals, …</li>
<li>Make the contexts explicit. (“A Product in the context of Shipping is …”)</li>
<li>Make the boundaries explicit. (“Shipping uses database A, Inventory uses database B…”)</li>
<li>Keep the models consistent within a Bounded Context. Well, keeping your models consistent is a good idea anyway.</li>
</ul>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/01/bandwidth-and-context-mapping/">Bandwidth and Context Mapping</a></li>
<li><a href="/2015/04/context-mapping-life-expectancy/">Context Mapping: Life Expectancy</a></li>
</ul>
Domain-Driven Design: Bounded Contexts, Modelling2014-02-12T00:00:00+00:00https://verraes.net/2014/02/domain-driven-design-basics<p>Yesterday, we did a DDD Basics session with the <a href="http://domaindriven.be">Domain-Driven Design Belgium</a> community. We only covered Strategic DDD, and split up the work amongst the five founders of the group:</p>
<ul>
<li><a href="http://twitter.com/yreynhout">@yreynhout</a> (Defining Domain-Driven Design),</li>
<li><a href="http://twitter.com/JefClaes">@JefClaes</a> (Ubiquitous Language),</li>
<li><a href="http://twitter.com/ONE75">@ONE75</a> (Context Mapping),</li>
<li><a href="http://twitter.com/ToJans">@ToJans</a> (Selling Domain-Driven Design to the business),</li>
<li><a href="http://twitter.com/mathiasverraes">myself</a> (Bounded Contexts and Modelling).</li>
</ul>
<p>Update June 10, 2014: new & improved slides for the second edition of this session.</p>
<script async="" class="speakerdeck-embed" data-id="a75cfd80d2e40131173366cd45aa8ef5" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<p><a href="https://speakerdeck.com/mathiasverraes/ddd-basics-bounded-contexts-modelling-kortrijk-edition">Watch the slides on Speakerdeck</a></p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="https://verraes.net/2014/02/buzzword-free-bounded-contexts/">Buzzword-free Bounded Contexts</a></li>
</ul>
Bandwidth and Context Mapping2014-01-15T00:00:00+00:00https://verraes.net/2014/01/bandwidth-and-context-mapping<p>Context Mapping, first described by Eric Evans in Domain-Driven Design, is lightweight method of drawing out the relations between systems, and parts of systems. It’s not technical per se. It exposes the politics of the organisations and the teams building the systems. As you start mapping, every edge that connects two systems, defines an upstream, and a downstream point; the latter system being the one that is affected by changes in the former. Or, to put it more graphically: if the upstream people piss in the river, the downstream people are drinking it. Pardon my French. In the book, Eric defines a number of patterns, that characterise those relations: Anticorruption Layer, Custom/Supplier, Open Host Service, Published Language, Shared Kernel, etc.</p>
<p>Mind that it’s not about the direction of the data. You might be sending data to another system, but still be downstream, because they decide what that data should look like. Eric, and <a href="https://twitter.com/ziobrando/status/423531883893252096">Alberto Brandolini</a>, have <a href="http://www.infoq.com/articles/ddd-contextmapping">explained it better</a> than I could, so I won’t go into detail.</p>
<p>In the workshop today, and over dinner, <a href="https://twitter.com/ziobrando/status/423531883893252096">Alberto Brandolini</a> explained his recent idea about Bandwidth. For me, that ties the whole of Context Mapping together. I will try to do it justice here.</p>
<p><img src="/img/posts/2014-01-15-bandwidth-and-context-mapping/the-deepest-explanation-of-ddd.png" alt="I think today I gave the deepest explanation ever of Context Mapping ...and I feel good!" /></p>
<h2 id="whitespace-and-bandwidth">Whitespace and Bandwidth</h2>
<p>Claude Debussy predicted funk music:</p>
<blockquote>Music is the space between the notes.</blockquote>
<p>When you draw a Context Map, you visualize the different Bounded Contexts of the system. This is of course useful information, especially when tackling a legacy system. Invisible to the untrained eye, it is in fact more than a static representation. In the whitespace between the parts, there is implicit information about the future success of the project. Say you are downstream from another team’s system. You’ve labeled the relationship as Conformist on your side. Now someone wants you to make a significant change, and this change touches on this relationship with the upstream system. Will this project succeed?</p>
<p>You can not answer this from just looking at the Context Map. Some missing concept is hiding in the whitespace. The patterns describe the character of the communication, but not how much communication is going on between the two teams. Alberto names this quantity the Bandwidth.</p>
<p><img src="/img/posts/2014-01-15-bandwidth-and-context-mapping/context-map-bandwidth.jpg" alt="Context Map with Bandwidth" /></p>
<p>Bandwidth will affect you a lot. If the upstream team is in a different country, and you only have one monthly call with them, the Bandwidth is low. If they are in the office next door, and you have lunch with them daily, your Bandwidth is going to be higher. Even though in my example, you are mostly Conformist, you still have a chance of altering the relation slightly, perhaps pushing it more in the direction of a Customer/Supplier pattern. The Bandwidth allows more communication, more opportunity for shared understanding, more opportunity for having at least some say in the service they provide. Many things affect this Bandwidth: obvious things like physical availability, frequency of communications, responsiveness… But also things like seniority: people who have been in the organisation a long time, are usually the ones setting the rules. Sometimes it will be determined by which system is more essential to the business, although, in some case, a less important system might still call the shots. If one system is legacy, it will be upstream, simply because it is too hard to change. Or perhaps the upstream team simply has too much ego to want to empathise with other teams.</p>
<p>Making this Bandwidth explicit, by drawing fatter and thinner edges, helps you make more informed decisions. Low Bandwidth means higher risk for projects that depend on communications between Bounded Contexts. With Bandwidth, the Context Map is more complete: it now shows the structure, the character of the relations, and the quantity of the communications.</p>
<p><strong>Update 2014-01-16</strong>
I missed a small nuance. It’s not about the amount of communication going on per se, but rather about the <em>potential</em>, or the <em>capacity</em>, for communication.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/02/buzzword-free-bounded-contexts/">Buzzword-free Bounded Contexts</a></li>
<li><a href="/2015/04/context-mapping-life-expectancy/">Context Mapping: Life Expectancy</a></li>
</ul>
Build for Failure2014-01-05T00:00:00+00:00https://verraes.net/2014/01/build-for-failure<p>We can’t avoid failure. Systems fail all the time. The deep assumption of failure needs to be engraved in everything we do or build.</p>
<p>This insight has been hitting me in the face for the past year, with the BuildStuff conference as the pinnacle. Failure in Systems is inevitable (<a href="/2013/08/john-gall-systemantics-the-systems-bible/">Gall</a>). Predictions fail, and anything that depends on randomness (<a href="http://www.amazon.com/Fooled-Randomness-Hidden-Chance-Markets/dp/0812975219">Taleb</a>). We’re not rational beings, even if we understand to what extent we are not rational (<a href="http://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman-ebook/dp/B005MJFA2W">Kahneman</a>). It’s the philosophy of Erlang, to build for failure. It’s the idea behind the Lean Startup: fail fast and fail often, until you start to suck at failing. Instead of aiming for small known potential gains (with unknown potential losses), aim for small known losses (with unknown potential gains) (<a href="http://localhost:4000/2013/08/antifragile-nassim-nicholas-taleb/">Taleb</a> again). Publish blogs first, fix later (<a href="http://twitter.com/ToJans">@ToJans</a>). Model it wrong, refactor towards deeper insight (<a href="http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215/">Evans</a>). Visualise divergence of interpretations (<a href="http://twitter.com/ziobrando">@ziobrando</a>). Cause frequent failures, to force yourself to build resilient systems (Netflix Chaos Monkey). And so on…</p>
<p>I’ve had a sticky note on my team’s whiteboard for the past six months, saying “Find at least three solutions”. Problems do not have solutions. All they have, are a bunch of random approximations of solutions, each with a bunch of known and unknown drawbacks. The thing with the number three, is that after you’ve found three solutions, your collective brains are nicely warmed up, and you find three more, by combining parts from the first three. Challenge the solutions, break them down, and when you pick one, build the most minimal version of it. Evaluate. If it’s not what you thought it could be, you still have five more scenario’s to try.</p>
<p>As if BuildStuff didn’t fry my brain as it was, I was lucky to <a href="http://hintjens.com/blog:73">spend three hours on a plane</a> with <a href="http://twitter.com/hintjens">Pieter Hintjens</a>, the founder of the ZeoMQ community and the author of the book. Here’s a quote:</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2014-01-05-build-for-failure/zeromq-book.jpg" alt="ZeroMQ" /></p>
<blockquote>
We pretend to ourselves and others that we can be (could be) perfect, when in fact we consistently make mistakes. Bugs in code are seen as "bad", rather than "inevitable", so psychologically we want to see fewer of them, not uncover more of them. "He writes perfect code" is a compliment rather than a euphemism for "he never takes risks so his code is as boring and heavily used as cold spaghetti".
<br />
<br />
Some cultures teach us to aspire to perfection and punish mistakes in education and work, which makes this attitude worse. To accept that we're fallible, and then to learn how to turn that into profit rather than shame is one of the hardest intellectual exercises in any profession. We leverage our fallibilities by working with others and by challenging our own work sooner, not later.
<br />
<br />
(...)
<br />
<br />
The faster you can prove code incorrect, the faster and more accurately you can fix it. Believing that code works and proving that it behaves as expected is less science, more magical thinking. It's far better to be able to say, "libzmq has five hundred assertions and despite all my efforts, not one of them fails".
</blockquote>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/10/software-design-is-just-theory/">“Software design is just theory”</a></li>
<li><a href="/2013/08/john-gall-systemantics-the-systems-bible/">The Systems Bible</a></li>
</ul>
The Henry Ford Fallacy2014-01-01T00:00:00+00:00https://verraes.net/2014/01/henry-ford-fallacy<blockquote>
If I had asked people what they wanted, they would have said faster horses.
<small>Attributed to Henry Ford</small>
</blockquote>
<p>The line is often quoted as an excuse for deliberately ignoring users’ requests for features. It’s usually followed by some mention of the iPhone, Steve Jobs, or Apple in general. And, usually, it’s uttered by people who build a €20 app, not a hardware device or an entire car.</p>
<p>It’s the wrong question. If you ask users what they want, you’ll get an answer that is unimaginative, and that is additive. They’ll seldom ask to rethink an existing feature, and they’ll never ask to remove a feature. If Henry Ford would have asked what problem people had, and kept asking it over and over again, and drilled down to the core of the problem, they would have said: we want to get places faster, cheaper, more comfortable, and with less fuss.</p>
<p><img src="/img/posts/2014-01-01-henry-ford-fallacy/iron-dobbin.png" alt="The iron Dobbin is one creepy looking vehicle" /></p>
Domain-Driven Design is Linguistic2014-01-01T00:00:00+00:00https://verraes.net/2014/01/domain-driven-design-is-linguistic<p><img style="float:left;margin-right: 10px" src="/img/posts/2014-01-01-domain-driven-design-is-linguistic/hoarding small.jpg" title="Hoarding is an extreme form of loss aversion. http://jamesgulliverhancock.com/projects/hoarding/" /></p>
<p>If I give you half a million euros, you will be quite happy. If I give you a million euros, but on the way home, you get robbed and lose half a million, you will be unhappy, even though the end result is the same. Even if you only lose 200k, you will be unhappier than if you only had the half million to begin with. In psychology, this is called <a href="http://www.amazon.com/gp/product/B00555X8OA/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B00555X8OA&linkCode=as2&tag=verraesnet-20&linkId=WRTRBQWNNPU4DUAC">Loss Aversion</a>. That is, loss has a higher impact on us than gains. We are not purely rational economic beings. The behaviour that leads to a certain state is more important than the actual state.</p>
<p>Businesses don’t make decisions based on the money they have in the bank. It’s the successes and the failures that lead that profit, that are going to determine what you’re going to do next.</p>
<p>This is what, to me, is so appealing in Domain-Driven Design. Software development has traditionally been mostly about mathematics, about technical matters. DDD makes the linguistics the central concern. To express it in a formula:</p>
<p><code class="language-plaintext highlighter-rouge">f(x) = y and g(x) = y =/=> f = g</code></p>
<p>Or: Just because two behaviours lead to the same outcome, does not mean they are replaceable. In Domain-Driven modelling, state is a side effect of behaviour. State is only interesting as far as it has the potential to influence new behaviour. An Aggregate is not just an composition of objects that belong together to represent a certain state. An Aggregate manages behaviours that belong together, and keeps state only where they are relevant to those behaviours. Event Sourcing takes the idea to the its logical conclusion: We forget about persisting state altogether, and instead, we only persist the Domain Events that lead up to that state. We preserve the meaning, instead of the outcome. When we need the state, to make a decision, we replay the events and interpret them. The beauty of this, is that if we change our minds about how to interpret the behaviour, we are not stuck with the state the we decided to persist in the past.</p>
<p><strong>Update May 11, 2014</strong> I just had to add <a href="https://twitter.com/abdullin/status/465463982347026432">this quote from Rinat Abullin</a>:</p>
<blockquote>Behavior is not data. To be more precise, data is often an approximation of observed behaviors under fixed circumstances.</blockquote>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/01/model-storming-workshop/">Model Storming Workshop</a></li>
</ul>
Fighting Bottlenecks with CQRS2013-12-01T00:00:00+00:00https://verraes.net/2013/12/fighting-bottlenecks-with-cqrs<p>I hadn’t done any presentations or blog posts about Command/Query Responsibility Segregation yet, even though I’ve been
working with it for more than year (and have been wanting to use it for another year). <a href="https://twitter.com/__edorian">Volker Dusch</a>
invited me to speak at the first <a href="http://news.researchgate.net/index.php?/archives/181-ResearchGate-Developer-Day.html">ResearchGate Developer Day</a> in Berlin,
and offered me the luxury to choose any SE topic I wanted, I just had to do it.</p>
<script async="" class="speakerdeck-embed" data-id="9954fc103bd40131db9c2eaec66c710f" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<p><a href="https://speakerdeck.com/mathiasverraes/fighting-bottlenecks-with-cqrs">See the slides on Speakerdeck</a></p>
<p>Volker gave me a tour of the ResearchGate office. They’re all about sharing scientific research. It’s a fast growing
startup, with an awesome culture, and <a href="http://www.researchgate.net/careers/">they’re hiring</a>, so check them out.</p>
Value Objects and User Interfaces2013-11-15T00:00:00+00:00https://verraes.net/2013/11/value-objects-and-user-interfaces<p>I got this mail from a listener of the <a href="http://elephantintheroom.io/blog/2013/10/episode-2-heart-and-soul-of-oop/">podcast episode on Value Objects</a> I recorded with Konstantin:</p>
<blockquote>
I've been listening to your Elephant in the room podcast Episode #002, i was wondering if you have a coding example of where you use the Country as a value object and not as an entity.
I've come across this problem too, and was hoping you may have an example that i can look at to give me a better understanding in terms of how to code this?
My first question would really be how would you logically store a list of all countries outside of a database if you were using value objects for use in a Form lookup field?
</blockquote>
<p>There’s nothing intrinsically wrong with modelling countries as entities and storing them in the database. But in most cases,
that overcomplicating things. Countries don’t change often. When a country’s name changes,
it is in fact, for all practical purposes, a new country. If a country one day does not exist anymore, you can’t simply
change all addresses, because possibly the country was split into two countries.</p>
<p>Whenever you have this kind of friction, there’s usually some missing concept screaming to be discovered. In this case,
two concerns are being mixed. One is a modelling concern: we want to treat <code class="language-plaintext highlighter-rouge">Country</code> as a value object with all the
benefits that it brings. The other is a user experience concern: we want to help the user to pick a country from a list, so we don’t
end up with twenty different spellings of the same country. And finally, we’d probably want to validate somehow that the
submitted country indeed exists.</p>
<p>So what’s the missing concept? The value object represents an actual country, and the UX and validation require a list of
available countries. Even though both are closely related, the nuance is slightly different. So let’s make it explicit and
call the new concept <code class="language-plaintext highlighter-rouge">AvailableCountries</code>. These available countries can be entities in a database, records in a JSON, or
even simply a hardcoded list in your code. (That depends on whether the business wants easy access to them through a UI.)</p>
<p>In code, it could look something like this:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">Country</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$countryCode</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$countryCode</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">countryCode</span> <span class="o">=</span> <span class="nv">$countryCode</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__toString</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="n">countryCode</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">AvailableCountry</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$country</span><span class="p">;</span>
<span class="k">private</span> <span class="nv">$name</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">Country</span> <span class="nv">$country</span><span class="p">,</span> <span class="nv">$name</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">country</span> <span class="o">=</span> <span class="nv">$country</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">name</span> <span class="o">=</span> <span class="nv">$name</span><span class="p">;</span>
<span class="p">}</span>
<span class="cd">/** @return Country */</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">getCountry</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="n">country</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">getName</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="n">name</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="k">final</span> <span class="kd">class</span> <span class="nc">AvailableCountryRepository</span>
<span class="p">{</span>
<span class="cd">/** @return AvailableCountry[] */</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">findAll</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="p">[</span>
<span class="s1">'BE'</span> <span class="o">=></span> <span class="k">new</span> <span class="nc">AvailableCountry</span><span class="p">(</span><span class="k">new</span> <span class="nc">Country</span><span class="p">(</span><span class="s1">'BE'</span><span class="p">),</span> <span class="s1">'Belgium'</span><span class="p">),</span>
<span class="s1">'FR'</span> <span class="o">=></span> <span class="k">new</span> <span class="nc">AvailableCountry</span><span class="p">(</span><span class="k">new</span> <span class="nc">Country</span><span class="p">(</span><span class="s1">'FR'</span><span class="p">),</span> <span class="s1">'France'</span><span class="p">),</span>
<span class="c1">//...</span>
<span class="p">];</span>
<span class="p">}</span>
<span class="cd">/** @return AvailableCountry */</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">findByCountry</span><span class="p">(</span><span class="kt">Country</span> <span class="nv">$country</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="nf">findAll</span><span class="p">()[(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$country</span><span class="p">];</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>So now you can query <code class="language-plaintext highlighter-rouge">AvailableCountryRepository</code> to populate select boxes, and use it to see if a Country exists.</p>
<h2 id="when-to-use">When to use</h2>
<p>This pattern should be easy to implement in many different ways, suited to whatever your use case is. Deciding when and
when not to use this pattern, is probably the hardest part. Start by imagining you are building a ‘pure’ model. There’s
no UI, no database, just code. Is the thing you’re modelling an entity or a value object? Does it change over time? If two
instances have the same value, can they have a different identity? Or is it the value alone that defines what they are?
Try listing the problems you are trying to solve (model, UX, validation, …) and try to come up with a separate solution
for each of them. Take decoupling to the max, poke around for missing concepts, and make the implicit explicit. Don’t think
of database normalisation, relational integrity, performance, … those are all optimisations that you can add on top of
your solution, as opposed to allowing your logical solution to be defined by the infrastructural constraints. And above all,
experiment, fail, try again, learn!</p>
<h2 id="read-more">Read More</h2>
<ul>
<li>Elephant in the Room podcast, <a href="http://elephantintheroom.io/blog/2013/10/episode-2-heart-and-soul-of-oop/">Episode #002 on Value Objects</a></li>
<li><a href="http://www.jefclaes.be/2013/05/accidental-entities-you-dont-need-that.html">Accidental Entities - You don’t need that identity</a> by <a href="http://twitter.com/JefClaes">@JefClaes</a></li>
<li><a href="http://www.jefclaes.be/2013/05/accidental-entities-you-dont-need-that.html">Accidental Entities - What about the UI?</a> by <a href="http://twitter.com/JefClaes">@JefClaes</a></li>
<li><a href="/2013/12/related-entities-vs-child-entities/">Related Entities vs Child Entities</a></li>
</ul>
Pre-merge Code Reviews2013-10-26T00:00:00+00:00https://verraes.net/2013/10/pre-merge-code-reviews<p>I’m a big proponent of pre-merge code reviews. From my experience consulting for teams in problematic projects, I can say that
(along with daily standup meetings) pre-merge code reviews are one of the most effective and yet fairly easy changes a team can introduce to radically improve
the condition of the project. And even if you consider your project to be healthy, there’s always room for improvement.</p>
<h2 id="the-rules">The Rules</h2>
<ol>
<li>
<p><strong>Never Push to Master</strong><br />
Always push to a separate branch per logical unit (story, feature, bug, optimisation, refactor, improvement). Branches are easy to make and easy to merge when you use git (and you apply some of tips further down in this post).</p>
</li>
<li>
<p><strong>Never Merge Your Own Branch</strong><br />
This helps to ensures that code is in fact reviewed. If you are caught merging into master, you will order pizza for the whole team.</p>
</li>
<li>
<p><strong>Review Work in Progress First</strong><br />
When you are finished with a task, you notify the other team members that your work is ready for final review. Then you review existing branches. Before picking up a new task, you look at all open pull requests (including unfinished ones) and review the changes since the last time you checked.</p>
</li>
<li>
<p><strong>Merge responsibly</strong><br />
Merging a pull request is the responsibility of the whole team. A pull request can not be merged when someone in the team does not understand the code or the reasoning, or does not agree with the solution.</p>
</li>
</ol>
<p>(Note that these rules are starting points. Figure out what works in your team, adapt continuously.)</p>
<h2 id="the-benefits">The Benefits</h2>
<p>Having multiple sets of eyes review a pull request before it gets merged to master or an integration branch, is a great
way to catch defects early. At this time, they are usually still cheap to fix.</p>
<p>There are however much more important benefits. Instead of individual developers, the team is responsible for the internal and external quality of the code.
This is a great remedy against the blame culture that is still present in many organisations. Managers or team members can no longer point
fingers to an individual for not delivering a feature within the expected quality range. You tend to become happier and more productive, knowing that the
team has your back. You can afford to make a mistake; someone will find it quickly.</p>
<p>Another effect is something called ‘swarming’ in Kanban. Because you are encouraged to help out on other branches before starting your own,
you start to help others finishing work in progress. Stories are finished faster, and there’s a better flow throughout the system.
Especially when stories are difficult, or when stories block other stories, it’s liberating to have people come and help you to get it done.</p>
<p>And of course, there’s all the benefits from the clear sense of code co-ownership. It’s invaluable to have a team where everybody
knows what the code does, why it’s designed that way, how everything fits together. It also reduces the Bus Factor: no single team
member is a bottleneck. Best practices are shared, and the code is more consistent. Opportunities for reuse are spotted before
lots of duplication happens.</p>
<p>In short, pre-merge code reviews grow the team’s maturity.</p>
<h2 id="making-it-easier">Making It Easier</h2>
<p>Reading code is hard, much harder than writing it. Here are some ideas that I have found to make things easier.</p>
<ul>
<li>
<p><strong>Atomic Stories</strong><br />
Break down stories in smaller, more atomic stories. “Manage products” is way to large. “Add products”, “Add an image to
a product”, “Modify the description of a product”, “Remove a product”, “Set a new price”: these are atomic, and it will make branches much easier to review and merge.</p>
</li>
<li>
<p><strong>Acceptance Criteria</strong><br />
List the acceptance criteria in the pull request. (In GitHub, you can use <a href="https://github.com/blog/1375-task-lists-in-gfm-issues-pulls-comments">checkboxes in Markdown</a>,
making it easy to tick them).</p>
</li>
<li>
<p><strong>Atomic Commits</strong><br />
Commit atomically. For example, don’t do code formatting and code changes in one commit, it makes the diffs very hard to
read. Each commit should preferably leave the code base in a consistent state, but apart from that, there’s no limit to how small the commits can be. I often commit single lines.</p>
</li>
<li>
<p><strong>Pull Requests as Conversations</strong><br />
A pull request is a conversation. Open it when you push your first commit. Comment, ask for feedback, commit to other
people’s branches if that helps them out. The pull request is the story of how the feature came into being.</p>
</li>
<li>
<p><strong>Integrate Frequently</strong><br />
Pull master into your branch regularly, to avoid difficult merges.</p>
</li>
<li>
<p><strong>Pair on Merges</strong><br />
Pair program on difficult merge conflicts, preferably with the person who wrote the code you are merging with. (Pair
programming in general is highly recommended, but off topic here.)</p>
</li>
<li>
<p><strong>Review Tests Only</strong><br />
If you use TDD, and you are short on time, review only the tests. If these are satisfactory, the implementation will
probably be ok too.</p>
</li>
<li>
<p><strong>Solve the Problem Before Coding</strong><br />
Catching problems early when the code is written is great, but catching them before the code is written, is better. Go
to the whiteboard with a couple of team members and draw out a solution everybody agrees with. Reviewing the code afterwards will be easier because you already have a model on the whiteboard.</p>
</li>
</ul>
<p>I’m assuming in this post that you use GitHub, but there are other solutions, such as ReviewBoard, BarKeep, Phabricator,
and Gerrit. I did a little bit of research last year, and felt that GitHub was the best tool, but I have no hands-on
experience with the others. YMMV. The important factor for me is that you can review stories or branches as a whole,
and comment inline. (Update: @tvlooy suggested GitLab.)</p>
<h2 id="pitfalls-and-anti-patterns">Pitfalls and anti-patterns</h2>
<ul>
<li>
<p><strong>Merge Buddies</strong><br />
As soon as you impose rules, you’ll find people who try to game the system. Sometimes two team members consistently
merge each other’s pull requests, without actually looking at them. “I’ll merge
yours if you merge mine.” They appear to conform to the rules, but of course it’s just a variation on their old habit of
pushing to master. Catch this early. Gatekeeper can be a good remedy.</p>
</li>
<li>
<p><strong>GateKeeper</strong><br />
One person is responsible for doing all the reviews and the merges. It’s not necessarily an anti-pattern, as it can be useful to prevent
people from cheating. But if you keep having a long term Gatekeeper, you have serious problems. People will not feel responsible
for their code, let alone others. They will not learn from other people’s code, and they will adapt their code to the Gatekeeper’s
comments, without bothering to understand why. Avoid when possible.</p>
</li>
<li>
<p><strong>Total Lockdown</strong><br />
One step further than having a Gatekeeper, is taking away everybody’s permissions to touch the main repository, and forcing
them to use their own forks and send pull requests from there. It an be a good measure if the situation is really problematic, but it’s
a terrible downer for morale. Deserved or not, people will feel distrusted. (Update: To clarify, it’s not a problem if they never
had pull permissions from the beginning.)</p>
</li>
<li>
<p><strong>The Apologists</strong><br />
“We don’t have time for code reviews.” Or: “Code reviews do not apply in this project.” As with all discussions about
software quality, there will always be people who feel it’s perfectly alright to save time writing junk first,
and then waste huge amounts of it bugfixing and putting out fires. They probably don’t write tests either, or refactor code.
This is a deeper issue that won’t be fixed by enforcing code reviews alone.</p>
</li>
<li>
<p><strong>Cowboys and Hermits</strong><br />
Some people don’t want their code looked at. They write their share, and guard it from others, by making sure they are the only ones
working on that area. That’s unacceptable, and you’ll need to figure out why they act this way.</p>
</li>
<li>
<p><strong>Issue Trackers</strong><br />
Many teams discuss issues inside their issue trackers. The distance from the actual code is too great. GitHub’s inline commenting,
on individual lines, allows developers to talk about actual code instead of abstract descriptions of code.</p>
</li>
<li>
<p><strong>Focus on Junk Details</strong><br />
If your code reviews focus on having the correct amount of spaces in the indentation, or typos in comments, you are wasting the effort.
Focus on the important issues: Are the tests sufficiently conclusive? Is the naming expressive? Is there coupling where none should be?
Will we still understand the code 12 months from now? Is it refactorable? …</p>
</li>
</ul>
<p>As usual, I will keep adding to this post as I find more patterns. Got tips?</p>
<p>Update: Apparently Phil Haack <a href="http://haacked.com/archive/2013/10/28/code-review-like-you-mean-it.aspx">blogged about code reviews</a>
almost at the exact same time. He has some great tips like keeping a checklist, focusing on the code and
not the author, and stepping through the code. Worth reading!</p>
<h2 id="read-next">Read next</h2>
<ul>
<li><a href="/2016/04/code-reviews-and-blame-culture/">Code Reviews and Blame Culture</a></li>
<li><a href="/2014/12/how-much-testing-is-too-much/">How Much Testing is Too Much?</a></li>
<li><a href="/2014/03/small-controlled-experiments/">Small Uncontrolled Experiments</a></li>
</ul>
Verbs in Class Names2013-10-06T00:00:00+00:00https://verraes.net/2013/10/verbs-in-class-names<p>When you first learned Object Oriented Programming, somebody probably told you that objects map to things. And that still holds true
most of the time. And maybe somebody explained it using the simple heuristic to “look for the nouns”, which can indeed be a great
discovery technique. But then somebody probably phrased that as “class names should not have verbs in them”. To me,
that rule is severely limiting the possibilities for your models. So here are some cases where I prefer to use verbs.</p>
<h2 id="messages">Messages</h2>
<p>The king sends a message to the general: “Attack the enemy on the south flank with three divisions”. The general does as
he’s told, and succeeds. He sends a message back to the king: “The enemy is defeated!”. Peace is restored in the
<a href="http://steve-yegge.blogspot.co.uk/2006/03/execution-in-kingdom-of-nouns.html">Kingdom of Nouns</a>.</p>
<p>The messages that are sent back and forth, are “things”, so they can be modeled as objects. Message objects have all the benefits
of OOP: you can pass them around, have type hinting, encapsulation… They can have properties: <code class="language-plaintext highlighter-rouge">private $flank = 'south'</code>,
<code class="language-plaintext highlighter-rouge">private $numberOfDivisions = 3</code>. You can serialize the messages, send them over the wire,
persist them in an audit log, or even an event store. You can publish the messages, or put them in a message queue, using
ZeroMQ or an alternative.</p>
<p>If you know me at all, you know I’m obsessed with naming. So what should we name these two messages? We could use the
nouns “Attack” and “Defeat”, as in <code class="language-plaintext highlighter-rouge">AttackMessage</code>, <code class="language-plaintext highlighter-rouge">DefeatMessage</code>. (English is a bit messy in this regard, as nouns and
verbs are often exactly the same word – an attack, to attack.) To me, this naming is too vague, too implicit. Let’s try
a bit harder. Both message objects are very much alike. Are we missing a concept? If we focus on the differences, then we
discover that one of them is a (Domain-) Command: The king tells the general what to do. The other one is a (Domain) Event: it’s something
that happened in the past, that is of interest to the king: the enemy was defeated. <code class="language-plaintext highlighter-rouge">AttackCommand</code>, <code class="language-plaintext highlighter-rouge">DefeatEvent</code>? Close.</p>
<p>The most expressive option is to name it after its purpose. <code class="language-plaintext highlighter-rouge">AttackEnemy implements Command</code>, <code class="language-plaintext highlighter-rouge">EnemyWasDefeated implements Event</code>.
Again, we have the problem with English here. But this time it’s either an imperative verb (“Do This”), or a past tense (“This happened”).
More examples are <code class="language-plaintext highlighter-rouge">CheckoutBasket</code>, <code class="language-plaintext highlighter-rouge">PayForOrder</code>, <code class="language-plaintext highlighter-rouge">OrderWasPaid</code>, <code class="language-plaintext highlighter-rouge">OrderWasShipped</code>.</p>
<p>There’s no ambiguity about what any of these objects represents. And you can use them to build expressive api’s:
<code class="language-plaintext highlighter-rouge">$auditLog->recordThat(new EnemyWasDefeated($onDate));</code> Imagine the <code class="language-plaintext highlighter-rouge">new</code> keyword isn’t there, and what you end up with is a nice
sentence, saying exactly what it does: the audit log records that the enemy was defeated on a certain date.</p>
<h2 id="specifications">Specifications</h2>
<p>The Specification pattern is a way to model business rules as individual objects – I talk about this in
<a href="/2013/06/unbreakable-domain-models/">Unbreakable Domain Models</a>. The idea is that a question about an object, is answered by a
<code class="language-plaintext highlighter-rouge">isSatisfiedBy()</code> method:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">CustomerIsPremium</span> <span class="kd">implements</span> <span class="nc">CustomerSpecification</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$someCollaborator</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* ... */</span> <span class="p">}</span>
<span class="cd">/** @return bool */</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">isSatisfiedBy</span><span class="p">(</span><span class="kt">Customer</span> <span class="nv">$customer</span><span class="p">)</span> <span class="p">{</span>
<span class="c1">// figure out if the customer is indeed premium and return true or false.</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Other examples are <code class="language-plaintext highlighter-rouge">OrderIsReadyForShipment</code>, <code class="language-plaintext highlighter-rouge">BankAccountIsWithinCreditLimit</code>, …
Again, having the verb in there makes it much more natural. They are small, composable objects, and you can build sentences from them.</p>
<h2 id="exceptions">Exceptions</h2>
<p>Exceptions can be similar to events, as they describe something that has happened in your system, with the nuance that it is
an undesirable event, a warning that some operation was inconsistent with, for example, business rules that are in effect. For
application- or infrastructure-level exceptions, I don’t really bother much: <code class="language-plaintext highlighter-rouge">DbException</code> or <code class="language-plaintext highlighter-rouge">FSException</code> are fine, the
message string will usually tell you more, and it’s all very technical anyway. But for domain model exceptions, I want it,
once more, in the language of the business: <code class="language-plaintext highlighter-rouge">OrderShipmentHasFailed</code>, <code class="language-plaintext highlighter-rouge">BackAccountWasOverdrawn</code>. I don’t generalize them
either: I have an individual exception class for each specific occasion.</p>
<h2 id="interfaces">Interfaces</h2>
<p>Interfaces are not classes, but close enough for inclusion. I go on about them in <a href="/2013/09/sensible-interfaces/">Sensible Interfaces</a>,
but let’s hammer it some more.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">FireEmployee</span> <span class="kd">implements</span> <span class="nc">DomainCommand</span><span class="p">,</span> <span class="nc">HasPermissions</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">getPermissions</span><span class="p">()</span> <span class="p">{</span>
<span class="k">return</span> <span class="p">[</span><span class="s1">'ROLE_CEO'</span><span class="p">,</span> <span class="s1">'ROLE_HR_MANAGER'</span><span class="p">];</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>I feel like I’m ranting here, but once, again, it’s a sentence in natural language. You can talk to another developer and
say “This domain command has permissions”. There is no additional translation in your brain that converts that sentence to
code such as <code class="language-plaintext highlighter-rouge">implements Permissionable</code>.</p>
<p>The sentence <em>is</em> the code.</p>
<p>If you have more types of classes where you use verbs, let me know!</p>
<h3 id="read-more">Read more</h3>
<ul>
<li><a href="/2013/09/sensible-interfaces/">Sensible Interfaces</a></li>
<li><a href="http://steve-yegge.blogspot.co.uk/2006/03/execution-in-kingdom-of-nouns.html">Execution in the Kingdom of Nouns</a></li>
<li><a href="/2013/06/unbreakable-domain-models/">Unbreakable Domain Models</a></li>
</ul>
Sensible Interfaces2013-09-24T00:00:00+00:00https://verraes.net/2013/09/sensible-interfaces<p>Here are some guidelines to help you live in friendly cohabitation with your fellow rockstar ninja code monkeys. I’ll update them
as my ideas about the matter evolve.</p>
<h2 id="naming">Naming</h2>
<p>In my team, suffixing your interface with the word <code class="language-plaintext highlighter-rouge">Interface</code>, is a sackable offence. Well,
maybe it’s not <em>that</em> bad, but don’t bother trying it. The same goes for prefixing it with <code class="language-plaintext highlighter-rouge">I</code> – looking at you Microsoft.
Here’s the reasoning.</p>
<p>If you have an interface, then you are suggesting
that multiple implementations of that interface are possible. Usually, when you see something like <code class="language-plaintext highlighter-rouge">TranslatorInterface</code>, chances
are there’s an implementation called <code class="language-plaintext highlighter-rouge">Translator implements TranslatorInterface</code>. It makes me wonder: what makes the <code class="language-plaintext highlighter-rouge">Translator</code> so special, that it has
the unique right of being called <code class="language-plaintext highlighter-rouge">Translator</code>? Every other implementation needs a descriptive name, such as <code class="language-plaintext highlighter-rouge">XmlTranslator</code> or
<code class="language-plaintext highlighter-rouge">CachedTranslator</code>, but that one is somehow “the default”, as suggested by it’s preferential treatment in being named <code class="language-plaintext highlighter-rouge">Translator</code>
without a description.</p>
<p>Is that bad? I believe it is. It confuses people, as they tend to misunderstand whether they should typehint for <code class="language-plaintext highlighter-rouge">Translator</code>
or <code class="language-plaintext highlighter-rouge">TranslatorInterface</code>. So both are being used in the client code. Program sometimes to an interface, sometimes to an implementation?</p>
<p>Just as bad, is that it suggests that the implementation is the real thing, and the interface is a label. It should be the
other way around. Look at it from the point of view of the client code:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">KlingonDecoder</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">TranslatorInterface</span> <span class="nv">$translator</span><span class="p">)</span>
</code></pre></div></div>
<p>This constructor definition is saying: “I need a translator <strong>interface</strong> to operate”. But that would be silly. It needs an
object that is a <code class="language-plaintext highlighter-rouge">Translator</code>. It does not need an interface. And that object has a certain role, a certain contract, namely that of a <code class="language-plaintext highlighter-rouge">Translator</code>.
I hope I’m making this clear. The interface <code class="language-plaintext highlighter-rouge">Translator</code> is the essential concept, the thing that clients use. They don’t care
whether <code class="language-plaintext highlighter-rouge">Translator</code> is a concrete class or an interface, and they don’t care how it’s implemented. The client wants to be
decoupled from all those details. That’s the power of interfaces.</p>
<h2 id="default-implementations">Default implementations</h2>
<p>The burden of having a descriptive name then lies with the implementations. If we rename <code class="language-plaintext highlighter-rouge">TranslatorInterface</code> to <code class="language-plaintext highlighter-rouge">Translator</code>, our
former <code class="language-plaintext highlighter-rouge">Translator</code> class needs a new name. People tend to solve this problem by calling it <code class="language-plaintext highlighter-rouge">DefaultTranslator</code>. Again, what makes it so
special to be called <code class="language-plaintext highlighter-rouge">Default</code>? Don’t be lazy, think really hard about what it does, and why that’s different from other
possible implementations. You might even discover a thing or two about that class, such as having too many responsibilities.</p>
<h2 id="nameable">Nameable</h2>
<p>Another bad habit, is using the <code class="language-plaintext highlighter-rouge">-able</code> suffix for interface names. I guess I can live with something like <code class="language-plaintext highlighter-rouge">Translatable</code>,
or maybe <code class="language-plaintext highlighter-rouge">Serializable</code>. But <code class="language-plaintext highlighter-rouge">Timestampable</code>? <code class="language-plaintext highlighter-rouge">Jsonable</code>? Is that the world we want our children to inherit? English motherfucker,
do you speak it? Try making a sentence, it’s so much nicer.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">Product</span> <span class="kd">implements</span> <span class="nc">CastsToJson</span><span class="p">,</span> <span class="nc">HasTimestamp</span>
</code></pre></div></div>
<p>Say it out loud: “Product casts to json and has a timestamp”. It’s beautiful, it’s – dare I say it? – Shakespearian.</p>
<h2 id="respect-the-contract">Respect the contract</h2>
<p>PHP, having grown organically (to put it politely), is rather permissive when it comes to interfaces. Look at this code:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">interface</span> <span class="nc">Animal</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">makeNoise</span><span class="p">();</span>
<span class="p">}</span>
<span class="kd">class</span> <span class="nc">Dog</span> <span class="kd">implements</span> <span class="nc">Animal</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">makeNoise</span><span class="p">()</span> <span class="p">{}</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">fetchStick</span><span class="p">()</span> <span class="p">{}</span>
<span class="p">}</span>
<span class="c1">// elsewhere:</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">myClient</span><span class="p">(</span><span class="kt">Animal</span> <span class="nv">$animal</span><span class="p">)</span> <span class="p">{</span>
<span class="nv">$animal</span><span class="o">-></span><span class="nf">fetchStick</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Even though <code class="language-plaintext highlighter-rouge">myClient()</code> accepts an <code class="language-plaintext highlighter-rouge">Animal</code>, and should have no knowledge whether the <code class="language-plaintext highlighter-rouge">$animal</code> is a <code class="language-plaintext highlighter-rouge">Dog</code>b, PHP allows you to call
<code class="language-plaintext highlighter-rouge">fetchStick()</code> if <code class="language-plaintext highlighter-rouge">$animal</code> has a method by that name. That flexibility can be very useful, in a very limited set of cases.
For all normal cases: never call a method on an object that is not part of the interface you are typehinting for.</p>
<h2 id="interface-segregation">Interface segregation</h2>
<p>If you find that your client code depends on an interface with many methods that your client doesn’t care about, your
interface may be too big. The same is true if your classes implementing an interface, have a lot of unused stub methods:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">Fish</span> <span class="kd">implements</span> <span class="nc">Animal</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">makeNoise</span><span class="p">()</span> <span class="p">{</span>
<span class="k">throw</span> <span class="k">new</span> <span class="nc">NotImplemented</span><span class="p">(</span><span class="s2">"Fish don't make noise"</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>This is a good sign that you need to split off the <code class="language-plaintext highlighter-rouge">makeNoise()</code> method into a separate interface. Perhaps <code class="language-plaintext highlighter-rouge">MakesNoise</code>, or <code class="language-plaintext highlighter-rouge">Noisy</code>?</p>
<h2 id="roles">Roles</h2>
<p>Interfaces can be a nice way to share code without the client knowing, by seeing them as roles. Say the product prices
are in a database. You have some logic in <code class="language-plaintext highlighter-rouge">OrderBuilder</code>, but you don’t want <code class="language-plaintext highlighter-rouge">OrderBuilder</code> to know that the prices are in the
database, because that might change in the future. You could solve this with composition.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">interface</span> <span class="nc">ProductRepository</span> <span class="p">{</span>
<span class="cm">/* defines find(), add()... */</span>
<span class="p">}</span>
<span class="kd">interface</span> <span class="nc">ProductPricer</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">priceProduct</span><span class="p">(</span><span class="kt">Product</span> <span class="nv">$product</span><span class="p">);</span>
<span class="p">}</span>
<span class="kd">class</span> <span class="nc">DbProductPriceRepository</span> <span class="kd">implements</span> <span class="nc">ProductRepository</span> <span class="p">{</span>
<span class="cm">/* implements find(), add()... */</span>
<span class="p">}</span>
<span class="kd">class</span> <span class="nc">DbProductPricer</span> <span class="kd">implements</span> <span class="nc">ProductPricer</span> <span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">ProductRepository</span> <span class="nv">$productRepository</span><span class="p">){</span> <span class="cm">/* ... */</span> <span class="p">}</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">priceProduct</span><span class="p">(</span><span class="kt">Product</span> <span class="nv">$product</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* ... */</span> <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>To save a bit of typing, you can give the <code class="language-plaintext highlighter-rouge">ProductPricer</code> role to <code class="language-plaintext highlighter-rouge">DbProductPricer</code>. Sure, it’s doing double duty, but the clients
don’t know that. In this example, it’s probably not a good long term solution when your application grows, but if nothing else,
it’s a great timesaver when prototyping.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">DbProductPriceRepository</span> <span class="kd">implements</span> <span class="nc">ProductRepository</span><span class="p">,</span> <span class="nc">ProductPricer</span> <span class="p">{</span>
<span class="cm">/* implements find(), add()... */</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">priceProduct</span><span class="p">(</span><span class="kt">Product</span> <span class="nv">$product</span><span class="p">)</span> <span class="p">{</span> <span class="cm">/* ... */</span> <span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Roles of course work great with entities:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">Teacher</span> <span class="kd">implements</span> <span class="nc">User</span> <span class="p">{}</span>
<span class="kd">class</span> <span class="nc">Pupil</span> <span class="kd">implements</span> <span class="nc">User</span> <span class="p">{}</span>
<span class="kd">class</span> <span class="nc">Parent</span> <span class="kd">implements</span> <span class="nc">User</span> <span class="p">{}</span>
</code></pre></div></div>
<h2 id="one-implementation">One implementation</h2>
<p><code class="language-plaintext highlighter-rouge">ProductPricer</code> is a great use case for an interface, because it’s easy to imagine different business rules applying in
different circumstances: <code class="language-plaintext highlighter-rouge">GermanProductPricer</code>, <code class="language-plaintext highlighter-rouge">BelgianProductPricer</code>. There could be different technical implementations:
<code class="language-plaintext highlighter-rouge">DbProductPricer</code>, <code class="language-plaintext highlighter-rouge">SoapProductPricer</code>, or a <code class="language-plaintext highlighter-rouge">CachedProductPricer</code> that wraps one of the other implementations.</p>
<p>But often, it’s less clear. If your business has only One True Way to calculate prices, and One True Datasource to store them.
My rule of thumb would be that if you can imagine that there could be more implementations than just the one, it’s good to
have an interface. If you can’t imagine different implementations, don’t have an interface. An example could be <code class="language-plaintext highlighter-rouge">OrderTotalCalculator</code>:
There’s only one valid way to sum the different prices, so an interface does not make sense.</p>
<h2 id="aware">Aware</h2>
<p>I’m sort of undecided about the <code class="language-plaintext highlighter-rouge">Aware</code> suffix. I don’t think it’s a problem per se. But of course <code class="language-plaintext highlighter-rouge">ContainerAware</code> is evil
and should never have been in Symfony. But that is a Dependency Injection anti-pattern, not an interface naming problem. Thoughts?</p>
<p>Read next: <a href="/2011/03/interface-discovery-with-phpunit-mock-objects/">Interface discovery with PHPUnit’s Mock objects</a> - March 21, 2011</p>
Extract Till You Drop2013-09-20T00:00:00+00:00https://verraes.net/2013/09/extract-till-you-drop<p>“Extract Till You Drop” is a live refactoring that I used to do at conferences and as a webinar.</p>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/1_dpOZmKXBw" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<h3 id="abstract">Abstract</h3>
<p>We’ve all seen them: applications out of control. Under the pressure of deadlines and endless change requests, with the
weight of years of legacy, the code has become unmaintainable. Adding features is a slow hit and miss process. You know
something needs to be done, but nobody knows how. To change the code safely, you need tests, but to make it testable,
you need to change it. Rebuilding the system from scratch is not an option.</p>
<p>With the right tools, techniques, and mindset, any codebase can be brought under test, and be refactored towards a better
architecture. All without affecting the behavior of the system, and allowing the business to continue.</p>
<p>This presentation is not for the weak of heart. We’ll skip the theory and dive straight into the spaghetti code. We’ll
wrestle in the mud, and we’ll tame the beast, whether it wants to or not. It’s live coding, and it may fail. But it will
change how you approach legacy code forever.</p>
The DDDBE Modellathon2013-09-05T00:00:00+00:00https://verraes.net/2013/09/dddbe-modellathon<h3 id="goal">Goal</h3>
<p>Many teams are not spending enough time on modeling. Often, they’re not particularly good at it, and often, the business
considers time spent at the whiteboard “unproductive”. The goal of the Modellathon workshop, was to give people a safe
environment to play around with different modeling techniques, without the pressure of producing something of immediate
value. For more experienced attendees, it’s an opportunity to exercise their modeling skills on a new and unfamiliar domain.</p>
<p>By collaborating with people, and by learning from the models of the other teams, the attendees get a glimpse of the myriad
of techniques that exist (often not documented anywhere) to explore, grasp, and visualize, their understanding of the domain.</p>
<h3 id="slides">Slides</h3>
<script async="" class="speakerdeck-embed" data-id="105cfa00f86b0130e851227600bb7d26" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<p><a href="https://speakerdeck.com/mathiasverraes/dddbe-modellathon-2013">See the slides on Speakerdeck</a></p>
<h3 id="what-we-did">What we did</h3>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2013-09-05-modellathon/mathias_verraes_at_dddbe.jpg" alt="Mathias at DDDBE" /></p>
<p>We formed teams of four people. Each team was provided with plenty of <a href="http://www.ikea.com/be/nl/catalog/products/20152281/">paper</a>,
markers, and stickies. Stijn and me played the role of domain experts. We gave a short introduction about “The United Schools of Kazachstan”.
This domain was obviously made up, but most of what we talked about, comes from a real project we are working on. We gave the people a
paper report card for <a href="https://twitter.com/DDD_Borat">a Kazach pupil</a>, and told them we wanted a better system for making the reports. Along the way we introduced
more complexity: the different ways pupils can be evaluated, the laws and governmental regulations, the importance of the progress a pupil
makes, and the confidentiality of the pupils.</p>
<p>In two rounds of 25 minutes, the teams got to work to try and draw the problem space into a model. Meanwhile, Stijn and I walked around to answer questions.
Just like real domain experts, we sometimes took things for granted, or casually mentioned some new piece of critical information.</p>
<p>For the third round, we shifted the focus from understanding the domain, to modeling a somewhat workable solution. Yves and me
now tried to give guidance on techniques, and give the teams some pointers. For example, I showed one team how to get started with
<a href="https://verraes.net/2013/08/facilitating-event-storming/">Event Storming</a>. Another team had already produced a very detailed
and complete model, so I told them that if this were a real application, the teachers would have to spend too much time entering
data, and too little time actually teaching. This forced the team to re-evaluate some of their assumptions.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2013-09-05-modellathon/presenting_a_model_at_dddbe.jpg" alt="Presenting a model at DDDBE" /></p>
<p>After each round, a couple of teams presented their progress. Some had UML-like drawings, others had mostly stickies with words, some had
drawn UI mockups as a starting point. It was quite wonderful to see how distinct each team’s model was, despite the short time frame.
Even after the first 25 minutes, it was surprising to see how much the teams had already captured of the problem space.</p>
<p>At the end, we had an improvised round table discussion, about the challenges involved with modeling complex domains, where many people
contributed their insights.</p>
<h3 id="what-ive-learned-as-a-facilitator">What I’ve learned as a facilitator</h3>
<h4 id="ask-the-right-question">Ask the right question</h4>
<p>In attempt to distribute people of different levels evenly across the teams, I asked who was an experienced modeler. No reaction.
(Belgians especially can be shy in this regard.)
<del>Somebody (I think it was Jef)</del> Tom then asked “who had previously screwed up a model”. Some people raised their hands, so that
proved a much more effective way to find the attendees with modeling experience!</p>
<h4 id="struggling">Struggling</h4>
<p>Somebody told me aftwerwards that he would have preferred to first get an overview of modeling techniques. On the other hand,
struggling with a problem is a really good way to learn. It triggers your instinct to look for better ways. And, when somebody
then shows you a new technique, you immediately recognize why it’s better. Perhaps, a good approach would be to have the teams
try it for themselves during one or two rounds, and then have a short presentation on one or two techniques they can try.
You’d probably need more time, 3-4 hours at least.</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2013-09-05-modellathon/tom_janssens_at_dddbe.jpg" alt="Presenting a model at DDDBE" /></p>
<h4 id="be-very-attentive">Be very attentive</h4>
<p>As a facilitator, it’s easy to get carried away with explaining the domain. It’s important to be aware of teams that have trouble
moving forward. It’s better to step out of the domain expert role, and help them move forward. Perhaps a good approach would be to
have different people in the expert and in the facilitator role (which is more or less what we did when Yves starting helping
out in the third round.</p>
<h4 id="throw-away-the-models">Throw away the models</h4>
<p>We gave teams the choice of working on the same model round after round. In retrospect, it might have been better to explicitly
ask them to throw everything away. This way, you give yourself the opportunity to start over with a different technique, borrowed
from another team’s presentation. Some teams became quite attached to their models, which limited their progress. <a href="http://c2.com/cgi/wiki?PlanToThrowOneAway">Kill your darlings!</a></p>
<h4 id="feedback-feedback-feedback">Feedback, Feedback, Feedback!</h4>
<p>Using the Pomodoro’s and having a few minutes of feedback proved extremely useful. I tended to provide the options, and have
the attendees choose, but soon I noticed that they came up with better ideas than me. So as a reminder to myself: the group
is smarter than the individual!</p>
<h3 id="conclusion">Conclusion</h3>
<p>It was a miracle that we had so much attendees for our brand new <a href="http://domaindriven.be">Belgian Domain-Driven Design community</a>,
and that all these people were prepared to run this experiment with us. In fact, I don’t think anything like a modellathon
was ever attempted anywhere else. Let me know if you do.</p>
<p>In any case, I can highly recommend running a modellathon of your own. And, incidentally, I’d love to come facilitate it :-)</p>
<h3 id="more">More</h3>
<ul>
<li>Belgian Domain-Driven Design community <a href="http://domaindriven.be">domaindriven.be</a></li>
<li><a href="http://twitter.com/DDDBE">@DDDBE</a> on Twitter</li>
<li>Read next: <a href="/2013/08/facilitating-event-storming/">Facilitating Event Storming</a></li>
</ul>
Antifragile - Things That Gain from Disorder2013-08-29T00:00:00+00:00https://verraes.net/2013/08/antifragile-nassim-nicholas-taleb<p>What is the opposite of <strong>fragile</strong> (breaks when confronted with unexpected events)? Changes are people answer with
<strong>robustness</strong> (stays the same under unexpected events). But there’s another way to look at it.</p>
<table class="table table-bordered">
<thead>
<tr>
<th> </th>
<th>Fragile</th>
<th>Robust</th>
<th>Antifragile</th>
</tr>
</thead>
<tbody>
<tr>
<th>Best case scenario</th>
<td>Remains unchanged</td>
<td>Remains unchanged</td>
<td>Improves</td>
</tr>
<tr>
<th>Worst case scenario</th>
<td>Breaks</td>
<td>Remains unchanged</td>
<td>Remains unchanged</td>
</tr>
</tbody>
</table>
<p>Taleb introduces the term <em>antifragile</em> for systems that don’t simply stay the same when subjected to stressors, volatility,
and exceptions, but that actually change for the better.</p>
<h2 id="wildfire">Wildfire</h2>
<p>We tend to naively intervene in systems, without taking into account their natural antifragility. Take wildfire. If we attempt to
stabilize - that is, prevent wildfires from happening - we believe we make the forest more robust. But over time, the combustible
materials in the forest accumulate. When they finally do catch fire, the resulting damage is much larger. Even though
the parts of the system, when left alone, are more fragile, the whole is antifragile. Just like bailing out banks can give us some
temporary perceived stability, but can lead to bigger disasters in the long term.</p>
<blockquote><p>Fire feeds on obstacles. <small>Marcus Aurelius</small></p></blockquote>
<h2 id="taxi-driver">Taxi driver</h2>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2013-08-29/antifragile-taxidriver.png" alt="Antifragile taxidriver" /></p>
<p>The book is full of great examples. Taleb tells of of a taxi driver and an office clerk. The clerk has a stable job:
he gets paid at the end of the month, and he gets a promotion every five years. The taxi driver on the other hand, has
good days and bad days. He’s constantly forced to be creative, experiment to find better spots to pick up customers, try
honest as well as less honourable ways to increase his income. Then 2008 hits, the clerk suddenly loses all of his income at once,
and finds himself on the street with no survival skills to speak of, and no experience with selling himself in interviews.
The taxidriver might take a hit from the recession, but his years of being subjected to highs and low, made him antifragile.</p>
<p>Even though small economic units may be fragile, the fragility of the parts can make the system as a whole more antifragile.
(Compare to John Gall’s “An airplane is a system that consists of parts that tend to fall down.”)</p>
<h2 id="the-book">The Book</h2>
<p><a href="http://www.amazon.com/gp/product/B0083DJWGO/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B0083DJWGO&linkCode=as2&tag=verraesnet-20&linkId=Q2UYVYAWOKWT7F5V">
<img style="float:left;margin-right: 10px" src="/img/posts/2013-08-29/antifragile-book.png" alt="Antifragile - Things That Gain from Disorder" />
</a><img src="http://ir-na.amazon-adsystem.com/e/ir?t=verraesnet-20&l=as2&o=1&a=B0083DJWGO" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /></p>
<p>I feel Antifragile is one of the most important books I’ve read in ages. Neither
my short explanation above, nor the quotes below do it justice. It’s hard to convey why it’s so gripping. Already I’m seeing
everything in my life in the light of the triad fragile - robust - antifragile.</p>
<h2 id="quotes">Quotes</h2>
<blockquote>The chief ethical rule is the following: Thou shalt not have antifragility at the expense of the fragility of others.</blockquote>
<blockquote>How do you innovate? First, try to get in trouble. I mean serious, but not terminal, trouble. I hold—it is
beyond speculation, rather a conviction—that innovation and sophistication spark from initial situations of necessity,
in ways that go far beyond the satisfaction of such necessity (...) The excess energy released from overreaction to setbacks is what innovates!</blockquote>
<blockquote>we can estimate, even measure, fragility and antifragility, while we cannot calculate risks and probabilities
of shocks and rare events, no matter how sophisticated we get. Risk management as practiced is the study of an event taking
place in the future, and only some economists and other lunatics can claim—against experience—to “measure” the future
incidence of these rare events, with suckers listening to them—against experience and the track record of such claims.
But fragility and antifragility are part of the current property of an object, a coffee table, a company, an industry, a
country, a political system. We can detect fragility, see it, even in many cases measure it, or at least measure comparative
fragility with a small error while comparisons of risk have been (so far) unreliable.</blockquote>
<blockquote>A complex system, contrary to what people believe, does not require complicated systems and regulations and
intricate policies. The simpler, the better. Complications lead to multiplicative chains of unanticipated effects. Because
of opacity, an intervention leads to unforeseen consequences, followed by apologies about the “unforeseen” aspect of the
consequences, then to another intervention to correct the secondary effects, leading to an explosive series of branching
“unforeseen” responses, each one worse than the preceding one.</blockquote>
<blockquote>We hear about the more lurid post-traumatic disorder, not post-traumatic growth, in the intellectual and so-called
learned vocabulary. But popular culture has an awareness of its equivalent, revealed in the expression “it builds character.”
So do the ancient Mediterranean classics, along with grandmothers.</blockquote>
<blockquote>In the complex world, the notion of “cause” itself is suspect; it is either nearly impossible to detect or not
really defined—another reason to ignore newspapers, with their constant supply of causes for things.</blockquote>
<blockquote>So some parts on the inside of a system may be required to be fragile in order to make the system antifragile
as a result. Or the organism itself might be fragile, but the information encoded in the genes reproducing it will be antifragile.
The point is not trivial, as it is behind the logic of evolution. </blockquote>
<blockquote>This is the central illusion in life: that randomness is risky, that it is a bad thing—and that eliminating
randomness is done by eliminating randomness.</blockquote>
<p>Read next: <a href="/2013/08/john-gall-systemantics-the-systems-bible/">Systemantics - The Systems Bible</a> - August 10, 2013</p>
Facilitating Event Storming2013-08-28T00:00:00+00:00https://verraes.net/2013/08/facilitating-event-storming<p>Event Storming is a technique where you get the developers and the business stakeholders in a room, and visualize the
business processes. You do this using stickies with domain events and causality (such as commands), and end with drawing
boundaries for aggregates, bounded contexts and subdomains. It’s developed by <a href="http://ziobrando.blogspot.be/">Alberto Brandolini</a> and is deeply rooted in
Domain Driven Design, and CQRS/ES.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2013-08-28/event-storming.jpg" alt="Event Storming at Qandidate.com" /></p>
<p>I facilitated an Event Storming session with the team of Qandidate.com in Rotterdam. I’ve read an early draft of Alberto’s
paper on Event Storming, that inspired most of what I did. It was my first time doing this as a facilitator, so I’d thought
I’d share my notes. It’s just the raw material – I was going to process them into something more
readable first, but <a href="http://www.jefclaes.be/">Jef Claes</a> convinced me to just post them as is. So forgive me if they are vague; I promise I will blog about
Event Storming more later.</p>
<p>Despite my rookie mistakes (too much intervention, then too little) the whole thing went very well. The team was very
excited, they want to do it again with their founders. We also did a bit of context mapping for the legacy systems of
their parent company, but time ran out.</p>
<h2 id="notes">Notes</h2>
<p>I think there are three main reasons for Model Storming:</p>
<ul>
<li>Discovering complexity early on, finding missing concepts, understanding the business process;</li>
<li>Modelling or solving a specific problem in detail;</li>
<li>Learning how to model and think about modelling.</li>
</ul>
<p>Facilitator tips:</p>
<ul>
<li>Hang the first sticky yourself (a tip from Alberto, works really well)</li>
<li>Know when to step back. Don’t do the modelling, guide the modelling</li>
<li>Ask questions:
<ul>
<li>Is there something missing here? Why is there a gap?</li>
<li>How does this make money?</li>
<li>How does the business evaluate that this is working? What are the targets, how will we know we’ve reached them?</li>
<li>For whom is this event of importance (end user, business, tenant,…) ?</li>
<li>I can’t see this particular role, or type of user, in this model. Should they be on here somewhere?</li>
</ul>
</li>
<li>Change the direction, e.g. start at the end of the flow, move back in time, then later start at the beginning and move forward.</li>
<li>Interrupt long discussions. Visualise both opinions, and, very important: ask both parties if they feel their opinion is accurately represented.</li>
<li>Timebox, using pomodoro’s (25 minutes). After each pomodoro, ask what is going well and what isn’t. It’s a good opportunity to move to the next phase (e.g. from adding events to adding causality, to drawing aggregate boundaries). You may want to move on even if you don’t feel the model is complete.</li>
<li>Constantly move stickies to create room for hotspots.</li>
<li>Hang red stickies with exclamation marks, question marks, or other notes, anywhere you feel there’s an issue.</li>
<li>At the end, make a photo. Then tell them to throw the model away, and to do it over the next day. If possible in the presence of other stakeholders.</li>
<li>I personally prefer sentence-style event names (“A product was added to a basket” vs “Product added to basket”). I believe this makes the business people feel more comfortable.</li>
</ul>
<h2 id="read-more">Read More</h2>
<ul>
<li><a href="/2014/07/event-storming-fake-domains-happy-paths/">Event Storming: on Fake Domains and Happy Paths</a></li>
<li><a href="/2015/03/event-storming-storytelling-visualisations/">Event Storming, Storytelling, Visualisations</a></li>
<li><a href="/2014/01/domain-driven-design-is-linguistic/">Domain-Driven Design is Linguistic</a></li>
<li><a href="/2013/09/dddbe-modellathon/">The DDDBE Modellathon</a></li>
<li><a href="/2014/01/model-storming-workshop/">Model Storming Workshop</a></li>
</ul>
Switch - How to Change Things When Change Is Hard2013-08-18T00:00:00+00:00https://verraes.net/2013/08/chip-dan-heath-switch<p><img style="float:left;margin-right: 10px" src="/img/books/switch.jpg" alt="Switch - How to Change Things When Change Is Hard" /></p>
<p>As I <a href="/2013/08/john-gall-systemantics-the-systems-bible/">wrote earlier</a>, I want to post some quotes and a mini-review of
the books I read. “<a href="http://www.amazon.com/Switch-Change-Things-When-Hard/dp/0385528752">Switch</a>” compares humans in
change processes to Rider (our logical side), riding an Elephant (our emotional side)
down a Path (the environment). By directing the Rider, motivating the Elephant, and shaping the Path, you can change yourself,
a team, an organization, or even society. As with all frameworks, it’s limited (which the authors admit upfront). However,
as a consultant who’s made it his professional ambition to help teams and organizations build better software, I definitely found
a lot of useful advise. If nothing else, the book will help me see which areas of a change process I haven’t paid sufficient
attention to. What most appealed to me, was the idea of finding the bright spots. We tend to focus on problems, but if we look at the
things people are doing that are in fact working, we can imitate and multiply those.</p>
<p><br /><br /><br /></p>
<h3 id="environment">Environment</h3>
<blockquote>What looks like a people problem is often a situation problem.</blockquote>
<p>(Contrast this with Weinberg’s «It’s always a people problem.» I don’t think the statements are mutually exclusive.)</p>
<blockquote>(...) people have a systematic tendency to ignore the situational forces that shape other people's behavior. He called
this deep-rooted tendency the "Fundamental Attribution Error." The error lies in our inclination to attribute people's
behavior to *the way they are* rather than to *the situation they are in*.</blockquote>
<h3 id="identity">Identity</h3>
<blockquote>Because identities are central to the way people make decisions, any change effort that violates someone's identity
is likely doomed to failure. (That's why it's so clumsy when people instinctively reach for 'incentives' to change other
people's behavior.) So the question is this: How can you make your change a matter of identity rather than a matter of
consequences?</blockquote>
<p>I can recall a number of occasions where a manager attempted to bribe a team into delivering faster, at lower quality,
even though the team was priding itself on its craftsmanship.</p>
<blockquote>If you want to change the culture of your organization, you've got to get the reformers together. (...) Counterintuitively,
you've got to let your organization have an identity conflict. For a time, at least, you've got to permit
an 'us versus them' struggle to take place. We know this violates our 'we're all on the same team,' Kumbaya-ish instincts.
It's not desirable, but it's necessary.</blockquote>
<blockquote>Also, *cognitive dissonance* works in your favor. People don't like to act in one way and think in another. So once a small
step has been taken, and people have begun to act in a new way, it will be increasingly difficult for them to dislike the
way they're acting. Similarly, as people begin to act differently, they'll start to think of themselves differently, and as
their identity evolves, it will reinforce the new way of doing things.</blockquote>
<h3 id="growth-mindset">Growth mindset</h3>
<blockquote>In the business world, we implicitly reject the growth mindset. Businesspeople think in terms of two stages: You plan,
and then you execute. There's no 'learning stage' or 'practice stage' in the middle. From the business perspective, practice
looks like poor execution. Results are the thing: *We don't care how ya do it, just get it done.*</blockquote>
<p>Learning is the bottleneck. Also see: «The management question, therefore, is not <em>whether</em> to build a pilot
system and throw it away. You will do that. (…) Hence plan to throw one away; you will, anyhow.» - Fred Brooks</p>
<blockquote>Everything can look like a failure in the middle.</blockquote>
Systemantics - The Systems Bible2013-08-10T00:00:00+00:00https://verraes.net/2013/08/john-gall-systemantics-the-systems-bible<p><a href="http://jonjagger.blogspot.com/">Jon Jagger</a> regularly posts snippets from books he’s read on his blog. I think this is an
excellent habit, that has helped me to discover some very interesting books, so I want to start doing the same.</p>
<h3 id="the-book">The book</h3>
<p><a href="http://www.amazon.com/dp/B00AK1BIDM/ref=r_soa_w_d">“Systemantics. The Systems Bible” by John Gall</a> is a classic work from 1975. It deals mostly with the idea that one cannot
simply design a large complex system, and expect to predict its entire set of behaviours. It’s one of those books that
help you see the world as it is. Although it mostly deals with failing systems and the dangers of complexity, it is,
almost inadvertently, a book on problem solving – even though it claims that “problems are not the problem. Coping is the problem.”</p>
<h3 id="quotes">Quotes</h3>
<p><img style="float:left;margin-right: 10px" src="/img/books/systemantics.jpg" alt="Systemantics - The Systems Bible" /></p>
<blockquote>In setting up a new System, tread softly. You may be disturbing another System that is actually working.</blockquote>
<blockquote>The word "Solution" is only a fancy term for the Response of System A (ourselves) to System B (the Problem). And its a
misleading word, because it implies something that can be done once and for all. But System B is sure to Kick Back in response
to our Response, and then we must respond once again.</blockquote>
<blockquote>As events get further and further ahead of the System's inappropriate responses, the Model of the Universe
generated in the control room by such Wishful Feedback bears less and less resemblance to outside reality.
The System *hallucinates* its way to Terminal Instability. In summary:
Just calling it "Feedback" doesn't mean that is had actually fed back. To speak precisely: It hasn't fed back
until the System changes course.</blockquote>
<blockquote>And when Anthropologists began to study tribes in their native habitats, they increasingly began to notice
that the typical New Guinea family consisted of father, mother, three children, and one or more Anthropologists.
In brief, there can be no System without its Observer.</blockquote>
<blockquote>Systems are seductive. They promise to do a hard job faster, better, and more easily than you could do it by yourself.
But if you set up a System, you are likely to find your time and effort now being consumed in the care and feeding
of the System itself. New Problems are created by its very presence. Once set up, it won't Go Away; it Grows
and Encroaches. It begins to do Strange and Wonderful Things and Breaks Down in Ways You Never Thought Possible.
It Kicks Back, Gets In The Way and Opposes Its Own Proper Function. Your own perspective becomes distorted by being In
The System. You become anxious and Push On It To Make It Work. Eventually you come to believe that the misbegotten product
it so grudgingly delivers is What You Really Wanted all the time. At that point, Encroachment has become complete. You have
become absorbed. You are now a Systems-person.</blockquote>
<blockquote>(..) persons or Systems in this unblessed state often adhere strongly to a bizarre Systems-delusion, the
Inevitability-of-Reality fallacy, the belief that: Thing have to be the way they are and not otherwise because that's
just the way they are.</blockquote>
<blockquote>The problem is a Problem precisely because it is incorrectly conceptualized in the first place, and a large System for
studying and attacking the Problem merely locks in the erroneous conceptualization into the minds of everyone concerned.
What is required is not a large System, but a *different approach*. Trying to design a System in the hope that the
System will somehow solve the Problem, rather than simply solving the Problem in the first place, is to present oneself
with two problems in place of one.</blockquote>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/01/build-for-failure/">Build for Failure</a></li>
<li><a href="/2013/08/antifragile-nassim-nicholas-taleb/">Antifragile - Things That Gain from Disorder</a></li>
</ul>
Managed Technical Debt2013-07-21T00:00:00+00:00https://verraes.net/2013/07/managed-technical-debt<p><strong>Update 2020-01-22 <a href="/2020/01/wall-of-technical-debt/">I wrote a new version of this</a>.</strong></p>
<p><a href="https://twitter.com/mathiasverraes/status/326435664319111170">Alberto Brandolini</a> said at IDDD Belgium (quoting from memory):</p>
<blockquote>We've always explained 'technical debt' badly to the business. If you have debt with a bank, you can talk to someone,
negotiate, and agree on a payment plan. But technical debt is like debt with the mob: they come at night, pointing a gun
to your head, and they want their money NOW.</blockquote>
<p><a href="http://codebetter.com/gregyoung/2013/03/06/startups-and-tdd/">Greg Young</a> on the other hand, has talked on several occasions about how technical debt is not necessarily bad. Businesses
take loans all the time. They are a powerful tool to make an investment now, and pay for it later. It’s a fundamental
aspect of our economy.</p>
<h3 id="shortcuts">Shortcuts</h3>
<p>I get very irritated when a non-technical person tells me to take shortcuts to meet a deadline. Or – even worse – when
a developer-turned-project-manager does so. Technical debt is often what made the project slow in the first place. And as
a consultant being more and more specialized in “problem projects”, I’ve seen a lot of crappy code that grinds projects
to a halt.</p>
<p>And yet, I myself tell teams to take shortcuts all the time. But there’s a difference. I know the codebase intimately.
I know where it’s problematic, and where it’s under control. I know what is covered with tests, which parts are hurting us,
and which parts are simply not elegant. And most importantly: when I introduce technical debt, I know how to get rid of
it, and I refactor as soon as I feel it’s needed. I add comments in the code suggesting how to change it, and often I
hang a sticky note in the backlog. Whenever new code is affected by earlier debt, I bump the priority of the backlog item.</p>
<p><img style="float:left;margin-right: 10px" src="/img/posts/2013-07-21-managed-technical-debt/map-of-technical-debt.jpg" alt="Mapped Technical Debt (bottom)" /></p>
<h3 id="team-discipline">Team discipline</h3>
<p>When opinions are at odds, like in the quotes above, there’s often a problem in the language. There’s one term, and two
concepts. In the interest of making
the implicit explicit, I’d like to propose that we make a clear distinction between <strong>Managed Technical Debt</strong>, and <strong>Unmanaged
Technical Debt</strong>. The former is defined as technical debt where most of the following conditions are present:</p>
<ul>
<li>The team is disciplined: it applies techniques such as refactoring, pair programming, iterative development, CI, TDD (or other testing schemes) …</li>
<li>The codebase is well structured, well tested, and stable.</li>
<li>When introducing new technical debt, the team estimates the cost/benefit.</li>
<li>Even though the introduced solution is not optimal, the team understands what the optimal solution would be, and how
the current solution can later be evolved towards that goal.</li>
<li>Technical debt is documented.</li>
<li>Technical debt is not introduced on top of existing technical debt.</li>
<li>Existing technical debt is paid off as soon as there’s a clear need.</li>
</ul>
<p>For Unmanaged Technical Debt on the other hand, most of the opposite conditions are present:</p>
<ul>
<li>The team lacks maturity.</li>
<li>Hacks are built as workarounds for the effects of previous hacks.</li>
<li>The team is unaware that a newly introduced solution is not optimal,</li>
<li>or the team is aware, but does not understand what an optimal solution would be or how to achieve it.</li>
</ul>
<h3 id="how-we-manage-it">How we manage it</h3>
<p>(Update September 12, 2013)</p>
<p>The photo above shows a project’s story map, and the bottom part is the map of technical debt, in columns per categories
(blue stickies). The green stickies are prioritized, and the priorities regularly change: whenever we find that something
makes us slower, the stickies move up. Sooner or later they move to the kanban board to be addressed in the sprint. We informally use the “three
strikes and you refactor” rule of thumb, but mostly it’s gut feeling, experience, and discussion that help us decide
whether something needs fixing – for example when we believe that future stories will be affected.</p>
<p>Visualizing the story map and the technical debt, is incidentally a great way to relieve stress: sure, there’s a lot of
work ahead, but it never feels unmanageable.</p>
<p>(Update July 22, 2014)</p>
<p>Some people have started using the technique after seeing my lightning talk about it. Some early pictures:</p>
<p><img style="width:340px; border: 1px solid gray" src="/img/posts/2013-07-21-managed-technical-debt/technical-debt-ramon.png" />
<img style="width:340px; border: 1px solid gray" src="/img/posts/2013-07-21-managed-technical-debt/technical-debt-pascal.png" /></p>
<p>Be sure to let me know your experience with if you’ve tried it as well!</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="https://verraes.net/2011/04/beautiful-code/">Beautiful Code</a></li>
<li><a href="/2014/07/managed-technical-debt-revisited/">Managed Technical Debt - Revisited</a></li>
<li><a href="/2016/04/repair-replace-heuristic-for-legacy-software/">The Repair/Replace Heuristic for Legacy Software</a></li>
</ul>
Unbreakable Domain Models2013-06-09T00:00:00+00:00https://verraes.net/2013/06/unbreakable-domain-models<h3 id="video">Video</h3>
<figure class="image is-16by9 loading">
<iframe class="has-ratio" width="640" height="360" src="https://www.youtube-nocookie.com/embed/ZJ63ltuwMaE" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen=""></iframe>
</figure>
<h3 id="slides">Slides</h3>
<script async="" class="speakerdeck-embed" data-id="838642907d6c0131238f2adccd741a3a" data-ratio="1.33333333333333" src="//speakerdeck.com/assets/embed.js"></script>
<h3 id="abstract">Abstract</h3>
<p>Data Mappers (like Doctrine2) help us a lot to persist data. Yet many projects are still struggling with tough questions:</p>
<ul>
<li>Where to put business logic?</li>
<li>How to protect our code from abuse?</li>
<li>Where to put queries, and how test them?</li>
</ul>
<p>Let’s look beyond the old Gang of Four design patterns, and take some clues from tactical Domain Driven Design. At the heart of our models, we can use Value Objects and Entities, with tightly defined consistency boundaries. Repositories abstract away the persistence. Encapsulated Operations helps us to protect invariants. And if we need to manage a lot of complexity, the Specification pattern helps us express business rules in the language of the business.</p>
<p>These patterns help us evolve from structural data models, to rich behavioral models. They capture not just state and relationships, but true meaning.</p>
<p>The presentation is a fast paced introduction to some patterns and ideas that will make your Domain Model expressive, unbreakable, and beautiful.</p>
<h2 id="learn-more">Learn More</h2>
<p>Check out <a href="http://elephantintheroom.io/blog/2013/10/episode-2-heart-and-soul-of-oop/">Elephant in the Room #002: The Heart and Soul of OOP</a> for more info about Value Objects.</p>
Decoupling (Symfony2) Forms from Entities2013-04-20T00:00:00+00:00https://verraes.net/2013/04/decoupling-symfony2-forms-from-entities<p>I usually tell people to stay away from the Symfony2 Form component, or forms libraries in general. The problem is that
all these libraries are designed in a very CRUD mindset. You’ve done all the work to keep your models, views, and controllers
nicely separated, and then this component comes along and violates every boundary.</p>
<p>The good news is that there is a solution, that allows you to cleanly separate your model from UI.</p>
<h3 id="another-example-of-encapsulating-operations">Another example of Encapsulating Operations</h3>
<p>In my post about the <a href="https://verraes.net/2013/04/crud-is-an-anti-pattern/">CRUD anti-pattern</a>, I showed how to get rid of setters, using the Encapsulate Operations pattern. Let’s now take the example of an Employee in an HR system.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$employee</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Employee</span><span class="p">;</span>
<span class="nv">$employee</span><span class="o">-></span><span class="nf">setName</span><span class="p">(</span><span class="s1">'Sarah Jones'</span><span class="p">);</span>
<span class="nv">$employee</span><span class="o">-></span><span class="nf">setPosition</span><span class="p">(</span><span class="s1">'Developer'</span><span class="p">);</span>
<span class="nv">$employee</span><span class="o">-></span><span class="nf">setSalaryScale</span><span class="p">(</span><span class="mi">3</span><span class="p">);</span>
<span class="c1">// a few years later</span>
<span class="nv">$employee</span><span class="o">-></span><span class="nf">setPosition</span><span class="p">(</span><span class="s1">'Lead Developer'</span><span class="p">);</span>
<span class="nv">$employee</span><span class="o">-></span><span class="nf">setSalaryScale</span><span class="p">(</span><span class="mi">4</span><span class="p">);</span>
</code></pre></div></div>
<p>Imagine we have a good talk with the client’s HR director, and we’ve learned two important things about the domain:</p>
<ul>
<li>Once an employee is entered into the system, the name never changes.</li>
<li>Promotions and Salary increases always go hand in hand.</li>
</ul>
<p>We want to reflect these new insights in our domain model. First, we get rid of the <code class="language-plaintext highlighter-rouge">setName()</code> method, and make sure the
name can never be changed, by passing it in the constructor:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$employee</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Employee</span><span class="p">(</span><span class="s1">'Sarah Jones'</span><span class="p">);</span>
</code></pre></div></div>
<p>Next we make sure that an employee always has a position and a salary scale, from the moment they are registered in the system.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$employee</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Employee</span><span class="p">(</span><span class="s1">'Sarah Jones'</span><span class="p">,</span> <span class="s1">'Lead Developer'</span><span class="p">,</span> <span class="nv">$salaryScale</span><span class="p">);</span>
</code></pre></div></div>
<p>We get rid of the other setters, and add a method that communicates that a promotion is always accompanied by a salary change:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$employee</span><span class="o">-></span><span class="nf">promote</span><span class="p">(</span><span class="nv">$newPosition</span><span class="p">,</span> <span class="nv">$newSalaryScale</span><span class="p">);</span>
</code></pre></div></div>
<p>Finally, we realize that in the real world, employees are not ‘instantiated’, they are hired. We make the constructor
private, and add a static method to make that explicit. The resulting code looks something like this:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">Employee</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$name</span><span class="p">;</span>
<span class="k">private</span> <span class="nv">$position</span><span class="p">;</span>
<span class="k">private</span> <span class="nv">$salaryScale</span><span class="p">;</span>
<span class="k">private</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$name</span><span class="p">,</span> <span class="nv">$position</span><span class="p">,</span> <span class="nv">$salaryScale</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">name</span> <span class="o">=</span> <span class="nv">$name</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">position</span> <span class="o">=</span> <span class="nv">$position</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">salaryScale</span> <span class="o">=</span> <span class="nv">$salaryScale</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">static</span> <span class="k">function</span> <span class="n">hire</span><span class="p">(</span><span class="nv">$name</span><span class="p">,</span> <span class="nv">$forPosition</span><span class="p">,</span> <span class="nv">$withSalaryScale</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="k">new</span> <span class="nc">self</span><span class="p">(</span><span class="nv">$name</span><span class="p">,</span> <span class="nv">$forPosition</span><span class="p">,</span> <span class="nv">$withSalaryScale</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">promote</span><span class="p">(</span><span class="nv">$toNewPosition</span><span class="p">,</span> <span class="nv">$withNewSalaryScale</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">position</span> <span class="o">=</span> <span class="nv">$toNewPosition</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">salaryScale</span> <span class="o">=</span> <span class="nv">$withNewSalaryScale</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nv">$employee</span> <span class="o">=</span> <span class="nc">Employee</span><span class="o">::</span><span class="nf">hire</span><span class="p">(</span><span class="s1">'Sarah Jones'</span><span class="p">,</span> <span class="s1">'Developer'</span><span class="p">,</span> <span class="mi">3</span><span class="p">);</span>
<span class="nv">$employee</span><span class="o">-></span><span class="nf">promote</span><span class="p">(</span><span class="s1">'Lead Developer'</span><span class="p">,</span> <span class="mi">4</span><span class="p">);</span>
</code></pre></div></div>
<h3 id="towards-a-task-based-ui">Towards a task based UI</h3>
<p>Now that we have our shiny model, we want to add a create and an edit form for the employee. We try using the
Symfony2 Form component, but very soon we get stuck. The component expects there to be a bunch of setters, and it doesn’t
know how to deal with constructor arguments, let alone private constructors and static factory methods.</p>
<p>We don’t want to reintroduce the setters, and screw up our domain model for the sake of the UI. But we don’t want to make
forms by hand. How do we get out of this situation?</p>
<p>Let’s think again about what the HR Director told us, about how they work. The department <strong>hires</strong> and employee, and later <strong>promotes</strong> them.
At no point he mentioned anything about <strong>creating</strong> and <strong>editing</strong> employees. Our own old-fashioned CRUD-thinking added those
words. So why are we still considering a “create” and an “edit” form? We really should be talking about a “hire” form and
and a “promote” form. This is a central idea in Task Based UI’s.</p>
<p>So we need a way to translate the result of our <code class="language-plaintext highlighter-rouge">HireEmployeeForm</code> and <code class="language-plaintext highlighter-rouge">PromoteEmployeeForm</code>, into our <code class="language-plaintext highlighter-rouge">Employee::hire()</code> and <code class="language-plaintext highlighter-rouge">$employee->promote()</code> methods.
This is starting to sound a lot like the Gang of Four Command pattern. As you may recall, a Command is an object that represents
all the information needed to call a method. To represent the <code class="language-plaintext highlighter-rouge">Employee::hire()</code> method, we use a <code class="language-plaintext highlighter-rouge">HireEmployeeCommand</code>:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">HireEmployeeCommand</span>
<span class="p">{</span>
<span class="k">public</span> <span class="nv">$name</span><span class="p">;</span>
<span class="k">public</span> <span class="nv">$forPosition</span><span class="p">;</span>
<span class="k">public</span> <span class="nv">$withSalaryScale</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>You can use public properties, or privates with getters and setters, that’s really a matter of taste here. Both are ok, because the Command
is just a very simple DTO object, with no behaviours. (I personally prefer getters and setters.)</p>
<p>Let’s add our <code class="language-plaintext highlighter-rouge">PromoteEmployeeCommand</code> as well:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">PromoteEmployeeCommand</span>
<span class="p">{</span>
<span class="k">public</span> <span class="nv">$employeeId</span><span class="p">;</span>
<span class="k">public</span> <span class="nv">$toNewPosition</span><span class="p">;</span>
<span class="k">public</span> <span class="nv">$withNewSalaryScale</span><span class="p">;</span>
<span class="p">}</span>
</code></pre></div></div>
<p>You’ll notice that the Command nicely matches with the form fields we need.</p>
<p>The next step is self-evident: Instead of making the Form component work off our Employee Entity, it now uses our Commands.
The Form and the Domain Model have no knowledge of each other, apart from the Commands. The Commands function as a clear,
explicit API that outsiders use to pass instructions to the Domain Model. You can use these Commands for your Symfony2
Form, but they can just as well be used for your REST API or other clients.</p>
<p>That leaves us the matter of getting the data from the Command into the Employee instance. We could do this in our Controller:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$employee</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-></span><span class="n">employeeRepository</span><span class="o">-></span><span class="nf">find</span><span class="p">(</span>
<span class="nv">$promoteEmployeeCommand</span><span class="o">-></span><span class="n">employeeId</span>
<span class="p">);</span>
<span class="nv">$employee</span><span class="o">-></span><span class="nf">promote</span><span class="p">(</span>
<span class="nv">$promoteEmployeeCommand</span><span class="o">-></span><span class="n">toNewPosition</span><span class="p">,</span>
<span class="nv">$promoteEmployeeCommand</span><span class="o">-></span><span class="n">withNewSalaryScale</span>
<span class="p">);</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">entityManager</span><span class="o">-></span><span class="nb">flush</span><span class="p">();</span>
</code></pre></div></div>
<p>A better solution is to move that particular code into a CommandHandler. Maybe something for a later blog post.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2015/02/form-command-model-validation/">Form, Command, and Model Validation</a></li>
<li><a href="https://verraes.net/2014/06/when-to-use-static-methods-in-php/">When to Use Static Methods</a></li>
</ul>
<h2 id="comments">Comments</h2>
<h3 id="cryptocompress----20130420"><a href="https://twitter.com/CryptoCompress">CryptoCompress</a> - 2013/04/20</h3>
<p>Very nice Blog! Thank You!</p>
<p>off-topic:</p>
<h4 id="static-function-hire-1">“static function hire()” (1):</h4>
<p>Static methods considered harmful. I use static-method-in-object instead in-factory for stateful objects too. Is this a
common approach? Maybe you have more information about it?</p>
<h4 id="static-function-hire-2">“static function hire()” (2):</h4>
<p>A company hire “instantiated” employees. Bonus points for <code class="language-plaintext highlighter-rouge">$company->hire($employee)</code> :D</p>
<h4 id="last-code-snippet">last code snippet</h4>
<p>There is no obvious connection between repository and entity manager.
<code class="language-plaintext highlighter-rouge">$this->employeeRepository->persist($employee);</code>
or
<code class="language-plaintext highlighter-rouge">$this->entityManager->employeeRepository->find();</code></p>
<h3 id="bernhard-schussek---20130421"><a href="https://twitter.com/webmozart">Bernhard Schussek</a> - 2013/04/21</h3>
<p>FYI, the Form component is able to deal with constructors by using the (empty_data)[http://symfony.com/doc/current/cookbook/form/use_empty_data.html] option.</p>
<p>Of course, if you don’t want to use any getters/setters, the automatic data mapping is pointless. It is there only as a
RAD helper to save some lines of code (especially when the entities also define validation constraints). But if you don’t
want that, your approach is a very good alternative. Another alternative would be to use an array-backed form.</p>
CRUD is an antipattern2013-04-19T00:00:00+00:00https://verraes.net/2013/04/crud-is-an-anti-pattern<p>CRUD is, as Greg Young calls it, our industry’s <a href="http://herdingcode.com/?p=189">Grand Failure</a>. We have deluded our
users, and ourselves, that applications are nothing more than a thin layer around our database. We present the user
with grids and edit forms, and ask them to fill in fields, that we map more or less directly to our relational tables.
We use terms like “data model”.</p>
<p>Real processes and businesses are not CRUD though. Real people don’t say “I’ve set the paid amount of the order to ‘120’,
the paid currency to ‘EUR’, and status to ‘paid’”. They say “I’ve paid €120 for the order”.</p>
<p>A good domain model is not a data model. A domain model describes behaviour, and the data is an artefact of that. Let’s
translate our user story to code.</p>
<h3 id="crud-thinking">CRUD thinking:</h3>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$order</span><span class="o">-></span><span class="nf">setStatus</span><span class="p">(</span><span class="s1">'paid'</span><span class="p">);</span>
<span class="nv">$order</span><span class="o">-></span><span class="nf">setPaidAmount</span><span class="p">(</span><span class="mi">120</span><span class="p">);</span>
<span class="nv">$order</span><span class="o">-></span><span class="nf">setPaidCurrency</span><span class="p">(</span><span class="s1">'EUR'</span><span class="p">);</span>
<span class="nv">$order</span><span class="o">-></span><span class="nf">setCustomer</span><span class="p">(</span><span class="nv">$customer</span><span class="p">);</span>
</code></pre></div></div>
<h3 id="step-1-replace-a-setter-with-an-expressive-method">Step 1: Replace a setter with an expressive method</h3>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$order</span><span class="o">-></span><span class="nf">pay</span><span class="p">();</span>
</code></pre></div></div>
<p>Note that internally, the Order object still stores the <code class="language-plaintext highlighter-rouge">status</code> field. That’s perfectly fine.</p>
<h3 id="step-2-replace-values-that-belong-together-with-a-value-object">Step 2: Replace values that belong together with a Value Object:</h3>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$money</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="p">(</span><span class="mi">120</span><span class="p">,</span> <span class="k">new</span> <span class="nc">Currency</span><span class="p">(</span><span class="s1">'EUR'</span><span class="p">));</span>
<span class="nv">$order</span><span class="o">-></span><span class="nf">setPaidMoney</span><span class="p">(</span><span class="nv">$money</span><span class="p">);</span>
</code></pre></div></div>
<p>I’ve written about Value Objects and Money elsewhere on this blog already.</p>
<h3 id="step-3-encapsulate-operation">Step 3: Encapsulate Operation</h3>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$order</span><span class="o">-></span><span class="nf">pay</span><span class="p">(</span><span class="nv">$customer</span><span class="p">,</span> <span class="nv">$money</span><span class="p">);</span>
</code></pre></div></div>
<p>It should be clear that the code in step 3 is a lot closer to demonstrating the behaviour and the actual purpose of our application.
We haven’t changed anything in the internal representation of our Order object. It still has fields like
<code class="language-plaintext highlighter-rouge">paidAmount</code>, <code class="language-plaintext highlighter-rouge">paidCurrency</code>, <code class="language-plaintext highlighter-rouge">customerId</code>, and <code class="language-plaintext highlighter-rouge">status</code>, that are mapped to the database. Order no longer exposes it’s
internals directly, but instead exposes a public api that is intention-revealing. It matches how users see our application:
as a set of behaviours, instead of a bunch of relational tables.</p>
<h3 id="bonus-points">Bonus points</h3>
<p>Optionally, we can make it even more expressive by stating that it’s the Customer who’s paying:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$customer</span><span class="o">-></span><span class="nf">pay</span><span class="p">(</span><span class="nv">$order</span><span class="p">,</span> <span class="nv">$money</span><span class="p">);</span>
</code></pre></div></div>
<p>The downside of this, is that we are coupling the Customer to the Order. This may or may not be desirable, depending on
context. In any case, it’s always worth considering.</p>
<h2 id="read-more">Read more</h2>
<ul>
<li><a href="/2014/09/decoupling-model-framework/">Decoupling the Model from the Framework</a></li>
<li><a href="/2013/04/decoupling-symfony2-forms-from-entities/">Decoupling Forms from Entities</a></li>
</ul>
<h2 id="comments">Comments</h2>
<h3 id="daniel-lo-nigro---20130420"><a href="http://dan.cx/">Daniel Lo Nigro</a> - 2013/04/20</h3>
<p>Hmm… I don’t know if CRUD is the right term here. Your <code class="language-plaintext highlighter-rouge">pay</code> method is only setting values and saving the entity,
and thus is still a CRUD operation. I think you’re complaining about using “raw” getters and setters as opposed to
methods that encapsulate the business or domain logic. There’s definite advantages to containing domain logic, but
domain logic and CRUD aren’t mutually exclusive.</p>
<h3 id="mathias-verraes---20130420"><a href="http://twitter.com/mathiasverraes">Mathias Verraes</a> - 2013/04/20</h3>
<p>Care to elaborate why you believe <code class="language-plaintext highlighter-rouge">$order->pay(...)</code> is still CRUD?</p>
<p>CRUD is the idea that everything in a model can be described using only four verbs: <code class="language-plaintext highlighter-rouge">create</code>, <code class="language-plaintext highlighter-rouge">read</code>, <code class="language-plaintext highlighter-rouge">update</code>, <code class="language-plaintext highlighter-rouge">delete</code>. In this
post, and in <a href="https://verraes.net/2013/04/decoupling-symfony2-forms-from-entities/">the next one</a> I show some examples where
I introduce new verbs: <code class="language-plaintext highlighter-rouge">pay</code>, <code class="language-plaintext highlighter-rouge">hire</code>, <code class="language-plaintext highlighter-rouge">promote</code>. Having domain logic in CRUD systems is possible, but it’s harder to express.
My point here is mostly that we should build applications that express user intent using a rich language with verbs that users
actually use. And that is in fact mutually exclusive with CRUD.</p>
<h3 id="daniel-lo-nigro---20130420-1"><a href="http://dan.cx/">Daniel Lo Nigro</a> - 2013/04/20</h3>
<p>I think that <code class="language-plaintext highlighter-rouge">$order->pay(...)</code> might still be CRUD because in the end, it’s just doing a single database update (like
<code class="language-plaintext highlighter-rouge">UPDATE order SET status = 'paid', amount = 120 WHERE order_id = 123</code>) and no other logic, which is a CRUD
operation. It’s a thin domain model abstraction of the CRUD operation, but it’s just a regular update statement with a
different name, and hence it’s still a CRUD operation. I wouldn’t consider it CRUD if it had more logic. I could be
totally wrong though!</p>
<p>And in any case, getters and setters aren’t CRUD, and calling <code class="language-plaintext highlighter-rouge">$order->setStatus</code> doesn’t automatically imply CRUD.</p>
<h3 id="mathias-verraes---20130420-1"><a href="http://twitter.com/mathiasverraes">Mathias Verraes</a> - 2013/04/20</h3>
<p>You’re right that setters don’t imply CRUD, but they are mostly seen together. The fact that the backend could use an UPDATE
SQL has nothing to do with the problem at hand though. Databases use primitive verbs, because they have no knowledge of the
meaning of the data.</p>
<p>Whether or not we have additional business logic in our methods, is a different matter. My example is oversimplified, and indeed doesn’t have
much logic in the <code class="language-plaintext highlighter-rouge">pay()</code> method. It would however be a perfect place to put some invariants: What if the paid amount is incorrect?
What if the currency is wrong? What if the order was already paid? This is all logic that is harder to find a place for in a
CRUD system.</p>
<h3 id="tobias-wooldridge---20130420"><a href="http://tobias.wooldridge.id.au/">Tobias Wooldridge</a> - 2013/04/20</h3>
<p>One of the most useful things about design patterns and anti-patterns is that, outside of highlighting problems and
solutions, they have a widely understood, unique name.</p>
<p>In this case, my confusion came about because CRUD is an inappropriate term for what you’re describing. The
anti-pattern definition unfortunately takes the word “CRUD” from the database implementation level of design and applies
it to the domain level.</p>
<p>Compared to names such as ‘God object’ and ‘Spaghetti code’, ‘CRUD’ is ambiguous. While I’m not a fan of
<a href="http://en.wikipedia.org/wiki/Object_orgy">other names given to this anti-pattern</a>, I don’t feel CRUD is an
improvement, as it fails to clearly identify the problem at hand.</p>
<h3 id="paul-mitchum---20130420"><a href="http://github.com/paul-m">Paul Mitchum</a> - 2013/04/20</h3>
<p>CRUD is a pattern that encapsulates certain behavior, related to persisting and managing data outside the script.</p>
<p>If you need that pattern, then use it. If you don’t, then don’t. :-)</p>
<p>Your customer doesn’t care whether you implement <code class="language-plaintext highlighter-rouge">$order->setStuff()->persist();</code> or
<code class="language-plaintext highlighter-rouge">$order->pay(new SemtanticallyCleanObjectName($currency, $amt));</code> Your work is to abstract these details away
for your customer, in exactly the same way that <code class="language-plaintext highlighter-rouge">setStuff()->persist();</code> abstracts CRUD away for you.</p>
<p>The real question is what your customer needs, and how you can deliver it to them in an efficient and maintainable way.
CRUD isn’t an anti-pattern working against this goal. Viewing it as the only required layer of abstraction is,
however, most likely a design error.</p>
<h3 id="rasmus-schultz---2013-04-22"><a href="http://twitter.com/mindplaydk">Rasmus Schultz</a> - 2013-04-22</h3>
<p>Hi Mathias,</p>
<p>Two things :-)</p>
<p>First, your post explains how to “replace” the code that uses the accessors directly to perform the change - but “replace” is not the right term here, that’s not what’s happening. You’re not “replacing” anything - you’re introducing better encapsulation for an identified type of transaction, and your code is now self-documenting and clearly reflects the fact that the code that uses the accessors is in fact a transcation. Your Order::pay() implementation will still contain the “replaced” code - and possibly other aspects of the transaction, such as logging, statistics, e-mail notifications, etc.</p>
<p>The code that uses the accessors of course should not go directly in a Controller - and I think that was your point? Although you never used the word Controller. But it has got to go somewhere, right? I think we agree on that much, and your post definitely points out something very important. So far so good :-)</p>
<p>Now secondly, you give “bonus points” for implementing the transaction method in the Customer entity instead of the Order entity. But this particular transaction depends on two things equally: an Order and a Transaction - while implementing the method in the Customer entity makes the controller-code read out more like english (“Customer pay order”) there is no logical reason why one is better than the other, and the semantics remain essentially the same.</p>
<p>You can find countless examples of this in the real world - for any transaction that involves more than one entity, you will have to make this decision. It gets particularly tricky when you have three or more entities involved in a transaction - which party is going to be responsible? And even worse, when one of those entities are optional in the same type of transaction. Of course, these are all questions you can answer, but your answers and thinking might not be the same as the next guy who has to work with your code.</p>
<p>That’s why I prefer to avoid those questions and use a service-oriented approach instead. Introduce a static PaymentService class to act as a mediator for that transaction. Or introduce an encapsulated PaymentTransaction object, since encapsulation was what you were questing for in the first place. This eliminates all of the above questions - the responsible party is now the transaction service or object, not an arbitrary entity. If an entity is optional in that transaction, make it obvious by allowing a required null-argument for that entity in the transaction service or object.</p>
<p>Adhering to this pattern has other advantages too - in particular, it enables composition: more than one type of service or transaction probably sends e-mail notifications or performs logging, so you can encapsulate those requirements in a base-class, and so on. Which furthermore helps with testing, since you can now inject a mock e-mail client or logger during tests.</p>
<p>But perhaps most importantly, it helps with perception - because your transactions are no longer scattered across entities, but encapsulated in services or transaction objects, it’s easier for somebody else to gain an overview of all the possible transactions in an application. It’s also easier to look at an entity and expect to find only methods that operate on the entity itself - rather than methods that depend on other entities. It scales better in terms of complexity, because each entity and service will have a fixed scope - rather than growing each class to meet new requirements, you introduce more classes, each with an isolated responsibility and fixed scope.</p>
<p>In the case of PHP specifically, it also marginally helps with performance, because you’re no longer loading transaction code that doesn’t get executed.</p>
<p>Maybe this is just way beyond the scope of your post :-)</p>
<p>And under any circumstances, having done this little write-up, I will probably end up posting that on my own blog. Feel free to post as comments on your blog though, if you wish. Though it’s almost longer than your blog-post at this point ;-)</p>
<p>I know, I do go on. Sorry about that.</p>
Casting Value Objects to strings2013-02-16T00:00:00+00:00https://verraes.net/2013/02/2013-02-16-casting-value_objects<p>I’m a big fan of Value Objects, as they are very helpful in encapsulating behavior, and communicating intent. In fact, as a
friend remarked, Value Objects are the heart and soul of Object Oriented Programming. Discussing the uses of Value Objects
is not what I want to talk about here though.</p>
<p>Let’s set up two simple examples first:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">TwitterHandle</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$handle</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$handle</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// Put some validation here, like a regex check</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">handle</span> <span class="o">=</span> <span class="nv">$handle</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">class</span> <span class="nc">DateRange</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$start</span><span class="p">,</span> <span class="nv">$stop</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">DateTime</span> <span class="nv">$start</span><span class="p">,</span> <span class="kt">DateTime</span> <span class="nv">$end</span><span class="p">)</span>
<span class="p">{</span>
<span class="c1">// Put some validation here, like checking if $end > $start</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">start</span> <span class="o">=</span> <span class="nv">$start</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">stop</span> <span class="o">=</span> <span class="nv">$end</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="simple-string-casting">Simple string casting</h3>
<p>Value Objects are one of the few places where using <a href="http://www.php.net/manual/en/language.oop5.magic.php#object.tostring">PHP’s magic __toString()</a>
makes perfect sense. For single-value Value Objects like the <code class="language-plaintext highlighter-rouge">TwitterHandle</code>, it’s obvious what should be in there. The
_toString() method simply returns the string representation.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">TwitterHandleTest</span> <span class="kd">extends</span> <span class="nc">PHPUnit_Framework_TestCase</span>
<span class="p">{</span>
<span class="cd">/** @test */</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">ItShouldCastToString</span><span class="p">()</span>
<span class="p">{</span>
<span class="nv">$twitterHandle</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">TwitterHandle</span><span class="p">(</span><span class="s1">'@mathiasverraes'</span><span class="p">);</span>
<span class="nv">$this</span><span class="o">-></span><span class="nf">assertEquals</span><span class="p">(</span>
<span class="s1">'@mathiasverraes'</span><span class="p">,</span>
<span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$twitterHandle</span><span class="p">,</span>
<span class="s2">"Casting a TwitterHandle to a string "</span>
<span class="mf">.</span><span class="s2">"should return the string version of the handle"</span>
<span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Implemented inside TwitterHandle, it looks like this:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">TwitterHandle</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$handle</span><span class="p">;</span>
<span class="c1">// constructor ....</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__toString</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="n">handle</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>That’s quite self-evident, but what about other cases?</p>
<h3 id="rules-for-casting-value-objects-to-string">Rules for casting Value Objects to string</h3>
<p>I had a couple of basic rules, that I have been using for a while for Value Objects. They were always implicit, but
talking about them in a pair programming session made them explicit.</p>
<h3 id="1-dont-use-_tostring-for-presentation">1. Don’t use _toString() for presentation.</h3>
<p>It’s tempting to use a __toString() method to render the Value Object in a View. More often
than not, you would need to put presentation logic in your Value Object, and that’s a big no-no. It’s better to perform
presentation rendering in a separate class or function, like a filter for your templating engine. It allows you to fine-tune
the rendering for your use case.</p>
<div class="language-html highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="nt"><span></span>
Find me on Twitter:
<span class="cp"><?php renderAsTwitterLink($user->getTwitterAccount()) ?></span>
<span class="nt"></span></span>
<span class="c"><!-- renders to: --></span>
<span class="nt"><span></span>
Find me on Twitter:
<span class="nt"><a</span> <span class="na">href=</span><span class="s">"https://twitter.com/mathiasverraes"</span> <span class="na">title=</span><span class="s">"Visit Twitter"</span><span class="nt">></span>
@mathiasverraes
<span class="nt"></a></span>
<span class="nt"></span></span>
</code></pre></div></div>
<p>Imagine we would have put the HTML rendering inside the TwitterHandle class; that would have been a sackable offense in
my book! You’ll notice that this is especially important with Value Objects like dates and monetaries, where the
rendering depends on the language.</p>
<h3 id="2-always-return-a-complete-representation-of-the-value-object">2. Always return a complete representation of the Value Object.</h3>
<p>Another temptation is to use string casting to get the numeric part of a measurement.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="cp"><?php</span>
<span class="nv">$aDistance</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Distance</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'meters'</span><span class="p">);</span>
<span class="nv">$this</span><span class="o">-></span><span class="nf">assertEquals</span><span class="p">(</span><span class="s1">'5'</span><span class="p">,</span> <span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$aDistance</span><span class="p">);</span> <span class="c1">// bad, don't do this</span>
</code></pre></div></div>
<p><code class="language-plaintext highlighter-rouge">'5'</code> is not a valid representation of a distance. Is it meters, centimeters, or inches? That information is now lost. If you start doing
calculations with that <code class="language-plaintext highlighter-rouge">'5'</code>, you may get nasty bugs. <a href="http://en.wikipedia.org/wiki/Mars_Climate_Orbiter">The kind of bugs that crash spacecrafts</a>.
So make sure that the string represents all of the Value Object:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="cp"><?php</span>
<span class="nv">$aDistance</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Distance</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'meters'</span><span class="p">);</span>
<span class="nv">$this</span><span class="o">-></span><span class="nf">assertEquals</span><span class="p">(</span><span class="s1">'5 meters'</span><span class="p">,</span> <span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$aDistance</span><span class="p">);</span> <span class="c1">// much better</span>
</code></pre></div></div>
<h3 id="3-return-a-parsable-string">3. Return a parsable string</h3>
<p>The most useful string representation you can come up with, is one that can be parsed back into the Value Object. For the
TwitterHandle example, that’s again very easy, so let’s use DateRange as an example:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="cp"><?php</span>
<span class="nv">$aDateRange</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">DateRange</span><span class="p">(</span>
<span class="k">new</span> <span class="nc">DateTime</span><span class="p">(</span><span class="s1">'2013-01-01'</span><span class="p">),</span>
<span class="k">new</span> <span class="nc">DateTime</span><span class="p">(</span><span class="s1">'2013-02-07'</span><span class="p">)</span>
<span class="p">);</span>
<span class="nv">$stringVersion</span> <span class="o">=</span> <span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="nv">$aDateRange</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="nf">assertEquals</span><span class="p">(</span><span class="s1">'2013-01-01 - 2013-02-07'</span><span class="p">,</span> <span class="nv">$stringVersion</span><span class="p">);</span>
<span class="nv">$parsedDateRange</span> <span class="o">=</span> <span class="nc">DateRange</span><span class="o">::</span><span class="nf">parse</span><span class="p">(</span><span class="nv">$stringVersion</span><span class="p">);</span>
<span class="nv">$this</span><span class="o">-></span><span class="nf">assertEquals</span><span class="p">(</span><span class="nv">$aDateRange</span><span class="p">,</span> <span class="nv">$parsedDateRange</span><span class="p">);</span>
</code></pre></div></div>
<p>So as you can see, <code class="language-plaintext highlighter-rouge">'2013-01-01 - 2013-02-07'</code> is a string representation of a DateRange, that can be parsed back into a DateRange object.
In pseudocode:</p>
<p><code class="language-plaintext highlighter-rouge">DateRange == parse(string(DateRange))</code></p>
<p>It works just as well for our other examples:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="cp"><?php</span>
<span class="nv">$this</span><span class="o">-></span><span class="nf">assertEquals</span><span class="p">(</span>
<span class="k">new</span> <span class="nc">Distance</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'meters'</span><span class="p">),</span>
<span class="nc">Distance</span><span class="o">::</span><span class="nf">parse</span><span class="p">(</span>
<span class="p">(</span><span class="n">string</span><span class="p">)</span> <span class="k">new</span> <span class="nc">Distance</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="s1">'meters'</span><span class="p">)</span>
<span class="p">)</span>
<span class="p">);</span>
<span class="c1">// leaving the implementation as an exercise for the reader :-)</span>
</code></pre></div></div>
<p>This will turn out to be very handy, especially when integrating with other systems, like a REST api or a database. But
that’s something we’ll cover in a future blog post.</p>
<p>Related posts:</p>
<ul>
<li><a href="/2011/04/fowler-money-pattern-in-php/">Representing Money in PHP, Fowler-style</a></li>
<li>Check out <a href="http://elephantintheroom.io/blog/2013/10/episode-2-heart-and-soul-of-oop/">Elephant in the Room #002: The Heart and Soul of OOP</a> for more info about Value Objects.</li>
</ul>
How to read more books2012-12-23T00:00:00+00:00https://verraes.net/2012/12/2012-12-23-how-to-read-more-books<p><img src="/img/posts/2012-12-23/reading_improves_your_view_of_the_world.jpg" alt="Reading improves your view of the world" /></p>
<p>“You can’t learn everything from books”, people say. It’s nothing but an excuse not to read books. It’s a straw man argument,
because no-one has ever claimed that you can. Or, “knowledge from experience can’t be taught in a book”. Even though that’s true,
there are many people out there with years of experiences that you will never have. Some of them are very good at sharing at least
some of that gained knowledge through writing. Books broaden your horizon, and deepen your insight.</p>
<p>With that out of the way, here are some of the tips that help me to read more books.</p>
<h3 id="get-glasses">Get glasses</h3>
<p>If reading makes you tired, gives you headaches, or is generally uncomfortable, you may need glasses.</p>
<h3 id="stop-watching-tv">Stop watching tv</h3>
<p>When people tell me they don’t have time to read books, I always ask if they have time to watch tv. You don’t have to stop entirely,
but only watch things that you consciously choose to watch.</p>
<ul>
<li>Don’t watch live television. Most programs last either 22 or 44 minutes, but that’s 30 or 60 minutes with commercials. Watch recorded tv instead.</li>
<li>Don’t zap. The ultimate time sink hole.</li>
<li>Don’t watch the news. It’s always the same. The little information you can get from a 22 minute news program, can be read online in 5 minutes. <a href="http://www.guardian.co.uk/media/2013/apr/12/news-is-bad-rolf-dobelli">Giving up news will make you happier</a>.</li>
</ul>
<h3 id="stop-reading-newspapers">Stop reading newspapers</h3>
<p>Newspapers and magazines compete by getting fatter year after year, with declining quality as the result. Everything you read will be outdated tomorrow anyway. And let’s be honest: first you read the funnies, then the headers, then some celebrity news. Do you really need a newspaper anymore?</p>
<blockquote>"Never read anything that isn't worth reading." — Gerald M. Weinberg</blockquote>
<h3 id="get-an-e-reader">Get an e-reader</h3>
<p><a href="http://www.amazon.com/gp/product/B007HCCNJU/ref=as_li_tl?ie=UTF8&camp=1789&creative=390957&creativeASIN=B007HCCNJU&linkCode=as2&tag=verraesnet-20&linkId=PAXYR27D7TJUIIG3">
Or get two, or three.
</a><img src="http://ir-na.amazon-adsystem.com/e/ir?t=verraesnet-20&l=as2&o=1&a=B007HCCNJU" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" />
They’re cheap, small, and light. They carry thousands of books. They’re a pleasure to read, because
they don’t blast bright light at you like iPads, monitors, and TV’s. Rid yourself of the romantic idea that books must
be touchable, sniffable, paper objects. They’re just dead trees collecting dust in your living room, and you’ll be
really sorry when you move to a new house and have to carry them down four flights of stairs. The content is what matters.</p>
<p><img style="" src="/img/posts/2012-12-23/matilda-roald-dahl.jpg" alt="Matilda" /></p>
<h3 id="read-everywhere">Read everywhere</h3>
<p>Kindle has apps for every device, and syncs your progress. When I’m standing in line in the supermarket, I read a couple of
pages on my phone. I read in bed, in the bathroom, on lunch breaks, in transit, and when I’m early for a meeting.</p>
<h3 id="make-it-a-habit">Make it a habit</h3>
<p>I read when I’m too tired to read. Obviously, some of it gets lost, and I often need to reread the last couple of pages
the next day. I do it because I feel that even reading only a little bit, keeps the habit alive. That’s also why I try to make
sure I always have my next book lined up. Some e-book stores give you the first chapter for free, which is great way to find
out if a book is worth reading.</p>
<h3 id="train-your-muscle">Train your muscle</h3>
<p>Like many things, the more you read, the easier it gets. Even if it’s hard in the beginning, if you read daily, you’ll find that
it gets easier, that you learn new concepts faster, that your brain gets more adapted to obtaining and processing information.</p>
<h3 id="keep-a-list">Keep a list</h3>
<p>Keep track of what you’ve read, on sites like <a href="https://kindle.amazon.com/">Kindle Reading List</a>, or <a href="http://www.goodreads.com/">GoodReads</a>.
You can share your list, follow others, turn it into a game. It’s very rewarding to watch your reading list grow, realizing that all that
knowledge is now in your head as well and makes you better at what you do.</p>
<p><img src="/img/posts/2012-12-23/adopt_books_small.jpg" alt="I don't read books, I adopt them" /></p>
<p>(Update September 10, 2013:)</p>
<h3 id="skip-to-a-better-book">Skip to a better book</h3>
<p><a href="https://verraes.net/2013/08/antifragile-nassim-nicholas-taleb/">Nassim N. Taleb</a> spent years reading 30-60 hours a week:</p>
<blockquote>"The minute I was bored with a book or a subject I moved to another one, instead of giving up on reading altogether (...)
The trick is to be bored with a specific book, rather than with the act of reading."</blockquote>
<h3 id="start-easy">Start easy</h3>
<p>If you have a backlog of books you want to read, you can start with an easy or short one and build your way up.
A finished book is a small victory that boosts your confidence, and prepares you for tougher tomes.</p>
<h3 id="listen-to-audiobooks">Listen to audiobooks</h3>
<p>I’d never given audiobooks much thought, but the last few months I’ve been driving a lot, so I wanted to spend the time well.
I finished my first two audiobooks – easy and short ones to begin with. I find that I’m more relaxed and focused when driving with
the gentle voice of a good narrator, than with blaring radio.</p>
<p>(Update December 22, 2013:)</p>
<h3 id="read-it-before-you-need-it">Read it before you need it</h3>
<p>In an ideal world, you’d read (technical) books just in time, precisely before you need the information. But you can’t plan learning like that.
You don’t know how much time you’ll need to read it, let alone to have it sink in. When you need information to finish a task
you’re working on, a book is often not the right choice.</p>
<p>A better approach is to read a book long before you might need it. Learning takes time, so giving your brain ample time to process the new
ideas, will give you a much better understanding. When the time comes to use the knowledge, a quick refresher will be all you need to
reactivate it in your brain.</p>
<h3 id="make-room-and-make-options">Make room and make options</h3>
<p><a href="http://www.noop.nl/2013/12/dont-make-goals-make-room-and-make-options.html">Jurgen Appelo</a> recommends not to set a target
amount of books to read. Targets lead to sub-optimisation. And how would you know what amount is ‘just right’? Focus on removing
waste (he cites watching TV and drinking in bars), and on making options (have a backlog and always bring reading material).</p>
<p>(Update June 24, 2014:)</p>
<h3 id="walk-n-read">Walk ‘n’ read</h3>
<p>I’ve always felt walking was good for thinking and getting the creative juices flowing – back when I was a film music composer, that’s how I came up with my best melodies. For the past few weeks, I’ve been trying something new: every morning, I take the same 45 minute walk, while reading on my Kindle. I’m taking small farmers’ roads between the fields, so there’s no traffic and it’s safe. It’s good for your health, so it’s a double win. And if reading on the couch makes you drowsy, walking is the perfect remedy.</p>
<h3 id="pomodoro">Pomodoro</h3>
<p><a href="https://twitter.com/tvlooy/status/481500378362818560">@tvlooy</a> uses the <a href="http://pomodorotechnique.com/">Pomodoro technique</a>: set an alarm, read for 25 minutes, take a 5 minute break away from the book or screen, repeat ad lib, with the same or a different book. For the full Pomodoro experience, keep track of the finished (uninterrupted) reading sessions, and set goals.</p>
<p>(Update June 30, 2014:)</p>
<p><img style="float:right;margin-left: 10px" src="/img/posts/2012-12-23/install-software-in-brain.jpg" alt="It's called reading. It's how people install new software into their brains" /></p>
<h3 id="dont-read-biographies">Don’t read biographies</h3>
<p>Although I’m sure some are valuable, don’t read books written <em>about</em> interesting people. Read books written <em>by</em> interesting people.</p>
<h3 id="dont-read-single-idea-books">Don’t read single-idea books</h3>
<p>“This book was based on a popular article in the New York Times”? Dead give away: one idea, lots of air. “The Three Secrets to…”, “Five steps to…”: It’s pretty likely that the book can be explained in a just a few paragraphs. Sometimes, a helpful Amazon customer review will in fact summarize the whole book for you. Read that instead.</p>
<p>(Update July 22, 2014:)</p>
<h3 id="read-what-gives-you-delight">Read what gives you delight</h3>
<p>From <a href="http://www.farnamstreetblog.com/2013/01/the-pleasures-of-reading-in-an-age-of-distraction/">The Pleasures of Reading in an Age of Distraction</a>:</p>
<blockquote>"Read what gives you delight — at least most of the time — and do so without shame. And even if you are that rare sort of person who is delighted chiefly by what some people call Great Books, don’t make them your steady intellectual diet, any more than you would eat at the most elegant of restaurants every day."</blockquote>
<p>(Update Feb 10, 2015:)</p>
<h3 id="riffle-the-pages">Riffle the pages</h3>
<p>I don’t know how much Gerald M. Weinberg <em>reads</em>, but he has <em>written</em> over 50 books. From “Weinberg on Writing”, I’m quoting a tip on reading nonfiction:</p>
<blockquote>"My own way (...) is to riffle the pages, scanning from back to front for something interesting that catches my eye, usually a picture or diagram of some sort. I start reading from there and continue until I lose interest. The I riffle again and repeat the process." </blockquote>
<h3 id="stop-means-stop">Stop means stop</h3>
<p>Also from “Weinberg on Writing”, similar to Taleb’s advice:</p>
<blockquote>"If you decide that it's not worth finishing, do you stop, or do you drag yourself through it? If you stop, do you *really* stop, or do you leave the book partly finished, as a nagging chore in the back of your mind?"</blockquote>
<p>Just remember that finishing a bad book is a waste of time and motivation. On the other hand, a great book is a window to a world you could never have found on your own. You still need to get out the door and discover that world. But that, dear reader, is out of reach for any book, and for this blog post.</p>
<p>(Update March 26, 2016:)</p>
<h3 id="diversify-before-re-reading">Diversify before Re-reading</h3>
<p>If you read a book you really want to (or need to) understand, but you’re struggling, it may seem tempting but drudging to re-read it. Find a different book on the same topic instead. Often the different point of view helps you grasp the original book.</p>
<p>(Update February 12, 2021:)</p>
<h3 id="the-sigert-method">The Sigert Method</h3>
<p>My friend Sigert went to music school at age 15, and didn’t learn much math or science there. Ten years later he found his passion after designing a small radio studio for the university. He wanted to become a freelance radio & music recording studio designer/advisor, but he knew he needed to get a lot better at acoustics than only relying on what his ears told him.</p>
<p>Sigert started reading what was essentially the bible of acoustics theory. As soon as he didn’t understand something (which happened at page 2 or so), he stopped and found a book to explain it. He worked his way back, from acoustics to waves to general physics to basic math. Then he moved forward again. By the time he got to the acoustics bible, not only was it a fast and easy read, but he was already helping clients, which he attracted through his blog on designing studios.</p>
<h3 id="what-are-your-reading-tips">What are your reading tips?</h3>
<p>Comment below.</p>
Code Folder Structure2011-10-22T00:00:00+00:00https://verraes.net/2011/10/code-folder-structure<p>All code bases I ever read or worked with, share a similar folder structure:</p>
<ul>
<li>Controllers
<ul>
<li>BlogPostController</li>
<li>CommentController</li>
</ul>
</li>
<li>Models
<ul>
<li>BlogPostModel</li>
<li>CommentModel</li>
</ul>
</li>
<li>Views
<ul>
<li>BlogPostsView</li>
<li>BlogPostDetailView</li>
</ul>
</li>
<li>Helpers
<ul>
<li>…</li>
</ul>
</li>
</ul>
<p>The ones where the developer has read Domain Driven Design, or is using Doctrine2 or Hibernate, usually have a better focus on the domain model:</p>
<ul>
<li>Model
<ul>
<li>Entities
<ul>
<li>BlogPost</li>
<li>Comment</li>
<li>User</li>
</ul>
</li>
<li>Repositories
<ul>
<li>BlogPostRepository</li>
<li>CommentRepository</li>
<li>UserRepository</li>
</ul>
</li>
<li>Services
<ul>
<li>UserService</li>
<li>…</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>The philosophy of these folder structures is usually inspired by the frameworks they use. After all, if your framework is organized like this, it must be a best practice, right? It does make really good sense for a framework to be organized in packages [modules, components, …] like this.</p>
<p>For your application, it’s a missed opportunity. You are not communicating the role each class has in relation to others, or the dependencies it has. A BlogPostRepository and a CommentRepository have no direct relation to each other, apart from the fact that they are both Repositories. However, a BlogPostRepository has a very tight dependency on BlogPost.</p>
<ul>
<li>BlogDomain
<ul>
<li>BlogPost
<ul>
<li>BlogPost</li>
<li>BlogPostRepository</li>
</ul>
</li>
<li>Comment
<ul>
<li>Comment</li>
<li>CommentRepository</li>
</ul>
</li>
</ul>
</li>
<li>CoreDomain
<ul>
<li>User
<ul>
<li>User</li>
<li>UserRepository</li>
</ul>
</li>
</ul>
</li>
</ul>
<p>This makes it a lot easier to communicate bounded contexts, and to illustrate dependencies. For example, the BlogDomain depends on the CoreDomain. On a smaller scale, the BlogPost package depends on the Comment package. Zooming in even further, BlogPostRepository depends on BlogPost.</p>
<p><img src="/img/posts/folderstructure1.png" alt="Folder structure" /></p>
<p>In other words: A BlogPost and a Comment know about their author. A BlogPost has zero or more Comments, but the Comments are not aware that they belong to BlogPost. A BlogPostRepository manages BlogPost entities, but those entities have no idea that they are being managed.</p>
<p>Obviously the whole example is too simple, as examples usually are. The point is that, to keep code clean, it’s important to think hard about coupling between elements. A folder structure aides to delineate depedencies. Close proximity in the tree suggests closer coupling. Documentation can further help to explain the direction of the coupling. We may just as well decide that Comments do know about the BlogPost they belong too, but that should be a conscious decision.</p>
<p><strong>Update August 18, 2014:</strong> I still agree with the general idea of this post, which is to group files or classes according to their cohesion in the domain and the model, as opposed to their underlying patterns or infrastructure. However, I wouldn’t use names like BlogDomain and CoreDomain anymore. Whatever your core domain is, it might change in the future. Businesses evolve. Authentication might be a better name. I also wouldn’t use the suffix “Domain” anymore. And finally, I would attempt to reduce the coupling between blogs and authentication, for example by only sharing a small set of Domain Events.</p>
Coping with Change in Software Development2011-07-21T00:00:00+00:00https://verraes.net/2011/07/coping-with-change-in-software-development<p>In a discussion about keeping developers motivated, someone said:</p>
<p>“I also try as hard as I can to isolate our developers […] from out-of-spec change requests”</p>
<p>Sticking to the plan sure allows developers and managers to stay in their comfort zone. The project keeps moving forward, the deadlines are met. But we’re not in the business of meeting deadlines. We’re in the business of building software that delivers value to customers. As the customer gains new insight in what will get the most value, the specs change. <a href="http://www.google.com/?q=change+is+the+only+constant">And change they will.</a></p>
<p>The reaction of the software industry has traditionally been to avoid change. Get the customer to sign off on the plan, then stick to it. Make them understand that changes are going to cost them dearly. Shield the developers from change.</p>
<p>The effect is backwards: customers, managers and developers are instilled with a holy fear of change. When, after much resistance, a change becomes inevitable, nobody knows how to deal with it.</p>
<p>Knowing how to cope with change in a software project is a skill like any other, whether it’s atomic refactoring, or big architectural changes, or writing the automated tests that will help you keep your sanity in the process.</p>
<p>Like any skill, you can master it with study and practice. Perhaps the best way to learn how to cope with change in software development, is to deliberately introduce change. It would make an interesting experiment: tell your developers to change things, just to keep them alert and well-trained in the art of change. I don’t know if they will declare you crazy, or if they will be motivated by it. All I know is that I would.</p>
Lazy Loading in PHP with Closures2011-05-17T00:00:00+00:00https://verraes.net/2011/05/lazy-loading-with-closures<p>Closures are a great way to do all kinds of neat tricks in PHP, and they’re particularly useful for Lazy Loading. I’m
currently involved in a +200k SLoC legacy project, and the challenge is moving it to a Domain Driven implementation
(while improving the performance), with the ultimate goal of making it more testable.</p>
<h3 id="the-problem">The problem</h3>
<p>We want to find a Customer, and ask it for a list of Orders:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="c1">// client code</span>
<span class="nv">$customer</span> <span class="o">=</span> <span class="nv">$customerRepository</span><span class="o">-></span><span class="nf">find</span><span class="p">(</span><span class="nv">$id</span><span class="p">);</span>
<span class="nv">$orders</span> <span class="o">=</span> <span class="nv">$customer</span><span class="o">-></span><span class="nf">getOrders</span><span class="p">();</span>
</code></pre></div></div>
<p>With the ActiveRecord pattern, this is simple. The Customer object holds an instance of the database adapter, and queries
it for related Orders:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">Customer</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">getOrders</span><span class="p">()</span>
<span class="p">{</span>
<span class="nv">$ordersData</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-></span><span class="n">db</span><span class="o">-></span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select orders... */</span><span class="p">);</span>
<span class="nv">$orders</span> <span class="o">=</span> <span class="k">array</span><span class="p">();)</span>
<span class="k">foreach</span><span class="p">(</span><span class="nv">$ordersdata</span> <span class="k">as</span> <span class="nv">$orderdata</span><span class="p">)</span> <span class="p">{</span>
<span class="nv">$orders</span><span class="p">[]</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Order</span><span class="p">(</span><span class="nv">$orderdata</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nv">$orders</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The downside of ActiveRecord, is that it violates the principle of Separation of Concerns. The Customer class contains
domain knowledge (“What is a customer, how does it behave?”), as well as persistence knowledge (“How do we store a customer?”).
The Customer class in our example even knows how to find it’s Orders in the database. The tight coupling between Customer
and the database makes it less portable, and hard to test.</p>
<p>In DDD, we solve this by keeping the Customer class pure, and move the logic for storing the object to a CustomerRepository.
Clients of the Repository don’t know how or where it finds Customers, and the Customer class itself doesn’t know anything
about the Repository or the database it is stored in. As for the Orders, they are pushed in the Customer at creation time.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">Customer</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">getOrders</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="n">orders</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">class</span> <span class="nc">CustomerRepository</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">find</span><span class="p">(</span><span class="nv">$id</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$customerdata</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-></span><span class="n">db</span><span class="o">-></span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select customer ...*/</span><span class="p">);</span>
<span class="nv">$customer</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Customer</span><span class="p">(</span><span class="nv">$customerdata</span><span class="p">);</span>
<span class="nv">$ordersdata</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-></span><span class="n">db</span><span class="o">-></span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select orders ... */</span><span class="p">);</span>
<span class="k">foreach</span><span class="p">(</span><span class="nv">$ordersdata</span> <span class="k">as</span> <span class="nv">$orderdata</span><span class="p">){</span>
<span class="nv">$customer</span><span class="o">-></span><span class="nf">addOrder</span><span class="p">(</span><span class="k">new</span> <span class="nc">Order</span><span class="p">(</span><span class="nv">$orderdata</span><span class="p">));</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Note that the client code for this example is still the same as in the first snippet.</p>
<h3 id="adding-lazy-loading-to-the-mix">Adding Lazy Loading to the mix</h3>
<p>The problem with the previous example is that we always query the database for the Orders, even when we don’t need them.
We don’t want to move that query back to the Customer class, but we want to keep our client code intact. The trick is to move the logic for finding Orders into a Closure, push it into the Customer instance, and execute only when we actually need the Orders. In other words, Customer now holds a reference to the Orders, and only dereferences it at the very last moment – hence the term Lazy Loading.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">Customer</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">setOrdersReference</span><span class="p">(</span><span class="kt">Closure</span> <span class="nv">$ordersReference</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">ordersReference</span> <span class="o">=</span> <span class="nv">$ordersReference</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">getOrders</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">if</span><span class="p">(</span><span class="o">!</span><span class="k">isset</span><span class="p">(</span><span class="nv">$this</span><span class="o">-></span><span class="n">orders</span><span class="p">))</span> <span class="p">{</span>
<span class="nv">$reference</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-></span><span class="n">ordersReference</span><span class="p">;</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">orders</span> <span class="o">=</span> <span class="nv">$reference</span><span class="p">();</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="n">orders</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="kd">class</span> <span class="nc">CustomerRepository</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">find</span><span class="p">(</span><span class="nv">$id</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$db</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-></span><span class="n">db</span><span class="p">;</span>
<span class="nv">$customerdata</span> <span class="o">=</span> <span class="nv">$db</span><span class="o">-></span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select customer ...*/</span><span class="p">);</span>
<span class="nv">$customer</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Customer</span><span class="p">(</span><span class="nv">$customerdata</span><span class="p">);</span>
<span class="nv">$ordersReference</span> <span class="o">=</span> <span class="k">function</span><span class="p">(</span><span class="nv">$customer</span><span class="p">)</span> <span class="k">use</span><span class="p">(</span><span class="nv">$id</span><span class="p">,</span> <span class="nv">$db</span><span class="p">)</span> <span class="p">{</span>
<span class="nv">$ordersdata</span> <span class="o">=</span> <span class="nv">$db</span><span class="o">-></span><span class="nf">query</span><span class="p">(</span><span class="cm">/* select orders ... */</span><span class="p">);</span>
<span class="nv">$orders</span> <span class="o">=</span> <span class="k">array</span><span class="p">();</span>
<span class="k">foreach</span><span class="p">(</span><span class="nv">$ordersdata</span> <span class="k">as</span> <span class="nv">$orderdata</span><span class="p">)</span> <span class="p">{</span>
<span class="nv">$orders</span><span class="p">[]</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Order</span><span class="p">(</span><span class="nv">$orderdata</span><span class="p">);</span>
<span class="p">}</span>
<span class="k">return</span> <span class="nv">$orders</span><span class="p">;</span>
<span class="p">};</span>
<span class="nv">$customer</span><span class="o">-></span><span class="nf">setOrderReference</span><span class="p">(</span><span class="nv">$ordersReference</span><span class="p">);</span>
<span class="k">return</span> <span class="nv">$customer</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The client code is still exactly as in the very first code snippet. But this time, when we call getOrders(), the Closure
we prepared in find() is executed. It’s self-contained: it has the database instance and the Customer’s id, and it knows
what to do. Customer on the other hand has no idea what goes on inside the Closure, but can perfectly deliver on getOrders() calls.</p>
<h3 id="conclusion">Conclusion</h3>
<p>Clearly this is a simplified example. My colleague, who has a lot more experience with the project, pointed out a whole
bunch of real life use cases where this method would cause an abundance of queries. I’ll keep you posted when we figure
out how to deal with those.</p>
<h2 id="comments">Comments</h2>
<h3 id="alessandro-nadalin---20110517">Alessandro Nadalin - 2011/05/17</h3>
<p>One of the good things of doctrine 1 was that it used lazy loading by default.
That meant if you didn’t want to do N bunch of separated queries you only needed to do a ->leftJoin() when quering a Doctrine_Table (which, conceptually, is similar - but far away - to the repository).</p>
<p>But it was active record… gosh :)</p>
<p>BTW, i’d suggest not to inject the whole instance of the DB abstraction into the closure, but something like a query object ( Doctrine_Query, for instance )</p>
<p>In order to avoid “a whole bunch of real life use cases where this method would cause an abundance of queries” you should not use a ->find(), which should be relation-agnostic, but some other pre-defined methods like:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">UserRepository</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">getUserWithFriends</span><span class="p">(</span><span class="nv">$userId</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="nf">createQuery</span><span class="p">(</span><span class="s1">'UserTable'</span><span class="p">)</span><span class="o">-></span><span class="nf">leftJoin</span><span class="p">(</span><span class="s1">'friends'</span><span class="p">)</span><span class="o">-></span><span class="nf">blablabla</span><span class="p">();</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="herman-peeren---20110517">Herman Peeren - 2011/05/17</h3>
<p>My idea about Lazy Loading:</p>
<p>The Lazy Loading in this example only is more efficient when the total number of hits to getOrders() is less than the number of customer-objects. So, it depends on the situation when to use Lazy Loading or not.</p>
<p>To be able to take this design decision, you need to know in what situations it will be used: the “art” of programming.</p>
<h3 id="mathias-verraes---20110517">Mathias Verraes - 2011/05/17</h3>
<p>Exactly. I failed to mentioned that normally, you’d use an ORM like Doctrine2 to do this kind of stuff. Unfortunately the project is too large, too complex, and too critical to refactor to Doctrine2 (or at least not in one go).</p>
<h3 id="herman-peeren---20110517-1">Herman Peeren - 2011/05/17</h3>
<p>But, as far as I know, Doctrine2 ALWAYS uses Lazy Loading. And my point is: there might be situations where that is not the best choice. So, we have a lot of ease of use when using an ORM-framework, but will sometimes get a lack of performance.</p>
<p>An advantage when you need to program everything yourself, is that you can still take the decision yourself to use Eager or Lazy Loading. As Johan Cruijff (Dutch Ajax Champion) would say: “every disadvantage has it’s advantage”.</p>
Ubiquitous Language2011-05-12T00:00:00+00:00https://verraes.net/2011/05/ubiquitous-language<p>Customers usually have never been forced to really think about their domain in a structured way – let alone explain it in detail to a developer, who doesn’t know anything about that domain. Often the customer doesn’t have a set of clear definitions for the concepts his business uses daily, or has multiple terms for the same concepts.</p>
<p>Developers are even worse. They also tend to have vague concepts and lots of ways to name the same thing. Depending on where you look in the code or the documentation (if any), you might see: “bank account”, “account”, “bank_account”, “bankAccount”, “BankAccount”. And worse: “BA”, “bank_acc”, “acnt”. When storage is involved, you’ll find “item”, “row”, “record”, “model”. And another set of words for a collection of bank accounts: “list”, “array”, “collection”, “bankaccounts”, “acntset”, … you get the picture.</p>
<p>Even on small projects, this hinders communication. Developers, project managers, customers, and others involved with the project, need to communicate about the domain, but everybody uses a different, but equally vague terminology.</p>
<h3 id="a-common-language">A common language</h3>
<p>The overcome this, you should start building a ubiquitous language, from the moment you start talking to the customer. Use a single term for a single concept, be it a thing, a relationship, or a behavior. Use the exact same term in the code, in the documentation, and in speech, and give it a strict spelling, down to the capitalization. Always “BankAccount”, never “bank account”. Don’t use abbreviations, as they make everything harder to read (except for really common ones, like “ISBN”).</p>
<p>You need to be disciplined about it, and expect the same from everybody involved: correct the customer when he uses a different word, fix the spelling when someone writes it down incorrectly, and change other’s code. Some people might get annoyed at first at your insistence, but they will soon get comfortable with using the language.</p>
<p>Maintain this ubiquitous language throughout the project. When concepts change, or your understanding of the domain changes, adjust the language, make sure it is adapted everywhere, and tell everyone.</p>
<h3 id="benefits">Benefits</h3>
<p>You’ll notice that soon, communication between everyone in the project will go much easier. When something is unclear, you can refer to the documentation. For new people on the project, it will be a lot easier to get up to speed. Code becomes easier to read, and easier to understand, as there are less concepts, and they are clearly defined. The application is much less vulnerable to regressions, and changing or refactoring is a breeze.</p>
Beautiful code2011-04-05T00:00:00+00:00https://verraes.net/2011/04/beautiful-code<p>Code doesn’t have to be beautiful. Some of the most successful projects have the ugliest code. Marketing, features, vendor lock-in, shiny GUI’s and all kinds of real or imagined qualities are what make or break a software product. Of all the factors that’ll make you reach your audience, beautiful code dangles somewhere at the bottom. The business value of beautiful code is near zero. CEO’s, managers, customers, and users, they don’t give a shit about the code. So neither should we. Just keep pumping out those LOC’s.</p>
<p>But we do care. All good programmers care about beautiful code. A well-crafted code base makes us happy to go to work every day. It makes us confident that anything they’ll throw at us, can be handled with grace.</p>
<p>And even if the code isn’t there yet, just having the opportunity to improve it gives us energy. Making all kinds of small, hardly visible changes, and over time seeing how all the wrinkles and hiccups disappear, is like watching natural evolution produce the perfect flower.</p>
<p>A last-minute change request from a customer is no problem for beautiful code. We welcome it as a chance to prove that the painstaking refactoring was worth it. We’re thrilled at how ridiculously easy it has become to respond to change.</p>
<p>The world is littered with garbage. So let’s pick up the trash in our own neighbourhood. Leave every piece of code behind a little cleaner than you found it. Beautiful code does make a difference.</p>
<p>Read next: <a href="https://verraes.net/2013/07/managed-technical-debt/">Managed Technical Debt</a> - July 21, 2013</p>
Representing Money in PHP, Fowler-style2011-04-04T00:00:00+00:00https://verraes.net/2011/04/fowler-money-pattern-in-php<p>Whenever working with values in object oriented programming, it’s often a good idea to wrap them in a ValueObject. Money is a perfect candidate for a ValueObject: When talking about money, numbers are meaningless if they are not combined with a currency.</p>
<p>I’ve been using a very simple version of the Money pattern as described in <a href="http://martinfowler.com/books.html">Martin Fowler’s PoEAA</a>. I couldn’t find a PHP implementation anywhere, so I decided to make my own little open source library for it. You can find it on <a href="https://github.com/mathiasverraes/money">my GitHub account</a> (where else?).</p>
<h3 id="immutability">Immutability</h3>
<p>An important aspect of ValueObjects is their immutability:</p>
<p>Let’s say Jim and Hannah both want to buy a copy of book priced at EUR 25.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$jim_price</span> <span class="o">=</span> <span class="nv">$hannah_price</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="p">(</span><span class="mi">2500</span><span class="p">,</span> <span class="k">new</span> <span class="nc">Euro</span><span class="p">);</span>
</code></pre></div></div>
<p>Jim has a coupon for EUR 5.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$coupon</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="p">(</span><span class="mi">500</span><span class="p">,</span> <span class="k">new</span> <span class="nc">Euro</span><span class="p">);</span>
<span class="nv">$jim_price</span><span class="o">-></span><span class="nf">subtract</span><span class="p">(</span><span class="nv">$coupon</span><span class="p">);</span>
</code></pre></div></div>
<p>Because <code class="language-plaintext highlighter-rouge">$jim_price</code> and <code class="language-plaintext highlighter-rouge">$hannah_price</code> are the same object, you’d expect Hannah to now have the reduced price as well. To prevent this problem, Money objects are immutable. With the code above, both <code class="language-plaintext highlighter-rouge">$jim_price</code> and <code class="language-plaintext highlighter-rouge">$hannah_price</code> are still EUR 25:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$jim_price</span><span class="o">-></span><span class="nf">equals</span><span class="p">(</span><span class="nv">$hannah_price</span><span class="p">);</span> <span class="c1">// true</span>
</code></pre></div></div>
<p>The correct way of doing operations is:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$jim_price</span> <span class="o">=</span> <span class="nv">$jim_price</span><span class="o">-></span><span class="nf">subtract</span><span class="p">(</span><span class="nv">$coupon</span><span class="p">);</span>
<span class="nv">$jim_price</span><span class="o">-></span><span class="nf">lessThan</span><span class="p">(</span><span class="nv">$hannah_price</span><span class="p">);</span> <span class="c1">// true</span>
<span class="nv">$jim_price</span><span class="o">-></span><span class="nf">equals</span><span class="p">(</span><span class="nc">Money</span><span class="o">::</span><span class="nf">euro</span><span class="p">(</span><span class="mi">2000</span><span class="p">));</span> <span class="c1">// true</span>
</code></pre></div></div>
<h3 id="allocation">Allocation</h3>
<p>My company made a whopping profit of 5 cents, which has to be divided amongst myself (70%) and my investor (30%). Cents can’t be divided, so I can’t give 3.5 and 1.5 cents. If I round up, I get 4 cents, the investor gets 2, which means I need to conjure up an additional cent. Rounding down to 3 and 1 cent leaves me 1 cent. Apart from re-investing that cent in the company, the best solution is to keep handing out the remainder until all money is spent. In other words:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$profit</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Money</span><span class="p">(</span><span class="mi">5</span><span class="p">,</span> <span class="k">new</span> <span class="nc">Euro</span><span class="p">);</span>
<span class="k">list</span><span class="p">(</span><span class="nv">$my_cut</span><span class="p">,</span> <span class="nv">$investors_cut</span><span class="p">)</span> <span class="o">=</span> <span class="nv">$profit</span><span class="o">-></span><span class="nf">allocate</span><span class="p">(</span><span class="mi">70</span><span class="p">,</span> <span class="mi">30</span><span class="p">);</span>
</code></pre></div></div>
<p>Now <code class="language-plaintext highlighter-rouge">$my_cut</code> is 4 cents, and <code class="language-plaintext highlighter-rouge">$investors_cut</code> is 1 cent. The order in which you allocate the the money is important:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="k">list</span><span class="p">(</span><span class="nv">$investors_cut</span><span class="p">,</span> <span class="nv">$my_cut</span><span class="p">)</span> <span class="o">=</span> <span class="nv">$profit</span><span class="o">-></span><span class="nf">allocate</span><span class="p">(</span><span class="mi">30</span><span class="p">,</span> <span class="mi">70</span><span class="p">);</span>
</code></pre></div></div>
<p>Now <code class="language-plaintext highlighter-rouge">$my_cut</code> is 3 cents, and <code class="language-plaintext highlighter-rouge">$investors_cut</code> is 2 cents.</p>
<h3 id="conclusion">Conclusion</h3>
<p>At the moment my Money class has all the features that Fowler lists. You can see some more example by looking at the <a href="https://github.com/mathiasverraes/money/tree/master/tests">unit tests</a>. I hope to add a lot stuff more if time permits: currency conversion, parsing of strings like ‘$2.00’ and ‘USD 2.00’, dealing with major units and subunits in currencies, etc. In any case, I hope it’s useful to somebody in it’s present form.</p>
Random thoughts on using Git2011-03-29T00:00:00+00:00https://verraes.net/2011/03/git-vs-svn<p>A couple of weeks ago, I switched from Subversion to Git for a couple of smaller projects. Below are some of my findings.</p>
<h3 id="switching-from-subversion">Switching from Subversion</h3>
<p>Git is a little harder to understand than Subversion, because of it’s lack of a centralized repository, and because it has more concepts you need to understand. But if you already understand SVN well, it’s very manageable. I’ve no idea how the learning curve compares to SVN for a complete newbie to version control.</p>
<p>If you are on the fence, there are two things that I wish I knew about sooner:</p>
<ul>
<li><a href="http://gitimmersion.com/">Git Immersion</a> only takes an hour or so, but you’ll understand so much more of git than when using conventional books or tutorials.</li>
<li><a href="http://help.github.com/svn-importing/">Migrating an SVN repo to git</a> is dead easy.</li>
</ul>
<h3 id="open-source-community">Open source community</h3>
<p>GitHub alone is plenty of reason to switch to Git. It seems like the whole PHP community (and the whole open source community) are moving to GitHub. And you can’t blame them. GitHub works really, really well. It makes it easy to maintain your own fork of a project, make changes and send them as pull requests. Symfony2 for example noticed a huge increase in contributions since the move.</p>
<p>Even if you only consume open source code and don’t plan on contributing patches, you’ll find you need to understand at least how cloning and pulling work, so you can easily download code.</p>
<h3 id="svnexternals">Svn:externals</h3>
<p>I usually include third party libraries in my projects using svn:externals. At the moment, a lot of PHP libraries that switched to git, still sync to SVN. But I’m sure that people will stop bothering with that. That’s a problem if your SVN repository depends externals. You could set up your own syncing, or take the hint and move to git.</p>
<h3 id="fixing-oops-moments">Fixing ‘oops’ moments</h3>
<p>With SVN, commits are pretty much written in stone. Because of that, a convention has emerged:</p>
<ul>
<li>Faulty commits are followed by a commit labeled ‘reverted rev 123’</li>
<li>Incomplete commits are followed by a commit with the same message</li>
</ul>
<p>Git has a lot more possibilities to fix errors, amend to commits, etc. That’s a blessing and a curse. I find that by trying to make my history look pretty, I seem to make a mess of things. It’s not a big deal, because in Git, nothing is ever lost. Perhaps it’s just my lack of experience with git. In any case, I think that for now, I will stick to the SVN convention and won’t try to hide my clumsiness from the world.</p>
Accessing private properties from other instances2011-03-24T00:00:00+00:00https://verraes.net/2011/03/accessing-private-properties-from-other-instances<p>In PHP, when a property or method is marked private, it can only be accessed from within that class. That includes <strong>other instances of the same class</strong>. This may seem counter-intuitive at first, because we are used to dealing with instances of classes. The visibility operator however works not on object-level, but on class level.</p>
<p>An example:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">Foo</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$private</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="nv">$value</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="k">private</span> <span class="o">=</span> <span class="nv">$value</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">getOther</span><span class="p">(</span><span class="kt">Foo</span> <span class="nv">$object</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$object</span><span class="o">-></span><span class="k">private</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nv">$foo1</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Foo</span><span class="p">(</span><span class="s1">'foo1'</span><span class="p">);</span>
<span class="nv">$foo2</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">Foo</span><span class="p">(</span><span class="s1">'foo2'</span><span class="p">);</span>
<span class="k">echo</span> <span class="nv">$foo1</span><span class="o">-></span><span class="nf">getOther</span><span class="p">(</span><span class="nv">$foo2</span><span class="p">);</span> <span class="c1">// outputs 'foo2'</span>
</code></pre></div></div>
<p>This should make it clear that both instances of Foo have access to each other’s private properties.</p>
<p>What practical use does this have? A great candidate for this are <a href="http://domaindrivendesign.org/node/135">Value Objects</a>. If we want to make sure that to separate instances of Foo are actually equal, we can easily compare their private properties:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">Foo</span>
<span class="p">{</span>
<span class="c1">// ...</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">equals</span><span class="p">(</span><span class="kt">Foo</span> <span class="nv">$other</span><span class="p">)</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="k">private</span> <span class="o">===</span> <span class="nv">$other</span><span class="o">-></span><span class="k">private</span><span class="p">;</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="c1">// ...</span>
<span class="k">echo</span> <span class="nv">$foo1</span><span class="o">-></span><span class="nf">equals</span><span class="p">(</span><span class="nv">$foo2</span><span class="p">)</span> <span class="o">?</span> <span class="s1">'Equal'</span> <span class="o">:</span> <span class="s1">'Different'</span><span class="p">;</span>
</code></pre></div></div>
<h2 id="comments">Comments</h2>
<h3 id="nicholas-k-dionysopoulos---20110324">Nicholas K. Dionysopoulos - 2011/03/24</h3>
<p>Nice info, Mathias! It’s very counter-intuitive indeed. On the other OO languages I know of (Delphi, various .NET dialects) a private variable is inaccessible outside the context of the owner object instance.</p>
<h3 id="jonathan-mayhak---20110324">Jonathan Mayhak - 2011/03/24</h3>
<p>Good info. Had no idea php worked like this.</p>
<p>Also, great use case….very practical!</p>
<h3 id="lolol---20110324">Lolol - 2011/03/24</h3>
<p>This is also the case in Java, which i found rather awkward when i found it.. but apparently it has some use. kind of takes away the principle behind private if you can extend a class to create functions to read another instance of that base class` privates :)</p>
Keep your controllers thin with Doctrine22011-03-23T00:00:00+00:00https://verraes.net/2011/03/keep-you-controllers-thin-with-doctrine2<p>Doctrine2 does such a nice job abstracting everything related to the database, that you might be tempted to do everything else in your controllers. Say we have a Bug entity:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="cd">/** @Entity */</span>
<span class="kd">class</span> <span class="nc">Bug</span>
<span class="p">{</span>
<span class="cd">/** @Column(type="integer") */</span>
<span class="k">private</span> <span class="nv">$id</span><span class="p">;</span>
<span class="cd">/** @Column(length=50) */</span>
<span class="k">private</span> <span class="nv">$status</span><span class="p">;</span>
<span class="c1">//...</span>
<span class="p">}</span>
</code></pre></div></div>
<p>To get a list of fixed bugs, we get the Bug repository from the EntityManager and ask for a list of Bugs where status equals ‘fixed’.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="o"><?</span><span class="n">php</span>
<span class="c1">// $em instanceof Doctrine\ORM\EntityManager</span>
<span class="nv">$fixedbugs</span> <span class="o">=</span> <span class="nv">$em</span><span class="o">-></span><span class="nf">getRepository</span><span class="p">(</span><span class="s1">'Bug'</span><span class="p">)</span>
<span class="o">-></span><span class="nf">findBy</span><span class="p">(</span><span class="k">array</span><span class="p">(</span><span class="s1">'status'</span> <span class="o">=></span> <span class="s1">'fixed'</span><span class="p">));</span>
</code></pre></div></div>
<p>That’s easy enough. Surely there can be no harm in having this code inside a controller? Although this code doesn’t look like one, it is in a fact a database query. It’s a shortcut for this:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$fixedbugs</span> <span class="o">=</span> <span class="nv">$em</span>
<span class="o">-></span><span class="nf">createQuery</span><span class="p">(</span><span class="s2">"SELECT b FROM Bug b WHERE b.status = 'fixed'"</span><span class="p">)</span>
<span class="o">-></span><span class="nf">getResult</span><span class="p">();</span>
</code></pre></div></div>
<p>Having a query in our controller should ring some serious alarm bells. It means that despite all abstraction Doctrine2 provides, we are at this point still coupling the controller to the database. If one day we decide to change how bug status is represented in the database, we’d need to modify all our controllers.</p>
<p>Let’s have a closer look at the Repository. <a href="http://domaindrivendesign.org/books/evans_2003">Evans</a> defines it as “an object that can provide the illusion of an in-memory collection of all objects of that type”, that clients talk to using the domain language. In other words, findBy(array(‘status’ => ‘fixed’)) is too generic: in domain language, we want to ask the repository to findAllFixedBugs(). If the database schema changes, we’ll only have to change that method. Luckily Doctrine2’s repositories can be extended:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="cd">/**
* @Entity(repositoryClass="BugRepository")
*/</span>
<span class="kd">class</span> <span class="nc">Bug</span>
<span class="p">{</span> <span class="cm">/* ... */</span> <span class="p">}</span>
<span class="kd">class</span> <span class="nc">BugRepository</span> <span class="kd">extends</span> <span class="nc">EntityRepository</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">findAllFixedBugs</span><span class="p">()</span>
<span class="p">{</span>
<span class="k">return</span> <span class="nv">$this</span><span class="o">-></span><span class="n">_em</span>
<span class="o">-></span><span class="nf">createQuery</span><span class="p">(</span><span class="s2">"SELECT b FROM Bug b WHERE b.status = 'fixed'"</span><span class="p">)</span>
<span class="o">-></span><span class="nf">getResult</span><span class="p">();</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Finally we can replace the code in our controller with this</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$fixedbugs</span> <span class="o">=</span> <span class="nv">$em</span><span class="o">-></span><span class="nf">getRepository</span><span class="p">(</span><span class="s1">'Bug'</span><span class="p">)</span><span class="o">-></span><span class="nf">findAllFixedBugs</span><span class="p">();</span>
</code></pre></div></div>
<p>Our controller is now decoupled from the database. As an added bonus, the code does a much better job of communicating it’s intent than our first version ever could.</p>
<h2 id="comments">Comments</h2>
<h3 id="torkil-johnsen---20120323">Torkil Johnsen - 2012/03/23</h3>
<p>Looks extremely nifty, but left me with questions. I’m not familiar with Doctrine, and you probably just wrote this as a crude example, so forgive the silly question! :)</p>
<p>I’m wondering why you write something as specific as findAllFixedBugs()? Seems to me like you’ll need a findAllFixedBugsFilteredByUser() and findAllFixedBugsWithStatusCritical() around the next bend?</p>
<p>And why did you not use findBy() in your final version of the method, but instead write a regular SQL statement?</p>
<p>Other than that: Step up the pace on the blog posts, we need more of these :)</p>
<h3 id="mathias-verraes----20120323">Mathias Verraes - 2012/03/23</h3>
<p>It’s indeed a crude example.</p>
<p>You would indeed need more methods, which you would discover as you write tests (see my previous post). The point of having these methods is that your code is speaking the same language as you speak with your customer (called ‘ubiquitous language’ in DDD). If you decide to move your bugs from a relational database to, say, a remote service, your methods would still be there, but the implementation would be entirely different. And of course, you can easily replace BugRepository with a mock object during testing, so you wouldn’t need a database to test your controller.</p>
<p>There’s no particular reason I didn’t use findBy() in the last snippet, except that I wanted to illustrate that the method can contain complex queries that can’t be done with find by.</p>
<p>Off topic: the queries in the example are DQL, not SQL. http://docs.doctrine-project.org/projects/doctrine-orm/en/2.0.x/reference/dql-doctrine-query-language.html</p>
<h3 id="alessandro-nadalin---20120323">Alessandro Nadalin - 2012/03/23</h3>
<p>Hi Torkil,</p>
<p>yes, the repository patterns was specifically born to incapsulate all the methods you need to retrieve objects from the DB.
So you basically add there all the possible queries you need in your application…</p>
<p>You are obviously able to abstract some methods using a custom class between your repositories and the EntityRepository, which inherits from the latter.</p>
<p>To use OO api and not raw DQL statements you need a QueryBuilder instance:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="nv">$qb</span> <span class="o">=</span> <span class="nv">$em</span><span class="o">-></span><span class="nf">getRepository</span><span class="p">(</span><span class="s1">'Odino\BlogBundle\Entity\Content'</span><span class="p">)</span><span class="o">-></span><span class="nf">createQueryBuilder</span><span class="p">(</span><span class="s1">'u'</span><span class="p">)</span>
<span class="o">-></span><span class="nf">where</span><span class="p">(</span><span class="s1">'u.isActive = 1'</span><span class="p">)</span>
<span class="o">-></span><span class="nf">andWhere</span><span class="p">(</span><span class="s2">"u.keywords LIKE ?1"</span><span class="p">)</span>
<span class="o">-></span><span class="nf">setParameter</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="s2">"%</span><span class="nv">$tag</span><span class="s2">%"</span><span class="p">);</span>
<span class="nv">$qb</span><span class="o">-></span><span class="nf">getQuery</span><span class="p">()</span><span class="o">-></span><span class="nf">execute</span><span class="p">();</span>
</code></pre></div></div>
<p>ciao!</p>
<h3 id="torkil-johnsen---20120323-1">Torkil Johnsen - 2012/03/23</h3>
<p>Thanks to you both, enlightening!</p>
Interface discovery with PHPUnit’s Mock objects2011-03-21T00:00:00+00:00https://verraes.net/2011/03/interface-discovery-with-phpunit-mock-objects<p><a href="https://github.com/sebastianbergmann/phpunit/">PHPUnit</a> provides some great features to create <a href="http://en.wikipedia.org/wiki/Mock_object">mock objects</a>. The idea is that when you are testing code that depends on another class, you provide the object with a mock instance of that class, instead of a real object. That way, you are making sure that your test will only fail if the system under test is broken, and not if one of it’s dependencies is broken. You could simply write a mock class and instantiate it, but PHPUnit can generate them for you.</p>
<p>The PHPUnit documentation doesn’t explicitly state this, but you can also create mock objects from interfaces. This makes a lot of sense if you think about it. In many cases, you should actually use mocked interfaces in your tests instead of mocked concrete classes. After all, the interface is the contract by which classes agree to talk to the outside world.</p>
<h3 id="a-simple-example">A simple example</h3>
<p>Let’s write some code, in true <a href="http://en.wikipedia.org/wiki/Test-driven_development">TDD</a> style. Let’s say we want to post to Twitter whenever someone deposits money in our bank account. We don’t want the test to actually send out tweets. In fact, we haven’t even thought about what our Twitter class will look like. This is our test:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">BankAccountTest</span> <span class="kd">extends</span> <span class="nc">PHPUnit_Framework_TestCase</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">testSendEmailWhenReceivingMoney</span><span class="p">()</span>
<span class="p">{</span>
<span class="nv">$twitter</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-></span><span class="nf">getMock</span><span class="p">(</span><span class="s1">'Twitter'</span><span class="p">);</span>
<span class="nv">$account</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">BankAccount</span><span class="p">(</span><span class="nv">$twitter</span><span class="p">);</span>
<span class="nv">$account</span><span class="o">-></span><span class="nf">deposit</span><span class="p">(</span><span class="mi">10</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>Running this test fails, as we haven’t got a BankAccount class yet.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PHP Fatal error: Class 'BankAccount' not found
</code></pre></div></div>
<p>Let’s add it:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">BankAccount</span>
<span class="p">{</span>
<span class="k">private</span> <span class="nv">$twitter</span><span class="p">;</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">__construct</span><span class="p">(</span><span class="kt">Twitter</span> <span class="nv">$twitter</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">twitter</span> <span class="o">=</span> <span class="nv">$twitter</span><span class="p">;</span>
<span class="p">}</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">deposit</span><span class="p">(</span><span class="nv">$amount</span><span class="p">){</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The test now succeeds, for the simple reason we are not really testing anything. Let’s make sure that BankAccount::deposit() actually sends out a tweet. We do this by telling the mock to expect a call to it’s tweet() method.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">BankAccountTest</span> <span class="kd">extends</span> <span class="nc">PHPUnit_Framework_TestCase</span>
<span class="p">{</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">testSendEmailWhenReceivingMoney</span><span class="p">()</span>
<span class="p">{</span>
<span class="nv">$twitter</span> <span class="o">=</span> <span class="nv">$this</span><span class="o">-></span><span class="nf">getMock</span><span class="p">(</span><span class="s1">'Twitter'</span><span class="p">);</span>
<span class="nv">$twitter</span><span class="o">-></span><span class="nf">expects</span><span class="p">(</span><span class="nv">$this</span><span class="o">-></span><span class="nf">once</span><span class="p">())</span>
<span class="o">-></span><span class="nf">method</span><span class="p">(</span><span class="s1">'tweet'</span><span class="p">);</span>
<span class="nv">$account</span> <span class="o">=</span> <span class="k">new</span> <span class="nc">BankAccount</span><span class="p">(</span><span class="nv">$twitter</span><span class="p">);</span>
<span class="nv">$account</span><span class="o">-></span><span class="nf">deposit</span><span class="p">(</span><span class="mi">10</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>The test fails with the following message:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>1) BankAccountTest::testSendEmailWhenReceivingMoney
Expectation failed for method name is equal to <string:tweet>
when invoked 1 time(s)
Method was expected to be called 1 times, actually called 0 times.
</code></pre></div></div>
<p>Let’s add some code that calls tweet() to our deposit() method.</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">class</span> <span class="nc">BankAccount</span>
<span class="p">{</span>
<span class="c1">// ...</span>
<span class="k">public</span> <span class="k">function</span> <span class="n">deposit</span><span class="p">(</span><span class="nv">$amount</span><span class="p">)</span>
<span class="p">{</span>
<span class="nv">$this</span><span class="o">-></span><span class="n">twitter</span><span class="o">-></span><span class="nf">tweet</span><span class="p">(</span><span class="s2">"Yay, someone deposited </span><span class="nv">$amount</span><span class="s2">"</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</code></pre></div></div>
<p>We get a new error:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PHP Fatal error:
Call to undefined method Mock_Twitter_28053312::tweet()
</code></pre></div></div>
<p>Mock_Twitter_28053312 is the class that PHPUnit generated based on the Twitter interface, which we haven’t written yet. The good news is that by now, we have discovered what the interface should look like:</p>
<div class="language-php highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="cp"><?php</span>
<span class="kd">interface</span> <span class="nc">Twitter</span>
<span class="p">{</span>
<span class="k">function</span> <span class="n">tweet</span><span class="p">(</span><span class="nv">$message</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>
<h3 id="conclusion">Conclusion</h3>
<p>The whole point of this technique, is that we have now written BankAccount, without worrying how our Twitter implementation will work. In the process, we discovered that we will need an implementation of a Twitter interface, and we discovered what that interface looks like.</p>
<p>Later on, we probably build a RESTfulTwitter implements Twitter, which would post messages using the Twitter REST API. If we had build the Twitter implementation first, we’d probably would have been tempted to add all kinds of features that we thought we might need. Instead of that, we discovered what we actually needed. Our test proves that our code adheres to <a href="http://en.wikipedia.org/wiki/You_ain't_gonna_need_it">YAGNI</a>.</p>
<p>Read next: <a href="/2013/09/sensible-interfaces/">Sensible Interfaces</a> - September 24, 2013</p>