Patterns for Decoupling in Distributed Systems: Fat Event

Add redundant information to a Domain Event to reduce complexity in the consumer.

Published on 11 May 2019 by @mathiasverraes

Fat Event

Add redundant information to a Domain Event to reduce complexity in the consumer.

Problem

A consumer is interested in one event type from a producer, to react to it or report information to a user. The producer’s events are designed with a Completeness Guarantee. The event only contains the attributes that have changed, and none of the other state of the resources that the consumer is interested in. So the consumer has to listen to a number of other events as well, that each contain some of the state changes of the resource. Most of the work the consumer is now doing, is building state based on events, and doing lookups to associate things with each other. There’s a lot of complexity in the consumer, just to achieve a small task.

Solution

A Fat Event is an event that not only contains the attributes that have changed, but also a number of attributes that have not changed at the time the event is emitted. These are specifically chosen to lighten the work that the consumer has to do: it doesn’t have to persist intermediate states from other event types, as everything it needs to do its core job is the one (or few) Fat Events.

The consumer now needs to be aware of fewer event types, and is therefore better isolated from changes in the producer.

Example

A consumer wants to display a list of overdue invoices. It listens to InvoiceBecameOverdue {invoiceId}. This event only contains the invoiceId, but the user needs to see the customer’s name as well (and other informationbut let’s keep this example simple). So the consumer listens to CustomerWasInvoiced {invoiceId, customerId, amount, lineItems...}. Now the consumer knows which customer the invoice belongs to, but to get the name, it needs to listen to CustomerHasSignedUp {customerId, customerName, ...} and CustomerWasRenamed {customerId, newCustomerName}. The consumer keeps lookup tables:

class OverdueInvoices implements Projector
    when(CustomerHasSignedUp e)
        INSERT INTO CustomerNames (e.customerId, e.customerName)
    when(CustomerWasRenamed e)
        UPDATE CustomerNames (e.customerId, e.customerName)
    when(CustomerWasInvoiced e)
        INSERT INTO Invoices (e.invoiceId, e.customerId)
    when(InvoiceBecameOverdue e)
        INSERT INTO OverdueInvoices (
            SELECT invoiceId, customerName FROM Invoices i 
            LEFT JOIN CustomerNames cn ON cn.customerId = i.customerId
            WHERE invoiceId = e.invoiceId
        )
    when(WhichInvoicesAreOverdue q) 
        SELECT * FROM OverdueInvoices

In the new design, we adapt the producer first. InvoiceBecameOverdue will now carry the attribute customerName as well. The consumer becomes very simple:

class OverdueInvoices implements Projector
    when(InvoiceBecameOverdue e)
        INSERT INTO OverdueInvoices (e.invoiceId, e.customerName)
    when(WhichInvoicesAreOverdue q) 
        SELECT * FROM OverdueInvoices

Anti-pattern

All patterns become anti-patterns when applied incorrectly, but Fat Events certainly have their share of risks.

Adapting the API of a producer to fit the needs of the consumer, should always be done with care. With lots of consumers that each have their own specific demands, it could quickly become mess. Events become huge, and as consumers come and go, it becomes unclear wich attributes are still relevant. Changing the producer becomes harder. This is exactly the kind of situation that the Completeness Guarantee pattern was supposed to avoid. A useful heuristic is to consider the number of consumers and their ownership. If a single team owns both the producer and consumer, Fat Events pose less of a risk than if a large number of teams each want different redundant attributes in the messages.

Eventual consistency and immutability

In many cases, there’s a risk that the consumer’s state becomes inconsistent with the producer’s state. The invoice example above is chosen to illustrate this. In the second design, where we use the fat InvoiceBecameOverdue event, suppose the customer is renamed after the invoice became overdue. Now the consumer has a list of overdue invoices with some outdated customer names. In the original design, this problem did not exist, because there it doesn’t matter if the customer name changes before or after the invoice became overdue, the state is always updated.

This inconsistency may not be a problem. If, say, only a very small number of customers do in fact change their name, and the invoies only stay unpaid for a short while, the chances of actual inconsistency become slim. And in the rare case it happens, the user might see the old name in one place and the new name in another place, and work out easily that it’s the same invoice. As always, it helps to carefully consider the domain, the users, their expectations and behaviours.

A very powerful heuristic is that immutable values are always safe to copy. If in the producer, a value can never change after it was initially set, there is no risk that the copy in the consumer becomes inconsistent. In our example, if we don’t allow customers to change their name, the Fat Event that carries the name is a fine solution to simplify the consumer.

Identifiers

A special case of immutability are identifiers. In a well-designed system, IDs never change. So assuming that the customers can change their name, but that all IDs are stable, the InvoiceBecameOverdue event can be designed to contain only invoiceId and customerId, but no customerName. The consumer now needs to track customer names again, but doesn’t have to deal with a separate Invoices lookup table.

Because of the safety of copying IDs, as a rule of thumb I recommend to always include redundant IDs in events. The cost is extremely low, and it simplifies consumers. It becomes especially important when dealing with multiple relations between the entities described by the events.

Consider:

CustomerHasSignedUp {customerId, customerName}
OrderWasPlaced {orderId, customerId}
InvoiceWasMadeForOrder {invoiceId, orderId}
PaymentWasReceived {paymentId, invoiceId}

In this example, a consumer interested in payment, needs three lookup tables to associate a customer name with a payment. With Fat Events, each event has the redundant, but immutable, IDs, and only a single lookup table is required:

CustomerHasSignedUp {customerId, customerName}
OrderWasPlaced {orderId, customerId}
InvoiceWasMadeForOrder {invoiceId, orderId, customerId}
PaymentWasReceived {paymentId, invoiceId, orderId, customerId}

Separated payload

If you’re concerned about putting redundant attributes in a an event, a nice trick is to separate the payload into the actual state changes, and the “fat”. This makes it obvious to other developers that these attributes are not part of the core event.

InvoiceBecameOverdue { core: {invoiceId}, fat: {customerName} }

Summary Event

The Fat Event pattern is distinct from the Summary Event in a few ways:

  1. A Summary Event usually happens at the end of a business process, whereas all events can be Fat Events.
  2. A Summary Event is often (but not necessarily) a new event type, whereas Fat Events are existing event types with additional attributes.
  3. A Summary Event contains often a complete representation of the resource or artefact that resulted from the business process, whereas Fat Events usually only contain a few specific extra attributes the consumer cares about. Summary Events are more likely to be useful for a range of different consumers.

Patterns for Decoupling in Distributed Systems

  1. Domain Queries
  2. Summary Event
  3. Completeness Guarantee
  4. Passage of Time Event
  5. Fat Event
  6. Explicit Public Events
  7. Segregated Event Layers

Intrigued? I teach workshops about DDD for Messaging Architectures.

Follow @mathiasverraes on Twitter.



Upcoming

Have a look at our upcoming workshops
Past agenda entries...

Blog

2019

2016

2015

2014

2013

2012

2011

Creative Commons License This work by Mathias Verraes is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License.