SPONSORED

Sponsored by Databricks. Databricks commissioned and funded this guide and reviewed it before publication for factual accuracy. They did not get to change the conclusions.
Full disclosure at the foot of this post.

Before I begin, let me tell you a true story. I was working with an online fashion retailer whose WooCommerce instance kept its own customer records, with a second Segment customer database behind it that everybody treated as the golden record.

Both systems gave every customer an ID. The two sets of IDs looked alike, and they were not the same thing.

The first time the new activation layer went live, it did exactly what it was told. It took the segment, looked each customer up by ID, and sent. The IDs passed every check.

Right format, right type.

But because this issue had never garnered any attention, nobody knew that they just belonged to different people.

So some Italian customers started getting the Dutch campaign. Germans, French. Male, female. There was no match.

Neither system was broken. Each did precisely what it was built to do, and the join between them was exact, on two things that happened to look identical.

How it came to light is the part I think about most. No dashboard caught it. And I am confident AI would not have caught it.

The project manager did, herself Italian, with her account set to Italian, when the campaign reached her in Dutch. She reads Dutch, so she knew what she was looking at, and she knew it should not have arrived in that language. She asked customer support whether anyone else had mentioned it.

They had.

The tickets were already there, and they had not gone anywhere yet, which is normal. From a support desk, a complaint about the wrong language looks like a content problem, and nobody thinks to ask about identity.

Once it was found, everyone came together.

Data engineering, marketing, the project manager, all of them, working out which messages had gone to whom and putting the records right. It got fixed, and it cost time nobody had budgeted for.

In guide 2 I said this one would pick up where the key is right and the matching is not. This story is the mirror image of that.

The join was perfect. The key only looked shared.

People tend to expect the fuzzy matching to be the dangerous part of identity. In my experience the expensive mistakes are far more often an exact join on a key that meant something different on each side, and a confidence threshold is unlikely to flag it, because exact-key rules usually run before any scoring does.

Two directions, and they get fixed in opposite ones

When two systems disagree about who somebody is, the mistake goes one of two ways. Two people get treated as one, or one person gets treated as several. Engineers tend to call these a false match and a missed match, or over-merging and under-merging (and occasionally worse things, in the incident channel).

It helps to know both pairs of words, because you will hear them in the same meeting.

It matters which one you have, because the fixes usually point in opposite directions. Tighter rules or a corrected key for the first, looser rules or more identifiers for the second. Loosen the rules to stop a double send when the audience was really built per address, and you have taught the system to join more strangers while the double sends carry on.

Side 1 of this sheet sorts the common symptoms into those two columns.

Over-merging, two people treated as one, shows up as somebody seeing another customer's details, messages in the wrong language, or one partner's unsubscribe stopping mail to the other.

Under-merging, one person treated as several, shows up as the same campaign arriving at two addresses, an unsubscribe on one record while another keeps getting mail, or more customers than finance counts.

Every symptom on the sheet also has an "also check" line under it, and I would not skip those. A wrong name on the site can be a caching bug. Existing customers still seeing acquisition ads is often the ad platform failing to match your list, before it is anything in your match rules. A good part of what reaches you as an identity problem proves to be something else, and it is much cheaper to find that out before the match rules get touched.

The cheapest way to find out is ten records. Pull ten real customers behind the complaints, look each one up in every system you are allowed to see, and ask whether the same ID is the same person.

This is what I did.

Then sort them into the two columns. Bring engineering the IDs and your sorting, not copied personal data. Ten records will not tell you how good your matching is overall. That takes a properly labeled sample, and engineering's help. What they will tell you is which way this particular problem is failing, which is what you need before anybody touches the rules. Read against both systems before the first send, they would very likely have caught the retailer's problem.

The mistake you can afford

No match rule avoids both mistakes entirely. With the same data, tightening it misses more real matches and loosening it joins more strangers. So the useful question is which mistake a given use case can afford, and that is a business decision, which makes it the part of identity that belongs to marketing.

In July I wrote in the practitioner's guide to who owns the match that suppression wants to over-merge. For opt-outs, I still think so. If in doubt, suppress, because holding back one stranger costs a send, and mailing someone who opted out costs a great deal more. For opt-in consent, it is backward, and I should have split them. Borrowing a permission from a record that may be somebody else is sending without consent, so that use wants the rule strict.

Side 2 of the sheet puts seven common uses on a line between those two poles. Suppression, frequency capping, and ad exclusion lean towards joining. Order history, consent, and on-site recommendations lean towards keeping apart. The customer count for the board sits in the middle, and it only needs a sentence saying which rule it was counted with.

The household example on the sheet is there to show that one pair of records can deserve opposite treatment. Two partners, one address, one phone number.

For a printed catalog they are one household, and a second copy is usually waste, while for personalized product emails they are two people with different tastes. Whether your platform lets you set the grain per use varies a lot, and it is worth asking before assuming it can or cannot.

Can a wrong match be undone?

This is the question on the sheet I would ask first if I were only allowed one, and it is the one marketers rarely ask.

Some systems merge records destructively.

The originals are combined, one value wins where they disagree, and the history of how they got there is thin. Many keep the source records and link them to a shared ID instead, which means a wrong match can be split again.

Plenty sit somewhere in between. None of these is wrong in itself, but the same mistake costs very different amounts in each. In a warehouse setup, you can often check the tables yourself. With a packaged platform it is rarely visible from outside, so ask.

If a bad match is a link, the fix is mostly a technical one: split the link, correct the rule, and the records go back to being two people. Messages already sent on the wrong match cannot be recalled, but no data has to be untangled. If it is a merge, it can be a project, with somebody working out which email, which consent flag and which order belonged to whom, after messages have already gone out on the combined record.

I have argued before that reversibility is one of the best proxies for safe autonomy when agents start making decisions. The same test works for match rules, and it is a much easier conversation to have before the rules change than after the first privacy complaint. It is also the question in the six most likely to change what engineering actually builds.

After the last incident

The first three guides ended before the next meeting or the next request. This one ends after the last incident. Most of what is in it, I learned after one.

Fixing an incident like the retailer's only gets you part of its value. A proper post-mortem gets the rest. A while ago I made the case for pre-mortems before a CDP launch, and I still like them, but most teams already have an identity mix-up sitting in recent memory that nobody has looked at properly.

So take that one. A double send, a wrong name, a complaint that made no sense at the time. Work out which direction it failed, using side 1. Then find the question on side 2 that would have caught it, and ask that one before the next change to the match rules. And make sure support knows who to tell when the complaints start to cluster, because in my story they knew first.

Next in this series: who is allowed to say yes. All six live on the Helping Marketing Talk to Data Engineering page as they publish, and every term across the series is collected in the glossary.

About the series