In the business of client services, knowing your customer is everything. For a firm managing relationships across thousands of advisors and hundreds of thousands of customers this is no small feat on a normal day, but when you add in that records live in several different applications and systems, it quickly becomes a data engineering challenge.
Throw in a marketing team trying to maximize their outreach into the mix and this becomes a serious problem.
When we began investigating this issue, we found each source system identified a person differently with different IDs. So, when a customer has multiple services, they may appear in three or four places with no connection between them.
The Solution: A Golden Marketing Person ID
Using our existing Databricks platform with our medallion architecture, we came up with a plan for this initiative to build what we called the “Golden Marketing Person ID”, in which we wanted to create a single, stable, deduplicated identifier for every person across all source systems.
Rather than trying to implement complex fuzzy matching from the start, we built a rules-based deduplication engine where each person was assigned a generated ID computed from their deduplication rules and matching value. For example, an email address match produces a consistent ID regardless of which source system the record came from and when the same person appears in multiple systems, they match to the same ID every time.
After testing our initial implementation, we did end up adding some basic fuzzy matching on names to account for when the same email was used for multiple people in the same household. In this case, we created additional marketing records for those individuals even though they shared an email.
The result lives in a gold-layer table that maps every source system record to a single Marketing Person ID. Our initial load of the table handled the full historical deduplication across all records, while the ongoing incremental loads apply the same rules to new and changed records without reprocessing the full history.
We also maintain a historical table using SCD2 (Slowly Changing Dimension Type 2), so that we can get a complete audit history of every change to a person’s record over time, not just the current state.
This kind of design matters for compliance and reporting as much as for marketing. When a regulator asks what our client knew about a customer at a specific date, they can answer precisely.
A key part of this initiative was opening the data platform up to our client’s external marketing partners, which included a marketing agency, a data enrichment and activation platform as well as a customer data platform for web and user behavior data, without compromising our data governance.
Rather than extracting data and copying it to an external system, which could create drift and governance headaches, we built a dedicated external-access schema populated entirely with views pointing at the gold tables. External partners can then get read access to a curated view of the data. The source of truth never moves and when the underlying data updates, the views reflect it automatically.
The metadata framework driving this is configurable so new views can be added to the external schema through a metadata table, not a code deployment. That flexibility matters when marketing requirements change faster than typical sprint cycles.
On the enrichment side, we built pipelines to pull behavioral and marketing data back from those marketing platforms to further enrich the Marketing Person Golden record. Web engagement signals, email open rates, campaign attribution are all flowing back into the same unified identity, making the gold record progressively more valuable over time.
What This Looks Like in Practice
Before this initiative, the marketing team had to work around data fragmentation, audience lists were built by hand, personalization was limited by uncertainty about who was who and coordination between the data platform and marketing platforms required manual data extracts.
After this initiative:
- Every person across every source system has a single, stable ID that persists across rebuilds and incremental updates
- Marketing campaigns can target customers with confidence so the same person won’t receive duplicate outreach because they appeared in three different systems
- Enrichment from marketing platforms flows into the same record, building a progressively richer profile over time
- Marketing partners consume data through governed, zero-copy views which means no data sprawl, no synchronization lag
- The full history of every person record is preserved for compliance and analytics
Why This Work Matters Beyond Marketing
What we built here is a pattern, not just a solution. The identity resolution approach, the external access framework, the enrichment pipelines, the unified library – all of it is generalizable. Any firm that manages client data across multiple systems and wants to activate that data for marketing, compliance, or analytics can benefit from this same architecture.
The investment in getting identity right at the data layer pays dividends across every downstream use case. When you know with confidence who a person is, every system that touches that person gets better marketing personalization, advisor tools, reporting, regulatory compliance, which provides a better experience for the advisor and customer.
