Mary, Mary, Quite Contrary: Where Does Your Data Flow?
I've had this rhyme stuck in my head for a few weeks, which is either a sign I need more sleep (I definitely do), a subliminal nag that I really have to get round to controlling our fig tree at the back, or an acceptance that everything, EVERYTHING, is a systems problem if you look at it long enough.
Even nursery rhymes? Well, sure, Bob.
Strip away the silver bells and what do we have? An operating model. (I know, I know, stay with me a sec). A garden that's expected to grow in a straight, orderly, auditable row. A gardener who is, supposedly, contrary. And a question that every ops person I know is screaming somewhere: where does your data need to go? Who's standing downstream when it gets there and what do they do with it? The florist trimming and putting together a beautiful bouquet? What info does she need to build? The grocer who decides how much to charge for it? What feeds their pricing strategy? The customer who takes it home to enjoy?
Every process or model is based on the clean swimlane diagram. Attributes flowing left to right. A field is created once, upstream, and it ripples obediently into every system that needs it - the catalogue, the partner feed, the AI layer to make sense of all of it. Pretty maids, all in a row.
Is your garden reflective of those process slides?
Where the contrary part comes in
The contrary part isn't the data; data, after all, doesn't have opinions. The contrary part is everywhere a human made a reasonable, defensible, entirely-in-isolation decision that breaks the row.
Someone renamed a field because the old name didn't make sense for their team. Someone hand-keyed a value because the source didn't reflect what was needed. Someone built a workaround eighteen months ago that became load-bearing infrastructure nobody remembers agreeing to. None of these were SABOTAGE!!!, each one was somebody tending their own patch of the garden rather sensibly.
This is the bit that I missed: when data quality gets treated as a hygiene problem (as if the fix is a stricter template or a sterner message) It isn't. Contrariness is structural. It's what happens when a system has no place where "this is the definition, this is the source, this is where it's allowed to change". You don't discipline your way out of that. You design your way out of it.
Why integrity is the whole game
A garden doesn't fail all at once. It fails one contrary decision at a time, and every one looks harmless from where the person is standing.
A mislabelled seed at planting doesn't always announce itself. It grows and it can look like everything else in the row, it’s harvested, it’s priced up, but why does the customer’s corinader chicken curry taste of parsely? By the time it’s discovered, nobody can trace it back to the label. It’s too far along, too entangled with everything that grew around it. That's data integrity: not a compliance checkbox, but the what makes it a garden you can trust.
Weeds are the easy metaphor I was thinking around here, but they're not quite right. Weeds are obviously wrong, and you can pull them the moment you see them. The real risk is cross-pollination. Two beds too close together, sharing a name for something that isn't the same thing, and by the time it's flowering you can't separate what came from where. That's the field renamed by one team and never reconciled with the other three that manage their work from it. Everything downstream - the pricing, the bouquet, the customer's expectation inherits the confusion… and has no way of knowing it's inherited anything at all until supper time.
This is why integrity has to be designed in at the root, not just inspected in at the harvest. You can audit a garden forever and still not catch this, because by the time you're auditing, it looks exactly like everything that grew correctly. The only real fix is knowing, at planting, what's been planted, by whom, and on whose authority that anything that impacts others is allowed to change.
Silver bells, cockle shells, and the attribute registry
The fix looks a lot like gardening, not engineering. You don't necessarily rent an excavator and rip the whole thing out and start fresh. You investigate bed by bed, especially where functional teams with their own goals are concerned. You find out what's planted where and why, and you build a registry of what's really there: source of truth, owner, what depends on it downstream. Silver bells are, after all, silver bells because someone wrote it down and there's shared agreement that they're silver bells (Mark Catesby, The Natural History of Carolina, Florida, and the Bahama Islands, 1731 if you’re interested).
Conceptually this is both duller and more valuable than it sounds; two things can be true. Every AI initiative I've touched features me dying on this exact hill. I've set up a second home here. You cannot, I mean it, cannot layer intelligence on top of a garden where nobody can say with confidence which row is real.
Someone has to walk the rows
The other thing: a diagram doesn't tend itself. Someone has to actually walk the rows; not own the garden on an org chart, but notice when something's drifted, and know which contrary bit is a problem and which one is just how this particular bed grows.
That bit is not system design really, it's a role. It looks like admin until the day the AI layer trips over a field that means three different things in three different systems (and suddenly it's the most important project in the building). Give it focus before you need it, not after.
So, where does your data go?
Not where it's supposed to. Actually. Trace one field, right now, to every place it lands and every decision made off the back of it once it does.
If you can't answer that in a sentence and the honest answer is "depends who you ask, and depends which quarter", you don't have a data quality problem. You, my friend, have an unattended garden with a very tidy diagram taped to the fence, and no idea who's downwind of it. Worth going and looking at what's actually in the ground, and where it's rooting on the other side.
The framework I used.
I didn't just diagnose this in the abstract - I mapped it in Miro based workshops. Reconciled into one journey from interest in a vendor to content being available for customers. Every milestone modelled the same way: Spark (what triggers the event), Gatekeeper (what needs to be true) Outcome (what happens), with the data in, the data out, and the owner named at each step. Magic-wand (how it should work), role-based, data-first.
If you're starting your own bed, five things worth doing, in this order:
Name the problem before the tool. One sentence, no tools or solutions allowed. "I can’t do A because I don’t know about B" is a problem. "We need to implement X" is a solution looking for a problem to fit it.
Diagnose with six honest questions to understand scalability. Every "no" tells you exactly where to dig.
Do you have one front door? i.e. Do the requests come in one way, consistently?
Is the same info captured every time to process your work? Do you get what you need to do your job?
Does the data move on its own? Or do you have to manually push?
Can you prioritise by impact, not by who shouted loudest?
Can you see what's in flight (and where in the sky) without asking?
Will this hold as volume grows?
Design the triggers before touching a tool. Map each milestone as Spark - Gatekeeper - Outcome - "when X happens, Y is created, but only if Z." Workshop it with the people involved. Document it, ask then to check your work. Twice. Only then go near a planning your build.
Have an owner per field. Frankly, two owners is the same as no owner in this work.
If a system changes: cut over, then deprecate. Port the data, retire the old system, and don't run both indefinitely. Two systems side by side is double the work and half the truth.
Automation applied to a defined process multiplies the gain. Applied to an undefined one, it multiplies the mess. And yes I’ve said it multiple times before, but hey! It’s my hill !
Methodology and sequence matters too. Skip a step and you'll be back here in six months, figuring out where that blooming Marigold came from?!?!
