Process Hygiene: Why Clean Data Never Stays Clean

Cinderhaven Provisions (a worked example, not a client case) gets a product data health audit: every record traced against GS1 standards, every downstream failure tracked to the field that caused it. The result: $93,000 a year in retailer chargebacks traced to five product data defects. Missing case dimensions, invalid UPCs, bad GTINs, missing country of origin, missing case weight. Five fields. Fix them and the fines stop.

Here is the uncomfortable part: fixing them is the easy half. Every one of those five defects was manufactured by a routine. An item setup form filled out by whoever had a free hour. A packaging change that updated the dieline but not the case weight. A unit conversion done in someone's head. Correct the fields and leave the routines, and the next product launch re-creates the defects with fresh annual price tags. The audit bought data hygiene. Whether it lasts depends on process hygiene.

What process hygiene is

Data hygiene is the state of your records at rest: product masters that agree across systems, transactions that reconcile, fields that mean what they say. Process hygiene is the condition of the routines that create, move, and change those records.

A case dimension is data. The way that dimension gets measured, entered, converted, and updated after a packaging change is process. A deduction is data. Whether anyone reads the remittance before the dispute window closes is process. When the routine is undefined, unowned, or manual, the data it produces is dirty on arrival, and no cleanup survives contact with it.

The data quality market sells tooling for the first half. Audits, cleansing tools, syndication platforms, MDM systems: all of them act on the state of the records, and the good ones act on it well. What none of them can ship you is an owner. Who fills out the item setup form, what they check before submitting, and how fast a bad field gets caught are decisions inside your building. That is why the term needs to exist. The half of the discipline that determines whether the other half lasts rarely has a name, a budget line, or a person attached.

Why clean data decays

Data has a half-life because defects are a flow, not a stock. An audit corrects the records that exist today. Tomorrow the flow resumes: every new item form, every ASN, every promo agreement, every packaging change is another chance to reintroduce the same defect the audit just removed. The flow is not a rounding error, either. When Harvard Business Review had managers score their own operations, 47 percent of newly created data records contained at least one critical error, and only 3 percent of the datasets studied rated acceptable.

The compounding is concrete. A case dimension that disagrees with the retailer's system by a rounding error generates a chargeback on every shipment of that SKU until someone reconciles the records: a $200 fine on a weekly shipment is $10,400 a year from one field on one product. Fix the field and the bleeding stops. Leave the conversion habit that produced it, and the next packaging change manufactures the same defect with a fresh twelve-month price tag.

This is the decay curve hiding inside every "we already did a data cleanup" conversation. The cleanup worked. The factory that makes the defects never shut down.

The five failure patterns

Process hygiene fails in recognizable ways. Five patterns cover most of the damage. Each one comes with the question that tells you whether it is running in your building right now.

Nobody owns the routine. Item setup gets done by whoever is free the week the retailer form is due, usually by copying the last product that got accepted. Ownership by availability means the routine has no memory: the same mistake gets made fresh each time, and the same GDSN attributes bounce at the same retailers.

Ask: who owns item setup, by name? Not a department. A person whose routine it is, with a checklist that outlives them.

Manual re-keying at handoffs. Every place a human retypes what a system already knows is a defect generator with a schedule. The ASN typed by hand on shipping day is the canonical case: EDI 856 errors trigger deductions even when the physical shipment is perfect, at $1,000 to $3,000 per incident, weekly if you ship weekly.

Ask: how many times does the same number get typed between your ERP and the retailer's dock? Every retype is a place the two can diverge, and nothing tells you when they have.

No validation before send. Submissions get checked by the retailer instead of by you. The rejection arrives days later with a cryptic code, the resubmission burns a week, and the launch date does not move. Everything the retailer's system checks could have been checked at your desk first, which is the entire argument for preflight validation.

Ask: when a retailer rejects a submission, do you learn which field caused it? If rejections get fixed by trial and error and teach nothing, the routine has no feedback loop.

Nobody reconciles. Remittances pile up unread while deductions age past their dispute windows. 65 to 80 percent of shortage claims are invalid: winnable, if anyone had opened them in time. Deductions run 5 to 15 percent of gross sales at brands whose net margins sit at 3 to 5 percent. Reading the remittance is not bookkeeping. It is margin defense.

Ask: how many days pass between a deduction landing and a human reading it? Dispute windows run 30 to 90 days. A monthly glance at the AP aging is not a reconciliation routine.

Two systems, no tiebreaker. The product master says one thing, the ERP another, the retailer portal a third, and no routine decides which is true. This is how a brand's team reports 95 percent fulfillment while Walmart scores the same shipments at 86. Both numbers came out of real data. Only one is writing fines against the remittances, and the gap persists because each side keeps trusting its own. The same pattern hides trade spend: the promo plan lives in email, the settlements live in the ERP, and because nobody joins them, double-funded and phantom promotions ride through unchallenged. Reconciling two sources, quantifying the gap, and attributing root causes is what the Fulfillment & OTIF Diagnostic works through on Cinderhaven's data.

Ask: can you match last quarter's promo plan to its settlements, line by line? If the plan and the money live in different places and have never been joined, assume leakage until proven otherwise.

Most brands answer "no" or "I don't know" to two or three of the five. That is not an indictment; it is a to-do list, and each item has a dollar figure attached. Across the ten decisions a growing brand answers by reflex, a $25M brand leaks $1.4M to $2.3M a year through exactly these unowned routines.

Data hygiene + process hygiene = business intelligence you can trust

Reports inherit the hygiene of what feeds them. That is the whole equation, and both terms are load-bearing.

Fix the data without the process and you are clean for a quarter, then the decay curve does its work. Fix the process without the data and pristine routines flow into a stock of records that is still wrong. Fix both and something changes that neither cleanup nor software delivers on its own: the numbers on the Monday report become numbers you can act on without double-checking, because the routines that produced them are known, owned, and validated.

Consider what the alternative costs in decisions rather than fines. A founder looks at a channel report showing one retailer as the top account by revenue and commits the next production run and the next round of trade dollars to it. The report is accurate about revenue. It is silent about deductions, because the deduction ledger was never coded to channel. That coding is a routine nobody owns. Netted out, that account ranks fourth by contribution, and the brand has just funded its least profitable shelf. Nothing in the report was wrong. It simply could not carry the weight of the decision made on it, and no one could tell.

This is also why dashboard projects disappoint. A brand that does not trust its numbers buys a BI tool, and the tool faithfully visualizes the same unreconciled inputs with better fonts. The dashboard was never the problem. The audit your CFO should be requesting is not a report on what the numbers say; it is an assessment of whether the numbers deserve belief.

What good looks like

Process hygiene is not a transformation program. It is a small set of routines with names on them, in three layers.

Recover. Run the twelve-month diagnostic once: pull the deductions, standardize the codes, group by root cause, split the result into a fix list and a dispute list. This is where the money that already leaked comes back, and it funds everything else.

Prevent. Put validation where the data enters. Item-setup preflight before the form goes to the retailer. EDI checks before the ASN transmits. Automated quality scoring on the product master so a bad field is caught the day it appears, not the quarter the chargebacks arrive. The audit finds the problems once; the pipeline stops them forever.

Preserve. Cadence. A weekly deduction triage with a fifteen-minute agenda. Three numbers reviewed every Monday. A monthly promo-to-settlement reconciliation. None of these routines is sophisticated. Their entire power is that they run every week whether or not anyone feels like it, which is exactly what a routine held together by memory and good intentions cannot promise.

Start with one routine

Do not start with a transformation. Pick the routine feeding your largest loss and put a name and a cadence on it. For most brands that is deductions, because the money is countable and the dispute clock is running: a free deduction scan will tell you what your retailers took and what is recoverable. If you want the wider picture first, the Retail Readiness Scorecard locates your weakest routines in ten minutes, and the services page shows what a scoped engagement looks like when you want the diagnostic run for you.

Or skip the tools. Tell me which of the five questions you could not answer and I will tell you what that routine is most likely costing you, in writing, no call.

Clean data is a state. Process hygiene is what keeps you in it. If you paid for a cleanup more than a year ago and nothing changed about how the data gets made, the decay has already started. You just have not been billed for it yet.