Data integration challenges in B2B enrichment: a recovery guide

What makes B2B data integration difficult?

Data integration combines information across systems so that another system can use it. In enrichment, the incoming information is not simply a copy of your existing record. It is a proposed addition or correction from another source, with its own identifiers, field meanings, observation dates and uncertainty.

A company identifier may describe a subsidiary while your CRM account represents a group. A person's returned employer may have changed since the original request. A phone number may be present without an independently confirmed connection to that person. Moving these fields successfully does not settle whether they belong in the destination.

Separate three responsibilities: transport moves the response; interpretation decides what it means; application decides which authorised change may occur. This division helps locate failures. A provider can complete a lookup while the destination rejects a field. Conversely, a technically successful write can still carry a stale employment relationship.

General integration guidance already recognises quality, incompatible schemas, volume and latency as important problems. IBM's Top Data Integration Challenges and Solutions, dated 19 December 2025, covers those themes. The enrichment-specific task is to connect those concerns to a traceable decision for each requested record.

Start with a reconciliation ledger

Give every eligible input a stable internal operation identifier before making a paid lookup. Retain the destination record ID, requested fields, input version, policy version and permitted purpose. Store provider request identifiers separately: they describe external attempts, whereas your operation describes the business intent that may span several attempts.

Use explicit states such as eligible, submitted, awaiting result, result received, held, accepted and committed. Record unsuccessful terminal states as well: invalid input, no result, expired and rejected write. A timeout belongs in an unresolved operational state until you establish what happened; it should not become a negative finding about a person.

Reconcile unique operations, not log lines. One operation can generate several HTTP requests, webhook deliveries and write attempts. Count those attempts for cost and reliability analysis, but do not let them inflate the number of people enriched. Keep the identity decision separate from transport deduplication; the B2B matching guide explains entity-level rules.

The chart's committed yield is 5,700 ÷ 9,000, or 63.3% of eligible operations. Candidate coverage is 7,200 ÷ 9,000, or 80%. Neither figure independently measures factual accuracy. Keeping both prevents a high lookup coverage number from concealing a destination bottleneck.

Challenge 1: incompatible meanings behind compatible fields

A field called company_size could mean an employee count, a band or a provider estimate. A field called updated_at could describe retrieval, ingestion or an actual source change. Map meaning before format. Record the source field, expected type, unit, entity scope, transformation, destination field and missing-value behaviour.

Distinguish absent, null, empty and zero. An omitted phone number normally means there is no proposed replacement; it should not automatically erase a previously accepted number. A genuine correction or deletion requires its own explicit action. Unknown verification status must remain unknown rather than being promoted to valid during a transformation.

Validate the incoming structure, then apply business rules. JSON Schema's object reference explains how required properties and additional properties work; those mechanisms validate structure, not truth. An additive field can be recorded without authorising a new CRM write. A changed type or unfamiliar status should stop the affected mapping until its meaning is understood.

Challenge 2: retries that repeat paid work

A lost response does not prove that the provider failed to process the request. Retrying can create another paid lookup even if your eventual CRM update is deduplicated. Treat request execution, billing and destination application as separate effects, and establish the provider's documented behaviour for each.

RFC 9110, section 9.2.2, cautions against automatically retrying non-idempotent requests unless their semantics are known to support it or the original request is known not to have been applied. Do not assume an arbitrary idempotency header works. Check whether the endpoint supports a caller-supplied key, how long it is retained and what happens when the same key carries different input.

When a provider offers a status or result lookup, reconcile an uncertain request before starting another chargeable operation. Where that facility is unavailable, document the duplicate-charge risk and route ambiguous cases to a bounded recovery policy. Use attempt limits, deadlines and a shared rate limiter rather than independent retry loops inside every worker.

Authentication failures need credential or permission repair. Invalid input needs correction. Rate limits need the documented waiting policy; transient server failures may justify a retry only when execution semantics permit it. Add jitter where appropriate, but never let random delay shorten a provider's required wait. AWS's Making retries safe with idempotent APIs provides the underlying design rationale.

Challenge 3: real-time data integration without a completion deadline

Define the deadline for the business action before choosing synchronous or asynchronous processing. A form response may need a fallback within two seconds; an account backfill may tolerate several hours. A fresh HTTP response is also not proof of fresh underlying employment data. Timing the transport and dating the evidence answer different questions.

Allocate time to input checks, the provider, policy evaluation and the destination. Measure end-to-end percentiles from real operation traces. Do not add the p95 latency of each stage and call the sum an end-to-end p95: stage distributions and their relationships determine the actual result.

Acknowledge an asynchronous job separately from completing it. Apollo's People Enrichment documentation, reviewed 20 October 2026, describes asynchronous phone and waterfall results and warns that webhook calls may be repeated. This is evidence of a documented integration contract, not a measured delivery guarantee or a universal design shared by all suppliers.

Specify what happens to late results. They may be useful for a later account refresh while being too late for the original routing decision. Recheck the destination version and current suppression policy before applying them. A delayed response should never overwrite a newer accepted value simply because it arrived last.

Challenge 4: duplicate events and split transactions

Authenticate callbacks according to the provider's documented mechanism, validate their shape and persist them durably before acknowledging successful receipt. Keep callback handling short; perform enrichment interpretation and CRM changes in workers. An acknowledgement sent before durable storage creates a loss window if the process stops immediately afterwards.

Use a uniqueness constraint for the provider event identifier within its documented scope. If no stable identifier exists, agree a deterministic event key that includes the operation and result version; a hash of the contact alone can incorrectly collapse legitimate later changes. Preserve permitted evidence long enough to distinguish a duplicate delivery from a new observation.

Coordinate the processing ledger and queued work through a transaction or an outbox pattern where the storage system supports it. That protects the local hand-off; it does not create a transaction with a remote CRM. AWS's transactional outbox guidance explicitly notes duplicate-message handling and ordering as design considerations.

For the remote write, use stable destination identifiers and appropriate conditional updates. After a timeout, reconcile the intended field values and operation marker before trying again. An upsert is not automatically update-only: Salesforce's external-ID documentation states that an unmatched ID can create a new record. Make creation permission a deliberate choice.

Challenge 5: automated data integration that outruns its destination

Concurrency does not remove destination capacity limits. Separate new work from retries and backfills, and allocate capacity across them. An unlimited retry queue can starve today's operational requests. Conversely, allowing new traffic to consume every available slot prevents recovery after an outage.

Track queue age as well as queue length. A short queue can contain a small number of permanently failing records; a long queue can be healthy during a planned backfill. Track the oldest eligible operation, arrival rate, effective processing rate, retry share and time remaining before the business deadline.

At 180 operations per minute, spare capacity above a continuing arrival rate of 120 is 60. A backlog of 300 then needs at least five minutes to drain, assuming those rates remain stable. Add retries, API windows and variable service times before using this lower-bound calculation as an operational forecast.

Challenge 6: a cheap lookup with an expensive usable result

Compare data integration techniques on the cost of their completed purpose. Batch, micro-batch and event-driven approaches have different latency, operating and recovery costs. Select the simplest approach that meets the deadline and recovery requirement; sub-second processing is unnecessary for a scheduled account-research refresh.

Return to the illustrative cohort. Assume 9,000 initial requests at £0.08 each, 600 additional billable attempts at the same price, £180 of allocated operational cost and 1,200 reviews. At 1.5 minutes per review and £40 per reviewer hour, review costs £1,200. Total cost is £720 + £48 + £180 + £1,200 = £2,148.

Divide by the 5,700 confirmed usable writes, giving approximately £0.38 per result. This is a cost-allocation example, not TargetWise pricing. Supplier charging rules differ: retries, unsuccessful lookups, phone results and waterfall stages may have separate treatment. Reconcile actual invoices and credit transactions rather than assuming every attempted call costs the same.

Run a failure drill before expanding production

Use synthetic records or an authorised test set. Record the initial destination values, intended changes and expected outcomes. Exercise the same integration path used in production, with paid operations limited by an explicit budget. A happy-path demonstration alone cannot establish recovery behaviour.

Add no-match responses, catch-all domains, unverified numbers, stale employment and conflicting providers to the data cases. Add revoked credentials, rate limits, malformed callbacks and destination outages to the operational cases. State the expected result before each run; preserve failures rather than adjusting expectations afterwards.

Assign owners for the mapping, acceptance policy, queue and destination. Specify an exception deadline and who may authorise replay. Keep permitted before-and-after values for correction, but avoid a blind rollback that overwrites subsequent human edits. Recovery must compare the current destination with the change it intends to reverse.

Apply the method to TargetWise without assuming guarantees

TargetWise's published API contract distinguishes matched, partial and not_found enrichment results and exposes request identifiers, unresolved fields and grounding information. Its documentation explicitly separates retrieval time from the last update of underlying records and tells consumers not to infer missing source provenance. These documented fields can feed a ledger; they do not replace your acceptance policy.

Review the current enrichment API implementation guide for the broader lookup workflow and the freshness guide for evidence-age decisions. Do not infer native idempotency, callback guarantees or a universal source lineage from the existence of an API or MCP interface. Validate the particular operation and commercial agreement you plan to use.