Telemetry arriving at the new platform is not the same thing as operational capability existing on the new platform.
Short answer
A controlled observability migration runs both platforms simultaneously for a defined period, validates that operational capability — not just data — has transferred, and moves in waves rather than a single cutover. Dual collection, typically through an OpenTelemetry Collector or an equivalent fan-out layer, sends telemetry to both the source and target platforms during the transition. But "the data arrived at the new platform" is a weak acceptance criterion. The correct chain to validate is telemetry → queries → dashboards → alerts → SLOs → incident workflows → operational ownership, because a monitor that silently stops firing, or a dashboard that renders successfully while showing the wrong numbers, is a failure that a volume-based data check will not catch.
This article assumes the decision is already made
Whether to migrate at all — and whether Datadog, OpenSearch, or another platform is the right target — is a separate question, covered in Should You Optimize Datadog or Migrate? A Decision Framework and Datadog vs OpenSearch: A Practical TCO Comparison. This article starts from the point where that decision has been made and focuses entirely on execution: how to move without a big-bang cutover, and how to know, with evidence rather than confidence, that the new platform is actually ready to carry production responsibility.
Why "same data arrived" is not the acceptance test
A migration team can point to matching event counts between source and target and reasonably believe the migration succeeded. That belief is frequently wrong, for reasons that have nothing to do with data loss.
A dashboard can render on the new platform and still be wrong, because the underlying query semantics differ. A metric that was a simple average on the old platform might be computed differently — a different aggregation window, a different handling of missing data points — on the new one, producing a chart that looks complete but tells a different story. A log-based alert that depended on a specific field extraction the old platform performed automatically may silently stop matching anything if that extraction was not rebuilt. None of these failures show up in a row-count comparison between source and target.
The correct validation chain has seven links, and each one has to be checked independently:
Telemetry — did the expected volume and shape of data arrive? Queries — do the underlying queries return the same results against the new platform's data model? Dashboards — do the visualizations built on those queries render the same operational picture? Alerts — do the same conditions still trigger the same notifications, at the same thresholds? SLOs — do service-level objectives compute against equivalent underlying data and produce comparable numbers? Incident workflows — does the on-call and investigation process actually work end to end on the new platform, under realistic conditions? Operational ownership — does the team that owns a given service actually know how to operate it on the new platform, not just in principle but in practice?
A migration that stops validating after the first link is not actually finished, even if every dashboard technically exists on both platforms.
Dual collection: the architecture that makes a controlled transition possible
The alternative to a big-bang cutover is running both platforms in parallel long enough to validate the full chain above, which requires telemetry to reach both systems simultaneously without duplicating engineering effort at every instrumentation point.
Fan-out at the collector layer
The OpenTelemetry Collector is commonly used as the fan-out point precisely because it decouples instrumentation from destination. Applications and infrastructure send telemetry once, to the Collector, which then routes copies — or filtered, transformed subsets — to multiple destinations through its exporter layer. This means dual collection during a migration does not require re-instrumenting applications twice; it requires configuring the Collector's pipeline to export to both the source and target platforms for the duration of the transition.
Where an OpenTelemetry Collector is not already in the architecture, equivalent fan-out is achievable through dual-shipping in existing agents, a log-routing or observability-pipeline layer, or vendor-specific dual-export tooling — the architectural principle (one collection point, multiple destinations) matters more than the specific tool.
What dual collection does not solve by itself
Fan-out solves the "both platforms receive the data" problem. It does not solve schema translation, and this is where migrations that looked straightforward on paper accumulate the most unplanned engineering work.
Different platforms use different conventions for the same concepts. A field that was a search-time extraction on one platform might need to become an explicit attribute set at collection time on another. Standard attribute or semantic-convention models differ between ecosystems — OpenTelemetry's semantic conventions are not identical to every vendor's native tagging model, and a Collector processor or transformation step is usually required to normalize service identity, environment, severity and other cross-cutting fields consistently across both destinations during the dual-collection period.
Get this normalization wrong and the two platforms will show subtly different pictures of the same system for the entire migration window — which undermines the validation work the parallel period exists to do, because nobody can tell whether a discrepancy is a genuine data problem or a schema-mapping artifact.
What changes semantically between platforms, and needs explicit validation
Retention and time semantics
Retention periods rarely match exactly between source and target during a transition, and comparing a seven-day window on one platform against a thirty-day window on the other will produce different-looking trends even when the underlying data is identical. Validation windows need to account for this deliberately, not assume retention parity that does not exist yet.
Trace, log and metric semantic differences
A trace on one platform is not guaranteed to decompose into spans the same way on another, particularly where automatic instrumentation libraries differ between ecosystems. A metric's aggregation temporality — whether it accumulates over time or resets per interval — can differ between an OpenTelemetry-native pipeline and a platform-specific agent, producing charts that diverge even from identical underlying measurements. These are not migration bugs to be fixed once; they are structural differences that need to be understood and, where necessary, explicitly normalized in the Collector pipeline before dashboards are trusted.
Sampling and cardinality differences
If the source platform's sampling behavior — head-based, tail-based, or a fixed percentage — is not reproduced equivalently on the target during the dual-collection period, the two platforms will show different trace populations, and any comparison between them will be comparing different samples of reality rather than the same reality twice. Cardinality behavior can also differ: a tag or attribute treated as low-cost on one platform's pricing model might be a meaningful cost driver on the other, which is a reason to review dimension design as part of the migration rather than porting the old tagging convention unexamined — see How Kubernetes Labels and Container Churn Increase Observability Costs and Datadog Custom Metrics and Cardinality: How Costs Grow for the mechanics that make this worth checking during a migration specifically, not only afterward.
Ingestion validation: closer to the source than the dashboard
Waiting until a dashboard looks wrong to discover an ingestion problem is a slow and expensive way to find it. Validate closer to the source:
- Compare event and datapoint counts between source and target over matching time windows, accounting for retention and sampling differences explicitly rather than assuming parity.
- Spot-check field-level fidelity on a sample of events — not just that a field exists on both platforms, but that its value matches.
- Watch for silent data loss patterns specifically: a Collector pipeline that drops data under memory pressure without an alert, an exporter that fails silently on a schema mismatch, or a receiver that stops accepting a specific telemetry type after a configuration change nobody reviewed.
- Instrument the pipeline itself — Collector-level metrics on received, processed, and exported telemetry volume are the fastest way to catch a fan-out failure before it becomes an operational gap discovered during an actual incident.
Rollout waves, not a single cutover
Group the migration into waves defined by a boundary that makes sense for the organization — by service, by team, by environment, or by telemetry type — rather than moving the entire estate at once. A wave-based approach has two structural advantages over a single cutover: it bounds the blast radius of any single wave's problems, and it lets each wave's validation findings inform the next wave's approach before the mistake is repeated at full scale.
A reasonable wave sequence starts with a service or team that is technically representative — meaning it exercises the harder cases (high cardinality, complex alerting, cross-service correlation) rather than the easiest possible workload — but is not simultaneously the most business-critical system in the estate. Testing only easy workloads produces a successful-looking pilot that does not predict how the harder 80% of the estate will behave.
Cutover criteria and rollback
Before any wave cuts over, define explicitly, in writing:
- Who owns the cutover decision, and who has authority to halt it.
- The specific validation results required before cutover — not "it looks fine," but which queries, which alert-parity checks, and which SLO comparisons must pass.
- Stop conditions — the specific signals that trigger a pause or rollback, defined before the pressure of a live cutover makes that judgment harder to make well.
- The rollback path — how quickly and cleanly the wave can revert to the source platform if validation fails after cutover, and what state the target platform is left in when that happens.
- What "done" means for this wave — because "zero downtime" is not a precise criterion until it specifies downtime of what: ingestion, search, dashboards, or alerting can each fail independently, and a cutover plan needs to define acceptable behavior for each.
Decommissioning the source platform for a given wave should happen only after data has been reconciled, alerts have proven stable under real production load for a meaningful period, the team has signed off, and any audit or retention obligation tied to historical data on the source platform is understood and satisfied — not immediately after the first successful day on the new platform.
Security, RBAC and access parity
Access control rarely maps one-to-one between platforms, and a migration that reproduces data and alerting fidelity while leaving access control as an afterthought creates real risk: either overly broad access granted during the transition to avoid blocking anyone, or gaps where a team that had appropriate access on the source platform cannot reach equivalent data on the target. RBAC design deserves the same explicit validation pass as dashboards and alerts, on the same wave timeline, not as a final step before decommissioning.
Historical data: decide what actually needs to move
Migrating years of historical data into the new platform's active, expensive storage tier is frequently the least justified line item in a migration budget. Before committing to it, ask which of three categories a given historical dataset actually falls into: data that must remain searchable at low latency (a genuine case for migrating into the new platform's primary storage), data that must be retained but is rarely queried (a case for an archive tier rather than active migration), or data that has no remaining business or compliance justification for retention at all (a case for retiring it rather than paying to move it). The cheapest historical-data migration is frequently the data that does not need to migrate.
Cost of the dual-running period
Running two platforms in parallel is not free, and migration business cases that model only the future steady-state cost of the new platform — without the dual-running period — understate the true cost of the transition. Budget explicitly for duplicated ingestion and processing, duplicated storage on both platforms during the overlap, the engineering time spent on reconciliation and validation, and the monitoring required to watch both platforms simultaneously. This period has a defined cost and a defined — ideally short — duration; treat it as a planned project cost, not a rounding error absorbed silently by existing budgets.
FAQ
A parallel migration runs the source and target observability platforms simultaneously for a defined period, with telemetry fanned out to both — typically through an OpenTelemetry Collector or an equivalent routing layer — so that dashboards, alerts and operational workflows can be validated on the new platform under real production conditions before the source platform is decommissioned.
No. Event-count parity confirms data arrived; it does not confirm that queries return equivalent results, that dashboards render the correct operational picture, that alerts still trigger under the same conditions, or that the on-call team can actually operate the new platform during an incident. Each of those needs to be validated independently.
An OpenTelemetry Collector is commonly used as the fan-out point during dual collection, because it lets applications send telemetry once while the Collector routes copies to both the source and target platforms. It does not automatically solve schema or semantic differences between platforms — attribute conventions, aggregation behavior and sampling still need explicit normalization in the Collector pipeline.
Differences in query semantics, metric aggregation temporality, retention windows, or sampling behavior between platforms can all produce different-looking results from equivalent underlying telemetry. These are structural differences between platforms, not necessarily migration errors, and they need to be identified and, where necessary, normalized rather than assumed away.
Classify it before migrating it: data that needs continued low-latency search access is a candidate for the new platform's active storage; data that must be retained but is rarely queried belongs in an archive tier rather than active storage; data with no remaining retention justification should be retired rather than migrated. Moving every historical byte into active storage on the new platform by default is frequently unnecessary cost.
The specific stop conditions should be defined in writing before the wave cuts over — for example, a defined threshold of alert-parity failures, missing data in a critical dashboard, or a failed SLO comparison — rather than decided under the pressure of a live cutover. The rollback path and the state the target platform is left in afterward should also be defined in advance, not improvised.
Sources
- OpenTelemetry — Collector architecture, receivers, processors, connectors and exporters documentation (fan-out and routing mechanics).
- OpenTelemetry — Semantic conventions documentation (why attribute and service-identity mapping needs explicit normalization across platforms).
- OpenTelemetry Collector Contrib — Tail Sampling Processor and Routing Connector documentation (sampling and routing behavior relevant to dual-collection design).
Migration tooling and platform-specific behavior change. Validate collector component status, exporter capabilities and target-platform semantics against current official documentation before finalizing a migration architecture.