Observability Cost Optimization: Where Your Logs, Metrics and Traces Budget Actually Goes

Observability costs are rarely explained by one expensive vendor or one bad contract.

In many environments, the invoice is the downstream result of technical decisions made much earlier: what applications emit, what collectors forward, which dimensions are attached, what gets indexed, how frequently metrics are sampled, how traces are retained, how long telemetry stays searchable, and how much infrastructure is required to process it.

Procurement still matters. Contract structure, commitments, discounts and vendor pricing can materially change spend. But an invoice usually tells you what the platform charged for, not which architectural decisions created that consumption.

Observability cost optimization is the discipline of controlling the cost of collecting, processing, storing, querying and operating telemetry without removing the diagnostic and operational value the organization actually needs.

The objective is not minimum telemetry volume.

It is better economics for useful telemetry.

Executive summary

Observability spending can be understood through three economic layers:

1. Telemetry generation — logs, metric series, spans, events and profiles produced by applications and infrastructure.
2. Platform consumption — ingestion, processing, indexing, active series, datapoints, spans, storage, retention, search capacity or other resources consumed by the observability platform.
3. Operating cost — engineering labor, compute, storage, networking, collectors, upgrades, reliability work and governance required to operate the capability.

Those layers interact.

A high-cardinality label introduced at instrumentation time can create thousands of metric series downstream. A verbose application log can increase collection, ingestion, processing, indexing, storage and retention together. Recording every span from a high-throughput service can increase collector load and network traffic before the observability backend stores anything.

The resulting economic chain is:

application and infrastructure behavior → telemetry generation → collection → processing and enrichment → platform consumption → storage and retention → query and analysis → vendor or infrastructure charges → operating cost

This is why observability cost optimization is often more of a telemetry architecture and governance problem than a procurement exercise.

But not always.

A pricing model that is structurally mismatched to the workload or an unfavorable contract can remain expensive even after telemetry has been well governed. Architecture and procurement therefore need to be analyzed separately.

The better objective is not simply lower spend.

It is lower cost without disproportionate loss of diagnostic value.

Why observability spending is difficult to understand

A finance system sees meters and SKUs.

An engineering organization sees services, namespaces, clusters, traces, logs, teams and incidents.

Those views rarely align naturally.

Different observability platforms also meter fundamentally different resources.

Splunk supports ingest-based pricing as well as workload pricing based primarily on compute capacity. Datadog exposes dimensions including ingested log GB, indexed log events, custom metric timeseries or datapoints depending on the contract, APM hosts, ingested spans and indexed spans. Grafana Cloud measures Prometheus metrics through active series and datapoints per minute while logs and traces are volume-based. Elastic Observability Serverless uses consumption dimensions including ingest and retention. Amazon OpenSearch Service managed clusters charge primarily for infrastructure resources such as instance hours, storage and data transfer, while its Serverless model separates compute from storage. Dynatrace currently exposes several models across hosts, pods, datapoints, GiB ingested, GiB retained and GiB scanned.

Therefore:

There is no universal economic unit called “observability volume.”

A GB of logs, 100,000 active metric series and one billion spans are economically different workloads.

Even a GB is not necessarily comparable across platforms.

Datadog currently meters Log Management ingestion using uncompressed data, while Elastic Observability Serverless measures fully enriched normalized data before Elasticsearch compression. Azure Monitor has its own billable-record-size calculation, and an infrastructure-priced OpenSearch deployment ultimately consumes provisioned compute and storage rather than charging for an abstract number of log GB.

That billing boundary matters.

Comparing two nominal “per GB” numbers without understanding what each GB represents can produce a misleading TCO comparison.

Observability FinOps therefore needs to translate platform billing dimensions back into the workloads, services and telemetry decisions that produced them.

The observability cost chain

A useful cost model starts before ingestion.

Consider an API team that decides to log complete request payloads.

The code change may be one additional log statement.

At sufficient request volume, however, it can increase:

• application I/O;
• bytes collected;
• collector throughput;
• network traffic;
• ingestion volume;
• parsing and enrichment work;
• indexed data;
• searchable storage;
• replicas;
• retained data;
• archive volume;
• query scan volume.

If additional high-variability fields are extracted or indexed, the downstream effect can grow again.

The vendor's price per unit did not change.

The workload did.

This is the central concept of observability cost anatomy:

A high observability bill is often the final output of decisions made upstream in instrumentation, telemetry pipelines, retention and governance.

Commercial pricing can amplify or mitigate those decisions, but it usually does not explain them on its own.

The three layers of observability cost

Layer 1: Telemetry generation

This is what applications and infrastructure create.

Examples include:

• log events and event size;
• metric samples;
• unique time series;
• spans per request;
• span attributes;
• profiles;
• Kubernetes events;
• infrastructure metrics.

Generation is driven by application code, instrumentation libraries, framework defaults, platform configuration and operational practices.

An expensive downstream workload can therefore originate in something as small as a new label, a shorter scrape interval or an additional debug statement.

Layer 2: Platform consumption

Once telemetry enters the observability pipeline, platforms perform work on it.

Depending on the product and contract, economic dimensions can include:

• GB ingested;
• events indexed;
• active metric series;
• datapoints;
• spans ingested;
• spans indexed;
• monitored hosts;
• pods or containers;
• compute capacity;
• storage;
• retention;
• data scanned by queries;
• rehydration;
• data transfer.

Datadog, for example, distinguishes log ingestion from indexed log events. Its APM model separately exposes APM hosts, GB of ingested spans and indexed spans. Elastic Observability Serverless instead measures ingest and retained data volume. Amazon OpenSearch Service can express the same underlying workload as instance or OCU consumption plus storage rather than a per-log-event charge.

The architecture may be similar.

The economic meter is not.

Layer 3: Operating cost

The vendor invoice is only one component of observability TCO.

Depending on the architecture, additional cost can include:

• platform engineering labor;
• OpenTelemetry collectors;
• Kafka or other buffering infrastructure;
• compute;
• memory;
• block storage;
• object storage;
• backups;
• network transfer;
• Kubernetes resources;
• upgrades;
• security hardening;
• lifecycle management;
• access control;
• capacity planning;
• incidents affecting the observability platform itself.

Network costs require particular care because treatment depends on the cloud service and topology. AWS, for example, applies standard data-transfer charges to Amazon OpenSearch Service but documents specific exceptions such as traffic between nodes inside a multi-AZ domain.

A lower software invoice can therefore coexist with a higher total observability cost.

AWS itself recommends considering operational management, networking, backups and availability when evaluating OpenSearch TCO rather than comparing infrastructure or licensing alone.

Where your log budget actually goes

Logs are frequently blamed for observability overspend because their volume is highly visible.

But log volume is not one economic state.

A log can move through:

generated → collected → ingested → processed → indexed → retained → archived → rehydrated → queried

Each stage can have different economics.

Log generation

Log economics begin in the application.

Important volume multipliers include:

• request throughput;
• events emitted per request;
• average event size;
• logging level;
• repeated stack traces;
• complete request or response bodies;
• framework verbosity;
• Kubernetes and infrastructure logs.

A service receiving ten times more traffic can generate approximately ten times the events even if the observability configuration itself has not changed.

This is why useful cost attribution eventually needs to reach the service and workload, not just the index or observability account.

Log ingestion

Ingestion is a major economic dimension on several platforms.

Microsoft states that log ingestion is the largest component of Azure Monitor charges for most customers. Datadog charges Log Management ingestion based on GB submitted to the service, while Splunk retains an ingest-pricing model based on GB/day. Grafana Cloud also measures log usage by ingested volume.

This creates an important architectural rule:

Filtering telemetry after a billable ingestion boundary may reduce indexing, retention or query cost without reducing the ingestion charge that has already been incurred.

Location matters.

Dropping an unnecessary event inside an application or collector can have different economics from ingesting it first and excluding it only from a later searchable tier.

Processing and enrichment

Parsing and enrichment consume resources somewhere.

They may also increase billable data size.

Azure Monitor explicitly documents that billable record size can include fields added during ingestion and transformations. Elastic Observability Serverless likewise measures fully enriched normalized data after the ingest pipeline and before Elasticsearch compression.

Enrichment therefore needs two tests:

Does this field create operational value?

and:

Does that field need to exist on every event?

Attributes such as service, team and environment can materially improve troubleshooting and cost attribution.

Adding dozens of rarely used attributes to billions of events may simply create more processing and retention work.

Indexing

Collection, ingestion and indexing are not synonyms.

Some architectures ingest a broad telemetry stream but give only a subset expensive, low-latency search treatment.

Datadog illustrates this distinction clearly: it meters logs submitted for ingestion separately from events sent to an index under a selected retention policy.

That creates a major economic decision:

How quickly and flexibly does this dataset actually need to be searchable?

A production error stream used during active incident response may justify immediate indexing.

A high-volume dataset kept mainly for historical or audit purposes may have very different retrieval requirements.

Storage, replication and retention

Retention multiplies accumulated data over time.

If approximately the same amount of data arrives every day, keeping 90 days instead of 15 days means maintaining roughly six times as many days of data before accounting for compression, lifecycle tiers or deletion.

Physical storage can then be affected further by replica policy.

Modern architectures mitigate this using different storage tiers.

Elastic, for example, separates hot, warm, cold and frozen data tiers. Fully mounted searchable snapshots on the cold tier do not require normal replicas and can reduce local disk requirements, while the frozen tier trades additional query latency for substantially lower local storage requirements.

Azure Monitor similarly distinguishes interactive retention from cheaper long-term retention options.

For datasets subject to specific immutability requirements, WORM-capable object storage may also be relevant. Amazon S3 Object Lock, for example, can prevent objects from being overwritten or deleted for a defined retention period. Whether such storage satisfies a particular regulatory requirement must be validated against the organization's legal and compliance obligations rather than assumed.

The useful retention question is therefore not:

How many days should we keep our logs?

It is:

For how long does each dataset need each level of retrieval performance and mutability?

Query workload

Queries can also generate consumption.

The effect depends heavily on platform economics.

Dynatrace currently offers a log model that separately meters ingestion, retention and GiB scanned by queries. Grafana Cloud Logs includes a large fair-use query allowance, but excess query volume can become billable. Grafana specifically identifies misconfigured alerting rules that repeatedly scan large windows as one possible source of excess query consumption.

A dashboard querying a large period every 30 seconds is therefore economically different from a narrowly scoped troubleshooting query executed twice during an incident.

The same applies to:

• alert rules;
• scheduled searches;
• recording rules;
• automated analytics.

Query architecture can be a cost driver, not merely a user-experience concern.

Where your metrics budget actually goes

Metrics are compact compared with verbose logs.

That does not automatically make them inexpensive.

Their economics are driven heavily by series count and sampling frequency.

A metric name is not a time series

Consider:

http_requests_total

It appears to be one metric.

Now attach:

• service;
• environment;
• region;
• status;
• method;
• endpoint;
• namespace.

Every label combination that occurs can create a distinct time series.

If an environment contains:

20 services
× 3 environments
× 5 regions
× 6 status categories
× 10 endpoints

the theoretical combination space is already:

18,000 time series for one metric family.

Not every combination necessarily exists in production.

The important mechanism is multiplication.

Adding user IDs, container IDs, pod identities or other highly variable dimensions can increase series count dramatically.

This is why:

Metric cardinality is not simply “more metadata.” It changes the number of time series the observability system must ingest, store and query.

Active series and datapoints are different cost drivers

Grafana Cloud illustrates another important distinction.

Its Prometheus metrics model evaluates both:

• active series — unique series recently receiving data;
• datapoints per minute — the rate at which samples are submitted.

Its billing calculation can depend on both.

That means two independent decisions can increase metric consumption.

Cardinality

More unique label combinations create more series.

Sampling frequency

A shorter scrape interval produces more samples for those series.

Changing an interval from 60 seconds to 15 seconds does not necessarily create new time series.

It does generate approximately four times as many samples over the same duration.

Those are different optimization levers.

Cardinality is not only a storage problem

More series can also create:

• greater ingestion load;
• larger indexes or metadata structures;
• more memory pressure;
• more query fan-out;
• aggregation work;
• additional storage;
• vendor-specific billable consumption.

Datadog demonstrates why vendor terminology must remain explicit.

Under its cardinality-based custom-metrics model, a custom metric is identified by the metric name plus tag-value combinations, and billing uses the monthly average number of distinct custom metric timeseries per hour.

Datadog also offers a separate Metric Name pricing model for contracts using those SKUs. That model bills unique metric names and datapoint volumes rather than the older timeseries-cardinality model. The two models are mutually exclusive for a given contract.

“Datadog custom metric cost” therefore does not have one universally valid counting rule.

The contract matters.

Histograms multiply series by design

Latency should rarely be reduced to a simple average.

Histograms preserve distribution information and make tail latency observable.

But classic histogram economics need to be understood.

In Prometheus, a classic histogram produces:

one time series for each exposed bucket + _sum + _count

for every unique combination of the histogram's other labels.

If a histogram exposes ten bucket series, each label combination produces twelve series before considering any other metric families.

With 18,000 active label combinations, that would represent:

216,000 time series.

This does not mean histograms should be removed.

It means bucket configuration and label design matter economically.

Prometheus native histograms change this model. A native histogram stores the count, sum and dynamic buckets in a composite sample represented as one time series rather than separate bucket series. Prometheus can also map OpenTelemetry exponential histograms into its native histogram representation when the pipeline and backend support it.

Native histograms have their own storage, resolution and compatibility considerations.

The architectural lesson is simpler:

Histogram representation is an observability cost decision, not merely an instrumentation detail.

Where your traces and APM budget actually goes

Tracing follows another multiplication chain:

request volume × spans per request × span size × collection/sampling policy × retained volume

A high-throughput service can therefore create substantial trace consumption even when an individual span is small.

Requests are not traces, and traces are not spans

A trace describes a distributed operation.

That trace may include spans from:

• gateways;
• authentication;
• application services;
• database calls;
• caches;
• queues;
• external APIs.

As systems become more distributed or instrumentation becomes deeper, the number of spans associated with one request can increase.

Tracing economics therefore depend on both:

request throughput

and:

instrumentation depth

Span attributes have a cost

Attributes improve trace usefulness.

They also increase the information attached to every span.

The right decision is not to remove attributes indiscriminately.

It is to distinguish attributes that help with:

• diagnosis;
• correlation;
• routing;
• ownership;
• filtering;

from attributes retained simply because instrumentation makes them available.

In volume-based tracing systems, span size can directly affect ingestion and retention. In other platforms, the economic effect may appear through indexed spans, hosts, compute or another unit instead.

Sampling changes trace economics

OpenTelemetry distinguishes two major strategies.

Head sampling makes the sampling decision early.

It is efficient, but the sampler does not yet know how the full request will end. It cannot guarantee that a trace containing a later error or unusual latency will be retained.

Tail sampling evaluates all or most of the completed trace, allowing policies based on characteristics such as error status or latency.

Tail sampling can preserve higher-value traces while dropping less informative traffic.

But that capability has infrastructure consequences.

The OpenTelemetry Collector tail-sampling processor groups spans by trace ID, requires all spans for a trace to reach the appropriate collector instance for effective sampling, retains traces while waiting for a decision and exposes configuration limits around traces kept in memory. Its documentation explicitly notes that increasing trace capacity can increase memory usage.

At scale, tail sampling may therefore require:

• trace-aware routing;
• sampler capacity;
• memory;
• buffering;
• failure handling;
• additional operational work.

This produces an important Observability FinOps principle:

An optimization can move cost from one layer of the telemetry architecture to another rather than eliminating it.

Reducing backend trace storage can still be economically attractive.

The sampler must simply be included in the TCO.

Sampling should optimize information, not only percentage

A flat 1% sampling rate can be inexpensive.

It can also miss rare failures.

A more useful policy may preserve disproportionately valuable traces such as:

• errors;
• high latency;
• selected critical services;
• unusual endpoints;
• representative successful requests.

The objective is not the lowest possible sampling rate.

It is the best diagnostic coverage that the organization can justify economically.

Kubernetes: the observability cost multiplier

Kubernetes does not inherently make observability expensive.

It creates conditions in which telemetry can grow quickly if collection is not governed.

A relatively static infrastructure model may look like:

host → application

Kubernetes introduces:

cluster → node → namespace → workload → deployment → ReplicaSet → pod → container

Many of those objects are ephemeral.

Ephemeral dimensions create churn

Pod names, container IDs and generated workload identifiers can be useful during diagnosis.

But when those dimensions propagate indiscriminately into metric series or attribution models, every deployment can create new identities.

Grafana's cost-attribution guidance therefore recommends stable dimensions such as service, team and environment and advises against using unstable values such as pod names, container IDs and frequently changing versions for cost attribution.

This does not mean ephemeral dimensions should never exist.

It means they should be retained where they provide diagnostic value rather than automatically promoted into every economic dimension.

Kubernetes produces several telemetry layers

A cluster may expose telemetry from:

• applications;
• containers;
• kubelet;
• cAdvisor;
• kube-state-metrics;
• control-plane components;
• ingress;
• service mesh;
• networking;
• autoscalers;
• storage;
• cloud integrations.

Kubernetes itself documents separate kubelet metric endpoints including /metrics, /metrics/cadvisor, /metrics/resource and /metrics/probes. kube-state-metrics independently creates telemetry from the state of Kubernetes API objects such as pods, deployments and nodes.

kube-state-metrics explicitly warns that large numbers of frequently updated Kubernetes resources can create high ingestion volumes and recommends controlling which metrics and resources are exposed.

Multiple telemetry pipelines can also overlap.

For example, a Prometheus deployment and another monitoring integration may both collect related container or infrastructure signals.

That does not automatically mean one copy is useless. Different systems may have legitimate consumers.

It does mean new collectors and integrations should be checked for incremental diagnostic value versus duplicate consumption.

Autoscaling changes telemetry consumption

Kubernetes can scale application resources correctly while observability follows a separate cost curve.

More workloads can mean:

• more log sources;
• more infrastructure series;
• more resource identities;
• more traces;
• more monitored pods or containers under entity-based pricing.

Dynatrace, for example, currently exposes Kubernetes Platform Monitoring priced per pod, while other capabilities use host, memory, datapoint or telemetry-volume dimensions.

Kubernetes FinOps and Observability FinOps are therefore related but not identical.

Reducing cluster compute does not necessarily reduce every observability meter.

Retention is an architectural decision, not a global number

Requirements such as:

“Keep everything for 90 days.”

are not complete retention strategies.

They apply one number to datasets with different value profiles.

Retention should distinguish at least:

• incident-response data;
• trend data;
• capacity-planning data;
• security telemetry;
• audit records;
• regulatory records.

These uses do not necessarily require the same:

• granularity;
• latency;
• indexing;
• retention period;
• mutability.

A detailed debug event may be highly valuable during the first week after an incident and almost irrelevant months later.

High-resolution infrastructure metrics may be useful for recent diagnosis while aggregates are sufficient for long-term capacity analysis.

Certain audit datasets may require long retention but little interactive querying.

Modern platforms increasingly expose these differences economically through tiering and retention models. Elastic supports hot, warm, cold and frozen data tiers. Azure Monitor distinguishes table plans and long-term retention. Dynatrace separately meters ingestion, retention and, for some telemetry types, query consumption.

The right question is therefore:

What retrieval capability does this telemetry require at each stage of its life?

That question is more useful than assigning one retention period to every signal.

Vendor bill versus total observability cost

Suppose an organization compares:

Platform A: €400,000 managed-service invoice

with:

Platform B: €150,000 software or service cost

It has not yet calculated TCO.

A comparable total cost needs to consider the operational capability required around each platform.

For a managed system, parts of the following may already be embedded in the price:

• infrastructure;
• upgrades;
• backups;
• platform availability;
• capacity management;
• product maintenance.

For a self-managed architecture, some of those responsibilities move inside the organization.

A practical TCO model therefore considers:

vendor or software cost
+ cloud/on-prem infrastructure
+ telemetry pipeline infrastructure
+ storage
+ network
+ platform engineering labor
+ operational overhead and risk

The exact weighting is organization-specific.

This is why neither of these statements is generally true:

“SaaS is always more expensive.”

“Self-hosting is always cheaper.”

Amazon OpenSearch Service itself distinguishes managed-cluster costs such as instances, storage and transfer while highlighting operational work covered by a managed service as part of TCO.

At sufficient scale, an organization with strong platform engineering capabilities may operate a self-managed architecture economically.

In a smaller team, engineering and reliability overhead can erase much of the apparent licensing advantage.

Platform economics matter.

But they should generally be analyzed after the workload itself is understood.

Otherwise the organization risks moving inefficient telemetry to a different backend without correcting the mechanism generating the cost.

Cost attribution: who is generating the observability bill?

Observability spend is difficult to govern when nobody can answer:

• Which team generated this consumption?
• Which service?
• Which application?
• Which namespace?
• Which environment?
• Which business unit?
• Which telemetry type?

Without attribution, observability behaves like an unlimited shared resource.

The team adding telemetry receives its diagnostic benefit.

The platform organization often absorbs the incremental cost.

That weakens the economic feedback loop.

Build attribution around ownership

Useful attribution dimensions can include:

• team;
• service;
• application;
• environment;
• namespace;
• business_unit;
• cost_center;
• telemetry type.

Not every telemetry item needs every dimension.

The objective is sufficient coverage to aggregate consumption back to meaningful owners.

Grafana Cloud, for example, supports cost attribution for metrics, logs and traces using telemetry labels and can break consumption down across teams, services, departments or systems. Datadog similarly provides usage-attribution capabilities based on tags for supported products.

Stable dimensions matter.

A team or service identifier is generally more useful for monthly economic accountability than a pod ID that disappears during the next deployment.

Showback before chargeback

Formal internal billing is not always necessary.

A useful first step is often showback:

make consumption visible to the team that owns it.

For example:

• Team A generates 31% of log ingestion.
• Service B represents 22% of trace volume.
• One metric family represents 18% of active series.
• A non-production environment generates 27% of retained logs.

Those observations create an engineering conversation.

They are much more actionable than:

“The observability bill went up 14%.”

What should an Observability FinOps team measure?

Observability FinOps is the practice of connecting telemetry consumption and architecture to financial accountability so that cost can be controlled without indiscriminately removing diagnostic capability.

Do not begin with one global invoice number.

Begin with consumption.

And do not force every platform into the same artificial unit.

Track the economic dimensions that actually drive the contract or infrastructure.

Logs

Measure where available:

• generated or collected GB;
• ingested GB;
• indexed events or GB;
• average event size;
• retention by dataset;
• archive volume;
• retrieval or rehydration;
• query consumption where material;
• ingestion by team and service.

Metrics

Measure:

• active series;
• samples or datapoints;
• scrape intervals;
• highest-cardinality metric families;
• label keys responsible for series growth;
• histogram series;
• unused or rarely queried metrics;
• series by team and service.

Traces

Measure:

• spans generated;
• bytes ingested;
• spans per request;
• sampling rate by service;
• retained or indexed spans where relevant;
• average span size;
• retention;
• collector and sampler resources.

Platform

Measure:

• compute;
• memory;
• storage by tier;
• network transfer where billable;
• collector resources;
• buffering infrastructure;
• platform-engineering effort;
• operational incidents.

Derive unit economics only where they are meaningful

Depending on the architecture, useful derived indicators might include:

• cost per GB of log ingestion;
• cost per active metric series;
• cost per million spans;
• observability cost per service;
• observability cost per environment;
• observability cost per business unit.

These are analytical ratios.

They are not universal pricing units and should not be used to normalize vendors blindly.

The purpose is to turn consumption into engineering decisions.

How to reduce observability costs: the order of operations

Optimization order matters.

Deleting telemetry before understanding its value creates risk.

Changing platform before understanding the workload can simply relocate the problem.

A stronger sequence is:

1. Measure

Establish the baseline.

Identify the actual economic dimensions:

• ingestion;
• series;
• datapoints;
• spans;
• hosts;
• pods;
• storage;
• retention;
• queries;
• compute.

2. Attribute

Connect consumption to ownership:

team → service/application → environment → telemetry type

If a large part of spend cannot be attributed, fixing attribution may be more valuable than immediately optimizing it.

3. Find high-cost, low-value telemetry

Investigate telemetry that appears to be:

• duplicated;
• accidentally generated;
• excessively verbose;
• produced by obsolete services;
• rarely used;
• collected by overlapping integrations;
• retained without a documented purpose.

“Rarely queried” does not automatically mean “worthless.”

Audit, security and forensic telemetry can be valuable precisely because it is used infrequently.

4. Control cardinality and dimensionality

For metrics, identify the labels and metric families responsible for series growth.

Review histogram configuration.

For logs and traces, investigate whether high-variability fields require the current indexing, enrichment or retention treatment.

5. Align retention with value

Separate:

• high-performance searchable retention;
• lower-cost searchable retention;
• archive;
• immutable retention where required;
• deletion.

Do this by dataset, not globally.

6. Optimize tracing and sampling

Determine which workloads justify:

• full capture;
• probabilistic sampling;
• tail-based policies;
• error or latency preservation;
• differentiated service policies.

Include sampler infrastructure in the economic model.

7. Optimize routing and processing

Ask whether every signal needs the same destination.

Possible paths include:

• premium searchable storage;
• lower-cost search tiers;
• object storage;
• security-specific systems;
• aggregated metrics;
• pre-ingestion filtering.

8. Optimize platform architecture

Only now should the organization aggressively tune:

• cluster topology;
• storage tiers;
• compute;
• shards or partitions;
• collectors;
• queues;
• data movement.

9. Evaluate commercial and platform economics

Finally, test the normalized workload against:

• the current contract;
• alternative commercial models;
• managed versus self-managed economics;
• alternative architectures.

A platform change may be justified.

But it is the final economic question in this framework, not the starting point.

Which telemetry should you optimize first?

There is no universal rule that logs are always the most expensive signal.

Logs often dominate high-volume environments. Microsoft, for example, states that ingestion is the largest Azure Monitor log-related charge for most customers.

Another environment may instead be dominated by:

• high-cardinality custom metrics;
• dense histogram metrics;
• full-fidelity distributed traces;
• APM entity licensing;
• long-term storage;
• heavy query workloads;
• the labor required to operate a self-managed platform.

The right priority is:

Optimize the largest material cost driver where consumption can be reduced without disproportionate loss of diagnostic or operational value.

A 20% reduction in a minor telemetry category can matter less than a 5% reduction in the workload responsible for half the observability TCO.

Observability cost-driver decision table

When optimization is enough — and when architecture needs to change

Optimization is usually the first response when cost is driven primarily by:

• unnecessary telemetry;
• excessive retention;
• cardinality growth;
• ineffective sampling;
• duplicate collection;
• poor attribution;
• inappropriate storage treatment.

Architecture deserves deeper reconsideration when these issues have already been addressed and the normalized workload remains structurally expensive.

Examples include:

• the platform's pricing dimension is poorly matched to the workload;
• required retention cannot be delivered economically;
• query requirements conflict with the storage model;
• platform operating effort becomes disproportionate;
• contractual economics remain unacceptable after consumption is controlled.

That is different from:

“The invoice is high, therefore we should migrate.”

At that stage, platform-specific economics can be investigated using dedicated analyses rather than turning observability cost optimization itself into a migration project.

What should a CTO ask before approving an observability optimization project?

1. What are we actually paying for?

Not products.

Economic dimensions:

ingestion, hosts, series, spans, retention, compute, storage, network and engineering labor.

2. Who generates that consumption?

Which:

• teams;
• services;
• applications;
• clusters;
• namespaces;
• environments?

If that cannot be answered, attribution is an immediate priority.

3. Which telemetry produces operational value?

Look beyond simple query frequency.

Determine which telemetry contributes to:

• incident diagnosis;
• alerts;
• SLOs;
• security investigations;
• capacity planning;
• compliance.

4. Which consumption dimensions are growing fastest?

The current invoice is backward-looking.

Growth in active series, spans per request, ingestion or retained data can reveal the next cost problem before the next renewal.

5. Which retention requirements are genuinely different?

Separate operational requirements from audit, security and regulatory requirements.

They may need different storage architectures.

6. What capability disappears if we optimize this?

Every major reduction should include an explicit diagnostic and reliability impact assessment.

7. Are we optimizing unit price or unit consumption?

Both matter.

Negotiation changes what each unit costs.

Telemetry architecture changes how many units the system consumes.

They are different economic levers.

The governing principle: cost per useful diagnostic signal

“Cost per GB” is a commercial ratio.

It is not an observability outcome.

Organizations pay for observability because telemetry helps them:

• detect failures;
• understand systems;
• isolate causes;
• recover faster;
• protect reliability;
• investigate security events;
• demonstrate compliance.

A cheap dataset that nobody can use is waste.

An expensive signal that materially reduces the impact or duration of critical incidents can be valuable.

This does not make cost per useful diagnostic signal a perfectly measurable universal KPI.

Diagnostic value is contextual.

It is better treated as a decision principle:

Reduce observability consumption where the marginal cost exceeds the operational value of the telemetry being retained.

That requires context invoices cannot provide automatically.

Ask:

• Which telemetry contributes to alerts?
• Which data is actually used during incidents?
• Which signals support SLOs?
• Which datasets have security or compliance value?
• Which dimensions enable root-cause isolation?
• Which telemetry could be aggregated?
• Which data could move to a slower tier?
• Which signals are collected but have no identifiable consumer?

This is where Observability FinOps becomes more than cost reporting.

It connects financial management with observability architecture and reliability engineering.

FAQ

What drives observability costs?

Observability costs are driven by telemetry generation and by what happens to that telemetry afterward: collection, ingestion, processing, indexing or time-series creation, storage, retention, querying, infrastructure consumption and platform operations. Vendor pricing determines how those activities appear on the invoice, but upstream architecture determines much of the consumption.

Why is our observability bill so high?

A high bill can result from vendor pricing, contract structure or inefficient telemetry consumption — and frequently from a combination of all three. The fastest way to understand the cause is to map billable dimensions back to teams, services and telemetry types rather than looking only at the invoice total.

Are logs usually more expensive than metrics or traces?

Not necessarily. Logs can dominate high-volume environments, while high-cardinality metrics, dense histograms or full-fidelity distributed tracing can dominate others. The relevant question is which signal generates the largest marginal cost in your architecture.

Why does metric cardinality increase observability cost?

Every distinct label combination can create another time series. Adding labels with many possible values can therefore multiply series count rather than simply adding metadata. More series can increase ingestion, memory, storage, query work and vendor-specific billable usage. Datadog's cardinality-based custom-metrics model and Grafana Cloud's active-series model are examples of pricing structures in which series count has direct economic significance.

Why can histograms become expensive?

A Prometheus classic histogram emits one time series per bucket plus separate sum and count series for every label combination. Histograms combined with high-cardinality dimensions can therefore produce very large series counts. Native histograms change this representation by storing the distribution in composite histogram samples within one series.

What is the difference between log ingestion and retention cost?

Ingestion is the cost or resource consumption associated with bringing telemetry into the platform. Retention is the cost of keeping that telemetry available over time. Some products bundle parts of these economics; others meter them independently. Filtering after the billable ingestion boundary may therefore reduce downstream storage without reducing the ingestion consumption already incurred.

How does sampling reduce tracing cost?

Sampling reduces the amount of trace data that proceeds through later stages of the pipeline. Head sampling makes the decision early and cheaply. Tail sampling can preserve traces based on completed outcomes such as errors or latency, but it requires additional state and processing infrastructure.

Why can Kubernetes observability become expensive?

Kubernetes creates many dynamic resources and telemetry sources. Kubelet, cAdvisor, kube-state-metrics, applications, service meshes and monitoring integrations can all generate signals. Ephemeral dimensions can create series churn, while overlapping collectors can duplicate some consumption. Kubernetes itself and kube-state-metrics expose numerous resource-specific metrics, and kube-state-metrics explicitly warns that frequently changing resources can create significant ingestion volumes.

How does retention affect observability TCO?

Longer retention increases the amount of data maintained over time, but cost depends on the storage architecture. Hot searchable storage, cold searchable data, archive and immutable retention can have very different economics. Retention should therefore be designed by dataset and retrieval requirement rather than through one global number.

What is Observability FinOps?

Observability FinOps connects telemetry consumption to financial accountability. It measures the economic drivers of logs, metrics and traces, attributes those drivers to owners, governs telemetry generation and optimizes architecture while protecting necessary diagnostic capability.

How should observability costs be allocated?

Start with reliable showback using stable ownership dimensions such as team, service, application, environment or business unit. Cost attribution should identify who generates telemetry consumption before an organization considers formal chargeback. Grafana Cloud and Datadog both expose mechanisms for attributing supported telemetry usage through labels or tags.

When is self-hosted observability cheaper?

Self-hosting is cheaper only when the reduction in managed-service or software spend exceeds the additional infrastructure, storage, network, platform-engineering and operational costs required to provide the needed capability and reliability. Software price alone is not sufficient to calculate observability TCO.

Should we change observability platforms to reduce costs?

Not before establishing the workload economics. If excessive telemetry, cardinality, retention or duplicate collection is the primary driver, those problems can survive a migration. Platform changes deserve investigation when the normalized workload remains structurally expensive under the current commercial or architectural model.

Conclusion: understand the telemetry economics before changing the platform economics

An observability bill is the end of a chain.

Applications generate telemetry → collectors transport it → pipelines process it → platforms ingest or index it → retention accumulates it → engineers and automation query it → infrastructure and vendors meter the resulting consumption.

That chain eventually becomes:

total observability cost.

Optimizing only the final vendor price leaves the upstream architecture untouched.

Optimizing only telemetry volume can be equally dangerous if it removes the evidence engineers need during incidents.

The first objective should therefore be to establish an economic map of the observability environment:

• what is generated;
• what is consumed;
• what is charged;
• who generates it;
• why it is retained;
• which signals provide diagnostic value;
• which architectural decisions multiply cost downstream.

Once that map exists, an organization can distinguish:

quick wins from risky cuts,

and:

telemetry problems from platform-economic problems.

Observability FinOps Assessment

SAB Consulting's Observability FinOps Assessment is designed to establish that baseline before significant optimization or platform decisions are made.

The assessment focuses on:

• establishing the current observability cost and consumption baseline;
• attributing telemetry consumption to services, teams and environments;
• identifying high-cost telemetry with weak or unclear operational value;
• analyzing metric cardinality and dimensionality;
• reviewing retention and storage economics;
• reviewing trace-generation and sampling economics;
• separating vendor spend from infrastructure and operating cost;
• distinguishing low-risk optimization opportunities from reductions that could weaken diagnosis or reliability;
• identifying architecture-level optimization opportunities;
• determining whether the remaining economics justify investigating a platform or commercial-model change.

Samuel Abouelfateh, Founder of SAB Consulting, works across Observability FinOps, observability architecture, Kubernetes, Elasticsearch and OpenSearch environments.

The objective of the assessment is not to begin with a preferred vendor outcome.

It is to answer a more fundamental question:

What exactly are you paying to observe, who is generating that cost, and which parts of it are economically justified?