OpenTelemetry OpenTelemetry

The Collector Configuration Model: Components, Pipelines, and Providers

Learn the rules of the Collector's YAML: naming, pipelines, connectors, merging files, and environment variables.

12 min Beginner

Every Collector setup is one YAML document that follows the same few rules. Learning those rules once lets you read any Collector config, including the ones that ship with Helm charts and the Operator. This guide covers the structure, the naming scheme, how components are switched on, how to split and merge files, and how to inject environment variables.

The five component types

SectionJobExamples
receiversTake data inotlp, prometheus, filelog, hostmetrics
processorsTransform, filter, enrich, batchbatch, memory_limiter, k8s_attributes, tail_sampling
exportersSend data outotlp, otlphttp, debug, prometheus
connectorsJoin two pipelines, acting as an exporter in one and a receiver in anotherspanmetrics, forward, routing
extensionsAdd capabilities beside the pipelineshealth_check, zpages, file_storage, auth extensions

A sixth section, service, wires them together.

Components are off until service enables them

This is the single most common source of confusion. Configuring a receiver does not turn it on. A component only takes effect when a pipeline in service references it:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317

exporters:
  debug:
  otlp_grpc:
    endpoint: gateway.example.com:4317

service:
  extensions: [health_check, zpages]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp_grpc]
    metrics:
      receivers: [otlp]
      exporters: [otlp_grpc]
    logs:
      receivers: [otlp]
      exporters: [otlp_grpc]

A component that is defined but never referenced is ignored, which is useful for keeping variants in a file but easy to forget when nothing arrives. The pipelines are named traces, metrics, and logs, and a receiver, processor, or exporter must support the signal of the pipeline it joins.

Naming: type and type/name

A component identifier is its type, optionally followed by /name. Use names to configure the same component type more than once:

exporters:
  otlp/tempo:
    endpoint: tempo:4317
    tls:
      insecure: true
  otlp/vendor:
    endpoint: otlp.vendor.example.com:4317

service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [otlp/tempo, otlp/vendor]   # fan out to two backends

Each identifier must be unique. The fan-out above sends every span to both destinations.

Pipelines in detail

  • Processor order matters. Data passes through processors in the order listed. Put memory_limiter first and batch last (see the resiliency guide).
  • A component can appear in several pipelines. Each pipeline that uses a processor gets its own instance.
  • Connectors bridge pipelines. A connector is listed as an exporter of one pipeline and a receiver of another. For example, spanmetrics turns spans into metrics:
connectors:
  spanmetrics:

service:
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp/tempo, spanmetrics]
    metrics:
      receivers: [otlp, spanmetrics]
      processors: [batch]
      exporters: [otlphttp/prometheus]

Environment variables

Use ${env:NAME} anywhere a value is expected, and bash-style defaults with :-:

exporters:
  otlp:
    endpoint: ${env:OTLP_ENDPOINT:-gateway.example.com:4317}
    headers:
      authorization: Bearer ${env:API_TOKEN}

To write a literal dollar sign, use $$. Because secrets arrive by environment variable, the file itself stays free of them and can be committed.

Config sources and merging files

--config accepts a path, or a scheme:value URI with the schemes file, env, yaml, http, and https:

otelcol --config=file:/etc/otelcol/base.yaml --config=file:/etc/otelcol/overrides.yaml
otelcol --config=env:MY_CONFIG_IN_AN_ENVVAR
otelcol --config="yaml:exporters::debug::verbosity: normal"

When you pass several --config options, the Collector merges them into one configuration. Each file may be partial, but the merged result must be complete, because the Collector doesn’t add components for you. Use this to keep a shared base file and a small per-environment override.

You can also embed another file into a value:

exporters: ${file:exporters.yaml}

The http and https providers load configuration from a URL, which is convenient but means the Collector’s behavior depends on that server, so use them deliberately.

Validate

otelcol validate --config=config.yaml
# or with the contrib image
docker run --rm -v $(pwd)/config.yaml:/etc/otelcol/config.yaml:ro \
  otel/opentelemetry-collector-contrib:0.163.0 validate --config=/etc/otelcol/config.yaml

validate catches unknown components, bad syntax, and pipelines that reference something you didn’t define. It doesn’t prove that a backend is reachable.

Security default to know

Receivers bind to localhost by default. That’s safe when every sender runs on the same host, but it means a Collector in a container is unreachable from outside until you set an endpoint such as 0.0.0.0:4317. When you open an endpoint to the network, add TLS and authentication (see the backends guide), and restrict access with network policy or firewall rules.

A readable layout for real projects

collector/
├── base.yaml            # receivers, processors, shared pipelines
├── env/
│   ├── dev.yaml         # exporters and endpoints for dev
│   └── prod.yaml
└── README.md

Run it with --config=file:base.yaml --config=file:env/prod.yaml. Reviewers can see exactly what changes per environment.

Quick reference

  • Define components at the top, enable them in service.
  • Use type/name to repeat a component type.
  • Order processors deliberately.
  • Use ${env:VAR} for anything environment-specific or secret.
  • Merge several --config files, and run validate in CI.

Popular Guides