OpenTelemetry OpenTelemetry

Deploy the Collector on Kubernetes with Helm

Install the official chart, use presets for logs and metrics, and override the config through values files.

12 min Intermediate

The OpenTelemetry Helm chart deploys the Collector as a DaemonSet, a Deployment, or a StatefulSet, and generates a sensible default configuration. You customize it entirely through a values.yaml file, which makes it a natural fit for GitOps. This guide installs it, uses presets for common needs, and shows how to override the Collector config.

Install

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts
helm repo update

helm install my-opentelemetry-collector open-telemetry/opentelemetry-collector \
  --namespace observability --create-namespace \
  --set image.repository="otel/opentelemetry-collector-k8s" \
  --set mode=daemonset

Two settings are required:

  • mode must be daemonset, deployment, or statefulset. There is no default.
  • image.repository chooses the distribution. The Kubernetes distribution (otel/opentelemetry-collector-k8s) includes the components the presets need. Use the contrib image if you need components outside it.

Pin the chart with --version and the image with image.tag, and keep everything in a values file rather than --set flags, so it’s reviewable:

# values.yaml
mode: daemonset
image:
  repository: otel/opentelemetry-collector-k8s
  tag: "0.163.0"   # pin to a version you have tested
helm install otel-agent open-telemetry/opentelemetry-collector \
  -n observability --create-namespace -f values.yaml --version <chart-version>

Which mode for which job

  • daemonset: one pod per node. Use it for log collection, kubelet metrics, and host metrics, because those only exist on the local node.
  • deployment: a scalable service. Use it for a gateway that applications send to.
  • statefulset: when you need stable storage or identity, for example a persistent queue or the Target Allocator pattern.
  • For data that should be collected once per cluster (cluster metrics, Kubernetes events), use a Deployment or StatefulSet with a single replica, or you’ll get duplicates.

Presets

Presets switch on common configurations, including the receivers, processors, RBAC, and volume mounts they need:

# Agent: logs, host and kubelet metrics, and pod metadata on every node
mode: daemonset
presets:
  logsCollection:
    enabled: true
  hostMetrics:
    enabled: true
  kubeletMetrics:
    enabled: true
  kubernetesAttributes:
    enabled: true
# Cluster-wide metrics and events, once
mode: deployment
replicaCount: 1
presets:
  clusterMetrics:
    enabled: true
  kubernetesEvents:
    enabled: true

Presets need a Collector image that contains their components, so use the Kubernetes or contrib image. Treat them as a starting point: for unusual needs, configure the components and their RBAC and volumes yourself. All options are listed in the chart’s values.yaml.

Override the Collector configuration

The config key is merged over the chart’s default Collector config. Add or change what you need:

mode: deployment
replicaCount: 2

config:
  processors:
    memory_limiter:
      check_interval: 1s
      limit_percentage: 80
      spike_limit_percentage: 20
  exporters:
    otlp/tempo:
      endpoint: tempo.observability.svc.cluster.local:4317
      tls:
        insecure: true
  service:
    pipelines:
      traces:
        receivers: [otlp]
        processors: [memory_limiter, batch]
        exporters: [otlp/tempo]

Rules that trip people up:

  • The default exporter is debug. Replace it in each pipeline you use, as above.
  • Setting a default component to null removes it: config.receivers.zipkin: null.
  • Lists replace, they don’t merge. When you change a pipeline, list all of its components, including defaults you want to keep.
  • Disable unused ports to keep the Service tidy, for example ports.jaeger-compact.enabled: false.
  • Logs and the debug exporter: if you enable logsCollection with includeCollectorLogs: true and keep the debug exporter on the logs pipeline, the Collector can read its own logs and feed them back, causing a log explosion. Switch to a real exporter first.

A trimmed OTLP-only gateway:

config:
  receivers:
    jaeger: null
    prometheus: null
    zipkin: null
  service:
    pipelines:
      traces:
        receivers: [otlp]
        exporters: [otlp/tempo]
      metrics: null
      logs: null
ports:
  jaeger-compact:
    enabled: false
  jaeger-thrift:
    enabled: false
  jaeger-grpc:
    enabled: false
  zipkin:
    enabled: false

Secrets and environment variables

Inject credentials with environment variables and reference them in the Collector config with ${env:NAME}:

extraEnvs:
  - name: BACKEND_TOKEN
    valueFrom:
      secretKeyRef:
        name: otel-backend
        key: token

config:
  exporters:
    otlphttp/backend:
      endpoint: https://otlp.backend.example.com
      headers:
        authorization: Bearer ${env:BACKEND_TOKEN}

Keep the Secret out of Git in plain form: encrypt it with SOPS or sync it from a secret manager.

Deploy with Flux

If you use Flux, the chart is a HelmRelease:

apiVersion: source.toolkit.fluxcd.io/v1
kind: HelmRepository
metadata:
  name: open-telemetry
  namespace: flux-system
spec:
  interval: 1h
  url: https://open-telemetry.github.io/opentelemetry-helm-charts
---
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
  name: otel-agent
  namespace: observability
spec:
  interval: 10m
  chart:
    spec:
      chart: opentelemetry-collector
      version: "<chart-version>"
      sourceRef:
        kind: HelmRepository
        name: open-telemetry
        namespace: flux-system
  values:
    mode: daemonset
    image:
      repository: otel/opentelemetry-collector-k8s
    presets:
      kubernetesAttributes:
        enabled: true

A change to the values in Git then rolls out through Flux. The Flux series covers HelmRelease options such as remediation and drift detection.

Verify

kubectl get pods -n observability
kubectl logs -n observability daemonset/otel-agent-opentelemetry-collector | tail -n 20
kubectl port-forward -n observability svc/otel-agent-opentelemetry-collector 13133:13133 &
curl -s http://localhost:13133

Pod and Service names follow your release name. If a pod crash-loops, the log usually names the unknown component or invalid key. In that case, the image doesn’t contain a component your config references, or a value was indented under the wrong key.

Popular Guides