Fluentd is an open-source data collector that unifies log collection and consumption. It tails, parses, buffers, and routes log data from many sources to many destinations, decoupling how logs are produced from where they end up — a search index, object storage, a log analytics platform, or all three.

How it works

Fluentd runs as an agent (often as a Kubernetes DaemonSet, one per node) that reads log data through input plugins, optionally transforms it through filter plugins, and writes it out through output plugins. Its plugin architecture is what lets a single Fluentd pipeline read container logs from a node’s filesystem and fan them out to multiple destinations with different formats and retry semantics.

Core concepts

• Input plugin — reads log events from a source: a file, a TCP socket, the Kubernetes container log directory, and many others.
• Filter plugin — transforms or enriches events in the pipeline, such as parsing structured fields out of a raw log line or attaching Kubernetes metadata.
• Output plugin — writes events to a destination: Elasticsearch, S3, Kafka, a hosted log platform, and dozens more.
• Buffer — Fluentd’s internal queue between inputs and outputs, which absorbs backpressure and retries failed writes.

Why teams adopt Fluentd

• Vendor-neutral. A large plugin ecosystem means switching log destinations is a configuration change, not a re-architecture.
• Built for high volume. Its buffering and chunking model is designed to handle bursty, high-throughput log streams reliably.
• A structured log model. Events carry structured fields rather than raw text, making downstream filtering and routing far more precise.
• CNCF graduated. It’s the logging counterpart to Prometheus’ place in the observability stack — broadly adopted and actively maintained.

Getting started

Deploy Fluentd as a DaemonSet so an agent runs on every node, configure it to tail container logs, and add the output plugins for wherever you want logs to end up. The guides below cover the most common collection and routing setups.