Manage Grafana with the Grafana Operator
Declare instances, folders, dashboards, data sources, and alert rules as Kubernetes custom resources.
The Grafana Operator turns each Grafana resource into a Kubernetes object. A GrafanaDashboard is a manifest, a GrafanaDatasource is a manifest, and the operator reconciles them against a Grafana instance through its API. That fits GitOps tools like Flux and Argo CD naturally, and it works for Grafana instances inside or outside the cluster, including Grafana Cloud. All resources use apiVersion: grafana.integreatly.org/v1beta1.
Install the operator
Install it with Helm from its OCI chart (check the operator’s installation docs for the current chart location and version, and pin it):
helm upgrade -i grafana-operator \
oci://ghcr.io/grafana/helm-charts/grafana-operator \
--namespace grafana --create-namespace \
--version <operator-version>
The operator’s docs also describe a Kustomize installation. With Flux, put the same chart in a HelmRelease.
Describe the instance
An instance can be managed (the operator deploys Grafana) or external (it connects to one that already exists). A managed instance:
apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
name: grafana
namespace: grafana
labels:
dashboards: "grafana"
spec:
config:
log:
mode: console
security:
admin_user: root
admin_password: secret # example only; see below
An external instance, such as a Grafana Cloud stack, points at a URL and takes a token from a Secret:
apiVersion: grafana.integreatly.org/v1beta1
kind: Grafana
metadata:
name: my-stack
namespace: grafana
labels:
dashboards: my-stack
spec:
external:
url: https://my-stack.grafana.net/
apiKey:
name: grafana-cloud-credentials
key: GRAFANA_CLOUD_INSTANCE_TOKEN
The spec.config block takes the same sections and keys as grafana.ini. Don’t put a real password in a manifest. The operator documentation shows how to read admin credentials from a Secret.
The labels on the instance matter: every other resource selects its target instance with instanceSelector.matchLabels, using those labels.
Folders
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaFolder
metadata:
name: applications
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
title: Applications
uid: applications # set a stable UID so other resources can reference it
If title is omitted, it defaults to metadata.name. Set spec.uid whenever other resources will point at the folder with folderRef, because without it the reconciler may adopt an existing folder with the same name.
Data sources
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDatasource
metadata:
name: prometheus
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
datasource:
name: prometheus
type: prometheus
access: proxy
url: http://prometheus-service:9090
isDefault: true
jsonData:
timeInterval: 5s
Credentials come from a Secret in the same namespace through valuesFrom, so they never appear in the manifest:
apiVersion: v1
kind: Secret
metadata:
name: credentials
type: Opaque
stringData:
PROMETHEUS_USERNAME: grafana
PROMETHEUS_PASSWORD: change-me
---
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDatasource
metadata:
name: prometheus-secured
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
valuesFrom:
- targetPath: basicAuthUser
valueFrom:
secretKeyRef:
name: credentials
key: PROMETHEUS_USERNAME
- targetPath: secureJsonData.basicAuthPassword
valueFrom:
secretKeyRef:
name: credentials
key: PROMETHEUS_PASSWORD
datasource:
name: prometheus-secured
type: prometheus
access: proxy
basicAuth: true
url: http://prometheus-service:9090
secureJsonData:
basicAuthPassword: ${PROMETHEUS_PASSWORD}
Don’t put credentials in the url. Encrypt or sync the Secret with SOPS or External Secrets, as with any Kubernetes Secret.
Dashboards
A GrafanaDashboard can take its JSON from several sources, which is the operator’s biggest strength:
# inline JSON
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaDashboard
metadata:
name: simple
spec:
resyncPeriod: 30s
instanceSelector:
matchLabels:
dashboards: "grafana"
folderRef: applications
json: >
{ "uid": "simple", "title": "Simple Dashboard", "panels": [] }
# from a ConfigMap
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
configMapRef:
name: dashboard-definition
key: json
# from a URL, such as grafana.com or a raw Git file
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
url: "https://grafana.com/api/dashboards/1860/revisions/30/download"
# from an OCI artifact
spec:
instanceSelector:
matchLabels:
dashboards: "grafana"
oci:
reference: ghcr.io/team-a/dashboards:v1.4.7
path: board.json
pullSecretRef:
name: ghcr-pull
Other fields you will use:
folderRefpoints at aGrafanaFolderby name, andfolderUIDat an existing folder’s UID.gzipJsonholds the dashboard as base64-encoded gzip (cat dashboard.json | gzip | base64 -w0), for dashboards too large for a manifest.variablesoverrides template variables, for example to set anamespaceor data source per environment.resyncPeriodandcontentCacheDurationcontrol how often content is re-applied and how long fetched URLs are cached.
Alert rules
apiVersion: grafana.integreatly.org/v1beta1
kind: GrafanaAlertRuleGroup
metadata:
name: webapp-alerts
spec:
folderRef: applications
instanceSelector:
matchLabels:
dashboards: "grafana"
interval: 5m
rules:
- uid: webapp-down
title: Webapp targets down
condition: C
for: 5m
noDataState: NoData
execErrState: Error
data:
- refId: A
# the Prometheus query step, as in the alerting guide
- refId: C
# the threshold expression step
Every rule group needs a folder and at least one rule. The data steps use the same structure as provisioning files, so you can reuse the YAML from the alerting guide. The operator also has resources for contact points; see its examples for the current list.
Operate it
kubectl get grafanas,grafanafolders,grafanadatasources,grafanadashboards -A
kubectl describe grafanadashboard simple -n grafana
describe shows the resource’s status conditions, which say whether the last sync to Grafana succeeded and why not if it failed. Common causes: an instanceSelector that matches no instance, a wrong token, or a Secret in the wrong namespace.
Operator or file provisioning?
Use the Operator when you already run GitOps on Kubernetes, want each team to own its dashboards and alerts as separate objects, or manage Grafana Cloud or several instances. Use the Helm chart’s provisioning values when you have one Grafana and a handful of resources, since that adds no extra controller. Both can coexist, but let one tool own each resource.
Popular Guides
-
Operate Grafana as Code: The Big Picture What can live in files, what needs an API, and how the pieces fit together. -
Configure the Grafana Server with grafana.ini and Environment Variables Know where config files live, how overrides are ordered, and how to keep settings in Git. -
Provision Dashboards from JSON Files Load dashboards from disk, map folders from directories, and author them without losing changes. -
Provision Alert Rules, Contact Points, and Notification Policies Define the whole alerting pipeline in YAML so it is reviewed and versioned like code. -
Deploy Grafana on Kubernetes with Helm and Provisioning Values Run Grafana from a Helm chart where data sources, dashboards, and settings all come from values files. -
Reload, Verify, and Troubleshoot Provisioning from the Command Line Find out what Grafana loaded, reload without a restart, and fix the usual provisioning errors.