Kubernetes knows about CPU and memory. It does not know about queue depth, batch job duration, or active WebSocket connections. For most real-world applications, those are the signals that actually matter for scaling decisions. A custom metrics exporter bridges that gap by exposing application-specific metrics in a format Prometheus can scrape[reference:19].
This guide walks through building a custom exporter from scratch, packaging it as a container, and wiring it into a Kubernetes cluster so that Prometheus — and ultimately the HorizontalPodAutoscaler — can consume it[reference:20].
What an exporter actually does
An exporter is a small HTTP server with a single responsibility: expose application state as plain text on a /metrics endpoint[reference:21]. Prometheus scrapes that endpoint on a regular interval, stores the time-series data, and makes it available for queries, alerts, and autoscaling rules[reference:22].
In some cases you can instrument your application directly — embedding the Prometheus client library and exposing metrics from within the same process — rather than running a separate exporter. A standalone exporter makes more sense when the data source is external to your application or when you do not control the application code.
Choosing the right metric type
The Prometheus data model has three main types, and choosing the right one matters:
- Counters only ever increase. Use them for totals: requests served, jobs processed, errors encountered. Never use a counter for a value that can go down.
- Gauges represent a current snapshot that can rise and fall freely. Queue depth, active connections, and cache size are all gauges.
- Histograms record the distribution of observed values, such as request latency. They let you calculate percentiles (p99, p50) rather than just averages.
Once you know which type fits your signal, choose a name that follows Prometheus naming conventions. A job processor might expose jobs_processed_total (counter), queue_depth (gauge), and job_duration_seconds (histogram). Clear names save everyone debugging time later.
Writing the exporter in Go
The Go Prometheus client is the most common choice for exporters in the Kubernetes ecosystem, largely because the same library powers most of the official Kubernetes components. Start by creating a module and pulling in the dependency.
The first step is to declare the metrics and register them with Prometheus's default registry. Registration tells the library that these metrics exist so they appear in the output even before the first observation is recorded. prometheus.MustRegister panics on a duplicate registration, which makes misconfigurations obvious at startup rather than silently at runtime.
With the metrics registered, the next step is to keep them current. You can either update the data as it changes or run an internal refresh loop. A polling loop — a goroutine that periodically reads from whatever data source your application owns and updates the registered metrics — is a common pattern. Replace simulated values with real calls to your database, internal API, or message broker.
The polling interval should be shorter than Prometheus's scrape interval so that each scrape sees a fresh value. The default scrape interval in most cluster deployments is fifteen seconds, which gives you comfortable headroom.
Wire the collection loop and the HTTP handler together in main.go. A /health path alongside /metrics gives Kubernetes a liveness probe target without exposing metric data on the health route.
Containerizing the exporter
A multi-stage build keeps the final image small and avoids shipping a Go toolchain to production. The first stage compiles a statically linked binary; the second stage copies only that binary into a minimal base. Using scratch as the base image contains no shell, no package manager, and runs as a non-root user by default, which satisfies most cluster security policies without extra configuration.
Deploying to Kubernetes
Two manifests are enough to run the exporter: a Deployment that manages the pod lifecycle, and a Service that gives Prometheus a stable address to scrape. The Deployment should set conservative resource limits appropriate for a lightweight sidecar-style process and use the /health route for its liveness probe. The Service should name the port metrics so that the ServiceMonitor can reference it by name.
If you installed Prometheus using the Prometheus Operator or the Helm chart, the operator must be running in your cluster before you create a ServiceMonitor. The label on the ServiceMonitor must match the label selector configured on your Prometheus resource.
Verifying the pipeline
Port-forward to the Prometheus service and open the targets page to confirm the exporter has been discovered. Navigate to /targets. The target should appear with state UP. If it shows DOWN, check that the ServiceMonitor's label matches and that the pod is running.
Once the target is healthy, run a quick query in the expression browser to confirm data is flowing. A non-zero result here means the full pipeline is working: your application is producing data, Prometheus is scraping it, and the time-series are stored and queryable.
What comes next
A working exporter is the foundation, not the destination. The natural next step is surfacing these metrics to the HorizontalPodAutoscaler so that your workload scales on the signals that actually drive load, not just CPU. That requires a metrics adapter — the Prometheus Adapter is the most widely deployed option — which registers your custom metrics with the Kubernetes Custom Metrics API[reference:23]. Once registered, any HorizontalPodAutoscaler in the cluster can reference your custom metrics directly in its metrics block.