Skip to content

Log Collection Failed

Error message

                Error: Fluentd failed to send logs to Elasticsearch
              

Quick summary

The log collector (Fluentd, Filebeat) cannot forward logs to the storage backend, resulting in missing or delayed logs.

Why this happens

  • The log storage backend (Elasticsearch, Loki) is unavailable

  • The log collector is not running or is misconfigured

  • Network issues prevent the collector from reaching the log store

  • The log store is full or has insufficient disk space

Minimal example

✗ Broken code
# Fluentd cannot send to Elasticsearch
# Error: connection refused to elasticsearch:9200
✓ Fixed code
# Check and restart Elasticsearch
kubectl get pods -l app=elasticsearch
kubectl rollout restart deployment/elasticsearch
# Then restart Fluentd
kubectl rollout restart daemonset/fluentd

Elasticsearch is down, so Fluentd cannot send logs. The fix restarts Elasticsearch and then Fluentd.

How to diagnose

  • Check if the log collector is running: kubectl get pods -l app=fluentd

  • View collector logs for errors: kubectl logs <fluentd-pod>

  • Verify the log store is running and reachable

  • Check disk space on the log storage backend

How to fix

  • Restart the log collector: kubectl rollout restart daemonset/fluentd

  • Fix the log store connection (restart Elasticsearch, fix network)

  • Correct the collector configuration (URL, credentials, index)

  • Free up disk space on the log store or increase retention rotation

How to prevent

  • Monitor log collection health and alert on failures

  • Use buffer/queue in the collector to handle log store outages

  • Set up log retention policies to prevent disk full issues

Related Resources

Related Glossary

← Back to language