Analyze CrashLoopBackOff Pod Logs with Gemini
Automatically finds all Kubernetes pods in a CrashLoopBackOff state, extracts their recent logs, and uses Gemini to diagnose the root cause.
Setup
- → kubectl configured with cluster access
- → jq installed
- → gemini CLI installed and authenticated with an API key
Cost per run
Gemini API usage costs based on log token volume
The one-liner
$ kubectl get pods -A -o json | \
jq -r '.items[] | select(any(.status.containerStatuses[]?; .state.waiting.reason == "CrashLoopBackOff")) | "\(.metadata.namespace) \(.metadata.name)"' | \
xargs -r -n2 sh -c 'echo "\n--- Logs for $0/$1 ---"; kubectl logs -n "$0" "$1" --tail=50' | \
gemini -p "Analyze these logs from crashing Kubernetes pods and determine the root cause."What each stage does
- [01] kubectl
kubectl get pods -A -o jsonFetches the status of all pods across all namespaces in JSON format for reliable structured parsing. - [02] jq
jq -r '.items[] | select(any(.status.containerStatuses[]?; .state.waiting.reason…Filters the JSON to find pods with at least one container in CrashLoopBackOff, extracting the namespace and pod name as raw text. - [03] xargs
xargs -r -n2 sh -c 'echo "\n--- Logs for $0/$1 ---"; kubectl logs -n "$0" "$1" -…Reads the namespace and pod name pairs (-n2), safely passing them to a subshell to fetch the last 50 lines of logs for each pod. - [04] gemini
gemini -p "Analyze these logs..."Pipes the aggregated logs directly into the Gemini LLM CLI to generate a human-readable root cause analysis.
Expected output (sample)
**Root Cause Analysis:** 1. `frontend/web-app-7b89f`: The application is failing to start due to a missing environment variable `DATABASE_URL`. 2. `backend/api-worker-2c44a`: The container is running out of memory (OOMKilled) during the initial cache hydration phase. **Recommendation:** Update the ConfigMap for `web-app` and increase memory limits for `api-worker`.
Caveats & tips
- Cost/Permission: Requires a configured Gemini API key, which may incur token costs for large log payloads.
- Footgun: If many pods are crashing simultaneously, the combined logs might exceed the LLM's context window.
- Security: Ensure your pod logs do not contain sensitive PII or secrets before sending them to an external LLM API.