Kubernetes OOMKilled (Exit Code 137): Causes and Fixes

A container that was running fine suddenly restarts, and kubectl describe pod shows Last State: Terminated, Reason: OOMKilled, Exit Code: 137. No stack trace, no application error—the process was killed from underneath the app. This usually means Linux cgroup OOM handling killed a process after the container exceeded its memory limit. Exit code 137 alone is not proof of OOM, but Reason: OOMKilled is the decisive signal.

OOMKilled is one of the most common reasons pods restart, and unlike a code crash it leaves almost no trace in your application logs. This guide explains what's happening, why exit code 137 appears, and how to diagnose and fix out-of-memory kills for good.

What OOMKilled and Exit Code 137 Mean

OOM stands for Out Of Memory. When a container exceeds its memory limit, the Linux kernel's OOM killer steps in and sends the process a SIGKILL — an immediate, unblockable termination. Kubernetes reports this as OOMKilled.

The number 137 means the process was reported as terminated by SIGKILL: shells commonly encode that as 128 + 9. In a limited container OOM is a frequent cause, but a forced termination can produce the same code. Confirm Reason: OOMKilled, cgroup/node metrics, and events.

Because it's a SIGKILL, the process gets no chance to clean up or log anything. That's why OOM kills are confusing: the app logs just stop, with no error, and the only evidence is in the pod's state and events — not in the application's own output.

Requests vs Limits: The Root of It

To understand OOM kills you have to understand Kubernetes' two memory settings:

• requests — the memory amount the scheduler uses when placing the pod. It is not a reserved block of RAM or a runtime guarantee.
• limits — the cgroup memory boundary enforced for the container. Crossing it can trigger a container-level OOM kill. Linux node swap behavior is configurable on supported Kubernetes releases, so do not assume every cluster has swap disabled.

Two distinct OOM scenarios follow from this:

1. Container exceeds its own limit. The most common case: your container's limit is 512Mi, it tries to use 600Mi, and it's killed. This is a per-container limit kill.
2. Node runs out of memory. Under node memory pressure, kubelet may evict pods according to eviction signals and QoS/priority, or the kernel may invoke its system OOM killer before kubelet can react. This is distinct from a container exceeding its own cgroup limit, and the pod status and events can differ.

Requests and limits also contribute to the pod's Quality of Service class (Guaranteed, Burstable, BestEffort). A pod is Guaranteed only when every container has CPU and memory requests equal to its limits. QoS influences eviction and OOM scoring, but priority, actual usage, and whether usage exceeds requests also matter; Guaranteed is not absolute immunity.

How to Diagnose an OOM Kill

1. Confirm it is actually OOM. kubectl describe pod <name> and kubectl get pod <name> -o yaml can show Reason: OOMKilled under the previous container state. Exit 137 without that reason can also result from another forced SIGKILL. Events may separately show node-pressure eviction.
2. Distinguish limit-kill from node-pressure. If a single container is OOMKilled while the node looks healthy, it's hitting its own limit. If multiple pods are being killed or evicted at once, suspect node-level memory pressure.
3. Look at actual usage. kubectl top pod <name> (and kubectl top nodes) shows real-time memory; compare it to the configured limit. A metrics dashboard showing usage over time is even better — it reveals whether the app spikes or grows steadily.
4. Spike or leak? A steady climb toward the limit over hours or days is a memory leak. A sudden jump under load is a spike (a big request, a large batch, an unbounded cache). The fix differs.
5. Check recent changes. A new deploy, a dependency upgrade, or a traffic increase often precedes a fresh wave of OOM kills.

How to Fix It

Match the fix to what you found — don't just blindly raise the limit:

• Right-size the limit. If the app legitimately needs more memory than its limit allows, raise the limit (and usually the request with it). This is the correct fix when the limit was simply too low for the real workload.
• Fix the memory leak. If usage grows without bound, raising the limit only delays the kill. Profile the app, find what's retained (unbounded caches, accumulating buffers, leaked connections), and fix it. Monitoring memory as it trends abnormally upward catches leaks before the kill.
• Cap what the app thinks it has. Runtimes that auto-size to the node's memory rather than the container's limit will overshoot. Set heap/cache limits explicitly (e.g. JVM -XX:MaxRAMPercentage, Node --max-old-space-size) to fit inside the container limit.
• Set requests sensibly. Avoid heavy node oversubscription by setting realistic requests, so the scheduler doesn't pack more onto a node than it can hold.
• Bound spiky workloads. Page or stream large operations instead of loading everything into memory; cap batch sizes and cache growth.

A practical rule: raise the limit to fix an undersized container; fix the code to stop a leak. Confusing the two is how teams end up with pods that need 8Gi to do a 512Mi job.

Kubernetes' resource management documentation describes scheduler requests, cgroup-enforced limits, and current swap behavior.

How Webalert Helps

OOM kills are an internal cluster event — but their impact is users hitting errors and timeouts when a pod dies mid-request. That's what outside-in monitoring catches:

• Uptime and response-time checks that reveal when OOM-driven restarts cause real user-facing failures — the degradation the cluster's "pod recovered" status hides.
• External confirmation that endpoints respond correctly after you've resized limits or fixed a leak.
• Multi-region monitoring to tell a genuine service outage from a single pod cycling.
• Sustained-failure alerting so repeated OOM restarts that add up to real downtime page you, without noise from every brief blip.

Kubernetes restarts the OOMKilled pod automatically; Webalert tells you whether that automatic recovery was fast enough that users never noticed — or not.

Summary

OOMKilled with exit code 137 means the kernel sent SIGKILL to a container that exceeded its memory — either its own limit or the node's available memory under pressure. Because it's a hard kill, the app logs nothing; the evidence lives in kubectl describe pod (Reason: OOMKilled, exit 137) and in usage metrics.

Diagnose by confirming it's truly OOM, separating per-container limit kills from node memory pressure, and checking whether memory spikes or leaks. Then fix the right thing: raise an undersized limit, fix a genuine leak, cap the runtime's memory to the container limit, and set realistic requests to avoid oversubscription. Watch memory trends so you catch leaks before the kill — and use outside-in monitoring to confirm users stayed unaffected through the restart.

See when memory kills actually hurt your users

Start monitoring with Webalert ->

See features and pricing. No credit card required.