Exit code 137 linux Sep 18, 2017 · In Linux, there are a number of exit codes with Special Meanings, of note here is the 128+n section, which are the Kill levels for a process. One of the most common Kubernetes errors is exit code 137. Below is my docker-compose version: May 4, 2024 · Exit Code 137: This code signifies that a process was forcibly killed with a SIGKILL signal due to memory constraints. Jan 18, 2018 · Updated December 1, 2023, First Published January 18, 2018 by Pete Freitag When attempting to fire up a bunch of docker containers using docker-compose on a Mac, one of the containers was randomly exiting with the exit code 137. Since SIGKILL is signal 9, the resulting exit code is 128 + 9 = 137. However, like any complex system, Kubernetes can sometimes produce errors. Jan 14, 2020 · The number 137 is a sum of two numbers: 128+x, # where x is the signal number sent to the process that caused it to terminate. Cluster information: Kubernetes version: 1. This event is usually an indication that a container in a pod has exceeded its memory limit and the system cannot allocate additional memory Jan 1, 2025 · 3. This code is an integer that indicates the status of the container’s termination. Essential for diagnosing script or process outcomes effectively. Jul 9, 2023 · Unraveling the mystery of exit codes in Linux. When your container consumes more memory than is available, the Linux kernel might forcibly kill the container to free up resources. travis-ci is Oct 24, 2023 · To understand this code, let’s break it down: Exit code 137 results from the combination of two factors: 128, which represents an exit command with an invalid code. 11 4. OOM (Out of Memory Manager) is a tool on Linux systems that keeps track of memory usage by process. When a container uses more memory than allowed, Kubernetes uses the Linux Out-of-Memory (OOM) Killer to stop the process. In the example, x equals 9, which is the number of the SIGKILL signal, meaning the process was killed forcibly. This article will discuss what exit code 137 means, what causes it, and Jul 8, 2024 · The “OOMKilled” error is a serious issue marked by the exit code 137. Discover everything you need to know about exit code 137 on Kubernetes, what causes it, and best practices to prevent it. Mar 7, 2022 · I am running a CICD pipeline that runs a test script inside a docker container with docker exec and its failing with a return code of 137 every 3rd or 4th time. This Perl script exits with a warning and exit code 137. In Unix/Linux systems, when a process exits due to a signal, its exit code equals 128 plus the signal number. Try this as an experiment: 1) If you turn off the OOM killer completely. Kubernetes Exit Code 137: What It Means and How to Fix It Kubernetes is a powerful container orchestration system that can be used to deploy, manage, and scale containerized applications. The process gets terminated to prevent memory usage ballooning indefinitely, which could cause your host system to become unstable. in this case, 137 = 128 + 9, so this process was killed with the highest level. A code of 0 signifies success, while non-zero codes represent errors or specific conditions. If you were to issue the “kubectl describe pod “pod_name”” command, you’d get more information about the issue - The response will be “Exit Code: 137. Windows operating systems use different exit codes and signaling mechanisms. How do I fix exit code 137? The OOMKilled status of a pod will imply Out Of Memory as per the acronym at the start of the status. The calculation is based on the formula 128 + 9, where 128 indicates a process killed by a signal and 9 is the signal number for SIGKILL. Learn how to resolve the error. Jun 30, 2021 · [ rc=137 (** EXPECTED 0 **) ] #/vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv #| FAIL: exit code is 137; expected 0 #\^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ # [teardown] $ podman pod rm --all --force $ podman rm --all --force 52cff7db8ddd73a7fef9dc200d21a6e853f40b2e08c68273f6cc12b767b88562 Apr 4, 2025 · Understanding OOMKilled in Kubernetes: Beyond Exit Code 137 Exit code 137 shows up frequently in Kubernetes logs. env file and run docker-compose up -d or increase the memory on Linux. The message would look something like this: container_name exited with code 137 My first reaction was to try to quickly run docker exec -it container-id /bin/bash and Nov 21, 2024 · Linux exit status codes are numeric values returned by commands or scripts to indicate the outcome of their execution. Memory is to containers what water is to people: a vital resource. Unless this I'm trying to build 0ad on the travis-ci service and I'm randomly experiencing failures of apt-get install (or apt-get dist-upgrade) with returncode 137 without any further information. Low memory limits – Pod spec sets memory cap too low. Sep 23, 2021 · OOMKilled (exit code 137) occur when K8s pods are killed because they use more memory than their limits. The kubelet killed it to protect the node. This is part of an extensive series of guides about Observability. The exit codes that Docker containers report can also provide some diagnostic information (for example, exit code 137 means that the container received a SIGKILL signal). OutOfMemoryException" 100% of the time. Jul 8, 2023 · Referring to system logs or command-specific logs can provide additional information for better differentiation. But after sometime, mongodb is throwing exited 137 co… What does a non-zero exit code mean? When a container exits, it returns an exit code. The Kubernetes engine will allocate a certain amount of memory and CPU to every container. I could not Jan 17, 2024 · 2 Exit code 137 means a container or pod is trying to use more memory than it's allowed Decrease the MEM_LIMIT=1073741824 in . Knowing how to work with and display exit codes on the Linux command line or in scripts can help make errors more obvious. Thus you get an exit of 137. Sep 2, 2025 · Find out what Kubernetes exit code 137 means, what causes it, how to identify it, and the best ways to stop forced container terminations. Assuming this is not a live production box you are stressing ;-) You will should see the "System. Here is the code that's running: doc Is there a way I can do what stated in the title from the terminal commands, or will I have to look into the codes? Oct 12, 2022 · Learn about Linux exit codes, their meanings, and how to retrieve them. In practice, it almost always means: The pod consumed more memory than allowed. May 26, 2015 · Is there a way to eliminate a warning (exit code 137) in perl? I am running a Perl script on linux within another shell script. Apr 3, 2024 · Facing Docker Container Exit Code 137 errors? Learn exactly why your containers run out of memory and follow these steps to prevent unexpected shutdowns. List Docker containers in Amazon ECS. Nov 29, 2023 · We'll discuss what exit code 137 means for Kubernetes, how it reflects the operating system's intervention due to memory concerns, and how to mitigate memory issues. What does it mean and how do you fix it? 137 = killed by SIGKILL Exit code 137 is Linux-specific and means that your process was killed by a signal, namely SIGKILL. X Kubernetes OOMKilled Error: How to Fix and Tips for Preventing It What Is Kubernetes OOMKilled (Exit Code 137)? The Kubernetes OOMKilled (Exit Code 137) is a signal sent by the Linux Kernel to terminate a process due to an Out Of Memory (OOM) condition. Learn what are the exit codes and why and how they are used. Jul 10, 2025 · Exit Code 137 - Out of memory This is a Linux error code which you may see on CircleCI when your executor runs out of memory. This typically happens when the system or a user forcibly kills a process, often due to resource constraints like running out of Dec 12, 2019 · docker-compose up --build --exit-code-from combined Unfortunately, I consistently receive an exit code of 137 even when the tests in my combined container run successfully and I exit that container with an exit code of 0 (more details on how that happens are specified below). 137 & 127 = 9, so the mono process was sent a SIGKILL signal (kill -9) and your Stress test is not happy. Out-of-Memory (OOM) Killer: One of the most common causes is the Linux OOM killer. 14 Cloud being used: AWS EKS Installation method: EKS Host OS: Amazon Linux Node: c5. Docker Container Exit Code 137 means a container was terminated due to Out of Memory (OOM) issues or received a kill signal. Aug 16, 2023 · I have a docker container that is exiting with code 137 for mongodb connection. This mysterious signal needs a deeper look at the mechanisms of memory management in containerized environments. What are the common causes of a container Dec 8, 2023 · The exit code 137 is a result of 128 + 9, where 9 corresponds to the SIGKILL signal, indicating that the process was forcefully killed. Docker Jan 14, 2020 · Many jobs are throwing up exit code 137 errors and I found that it means that the container is being terminated abruptly. **Q: Is exit code 137 specific to Linux?** A: Exit code 137 and related signals are primarily used in Unix-like operating systems, including Linux. # What is Exit Code 137? Exit code 137 typically indicates that a process inside a container was killed by the system due to an out-of-memory (OOM) condition. Jul 14, 2022 · Usually this happens on a CI machine (Jenkins, TeamCity, GitLab). Common Causes of Exit 137 Code Several factors can lead to Exit 137 Code: 1. Jul 19, 2024 · This comprehensive guide will delve into the intricacies of error code 137, its common scenarios, and strategies to resolve it. The most common non-zero exit code is 137, which indicates that the container was killed by a signal. For more information, see Exit Status in the Docker documentation. The code shows a termination from a SIGKILL signal (128+9=137), which the Linux kernel’s Out-Of-Memory (OOM) Killer usually triggers. How or why the job may have been signaled, or exited with a certain exit code, can be application and/or system specific. Feb 18, 2023 · Exit code 137 means a container or pod is trying to use more memory than it’s allowed. Excessive memory usage can occur due to natural growth in your application’s use, or as the result of a memory leak in your Exit codes less than 128 relate to application exit values, while exit codes greater than 128 relate to system signal exit values (LSF adds 128 to system values). 2. Issue Pod failed due to container process exited with code 137 Environment Red Hat OpenShift Container Platform (RHOCP) 3. Use bhist to see the exit code for your job. 4xlarge After digging in, I found the below logs: OOM Killed stands for “out of memory killed,” and it is represented by Exit Code 137, which indicates that the Linux kernel has terminated a container due to exceeding its allocated memory limit. Exit Code 137 in Kubernetes = Linux process killed with SIGKILL (137). What if Exit Code 137 happens, but OOMKilled is false? In most cases, Exit Code 137 is caused by an out-of-memory (OOM) event where the Linux kernel forcibly kills the process. The Container Lifecycle To better understand the causes of container failure, let’s discuss the lifecycle of a container first. Exit code 137 indicates that a process was forcibly terminated using signal 9 (SIGKILL). What happens when Kubernetes Apr 15, 2025 · Let's explore what Linux exit codes are, the list of common exit codes, and how you can use them to troubleshoot your Linux system. So when I first start the server, the connection is successful and everything is fine. Nov 2, 2021 · Below we’ll explain how to troubleshoot failed containers on a self-managed host and in Kubernetes, and provide more details on all of the exit codes listed above. Mar 26, 2024 · In this blog post, we'll dive into the details of exit code 137 terminations and explore their underlying causes. Taking Docker as an example Jan 25, 2025 · SIGKILL (Exit Code 137) — When a process is killed by sending a kill signal (n=9), we get an exit code of 137. Why does Exit Code 137 Occur? The primary reason for an exit code 137 is that your pod is exceeding the memory limit set in its configuration. SIGTERM (Exit Code 143) — SIGTERM stands for Signal to Terminate and has a signal value of 15 (n=15). Here is the similar discussion. Jan 19, 2022 · OOMKilled, commonly known as Exit Code 137, is a type of error that originated in Linux. In short: Your pod ran out of memory (OOMKilled). A non-zero exit code means that the container did not terminate successfully. gp7o 92s co oov6hy eqrdzj wh tmsywg plqksi xpz bw2x