Stop AI Crashes: The Linux OOM-Killer Shield

The Silent Assassin: Linux OOM-Killer

You deployed a massive AI model. It runs perfectly. But suddenly, your API stops responding. You check the container logs, and there are no Python tracebacks, no errors—the process simply vanished. Welcome to the Linux OOM-Killer (Out-Of-Memory Killer).

When your server's system RAM gets dangerously low, the Linux kernel panics. To prevent a total system freeze, it acts as a sniper, targeting and terminating the process consuming the most memory. Since AI containers (like vLLM, ComfyUI, or NIM) are notoriously RAM-heavy during model loading, they are always the first target. Let's fix this permanently on your iRexta Bare Metal Server.

Step 1: Diagnosing the Ghost Crash

Before applying the shield, you need to prove the OOM-Killer is the culprit. Because it kills processes at the kernel level, the evidence is hidden in the system's kernel ring buffer, not your Docker logs.

# Search the kernel logs for recent OOM activity
dmesg -T | grep -i 'killed process'
# Example Output:
# [Tue Mar 19 12:45:00 2026] Out of memory: Killed process 10245 (python) total-vm:45000000kB...

If you see output like the one above, your AI was assassinated by the kernel.

Step 2: Deploying the Docker Shield (Compose)

Every process in Linux has an oom_score. The higher the score, the more likely it is to be killed. We can use the oom_score_adj parameter to tell the kernel: "Do not kill this AI container, kill a background logging service instead."

The value ranges from -1000 (complete immunity) to 1000. We recommend -500 for critical AI workloads in production.

version: '3.8'
services: ai-production-api: image: my-heavy-llm-container:latest container_name: bulletproof_ai # THE SHIELD: Lowers the OOM score to protect this container oom_score_adj: -500 deploy: resources: reservations: devices: - driver: nvidia count: all capabilities: [gpu] restart: always

Step 3: Deploying via Docker CLI

If you prefer running standalone Docker containers instead of Compose, you can pass the flag directly in your run command:

# Run container with OOM protection
docker run -d --gpus all \ --name bulletproof_ai \ --oom-score-adj -500 \ -p 8000:8000 \ my-heavy-llm-container:latest

Conclusion

By tweaking just one line of code, your critical AI APIs are now shielded from silent kernel terminations. However, software shields are just bandages. The ultimate fix for Out-Of-Memory errors is hardware abundance.