Context & Why Resource Isolation Matters
Picture this scenario: an unoptimized Celery worker or image processing batch job suddenly leaks 6 GB of RAM on an 8 GB server. Without enforced resource boundaries, the Linux Out-Of-Memory (OOM) killer intervenes abruptly. Instead of isolating the offending worker, it might terminate PostgreSQL or sshd to free memory immediately. This is the classic noisy-neighbor problem.
Control Groups (cgroups) solve this by metering, throttling, and isolating hardware resources. While Cgroups v1 split controllers across fragmented directory trees, Cgroups v2 introduces a single unified hierarchy. Every process belongs to exactly one cgroup path. This structure provides predictable resource allocation, cleaner delegation, and accurate accounting for both anonymous memory and filesystem page cache.
You do not need to write low-level scripts against /sys/fs/cgroup/ directly. Modern Linux distributions delegate this to systemd through units called Slices. Slices organize services into logical groups, letting you apply CPU weights, memory ceilings, and disk I/O limits across entire workload tiers with a few declarative directives.
Installation & Enabling Cgroups v2
Modern distributions—including Ubuntu 22.04+, Debian 11+, Fedora 31+, and RHEL 9+—enable Cgroups v2 by default. Start by verifying what your server currently runs.
Step 1: Check Current Cgroup Version
Inspect the cgroup filesystem type with stat:
stat -fc %T /sys/fs/cgroup/If the output returns cgroup2fs, your kernel is already running unified Cgroups v2. If it returns tmpfs, the system is on legacy Cgroups v1.
Step 2: Enable Unified Hierarchy (If on Cgroups v1)
To enable the unified hierarchy on older distributions, pass the corresponding kernel parameter via GRUB:
sudo nano /etc/default/grubAppend systemd.unified_cgroup_hierarchy=1 to the GRUB_CMDLINE_LINUX string:
GRUB_CMDLINE_LINUX="systemd.unified_cgroup_hierarchy=1"Rebuild your GRUB configuration and reboot the machine:
# On Ubuntu/Debian
sudo update-grub
sudo reboot
# On RHEL/AlmaLinux/Rocky
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
sudo rebootStep 3: Verify Available Subsystem Controllers
Inspect the active controllers available in your root cgroup:
cat /sys/fs/cgroup/cgroup.controllersConfirm that cpu, memory, io, and pids appear in the output.
Configuration: Systemd Slices and Resource Constraints
Systemd automatically partitions workloads into three default root slices:
system.slice: Background daemons and system services.user.slice: Interactive login sessions and per-user processes.machine.slice: Virtual machines and container workloads.
Creating custom slices allows you to sandbox background jobs, analytics engines, or microservices without starving essential system services.
Step 1: Define a Custom Slice Unit
Create a dedicated slice for backend workers at /etc/systemd/system/workload-workers.slice:
sudo nano /etc/systemd/system/workload-workers.sliceAdd the following configuration to enforce CPU quotas, memory limits, and disk write limits:
[Unit]
Description=Resource Limited Slice for Worker Services
Before=slices.target
[Slice]
# --- CPU Allocation ---
# Cap the entire slice at 200% CPU (the equivalent of 2 full cores)
CPUQuota=200%
# CPUWeight (1-10000) sets relative priority during CPU contention
CPUWeight=500
# --- Memory Constraints ---
# MemoryHigh throttles processes and aggressively reclaims page cache
MemoryHigh=2G
# MemoryMax is the hard limit: triggers slice-confined OOM if exceeded
MemoryMax=2.5G
# Cap swap memory usage to prevent disk thrashing
MemorySwapMax=512M
# --- I/O Throttling ---
# Limit maximum write and read throughput on the primary disk
IOWriteBandwidthMax=/dev/sda 25M
IOReadBandwidthMax=/dev/sda 50M
# --- Process Limiting ---
TasksMax=1024Step 2: Assign Services to the Custom Slice
Instead of editing unit files directly in vendor directories, assign services using systemd drop-in overrides. For example, to attach queue-worker.service to your new slice:
sudo systemctl edit queue-worker.serviceAdd the slice assignment inside the override file:
[Service]
Slice=workload-workers.sliceSave and exit the editor. Reload the systemd manager configuration and restart the worker:
sudo systemctl daemon-reload
sudo systemctl restart queue-worker.serviceStep 3: Direct In-Service Resource Directives (Alternative)
If you only need to throttle a single standalone service rather than an entire fleet, define the limits directly in that unit’s [Service] section:
[Service]
Type=simple
ExecStart=/usr/bin/python3 /opt/app/worker.py
Restart=on-failure
# Service-level CPU and Memory limits
CPUQuota=80%
MemoryHigh=800M
MemoryMax=1G
OOMPolicy=stopVerification & Real-Time Monitoring
Once limits are in place, test how the system behaves under heavy load.
1. Inspect Active Slice Membership
Verify that your service is running in the correct slice with systemctl status:
systemctl status queue-worker.serviceCheck the CGroup: line in the output. It should point directly to /workload-workers.slice/queue-worker.service.
2. Monitor Live Resource Usage with systemd-cgtop
Systemd includes systemd-cgtop, a top-like monitor organized by cgroup hierarchy:
systemd-cgtop -mUse interactive shortcuts to sort running groups:
- Press c to sort by CPU usage.
- Press m to sort by memory consumption.
- Press i to sort by disk I/O operations.
3. Check Applied Kernel Limits Directly
Query the cgroup virtual filesystem to inspect raw values applied by the kernel:
# View current memory usage in bytes
cat /sys/fs/cgroup/workload-workers.slice/memory.current
# View configured hard memory limit (in bytes or 'max')
cat /sys/fs/cgroup/workload-workers.slice/memory.max
# View CPU bandwidth quotas (quota and period in microseconds)
cat /sys/fs/cgroup/workload-workers.slice/cpu.max4. Stress Testing Resource Boundaries
Simulate memory pressure inside the slice with stress-ng to verify that enforcement works as expected:
systemd-run --slice=workload-workers.slice --unit=stress-test \
--wait stress-ng --vm 1 --vm-bytes 3G --timeout 20sInspect the kernel logs using journalctl -u stress-test.service or dmesg -T. You will see the kernel throttle memory allocation at 2 GB (MemoryHigh) and trigger an isolated OOM kill at 2.5 GB (MemoryMax)—leaving database, web, and SSH services completely unharmed.
