The 2:14 AM Post-Mortem
My pager went off at exactly 2:14 AM. After blocking over 1,200 SSH brute-force attempts earlier that night, I thought the worst was over. I was wrong. A Python web scraper, a minor tool I’d overlooked, had been exploited via a remote code execution (RCE) bug. Even though it ran as a non-root user, the attacker successfully dropped a 1.2MB malicious binary into /tmp and began scanning /etc for database credentials.
That incident proved that standard user permissions aren’t enough. If a service doesn’t need to touch the rest of your system, it shouldn’t even see it. Systemd sandboxing solves this by using Linux namespaces to wrap applications in a restricted environment. It turns a potential system-wide breach into a minor, contained incident.
Quick Start: Hardening a Service in 5 Minutes
You can significantly reduce your attack surface by adding just six lines to your [Service] block. Let’s look at a typical transition for a service file located at /etc/systemd/system/my-app.service.
The Vulnerable Setup
[Unit]
Description=My Vulnerable App
[Service]
ExecStart=/usr/bin/python3 /opt/my-app/app.py
User=myappuser
Restart=always
[Install]
WantedBy=multi-user.targetThe Hardened Setup
[Unit]
Description=My Secure App
[Service]
ExecStart=/usr/bin/python3 /opt/my-app/app.py
User=myappuser
Restart=always
# Security hardening
PrivateTmp=true
ProtectSystem=full
ProtectHome=true
NoNewPrivileges=true
PrivateDevices=true
DevicePolicy=closed
[Install]
WantedBy=multi-user.targetApply these changes by running systemctl daemon-reload followed by systemctl restart my-app. You’ve just locked the doors.
How the Sandbox Works
Systemd uses kernel-level namespaces—specifically mount, UTS, IPC, and PID—to enforce these restrictions. Here is what those specific directives actually do under the hood.
1. PrivateTmp=true
The /tmp and /var/tmp directories are notorious playgrounds for attackers because they are world-writable. By setting PrivateTmp=true, the service gets its own file system namespace. It sees a private /tmp that is totally isolated from the rest of the host. When the service stops, Systemd wipes the private directory, instantly deleting any malicious payloads left behind.
2. ProtectSystem=full
This directive controls what parts of the OS the service can modify. It has three main levels:
- true: Mounts
/usr,/boot, and/efias read-only. - full: Includes the above and also makes
/etcread-only. - strict: Makes the entire file system read-only. You must then manually grant write access to specific folders using
ReadWritePaths=.
For most APIs, full is the best balance. It prevents an attacker from swapping out your configuration files or binaries.
3. NoNewPrivileges=true
This prevents the service and any of its child processes from gaining new privileges. It sets the PR_SET_NO_NEW_PRIVS flag, which disables SUID/SGID bits. Even if an attacker finds a bug in a SUID binary like sudo or pkexec while inside your app, they cannot use it to escalate to root.
4. ProtectHome=true
Does your app really need to see /home/admin? Probably not. Setting this to true makes /home, /root, and /run/user appear completely empty. This protects your SSH keys and personal data from being exfiltrated if the application is compromised.
Advanced Hardening
If your service is public-facing, you should apply even stricter constraints to limit the kernel’s exposure.
Filtering System Calls
A standard web app has no business calling reboot() or kexec_load(). You can restrict which syscalls are allowed using a whitelist. Systemd provides a convenient @system-service group that covers most common needs:
# Block dangerous or unnecessary kernel calls
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERMStripping Kernel Capabilities
In Linux, root privileges are broken down into ‘capabilities.’ For instance, CAP_NET_BIND_SERVICE lets a process bind to port 443. If your app runs on port 8080, it needs zero capabilities. You can drop them all with one line:
CapabilityBoundingSet=Network Isolation
If you are running a local worker that only processes files, disable the network entirely. PrivateNetwork=true disconnects the service from everything except the local loopback interface (lo).
Production Implementation Tips
Hardening can break things if you aren’t careful. Here is how to implement these changes safely.
Audit with systemd-analyze
Don’t guess—check your work. Systemd includes a built-in security auditor. Run this command to see your current exposure:
systemd-analyze security my-app.serviceMost default services score a dangerous 9.6/10. Your goal for a production-grade service should be a score below 2.5.
Handling Filesystem Access
If you use ProtectSystem=strict, your app will fail when it tries to write logs. Use these modern directives to manage paths automatically:
RuntimeDirectory=my-app # Creates /run/my-app
StateDirectory=my-app # Creates /var/lib/my-app
LogsDirectory=my-app # Creates /var/log/my-appSystemd handles the directory creation and sets the owner to the User= defined in your service file. It’s cleaner and more secure than manual chown commands.
Streamline Logging
Avoid writing to flat files inside the sandbox. Configure your application to log to stdout or stderr. Systemd’s journald will automatically pick up these streams, removing the need for the service to have write access to /var/log.
The Bottom Line
Security is about layers. Systemd sandboxing ensures that a single vulnerability doesn’t lead to a total system takeover. By trapping attackers in a read-only box with no network and no way to escalate privileges, you buy yourself time to react. Start with PrivateTmp and ProtectSystem, then use systemd-analyze to bridge the remaining gaps.
