Loading...

Linux System Services: systemctl, journalctl, and systemd Unit Files

Back to blog
LinuxSystemDServicesDevOps

Linux System Services: systemctl, journalctl, and systemd Unit Files

Master systemd service management including systemctl commands, systemd unit files, journalctl logging, and creating custom services for DevOps.

8 min read

Linux System Services: systemctl, journalctl, and systemd Unit Files

systemd is the modern Linux system and service manager used by most Linux distributions. Understanding systemd is essential for managing services, automating tasks, and troubleshooting system issues.

systemctl - Manage Services

Service Control

# Start service
sudo systemctl start nginx
 
# Stop service
sudo systemctl stop nginx
 
# Restart service
sudo systemctl restart nginx
 
# Reload configuration (without stopping)
sudo systemctl reload nginx
 
# Check service status
sudo systemctl status nginx
# ● nginx.service - A high performance web server and a reverse proxy server
#    Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
#    Active: active (running) since Wed 2026-01-23 10:00:00 UTC; 5h ago
 
# Restart only if running
sudo systemctl try-restart nginx
 
# Reload or restart
sudo systemctl reload-or-restart nginx
 
# Enable service (start on boot)
sudo systemctl enable nginx
 
# Disable service (don't start on boot)
sudo systemctl disable nginx
 
# Check if enabled
sudo systemctl is-enabled nginx
# enabled
 
# Check if running
sudo systemctl is-active nginx
# active
 
# Mask service (prevent from running)
sudo systemctl mask nginx
 
# Unmask service
sudo systemctl unmask nginx

List Services

# List all services
systemctl list-units --type=service
 
# List active services
systemctl list-units --type=service --state=active
 
# List enabled services
systemctl list-unit-files --type=service
 
# List running services
systemctl list-units --type=service --state=running
 
# List failed services
systemctl list-units --type=service --state=failed
 
# List service with details
systemctl status service_name
 
# Search for service
systemctl | grep nginx
 
# Show service dependencies
systemctl list-dependencies nginx

Service Information

# Show service file location
systemctl cat nginx
# Shows /lib/systemd/system/nginx.service content
 
# Show service environment
systemctl show nginx -p Environment
 
# Show service properties
systemctl show nginx
# Type=notify
# Restart=no
# ExecStart=/usr/sbin/nginx -g 'daemon off;'
 
# Check service enable status
systemctl is-enabled apache2
 
# Verify service file syntax
systemctl verify nginx
 
# Edit service file (safe)
sudo systemctl edit nginx
# Creates override in /etc/systemd/system/nginx.service.d/
 
# Edit original service file
sudo nano /lib/systemd/system/nginx.service

systemd Unit Files

Understanding Unit Files

# Service unit file structure
cat /lib/systemd/system/nginx.service
 
# [Unit]
# Description=A high performance web server
# After=network.target
#
# [Service]
# Type=notify
# ExecStart=/usr/sbin/nginx -g 'daemon off;'
# ExecReload=/bin/kill -s HUP $MAINPID
# ExecStop=/bin/kill -s QUIT $MAINPID
# Restart=on-failure
#
# [Install]
# WantedBy=multi-user.target

Common Unit File Sections

# [Unit] - Service metadata
Description=              # Human-readable description
After=                   # Start after these units
Before=                  # Start before these units
Requires=                # Hard dependency
Wants=                   # Soft dependency
PartOf=                  # Part of another unit
Conflicts=               # Cannot run with other units
 
# [Service] - Service behavior
Type=simple              # Simple service (default)
Type=forking             # Fork to background
Type=notify              # Notify when ready
Type=dbus                # D-Bus activation
ExecStart=               # Command to start service
ExecStop=                # Command to stop service
ExecReload=              # Command to reload
Restart=no               # Restart policy
RestartSec=5             # Restart delay
Environment=             # Environment variables
EnvironmentFile=         # Load from file
User=                    # Run as user
Group=                   # Run as group
StandardOutput=journal   # Log to journal
StandardError=journal    # Error logging
 
# [Install] - Installation behavior
WantedBy=                # Units that want this
RequiredBy=              # Hard requirement
Also=                    # Install related units
Alias=                   # Alternative name
DefaultInstance=         # Default for templated units

Creating Custom Services

Simple Service File

# Create custom service
sudo nano /etc/systemd/system/myapp.service
 
# [Unit]
# Description=My Custom Application
# After=network.target
#
# [Service]
# Type=simple
# User=myuser
# WorkingDirectory=/opt/myapp
# ExecStart=/usr/bin/python3 /opt/myapp/app.py
# Restart=on-failure
# RestartSec=10
#
# [Install]
# WantedBy=multi-user.target
 
# Reload systemd
sudo systemctl daemon-reload
 
# Test service
sudo systemctl start myapp
 
# Check status
sudo systemctl status myapp
 
# Enable on boot
sudo systemctl enable myapp

Service with Environment Variables

# Create environment file
sudo nano /etc/myapp/myapp.conf
# DATABASE_URL=postgresql://user:pass@localhost/db
# API_KEY=secret123
# DEBUG=false
 
# Service file using environment file
sudo nano /etc/systemd/system/myapp.service
# [Unit]
# Description=My Application
# After=network.target postgresql.service
#
# [Service]
# Type=simple
# User=myuser
# EnvironmentFile=/etc/myapp/myapp.conf
# ExecStart=/usr/bin/python3 /opt/myapp/app.py
# Restart=on-failure
#
# [Install]
# WantedBy=multi-user.target
 
# Reload and test
sudo systemctl daemon-reload
sudo systemctl start myapp
sudo systemctl status myapp

Socket-Activated Service

# Create socket unit
sudo nano /etc/systemd/system/myapp.socket
# [Unit]
# Description=My App Socket
#
# [Socket]
# ListenStream=8080
# Accept=yes
#
# [Install]
# WantedBy=sockets.target
 
# Create service unit
sudo nano /etc/systemd/system/myapp@.service
# [Unit]
# Description=My App Service
#
# [Service]
# Type=simple
# ExecStart=/usr/bin/myapp
# StandardInput=socket
 
# Enable and start
sudo systemctl daemon-reload
sudo systemctl enable myapp.socket
sudo systemctl start myapp.socket
 
# Check socket
systemctl status myapp.socket

Templated Services

# Create templated service
sudo nano /etc/systemd/system/worker@.service
# [Unit]
# Description=Worker Instance %i
# After=network.target
#
# [Service]
# Type=simple
# ExecStart=/usr/bin/worker --id=%i
# Restart=on-failure
#
# [Install]
# WantedBy=multi-user.target
 
# Create instances
sudo systemctl enable worker@1.service
sudo systemctl enable worker@2.service
sudo systemctl enable worker@3.service
 
# Start instances
sudo systemctl start worker@{1,2,3}.service
 
# Check status
systemctl status worker@1.service
systemctl status worker@2.service

journalctl - Systemd Logging

View Logs

# View all logs
journalctl
 
# View recent logs
journalctl -e
# -e: jump to end of logs
 
# View recent N lines
journalctl -n 50
 
# View logs for specific service
journalctl -u nginx
journalctl -u mysql
 
# Continuous follow (like tail -f)
journalctl -f
journalctl -f -u nginx
# -f: follow mode
 
# View logs for specific unit and show full
journalctl -u nginx --no-pager
 
# View errors only
journalctl -p err
# -p: priority (emerg, alert, crit, err, warning, notice, info, debug)
 
# View last boot
journalctl -b
journalctl -b -u nginx
 
# View specific boot
journalctl --list-boots
journalctl -b 0  # Last boot
journalctl -b -1 # Previous boot
 
# View by time range
journalctl --since "2026-01-23 10:00:00" --until "2026-01-23 12:00:00"
journalctl --since today
journalctl --since "1 hour ago"

Filter Logs

# By priority
journalctl -p warning
journalctl -p 3  # Critical and higher (3=critical)
 
# By user
journalctl _UID=1000
 
# By process
journalctl _PID=1234
 
# By executable
journalctl /usr/bin/nginx
 
# Multiple filters (AND)
journalctl _UID=0 -u nginx
 
# Multiple units (OR)
journalctl -u nginx -u mysql
 
# Show only matched fields
journalctl --no-pager -o short
 
# With no wrap
journalctl --no-pager

Output Formats

# Short format
journalctl -n 5 --no-pager -o short
# Jan 23 10:30:45 server nginx[1234]: message
 
# Verbose
journalctl -n 1 -o verbose
 
# JSON
journalctl -n 1 -o json
 
# Cat format (fields only)
journalctl -n 1 -o cat
 
# Short-precise
journalctl -n 5 -o short-precise
 
# Export to file
journalctl -u nginx > nginx.log
journalctl --no-pager > full-logs.txt

Log Management

# View journal disk usage
journalctl --disk-usage
 
# Rotate logs
sudo journalctl --rotate
 
# Limit log size
sudo nano /etc/systemd/journald.conf
# SystemMaxUse=100M
# RuntimeMaxUse=50M
# MaxRetentionSec=30day
# SystemMaxFileSize=10M
 
# Apply changes
sudo systemctl restart systemd-journald
 
# Remove old logs
sudo journalctl --vacuum-time=30d
sudo journalctl --vacuum-size=100M
sudo journalctl --vacuum-files=10
 
# Make journal persistent
sudo mkdir -p /var/log/journal
sudo systemctl restart systemd-journald

Troubleshooting Services

Common Issues

# Service fails to start
sudo systemctl status myapp
# Shows error in red
 
# View detailed logs
journalctl -u myapp -n 50
 
# Check service file syntax
sudo systemd-analyze verify myapp.service
 
# Enable debug logging
sudo systemctl set-environment SYSTEMD_LOG_LEVEL=debug
sudo systemctl restart myapp
journalctl -u myapp
 
# View service dependencies
systemctl list-dependencies --before nginx
systemctl list-dependencies --after nginx
 
# Check if service is waiting for something
systemctl list-units --all --state=inactive
 
# View service startup time
systemd-analyze
systemd-analyze blame
systemd-analyze critical-chain

Reset Failed Status

# View failed units
systemctl list-units --failed
 
# Show failed service
systemctl status myapp
 
# Reset failed status
sudo systemctl reset-failed myapp
sudo systemctl reset-failed  # All units
 
# Retry failed service
sudo systemctl restart myapp

Timers - Scheduled Services

# Create timer unit
sudo nano /etc/systemd/system/backup.timer
# [Unit]
# Description=Daily Backup Timer
#
# [Timer]
# OnCalendar=daily
# OnCalendar=*-*-* 02:00:00
# Unit=backup.service
#
# [Install]
# WantedBy=timers.target
 
# Create service unit
sudo nano /etc/systemd/system/backup.service
# [Unit]
# Description=Backup Service
#
# [Service]
# Type=oneshot
# ExecStart=/usr/local/bin/backup.sh
 
# Enable and test
sudo systemctl daemon-reload
sudo systemctl enable backup.timer
sudo systemctl start backup.timer
 
# View timers
systemctl list-timers
 
# View timer logs
journalctl -u backup.timer

Best Practices

  1. Understand Unit Files - Know what each section does
  2. Set Proper Dependencies - Use After/Wants correctly
  3. Error Handling - Set Restart policy appropriately
  4. Resource Limits - Prevent resource exhaustion
  5. Logging - Direct output to journal
  6. Security - Run as non-root when possible
  7. Documentation - Comment your service files
  8. Testing - Test services before enabling
  9. Monitoring - Set up alerts for failed services
  10. Version Control - Track service file changes

Summary

systemd is essential for modern Linux administration:

  • systemctl controls and manages services
  • Unit files define service behavior
  • journalctl accesses system logs
  • Custom services automate application management
  • Timers schedule recurring tasks
  • Mastering systemd enables effective DevOps practices

Develop proficiency with systemd for professional Linux administration.

Continue Reading

More articles you might enjoy

View all
01

Linux Disk Management: df, du, fdisk, parted, mount, and LVM Basics

Master Linux disk management including disk usage with df and du, partitioning with fdisk and parted, mounting filesystems, and LVM for flexible storage.

9 min readJan 23
02

Linux Environment Variables: PATH, HOME, USER, and Shell Configuration

Master Linux environment variables including PATH, HOME, USER, SHELL, setting variables, .bashrc, .profile, and environment variable best practices.

8 min readJan 23
03

Linux File Permissions: chmod, chown, umask, SUID, and ACLs Explained

Master Linux file permissions including chmod, chown, umask, special permissions (SUID, SGID, sticky bit), and access control lists (ACLs) with practical examples.

11 min readJan 23
Back to all posts