- systemd services log to the journal, which you read with journalctl. Web servers and databases also write their own files in /var/log.
journalctl -u nginx -p err --since '1 hour ago'is the shape of nearly every useful log query: service, severity, time.journalctl -bshows the current boot and-b -1the previous one, which is how you read a crash.- Nginx and Apache write two logs: access for every request, error for what went wrong. You usually want the error log.
- If the journal is not saved to disk, it is lost on reboot. Create /var/log/journal to keep it.
Where are Linux logs stored?
In two places. Services managed by systemd (the program that starts and runs services on most Linux systems) log to the journal, which you read with journalctl. Nginx, Apache and MySQL also write plain text files under /var/log. Knowing which service writes where saves most of the searching.
| What | Where | Read with |
|---|---|---|
| systemd services and the kernel | The journal (binary) | journalctl |
| General system messages | /var/log/syslog or /var/log/messages | less, tail, grep |
| Logins and sudo | /var/log/auth.log or /var/log/secure | less, grep |
| Nginx requests | /var/log/nginx/access.log | awk, grep |
| Nginx errors | /var/log/nginx/error.log | tail -f |
| Apache | /var/log/apache2/ or /var/log/httpd/ | tail -f |
| MySQL and MariaDB | /var/log/mysql/ or /var/log/mariadb/ | less, mysqldumpslow |
| PHP-FPM | /var/log/php8.3-fpm.log | tail -f |
| WordPress debug | wp-content/debug.log | tail -f |
Debian and Ubuntu use syslog, auth.log and apache2. The RHEL family uses messages, secure and httpd.
# Largest log files, which is where the interesting things usually are
sudo du -sh /var/log/* | sort -rh | head -20
# Which logs were written to most recently?
sudo ls -lt /var/log/ | head -20
# Which log file does a running process have open?
sudo lsof -p $(pgrep -f 'nginx: master') | grep -i logHow do I use journalctl?
journalctl is the most useful troubleshooting command on a modern Linux server. Nearly every query combines three filters: which service, how severe, and what time range.
# One service, follow live
sudo journalctl -u nginx -f
# Errors only, last hour
sudo journalctl -u nginx -p err --since '1 hour ago'
# Everything from this boot
sudo journalctl -b
# The PREVIOUS boot: how you read a crash after a reboot
sudo journalctl -b -1 -p err
# Kernel messages only
sudo journalctl -k
# A time window
sudo journalctl --since '2026-07-29 09:00' --until '2026-07-29 10:00'
# Full detail on one entry, including the fields not normally shown
sudo journalctl -u nginx -o verbose -n 5
# Everything a single process wrote
sudo journalctl _PID=1234-b -1. After an unexplained reboot, it is the only way to see what happened just before.| Priority | -p value | Means |
|---|---|---|
| emerg | 0 | System is unusable |
| alert | 1 | Action required immediately |
| crit | 2 | Critical condition |
| err | 3 | Error: the usual starting point |
| warning | 4 | Warning |
| notice | 5 | Normal but notable |
| info | 6 | Informational |
| debug | 7 | Debug detail |
-p err also includes everything more severe: crit, alert and emerg.
Your journal may not survive a reboot
On some distributions the journal is kept in /run by default, which is memory. Restart the server and every entry is gone, including whatever explained the crash. Check with journalctl --disk-usage: if it reports a few megabytes in /run, the journal is not being saved.
# Create the directory and restart
sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal
sudo systemctl restart systemd-journald
# Confirm
journalctl --disk-usage
# Cap the size in /etc/systemd/journald.conf
# SystemMaxUse=1G
# MaxRetentionSec=1monthHow do I read web server logs?
Nginx and Apache write two separate logs. The access log records every request; use it to study traffic. The error log records what went wrong; read it when something is broken. People often search the wrong one.
# Follow it while you reproduce the problem
sudo tail -f /var/log/nginx/error.log
# Recent errors only
sudo grep -i 'error' /var/log/nginx/error.log | tail -50
# PHP errors surfacing through the proxy
sudo grep -iE 'PHP (Fatal|Parse|Warning)' /var/log/nginx/error.log | tail -30
# Upstream problems: 502 and 504 originate here
sudo grep -iE 'upstream|connect\(\) failed|timed out' /var/log/nginx/error.log | tail -30LOG=/var/log/nginx/access.log
# Status code distribution
awk '{print $9}' $LOG | sort | uniq -c | sort -rn
# Every 5xx, most recent last
awk '$9 ~ /^5/' $LOG | tail -30
# Top client IPs
awk '{print $1}' $LOG | sort | uniq -c | sort -rn | head -20
# Top user agents: this is where bots become visible
awk -F'"' '{print $6}' $LOG | sort | uniq -c | sort -rn | head -20
# Requests per hour
awk '{print substr($4,2,14)}' $LOG | sort | uniq -c | tail -24
# URLs consuming the most bandwidth, not just the most hits
awk '{b[$7]+=$10} END {for (u in b) print b[u], u}' $LOG | sort -rn | head -20Add $request_time to your Nginx log format and you can find slow requests straight from the access log. It is one of the most useful one-line changes you can make to a server.
How do I find the cause of a specific problem?
Start from the symptom and go to the log that owns it. This table covers most of what goes wrong on a web server.
| Symptom | Look here | Search for |
|---|---|---|
| 502 Bad Gateway | nginx error.log, then php-fpm log | upstream, connect() failed |
| 504 Gateway Timeout | nginx error.log | timed out |
| White screen of death | php-fpm log, wp-content/debug.log | Fatal error |
| Service will not start | journalctl -u <service> -n 50 | Read the last lines before it stopped |
| Server rebooted on its own | journalctl -b -1 -p err | panic, oom, hardware |
| Out of memory | dmesg -T | Killed process |
| Database connection errors | mysql error log, dmesg | crashed, OOM |
| Someone logged in | auth.log or secure | Accepted, sudo |
Rows one and three are the most common, and each takes two commands to answer.
# 1. When did it happen?
sudo grep '502' /var/log/nginx/access.log | tail -5
# 2. What did Nginx say at that moment?
sudo grep -A2 -B2 'upstream' /var/log/nginx/error.log | tail -20
# 3. What was PHP-FPM doing? This is where the real answer lives.
sudo grep -iE 'max_children|WARNING|ERROR' /var/log/php8.3-fpm.log | tail -20
# 4. Did the kernel kill something?
sudo dmesg -T | grep -i 'killed process' | tail# Everything from every source in one five-minute window
WHEN='2026-07-29 14:30'
sudo journalctl --since "$WHEN" --until "$WHEN:05" --no-pager
grep '29/Jul/2026:14:3' /var/log/nginx/error.log
grep '29/Jul/2026:14:3' /var/log/nginx/access.log | awk '$9 ~ /^5/'How do I stop logs filling the disk?
Logs are the most common cause of a full disk on a web server. A full disk stops the database writing, which takes the site down. logrotate trims the text files and journald caps its own size, but check both rather than assuming.
# Is logrotate configured for this service?
ls /etc/logrotate.d/
cat /etc/logrotate.d/nginx
# Test what it would do, without doing it
sudo logrotate -d /etc/logrotate.conf
# Force a rotation now
sudo logrotate -f /etc/logrotate.d/nginx
# Journal size
journalctl --disk-usage
sudo journalctl --vacuum-size=500M
sudo journalctl --vacuum-time=30dlogrotate -d is a dry run. Read it before you trust that rotation works the way you assume.Truncate a live log, never delete it
Deleting a log file that a process still has open does not free the space: the kernel keeps it in use until that process restarts. sudo truncate -s 0 /var/log/nginx/access.log empties the file and leaves it usable. Find files stuck in that state with sudo lsof +L1.
Logs tell you what happened afterwards. Knowing something is wrong in the first place is a different job: checking CPU, memory and disk usage covers the live view, and monitoring covers the hours nobody is watching. On G7Cloud, uptime is checked every minute and the dashboard shows each site's CPU and memory over the last 7 days. See platform monitoring.
Frequently asked questions
Where are Linux system logs stored?
systemd services and the kernel log to the journal, read with journalctl. Text logs live under /var/log: syslog or messages for general system output, auth.log or secure for logins, and a folder each for Nginx, Apache, MySQL and PHP-FPM.
How do I view logs for a specific service?
Run sudo journalctl -u servicename. Add -f to follow it live, -p err for errors only and --since '1 hour ago' for a time window. For services that write their own files, such as Nginx, read /var/log/nginx/error.log directly.
How do I see logs from before a reboot?
sudo journalctl -b -1 shows the previous boot, and -b -2 the one before that. This only works if the journal is saved to disk. If it lives in /run, it is held in memory and cleared on every restart. Create /var/log/journal to keep it.
What is the difference between the access log and the error log?
The access log records every request with its status code, size and user agent; use it to study traffic and bots. The error log records what went wrong: PHP fatal errors, upstream failures, permission problems. When something is broken, read the error log.
Why did my disk stay full after I deleted a log file?
A process still has the file open, so the kernel keeps the space in use until that process closes it. Use sudo lsof +L1 to find deleted files that are still open, then restart the process. Next time, use truncate -s 0 on live logs instead of rm.
How long should I keep server logs?
30 days covers most troubleshooting. Keep login logs longer, often 90 days, because a break-in is often found weeks after it happened. Set limits yourself in logrotate and journald rather than relying on the defaults.
About G7Cloud Engineering
Articles written by the engineers who build and run G7Cloud: UK managed hosting and the AI Website Builder. We write about what we operate every day: containers, backups, databases, and the small-business websites that run on them.
More about G7Cloud →