The Mindset
System logs matter to a pentester from two angles:
- Offensive — Logs contain credentials, activity trails, internal hostnames, and evidence of what’s running on the system
- Defensive awareness — Understanding what gets logged tells you what traces you’re leaving behind
Every action you take on a compromised system is potentially being logged somewhere. Know what those sources are.
Log Locations Overview
| Log File | What It Records |
|---|
/var/log/auth.log | SSH logins, sudo usage, PAM authentication (Debian/Ubuntu) |
/var/log/secure | Same as auth.log on RHEL/CentOS |
/var/log/syslog | General system events, service starts/stops, cron |
/var/log/kern.log | Kernel messages, hardware events, driver issues |
/var/log/messages | General messages (RHEL/CentOS equivalent of syslog) |
/var/log/dpkg.log | Package installs and removals |
/var/log/apache2/access.log | HTTP requests to Apache |
/var/log/apache2/error.log | Apache errors — often reveals paths and configs |
/var/log/nginx/access.log | HTTP requests to Nginx |
/var/log/mysql/error.log | MySQL errors and startups |
/var/log/postgresql/ | PostgreSQL activity |
/var/log/fail2ban.log | Blocked IPs, ban events |
/var/log/ufw.log | Firewall allow/deny events |
/var/log/mail.log | Mail server activity |
/var/log/journal/ | Systemd journal (binary — read with journalctl) |
Reading Logs
Basic Log Reading
# Read a full log
cat /var/log/syslog
# Last 50 lines (most recent activity)
tail -50 /var/log/auth.log
# Follow in real time
tail -f /var/log/auth.log
# First 20 lines
head -20 /var/log/syslog
# Page through a large log
less /var/log/syslog
Filtering Log Output
# Search for a specific string
grep "Failed password" /var/log/auth.log
# Case-insensitive search
grep -i "error" /var/log/syslog
# Show lines around a match (context)
grep -A 3 -B 3 "authentication failure" /var/log/auth.log
# Filter by date
grep "May 20" /var/log/auth.log
# Count occurrences
grep "Failed password" /var/log/auth.log | wc -l
Authentication Logs
/var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL/CentOS) is the most valuable log for a pentester. It records every login attempt, sudo command, and session event.
What to look for:
# Successful SSH logins — who logged in and from where
grep "Accepted" /var/log/auth.log
# Feb 28 18:15:01 sshd[5678]: Accepted publickey for admin from 10.14.15.2 port 43210
# Failed login attempts — brute force evidence, valid usernames
grep "Failed password" /var/log/auth.log
# Feb 28 15:04:22 sshd[3010]: Failed password for htb-student from 10.14.15.2 port 50223
# Sudo commands run — what privileged actions were taken
grep "sudo" /var/log/auth.log
# Feb 28 18:15:03 sudo: admin : TTY=pts/1 ; PWD=/home/admin ; USER=root ; COMMAND=/bin/bash
# New sessions created
grep "session opened" /var/log/auth.log
# su usage — user switching
grep "session opened for user" /var/log/auth.log
# All unique IPs that attempted to log in
grep "Failed password\|Accepted" /var/log/auth.log | grep -oE "[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}" | sort -u
# All usernames that were attempted (valid + invalid)
grep "Failed password" /var/log/auth.log | awk '{print $9}' | sort | uniq -c | sort -rn
# All commands run via sudo
grep "COMMAND" /var/log/auth.log | awk -F"COMMAND=" '{print $2}' | sort -u
System Logs
cat /var/log/syslog | tail -100
# Cron job execution — what ran and when
grep "CRON" /var/log/syslog
# Feb 28 15:00:01 server CRON[2715]: (root) CMD (/usr/local/bin/backup.sh)
# Service starts and stops
grep "systemd\[1\]" /var/log/syslog | grep -E "Started|Stopped|Failed"
# Network events
grep "NetworkManager\|dhclient" /var/log/syslog
# Errors and warnings
grep -E "error|warning|failed" /var/log/syslog -i | tail -30
Cron entries in syslog show you exactly what scripts run as root and when —
cross-reference with the actual scripts to find hijack opportunities.
Kernel Logs
cat /var/log/kern.log | tail -50
# Also accessible via dmesg
dmesg | tail -50
dmesg | grep -i "error\|fail\|usb\|eth"
# Driver and hardware issues (may reveal kernel version weaknesses)
grep -i "error\|fail" /var/log/kern.log
# USB device connections (removable media activity)
dmesg | grep -i "usb\|storage"
# Network interface events
dmesg | grep -i "eth\|wlan\|link"
# Out-of-memory events (reveals memory pressure — useful for DoS research)
grep "Out of memory\|oom" /var/log/kern.log
Application Logs
Web Server Logs
# Apache access log — every HTTP request
cat /var/log/apache2/access.log | tail -50
# Apache error log — application errors, path disclosures
cat /var/log/apache2/error.log | tail -50
# Nginx
cat /var/log/nginx/access.log | tail -50
cat /var/log/nginx/error.log | tail -50
# Credentials passed in GET parameters (bad practice but it happens)
grep -E "password=|passwd=|pass=|token=|key=|secret=" /var/log/apache2/access.log
# POST data isn't logged by default, but errors sometimes expose it
grep -i "password\|credential" /var/log/apache2/error.log
# User agents — identify tools used against this server
awk '{print $12}' /var/log/apache2/access.log | sort | uniq -c | sort -rn | head -20
# Most active IPs
awk '{print $1}' /var/log/apache2/access.log | sort | uniq -c | sort -rn | head -20
Database Logs
# MySQL errors (connection attempts, auth failures)
cat /var/log/mysql/error.log
# PostgreSQL logs
ls /var/log/postgresql/
cat /var/log/postgresql/postgresql-*-main.log | tail -50
Systemd Journal
On modern systems, many logs go to the systemd journal instead of flat files.
# All logs (most recent first)
journalctl -r
# Logs for a specific service
journalctl -u ssh.service
journalctl -u apache2.service
journalctl -u cron.service
# Logs since last boot
journalctl -b
# Logs from a specific time window
journalctl --since "2024-01-01 00:00" --until "2024-01-01 23:59"
# Follow in real time
journalctl -f
# Show only errors
journalctl -p err
# Without pager (pipe-friendly)
journalctl -u ssh --no-pager | grep "Accepted"
Security Logs
# Fail2ban — which IPs got banned and for what
cat /var/log/fail2ban.log
grep "Ban\|Unban" /var/log/fail2ban.log | tail -20
# UFW firewall events
cat /var/log/ufw.log | tail -50
grep "BLOCK" /var/log/ufw.log | awk '{print $12}' | sort | uniq -c | sort -rn
# AppArmor denials (processes hitting security boundaries)
grep "DENIED\|apparmor" /var/log/syslog | tail -20
# Audit daemon (if auditd is running)
cat /var/log/audit/audit.log 2>/dev/null | tail -50
ausearch -k passwd_changes 2>/dev/null # Search audit log by key
Log Analysis for Pentesters
Reconstructing What Happened
# Timeline of all auth events for a specific user
grep "htb-student" /var/log/auth.log | sort
# What commands did admin run via sudo today?
grep "sudo.*admin\|admin.*sudo" /var/log/auth.log | grep "COMMAND"
# What connected to this machine in the last hour?
grep "Accepted\|Failed" /var/log/auth.log | tail -100
# What services restarted recently?
grep "Started\|Stopped" /var/log/syslog | tail -30
# Any privilege escalation attempts?
grep "su\[" /var/log/auth.log
grep "sudo" /var/log/auth.log | grep -v "session\|pam"
Finding Cleartext Credentials in Logs
# Broad credential search across all readable logs
grep -rE "password|passwd|secret|token|credential|api.key" /var/log/ 2>/dev/null | grep -v "^Binary"
# FTP credentials (often logged in plaintext)
grep -i "pass\|user" /var/log/vsftpd.log 2>/dev/null
# SMTP AUTH credentials
grep -i "AUTH\|LOGIN" /var/log/mail.log 2>/dev/null
