Skip to content
Laviyx Learn

Try

Linux and Bash course menu

Bash Project: Analyse a Web Server Log

Build a Bash report for web server logs: top pages, top IP addresses, status codes, errors by hour and total traffic, using grep, awk, sort and uniq.

Intermediate8 min readUpdated
On this page

Every web server records each request. In this project, you will turn a raw log into a report that answers real questions: which pages are popular, who is hitting the site hardest, and what is failing?

The log format#

The file access.log is in the common format used by Nginx and Apache. The addresses are from ranges reserved for documentation:

Terminal
head -n 2 access.log
wc -l access.log
Output
192.0.2.44 - - [05/Nov/2026:09:23:00 +0530] "GET /products HTTP/1.1" 200 4252 "-" "Mozilla/5.0"
192.0.2.10 - - [05/Nov/2026:13:44:00 +0530] "GET /login HTTP/1.1" 200 1718 "-" "Mozilla/5.0"
400 access.log

Splitting on spaces gives these awk fields:

FieldValue
$1Client IP address
$4Time, starting with [
$7The page requested
$9Status code
$10Bytes sent

Step 1: Try each question#

Most popular pages:

Terminal
awk '{print $7}' access.log | sort | uniq -c | sort -nr | head -n 3
Output
     98 /products
     87 /login
     70 /missing

Status codes:

Terminal
awk '{print $9}' access.log | sort | uniq -c | sort -nr
Output
    177 200
     96 404
     57 403
     35 500
     35 301

Pages that fail with a server error:

Terminal
awk '$9 >= 500 {print $7}' access.log | sort | uniq -c | sort -nr | head -n 3
Output
     14 /products
     12 /login
      6 /cart

Requests per hour:

Terminal
awk '{print substr($4, 14, 2)}' access.log | sort | uniq -c
Output
     58 08
     65 09
     69 10
     67 11
     69 12
     72 13

Total data sent:

Terminal
awk '{total += $10} END {printf "%.1f MB\n", total / 1024 / 1024}' access.log
Output
0.9 MB

Step 2: The script#

Now combine them in report.sh:

Terminal
cat > report.sh <<'EOF'
#!/usr/bin/env bash
# Usage: report.sh <access log> [top N]
set -euo pipefail

log_file="${1:?Usage: report.sh <access log> [top N]}"
top="${2:-3}"

[[ -r "$log_file" ]] || { echo "Cannot read $log_file" >&2; exit 1; }

heading() {
  printf '\n== %s ==\n' "$1"
}

top_values() {
  awk "{print \$$1}" "$log_file" | sort | uniq -c | sort -nr | head -n "$top"
}

total=$(wc -l < "$log_file")
errors=$(awk '$9 >= 400' "$log_file" | wc -l)

echo "Report for $log_file"
echo "Requests: $total"
awk -v e="$errors" -v t="$total" 'BEGIN {printf "Errors:   %d (%.1f%%)\n", e, e * 100 / t}'

heading "Top pages"
top_values 7

heading "Top clients"
top_values 1

heading "Status codes"
top_values 9

heading "Pages with server errors"
awk '$9 >= 500 {print $7}' "$log_file" | sort | uniq -c | sort -nr | head -n "$top"
EOF
chmod +x report.sh
./report.sh access.log
Output
Report for access.log
Requests: 400
Errors:   188 (47.0%)

== Top pages ==
     98 /products
     87 /login
     70 /missing

== Top clients ==
    142 192.0.2.10
     96 198.51.100.7
     65 203.0.113.9

== Status codes ==
    177 200
     96 404
     57 403

== Pages with server errors ==
     14 /products
     12 /login
      6 /cart

How it works#

  • ${1:?message} stops with a message if there is no argument.
  • top_values is a reusable function that takes a field number. Double quotes let $1 become the number, and \$ keeps a literal $ for awk.
  • awk -v passes shell values in safely.

Step 3: Save and compare#

Keep a daily report by redirecting the output:

Terminal
./report.sh access.log 5 > "report-$(date +%F).txt"
ls report-*.txt | wc -l
./report.sh missing.log
echo "Exit code: $?"
Output
1
Cannot read missing.log
Exit code: 1

Step 4: Automate it#

Run it every morning with cron:

Text
0 8 * * * /home/asha/report.sh /var/log/nginx/access.log 10 > /home/asha/reports/daily.txt 2>&1

Extend the project#

  • Accept several log files, including .gz files, using zcat.
  • Show the top IP addresses that receive 404 errors, which often indicates scanners.
  • Print a warning, and exit with code 1, if errors go over a limit, so that cron or CI can react.
  • Produce the report as an HTML page.
  • Load the log into a database, and answer the same questions with SQL.

Key takeaway#

Next: Linux interview questions.

Finished this lesson?

Related lessons

Linux and BashLinux awk Tutorial: Columns, Filters and TotalsLearn awk with examples: print columns, set field separators, filter rows, calculate totals and averages, group with associative arrays, and format reports with printf.Intermediate5 min readLinux and BashLinux Text Processing: sort, uniq, cut, tr and pasteProcess text from the command line: sort lines, remove and count duplicates with uniq, extract columns with cut, translate characters with tr, and build a top 10 report.Beginner5 min readLinux and BashLinux grep Command with ExamplesSearch text with grep: ignore case, count matches, show line numbers and context, search folders recursively, and use regular expressions with grep -E.Beginner5 min read