Mashqlar — log fayllarni tahlil qilish

~25 daq · Linux · Matn bilan ishlash

Muammodan boshlaylik

Dushanba ertalab. Rahbar yozadi: "Kecha sayt sekin ishladi, ba'zilarda xato chiqdi. Nima bo'lgan? Kim ko'p so'rov yuborgan? Qaysi sahifalar muammoli?"

Sizda — Nginx access.log va ilova app.log. Grafana yo'q, monitoring yo'q. Faqat terminal va 12–16-darslarda o'rgangan vositalar. Bu — real hayotdagi eng tipik vaziyatlardan biri.

Tayyorgarlik

Sinov loglarini yaratuvchi skript:

mkdir -p ~/mashq-log && cd ~/mashq-log

cat > yarat.sh << 'EOF'
#!/bin/bash
ipler=(185.220.101.4 91.213.8.92 213.230.112.5 84.54.70.18 95.46.98.12 10.0.0.5)
yollar=("/" "/mahsulotlar" "/mahsulot/42" "/savat" "/api/buyurtma" "/login" "/wp-admin")
kodlar=(200 200 200 200 200 200 301 404 500 502)
for i in $(seq 1 5000); do
  ip=${ipler[$((RANDOM % 6))]}
  [ $((RANDOM % 3)) -eq 0 ] && ip=185.220.101.4
  yol=${yollar[$((RANDOM % 7))]}
  kod=${kodlar[$((RANDOM % 10))]}
  hajm=$((RANDOM % 50000))
  soat=$(printf "%02d" $((RANDOM % 24)))
  daq=$(printf "%02d" $((RANDOM % 60)))
  usul=GET; [ "$yol" = "/api/buyurtma" ] && usul=POST
  echo "$ip - - [05/Nov/2026:$soat:$daq:00 +0500] \"$usul $yol HTTP/1.1\" $kod $hajm \"-\" \"Mozilla/5.0\""
done > access.log

for i in $(seq 1 2000); do
  soat=$(printf "%02d" $((RANDOM % 24)))
  r=$((RANDOM % 10))
  if   [ $r -lt 6 ]; then echo "2026-11-05 $soat:00:00 INFO  So'rov bajarildi"
  elif [ $r -lt 8 ]; then echo "2026-11-05 $soat:00:00 WARN  Sekin so'rov: $((RANDOM % 5000))ms"
  elif [ $r -lt 9 ]; then echo "2026-11-05 $soat:00:00 ERROR Baza ulanmadi: timeout"
  else                    echo "2026-11-05 $soat:00:00 ERROR Buyurtma $((RANDOM % 100)) topilmadi"
  fi
done > app.log
EOF

bash yarat.sh
wc -l access.log app.log

Nginx log formati:

185.220.101.4 - - [05/Nov/2026:14:23:00 +0500] "GET /mahsulotlar HTTP/1.1" 200 12345 "-" "Mozilla/5.0"
$1                  $4                           $6   $7                  $9  $10

awk bo'shliq bo'yicha ajratganda: $1 — IP, $7 — yo'l, $9 — status kodi, $10 — hajm.

1-mashq: Umumiy manzara

  1. Jami nechta so'rov bo'lgan?
  2. Nechta so'rov xato bilan tugagan (status 500 yoki 502)?
  3. Nechta 404?
Yechim
wc -l < access.log
awk '$9 >= 500' access.log | wc -l
awk '$9 == 404' access.log | wc -l

Yoki grep bilan: grep -cE '" 5[0-9]{2} ' access.log.

2-mashq: Status kodlari taqsimoti

Har bir status kodi necha marta uchragan — kamayish tartibida.

Yechim
awk '{print $9}' access.log | sort | uniq -c | sort -rn

   3012 200
    498 301
    511 404
    492 500
    487 502

3-mashq: Shubhali IP

  1. Eng ko'p so'rov yuborgan 5 ta IP
  2. Eng faol IP ning ulushi necha foiz?
  3. Kim /wp-admin ga urinishgan (bizda WordPress yo'q — bu skaner)?
Yechim
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -5

awk '{c[$1]++} END {for (ip in c) printf "%s %.1f%%\n", ip, c[ip]*100/NR}' access.log \
  | sort -k2 -rn | head -1

grep "/wp-admin" access.log | awk '{print $1}' | sort | uniq -c | sort -rn

185.220.101.4 — so'rovlarning ~40% i. Bu bot yoki hujum bo'lishi mumkin; uni bloklash — 36-dars (firewall).

4-mashq: Muammoli sahifalar

  1. Qaysi yo'llar eng ko'p 5xx xato bergan?
  2. Har bir yo'l bo'yicha xatolar foizi (jami so'rovlarga nisbatan)
Yechim
awk '$9 >= 500 {print $7}' access.log | sort | uniq -c | sort -rn

awk '{jami[$7]++} $9 >= 500 {xato[$7]++}
     END {for (y in jami) printf "%-15s %5d %5d %5.1f%%\n", y, jami[y], xato[y], xato[y]*100/jami[y]}' \
  access.log | sort -k4 -rn

/savat            712   151  21.2%
/api/buyurtma     698   145  20.8%
...

Ikkinchi buyruqda ikki blok: birinchisi har qatorda, ikkinchisi faqat shart bajarilganda.

5-mashq: Vaqt bo'yicha

Qaysi soatda eng ko'p so'rov bo'lgan? Soatlar bo'yicha taqsimot.

Yechim
awk '{print $4}' access.log | cut -d: -f2 | sort | uniq -c

$4 = [05/Nov/2026:14:23:00 → : bo'yicha 2-qism — soat.

Eng ko'p:

awk '{print $4}' access.log | cut -d: -f2 | sort | uniq -c | sort -rn | head -3

Grafik ko'rinishda (oddiy "ustunlar"):

awk '{split($4, t, ":"); c[t[2]]++} END {for (h in c) {printf "%s ", h; for (i=0; i<c[h]/10; i++) printf "#"; print ""}}' access.log | sort

6-mashq: Trafik hajmi

  1. Jami qancha ma'lumot yuborilgan (MB)?
  2. Qaysi IP eng ko'p trafik olgan?
Yechim
awk '{s += $10} END {printf "%.1f MB\n", s/1024/1024}' access.log

awk '{t[$1] += $10} END {for (ip in t) printf "%s %.1f MB\n", ip, t[ip]/1024/1024}' access.log \
  | sort -k2 -rn | head -3

7-mashq: Ilova logi

app.log da:

  1. Har bir daraja (INFO, WARN, ERROR) necha marta?
  2. Xatolar turlari bo'yicha (raqamlarni hisobga olmasdan) — "Buyurtma 15 topilmadi" va "Buyurtma 87 topilmadi" — bitta tur
  3. Eng sekin so'rov necha ms?
Yechim
awk '{print $3}' app.log | sort | uniq -c

grep ERROR app.log | cut -d' ' -f4- | sed 's/[0-9]\+/N/g' | sort | uniq -c | sort -rn

    204 Baza ulanmadi: timeout
    198 Buyurtma N topilmadi

grep "Sekin" app.log | grep -oE "[0-9]+ms" | tr -d 'ms' | sort -n | tail -1

sed 's/[0-9]\+/N/g' — barcha raqamlarni N ga almashtirish: shunda bir xil turdagi xatolar guruhlanadi. Bu real log tahlilida juda foydali usul.

8-mashq: Hisobot

Yuqoridagilarni birlashtirib, hisobot.txt faylini yarating: sana, jami so'rovlar, xatolar soni, top-3 IP, top-3 muammoli sahifa.

Yechim
{
  echo "=== Kunlik hisobot: $(date +%F) ==="
  echo
  echo "Jami so'rovlar: $(wc -l < access.log)"
  echo "5xx xatolar:    $(awk '$9 >= 500' access.log | wc -l)"
  echo "404:            $(awk '$9 == 404' access.log | wc -l)"
  echo
  echo "--- Top 3 IP ---"
  awk '{print $1}' access.log | sort | uniq -c | sort -rn | head -3
  echo
  echo "--- Eng ko'p 5xx bergan sahifalar ---"
  awk '$9 >= 500 {print $7}' access.log | sort | uniq -c | sort -rn | head -3
  echo
  echo "--- Ilova xatolari ---"
  grep ERROR app.log | cut -d' ' -f4- | sed 's/[0-9]\+/N/g' | sort | uniq -c | sort -rn
} > hisobot.txt

cat hisobot.txt

{ ...; } > fayl — bir nechta buyruq chiqishini bitta faylga. 37–41-darslarda buni skriptga aylantirib, cron bilan har kuni avtomatik ishga tushiramiz.

Nima uchun shunday?

Nima uchun bu ko'nikma muhim — monitoring tizimlari (Grafana, ELK) bor-ku?

Chunki ular har doim ham yo'q: kichik loyiha, yangi server, mijozning serveri, yoki monitoringning o'zi ishlamay qolgan. Va ular bo'lsa ham, ko'pincha oldindan o'ylanmagan savol tug'iladi — "faqat /savat dagi, faqat shu IP dan, faqat 14:00–15:00 orasidagi 502 lar" — bunday so'rov uchun dashboard yo'q. Terminal vositalari esa istalgan savolga bir necha daqiqada javob beradi, har qanday formatdagi log bilan. Bu — "muammo nima" degan savolga mustaqil javob topa oladigan dasturchining ko'nikmasi.

Xulosa

Siz mashq qildingiz:

  • Nginx log ustunlari: $1 IP, $7 yo'l, $9 status, $10 hajm
  • awk '$9 >= 500' — filtrlash; sort | uniq -c | sort -rn | head — top-N
  • awk massivlari — guruhlash va foizlar
  • cut -d: — vaqtni ajratish
  • sed 's/[0-9]\+/N/g' — xatolarni turlar bo'yicha guruhlash
  • { ...; } > fayl — hisobot yaratish

Keyingi bo'lim

4-bo'lim: Foydalanuvchilar va ruxsatlar — sudo, chmod, va nega "Permission denied".