Loglar va monitoring — logging driver, stats, Uptime

~12 daq · Docker · Production

Muammodan boshlaylik

Dushanba ertalab mijoz yozadi: "Saytingiz kecha kechqurun 2 soat ishlamadi". Siz bilmagansiz. Nima bo'lgan? Loglar... qaysi konteynerda? Konteyner qayta yaratilgan — eski loglar yo'q. Xotira to'lganmi? CPU? Disk? Hech qanday ma'lumot yo'q.

Production da ikki savol doimo javobga ega bo'lishi kerak:

1. Hozir hammasi ishlayaptimi? — va yo'q bo'lsa, men birinchi bilishim kerak, mijoz emas
2. Nima bo'lgan edi? — loglar va metrikalar tarixi

Loglar: asos

Qoida (8-dars): ilova loglarni stdout/stderr ga yozadi, Docker ularni yig'adi.

docker compose logs -f --tail=100 api
docker compose logs --since 2h api
docker compose logs api | grep -i error

JSON loglar

Matnli log:

2026-11-06 14:23:01 ERROR Buyurtma yaratilmadi user=42 order=1001

Tuzilmali (JSON) log:

{"time":"2026-11-06T14:23:01Z","level":"error","msg":"Buyurtma yaratilmadi","userId":42,"orderId":1001,"reqId":"a3f9"}

JSON ni mashina tahlil qila oladi: "42-foydalanuvchining barcha xatolari", "oxirgi soatdagi xatolar soni". Node.js da — pino, Python da — structlog, Laravel — LOG_STDERR_FORMATTER=Monolog\Formatter\JsonFormatter.

Log driver lar

Docker loglarni qayerga yozishini log driver belgilaydi:

Xizmat darajasida:

services:
  api:
    logging:
      driver: local
      options:
        max-size: "10m"
        max-file: "5"

[!WARNING] Log hajmi chegarasini albatta qo'ying (42-darsdagi daemon.json yoki har xizmat uchun). Cheksiz loglar — production da diskni to'ldirishning eng keng tarqalgan sababi.

Markazlashgan loglar

Bir nechta server va o'nlab konteyner bo'lsa, docker logs bilan har birini alohida ko'rish imkonsiz. Kerak — barcha loglar bir joyda, qidiruv bilan.

Ommabop yengil to'plam: Grafana Loki + Promtail/Alloy + Grafana.

services:
  loki:
    image: grafana/loki:3.3.0
    volumes: [loki-data:/loki]

  promtail:
    image: grafana/promtail:3.3.0
    volumes:
      - /var/lib/docker/containers:/var/lib/docker/containers:ro
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./promtail.yaml:/etc/promtail/config.yaml:ro

  grafana:
    image: grafana/grafana:11.4.0
    ports: ["127.0.0.1:3001:3000"]
    volumes: [grafana-data:/var/lib/grafana]

volumes:
  loki-data:
  grafana-data:

Promtail konteyner loglarini o'qiydi → Loki ga yuboradi → Grafana da qidirasiz:

{container="dokon-api-1"} |= "error" | json | userId = 42

Boshqa variantlar: ELK (Elasticsearch + Logstash + Kibana) — kuchli, lekin og'ir; bulut xizmatlari — Better Stack, Datadog, Grafana Cloud (bepul tarif bor).

Healthcheck — holat

healthcheck:
  test: ["CMD", "wget", "-qO-", "http://localhost:3000/health"]
  interval: 15s
  timeout: 3s
  retries: 3
  start_period: 20s

docker compose ps
docker inspect api --format '{{json .State.Health}}' | jq

Yaxshi /health endpoint:

app.get('/health', async (req, res) => {
  try {
    await db.query('SELECT 1');
    await redis.ping();
    res.json({ status: 'ok' });
  } catch (e) {
    res.status(503).json({ status: 'error', error: e.message });
  }
});

Eslatma (40-dars): Compose unhealthy konteynerni avtomatik qayta ishga tushirmaydi — healthcheck ma'lumot beradi, ogohlantirish va harakat — monitoring vazifasi.

Metrikalar

Loglar — "nima bo'ldi". Metrikalar — "qancha": CPU, xotira, so'rovlar soni, javob vaqti, xatolar foizi — vaqt bo'yicha grafiklar.

Standart to'plam: Prometheus (yig'adi) + cAdvisor (konteyner metrikalari) + node-exporter (server metrikalari) + Grafana (grafiklar).

services:
  prometheus:
    image: prom/prometheus:v3.0.1
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
      - prom-data:/prometheus

  cadvisor:
    image: gcr.io/cadvisor/cadvisor:v0.49.1
    volumes:
      - /:/rootfs:ro
      - /var/run:/var/run:ro
      - /sys:/sys:ro
      - /var/lib/docker/:/var/lib/docker:ro

  node-exporter:
    image: prom/node-exporter:v1.8.2
    pid: host
    volumes:
      - /:/host:ro,rslave
    command: ["--path.rootfs=/host"]

# prometheus.yml
scrape_configs:
  - job_name: cadvisor
    static_configs: [{ targets: ["cadvisor:8080"] }]
  - job_name: node
    static_configs: [{ targets: ["node-exporter:9100"] }]
  - job_name: api
    static_configs: [{ targets: ["api:3000"] }]     # ilova /metrics beradi

Grafana da tayyor dashboard larni import qiling (masalan, "Docker monitoring", "Node Exporter Full") — bir necha daqiqada to'liq manzara.

Ilova metrikalari: Node.js — prom-client, Python — prometheus-client, Laravel — paketlar orqali. Eng muhim to'rttasi ("oltin signallar"): so'rovlar soni, xatolar, javob vaqti, to'yinganlik (resurslar).

Ogohlantirishlar

Grafik ni hech kim kechasi ko'rib o'tirmaydi. Kerak — ogohlantirish (alert):

Kanallar: Telegram, email, Slack — Grafana Alerting yoki Alertmanager orqali.

Eng oddiy boshlanish: tashqi uptime monitoring

Hech narsa o'rnatmasdan — tashqaridan tekshiradigan xizmat: UptimeRobot, Better Stack (bepul tariflar), yoki o'zingizniki — Uptime Kuma:

services:
  uptime-kuma:
    image: louislam/uptime-kuma:1
    restart: unless-stopped
    volumes: [kuma-data:/app/data]
    ports: ["127.0.0.1:3002:3001"]

[!TIP] Uptime monitoring ni boshqa serverda ishlating. Server o'chsa — o'sha serverdagi monitoring ham o'chadi va hech kim xabar bermaydi.

Qayerdan boshlash

Kichik loyiha uchun bosqichma-bosqich:

1. Log hajmi chegarasi + JSON loglar + /health — bugun
2. Tashqi uptime monitoring + Telegram ogohlantirish — bugun
3. docker stats va disk ni vaqti-vaqti bilan ko'rish
4. O'sganda: Prometheus + Grafana + Loki

Nima uchun shunday?

Nima uchun ham loglar, ham metrikalar, ham ogohlantirishlar kerak — bittasi yetmaydimi?

Chunki ular turli savollarga javob beradi. Ogohlantirish — "muammo bor" (va siz mijozdan oldin bilasiz). Metrikalar — "qayerda va qachondan": grafikda xotira 18:00 dan sekin o'sib, 20:00 da OOMKilled bo'lganini ko'rasiz. Loglar — "aynan nima": o'sha vaqtdagi xato xabari va stack trace. Birgalikda ular "2 soat ishlamadi, sababi noma'lum" ni "20:00 da xotira oqishi sababli API qulagan, 3 daqiqada ogohlantirish keldi, sabab — 1.4.2 dagi kesh xatosi" ga aylantiradi.

Xulosa

Eslab qoling:

• Loglar — stdout ga, iloji bo'lsa JSON; docker compose logs -f --since
• Log driver: json-file / local + albatta max-size, max-file
• Markazlashgan loglar: Loki + Promtail + Grafana (yoki ELK, bulut)
• /health — haqiqiy bog'liqliklarni tekshirsin
• Metrikalar: Prometheus + cAdvisor + node-exporter + Grafana
• Ogohlantirishlar: ishlamaslik, xatolar, disk, xotira, restart lar, sertifikat
• Tashqi uptime monitoring — boshqa serverda, birinchi qadam

Keyingi bo'lim

9-bo'lim: Keyingi qadam — Kubernetes ga kirish va yakuniy loyiha.