MultiCloud Classroom notes 14/Nov/2026

Azure vm app logs monitoring

1. gnerate app logs / run any application which provide logs

#!/usr/bin/env python3
"""
gen_logs.py - generate sample web-app logs as JSON Lines.

Each line is one JSON object whose keys match the Azure table columns, including
TimeGenerated, so the custom-table wizard finds the required column.

Run:
    sudo mkdir -p /var/log/webapp && sudo chmod 777 /var/log/webapp
    python3 gen_logs.py                       # runs until Ctrl+C
    python3 gen_logs.py --count 50 --stdout   # 50 lines, also print to screen
"""
import argparse
import json
import random
import time
from datetime import datetime, timezone

PATHS = ["/api/orders", "/api/users", "/api/cart", "/login", "/health"]
METHODS = ["GET", "POST"]


def make_record():
    status = random.choices([200, 201, 404, 500], [80, 8, 7, 5])[0]
    return {
        "TimeGenerated": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
        "Level": "ERROR" if status >= 500 else "WARN" if status >= 400 else "INFO",
        "Method": random.choice(METHODS),
        "Path": random.choice(PATHS),
        "Status": status,
        "LatencyMs": random.randint(20, 900),
    }


def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--output", default="/var/log/webapp/webapp.log")
    ap.add_argument("--count", type=int, default=0, help="0 = run forever")
    ap.add_argument("--stdout", action="store_true")
    args = ap.parse_args()

    written = 0
    try:
        with open(args.output, "a", buffering=1) as f:
            while args.count == 0 or written < args.count:
                line = json.dumps(make_record())      # one object per line, no pretty-printing
                f.write(line + "\n")
                if args.stdout:
                    print(line)
                written += 1
                time.sleep(random.uniform(0.2, 2))
    except PermissionError:
        raise SystemExit(f"Cannot write to {args.output}. Create the folder and fix permissions first.")
    except KeyboardInterrupt:
        pass


if __name__ == "__main__":
    main()

steps:

1. create sample.py and insert above data
2. sudo python3 sample.py # manual run scrpit
3. sudo nohup python3 sample.py > /dev/null 2>&1 & # run in backgroup to generate more logs run it 2-3 hours
4. verify logs tail -f /var/log/webapp/webapp.log

Log Analytics workspaces & Data collection rule

1. create Log Analytics workspaces select sub & resoucegroup
2. create table
   1. select any one table plans
      • Analytics (PROD)
      • Basic (test valiadate)
      • Auxiliary / Lake (long storage and cheep)
   2. table name having prefix _CL ex: qt now my table name qt_CL
   3. Data collection rule
      1. select Subscription, Resoucegroup and name of rule
   4. Data collection endpoint -> select Subscription, Resoucegroup and name of endpoint (it will create endpoint will not visual)
   5. uplaod sample json

[
    {
        "TimeGenerated": "2026-11-13T10:15:02Z",
        "Level": "INFO",
        "Method": "GET",
        "Path": "/api/orders",
        "Status": 200,
        "LatencyMs": 123
    },
    {
        "TimeGenerated": "2026-11-13T10:15:03Z",
        "Level": "INFO",
        "Method": "POST",
        "Path": "/api/cart",
        "Status": 201,
        "LatencyMs": 245
    },
    {
        "TimeGenerated": "2026-11-13T10:15:05Z",
        "Level": "INFO",
        "Method": "GET",
        "Path": "/health",
        "Status": 200,
        "LatencyMs": 31
    },
    {
        "TimeGenerated": "2026-11-13T10:15:06Z",
        "Level": "WARN",
        "Method": "GET",
        "Path": "/api/users",
        "Status": 404,
        "LatencyMs": 88
    },
    {
        "TimeGenerated": "2026-11-13T10:15:08Z",
        "Level": "INFO",
        "Method": "POST",
        "Path": "/login",
        "Status": 200,
        "LatencyMs": 412
    },
    {
        "TimeGenerated": "2026-11-13T10:15:09Z",
        "Level": "ERROR",
        "Method": "GET",
        "Path": "/api/orders",
        "Status": 500,
        "LatencyMs": 870
    },
    {
        "TimeGenerated": "2026-11-13T10:15:11Z",
        "Level": "INFO",
        "Method": "GET",
        "Path": "/api/cart",
        "Status": 200,
        "LatencyMs": 97
    },
    {
        "TimeGenerated": "2026-11-13T10:15:12Z",
        "Level": "INFO",
        "Method": "GET",
        "Path": "/health",
        "Status": 200,
        "LatencyMs": 29
    },
    {
        "TimeGenerated": "2026-11-13T10:15:14Z",
        "Level": "WARN",
        "Method": "POST",
        "Path": "/api/users",
        "Status": 404,
        "LatencyMs": 140
    },
    {
        "TimeGenerated": "2026-11-13T10:15:15Z",
        "Level": "INFO",
        "Method": "GET",
        "Path": "/api/orders",
        "Status": 200,
        "LatencyMs": 156
    }
]

verify table in log Analytics workspaces

select table click manage table (Data retention settings)

• Analytics retention – (how many days activitly access logs)
• Total retention – keep max days

Schema Editor

• manage columns and types
• ex: below

create DCR rule

• Now Review and create DCR rule

task

1. setup upto DCR and wait 2-3 hour
2. try select logs in log Analytics workspaces selects logs and run Below KQL querry.

logs (KQL querry)

qt_CL
| project TimeGenerated, Level, Method, Path, Status, LatencyMs
| order by TimeGenerated desc

qt_CL
| where Level == "ERROR"
| project TimeGenerated, Method, Path, Status, LatencyMs

qt_CL
| where Status >= 500
| project TimeGenerated, Level, Method, Path, Status, LatencyMs