---
name: logging-and-observability-review
description: "Reviews what a service logs and measures, then sets the levels, fields and alerts that let on-call find a problem in minutes without drowning in noise. Use when a service's logs are noisy or unhelpful during incidents and its metrics and alerts need reviewing for levels, fields and coverage. Do not use for choosing a monitoring vendor or debugging one specific outage. Related topics: devops, coding."
---

# Logging and Observability Review

Reviews what a service logs and measures, then sets the levels, fields and alerts that let on-call find a problem in minutes without drowning in noise.

## When to use

Use this skill when a service's logs are noisy or unhelpful during incidents and its metrics and alerts need reviewing for levels, fields and coverage. Do not use it for choosing a monitoring vendor or debugging one specific outage.

## Workflow

1. Ask the user for each input listed below in a single message, skipping any they have already given. Where options are listed, offer them, but accept other answers too.
2. Grade each sample log line on level, message stability and structured fields, and flag any that leak personal data or secrets.
3. List the events that are missing, including request boundaries, dependency calls, retries and state changes.
4. Propose metrics for latency, traffic, errors and saturation, with a name and unit for each.
5. Write one alert per symptom with a threshold, a window and the action expected from the responder.
6. Rank all changes by value and effort, and name the dashboard panels to build first.

## Inputs

- **Service description** (longer free text)
- **Sample log lines** (longer free text)
- **Current metrics and alerts** (longer free text)
- **Logging stack** (free text, e.g. "Datadog or Cloud Logging or ELK")
- **Typical incident** (longer free text)
- **Traffic level** (choose one: Low, Medium, High, Very high)

## Output

A graded table of log findings, a list of missing events, a metrics and alerts table with thresholds and actions, and a ranked change list with first dashboard panels.

## Prompt template

Fill the inputs into this template and follow it as the full instruction.

### Logging and Observability Review

Review the logging and monitoring of this service.

- Service description: {{Service description}}
- Sample log lines: {{Sample log lines}}
- Current metrics and alerts: {{Current metrics and alerts}}
- Logging stack: {{Logging stack}}
- Typical incident: {{Typical incident}}
- Traffic level: {{Traffic level}}

Judge the logs against the question an on-call engineer asks at 3 a.m.: what failed, for whom, and why. Check each sample line for level (errors only for things needing action, warnings for odd but handled events), a stable message, and structured fields such as request id, user or tenant id, operation, duration and error type. Flag logs that leak personal data, tokens or full payloads. Identify missing events: request start and end, calls to dependencies, retries, and state changes. Then propose the four golden signals as metrics (latency, traffic, errors, saturation) and for each a sample alert with a threshold and the action expected, favouring alerts on symptoms users feel over causes. Say which noisy logs to drop or sample at this traffic level. End with a prioritised list of changes and the dashboard panels to build first.

## Pitfalls

- Alerting on every error log, which trains the on-call to ignore pages.
- Logging full request bodies for debugging, which stores personal data and secrets in a searchable system.
- Adding fields with unbounded values as metric labels, which makes the metrics bill and the database explode.