kai.kzLab

KAI Lab

The DevOps engineer's AI toolkit

Prompts, repository rules for AI assistants, a skill and an agent that save hours on call, in reviews and in documentation. They work in ChatGPT, Claude, Cursor and any similar assistant: copy them and replace the <angle brackets> with your own data.

Five rules first

  1. Never paste secrets into a chat: tokens, keys, passwords, database connection strings or customers' personal data. Replace them with <REDACTED> before you hit send.
  2. AI is confidently wrong. Verify with a command everything it states as a fact about your system.
  3. Do not run commands blindly. In production start with the read-only variant: --dry-run, plan, --check, diff.
  4. Give an agent the least privilege: read-only in production, changes only through an MR reviewed by a human.
  5. Find out where code and logs may be sent: company policy may forbid external services. Ask your security team which tool to use.

Everyday prompts

Eight situations where AI genuinely speeds up the work.

Investigating an incident from logs

An alert fired, you have logs, you have no hypothesis.

You are a senior SRE. Help me investigate an incident. Do not invent facts: if data is missing, tell me which command to run to get it.

Symptom: <what users see or what the alert says>
Started at: <time>
Recent changes: <deploys, configs, migrations>
Logs (no secrets or personal data):
<30–100 lines around the first error>

Answer like this:
1. Three hypotheses, most likely first — for each: why you think so (a log line) and which command checks it.
2. What to do right now to reduce the impact (rollback, failover, limits) without waiting for the cause.
3. What not to do, so we do not make it worse.

What is wrong with this Kubernetes pod

CrashLoopBackOff, ImagePullBackOff, Pending, OOMKilled.

Explain what is happening to the pod and how to fix it. Answer from the output below, no generic advice.

kubectl describe pod <name>:
<output>

kubectl logs <name> --previous (if any):
<output>

Format: the cause in one sentence → evidence from the output → a command to verify it → the fix → how to prevent it happening again (limits, probes, alert).

Dockerfile review

Before merging a new service, or when an image has grown too big.

Review this Dockerfile as an experienced DevOps engineer. For each finding: the line, the problem, why it matters (size, security, build speed), and a fixed version.

Check at least: the base image and its version (not latest), layer order for caching, a multi-stage build, not running as root, extra files in the image (.dockerignore), secrets in layers, HEALTHCHECK.

<Dockerfile>

terraform plan review

Before apply, especially for someone else's MR.

Below is terraform plan output. Find everything that is dangerous to apply without looking.

1. Resources that will be destroyed or replaced (destroy, -/+, forces replacement) — and what is lost with them: data, IP addresses, DNS records.
2. Permission changes: IAM, security groups, ports exposed to the internet.
3. What should be applied separately, and in which order.
If everything is safe, say so — do not invent problems.

<terraform plan output>

A postmortem draft

The incident is closed and the notes are scattered across the chat.

Write a draft blameless postmortem from the notes below. Do not add facts that are not in the notes — mark gaps as [to confirm].

Sections:
- Impact — in business metrics: who was affected, for how long, what it cost.
- Root cause — what in the system and process allowed the failure, not “who made a mistake”.
- Timeline — time → event.
- What worked and what did not.
- Action items — each with an owner and a due date; at least one catches the problem automatically.

Notes:
<the timeline from the chat, times, numbers>

An alert on a symptom

You need a Prometheus alert that wakes people up for a reason.

I need an alert for Prometheus and Alertmanager. Service: <name>. Available metrics: <for example, http_requests_total{status}, the http_request_duration_seconds histogram>.

I want to know when: <the symptom as users see it — for example, more than 2% of responses are 5xx, or p95 is above 800 ms>.

Give me the rule: expr with an explanation, for:, labels (severity), annotations (summary, runbook_url). Explain why that for: value and how to avoid noise at night.

A bash one-liner, explained

You need a command you cannot afford to get wrong.

Write a Linux command for: <what needs doing — for example, find the 10 largest files in /var older than 7 days>.

First a safe variant that only shows what would happen — no deletions or changes. Then the working one. Explain every part of the command. If it could do harm in production, warn me.

A runbook from a ticket

The incident happened a second time — time to write down how to live with it.

From the incident description below, write a runbook for on-call:
- how to recognise it: the alert, the symptoms;
- how to check: commands step by step, and what counts as normal;
- how to mitigate: the safest action first;
- when to escalate and to whom.

Write for someone woken up at 3 a.m.: short, step by step, commands in code blocks.

<the incident or ticket description>

Repository rules and a skill for AI

Files you put into a repository so that an AI assistant (Claude Code, Cursor, Codex) follows the team's rules.

Rules for an infrastructure repository

Once — at the root of a repository with Terraform, Ansible or manifests.

File: CLAUDE.md · AGENTS.md · .cursor/rules/infra.mdc

# Rules for the AI assistant in this repository

This is an infrastructure repository. A mistake here takes production down.

- Do not run terraform apply, kubectl apply/delete, helm upgrade, or ansible-playbook without --check. Prepare the change and the command — a human applies it.
- Before any change, show what will be affected: terraform plan, kubectl diff, ansible-playbook --check --diff.
- Do not read or print secrets: .env, *.tfvars, kubeconfig, vault files. If you need them, ask a human.
- Change as little as possible: one task, one MR. Do not reformat other people's files.
- Anything that deletes data or resources happens only after explicit confirmation in the chat.
- Not sure which cluster or environment is active? Ask instead of guessing.

The k8s-debug skill for Claude Code

So the assistant investigates pod problems in one proven order.

File: .claude/skills/k8s-debug/SKILL.md

---
name: k8s-debug
description: Diagnose Kubernetes pods — CrashLoopBackOff, ImagePullBackOff, Pending, OOMKilled. Use when asked why a pod will not start or keeps restarting.
---

Read-only. Do not run apply, delete, edit, scale or rollout restart.

1. Check the context and namespace: kubectl config current-context. If it is production, say so.
2. kubectl get pod <pod> -o wide — status, restart count, node.
3. kubectl describe pod <pod> — Events from the bottom up, and Last State (Reason, Exit Code).
4. kubectl logs <pod> --previous — logs of the crashed container.
5. Then by symptom:
   - Exit Code 137 or OOMKilled → compare kubectl top pod with limits.memory.
   - ImagePullBackOff → image name and tag, imagePullSecrets, registry access.
   - Pending → not enough resources on nodes, taints, an unbound PVC.
   - CrashLoopBackOff with exit code 1 → an application error: logs and environment variables.
6. Result: the cause in one sentence, the evidence as an output line, the fix as a manifest diff. A human applies it.

An on-call assistant agent

A system prompt for an agent with read-only access to logs, metrics and the cluster.

The on-call assistant

When the agent is connected to logs and metrics and looks at an alert first.

You are an on-call engineer's assistant. You have read-only access: logs, metrics, kubectl get/describe/logs, deploy history. You change nothing and propose no change without a human's confirmation.

When an alert comes in:
1. Determine what is affected for users — or that nothing is, and it is noise.
2. Gather facts: when it started, what changed in the 2 hours before (deploys, configs), whether similar alerts happened before.
3. Give 1–2 hypotheses with evidence, and commands to verify them.
4. Propose the safest mitigation — rolling back the last deploy, shifting traffic — as a command, but do not run it.
5. Be brief: on-call reads on a phone. Conclusion first, details after.

If data is missing, say so and name where to look. Never invent log lines or metric values.

A prompt works better the more precise the context: which services, which environment, what changed. Keep the ones that work in the team's repository — then everyone uses them, not just whoever wrote them.

Back to the tasks