Skip to Content
User GuideRunbooks

How to Work with Runbooks

Runbooks are step-by-step remediation playbooks. This guide covers creating, configuring, approving, and executing runbooks in an enterprise workflow.

When to use runbooks

  • After RCA: When you have a root cause and a clear fix (e.g. restart service, scale pod, clear cache). Generate a runbook from the incident or pick a template.
  • Recurring fixes: When the same type of incident happens often — turn the fix into an approved runbook so next time you (or the AI) can run it with one click.
  • Controlled automation: When you want automation but need approval gates (L1/L2) so humans approve before risky steps run.

Why approval matters

Approval ensures that high-risk or irreversible actions are reviewed before execution. Draft runbooks must be approved before they can be run from Automation; per-step approval can be required depending on your Settings & RBAC and autonomy level.

Where to find Runbooks

  1. In the left sidebar, click Runbooks.
  2. The Runbook library lists all runbooks (draft, pending approval, and approved).

Runbook library — Title, Status, Steps, Updated

Capture from: Tenant Plane → AUTOMATE → Runbooks. Add as public/img/runbook-library.png.

Creating a runbook

Option A: Create from scratch

  1. Click New Runbook (or Create runbook).
  2. Enter a Title (e.g., “Restart Oracle listener on host”).
  3. Optionally set Applicability so the runbook is suggested only for matching services or CIs:
    • Service types — e.g., oracle, api-gateway
    • CI patterns — e.g., db-*, web-*
    • Labels — tags that match incident CIs or services
  4. Add Steps (see Adding and editing steps below).
  5. Click Save. The runbook is saved as Draft.

Option B: Generate from an incident (AI)

  1. Open an Incident that has RCA results.
  2. In the incident detail, click Run RCA if not already done.
  3. After RCA completes, click Generate Runbook (or Suggest runbook).
  4. The AI proposes steps based on root cause and evidence. Review and edit steps as needed.
  5. Save the runbook. It remains in Draft until approved.

Generate runbook from incident — RCA panel with Generate Runbook button

Capture from: Tenant Plane → Incidents → open incident → RCA. Add as public/img/runbook-generate-from-incident.png.

Adding and editing steps

Each step can be manual or automation (command/script).

  1. In the runbook editor, click Add step.
  2. For each step set:
    • Description — What this step does (e.g., “Restart Oracle listener”).
    • Action type — Manual or Command/automation.
    • Command — For automation: the actual command (e.g., systemctl restart oracle-xe or lsnrctl stop then lsnrctl start). Use variables for dynamic targets (see below).
    • Target — Where the command runs (e.g., {{target_ci}}, {{oracle_host}}). These are filled at execution time from the incident or job variables.
    • Rollback command (optional) — Command to undo this step if it fails.
    • Validation command (optional) — Command to verify success (exit 0 = success).
  3. Use Reorder to change step order. Save when done.

Using variables in steps

VariableMeaningExample
{{target_ci}}Primary CI linked to the incidentFilled when running from an incident
{{oracle_host}}Host for Oracle operationsSet in job variables or from incident CIs
{{service_name}}Affected service nameFrom incident’s affected service

When you execute a runbook from an incident, these variables are auto-populated from the incident’s CIs and services. For manual runs, provide them in the execution dialog.

Runbook step editor — Command, Target, Rollback, Validation

Capture from: Tenant Plane → Runbooks → Edit runbook → step. Add as public/img/runbook-step-editor.png.

Approval workflow

  1. When the runbook is ready, click Submit for approval (or Request approval). Status becomes Pending.
  2. Approvers (e.g., SRE lead) open Runbooks or Automation → Approvals, select the runbook, and click Approve (or Reject).
  3. After approval, status becomes Approved. The runbook is then available for execution and can be suggested by RCA.

Runbook approval — Approve/Reject

Capture from: Tenant Plane → AUTOMATE → Automation (Approvals). Add as public/img/runbook-approval.png.

Executing a runbook

From an incident

  1. Open the Incident.
  2. In the Diagnose or Remediate tab, find Suggested runbooks (from RCA) or Runbooks.
  3. Click Execute on the runbook you want to run.
  4. Review pre-filled variables (target_ci, oracle_host, service_name). Add or override if needed.
  5. Click Start execution. Monitor progress in the Automation / Jobs view.

From the Automation page

  1. Go to Automation (or Runbooks → execute from library).
  2. Select the runbook and click Run or Execute.
  3. Enter Variables (e.g., target_ci, oracle_host) for the run.
  4. Submit. The job appears in the execution list; open it for live output and status.

Scheduled execution

Runbooks can be run on a schedule via Automation → Schedules. See Scheduling & Automation.

Example: End-to-end runbook

  1. Create runbook “Restart Oracle listener.”
  2. Steps: (1) Validate connectivity to {{oracle_host}}, (2) Run lsnrctl stop then lsnrctl start on {{oracle_host}}, (3) Run sqlplus -s / as sysdba <<< "SELECT 1 FROM DUAL" to validate.
  3. Applicability: Service types: oracle; CI patterns: db-*.
  4. Submit for approval → Approve.
  5. When an incident affects an Oracle CI, RCA suggests this runbook; Execute uses the incident’s host/CI as {{oracle_host}} and {{target_ci}}.

Next steps

Using the InterfaceScheduling & Automation