Overview
A workflow runs actions in order to complete a task: diagnose a problem, fix it, or carry out a request. Workflows can ask for custom variables when they start, wait between steps, and decide whether to stop or carry on when a step fails.
Create a workflow
- Go to Automation > Automation Workflow, choose Global or a team, and click Add Workflow.
- Fill in the details:
| Field | What it means |
|---|---|
| Name, Description | What the workflow does. |
| Category | REMEDY for workflows that fix things, DIAGNOSTICS for workflows that collect information. |
| Service Related With | If the workflow fails, SQIBS opens an incident under this service. Requests for this workflow also belong to it. |
| Always consider as success | Mark the run successful whatever the actions return. Otherwise the result of the last action decides. |
| Custom Variables | Values asked for when the workflow starts. Variable Type: Text, Number or Drop-Down (with options). Use them in action arguments. |
-
Add the steps in order:
- Action: choose an action (or Show Callgoose Actions for the ready-made ones), its Profile or Runner Internal Profile, and its Arguments. Arguments are passed in the order you enter them; include quotes yourself if an argument needs them. For each action, choose whether the workflow should continue or break if it fails.
- Delay: wait a set time (seconds to days) before the next step, for example to let a service come back up.
-
Click Save.
Run it
| From | How |
|---|---|
| An incident | Open the incident and click Run Workflow. Enter the custom variables, optionally tick Resolve the incident if the workflow is successful, and run. |
| A request | Create a request and choose the workflow, or click Create Request for Workflow on the workflow. |
| The API | Copy the workflow's Public Id and send it as workflowId to the Request API. |
| Mobile app, Slack, Microsoft Teams | Use Run Workflow on the incident. |
| Automatically | Link the workflow to an API or email filter of the service (below). |
Fix known issues automatically
Many alerts have a known fix: restart a service, clear a full disk, run a health check. SQIBS can run that fix for you the moment the incident opens. This is event-driven auto remediation. For example, a DIAGNOSTICS workflow can collect logs and disk usage into the incident, and a REMEDY workflow can then restart the service.
Safe by design. SQIBS runs only workflows your own engineers wrote and approved for your environment. Nothing is guessed or generated: the same steps an engineer would run by hand, run sooner. If the workflow does not fix the issue, the engineer is paged as usual.
To set it up, in the same API filter:
- Select Workflow: choose the automation workflow to run when this filter opens an incident.
- If success then make incident resolved: turn on to resolve the incident when the workflow succeeds.
- Waiting time for escalate to escalation policies: how many minutes to give the workflow before paging the on-call engineer.
Start with low-risk workflows, such as a health check, and add fixes as you come to trust them.
Learn more:
- Prometheus: incident auto remediation, a full worked example.
- Automation workflow: build a workflow from actions.
- Run workflows where your systems are: SSH, Bash, PowerShell, Python, Ansible, kubectl, Terraform.