Overview

kubectl is the command-line tool for Kubernetes. With it a workflow can restart deployments, scale workloads or collect pod logs. In Callgoose SQIBS you can run kubectl commands as an action in an automation workflow: when an incident opens, on a schedule, or when someone submits a request.

You need
A SQIBS account with automation, and a system that kubectl (Kubernetes) can reach
Where it runs
On a Callgoose automation runner inside your network
Result
kubectl (Kubernetes) actions in your workflows

What you can do

  • Fix incidents automatically. Link a workflow to an API filter, and it runs the moment that filter opens an incident. If it succeeds, SQIBS can resolve the incident; if not, it escalates.
  • Event-driven automation. Run the same workflow for every matching alert, from any monitoring tool that sends to SQIBS.
  • Request automation. Let people trigger an approved workflow from a request, without giving them access to the systems.
  • Run a single command. A Kubernetes action can run one kubectl command.
  • Choose the version. Each kubectl (Kubernetes) profile has its own Kubernetes Version, so different workflows can use different versions.

Set it up

  1. Install a runner. The Callgoose automation runner runs your actions inside your network. Check the firewall requirements.
  2. Create a kubectl (Kubernetes) profile. It holds the version and the connection details. See kubectl (Kubernetes) profile.
  3. Create a kubectl (Kubernetes) action. It holds what to run. See kubectl (Kubernetes) action. You can keep scripts in Git: see automation with Git.
  4. Add the action to a workflow. See automation workflow.
  5. Link the workflow to an API filter in your service, so it runs when the incident opens (see below).

Fix known issues automatically

Many alerts have a known fix: restart a service, clear a full disk, run a health check. SQIBS can run that fix for you the moment the incident opens. This is event-driven auto remediation. For example: Run kubectl rollout restart on a deployment when its pods keep failing health checks.

Incident opens
Workflow starts
SQIBS runs the automation workflow linked to the filter.
Fixed
Incident resolved
The issue is fixed before anyone has to step in.
Not fixed
Escalated to the engineer
After the waiting time you set, the escalation policy pages on-call as usual.

Safe by design. SQIBS runs only workflows your own engineers wrote and approved for your environment. Nothing is guessed or generated: the same steps an engineer would run by hand, run sooner. If the workflow does not fix the issue, the engineer is paged as usual.

To set it up, in the same API filter:

  1. Select Workflow: choose the automation workflow to run when this filter opens an incident.
  2. If success then make incident resolved: turn on to resolve the incident when the workflow succeeds.
  3. Waiting time for escalate to escalation policies: how many minutes to give the workflow before paging the on-call engineer.

Start with low-risk workflows, such as a health check, and add fixes as you come to trust them.

Learn more: