Overview
Terraform describes infrastructure as code and applies changes in a planned, repeatable way across cloud providers. In Callgoose SQIBS you can run Terraform as an action in an automation workflow: when an incident opens, on a schedule, or when someone submits a request.
What you can do
- Fix incidents automatically. Link a workflow to an API filter, and it runs the moment that filter opens an incident. If it succeeds, SQIBS can resolve the incident; if not, it escalates.
- Event-driven automation. Run the same workflow for every matching alert, from any monitoring tool that sends to SQIBS.
- Request automation. Let people trigger an approved workflow from a request, without giving them access to the systems.
- Run a single command. A Terraform action can run one command, such as a plan.
- Choose the version. Each Terraform profile has its own Terraform Version, so different workflows can use different versions.
Set it up
- Install a runner. The Callgoose automation runner runs your actions inside your network. Check the firewall requirements.
- Create a Terraform profile. It holds the version and the connection details. See Terraform profile.
- Create a Terraform action. It holds what to run. See Terraform action. You can keep scripts in Git: see automation with Git.
- Add the action to a workflow. See automation workflow.
- Link the workflow to an API filter in your service, so it runs when the incident opens (see below).
Fix known issues automatically
Many alerts have a known fix: restart a service, clear a full disk, run a health check. SQIBS can run that fix for you the moment the incident opens. This is event-driven auto remediation. For example: Run a Terraform plan and apply that restores a resource to its declared state after drift is detected.
Safe by design. SQIBS runs only workflows your own engineers wrote and approved for your environment. Nothing is guessed or generated: the same steps an engineer would run by hand, run sooner. If the workflow does not fix the issue, the engineer is paged as usual.
To set it up, in the same API filter:
- Select Workflow: choose the automation workflow to run when this filter opens an incident.
- If success then make incident resolved: turn on to resolve the incident when the workflow succeeds.
- Waiting time for escalate to escalation policies: how many minutes to give the workflow before paging the on-call engineer.
Start with low-risk workflows, such as a health check, and add fixes as you come to trust them.
Learn more:
- Prometheus: incident auto remediation, a full worked example.
- Automation workflow: build a workflow from actions.
- Run workflows where your systems are: SSH, Bash, PowerShell, Python, Ansible, kubectl, Terraform.