How to troubleshoot runs not starting

This guide will walk you through the process of troubleshooting policies using the policy workbench, with a focus on a common scenario involving a tracked run or a proposed run not starting from a PR.

For more information on the policy workbench and sampling, refer to the Policy Workbench Documentation.

Scenario: Run Not Being Triggered

Let's consider a scenario where a run is not being triggered as expected. We'll use the default push policy as an example and review it in the policy workbench.

1. Create a New Policy:

• Navigate to the Policies section from the left-hand menu.

  • Click the Create policy button.

  • Provide a name for the policy, such as "Default Push Policy".

  • Choose Push as the policy type.

  • Pick the Space which will be assessable to your stack.

• Add the Default Push Policy Code:

  • In the policy editor, paste the default push policy and make sure you include the sample line

    sample := true 

• Save the Policy:

  • Click the Save button at the bottom of the policy editor.

2. Attaching the Policy to a Stack

• Navigate to Stacks:

  • Go to the Stacks section from the left-hand menu.

• Select the Stack:

  • Click on the stack to which you want to attach the policy.

• Attach the Policy:

  • Go to the Settings tab of the stack.

  • Find the Policies section.

  • Click Attach policy.

  • Select the "Default Push Policy" from the dropdown.

  • Click Attach.

3. Repeat the Action to Test the Policy

• Open a Pull Request:

  • Go to your GitHub repository linked to the stack.

  • Make a change and open a new pull request.

• Check Spacelift for the Run:

  • Navigate back to Spacelift.

  • Go to the Runs tab of the stack to see if the run was triggered.

4. Reviewing the Policy in the Workbench

• Navigate to the Policy Workbench:

  • Go to the Policies screen in Spacelift and click on the policy you want to review.

  • Click on the Show simulation panel link on the right-hand side of the screen.

• View Sampled Events:

  • If you have no samples - verify that the sample rule is set to true.

  • If the sample rule is true and you have no sampled events, then Spacelift did not receive this event. In this case, we recommend reviewing the relevant documentation for your source code provider and check your repository has the correct settings.

• Simulate Evaluations:

  • Select an input to simulate its evaluation. You can edit both the input and the policy body in the simulation panel.

  • The dropdown will display evaluations, allowing you to see the conditions and results of each rule.

• Analyze the Conditions:

  • Click on simulate and check the tracked and proposed results - if a run was not triggered, this should show as false

  • If it shows as false then all conditions within the tracked and proposed blocks were not met.

• Understanding the Conditions:

  • The track rule requires affected to be true and the push branch to be the same branch as the Stack's branch.

  • The propose rule requires either affected or affected_pr to be true. (In Rego - an overloaded rule acts as an OR)

  • The affected rules check if any file in the affected_files list either starts with the project_root or matches the project glob pattern.

  • Similarly, affected_pr rules check the pull_request.diff list against the same conditions.

Example Analysis:

• Given the following snippets from the sampled data:

   "stack": {
      "additional_project_globs": [],
      "branch": "main",
      "project_root": "wrongprojectroot",
  

  "push": {
      "affected_files": ["projectroot/main.tf"],
      "branch": "other",

  "pull_request": {
      "diff": ["projectroot/main.tf"],

• Both affected and affected_pr will be false because the file in both lists starts does not start with the project_root.

• If you change "project_root" in the stack section to projectroot both affected and affected_pr will be true, making propose true, and a proposed run would start.

• Since the stacks branch is main and the push events branch is other the input.push.branch == input.stack.branch condition is not true meaning track rule is not true as not all conditions have been met and tracked run would not start.