Cleaning Up After Incomplete Module Test Runs

Issue: Incomplete Destroy Phase Leaves Resources Behind

If a module’s destroy phase does not complete successfully, leftover resources may need manual cleanup. This guide provides a strategy for handling such situations using Spacelift hooks to assist with cleanup.

Solution: Use the after_run Hook for State Dump

To facilitate cleanup when a run fails, use the after_run hook to dump the remaining state to logs. This approach allows you to capture the state for further analysis and manual cleanup if necessary.

Setting Up the after_run Hook

1. Configure after_run Hook
   The after_run hook runs after the entire process completes. You can set this hook to log the remaining state if a failure occurs, enabling you to reload the state separately and execute a terraform destroy command as needed.

2. Use the TF_VAR_spacelift_final_run_state Variable
   The after_run hook has access to a special environment variable, TF_VAR_spacelift_final_run_state, which indicates the final state of the run. You can configure the hook to dump the state only if the run fails.

   You could use script to check the final state of the run and, if it’s not in a finished state, dumps the state to a file (e.g., failed_run_state.tfstate) for later use.