Setting Up Notifications for Unconfirmed runs after x amount of time

Overview

If you need to be notified when a stack remains in a specific state (e.g., Unconfirmed, Failed) Spacelift’s notification policy can help, however if you want to notify when a run has been in a state for a period of time, it will take some additional work outside of Spacelift.

Current Capabilities and Limitations

Spacelift’s notification policy can only respond to events generated inside Spacelift, such as state changes or run completions. However, it doesn’t track the duration of these states on its own.

Workaround: Track Stack State Duration with Webhooks and External Monitoring

1. Send a Notification Policy to a Third-Party Service
   Set up a notification policy to a webhook to send the stack state information to an external service, such as a monitoring tool or a custom API. This third-party service can track the duration that each stack remains in a particular state.

2. Monitor State Duration in the External Service
   Configure your third-party service to aggregate the state data and alert you if a stack remains in a specific state for too long. This setup enables more flexible and time-based notification.

You could also consider using the GraphQL API with a cron-triggered AWS Lambda for periodic checks of a stacks status.