Log Timeout Issue in Runs - No logs received for 7m0s, killing the process group

Issue: Logs Timing Out

Error Message: No logs received for 7m0s, killing the process group

With Spacelift runs, you may encounter log timeout errors that halt execution due to an absence of logs for a specified duration.

Troubleshooting Steps

To identify the cause of the log timeout, follow these steps:

Enable Debug Logging for Terraform Stacks
If you're using a Terraform stack, you can temporarily add the environment variable TF_LOG with the value's DEBUG/TRACE. This setting will increase the log verbosity and may reveal clues about the underlying issue.
How to set TF_LOG:
- Go to your stack's Environment tab, add two new variable's with:
  - Name: TF_LOG
  - Value: DEBUG

This is especially helpful for identifying potential issues within the Terraform stack processes.

Security Note on Debug Logging
Setting TF_LOG to DEBUG or TRACE can expose sensitive information in your run logs - including provider credentials, auth tokens, resource attribute values (even ones marked sensitive = true in your Terraform config), and raw HTTP request/response data. This is a known limitation reported in HashiCorp GitHub issues: Terraform's sensitive flag governs standard CLI/plan output, but debug-level logs have been observed printing sensitive values in plaintext for certain providers and backends.

Before enabling TF_LOG, take these precautions:

Enable Well Known Secret Masking on the stack. This automatically redacts common secret patterns (AWS keys, GitHub tokens, JWTs, private keys, etc.) from logs.

Turn TF_LOG off as soon as you've captured what you need - don't leave verbose logging enabled long-term.
If sharing logs with Spacelift support or elsewhere, review them first for anything that may have slipped through unmasked

Check Private Worker Configuration
Set SPACELIFT_LAUNCHER_LOGS_TIMEOUT as an environment variable on the private worker (launcher), for example in your Helm values or Kubernetes manifest, not in the stack's environment variables. Restart the worker pods to apply it. See Worker pool configuration options.

Common Causes for Log Timeouts

Log timeouts can occur for various reasons, including:

Rate Limiting
External systems (such as APIs or repositories) may impose rate limits, which can delay logging output and trigger timeouts.
Firewall or Networking Connectivity Issues
Network issues, such as firewall rules or DNS resolution problems, can prevent logs from reaching Spacelift within the required timeframe. Ensure that the worker has all necessary network permissions correctly configured.
Run-Specific Delays
Long-running tasks or stages that do not produce output for extended periods may also trigger timeouts. Consider modifying task behavior to periodically output logs or extending the SPACELIFT_LAUNCHER_LOGS_TIMEOUT to accommodate such cases.

Next Steps

If the above steps do not resolve your log timeout issues, consider reaching out to Spacelift support with the following details:

Stack ID or Name
Any relevant environment variable settings (e.g., TF_LOG, SPACELIFT_LAUNCHER_LOGS_TIMEOUT)
Description of the run being ran

These details will help support analyze the problem more effectively.