How do I fix "Could not read pre-apply state" errors after provider upgrades?

Context

After upgrading provider versions (particularly AWS provider), users may encounter errors during the apply phase where state file changes are not visible in the UI. This commonly manifests with errors like:

This run has succeeded however entity changes for this run will not visible in the UI due to an error parsing the state file in the initialization phase.
Could not read pre-apply state: couldn't decode state: EOF

This can also lead to related issues where stack outputs aren't being properly recognized for dependency references.

Answer

To resolve this issue, you can refresh the state using one of these methods:

1. Run a refresh-only task in Spacelift using the command:

   terraform refresh

   or

   tofu refresh

2. Alternatively, you can trigger a new Spacelift run

This issue occurs because the terraform show -json command (which Spacelift uses to generate the spacelift.state.before.json) fails on the first run with a new provider version. When this happens, there is no spacelift.state.before.json file, preventing Spacelift from updating the data behind the resource view.

Running a refresh-only task will update the state properly and resolve both the state parsing errors and any related dependency reference issues.