Context

When running Terraform/OpenTofu operations in Spacelift, you may experience slower execution times due to various factors such as module downloads, provider installations, and workspace preparation. This article covers different methods to optimize and speed up your Terraform/OpenTofu runs.

Answer

There are several ways to improve the performance of your runs in Spacelift:

1. Provider Caching

You can cache providers to avoid downloading them on every run:

For instance-based workers:

1. Create a cache directory in your startup script:

   mkdir -p /var/cache/terraform/
   chown 1983:1983 /var/cache/terraform/

2. Configure the mount points (note: do not include trailing slashes):

   export SPACELIFT_WORKER_WO_EXTRA_MOUNTS="/var/cache/terraform:/var/cache/terraform"
   export SPACELIFT_WORKER_RO_EXTRA_MOUNTS="/var/cache/terraform:/var/cache/terraform"

3. Set the TF_PLUGIN_CACHE_DIR environment variable to point to this directory

2. Increase the parallelism

Increase the -parallelism flag by adding the following environment variable in your stack environment settings
TF_CLI_ARGS_plan="-parallelism=X"

3. Custom Runner Images

Create a custom Docker image that includes:

• Pre-installed Terraform/OpenTofu

• Commonly used providers

• Pre-downloaded module archives

4. For Kubernetes Workers

If using Kubernetes workers, you can set up a binaries cache volume to cache Terraform modules and providers. Refer to the binaries cache volume documentation.

5. Git Checkout Optimization

Enable Git checkout support to potentially speed up the workspace preparation phase, especially for monorepos. See the Git checkout documentation for setup instructions.

6. Monitoring Performance

To identify bottlenecks in your runs, enable timestamps in your logs by adding the feature flag:

feature:enable_log_timestamps

You can view detailed timing information by disabling the simplified view in the run logs: