TIL Terraform does not clean old providers

21 Aug 2023 - TIL, Terraform

Terraform providers can use a significant amount of space on disk over time as they are not automatically cleaned up.

While looking at the disk usage of my computer, I noticed some of the projects I am working on, like Sweep, were using a huge amount of space, like dozens of GB. These projects are mostly standard web applications, without many assets or compiled code. Using du(1) I have quickly found it was linked to my Terraform usage.

> du -h -d1 .
 29G	./terraform
 ...
 12K	./tmp
 37G	.

While inspecting the various .terraform directories, I discovered they were full of old providers, some weighting more than 200MB. It seems to be a still opened, known issue.

One official workaround is to enable the Provider Plugin Cache to allow the use of a local directory as a shared plugin cache, which then allows each distinct plugin binary to be downloaded only once. There are two drawbacks:

• Over time, as plugins are upgraded, the cache directory may grow to contain several unused versions which you must delete manually;
• The plugin cache directory is not guaranteed to be concurrency safe. The provider installer’s behavior in environments with multiple terraform init calls is undefined.

Anyway, running rm -r ./terraform/environments/*/.terraform/providers fixed the issue for now. I definitively need a cleanup task to handle such situations, like when the disks are full of docker images.