[llvm] [github] remove more caches after downloading things (PR #133129)

[Aiden Grossman via llvm-commits]

https://github.com/boomanaiden154 commented:

Seems reasonable enough to me.

I believe there are some workflows that use this container and run `apt-get` inside of it. Theoretically they should be also running `apt-get` update just in case the containers get too out of date. I'll need to double check.

https://github.com/llvm/llvm-project/pull/133129