Optimizing GitHub Workflows: How to Auto cleanup your cache after use

Demo: Instant Cache cleanup

# ################################### 
# JOB 2 : terraform Plan
# ###################################    

Terraform_Plan:
name: 'Terraform Plan'
    runs-on: ubuntu-latest
    environment: test-labs  <--- Environment linked to git_action branch
    : write-all    : []   <---- dependency on the previous job successpermissions
needsterraform_setup_cache_load

# Use default shell     
    defaults:      
      run:
       shell: bash
   steps:
# ###################################### 
# Restore directory files from the cache
# ###################################### 
   # 1. Restore all plugin files, tf binaries,and working directory in a cache
    - name: Cache Terraform
      uses: actions/cache@v3
      with:
      path: |
          ~/.terraform.d/plugin-cache
          ./*
      key: ${{ runner.os }}-terraform-${{ github.run_id }} 
      restore-keys: |
          key: ${{ runner.os }}-terraform-${{ github.run_id }} <-- Our restore Cache Key

• Right after that, we can run additional steps like terraform plan

  # 2. Configure terraform in the new runner reusing the cache.
    - name: Config Terraform plugin cache
      run: |
        echo 'plugin_cache_dir="$HOME/.terraform.d/plugin-cache"' >~/.terraformrc
  # terraform Init not needed here. provider files already in the cache
  
  # 3. Execute terraform PLAN    
    - name: Terraform Plan
      run: |
        echo "== Reusing cached version of terraform binary 1.0.3 ==" 
        sudo cp ./terraform  /usr/local/bin/
        terraform -v      
        # example: terraform plan -input=false -no-color -out tf.plan

• Finally, we clean up our cache when done with our workflow using GitHub CLI (cache deletion requires a token with write permission)

• That’s all there is, I chose gh-action over rest API list/delete with our unique key. now let’s see logs

Workflow Execution result

• A new push to workflow test_cache_cleanup.yml to trigger our pipeline.

• The cache is visible for a short period during the execution
           

• But at the end, we can see the listed available caches and the matching cache being deleted.
   
  

Artifact vs. Caching

First Both are similar concepts to speed up the execution of CI/CD pipelines but each serve a slightly different purpose.

Caching

• Involves storing intermediate results or dependencies and commonly reused files from previous jobs runs.

• When your workflow/job runs again, it quickly retrieves the stored items instead of recreating them.

• This greatly speeds up the execution time of the pipeline, the same operations are performed only once.

• It’s ideal for components such as libraries, dependencies, or intermediate build outputs. It’s like keeping your tools on standby, so you don’t need to fetch them every time you work on a task.

• GitHub does not allow modifications once entries are pushed – cache entries are read-only records.

Artefacts

• Allow you to share data between running jobs and save them after the workflow is complete.

• An artifact is a file or collection of files produced during a workflow run.

• Example: docker image that is built early in the CI workflow but required to be pushed/run in a later stage.

  
   

Difference

• Caching is used to re-use non-changing files between jobs/workflows like sharing build dependencies.

• Artifacts are used to save files after workflow ended such as Logs, manifest, statefile, built binaries etc.

Conclusion:

• Caching optimizes workflows, but you should ensure they don’t become targets for malicious actors.

• We concentrated on instant GitHub cache cleanup today as one way to mitigate security vulnerabilities.
  

• GitHub CLI is the ideal tool that streamlines cache cleanup directly from within your workflow.

• Strategic setup and cleanup sustain a secure workflow (i.e Regularly clearing GitHub caches) 

• Next, I will implement these tips in OIDC based .

Stay tuned