Unveiling the Future of CI/CD Security: A Deep Dive into Advanced Practices
By Fabian Kammel
At ControlPlane, we’re committed to pushing the boundaries of (cloud) security, and our recent workshop on “Advanced CI/CD Security” was a testament to that commitment. In this blog post, we’ll recap the key insights from the workshop, where we explored cutting-edge practices to reinforce your Continuous Integration and Continuous Delivery (CI/CD) pipelines.
CI/CD Threat Modelling: Navigating the Security Landscape
The foundation of a robust CI/CD security strategy begins with a comprehensive threat model. In our workshop, we delved into CI/CD threat modelling, helping participants to identify potential vulnerabilities and weaknesses in their pipelines. In particular we discussed how the level-based approach of SLSA helps organisations to find and outline the next step forward.
In the workshop we delved into one technique for each threat category: source, build & dependency threats.
Git Commit Signing: Elevating Code Integrity
Code integrity is paramount in any CI/CD pipeline. We emphasized the significance of Git commit signing by taking the role of an attacker and explored ways to fake git meta data in an unsecured repository. We also discussed real life challenges to adopt git commit signing based on GPG keys: establishing trust, binding keys to identities and managing the key’s life cycle are hard problems. This is especially true for non-security experts, who are in dire need for usable security that is secure-by-default.
Sigstore is a Linux Foundation project that solves this problem through automatic key management and a transparent ledger to verify claims. The gitsign CLI was used to demonstrate how git commits are signed and verified using already available identities, e.g., GitHub logins.
Demonstrably, configuration is minimal and easy to understand, the user experience is the same as creating an unsigned commit, and verification can be done in a single line of code:
# 0. Configuration
git config commit.gpgsign true # Sign all commits
git config tag.gpgsign true # Sign all tags
git config gpg.x509.program gitsign # Use gitsign for signing
git config gpg.format x509 # gitsign expects x509 args
# Pre-select GitHub as default OIDC provider.
git config gitsign.connectorid https://github.com/login/oauth
# 1. Commit
git commit -sm "awesome change"
# 2. Verify
gitsign verify \
--certificate-identity-regex *@controlplane.io \
--certificate-oidc-issuer https://github.com/login/oauth
This provides a strong foundation to base our next pipeline steps on.
Build Attestation: Giving Bots an Identity
Where code is written and committed by humans, our build and release process is driven by a machine. This raises the question: Which identity should be used to sign our released binaries?
GitHub Actions together with cosign enables every developer to sign their release artifacts with an ephemeral key bound to the identity of the executing pipeline. This allows us to keep a high degree of automation in our release and deployment process.
jobs:
sign-image:
runs-on: ubuntu-22.04
permissions:
contents: read
packages: write
id-token: write
steps:
- name: Install Cosign CLI
uses: sigstore/cosign-installer@1fc5bd396d372bee37d608f955b336615edf79c8 # v3.2.0
- name: Log in to ghcr.io
id: docker-login
uses: docker/login-action@343f7c4344506bcbf9b4de18042ae17996df046d # v3.0.0
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Sign calculator-image
run: |
cosign sign -y \
${{ env.IMAGE_REF }}@${{ needs.build-image.outputs.digest }}
As the signature and attestation files are stored in the same place as our OCI image, the verification process is able to fetch and verify all required information, with very little context information.
cosign verify \
--certificate-identity https://github.com/controlplaneio/secure/.github/workflows/secure-service.yml@refs/tags/v0.1.0 \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
ghcr.io/controlplaneio/secure/secure-service@sha256:b2d2de552850cc7f99ebf0d0071357306159242ccf720b5b001ab694220f9547
Vulnerability Management: Shedding Light on Software Bill of Materials
As lot of words have been written on the topics of Software Bill of Materials (SBOM) generation, but rarely do we see information on how to:
- Protect the integrity of the SBOM file
- Deploy a vulnerability management process that does not break developer workflows
- Automatically scan for known vulnerabilities
In just 15 minutes practitioners were able to do just that, using the power of free and open source tools.
Syft is a an open source tool to generate an SBOM from container images or filesystems. It natively supports the generation of integrity protected SBOMs using the Sigstore toolchain:
syft attest --output spdx-json \
ghcr.io/controlplaneio/secure/secure-service@sha256:b2d2de552850cc7f99ebf0d0071357306159242ccf720b5b001ab694220f9547
Using cosign and Grype anyone is able to verify the integrity of the SBOM and scan for known vulnerabilities:
cosign verify-attestation \
ghcr.io/controlplaneio/secure/secure-service@sha256:b2d2de552850cc7f99ebf0d0071357306159242ccf720b5b001ab694220f9547 \
--certificate-identity https://github.com/controlplaneio/secure/.github/workflows/secure-service.yml@refs/tags/v0.1.0 \
--certificate-oidc-issuer https://github.com/login/oauth \
--type=spdxjson > spdx.json
# Extract SBOM from attestation and pipe into grype
cat spdx.json | jq -r '.payload | @base64d | fromjson | .predicate' | grype
Letting us and downstream users of our software know about vulnerabilities and updates to fix them.
Conclusion: Empowering Security Engineers for Tomorrow
The “Advanced CI/CD Security” workshop we ran at DevOpsCon 2023 in Munich provided a deep dive into the latest practices shaping the future of cloud security. Armed with insights into CI/CD threat modelling, git commit signing and secure SBOM management, security engineers are better equipped to safeguard their organisations against evolving cyber threats.
Both the slides and the workshop repository are available for anyone to explore in their own time.