# Introducing the AWS GitLab Helper URL: https://type2fun.net/introducing-the-aws-gitlab-helper ## What and why? The AWS GitLab Helper (AGH) is a tool that can be implemented in GitLab Pipelines to automatically fetch temporary credentials, secrets and parameters from AWS using [ABAC](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction_attribute-based-access-control.html). I wasn't happy with the [suggested way](https://docs.gitlab.com/ee/ci/cloud_services/aws/) of authenticating with AWS. Due to limitations of AWS OIDC provider implementation, you can only assert on the [ID Token](https://docs.gitlab.com/ee/ci/secrets/id_token_authentication.html)'s subject claim `{ "sub": "project_path:my-group/my-project:ref_type:branch:ref:feature-branch-1" }`, which means you're closely coupling security and your policies to Git References (tags or branches), which in my mind seems semantically wrong. Security should be coupled with environments and GitLab has [Protected Environments](https://docs.gitlab.com/ee/ci/environments/) — so why couldn't I use that. Well it transpires that you can. ## Say hello to AWS Cognito AWS OIDC when used on its own can only use limited claims (see [Available keys for AWS OIDC federation](https://docs.aws.amazon.com/en_en/IAM/latest/UserGuide/reference_policies_iam-condition-keys.html#condition-keys-wif)). After pouring over the docs I found [Passing session tags using AssumeRoleWithWebIdentity](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_session-tags.html#id_session-tags_adding-assume-role-idp), but alas it would only work if GitLab changed their ID Token to match AWS's schema. I eventually found [Using attributes for access control](https://docs.aws.amazon.com/cognito/latest/developerguide/attributes-for-access-control.html) in the AWS Cognito Identity Pools documentation and got very excited. Unfortunately you cannot hack with it using the `aws-cli`, so I had to use the [SDK for JavaScript](https://aws.amazon.com/sdk-for-javascript/). And it was incredibly hacky and ugly code but it worked. The magic is in Cognito mappings: ![Screen grab of Cognito attribute mappings](/assets/img/cognito-mappings.png) "Attributes for access control" is 50% of the magic sauce. This takes the claims from the GitLab ID Token (JWT), and maps them to the `AghCognito` role. However they're a bit janky to use directly. I don't want my AWS Resources having to have tags like `NamespacePath` and `ProjectPath`. I describe my infrastructure using `Domain`, `System`, `Service` and `Environment` tags. I thought, if I assume a 2nd role I could rename them — assuming a 2nd role opens up the environment for abuse (changing the tags to a claim you're not entitled to). However after too much coffee and some creative thinking I worked out a **safe** Trust Policy that would achieve what I wanted (it also has some extra features too). Introducing the wild Trust Policy for `AghAbac` — the other 50%: ```json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam:::role/agh/AghCognito" }, "Action": [ "sts:AssumeRole", "sts:TagSession" ], "Condition": { "StringEquals": { "aws:PrincipalTag/Environment": "${aws:RequestTag/Environment}", "aws:PrincipalTag/ProjectPath": "${aws:PrincipalTag/NamespacePath}/${aws:RequestTag/Service}", "aws:PrincipalTag/UserAccessLevel": [ "developer", "maintainer", "owner" ], "aws:PrincipalTag/EnvironmentProtected": "${aws:RequestTag/EnvironmentProtected}" }, "Null": { "aws:TagKeys": "false" }, "StringLike": { "aws:PrincipalTag/NamespacePath": "*/${aws:RequestTag/Domain}/${aws:RequestTag/System}" }, "ForAllValues:StringEquals": { "aws:TagKeys": [ "Domain", "System", "Service", "Environment", "EnvironmentProtected", "UserAccessLevel" ] } } } ] } ``` Let me explain it: | Desired Tag | Assertion | Explanation | | ---------------------- | ---------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ | | `Domain` and `System` | `"${aws:PrincipalTag/NamespacePath}" like "*/${aws:RequestTag/Domain}/${aws:RequestTag/System}"` | The `NamespacePath` contains `root-group/subgroup1/subgroup2` and my GitLab group hierarchy is `root-group/domain/system`. | | `Service` | `"${aws:PrincipalTag/NamespacePath}/${aws:RequestTag/Service}" == "${aws:PrincipalTag/ProjectPath}"` | The `ProjectPath` contains `root-group/subgroup1/subgroup2/service` and my GitLab service hierarchy is `root-group/domain/system/service`. | | `Environment` | `"${aws:RequestTag/Environment}" == "${aws:PrincipalTag/Environment}"` | | | `EnvironmentProtected` | `"${aws:RequestTag/EnvironmentProtected}" == "${aws:PrincipalTag/EnvironmentProtected}"` | I've added this as in my other Policies I assert that `"Environment" = "production"` also has `"EnvironmentProtected" = "true"`. | | `UserAccessLevel` | `"${aws:PrincipalTag/UserAccessLevel}" in ["owner", "maintainer", "owner"]` | This gives me the option to block `developers` from ever interacting with some resources. | So now I have a well-configured ABAC role, what can I do? In my `.gitlab-ci.yml` file I can add the following code: ```yaml include: - project: msyea-sa/aws-gitlab-helper file: /templates/AwsGitLabHelper.gitlab-ci.yml variables: AGH_CREDENTIAL_DEFAULT: "arn:aws:iam::123456789012:role/service-role/shared-system-role" AGH_CREDENTIAL_PROFILE1: "arn:aws:iam::123456789012:role/service-role/service-role" AGH_SECRET_SLACK_TOKEN: "arn:aws:secretsmanager:us-east-1:123456789012:secret:slack-token-V6d7a8" AGH_SECRET_TEST_DB: "arn:aws:secretsmanager:us-west-2:123456789012:secret:db-connection-string-F1r2t3" AGH_PARAMETER_MAX_RETRY: "arn:aws:ssm:us-west-2:123456789012:parameter/max-retry-attempts" test: extends: [.aws-gitlab-helper] script: - aws --profile profile1 sts get-caller-identity - echo "Max retry: ${MAX_RETRY}" # do not echo secrets — they will not be masked ``` The template executes in the GitLab Helper Container (`hook:pre_get_sources_script`). This is hugely advantageous as it doesn't interfere with job scripts (like `before_script` might) and `${GITLAB_ENV}` plays nicely. As it's in the GitLab Helper Container it knows `curl` is available and with `uname -s` it can download and execute the correct version from the GitLab Package Registry. The app is written in JavaScript/node and statically compiled for Linux and macOS. This makes the download quick with no dependencies. The AWS GitLab Helper parses the environment variables prefixed with `AGH_`, and, when necessary, executes `AssumeRole`, `GetSecretValue` and `GetParameter` commands and saves the responses. To make them available to the job it does something like: ```bash export AWS_SHARED_CREDENTIALS_FILE="${RUNNER_TEMP_PROJECT_DIR}/aws-credentials" echo "credentials" >> ${AWS_SHARED_CREDENTIALS_FILE} echo "secrets" >> ${GITLAB_ENV} ``` If you haven't used `${GITLAB_ENV}` before see [Pass an environment variable from the `script` section to another section in the same job](https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-from-the-script-section-to-another-section-in-the-same-job). It's much less janky than `export $(xargs < .env)`. Your AWS credentials file is populated outside the `${CI_PROJECT_DIR}`, so that it cannot be accidentally saved as an artifact or committed. The secrets and parameters are also made available as environment variables. ```mermaid sequenceDiagram participant Job as GitLab Job box transparent GitLab Helper Container hook participant AGH as AWS GitLab Helper end box transparent AWS participant Cognito participant STS participant SM as Secrets Manager participant PS as Parameter Store end Job->>+AGH: Implements AGH Template AGH->>AGH: Configures ID Token AGH->>AGH: Downloads AGH Binary and executes it AGH->>+Cognito: Authenticates Cognito->>Cognito: Maps claims to Principal Tags Cognito->>-AGH: Returns temporary credentials (AghCognito Role) AGH->>+STS: Assume AghAbac Role with AGH Tags STS->>-AGH: Returns temporary credentials (AghAbac Role) loop Fetch credentials for additional roles AGH->>+STS: Assume other Role with AGH Tags STS->>-AGH: Returns temporary credentials AGH->>Job: Write credentials to AWS_SHARED_CREDENTIALS_FILE end loop Fetch secrets AGH->>+SM: GetSecret SM->>-AGH: Returns secret AGH->>Job: Writes secret to GITLAB_ENV end loop Fetch parameters AGH->>+PS: GetParameter PS->>-AGH: Returns parameter AGH->>Job: Write parameter to GITLAB_ENV end AGH-xJob: AGH Binary exits and Helper Container completes alt GitLab Build Container Job->>Job: Runs your Job scripts end ``` And that is it. AGH is now available for you to use. I've been using it in production for a few months but I will improve the documentation and automated tests before I make it Generally Available. Note that the AWS GitLab Helper is available under BSL → MIT license. If you have 25 or fewer employees or less than £1 million in revenue you can use it in production for free. If you're larger than that then you will need a Commercial License or wait 4 years. ## Links | Component | Location | | ---------------- | --------------------------------------------------------------------- | | Template | | | Application | | | Terraform Module | |