GitLab CI CD - Variables

GitLab CI/CD – Variables

by

In the world of Continuous Integration and Continuous Delivery (CI/CD), flexibility and reusability are paramount. Hardcoding values directly into your pipeline scripts can quickly lead to unmanageable and inflexible configurations. This is where GitLab CI/CD variables come into play.

Variables allow you to store and manage dynamic values that your pipeline jobs can access and use. They are crucial for making your .gitlab-ci.yml files more robust, adaptable, and secure. From defining software versions to storing sensitive credentials, variables offer a powerful way to control your CI/CD workflows.

Why Use Variables in GitLab CI/CD?

  1. Reusability: Define a value once and use it across multiple jobs or even multiple pipelines.
  2. Flexibility: Easily change pipeline behavior without modifying the core .gitlab-ci.yml file. For example, changing a deployment target.
  3. Security: Store sensitive information (API keys, passwords) securely without exposing them in your repository.
  4. Consistency: Ensure that critical parameters, like application versions or environment names, are consistent across your pipeline stages.
  5. Dynamic Behavior: Allow pipelines to adapt based on runtime conditions, such as the branch name, commit message, or user who triggered the pipeline.

Types of GitLab CI/CD Variables

GitLab CI/CD provides several types of variables, each with its own scope, purpose, and level of security.

Predefined Variables

GitLab automatically sets a large number of predefined variables for every pipeline run. These variables provide useful information about the project, pipeline, commit, branch, and user who triggered the job. They are always available in your job scripts.

Common Examples:

  • $CI_COMMIT_REF_NAME: The name of the branch or tag for which the pipeline is running.
  • $CI_COMMIT_SHA: The SHA of the commit.
  • $CI_PROJECT_DIR: The path to the project’s root directory on the runner.
  • $CI_JOB_ID: The ID of the current job.
  • $CI_PIPELINE_ID: The ID of the current pipeline.
  • $CI_PIPELINE_SOURCE: The trigger source of the pipeline (e.g., push, web, schedule).
  • $CI_ENVIRONMENT_NAME: The name of the environment for deployment jobs.
  • $CI_COMMIT_TAG: The name of the Git tag if the pipeline was triggered by a tag.

How to use them: Just reference them directly in your scripts using $VARIABLE_NAME (or %VARIABLE_NAME% on Windows Batch, or ${VARIABLE_NAME} for Bash best practice).

deploy_job:
  script:
    - echo "Deploying branch: $CI_COMMIT_REF_NAME"
    - echo "Project directory: $CI_PROJECT_DIR"
    - deploy_script --version "$CI_COMMIT_SHA"

Custom Variables Defined in .gitlab-ci.yml

You can define your own custom variables directly within your .gitlab-ci.yml file. These variables can be global (applying to all jobs) or specific to individual jobs.

Global Variables:

Defined at the top level, under the variables: keyword.

variables:
  APP_VERSION: "1.0.0"
  BUILD_DIR: "build"
  TEST_SUITE: "unit_and_integration"

stages:
  - build
  - test

build_app:
  stage: build
  script:
    - echo "Building app version $APP_VERSION into $BUILD_DIR"
    - mkdir $BUILD_DIR
    - touch $BUILD_DIR/app-$APP_VERSION.zip

run_tests:
  stage: test
  script:
    - echo "Running $TEST_SUITE tests"
    - echo "Using artifact from $BUILD_DIR"
Job-Specific Variables:

Defined within a specific job’s configuration. These override any global variables with the same name for that particular job.

variables:
  API_URL: "https://api.dev.example.com" # Global variable

deploy_staging:
  stage: deploy
  variables:
    API_URL: "https://api.stg.example.com" # Job-specific override
  script:
    - echo "Deploying to staging using API: $API_URL"

deploy_production:
  stage: deploy
  variables:
    API_URL: "https://api.prod.example.com" # Job-specific override
  script:
    - echo "Deploying to production using API: $API_URL"

CI/CD Variables (UI/API)

These are variables defined directly within the GitLab UI (or via the API). They are powerful because they are stored outside your repository, making them ideal for sensitive information or environment-specific values that shouldn’t be hardcoded in version control.

Scopes:

  • Project Variables: Available to all pipelines within a specific project.
  • Group Variables: Available to all pipelines within a specific group and its subgroups/projects.
  • Instance Variables: (Admin Area only) Available to all pipelines across the entire GitLab instance.

Key Features:

  • Type: Can be Variable (regular text) or File (where the content is saved to a temporary file on the runner, and the variable holds the path to that file).
  • Protection: You can mark variables as “protected.” Protected variables are only exposed to jobs running on protected branches or protected tags. This is critical for production credentials.
  • Masking: You can mark variables as “masked.” This hides their values in job logs, preventing accidental exposure of sensitive data. Masked variables must meet specific character requirements (e.g., minimum 8 characters, no whitespace).
  • Environments: Variables can be scoped to specific environments (e.g., production, staging), meaning they are only exposed when a job targets that environment.

How to set them: Navigate to Project > Settings > CI/CD > Variables (or Group/Admin Area).

How to use them: Once set in the UI, they are automatically available in your .gitlab-ci.yml scripts like any other variable.

deploy_prod:
  stage: deploy
  script:
    - echo "Deploying with API Key: $PROD_API_KEY" # PROD_API_KEY is a masked, protected variable in UI
    - curl -H "Authorization: Bearer $(cat $SSH_PRIVATE_KEY_FILE)" ... # SSH_PRIVATE_KEY_FILE is a masked, protected file variable
  only:
    - main # Assuming 'main' is a protected branch
  environment: production # Exposes environment-scoped variables

Variables from trigger: and schedule:

When you trigger a pipeline manually, via a webhook, or using a scheduled pipeline, you can pass custom variables directly.

  • Manual Trigger: In the GitLab UI, when running a pipeline manually, you’ll see an “Enter variables” section.
  • API Trigger: When triggering via the API, you can include variables in the request body.
  • Scheduled Pipelines: When creating a schedule, you can define specific variables for that scheduled run.

These variables have the highest precedence, overriding any variables defined in .gitlab-ci.yml or the UI.

dotenv Variables

Jobs can create .env files (specifically dotenv format) containing variables that are then passed to subsequent jobs in the same pipeline. This is useful for passing dynamically generated values between stages.

generate_version:
  stage: build
  script:
    - echo "DYNAMIC_VERSION=build-$(date +%s)" > build.env
  artifacts:
    reports:
      dotenv: build.env # Export variables from build.env

deploy_app:
  stage: deploy
  script:
    - echo "Deploying version: $DYNAMIC_VERSION" # DYNAMIC_VERSION is now available

Variable Precedence (Hierarchy)

Understanding variable precedence is crucial to avoid unexpected behavior. Variables defined later in this list will override variables defined earlier:

  1. Trigger/Schedule Variables: Variables passed when manually running, triggering via API, or via schedules. (Highest precedence)
  2. dotenv Variables: Variables loaded from artifacts:reports:dotenv.
  3. Job-specific variables in .gitlab-ci.yml: Variables defined directly within a job.
  4. CI/CD Variables (UI/API):
    • Instance variables
    • Group variables
    • Project variables (environment-scoped variables override non-environment-scoped ones if applicable)
  5. Global variables in .gitlab-ci.yml: Variables defined at the top level of the file.
  6. Predefined CI/CD Variables: GitLab’s automatic variables. (Lowest precedence, unless overridden)

Best Practices for Using Variables

  • Separate Configuration from Code: Avoid hardcoding paths, URLs, versions, or credentials directly in your scripts. Use variables instead.
  • Mask Sensitive Variables: Always mask variables containing passwords, API keys, and other secrets to prevent them from appearing in job logs.
  • Protect Sensitive Variables: Use protected variables for production environments and ensure they are only exposed to protected branches/tags and trusted runners.
  • Use Environment Scoping: For environment-specific variables (e.g., different database URLs for staging vs. production), use the environment-scoped CI/CD variables in the UI.
  • Leverage Predefined Variables: Get familiar with the extensive list of predefined variables; they offer a lot of context for your pipelines.
  • Group Related Variables: Use prefixes (e.g., APP_VERSION, DB_HOST) to logically group your variables.
  • Document Your Variables: If you have many custom variables, especially in the UI, document their purpose and expected values.
  • Prefer UI Variables for Secrets: For anything truly sensitive, use the GitLab UI CI/CD variables, benefiting from masking and protection. .gitlab-ci.yml variables are committed to your repository and are less secure for secrets.

Conclusion

GitLab CI/CD variables are a cornerstone of effective pipeline management. They empower you to create dynamic, flexible, secure, and easily maintainable CI/CD configurations. By understanding the different types of variables, their precedence, and applying best practices, you can unlock the full potential of GitLab CI/CD to automate your software development lifecycle with confidence.

Author

Debjeet Bhowmik

Experienced Cloud & DevOps Engineer with hands-on experience in AWS, GCP, Terraform, Ansible, ELK, Docker, Git, GitLab, Python, PowerShell, Shell, and theoretical knowledge on Azure, Kubernetes & Jenkins.
In my free time, I write blogs on ckdbtech.com

Leave a Comment