GitLab CI CD - Variables

GitLab CI/CD – Variables

by

In today’s blog post, we will learn variables in GitLab CI/CD pipeline. Specifically, what are variables? How to define variables in .gitlab-ci.yml, different types of variables in GitLab with examples. So without ant further delay, let us get started.

What are Variables in GitLab CI/CD?

Variables allow you to store and manage dynamic values that your pipeline jobs can access and use. They are important for making your .gitlab-ci.yml files more dynamic, adaptable, and secure. From defining software versions to storing sensitive credentials, variables offer a powerful way to control your CI/CD workflows.

Why Use Variables in GitLab CI/CD?

  1. Reusability: Define a value once and use it across multiple jobs or even multiple pipelines.
  2. Flexibility: Easily change pipeline behavior without modifying the core .gitlab-ci.yml file. For example, changing a deployment target.
  3. Security: Store sensitive information (API keys, passwords) securely without exposing them in your repository.
  4. Consistency: Ensure that critical parameters, like application versions or environment names, are consistent across your pipeline stages.
  5. Dynamic Behavior: Allow pipelines to adapt based on runtime conditions, such as the branch name, commit message, or user who triggered the pipeline.

Types of GitLab CI/CD Variables

GitLab CI/CD provides several types of variables, each with its own scope, purpose, and level of security.

Predefined Variables

GitLab automatically sets a large number of predefined variables for every pipeline run. These variables provide useful information about the project, pipeline, commit, branch, and user who triggered the job. They are always available in your job scripts.

Common Examples:

  • $CI_COMMIT_REF_NAME: The name of the branch or tag for which the pipeline is running.
  • $CI_COMMIT_SHA: The SHA of the commit.
  • $CI_PROJECT_DIR: The path to the project’s root directory on the runner.
  • $CI_JOB_ID: The ID of the current job.
  • $CI_PIPELINE_ID: The ID of the current pipeline.
  • $CI_PIPELINE_SOURCE: The trigger source of the pipeline (e.g., push, web, schedule).
  • $CI_ENVIRONMENT_NAME: The name of the environment for deployment jobs.
  • $CI_COMMIT_TAG: The name of the Git tag if the pipeline was triggered by a tag.

How to use them: Just reference them directly in your scripts using $VARIABLE_NAME (or %VARIABLE_NAME% on Windows Batch, or ${VARIABLE_NAME} for Bash best practice).

deploy_job:
  script:
    - echo "Deploying branch: $CI_COMMIT_REF_NAME"
    - echo "Project directory: $CI_PROJECT_DIR"
    - deploy_script --version "$CI_COMMIT_SHA"

Custom Variables Defined in .gitlab-ci.yml

You can define your own custom variables directly within your .gitlab-ci.yml file. These variables can be global (applying to all jobs) or specific to individual jobs.

Global Variables:

Defined at the top level, under the variables: keyword.

variables:
  APP_VERSION: "1.0.0"
  BUILD_DIR: "build"
  TEST_SUITE: "unit_and_integration"

stages:
  - build
  - test

build_app:
  stage: build
  script:
    - echo "Building app version $APP_VERSION into $BUILD_DIR"
    - mkdir $BUILD_DIR
    - touch $BUILD_DIR/app-$APP_VERSION.zip

run_tests:
  stage: test
  script:
    - echo "Running $TEST_SUITE tests"
    - echo "Using artifact from $BUILD_DIR"
Job-Specific Variables:

Defined within a specific job’s configuration. These override any global variables with the same name for that particular job.

variables:
  API_URL: "https://api.dev.example.com" # Global variable

deploy_staging:
  stage: deploy
  variables:
    API_URL: "https://api.stg.example.com" # Job-specific override
  script:
    - echo "Deploying to staging using API: $API_URL"

deploy_production:
  stage: deploy
  variables:
    API_URL: "https://api.prod.example.com" # Job-specific override
  script:
    - echo "Deploying to production using API: $API_URL"

CI/CD Variables (UI/API)

These are variables defined directly within the GitLab UI (or via the API). They are powerful because they are stored outside your repository, making them ideal for sensitive information or environment-specific values that should not be hardcoded in version control.

Scopes:

  • Project Variables: Available to all pipelines within a specific project.
  • Group Variables: Available to all pipelines within a specific group and its subgroups/projects.
  • Instance Variables: (Admin Area only) Available to all pipelines across the entire GitLab instance.

Key Features:

  • Type: Can be Variable (regular text) or File (where the content is saved to a temporary file on the runner, and the variable holds the path to that file).
  • Protection: You can mark variables as “protected.” Protected variables are only exposed to jobs running on protected branches or protected tags. This is critical for production credentials.
  • Masking: You can mark variables as “masked.” This hides their values in job logs, preventing accidental exposure of sensitive data. Masked variables must meet specific character requirements (e.g., minimum 8 characters, no whitespace).
  • Environments: Variables can be scoped to specific environments (e.g., production, staging), meaning they are only exposed when a job targets that environment.

How to set them: Navigate to Project > Settings > CI/CD > Variables (or Group/Admin Area).

How to use them: Once set in the UI, they are automatically available in your .gitlab-ci.yml scripts like any other variable.

deploy_prod:
  stage: deploy
  script:
    - echo "Deploying with API Key: $PROD_API_KEY" # PROD_API_KEY is a masked, protected variable in UI
    - curl -H "Authorization: Bearer $(cat $SSH_PRIVATE_KEY_FILE)" ... # SSH_PRIVATE_KEY_FILE is a masked, protected file variable
  only:
    - main # Assuming 'main' is a protected branch
  environment: production # Exposes environment-scoped variables

Variables from trigger: and schedule:

When you trigger a pipeline manually, via a webhook, or using a scheduled pipeline, you can pass custom variables directly.

  • Manual Trigger: In the GitLab UI, when running a pipeline manually, you will see an “Enter variables” section.
  • API Trigger: When triggering via the API, you can include variables in the request body.
  • Scheduled Pipelines: When creating a schedule, you can define specific variables for that scheduled run.

These variables have the highest precedence, overriding any variables defined in .gitlab-ci.yml or the UI.

dotenv Variables

Jobs can create .env files (specifically dotenv format) containing variables that are then passed to subsequent jobs in the same pipeline. This is useful for passing dynamically generated values between stages.

generate_version:
  stage: build
  script:
    - echo "DYNAMIC_VERSION=build-$(date +%s)" > build.env
  artifacts:
    reports:
      dotenv: build.env # Export variables from build.env

deploy_app:
  stage: deploy
  script:
    - echo "Deploying version: $DYNAMIC_VERSION" # DYNAMIC_VERSION is now available

Variable Precedence (Hierarchy)

Understanding variable precedence is crucial to avoid unexpected behavior. Variables defined later in this list will override variables defined earlier:

  1. Trigger/Schedule Variables: Variables passed when manually running, triggering via API, or via schedules. (Highest precedence)
  2. dotenv Variables: Variables loaded from artifacts:reports:dotenv.
  3. Job-specific variables in .gitlab-ci.yml: Variables defined directly within a job.
  4. CI/CD Variables (UI/API):
    • Instance variables
    • Group variables
    • Project variables (environment-scoped variables override non-environment-scoped ones if applicable)
  5. Global variables in .gitlab-ci.yml: Variables defined at the top level of the file.
  6. Predefined CI/CD Variables: GitLab’s automatic variables. (Lowest precedence, unless overridden)

Best Practices for Using Variables

  • Separate Configuration from Code: Avoid hardcoding paths, URLs, versions, or credentials directly in your scripts. Use variables instead.
  • Mask Sensitive Variables: Always mask variables containing passwords, API keys, and other secrets to prevent them from appearing in job logs.
  • Protect Sensitive Variables: Use protected variables for production environments and ensure they are only exposed to protected branches/tags and trusted runners.
  • Use Environment Scoping: For environment-specific variables (e.g., different database URLs for staging vs. production), use the environment-scoped CI/CD variables in the UI.
  • Leverage Predefined Variables: Get familiar with the extensive list of predefined variables; they offer a lot of context for your pipelines.
  • Group Related Variables: Use prefixes (e.g., APP_VERSION, DB_HOST) to logically group your variables.
  • Document Your Variables: If you have many custom variables, especially in the UI, document their purpose and expected values.
  • Prefer UI Variables for Secrets: For anything truly sensitive, use the GitLab UI CI/CD variables, benefiting from masking and protection. .gitlab-ci.yml variables are committed to your repository and are less secure for secrets.

FAQs – Variables

What are variables in GitLab CI/CD?
Variables in GitLab CI/CD are key-value pairs used to store and reuse values across your pipeline configuration. They help simplify scripts, inject credentials, set environment flags, and control pipeline logic dynamically.

How do I define variables in .gitlab-ci.yml?
You can define variables globally or at the job level:

Global variable (applies to all jobs):

variables:
  APP_ENV: production
  DEPLOY_DIR: /var/www/app

Job-level variable (applies only to that job):

deploy:
  variables:
    DEPLOY_ENV: staging
  script:
    - echo "Deploying to $DEPLOY_ENV"

What is the difference between predefined and custom variables in GitLab?

  • Predefined variables are built into GitLab, like $CI_COMMIT_BRANCH, $CI_JOB_ID, $CI_PROJECT_NAME.
  • Custom variables are user-defined in .gitlab-ci.yml or GitLab UI.

Predefined variables are automatically available in every job, while custom ones must be explicitly declared.

Can I override global variables in a job?
Yes. Variables defined at the job level override global variables:

variables:
  APP_ENV: production

test:
  variables:
    APP_ENV: test
  script:
    - echo "Environment: $APP_ENV"

Here, APP_ENV will be test for the test job only.

Can I define secret variables in GitLab?
Yes. Secret variables (like tokens, passwords, API keys) should be stored in:

GitLab → Settings → CI/CD → Variables

These are masked in job logs and not stored in the .gitlab-ci.yml file, making them safe for sensitive data.

How do I use variables in job scripts?
Variables can be referenced using $VAR_NAME syntax:

variables:
  GREETING: "Hello"

job:
  script:
    - echo "$GREETING, World!"

This will output: Hello, World!

What is the scope of variables in GitLab CI/CD?
Variables can be defined at different scopes:

  • Global: In the top-level variables: section (applies to all jobs).
  • Job-level: Inside a specific job.
  • Environment/Group/Instance level: Set via GitLab UI (available to pipelines based on visibility).
  • Trigger-time: Passed when triggering a pipeline via API.

Can I use complex values or multi-line strings in variables?
Yes. Use double quotes or the | (YAML block scalar) for multi-line variables:

variables:
  MULTILINE: |
    Line one
    Line two
    Line three

In the script:

  script:
    - echo "$MULTILINE"

What happens if a variable is not defined?
If you reference an undefined variable, it resolves to an empty string (""). This can lead to unexpected behavior in scripts, so it is good practice to validate or default values when needed:

echo "${MY_VAR:-default_value}"

How do I use GitLab predefined CI/CD variables?
You can use any of GitLab’s built-in variables in your jobs:

print-info:
  script:
    - echo "Running job $CI_JOB_NAME on branch $CI_COMMIT_BRANCH"

Some useful predefined variables include:

  • $CI_COMMIT_BRANCH
  • $CI_JOB_STAGE
  • $CI_PIPELINE_ID
  • $CI_PROJECT_DIR
  • $CI_REGISTRY_IMAGE

Full list: GitLab Predefined Variables

Can I use variables to control pipeline behavior?
Yes. You can use variables with rules, only, or script to enable dynamic pipelines:

test:
  rules:
    - if: '$RUN_TESTS == "true"'
  script:
    - npm test

Here, the job runs only when RUN_TESTS is set to "true".

Are GitLab UI-defined variables available to all pipelines?
Yes, UI-defined variables (via GitLab Project or Group settings) are automatically available in all pipelines for that project or group. You can also mark them as:

  • Protected: Only available in pipelines running on protected branches/tags.
  • Masked: Hidden in job logs.

How do I debug or print all variables in a job?
You can output all available variables using:

debug-vars:
  script:
    - printenv | sort

Be cautious when printing variables, as this may reveal sensitive values. Avoid this in production pipelines.

Author

Debjeet Bhowmik

Experienced Cloud & DevOps Engineer with hands-on experience in AWS, GCP, Terraform, Ansible, ELK, Docker, Git, GitLab, Python, PowerShell, Shell, and theoretical knowledge on Azure, Kubernetes & Jenkins. In my free time, I write blogs on ckdbtech.com

Leave a Comment