In GitLab CI/CD, the traditional way pipelines execute is in stages: all jobs within a stage run in parallel, and all jobs in a stage must complete successfully before the next stage begins. While this sequential stage execution works well for many common workflows (build -> test -> deploy), it can become a bottleneck for more complex scenarios or highly optimized pipelines.
This is where the needs
keyword is useful. The needs
keyword allows you to break free from the strict stage-by-stage execution flow. It enables you to define explicit dependencies between jobs, creating a Directed Acyclic Graph (DAG) of your pipeline. This means a job can start as soon as its specific dependencies are met, regardless of what stage those dependencies belong to.
The Limitation of Stages (and Why needs Helps)
Consider a common pipeline:
stages:
- build
- test
- deploy
build_app:
stage: build
script: echo "Building app..."
unit_tests:
stage: test
script: echo "Running unit tests..."
integration_tests:
stage: test
script: echo "Running integration tests..."
deploy_staging:
stage: deploy
script: echo "Deploying to staging..."
In this setup:
build_app
runs.- Once
build_app
is finished,unit_tests
andintegration_tests
run in parallel. - Only when both
unit_tests
andintegration_tests
are completed (even ifunit_tests
finishes much earlier), doesdeploy_staging
begin.
What if deploy_staging
only requires the output of build_app
and unit_tests
, but not integration_tests
? With stages alone, deploy_staging
still waits for integration_tests
to finish. This creates an unnecessary delay.
This is where needs
comes into picture.
How needs Works
The needs
keyword is defined within a job and takes a list of other job names that must complete successfully before the current job can start.
Syntax:
job_name:
stage: <any_stage_name> # Stage still required, but primarily for visualization/grouping
needs:
- job_dependency_1
- job_dependency_2
script:
- # ... commands
When a job has needs
, it will:
- Start as soon as its
needs
are met: It does not wait for its entire stage to start, nor for previous stages to finish (unless those are implicitly part of itsneeds
graph). - Pass artifacts automatically: If the jobs listed in
needs
produce artifacts, those artifacts are automatically downloaded and made available to the needing job. This is a crucial benefit over manual artifact passing. - Visualized as a DAG: In the GitLab UI, pipelines using
needs
are beautifully visualized as a directed acyclic graph, showing the precise flow of dependencies.
Practical Examples of needs
Let us revisit our example with needs
:
stages:
- build
- test
- deploy
- qa # New stage for QA tasks
build_app:
stage: build
script: echo "Building app and creating artifact..."
artifacts:
paths:
- build/app.jar # Output artifact
unit_tests:
stage: test
script: echo "Running unit tests..."
needs: ["build_app"] # Requires build_app to finish
artifacts:
paths:
- test_results/unit.xml # Unit test results
integration_tests:
stage: test
script: echo "Running integration tests (takes longer)..."
needs: ["build_app"] # Also requires build_app
artifacts:
paths:
- test_results/integration.xml # Integration test results
deploy_staging:
stage: deploy
script: echo "Deploying app and unit test results to staging..."
needs: ["build_app", "unit_tests"] # Only needs build_app and unit_tests
# No need to explicitly download artifacts, they are available
qa_smoke_tests:
stage: qa
script: echo "Running QA smoke tests on deployed staging environment..."
needs: ["deploy_staging"] # Needs staging deployment to be done
full_regression_tests:
stage: qa
script: echo "Running full regression tests (requires all tests to pass first)..."
needs: ["integration_tests", "unit_tests"] # Needs all tests to complete
What happens here with needs
:
build_app
starts immediately.- As soon as
build_app
finishes:unit_tests
starts.integration_tests
starts.
- As soon as
unit_tests
finishes (it might finish beforeintegration_tests
):deploy_staging
can start, becausebuild_app
is also done. It does not wait forintegration_tests
.
qa_smoke_tests
waits fordeploy_staging
.full_regression_tests
waits for bothunit_tests
andintegration_tests
to complete.
This significantly speeds up the pipeline by allowing deploy_staging
to proceed concurrently with the (potentially longer) integration_tests
.
Advanced needs Features: Fine-Grained Control
The needs
keyword offers several additional attributes to provide even more specific control over job dependencies and artifact handling.
needs:artifacts (or artifacts: false
)
Purpose: By default, when a job needs
another job, all artifacts from the needed job are downloaded. Use artifacts: false
within a needs
entry if the needing job does not require the artifacts from a specific dependency, saving time and disk space.
Syntax:
job_name:
needs:
- job: <needed_job_name>
artifacts: false
Example:
lint_code:
stage: lint
script: echo "Linting code..."
artifacts:
paths:
- lint_report.txt # Produces a report, but not needed by build
build_app:
stage: build
script: echo "Building application..."
needs:
- job: lint_code
artifacts: false # build_app only needs lint_code to pass, not its report
Explanation: build_app
will wait for lint_code
to complete successfully, but it would not download lint_report.txt
, making the build_app
job start faster if that report is large.
needs:optional
Purpose: If a job listed in needs
is set with allow_failure: true
, you can mark it as optional: true
within the needs
definition. The dependent job will then run even if the optional needed job fails. If the optional job passes, the dependent job also runs.
Syntax:
job_name:
needs:
- job: <needed_job_name>
optional: true
Example:
security_scan:
stage: security
script: echo "Running optional security scan..."
allow_failure: true # This job would not fail the pipeline if it finds issues
artifacts:
paths: [security_report.json]
deploy_review_app:
stage: deploy
script: echo "Deploying review app, even if security scan has warnings..."
needs:
- job: security_scan
optional: true # This job runs even if security_scan fails due to allow_failure: true
artifacts: true # Still get the security report if it exists
Explanation: deploy_review_app
will start once security_scan
completes, regardless of whether security_scan
passed or failed (as long as security_scan
had allow_failure: true
).
needs:project and needs:pipeline:job (Cross-Project Pipeline Dependencies)
Purpose: These advanced features allow a job in one pipeline to depend on a job in a different project’s pipeline. This is crucial for microservice architectures or monorepos where components are built and tested in separate projects.
needs:project
: Specifies the project path where the dependent job resides.
needs:pipeline:job
: Specifies the exact job name in the dependent project’s pipeline.
Syntax:
job_name:
needs:
- project: <group>/<subgroup>/<project_path>
ref: <ref_name> # Optional: specific branch/tag/commit in the other project
job: <job_name_in_other_project>
artifacts: true/false # Optional: whether to download artifacts from the other job
Example:
deploy_frontend:
stage: deploy
script: echo "Deploying frontend using artifact from backend build..."
needs:
- project: my-group/backend-service # Dependent on a job in 'backend-service' project
ref: main # Depend on the 'main' branch of the backend
job: build_backend_api # Specifically on the 'build_backend_api' job
artifacts: true # Download its artifacts
Explanation: deploy_frontend
will only start after the build_backend_api
job in the my-group/backend-service
project’s pipeline (specifically from its main
branch) has successfully completed. This automatically fetches the build_backend_api
‘s artifacts.
needs:parallel:matrix
Purpose: When using parallel:matrix
to create multiple jobs from a single job definition, needs
can specifically target these generated jobs. This ensures dependencies are correctly mapped to each matrix combination.
Syntax:
needs:
- job: <job_name>
artifacts: true/false
optional: true/false
# For parallel:matrix jobs, specify the matrix variables
variables:
- VAR_1: value_A
- VAR_2: value_B
Example:
build_matrix_job:
stage: build
parallel:
matrix:
- OS: [linux, windows]
ARCH: [amd64, arm64]
script: echo "Building on $OS with $ARCH"
artifacts:
paths:
- build_$OS_$ARCH/
test_specific_build:
stage: test
script: echo "Testing Linux AMD64 build..."
needs:
- job: build_matrix_job
artifacts: true
variables:
OS: linux
ARCH: amd64 # This job specifically needs the 'linux-amd64' build from the matrix
Explanation: test_specific_build
will wait only for the build_matrix_job
that ran with OS: linux
and ARCH: amd64
to complete successfully, and it will automatically get its specific artifacts.
When to Use needs
- Speed Optimization: When you have long-running jobs that are not strict prerequisites for subsequent critical path jobs.
needs
helps reduce overall pipeline duration by running independent jobs concurrently. - Complex Dependencies: For pipelines where the flow is not a simple linear progression through stages, but rather a branching or converging set of tasks.
- Resource Management: If certain jobs require specialized runners or resources,
needs
can help ensure those resources are utilized efficiently by triggering jobs only when their exact prerequisites are met. - Monorepos & Microservices: Cross-project dependencies (
needs:project
) are invaluable for managing complex build and deployment flows across multiple repositories.
Best Practices for needs
- Keep Stages for Visualization: While
needs
breaks sequential stage execution, jobs still require astage
keyword. Use stages to logically group your jobs in the pipeline graph for better readability and organization in the GitLab UI. - Clear Dependencies: Only define
needs
for direct, actual dependencies. Avoid adding unnecessaryneeds
as it can complicate the graph and reduce parallelism. - Monitor the DAG View: Regularly check the pipeline’s DAG (available in the pipeline view within GitLab) to ensure your
needs
setup is working as intended and identify any bottlenecks or unexpected flows. - Artifact Management: Be mindful of artifact sizes.
needs
will automatically download artifacts by default. If a job isneeds
-ing many jobs with large artifacts but only requires a few, consider usingartifacts: false
for unnecessary ones. - Error Handling: Remember that if a required job in
needs
fails, the current job will not run. This is the default fail-fast behavior. Useallow_failure: true
on the needed job if its failure should not block the needing job, and then useoptional: true
in theneeds
definition for clarity.
FAQs – Needs
What is the needs
keyword in GitLab CI/CD?
The needs
keyword allows a job to explicitly declare its dependencies on other jobs, enabling jobs from later stages to run earlier, in parallel with earlier jobs. It overrides the default sequential stage execution and can speed up pipelines significantly.
How is needs
different from stages in GitLab?
By default, GitLab CI/CD executes jobs sequentially stage by stage. With needs
, jobs can run as soon as their dependencies finish, even if the dependent job is in a later stage.
Example comparison:
# Without needs: 'test' waits for all 'build' jobs to complete
stages:
- build
- test
# With needs: 'test' runs as soon as 'build-job' completes
test:
stage: test
needs: [build-job]
How do I use the needs
keyword in a job?
You define needs
as a list of job names the current job depends on:
build-job:
stage: build
script: make
test-job:
stage: test
needs: [build-job]
script: ./run-tests.sh
This allows test-job
to start immediately after build-job
finishes, instead of waiting for all build
stage jobs.
Can needs
improve pipeline performance?
Yes. By allowing jobs to run earlier instead of waiting for an entire stage to complete, needs
helps reduce total pipeline time through parallelism.
Can I specify artifacts
with needs
?
Yes. To download artifacts from the needed job, define it like this:
test:
stage: test
needs:
- job: build
artifacts: true
script:
- ./test-from-artifacts.sh
This ensures artifacts from build
are downloaded before test
runs.
What happens if I use needs
without specifying artifacts?
If you define a job using needs
without the artifacts: true
flag, artifacts are not downloaded automatically. If your job depends on output files from another, include artifacts: true
.
Can I use needs
across stages?
Yes. needs
is designed to cross stage boundaries. It enables jobs in later stages to start earlier based on actual job dependencies.
stages:
- build
- test
- deploy
deploy:
stage: deploy
needs: [test]
script: ./deploy.sh
In this case, deploy
runs as soon as test
completes.
Can I use needs
with jobs in the same stage?
Yes. You can define dependencies within the same stage:
job1:
stage: build
script: echo "Job 1"
job2:
stage: build
needs: [job1]
script: echo "Job 2"
This causes job2
to wait for job1
, even though both are in the same stage.
What is the maximum number of jobs I can reference in needs
?
In GitLab.com:
- Maximum: 50
needs
per job. - For self-managed GitLab, this can be configured by the admin.
Defining too many needs
can affect pipeline performance and readability.
What happens if a job listed in needs
fails?
If any job listed in needs
fails, the dependent job will not run. This ensures that jobs relying on earlier steps do not proceed with broken inputs.
Can I use needs
with trigger
jobs (multi-project pipelines)?
Yes, but only for downstream jobs, not upstream. Use needs
in combination with trigger
to define fine-grained control in child pipelines:
trigger-build:
trigger:
include: child-pipeline.yml
needs: []
This ensures the job does not wait for anything before triggering the child pipeline.
How do I visualize jobs with needs
in the pipeline graph?
GitLab displays a Directed Acyclic Graph (DAG) when using needs
, showing job dependencies clearly.
To view it:
CI/CD → Pipelines → View Pipeline Graph
Jobs connected with needs
will be linked with arrows, showing their execution order.
Can I use needs
and dependencies
together?
Yes, but they serve different purposes:
needs
: Controls execution order and concurrency.dependencies
: Controls which artifacts are downloaded from previous jobs.
Example:
test:
stage: test
needs:
- job: build
artifacts: true
dependencies:
- build
This ensures both early execution and artifact sharing.
Can I define an empty needs
list to skip automatic stage ordering?
Yes. You can use needs: []
to tell GitLab not to wait for earlier jobs or stages:
independent-job:
stage: deploy
needs: []
script: echo "Runs immediately"
This job runs as soon as a runner is available, regardless of the stage order.
Author

Experienced Cloud & DevOps Engineer with hands-on experience in AWS, GCP, Terraform, Ansible, ELK, Docker, Git, GitLab, Python, PowerShell, Shell, and theoretical knowledge on Azure, Kubernetes & Jenkins. In my free time, I write blogs on ckdbtech.com