Tutorial: Create and run your first GitLab CI/CD pipeline Create a .gitlab-ci.yml file

 GitLab CI/CD pipelines are defined using a YAML file called .gitlab-ci.yml which is stored in the root directory of your GitLab project. This YAML file specifies the stages, jobs, and commands needed to build, test, and deploy your project. Let me break down the basic structure and provide examples:

1. Define Stages:

Stages represent the different steps in your pipeline. Each stage can contain one or more jobs, and stages run sequentially.

stages: - build - test - deploy

2. Define Jobs:

Jobs are the individual tasks that need to be performed within each stage. Each job runs in a separate environment, and you can specify dependencies between jobs.

job_name: stage: stage_name script: - command1 - command2

Example

stages:

  - build

  - test

  - deploy

build:

  stage: build

  script:

    - npm install

    - npm run build

test:

  stage: test

  script:

    - npm test

deploy:

  stage: deploy

  script:

    - deploy_script.sh

  environment:

    name: production

    url: https://example.com

3. Define Scripts:

In each job, you can specify the commands that need to be executed. This can include build commands, testing commands, deployment scripts, etc.

Example:

build:

  stage: build

  script:

    - npm install

    - npm run build

4. Define Environment:

You can define environments for your jobs, such as staging or production. This helps in deploying to specific environments and managing releases.

Example:

deploy:

  stage: deploy

  script:

    - deploy_script.sh

  environment:

    name: production

    url: https://example.com

5. Define Triggers:

You can define triggers for your pipeline, such as when to run the pipeline (e.g., on every push to a specific branch, on a schedule, manually triggered, etc.).

Example:

trigger:

  branches:

    only:

      - main

6. Define Variables:

You can define variables to be used in your pipeline, such as API keys, environment-specific configurations, etc.

Example:

variables:

  ENV: production

  API_KEY: $API_KEY

Let's delve deeper into some advanced features and configurations available in GitLab CI/CD YAML pipelines:

1. Artifacts:

Artifacts are files generated by your jobs that you want to keep or use in subsequent stages. You can define artifacts to be passed between stages or to be downloaded after the job finishes.

build: stage: build script: - npm install - npm run build artifacts: paths: - dist/

2. Dependencies:

You can define dependencies between jobs, ensuring that one job runs only after the successful completion of another job.

test: stage: test script: - npm test dependencies: - build

3. Caching:

Caching allows you to cache dependencies between job runs, speeding up subsequent pipeline executions.

build: stage: build script: - npm install - npm run build cache: paths: - node_modules/

4. Rules:

Rules allow you to define conditions under which jobs will be executed. This provides more flexibility than simple only or except directives.

deploy: stage: deploy script: - deploy_script.sh rules: - if: '$CI_COMMIT_BRANCH == "main"' when: always

5. Parallel Jobs:

You can define jobs to run in parallel, improving the overall pipeline execution time.

test: stage: test script: - npm test parallel: 3

6. Manual Jobs:

Manual jobs require manual intervention before they can be executed, useful for tasks like deployment to production.

deploy_production: stage: deploy script: - deploy_script.sh environment: name: production url: https://example.com when: manual

7. Triggering External Pipelines:

You can trigger pipelines in other projects, enabling cross-project dependencies and workflows.

trigger: include: - project: 'group/project' ref: main

8. Custom Docker Images:

You can specify custom Docker images to use in your jobs, allowing for a specific environment setup.

image: node:14

9. Secrets:

You can securely store and use secrets such as API keys, SSH keys, etc., in your pipeline.

deploy: stage: deploy script: - deploy_script.sh environment: name: production url: https://example.com variables: SECRET_KEY: $SECRET_KEY

========================================

Here's a list of key keywords used in GitLab CI/CD YAML files along with examples and descriptions:

1. stages:

• Example:

stages:

  - build

  - test

  - deploy

1. • Description: Defines the different stages of the pipeline. Jobs within the same stage run in parallel, while stages run sequentially.

2. job:

• Example:

build: stage: build script: - npm install - npm run build

1. • Description: Defines a job to be executed in a particular stage. It includes the script or commands to be run.

2. script:

• Example:

build:

  stage: build

  script:

    - npm install

    - npm run build

1. • Description: Contains the commands or script to be executed for the job.

2. artifacts:

• Example:

build: stage: build script: - npm install - npm run build artifacts: paths: - dist/

1. • Description: Specifies files or directories generated by the job that should be passed to subsequent stages or available for download.

2. dependencies:

• Example:
3. test: stage: test script: - npm test dependencies: - build

1. • Description: Defines dependencies between jobs, ensuring that a job runs only after the successful completion of specified jobs.

2. cache:

• Example:

build:

  stage: build

  script:

    - npm install

    - npm run build

  cache:

    paths:

      - node_modules/

1. • Description: Configures caching of files or directories between job runs, improving pipeline execution speed.

2. rules:

• Example:

deploy: stage: deploy script: - deploy_script.sh rules: - if: '$CI_COMMIT_BRANCH == "main"' when: always

1. • Description: Specifies conditions under which jobs will be executed, providing more flexibility than simple only or except directives.

2. parallel:

• Example:

test: stage: test script: - npm test parallel: 3

1. • Description: Defines the number of parallel jobs to be executed for a job.

2. when:

• Example:

deploy_production: stage: deploy script: - deploy_script.sh environment: name: production url: https://example.com when: manual

1. • Description: Specifies when a job should be executed. It can be on_success, on_failure, always, or manual.

2. trigger:

• Example:

trigger: include: - project: 'group/project' ref: main

1. • Description: Triggers pipelines in other projects, enabling cross-project dependencies and workflows.

2. image:

• Example:

image: node:14

1. • Description: Specifies a custom Docker image to use for the job, allowing for a specific environment setup.

2. variables:

• Example:

deploy: stage: deploy script: - deploy_script.sh environment: name: production url: https://example.com variables: SECRET_KEY: $SECRET_KEY

1. • Description: Defines variables to be used in the job, such as secrets, environment-specific configurations, etc.

These are the key keywords used in GitLab CI/CD YAML files along with examples and descriptions. They provide the necessary structure and configuration options to define and customize your pipeline.

========================================

build-job: stage: build script: - echo "Welcome, $GITLAB_USER_LOGIN!" test-job1: stage: test script: - echo "This job tests something" test-job2: stage: test script: - echo "This job tests something, but takes longer than test-job1" - echo "After the echo command completes, it runs the sleep command for 30 seconds" - echo "This simulates the test running 30 seconds longer than test-job1" - sleep 30 deploy-prod: stage: deploy script: - echo "This job deploys a commit from $CI_COMMIT_BRANCH" environment: production

Source :

https://codefresh.io/learn/gitlab-ci/gitlab-ci-cd-3-quick-tutorials/

https://docs.gitlab.com/ee/ci/quick_start/

----- FT --- Working --- gitlab using variables

Variables defined in CICD

Project => Settings => CICD => Variables

requirements.txt

requests==2.26.0 Flask==2.0.2 pytest==6.2.5

.gitlab-ci.yml

image: python:3.9

stages:

  - build

  - test

  - deploy

variables:

  ENVIRONMENT: "production"

  API_KEY: "$API_KEY"

cache:

  paths:

    - .venv/

build:

  stage: build

  script:

    - python -m venv .venv

    - source .venv/bin/activate

    - pip install -r requirements.txt

test:

  stage: test

  script:

    - source .venv/bin/activate

    - export PYTHONPATH=$(pwd)  # Set PYTHONPATH to the root directory

    - pytest

deploy:

  stage: deploy

  script:

    - source .venv/bin/activate

    - cd scripts  # Navigate to the scripts directory    

    - python deploy_script.py

  environment:

    name: $ENVIRONMENT

  only:

    - main

tests/test_deploy_script.py

import pytest

import sys

print("Test 1111",sys.path)

from scripts.deploy_script import deploy

def test_deploy_successful():

    print("test_deploy_successful")

    # Your test implementation here

    pass

def test_deploy_failed():

    print("test_deploy_failed")

    # Your test implementation here

    pass

scripts/deploy_script.py

import requests

import sys

import os  # Import the os module for accessing environment variables

print("deploy 2222", sys.path)

def deploy():

    # Your deployment logic here

    api_key = get_api_key()

    environment = get_environment()

    # Example usage

    print("Deploying to:", environment)

    response = requests.post("https://api.example.com/deploy", headers={"Authorization": f"Bearer {api_key}"})

    print("response.status_code", response.status_code)

    if response.status_code == 200:

        print("Deployment successful!888888888")

    else:

        print("Deployment failed!8888888888")

def get_api_key():

    # Retrieve API_KEY from environment variable

    print("APIXXXXXXXXX", os.getenv("API_KEY"))

    return os.getenv("API_KEY")

def get_environment():

    # Retrieve ENVIRONMENT from environment variable

    print("Env 3333333", os.getenv("ENVIRONMENT"))

    return os.getenv("ENVIRONMENT")

if __name__ == "__main__":

    deploy()