Logo

dev-resources.site

for different kinds of informations.

The Art of Iteration: Loop in Pipeline Stage

Published at
1/4/2025
Categories
azure
devops
pipelines
community
Author
arindam0310018
Categories
4 categories in total
azure
open
devops
open
pipelines
open
community
open
Author
14 person written this
arindam0310018
open
The Art of Iteration: Loop in Pipeline Stage

Greetings my fellow Technology Advocates and Specialists.

This is the Chapter #2 of my Mastering Loops in Azure DevOps Series.

In this Session, I will walk you through The Art of Iteration: Loop in Pipeline Stage.

USECASE:-
Validate and Provision Multiple Azure Resource Groups using Iterations in an Azure DevOps YAML Pipeline.
IMPORTANT NOTE:-

The YAML Pipeline is tested on WINDOWS BUILD AGENT Only!!!

REQUIREMENTS:-
  1. Azure Subscription.
  2. Azure DevOps Organisation and Project.
  3. Service Principal with Required RBAC (Contributor) applied on Subscription or Resource Group(s).
  4. Azure Resource Manager Service Connection in Azure DevOps.
CODE REPOSITORY:-



HOW DOES MY CODE PLACEHOLDER LOOKS LIKE:-:-
Image description
EXAMPLE:-
PIPELINE CODE SNIPPET:-
AZURE DEVOPS YAML PIPELINE (azure-pipelines-Leverage-Loops-For-Automation-v1.0.yml):-
trigger:
  none

######################
#DECLARE PARAMETERS:-
######################
parameters:
- name: SUBSCRIPTIONID
  displayName: Subscription ID Details Follow Below:-
  type: string
  default: 210e66cb-55cf-424e-8daa-6cad804ab604
  values:
  - 210e66cb-55cf-424e-8daa-6cad804ab604

- name: RGNAME
  displayName: Please Provide the Resource Group Name:-
  type: object
  default: [AMRG001,AMRG002,AMRG003]

######################
#DECLARE VARIABLES:-
######################
variables:
  ServiceConnection: amcloud-cicd-service-connection
  BuildAgent: windows-latest
  loc: westeurope

#########################
# Declare Build Agents:-
#########################
pool:
  vmImage: $(BuildAgent)

###################
# Declare Stages:-
###################

stages:
- ${{ each rg in parameters.RGNAME }}:
  - stage: VALIDATE_AND_CREATE_${{ rg }} 
    jobs:
    - job: VALIDATE_AND_CREATE_${{ rg }} 
      displayName: VALIDATE AND CREATE ${{ rg }}
      steps:
      - task: AzureCLI@2
        displayName: SET AZURE ACCOUNT
        inputs:
          azureSubscription: $(ServiceConnection)
          scriptType: ps
          scriptLocation: inlineScript
          inlineScript: |
            az --version
            az account set --subscription ${{ parameters.SUBSCRIPTIONID }}
            az account show  
      - task: AzureCLI@2
        displayName: VALIDATE AND CREATE ${{ rg }}
        inputs:
          azureSubscription: $(ServiceConnection)
          scriptType: ps
          scriptLocation: inlineScript
          inlineScript: |      
            echo "Looped Value is: "${{ rg }}""

            $i = az group exists -n ${{ rg }}
              if ($i -eq "true") {
                echo "#####################################################"
                echo "Resource Group ${{ rg }} exists!!!"
                echo "#####################################################"
                }
                else {
                  echo "#############################################################"
                  echo "Resource Group ${{ rg }} DOES NOT EXISTS!!!"
                  echo "#############################################################"
                  echo "Creating Resource Group ${{ rg }}..."
                  echo "#############################################################"
                  az group create --name ${{ rg }} --location $(loc)
                  echo "#############################################################"
                  echo "Resource Group ${{ rg }} Successfully"
                  echo "#############################################################"
                }
Enter fullscreen mode Exit fullscreen mode
EXPLANATION:-

I.) It is a Single Stage Pipeline which first validates if Resource Group by the same name is available. If "No", then it will go ahead and create. If "Yes", it will then display in the pipeline console that "Resource Group Exists".

II.) Declare "Parameter" of type "Object". In this example, I have declared 3 values (Resource Group Names) as an array.

######################
#DECLARE PARAMETERS:-
######################
parameters:
- name: SUBSCRIPTIONID
  displayName: Subscription ID Details Follow Below:-
  type: string
  default: 210e66cb-55cf-424e-8daa-6cad804ab604
  values:
  - 210e66cb-55cf-424e-8daa-6cad804ab604

- name: RGNAME
  displayName: Please Provide the Resource Group Name:-
  type: object
  default: [AMRG001,AMRG002,AMRG003]
Enter fullscreen mode Exit fullscreen mode

III.) Declare "Each" at beginning of the Pipeline stage. The "rg" variable stores all the values that the "each" keyword iterates over in the parameter of type object.

stages:
  - ${{ each rg in parameters.RGNAME }}:
Enter fullscreen mode Exit fullscreen mode

IV.) As there are 3 values declared in "Parameter" of type "Object", there will be 3 STAGES CREATED.

V.) Name of each stage and job will be one of iterated value from the array declared in "Parameter" of type "Object".

- stage: VALIDATE_AND_CREATE_${{ rg }} 
    jobs:
    - job: VALIDATE_AND_CREATE_${{ rg }} 
      displayName: VALIDATE AND CREATE ${{ rg }}
Enter fullscreen mode Exit fullscreen mode

VI.) Validation of the iterated value from the array declared in "Parameter" of type "Object" followed by the decision, whether to create Resource Group or not, happens as part of Inline Script.

- task: AzureCLI@2
        displayName: VALIDATE AND CREATE ${{ rg }}
        inputs:
          azureSubscription: $(ServiceConnection)
          scriptType: ps
          scriptLocation: inlineScript
          inlineScript: |      
            echo "Looped Value is: "${{ rg }}""

            $i = az group exists -n ${{ rg }}
              if ($i -eq "true") {
                echo "#####################################################"
                echo "Resource Group ${{ rg }} exists!!!"
                echo "#####################################################"
                }
                else {
                  echo "#############################################################"
                  echo "Resource Group ${{ rg }} DOES NOT EXISTS!!!"
                  echo "#############################################################"
                  echo "Creating Resource Group ${{ rg }}..."
                  echo "#############################################################"
                  az group create --name ${{ rg }} --location $(loc)
                  echo "#############################################################"
                  echo "Resource Group ${{ rg }} Successfully"
                  echo "#############################################################"
                }
Enter fullscreen mode Exit fullscreen mode
OUTPUT OF EXAMPLE:-
1. As in the example, there were 3 values declared in "Parameter" of type "Object", hence 3 STAGES got created.
Image description
2. Name of each stage and Job is the iterated value from the array declared in "Parameter" of type "Object".
Image description
3. Below is how it looks when the Pipeline iterates and creates resource group(s).
Image description
4. Below is how it looks when the pipeline iterates, detects that a resource group with the same name already exists, and therefore skips creating the resource group(s).
Image description

Hope You Enjoyed the Session!!!

Stay Safe | Keep Learning | Spread Knowledge

pipelines Article's
30 articles in total
Favicon
The Art of Iteration: Starting the Cycle
Favicon
The Art of Iteration: Loop in Pipeline Stage
Favicon
Automating Docker Workflows with Jenkins: A Complete Guide
Favicon
TIL how to see the entire commit column on GitLab using JS
Favicon
Getting Started with Apache Kafka: A Backend Engineer's Perspective
Favicon
Building pipelines with IAsyncEnumerable in .NET
Favicon
DevOps Security Integrating Best Practices into Your Pipeline
Favicon
Creating a data pipeline using Dataproc workflow templates and cloud Schedule
Favicon
☸️ Kubernetes: A Convenient Variable Substitution Mechanism for Kustomize
Favicon
Setting Up a CI/CD Pipeline with AWS and Git: A Comprehensive Guide
Favicon
Enabling Pipelines: Easier than ever
Favicon
Optimizing GitLab CI for Readability and Maintainability: From 1K to 600 Lines!
Favicon
Building Robust Data Pipelines: A Comprehensive Guide
Favicon
Azure DevOps Pipelines breaks my "additional arguments" when using Deploy to Azure
Favicon
What is CI/CD Pipeline?-Comparing pipelines!
Favicon
🌟 The Power of Automation: Deploying an ARM Template in Microsoft Azure πŸš€
Favicon
Meet cici-tools, a multi-tool for building GitLab CI/CD pipelines
Favicon
Unlocking the Power of Data: 7 Key Factors to Consider When Building Data Pipelines
Favicon
Optimize Development with Jenkins Pipelines and Continuous Integration
Favicon
Amplify Your Tech Stack with Jenkins Shared Libraries
Favicon
Important Questions related to Data Engineering
Favicon
How To Secure Your CI/CD Pipeline
Favicon
Flexible and dynamic flow control of Azure DevOps YAML Pipelines using variables
Favicon
Go API Project Set-Up
Favicon
Sftp with Az Devops
Favicon
Error: Full scoped PAT is restricted by your organisation
Favicon
Error: No hosted parallelism has been purchased or granted
Favicon
SparrowCI - DSL is dead, long live DSL!
Favicon
Introducing the CircleCI Config SDK
Favicon
Run DB Scripts to Azure PostgreSQL Single Server using Azure CLI Task in pipeline

Featured ones: