The Talent500 Blog

Troubleshooting Azure Pipeline Errors: A Crash Course on Error Handling in Azure Pipelines in 2023

Neel Vithlani

Cloud computing can be attributed as one of the biggest stakeholders of the massive innovation in the digital world over the past decade.  Today, businesses use an average of 137 paid distinct SaaS apps while they use 3x more free SaaS apps, a figure that has continued to increase significantly over the past few years.

(Image Credits: Statista)

The SaaS market is estimated to be valued at $716.52 billion by 2028, with a CAGR of 27.5% during the forecast period 2021-2028. On the other hand, an average smartphone user among 6.5 billion users has 35 apps installed while using 9 apps on a daily basis. These figures reflect the mass utilization of cloud technology, and at the same time, they are indicative of the mammoth workforce and infrastructure required to keep such wide-scale systems running. Undoubtedly CI/CD plays a huge role in ensuring that the seamless user experience doesn’t turn obsolete and that businesses continue to innovate.

With a 21% market share, Microsoft Azure is among the leading cloud services providers. Combining the complexity of IT infrastructures with the technical sophistication it has to offer, running into pipeline errors.  As the Talent500 blog caters to a huge number of Azure DevOps engineers, our tech experts decided to curate this article on handling Azure Pipeline Errors.

Let’s get started.

Understanding Azure Pipelines And Errors Faced

To get a better understanding of the errors, let us first begin with Azure Pipelines. This service is used for automating the processes like building, compiling, deploying, and testing codes.  It helps minimize manual intervention, which in turn reduces errors as it enables DevOps engineers to configure CI/CD for various computational environments through several stages.

How to Find Out What Went Wrong When an Activity Fails in Azure Data Factory

Let us consider the case of three activities named Set variable1, Set variable2, and Set variable3 configured as conditional paths. To get the error details via stored procedure or email. 

For Parallel Pipelines In Combination With A Wait/Dummy Activity in Azure DataFactory:

To detect the failure, use the below syntax:

@concat(activity(‘Set variable1).Error?.message,’|’activity(‘Set variable2).Error?.message,’|’activity(‘Set variable3).Error?.message)

Here, the ‘?’ symbol is added to the respective properties to avoid any issues caused by null reference errors when an activity is completed successfully. 

Let us consider a scenario where the second activity fails. In that case, you will get the below error message:

Output

{

  “name”: “Error”,

  “value”: “|The variable ‘var1’ of type ‘String’ cannot be initialized or updated with value of type ‘Integer’. The variable ‘var1’ only supports values of types ‘String’./|”

}

However, in the case of series configuration for activities, using the syntax will result in the below error with Invalid Template code, type: User configuration issue:

The expression ‘concat(activity(‘Set

variable1).Error?.message, ‘|’,activity(‘Set

variable2′).Error?.message, ‘|’,activity(‘Set

variable3′).Error?.message)’ cannot be evaluated because

property “Error” cannot be selected.

Here, you need to use the ‘?’ symbol for the activity names also since the third activity, in this case, won’t be fired at all:

@concat(activity(‘Set variable1).Error?.message,’|’activity(‘Set variable2)?.Error?.message,’|’activity(‘Set variable3)?.Error?.message)

It is noteworthy that we used ‘?’ from the second activity onwards since the first activity is invoked by default. 

Here’s how the output error will appear in this case:

{

“name”: “Error”,

“value”: “[The variable ‘var1’ of type ‘String’ cannot be initialized or

updated with value of type ‘Integer’. The variable ‘var1’ only supports

values of types ‘String’.l

}

For more details on determining pipeline failure in Azure DataFactory, click here.

Error Handling In Azure Pipeline

In order to determine the reason behind Azure pipeline failure, you must follow the below process:

To begin with, most Azure pipeline errors fall into either of these three categories:

  1. The pipeline won’t trigger
  2. Pipeline queues but never gets an agent
  3. The pipeline fails to complete

For the first case where the pipeline won’t trigger at all, you can try the following solutions:

In the second scenario, where pipeline queues but fails to get an agent, follow the below procedures:

Now let us consider the cases where the pipeline fails to complete:

D:\home\python364x64\python.exe -m pip install -r requirements.txt

Access is denied.

Access to the path […] is denied.

Can’t move […] to […]

The process cannot access the file […] because it is being used by another process.

It is likely that you are encountering errors with the file or folder in use. In such cases, follow the below process:

  1. You can collect a record of file events within a particular directory using Process Monitor, Process Explorer, or Handle.
  2. You can compel MSBuild into running only one process at a time by including the /m:1 parameter in your build tasks, as MSBuild and Visual Studio Build tasks run MSBuild with the /m switch, causing problems such as multiple process file access issues. Also, you must pass the /nodeReuse:false argument. 
  3. You may make an antivirus exception for the “work folder” and agent directory that you have set up.

 

* text eol=lf

echo ##vso[task.setvariable variable=MY_VAR]my_value

set -x

# Script for Linux and macOS

pool: { vmImage: ubuntu-latest } # or whatever pool you use

steps:

– checkout: none

– bash: |

    SEARCH_PATH=$PATH  # or any colon-delimited list of paths

    IFS=’:’ read -r -a PathDirs <<< “$SEARCH_PATH”

    echo “##[debug] Found directories”

    for element in “${PathDirs[@]}”; do

        echo “$element”

    done;

    echo;

    echo;  

    echo “##[debug] Found files”

    for element in “${PathDirs[@]}”; do

        find “$element” -type f

    done

# Script for Windows

pool: { vmImage: windows-2019 } # or whatever pool you use

steps:

– checkout: none

– powershell: |

    $SEARCH_PATH=$Env:Path

    Write-Host “##[debug] Found directories”

    ForEach ($Dir in $SEARCH_PATH -split “;”) {

      Write-Host “$Dir”

    }

    Write-Host “”

    Write-Host “”

    Write-Host “##[debug] Found files”

    ForEach ($Dir in $SEARCH_PATH -split “;”) {

      Get-ChildItem $Dir -File -ErrorAction Continue | ForEach-Object -Process {

        Write-Host $_.FullName

      }

    }

Wrap Up

We hope that this mini-guide helps you handle Azure DevOps pipeline errors by diagnosing the issue and looking for the right solutions.With an average compensation of $105,017 per annum, making a career in Azure DevOps is better suited than many other software profiles. 

If you are looking to find the right Azure DevOps job for yourself, join Talent500 today!

 

0