Thursday, August 27, 2020

Quick Start with the JIRA REST API Browser

Milky Way over Guilderton Lighthouse, Western Australia - 35mm Panorama

Looking to start programming against the JIRA API? Me too! This post walks through setting up JIRA locally on your computer and installing the JIRA API Explorer to start poking at the APIs.

Why use the REST API Browser?

Good question. Postman and other tools can poke at JIRA endpoints just as well, but the API Explorer comes equipped with help documentation and a lets you quickly pick out an endpoint without having to worry about authentication, etc. You can easily toggle between GET, POST, PUT, DELETE commands for endpoints that support it, and the explorer provides a simple form entering required and optional fields.

jira-explorer

Setup a JIRA instance in Docker

Setting up JIRA in a docker container is pretty easy.

docker volume create --name jiraVolume
docker run -v jiraVolume:/var/atlassian/application-data/jira --name="jira" -d -p 8080:8080 atlassian/jira-software

After the docker image is downloaded, open your browser to http://localhost:8080

On first start, you will be prompted with a series of prompts:

  • Set it up for me
  • Create a trial license key
  • Setup an admin user account
  • Create an empty project, use a sample project or import from an existing project.

Quick Setup

When you first navigate to the docker instance, you'll be prompted with a choice:

jira-setup-1

Let's go with "Set it up for me", and then click on "Continue to MyAtlassian"

Create a Trial License

You'll be redirected to provide information for creating a trial license. Select "JIRA Server", provide a name for your organization and leave the default values for the server name.

jira-setup-2

When you click next, you'll be redirected to a screen that shows the license key and a Confirmation dialog.

jira-setup-3

Click Yes to confirm the installation.

Create an Admin User Account and default settings

Somewhat self-explanatory, provide the details for your admin account and click Next.

jira-setup-4

You should see a setup page run for a few minutes. When this completes, you'll need to:

jira-setup-5

  • Login with the admin account you created
  • Pick your default language
  • Choose an Avatar

Create your First Project

The last step of the Quick Setup is to create your first project. Here you can create an empty project, import from an existing project or use a sample project. As we're interested in playing with the API, pick the "See it in action" option to create a JIRA project with a prepopulated backlog.

jira-setup-7

Selecting "See it in action", prompts for the style of project and the project details.

jira-setup-8

Great job. That was easy.

Install the API Browser

The API Explorer is available through the Atlassian Marketplace. To install the add-on:

  1. In the top right, select Settings Applications
  2. Click the "Manage Apps" tab
  3. Search for "API Browser"

    manage-apps-1
  4. Click "Install"

Super easy.

Using the REST API Browser

The hardest part of using the REST API Browser is locating it after it's been installed. Fortunately, that's easy, too.

In the top-right, select Settings System and scroll all the way to the bottom to Advanced: REST API Browser

manage-apps-2

Try it out!

The REST API Browser is fairly intuitive, simply find the endpoint you're interested in on the right hand side and provide the necessary fields in the form.

rest-api-browser-1

Mind your verbs...

Pay special attention to the HTTP verbs being used: GET, POST, PUT, DELETE. Most of JIRA's endpoints differ only by the verb, items that are POST + PUT will either create or update records and not all endpoints will have a GET.

rest-api-browser-2

Wrap up

Now you have now excuse to start picking at the JIRA API.

Side note: I wrote this post using Typora, a super elegant markdown editor. I'm still experimenting on how to best integrate with my blog platform (Blogger), but I might look at some form of combination of static markdown.md files with a command-line option to publish. I will most likely post that setup when I get there.

Until then, happy coding.

Tuesday, August 18, 2020

Cleaning up stale git branches

Broken branches

If you have pull-requests, you likely have stale branches. What’s the best way to find which branches can be safely deleted? This post will explore some approaches to find and delete stale branches.

As there are many different ways we can approach this, I’m going to start with the most generic concepts and build up to a more programmatic solution. I want to use the Azure DevOps CLI and REST APIs for this instead of git-centric commands because I want to be able to run the scripts from any computer against the latest version of the repository. This also opens up the possibility of running these activities in a PowerShell script in an Azure Pipeline, as outlined in my previous post.

Table of Contents

Who can delete Branches?

One of the reasons branches don’t get deleted might be a permissions problem. In Azure DevOps, the default permission settings set up the creator of the branch with the permission to delete it. If you don’t have permission, the option isn’t available to you in the user-interface, and this creates a missed opportunity to remediate the issue when someone other than the author completes the PR.

pr-complete-merge-disabled-delete

pr-delete-source-branch

You can change the default setting by adding the appropriate user or group to the repository’s permissions and the existing branches will inherit. You’ll need the “Force push (rewrite history, delete branches and tags)” permission to delete the branch.  See my last post on ways to apply this policy programmatically.

branch-permission-forcepush

If we want to run this from a pipeline, we would have to grant the Build Service the same permissions.

Finding the Author of a Branch

One approach to cleaning-up stale branches is the old fashion way: nagging. Simply track down the branch authors and ask them to determine if they’re done with them.

We can find the authors for the branches with the following PowerShell + Az DevOps CLI:

$project    = "<project-name>"
$repository = "<repository-name>"

$refs = az repos ref list `
--query "[?starts_with(name, 'refs/heads')].{name:name, uniqueName:creator.uniqueName}" ` --project $project --repository $repository | ConvertFrom-Json $refs | Sort-Object -Property uniqueName

az-repos-ref-list-example1

Finding the Last Contributor to a Branch

Sometimes, the author of the branch isn’t the person doing the work. If this is the case, you need to track down the last person to commit against the branch. This information is available in the Azure DevOps user interface (Repos –> Branches):

branch-authors

If you want to obtain this information programmatically, az repos list ref provides us with the objectId SHA-1 of the most recent commit. Although az repos doesn’t expose a command to retrieve the commit details, we can use the az devops invoke command to call the Get Commit REST endpoint.

When we fetch the detailed information on the branch, we want to get the author that created the commit and the date that they pushed it to the server. We want the push details because a developer may have made the commit a long time ago but only recently updated the branch.

$project    = "<project-name>"
$repository = "<repository-name>"

$refs = az repos ref list –p $project –r $repository --filter heads | ConvertFrom-Json

$results = @()

foreach($ref in $refs) {

$objectId = $ref.objectId
# fetch individual commit details $commit = az devops invoke `
--area git `
--resource commits ` --route-parameters ` project=$project ` repositoryId=$repository ` commitId=$objectId |
ConvertFrom-Json $result = [PSCustomObject]@{ name = $ref.name creator = $ref.creator.uniqueName lastAuthor = $commit.committer.email
lastModified = $commit.push.date } $results += ,$result } $results | Sort-Object -Property lastAuthor

az-repos-ref-list-example

This gives us a lot of details for the last commit in each branch, but if you’ve got a lot of branches, fetching each commit individually could be really slow. So, instead we can use the same Get Commit endpoint to fetch the commit information in batches by providing a collection of objectIds in a comma-delimited format.

Note that there’s a limit to how many commits we can ask for at a time, so I’ll have to batch my batches. I could also use the Get Commits Batch endpoint that accepts the list of ids in the body of a POST message.

The following shows me batching 50 commits at a time. Your batch size may vary if the name of your server or organization name is a longer length:

$batchSize = 50
$batches = [Math]::Ceiling($refs.Length / $batchSize)

for( $x=0; $x -lt $batches; $x++ )
{
# take a batch $batch = $refs | Select-Object -First $batchSize -Skip ($x * $batchSize)

# grab the ids for the batch $ids = ($batch | ForEach-Object {$_.objectId}) -join ','

# ask for the commit details for these items $commits = az devops invoke --area git --resource commits ` --route-parameters ` project=$project `
repositoryId=$repository ` --query-parameters ` searchCriteria.ids=$ids ` searchCriteria.includePushData=true ` --query "value[]" | ConvertFrom-Json
# loop through this batch of commits for($i=0; $i -lt $commits.Length; $i++) {
$ref = $refs[($x*$batchSize)+$i] $commit = $commits[$i]

# add commit information to the batch $ref | Add-Member -Name "author" -Value $commit.author.email -MemberType NoteProperty $ref | Add-Member -Name "lastModified" -Value $commit.push.date -MemberType NoteProperty

# add the creator’s email on here for easier access in the select-object statement...
$ref | Add-Member –Name "uniqueName” –Value $ref.creator.uniqueName –MemberType NoteProperty } } $refs | Select-Object -Property name,creator,author,lastModified

Caveat about this approach: If you've updated the source branch by merging from the target branch, the last author will be from that target branch – which isn’t what we want. Even worse, there's no way to infer this scenario from the git commit details alone. One way we can solve this problem is to fetch the commits in the branch and walk up the parents of the commit until we find a commit that has more than one parent – this would be our merge commit from the target branch, which should have been done by the last author on the branch. Note that the parent information is only available if you query these items one-by-one, so this approach could be painfully slow. (If you know a better approach, let me know)

Check the Expiry

Now that we have date information associated to our branches, we can start to filter out the branches that should be considered stale. In my opinion anything that’s older than 3 weeks is a good starting point.

$date = [DateTime]::Today.AddDays( -21 )
$refs = $refs | Where-Object { $_.lastModified -lt $date }

Your kilometrage will obviously vary based on the volume of work in your repository, but on a recent project 10-20% of the branches were created recently.

Finding Branches that have Completed Pull-Requests

If you’re squashing your commits when your merge, you’ll find that the ahead / behind feature in the Azure DevOps UI is completely unreliable. This is because a squash merge re-writes history, so your commits in the branch will never appear in the target-branch at all. Microsoft recommends deleting the source branch when using this strategy as there is little value in keeping these branches around after the PR is completed. Teams may argue that they want to cherry-pick individual commits from the source-branch, but the practicality of that requires pristine

Our best bet to find stale branches is to look at the Pull-Request history and consider all branches that are associated to completed Pull-Requests as candidates for deletion. This is super easy, barely an inconvenience.

$prs = az repos pr list `
          --project $project `
          --repository $repository `
          --target-branch develop `
          --status completed `
--query "[].sourceRefName" | ConvertFrom-Json | $refs | Where-Object { $prs.Contains( $_.name ) } | ForEach-Object { $result = az repos ref delete ` --name $_.name ` --object-id $_.id ` --project $project ` --repository $repository | ConvertFrom-Json
Write-Host ("Success Message: {0}" –f $result.updateStatus) }

At first glance, this would remove about 50% of the remaining branches in our repository, leaving us with 10-20% recent branches and an additional 30-40% of branches without PRs. This is roughly a 40% reduction, and I’ll take that for now. It’s important to recognize this only includes the completed PRs, not the active or abandoned.

Wrapping Up

Using a combination of these techniques we could easily reduce the amount of stale branches, and then provide the remaining list to the team to have them clean-up the dredges. The majority of old branches are likely abandoned work, but there's sometimes scenarios where partially completed work is waiting on some external dependency. In that scenario, encourage the team to keep these important branches up-to-date to retain the value of the invested effort.

The best overall strategy is to adopt a strategy that does not let this situation occur: give the individuals reviewing and completing PRs the permission to delete branches and encourage teams to squash and delete branches as they complete PRs. Good habits create good hygiene.

Happy coding.

Friday, August 14, 2020

Securing Git Branches through Azure DevOps CLI

Permission Granted

I've been looking for an way to automate branch security in order to enforce branch naming conventions and to control who can create release branches. Although the Azure DevOps documentation illustrates how to do this using the tfssecurity.exe command, the documentation also suggests that the tfssecurity.exe command is now deprecated.

This post will walk through how to apply branch security to your Azure DevOps repository using the Azure DevOps CLI.

Table of Contents

Understanding Azure DevOps Security

The Azure DevOps Security API is quite interesting as security can be applied to various areas of the platform, including permissions for the project, build pipeline, service-connection, git repositories, etc. Each of these areas support the ability to assign permissions for groups or individuals to a security token. In some cases these tokens are hierarchical, so changes made at the root are inherited on children nodes. The areas that define the permissions are defined as Security Namespaces, and each token has a Security Access Control List that contains Access Control Entries.

We can obtain a complete list of security namespaces by querying https://dev.azure.com/<organization>/_apis/securitynamespaces, or by querying them using the az devops cli:

az devops security permission namespace list

Each security namespace contains a list of actions, which are defined as bit flags. The following shows the "Git Repositories" security namespace:

az devops security permission namespace list `
    --query "[?contains(name,'Git Repositories')] | [0]"
{
    "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87",
    "name": "Git Repositories",
    "displayName": "Git Repositories",
    "separatorValue": "/",
    "elementLength": -1,
    "writePermission": 8192,
    "readPermission": 2,
    "dataspaceCategory": "Git",
    "actions": [
        {
            "bit": 1,
            "name": "Administer",
            "displayName": "Administer",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 2,
            "name": "GenericRead",
            "displayName": "Read",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 4,
            "name": "GenericContribute",
            "displayName": "Contribute",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 8,
            "name": "ForcePush",
            "displayName": "Force push (rewrite history, delete branches and tags)",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 16,
            "name": "CreateBranch",
            "displayName": "Create branch",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 32,
            "name": "CreateTag",
            "displayName": "Create tag",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 64,
            "name": "ManageNote",
            "displayName": "Manage notes",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 128,
            "name": "PolicyExempt",
            "displayName": "Bypass policies when pushing",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 256,
            "name": "CreateRepository",
            "displayName": "Create repository",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 512,
            "name": "DeleteRepository",
            "displayName": "Delete repository",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 1024,
            "name": "RenameRepository",
            "displayName": "Rename repository",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 2048,
            "name": "EditPolicies",
            "displayName": "Edit policies",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 4096,
            "name": "RemoveOthersLocks",
            "displayName": "Remove others' locks",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 8192,
            "name": "ManagePermissions",
            "displayName": "Manage permissions",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 16384,
            "name": "PullRequestContribute",
            "displayName": "Contribute to pull requests",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        },
        {
            "bit": 32768,
            "name": "PullRequestBypassPolicy",
            "displayName": "Bypass policies when completing pull requests",
            "namespaceId": "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"
        }
    ],
    "structureValue": 1,
    "extensionType": "Microsoft.TeamFoundation.Git.Server.Plugins.GitSecurityNamespaceExtension",
    "isRemotable": true,
    "useTokenTranslator": true,
    "systemBitMask": 0
}

Security Tokens for Repositories and Branches

The tokens within the Git Repositories Security Namespace follow the naming convention repoV2/<projectId>/<repositoryId>/<branch>. As this is a hierarchical security namespace, you can target very specific and granular permissions by adding parameters from left to right.

Examples:

  • All repositories within the organization: repoV2/
  • All repositories within a project: repoV2/<projectId>
  • All branches within a repository: repoV2/<projectId>/<repositoryId>
  • Specific branch: repoV2/<projectId>/<repositoryId>/<branch>

As the tokens are hierarchial, a really cool feature is that we can define patterns for branches that do not exist yet.

While the project and repository elements are relatively self-explanatory, the git branch convention is case sensitive and expressed in a hex format. Both Jesse Houwing and the Azure DevOps blog have some good write-ups on understanding this format.

Converting Branch Names to git hex format

The following PowerShell script can produce a security token for your project, repository or branch.

function Get-RepoSecurityToken( [string]$projectId, [string]$repositoryId, [string]$branchName) {

   $builder = "repoV2/"

   if ( ![string]::IsNullOrEmpty($projectId) ) {

     $builder += $projectId
     $builder += "/"

     if ( ![string]::IsNullOrEmpty($repositoryId) ) {

        $builder += $repositoryId
        $builder += "/"

        if ( ![string]::IsNullOrEmpty( $branchName) ) {

            $builder += "refs/heads/"
            
            # remove extra values if provided
            if ( $branchName.StartsWith("/refs/heads/") ) {
                $branchName = $branchName.Replace("/refs/heads/", "")
            }
            if ( $branchName.EndsWith("/")) {
                $branchName = $branchName.TrimEnd("/")
            }

            $builder += (($branchName.Split('/')) | ForEach-Object { ConvertTo-HexFormat $_ }) -join '/'   
        }

     }
   }

   return $builder
}

function ConvertTo-HexFormat([string]$branchName) {
   return ($branchName | Format-Hex -Encoding Unicode | Select-Object -Expand Bytes | ForEach-Object { '{0:x2}' -f $_ }) -join ''
}

Obtaining Branch Name from Git Hex Format

If you're working with query results and would like to see the actual name of the branches you've assigned, this function can reverse the hex format into a human readable string.

function ConvertFrom-GitSecurityToken([string]$token) {
    $refHeads = "/refs/heads/"

    $normalized = $token

    if ($token.Contains($refHeads)) {

        $indexOf = $token.IndexOf($refHeads) + $refHeads.Length

        $firstHalf = $token.Substring(0, $indexOf)
        $secondHalf = $token.Substring($indexOf)

        $normalized = $firstHalf
        $normalized += (($secondHalf.Split('/')) | ForEach-Object { ConvertFrom-HexFormat $_ }) -join '/'  
    }

    return $normalized
}

function ConvertFrom-HexFormat([string]$hexString) {

    $bytes = [byte[]]::new($hexString.Length/2)
    
    for($i = 0; $i -lt $hexString.Length; $i += 2) {
        $bytes[$i/2] = [convert]::ToByte($hexString.Substring($i,2), 16)
    }

    return [Text.Encoding]::Unicode.GetString($bytes)
}

Fetching Group Descriptors

Although it's incredibly easy to fetch security permissions for individual users, obtaining the permissions for user groups requires a special descriptor. To make them easier to work with, I'll grab all the security groups in the organization and map them into a simple lookup table:

function Get-SecurityDescriptors() {

    $lookup = @{}
    
    $decriptors = az devops security group list --scope organization --query "graphGroups[]" | ConvertFrom-Json
    
    $descriptors | ForEach-Object { $lookup.Add( $_.principalName, $_.descriptor) }
    
    return $descriptors
}

Apply Branch Security

Given that we'll call the function several times, we'll wrap it in a method to make it easier to use.

$namespaceId = "2e9eb7ed-3c0a-47d4-87c1-0ffdd275fd87"

function Set-BranchSecurity( $descriptor, $token, $allow = 0 , $deny = 0) {

    $result = az devops security permission update `
                --id $namespaceId `
                --subject $descriptor `
                --token $token `
                --allow-bit $allow `
                --deny-bit $deny `
                --only-show-errors | ConvertFrom-Json
}

Putting it All Together

This crude example shows how to apply the guidance laid out in the Require branches to be created in folders article, to a specific repository:

$project   = "projectName"
$projectId = az devops project list --query "value[?name=='$project'].id | [0]" | ConvertFrom-Json

# grab the first repo in the project
$repoId    = az repos list --project $project --query [0].id | ConvertFrom-Json

$groups = Get-SecurityDescriptors
$administrators = $groups[ "[$project]\\Project Administrators" ]
$contributors   = $groups[ "[$project]\\Contributors" ]

# a simple array of tokens we can refer to
$tokens = @(
                Get-RepoSecurityToken( $projectId, $repoId ),             # repo - 0
                Get-RepoSecurityToken( $projectId, $repoId, "main" ),     # main - 1
                Get-RepoSecurityToken( $projectId, $repoId, "releases"),  # releases/* - 2
                Get-RepoSecurityToken( $projectId, $repoId, "feature"),   # feature/* - 3
                Get-RepoSecurityToken( $projectId, $repoId, "users")      # users/* - 4
           )
           
$CreateBranch = 16


# prevent contributors from creating branches at the root of the repository
Set-BranchSecurity $contributors, $tokens[0], -deny $CreateBranch

# limit users to only create feature and user branches
Set-BranchSecurity $contributors, $tokens[3], -allow $CreateBranch
Set-BranchSecurity $contributors, $tokens[4], -allow $CreateBranch

# restrict who can create a release
Set-BranchSecurity $admins, $token[2], -allow $CreateBranch

# allow admins to recreate master/main
Set-BranchSecurity $admins, $token[1], -allow $CreateBranch

To improve upon this, you could describe each of these expressions as a ruleset in a JSON format and apply them to all the repositories in a project.

Some considerations:

  • Granting Build Service accounts permissions to create Tags
  • Empowering the Project Collection Administrators with the ability to override branch policy
  • Empowering certain Teams or Groups with the ability to delete feature or user branches

Happy coding!

Friday, August 07, 2020

Running Azure DevOps CLI from an Azure Pipeline

pipelines

Having automation to perform common tasks is great. Having that automation run on a regular basis in the cloud is awesome.

Today, I'd like to expand upon the sweet Azure CLI script to manage Azure DevOps User Licenses I wrote and put it in a Azure Pipeline. The details of that automation script are outlined in my last post, so take the time to check that out if you're interested, but to recap: my azure cli script activates and deactivates Azure DevOps user licenses if they’re not used. Our primary focus in this post will outline how you can configure your pipeline to run your az devops automation on a reoccurring schedule.

Table of Contents

About Pipeline Security

When our pipelines run, they operate by default using a project-specific user account: <Project Name> Build Service (<Organization Name>). For security purposes, this account is restricted to information within the Project.

If your pipelines need to access details beyond the current Project they reside in, for example if you a pipeline that needs access to repositories in other projects, you can configure the Pipeline to use the Project Collection Build Service (<Organization Name>). This change is subtly made by toggling off the "Limit job authorization scope to current project for non-release pipelines"  (Project Settings -> Pipelines : Settings)

limit-job-scope

In both Project or Collection level scenarios, the security context of the build account is made available to our pipelines through the $(System.AccessToken) variable. There's a small trick that's needed to make the access token available to our PowerShell scripts and I'll go over this later. But for the most part, if you're only accessing information about pipelines, code changes or details about the project, the supplied Access Token should be sufficient. In scenarios where you're trying to alter elements in the project, you may need to grant some additional permissions to the build service account.

However, for the purposes of today's discussion, we want to modify user account licenses which requires the elevated permissions of a Project Collection Administrator. I need to stress this next point: do not place the Project Collection Build Service in the Project Collection Administrators group. You're effectively granting any pipeline that uses this account full access to your organization. Do not do this. Here by dragons.

Ok, so if the $(System.AccessToken) doesn't have the right level of access, we need an alternate access token that does.

Setup a PAT Token

Setting up Personal Access Tokens is a fairly common activity, so I'll refer you to this document on how the token is created. As we are managing users and user licenses, we need a PAT Token created by a Project Collection Administrator with the Member Entitlement Management scope:

pat-token-member-entitlement-management

Secure Access to Tokens

Now that we have the token that can manage user licenses, we need to put it somewhere safe. Azure DevOps offers a few good options here, each with increasing level of security and complexity:

My personal go-to are Variable Groups because they can be shared across multiple pipelines. Variable Groups also have their own Access Rights, so the owner of variable group must authorize which pipeline and users are allowed to use your secrets.

For our discussion, we'll create a variable group "AdminSecrets" with a variable "ACCESS_TOKEN".

Create the Pipeline

With our security concerns locked down, let's create a new pipeline (Pipelines -> Pipelines -> New Pipeline) with some basic scaffolding that defines both the machine type and access to our variable group that has my access token.

name: Manage Azure Licenses

trigger: none

pool:
  vmimage: 'ubuntu-latest'

variables:
 - group: AdminSecrets

I want to call out that by using a Linux machine, we're using PowerShell Core. There are some subtle differences between PowerShell and PowerShell Core, so I would recommend that you always write your scripts locally against PowerShell Core.

Define the Schedule

Next, we'll setup the schedule for the pipeline using a cron job schedule syntax.

We'll configure our pipeline to run every night as midnight:

schedules:
  # run at midnight every day
  - cron: "0 0 * * *"
    displayName: Check user licenses (daily)
    branches:
      include:
        - master
    always: true

By default, schedule triggers only run if there are changes, so we need to specify "always: true" to have this script run consistently.

Authenticate Azure DevOps CLI using PAT Token

In order to invoke our script that uses az devops functions, we need to setup the Azure DevOps CLI to use our PAT Token. As a security restriction, Azure DevOps does not make secrets available to scripts so we need to explicitly pass in the value as an environment variable.

- script: |
    az extension add -n azure-devops
  displayName: Install Azure DevOps CLI
  
- script: |
    echo $(ADO_PAT_TOKEN) | az devops login
    az devops configure --defaults organization=$(System.CollectionUri)
  displayName: Login and set defaults
  env:
    ADO_PAT_TOKEN: $(ACCESS_TOKEN)

Run PowerShell Script from ADO

Now that our pipeline has the ADO CLI installed, we're authenticated using our secure PAT token, our last step is to invoke the powershell script. Here I'm using the pwsh task to ensure that PowerShell Core is used. The "pwsh" task is a shortcut syntax for the standard powershell task.

Our pipeline looks like this:

name: Manage Azure Licenses

trigger: none

schedules:
  # run at midnight every day
  - cron: "0 0 * * *"
    displayName: Check user licenses (daily)
    branches:
      include:
        - master
    always: true

pool:
  vmImage: 'ubuntu-latest'

variables:
- group: AdminSecrets

steps:
- script: |
    az extension add -n azure-devops
  displayName: Install Azure DevOps CLI
  
- script: |
    echo $(ADO_PAT_TOKEN) | az devops login
    az devops configure --defaults organization=$(System.CollectionUri)
  displayName: Login and set defaults
  env:
    ADO_PAT_TOKEN: $(ACCESS_TOKEN)

- pwsh: .\manage-user-licenses.ps1
  displayName: Manage User Licenses

Combining with the Azure CLI

Keen eyes may recognize that my manage-users-licenses.ps1 from my last post also used the Azure CLI to access Active Directory, and because az login and az devops login are two separate authentication mechanisms, the approach described above won’t work in that scenario. To support this, we’ll also need:

  • A service-connection from Azure DevOps to Azure (a Service Principal with access to our Azure Subscription)
  • Directory.Read.All role assigned to the Service Principal
  • A script to authenticate us with the Azure CLI.

The built-in AZ CLI Task is probably our best option for this, as it provides an easy way to work with our Service Connection. However, because this task clears the authentication before and after it runs, we have to change our approach slightly and execute our script logic within the script definition of this task. The following shows an example of how we can use both the Azure CLI and the Azure DevOps CLI in the same task:

- task: AzureCLI@2
  inputs:
    azureSubscription: 'my-azure-service-connection'
    scriptType: 'pscore'
    scriptLocation: 'inlineScript'
    inlineScript: |
       echo $(ACCESS_TOKEN) | az devops login
       az devops configure --defaults organization=$(SYSTEM.COLLECTIONURI) project=$(SYSTEM.TEAMPROJECT)
       az pipelines list
       az ad user list

If we need to run multiple scripts or break-up the pipeline into smaller tasks as I illustrated above, we’ll need a different approach where we have more control over the authenticated context. I can dig into this in another post.

Wrap Up

As I’ve outlined in this post, we can take simple PowerShell automation that leverages the Azure DevOps CLI and run it within an Azure Pipeline securely and on a schedule.

Happy coding.