the urban canuk, eh: Azure DevOps CLI Examples

I've always been a fan of Azure DevOps's extensive REST API -- it's generally well documented, consistent and it seems like you can do pretty much anything you can do from within the web-interface. As much as I love the API, I hate having to bust it out. Nowadays, my new go to tool is the Azure DevOps CLI.

The Azure DevOps CLI is actually an extension of the Azure CLI. It contains a good number of common functions that you would normally use on a daily basis, plus features that I would normally rely on the REST API for. Its real power is unlocked when it's combined with your favourite scripting language. I plan to write a few posts on this topic, so stay tuned, but for today, we'll focus on getting up and running plus some cool things you can do with the tool.

Installation

Blurg. I hate installation blog posts. Let's get this part over with:

choco install azure-cli -y
az extension add --name azure-devops

Whew. If you don't have Chocolatey installed, go here: https://chocolatey.org/install

Get Ready

Ok, so we're almost there. Just a few more boring bits. First we need to login:

az login --allow-no-subscription

A quick note on the above statement. There are a number of different login options available but I've found az login with the --allow-no-subscription flag supports the majority of use cases. It'll launch a web-browser and require you to login as you normally would, and the --allow-no-subscription supports scenarios where you have access to the AD tenant to login but you don't necessarily have a subscription associated to your user account, which is probably pretty common for most users who only have access to Azure DevOps.

This next bit let's us store some commonly used parameters so we don't have to keep typing them out.

az devops configure --defaults organization=https://dev.azure.com/<organization>

In case your curious, this config file is stored in %UserProfile%\.azure\azuredevops\config

Our First Command

Let's do something basic, like getting a list of all projects:

az devops project list

If we've configured everything correctly, you should see a boatload of JSON fly by. The CLI supports different options for output, but JSON works best when paired with our automation plus there's some really cool things we can do with the result output by passing a JMESPath statement using the --query flag.

Understanding JMESPath

The JavaScript kids get all the cool tools. There are probably already a few dozen different ways of querying and manipulating JSON data, and JMESPath (pronounced James Path) is no different. The syntax is a bit confusing at first and it takes a little bit of tinkering to master it. So let's do some tinkering.

The best way to demonstrate this is to use the JSON output from listing our projects. Our JSON looks something like this:

{
   "value": [
     {
        "abbreviation": null,
        "defaultTeamImageUrl": null,
        "description": null,
        "id": "<guid>",
        "lastUpdateTime": "<date>",
        "name": "Project Name",
        "revision": 89,
        "state": "wellFormed",
        "url": "<url>",
        "visibility": "private"
     },
     ...
   ]
}

It's a single object with a property called "value" that contains an array. Let's do a few examples...

Return the contents of the array

Assuming that we want the details of the projects and not the outside wrapper, we can discard the "value" property and just get it's contents, which is an array.

az devops project list --query "value[]"

[
    {
        "abbreviation": null,
        "defaultTeamImageUrl": null,
        "description": null,
        "id": "<guid>",
        "lastUpdateTime": "<date>",
        "name": "Project Name",
        "revision": 89,
        "state": "wellFormed",
        "url": "<url>",
        "visibility": "private"
    },
    ...
]

Return just the first element

Because the "value" property is an array, we can get the first element.

az devops project --query "value[0]"

{
    "abbreviation": null,
    "defaultTeamImageUrl": null,
    "description": null,
    "id": "<guid>",
    "lastUpdateTime": "<date>",
    "name": "Project Name",
    "revision": 89,
    "state": "wellFormed",
    "url": "<url>",
    "visibility": "private"
}

You can also specify ranges:

[:2] = everything up to the 3rd item
[1:3] = from the 2nd up to the 4th items
[1:] = everything from the 2nd item

Return an array of properties

If we just wanted the id property of each element in the array, we can specify the property we want. The result assumes there are only 4 projects.

az devops project --query "value[].id"

[
    "<guid>",
    "<guid>",
    "<guid>",
    "<guid>"
]

Return specific properties

This is where JMESPath gets a tiny bit odd. In order to get just a handful of properties we need to do a "projection" which is basically like stating what structure you want the JSON result to look like. In this case, we're mapping the id and name property to projectId and projectName in the result output.

az devops project --query "value[].{ projectId:id, projectName:name }"

[
    {
        "projectId": "<guid>",
        "projectName": "Project 1"
    },
    {
        "projectId": "<guid>",
        "projectName": "Project 2"
    },
    ...
]

Filter the results

Here's where things get really interesting. We can put functions inside the JMESPath query to filter the results. This allows us to mix and match the capabilities of the API with the output filtering capabilities of JMESPath. This returns only the projects that are public.

az devops project list --query "value[?visibility=='public'].{ id:id, name:name }"

[
    {
        "id": "<guid>",
        "name": "Project 3"
    }
]

We could have also written this as:

--query "value[?contains(visibility,'private')].{id:id, name:name}"

Piping the results

In the above example, JMESPath assumes that the results will be an array. We can pipe the result to further refine it. In this case, we want just the first object in the resulting array.

az devops project list --query "value[?visibility=='private'].{ id:id, name:name} | [0]"

{
   "id": "<guid>",
   "name": "Project 3"
}

Piping can improve the readability of the query similar to a functional language. For example, the above could be written as a filter, followed by a projection, followed by a selection.

--query "value[?contains(visibility,'private')] | [].{id:id, name:name} | [0]"

Wildcard searches

Piping the results becomes especially important if we want just the single value of a wildcard search. For this example, I need a different JSON structure, specifically a security descriptor:

[
  {
    "acesDictionary": {
      "Microsoft.IdentityModel.Claims.ClaimsIdentity;<dynamic-value>": {
        "allow": 16,
        "deny": 0,
        "descriptor": "Microsoft.IdentityModel.Claims.ClaimsIdentity;<dynamic-value>",
        "extendedInfo": {
          "effectiveAllow": 32630,
          "effectiveDeny": null,
          "inheritedAllow": null,
          "inheritedDeny": null
        }
      }
    },
    "includeExtendedInfo": true,
    "inheritPermissions": true,
    "token": "repoV2"
  }
]

In this structure, i'm interested in getting the "allow" and "deny" and "token" values but the first element in the acesDictionary contains a dynamic value. We can use a wildcard "*" to substitute for properties we don't know at runtime.

Let's try to isolate that "allow". The path would seem like [].acesDictionary.*.allow but because JMESPath has no idea if this is a single element, so it returns an array:

If we pipe the result, [].acesDictionary.*.allow | [0] we'll get a single value.

[
    16
]

Following suit and jumping ahead a bit so that I can skip to the answer, I can grab the "allow", "deny" and "token" with the following query. At this point, I trust you can figure this out using by referencing all the examples I've provided. The query looks like:

--query "[].{allow:acesDictionary.*.allow | [0], deny:acesDictionary.*.deny | [0], token:token } | [0]"

{
    "allow": 16,
    "deny": 0,
    "token": "repoV2"
}

Ok! That is waay too much JMESPath. Let's get back on topic.

Using the Azure DevOps CLI

The Azure DevOps CLI is designed with commands and subcommands and has a few entry points. At each level, there are the obvious inclusions (list, add, delete, update, show), but there are a few additional commands per level.

az devops

admin

banner

extension
project
security

group
permission

namespace

service-endpoint

azurerm
github

team
user
wiki

page

az pipelines

agent
build

definition
tag

folder
pool
release

definition

runs

artifact
tag

variable
variable-group

az boards

area

project
team

iteration

project
team

work-item

relation

az repos

import
policy

approver-count
build
case-enforcement
comment-required
file-size
merge-strategy
required-reviewer
work-item-linking

policy
reviewer
work-item

az artifacts

universal

I won’t go into all of these commands and subcommands, I can showcase a few of the ones I’ve used the most recently…

List of Projects

az devops project list --query "value[].{id:id, name:name}"

List of Repositories

az repos list --query "[].{id:id, defaultBranch:defaultBranch, name:name}"

List of Branch Policies

az repos policy list --project <name> --query "[].{name: type.displayName, required:isBlocking, enabled:isEnabled, repository:settings.scope[0].repositoryId, branch:settings.scope[0].refName}"

Service Connections

az devops service-endpoint list --project <name> --query "[].name"

One More Thing

So while the az devops cli is pretty awesome, it has a hidden gem. If you can't find a supporting command in the az devops cli, you can always call the REST API directly from the tool using the az devops invoke command. There's a bit of hunting through documentation and available endpoints to find what you're looking for, but you can get a full list of what's available using the following:

az devops invoke --query "[?contains(area,'build')]"
az devops invoke --query "[?area=='build' && resourceName=='timeline']"

[
  {
    "area": "build",
    "id": "8baac422-4c6e-4de5-8532-db96d92acffa",
    "maxVersion": 6.0,
    "minVersion": 2.0,
    "releasedVersion": "5.1",
    "resourceName": "Timeline",
    "resourceVersion": 2,
    "routeTemplate": "{project}/_apis/{area}/builds/{buildId}/{resource}/{timelineId}"
  }
]

We can invoke this REST API call by passing in the appropriate area, resource, route and query-string parameters. Assuming I know the buildId of a recent pipeline run, the following shows me the state and status of all the stages in that build:

az devops invoke 
    --area build 
    --resource Timeline 
    --route-parameters project=myproject buildId=2058 timelineid='' 
    --query "records[?contains(type,'Stage')].{name:name, state:state, result:result}"

Tip: the route and query parameters specified in the routeTemplate are case-sensitive.

More to come

Today's post outlined how to make sense out of JMESPath and some cool features of the Azure DevOps CLI. My next few posts I'll dig deeper into using the cli in your favourite scripting tool

Happy coding.

the urban canuk, eh

Monday, July 27, 2020

Azure DevOps CLI Examples