Converting VSTS build definitions to YAML

Visual Studio Team System (VSTS) has a new feature in preview to allow builds to be defined in code and therefore in source control.

I’m not going to try to explain the why or how to do this. These blog posts already do a great job of that:

https://blog.kloud.com.au/2017/11/26/vsts-build-definitions-as-yaml-part-1-what-and-why/
https://blog.kloud.com.au/2017/11/26/vsts-build-definitions-as-yaml-part-2-how/

I just want to talk about converting existing builds to YAML and some of the problems I had. The process I followed was:

1. Created a file called .vsts-ci.yml in the root of the repository.
2. Export YAML from existing build (open the build definition and click “View YAML”) and put it in the .vsts-ci.yml.
3. Commit changes to the branch (so that there’s a record of the original definition).
4. Created a temporary build to test the YAML definition and fix any issues.
5. Commit changes to the branch.
6. Merge changes with other branches – this is a slightly awkward step to get to a common starting point to allow us to switch over to the new build definition.
7. Configure a new build to match existing build e.g. repo, triggers, retention etc.
8. Retire existing build but keep it until the new build beds in:
   Deactivate continuous integration.
   Rename with “-Deprecated” appended.
9. Change release artifacts to be the new build.

This process has worked well. However, the main problem I had was the exported YAML simply not working. This feature is in preview and Microsoft have been very responsive to issues I’ve raised at https://github.com/Microsoft/vsts-agent but still a bit disappointing that they don’t seem to have a good suite of round-trip tests.

Anyway, here are the changes I made from the exported YAML.

1. The following code was removed from the definition. There are some bugs reported around this in https://github.com/Microsoft/vsts-agent, the documentation is poor and it is very unclear in VSTS what effect this has on configuration defined in “Get sources”. I decided it was better to leave this out and be clear that the “Get sources” definition is entirely in
   VSTS.

   resources:
   - repo: self
     clean: true
   

2. I removed “condition: succeeded()” from the queue configuration. Leaving this in resulted in the error “Unexpected value ‘condition'”

   queue:
     name: Hosted VS2017
     condition: succeeded()
     demands: 
     - Cmd
     - DotNetFramework
   

3. I wrapped All displayName values in single quotes. This avoids errors such as “Mapping values are not allowed in this context” when the value contains a colon e.g.

   displayName: Publish Artifact: Deploy

4. I set the name to match the what we had configured in Options > Build number format. This populates the $(Build.BuildNumber) built in variable.

   name: $(Build.DefinitionName)-$(date:yyyyMMdd)$(rev:.r) # Populates $(Build.BuildNumber)
   

5. Corrected the PowerShell format.
   From:
   - powershell: . 'Deploy/Build.ps1' true
to
   - powershell: 'Deploy/Build.ps1'
6. Fixed indenting of projects in dotnet command tasks to fix errors like “While parsing a block mapping, did not find expected key.”
   From:

     - task: DotNetCoreCLI@2
       displayName: dotnet restore
       inputs:
         command: restore
         projects: |  
        **/Project1.csproj
        **/Project2.csproj
         feedsToUse: config
         nugetConfigPath: NuGet.config
   

   to

     - task: DotNetCoreCLI@2
       displayName: 'dotnet restore'
       inputs:
         command: restore
         projects: |  
           **/Project1.csproj
           **/Project2.csproj
         feedsToUse: config
         nugetConfigPath: NuGet.config
   

7. Fixed up Process Parameters. Any parameter defined on teh Build Process resulted in the message below. I changed these to be variables in the YAML.
   #Your build pipeline references an undefined variable named ‘Parameters.RestoreBuildProjects’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972

Notes

• It is also possible to specify build triggers in the YAML as below. These were not included in the exported YAML but I have tested the code below and it works well. At the moment I am going with the option to “Override YAML Continuous Integration trigger from here” in the Triggers tab of the build definition in VSTS. What triggers a build seems separate from what needs to be done to build a repository so I decided it should not be in the build configuration in source control.  Opinions welcomed regarding this

  trigger:
    branches:
      include:
      - master
      - develop

   

• Release definitions not supported yet but seem to be on the road map. I suspect it will be a while though.
• To use YAML you must have the “Build YAML definitions” preview feature enabled on your account – https://docs.microsoft.com/en-us/vsts/project/navigation/preview-features?view=vsts
• Must use spaces (2 or 4), not tabs, for indentation.
• For specific task help look at https://github.com/Microsoft/vsts-tasks/blob/master/Tasks. Find the task and look at task.json for the definition. The helpMarkDown attribute usually has a link to documentation with an example.
• Beware: variables in vsts are overridden by the yaml file.

Useful resources

Introductions:

• https://docs.microsoft.com/en-us/vsts/pipelines/get-started-yaml
• https://docs.microsoft.com/en-us/vsts/pipelines/build/yaml
• https://docs.microsoft.com/en-gb/vsts/pipelines

Documentation:

• https://github.com/Microsoft/vsts-agent/blob/master/docs/preview/yamlgettingstarted.md
• https://docs.microsoft.com/en-gb/vsts/pipelines/tasks & https://docs.microsoft.com/en-gb/vsts/pipelines for task syntax and documentation.
• http://yaml.org/refcard.html

Examples:

• https://www.marcusfelling.com/blog/2017/vsts-yaml-builds/

Tools:

• https://codebeautify.org/yaml-validator / http://www.yamllint.com – useful for validating before a commit and sometimes gives better error message than vsts.