MicroBuild v2: Fix binary and package version metadata; refactor and improve scripts.

Binary and Package Versions
===========================

Binary File Version Metadata
----------------------------

Enables correct embedding of version string in file info. All binaries will now
embed File Version metadata as follows:

    <MAJOR>.<MINOR>.<PATCH>.0 (<branch>:<hash>.<build_high>.<build_low>.<push_date>)

Example:

    1.2.1.0 (release/1.2:9786fcdbcaa6963a5ccc6648b7aa9bd9dd51d9ec.00008.12122.161007-1756)

`MAJOR`, `MINOR`, and `PATCH` correspond to SemVer fields and will match the
version of ChakraCore the package corresponds to. The fourth field would have
been used to indicate security fixes (a.k.a. `QFE`) but because a fourth field
is not allowed in NuGet package version numbers, but is standard for versions of
binaries produced from VS projects, we are keeping the fourth field in the
binary data and locking it to 0, and omitting it from NuGet package version
numbers and release numbers from now on.

Release NuGet Package Versions
------------------------------

The format of version strings for released NuGet packages will continue to be:

    Microsoft.ChakraCore.<MAJOR>.<MINOR>.<PATCH>.nupkg

Example:

    Microsoft.ChakraCore.1.2.1.nupkg

Preview NuGet Package Versions
------------------------------

The format of version strings for preview NuGet packages will now be:

    Microsoft.ChakraCore.<MAJOR>.<MINOR>.<PATCH>-<SPECIAL>.nupkg

Where `<SPECIAL>` is:

    preview-<build_high>-<build_low>

This solves the problem of getting enough information into the package preview
version string (`<SPECIAL>`) to order builds correctly while still guaranteeing
that we fit the requirements for the `<SPECIAL>` field:

1. It must start with a letter
2. It can only contain the characters [a-zA-Z0-9-]
3. It must be 20 characters or less

Example:

    Microsoft.ChakraCore.1.2.1-preview-00008-12122.nupkg

Miscellaneous Version-Related Changes
-------------------------------------

Add `PATCH` version to `CommonDefines.h` so that it doesn't need to be manually
coded into builds. (By reducing user intervention for package versions we make
it more feasible to automatically release preview packages from multiple
branches.)

Extract `MAJOR`, `MINOR`, `PATCH`, and `QFE` version numbers from the source
code by grepping for values of the `#defines` that correspond to those values.

Refactorings and Script Parameters
==================================

Merge `pogo_build.ps1` into `run_build.ps1`. Passing the required pogo-related
flags to `run_build.ps1` produces a POGO build. No need to have separate
scripts.

Compute `ObjectDirectory` and use that for `Chakra.Generated.BuildInfo.props`
unless `TF_BUILD_BUILDDIRECTORY` is specified in the PATH.

Infer `$binpath`.

For many parameters that were previously specified as environment variables from
the environment of the build infrastructure, it is now possible to specify them
as script parameters directly, where script parameters will take precedence over
the values of the corresponding environment variables. This can be helpful when
running scripts manually where you want to quickly experiment with the value of
a variable without having to write it to the environment first. Since it is
sometime valuable to get that variable from the environment, if the script
parameter is unspecified, the value of the environment variable will be used
instead. Additionally, if any parameter is unspecified by either script
parameter or environment variable, a reasonable default is encoded in the script
logic.

In order of precedence, all values will be determined by checking for values in
the following order, if applicable:

1. Script parameter
2. Environment variable
3. Script logic to retrieve value
4. Default value

Add parameters with corresponding environment variables to specify skipping
certain paths through the build scripts, which can save time when testing
certain changes by ignoring expensive but unimportant parts of the builds.
Specifying an environment value at the time the build is queued is easier than
modifying the build definition, so adding the environment variable option adds
convenience when testing with the production build definitions.

* `run_build.ps1 -skipPogo` or set `SKIP_POGO` to a non-empty string will skip
  the POGO phase of the build, if it would have been a POGO build. Useful if the
  build binaries produced at the correct locations is enough and you're most
  interested in testing the non-POGO-related aspects of the build scripts.
* `post_build.ps1 -skipTests` or set `SKIP_TESTS` to a non-empty string will
  skip running tests, which can be lengthy. This is useful if the quality of the
  binaries produced is not important and you're interested in testing other
  aspects of the build scripts.

Miscellaneous
-------------

Reduce dependencies on data from build server environments so that these scripts
can be used to perform local builds.

Update `UseValueOrDefault` to take any number of arguments.

Reduce dependency on `locate_msbuild.ps1`.

Add `$BuildSubtype` to 'not enough info' error.

Check for the existence of `nuget.exe` before trying to invoke it. This helps
when running in environments where NuGet is not installed or is not available.
Can avoid errors related to NuGet package restore when running on a build server
with a custom NuGet location and a specific NuGet Task separate from our
scripts. Keeping the NuGet invocation in our scripts allows us to still do local
builds without MicroBuild infrastructure configured on the dev machine (we don't
take a hard dependency on the aforementioned NuGet Task.

.gitignore

@@ -32,3 +32,5 @@ Build/Chakra.Core.VC.opendb
 test/benchmarks/*.txt
 test/benchmarks/*.dpl
 *.nupkg
+_DROP
+packages.config

Build/Common.Build.Default.props

@@ -58,7 +58,6 @@
     <PlatformPathNameAlt Condition="'$(Platform)'=='x64'">amd64</PlatformPathNameAlt>
   </PropertyGroup>
 
-
   <!-- Default output directories -->
   <PropertyGroup>
     <OutBaseDir Condition="'$(OutBaseDir)'!=''">$(OutBaseDir)\$(SolutionName)</OutBaseDir>
@@ -68,7 +67,11 @@
   </PropertyGroup>
 
   <!-- Import generated build info -->
-  <Import Project="$(TF_BUILD_BUILDDIRECTORY)\Chakra.Generated.BuildInfo.props" Condition="'$(TF_BUILD_BUILDDIRECTORY)' != '' AND exists('$(TF_BUILD_BUILDDIRECTORY)\Chakra.Generated.BuildInfo.props')" />
+  <PropertyGroup>
+    <ObjectDirectory Condition="'$(TF_BUILD_BUILDDIRECTORY)'!=''">$(TF_BUILD_BUILDDIRECTORY)</ObjectDirectory>
+    <ObjectDirectory Condition="'$(TF_BUILD_BUILDDIRECTORY)'==''">$(IntBaseDir)\obj\$(PlatformPathName.ToLower())_$(Configuration.ToLower())</ObjectDirectory>
+  </PropertyGroup>
+  <Import Project="$(ObjectDirectory)\Chakra.Generated.BuildInfo.props" Condition="'$(ObjectDirectory)'!='' AND exists('$(ObjectDirectory)\Chakra.Generated.BuildInfo.props')" />
 
   <!-- Output directories -->
   <PropertyGroup>

Build/scripts/finalize_build.ps1

@@ -101,7 +101,7 @@ $buildFlavorJson | Add-Member -type NoteProperty -name flavor -value $Env:BuildC
 $buildFlavorJson | Add-Member -type NoteProperty -name subtype -value $Env:BuildSubtype
 
 $buildFlavorJson | ConvertTo-Json | Write-Output
-$buildFlavorJson | ConvertTo-Json | Out-File $buildFlavorJsonFile -Encoding ascii
+$buildFlavorJson | ConvertTo-Json | Out-File $buildFlavorJsonFile -Encoding utf8
 
 #
 # Copy outputs to metadata directory

Build/scripts/init_build.ps1

@@ -12,23 +12,47 @@
 # before running the Pre-Build script.
 
 param (
+    [ValidateSet("x86", "x64", "arm", "")]
+    [string]$arch = "",
+    [ValidateSet("debug", "release", "test", "codecoverage", "")]
+    [string]$flavor = "",
+    [ValidateSet("default", "codecoverage", "pogo")]
+    [string]$subtype = "default",
+    [string]$buildtype,
+
     [string]$envConfigScript = "ComputedEnvironment.cmd",
 
     [string[]]$supportedPogoBuildTypes = @("x64_release", "x86_release"),
 
-    [Parameter(Mandatory=$True)]
+    [string]$verMajor = "",
+    [string]$verMinor = "",
+    [string]$verPatch = "",
+    [string]$verSecurity = "",
+
+    [string]$dropRoot,
+
+    [switch]$cleanBinDir,
+
     [string]$oauth
 )
 
-# If $Env:BuildType is specified, extract BuildPlatform and BuildConfiguration
-# Otherwise, if $Env:BuildPlatform and $Env:BuildConfiguration are specified, construct $BuildType
-$BuildPlatform = $Env:BuildPlatform
-$BuildConfiguration = $Env:BuildConfiguration
-$BuildType = $Env:BuildType
-$BuildSubtype = "default" # will remain as "default" or become e.g. "pogo", "codecoverage"
+#
+# Define values for variables based on parameters and environment variables
+# with default values in case the environment variables are not defined.
+#
+
+. $PSScriptRoot\util.ps1
+$gitExe = GetGitPath
+
+$BuildType = UseValueOrDefault $buildtype $Env:BuildType
+$BuildPlatform = UseValueOrDefault $arch $Env:BuildPlatform
+$BuildConfiguration = UseValueOrDefault $flavor $Env:BuildConfiguration
+$BuildSubtype = UseValueOrDefault $subtype $Env:BuildSubtype
 
-if (Test-Path Env:\BuildType) {
-    $BuildType = $Env:BuildType
+# If $BuildType is specified, extract BuildPlatform and BuildConfiguration
+# Otherwise, if $BuildPlatform and $BuildConfiguration are specified, construct $BuildType
+# $BuildSubtype will remain as "default" if not already specified, or become e.g. "pogo", "codecoverage"
+if ($BuildType) {
     $buildTypeSegments = $BuildType.split("_")
     $BuildPlatform = $buildTypeSegments[0]
     $BuildConfiguration = $buildTypeSegments[1]
@@ -42,12 +66,10 @@ if (Test-Path Env:\BuildType) {
         $BuildSubtype = "codecoverage" # keep information about codecoverage in the subtype
     }
 
-    if (-not ($BuildSubtype -in @("default","pogo","codecoverage"))) {
+    if (-not ($BuildSubtype -in @("default", "pogo", "codecoverage"))) {
         Write-Error "Unsupported BuildSubtype: $BuildSubtype"
     }
-} elseif ((Test-Path Env:\BuildPlatform) -and (Test-Path Env:\BuildConfiguration)) {
-    $BuildPlatform = $Env:BuildPlatform
-    $BuildConfiguration = $Env:BuildConfiguration
+} elseif ($BuildPlatform -and $BuildConfiguration) {
     $BuildType = "${BuildPlatform}_${BuildConfiguration}"
 } else {
     Write-Error (@"
@@ -56,67 +78,92 @@ if (Test-Path Env:\BuildType) {
         BuildType={0}
         BuildPlatform={1}
         BuildConfiguration={2}
+        BuildSubtype={3}
 
-"@ -f $Env:BuildType, $Env:BuildPlatform, $Env:BuildConfiguration)
+"@ -f $BuildType, $BuildPlatform, $BuildConfiguration, $BuildSubtype)
 
     exit 1
 }
 
+$CommitHash = UseValueOrDefault $Env:BUILD_SOURCEVERSION $(iex "${gitExe} rev-parse HEAD")
+
+$branchFullName  = UseValueOrDefault $Env:BUILD_SOURCEBRANCH $(iex "${gitExe} rev-parse --symbolic-full-name HEAD")
+
+$SourcesDirectory = UseValueOrDefault $Env:BUILD_SOURCESDIRECTORY $(GetRepoRoot)
+$BinariesDirectory = UseValueOrDefault (Join-Path $SourcesDirectory "Build\VcBuild")
+$ObjectDirectory = Join-Path $BinariesDirectory "obj\${BuildPlatform}_${BuildConfiguration}"
+
+$DropRoot = UseValueOrDefault $dropRoot $Env:DROP_ROOT (Join-Path $(GetRepoRoot) "_DROP")
+
 # set up required variables and import pre_post_util.ps1
 $arch = $BuildPlatform
 $flavor = $BuildConfiguration
 $OuterScriptRoot = $PSScriptRoot # Used in pre_post_util.ps1
 . "$PSScriptRoot\pre_post_util.ps1"
 
-$gitExe = GetGitPath
-
 $BuildName = ConstructBuildName -arch $BuildPlatform -flavor $BuildConfiguration -subtype $BuildSubtype
 
-$branch = $Env:BUILD_SOURCEBRANCH
-if (-not $branch) {
-    $branch = iex "$gitExe rev-parse --symbolic-full-name HEAD"
-}
-
-$BranchName = $branch.split('/',3)[2]
+$BranchName = $branchFullName.split('/',3)[2]
 $BranchPath = $BranchName.replace('/','\')
-$CommitHash = $Env:BUILD_SOURCEVERSION
+
 if (-not $CommitHash) {
-    $CommitHash = iex "$gitExe rev-parse HEAD"
+    $CommitHash = iex "${gitExe} rev-parse HEAD"
 }
+$CommitShortHash = $(iex "${gitExe} rev-parse --short $CommitHash")
 
-$Username = (iex "$gitExe log $CommitHash -1 --pretty=%ae").split('@')[0]
-$CommitDateTime = [DateTime]$(iex "$gitExe log $CommitHash -1 --pretty=%aD")
+$Username = (iex "${gitExe} log $CommitHash -1 --pretty=%ae").split('@')[0]
+$CommitDateTime = [DateTime]$(iex "${gitExe} log $CommitHash -1 --pretty=%aD")
 $CommitTime = Get-Date $CommitDateTime -Format yyMMdd.HHmm
 
 #
 # Get Build Info
 #
 
-$info = GetBuildInfo $oauth $CommitHash
+$buildPushDate = $null
+$buildPushIdString = $null
 
-$BuildPushDate = [datetime]$info.push.date
-$PushDate = Get-Date $BuildPushDate -Format yyMMdd.HHmm
+if (-not $oauth)
+{
+    $buildPushIdPart1 = 65535
+    $buildPushIdPart2 = 65535
+    $buildPushIdString = "65535.65535"
+    $buildPushDate = [DateTime]$CommitDateTime
+}
+else
+{
+    $info = GetBuildInfo $oauth $CommitHash
+    $_, $buildPushIdPart1, $buildPushIdPart2, $buildPushIdString = GetBuildPushId $info
+    $buildPushDate = [DateTime]$info.push.date
+}
+
+$PushDate = Get-Date $buildPushDate -Format yyMMdd.HHmm
+
+$VersionMajor       = UseValueOrDefault $verMajor       $Env:VERSION_MAJOR  (GetVersionField "CHAKRA_CORE_MAJOR_VERSION")       "0"
+$VersionMinor       = UseValueOrDefault $verMinor       $Env:VERSION_MINOR  (GetVersionField "CHAKRA_CORE_MINOR_VERSION")       "0"
+$VersionPatch       = UseValueOrDefault $verPatch       $Env:VERSION_PATCH  (GetVersionField "CHAKRA_CORE_PATCH_VERSION")       "0"
+$VersionSecurity    = UseValueOrDefault $verSecurity    $Env:VERSION_QFE    (GetVersionField "CHAKRA_CORE_VERSION_RELEASE_QFE") "0"
 
-$buildPushId, $buildPushIdPart1, $buildPushIdPart2, $buildPushIdString = GetBuildPushId $info
+$VersionString = "${VersionMajor}.${VersionMinor}.${VersionPatch}" # Only use MAJOR.MINOR.PATCH to align with SemVer
 
-$VersionMajor = UseValueOrDefault "$Env:VERSION_MAJOR" "1"
-$VersionMinor = UseValueOrDefault "$Env:VERSION_MINOR" "2"
-$VersionPatch = UseValueOrDefault "$Env:VERSION_PATCH" "0"
-$VersionQFE   = UseValueOrDefault "$Env:VERSION_QFE"   "0"
+$buildVersionString = "{0}-{1}" -f $buildPushIdPart1.ToString("00000"), $buildPushIdPart2.ToString("00000")
+$PreviewVersionString = "${VersionString}-preview-${buildVersionString}"
 
-$VersionString = "${VersionMajor}.${VersionMinor}.${VersionPatch}.${VersionQFE}"
-$PreviewVersionString = "${VersionString}-preview"
+$ShortBranch = "commit"
+if ($BranchName -eq "master") {
+    $ShortBranch = "master"
+} elseif ($BranchName.StartsWith("release")) {
+    $ShortBranch = $BranchName.replace("release/","")
+}
 
 # unless it is a build branch, subdivide the output directory by month
 if ($BranchPath.StartsWith("build")) {
     $YearAndMonth = ""
 } else {
-    $YearAndMonth = (Get-Date $BuildPushDate -Format yyMM) + "\"
+    $YearAndMonth = (Get-Date $buildPushDate -Format yyMM) + "\"
 }
 
 $BuildIdentifier = "${buildPushIdString}_${PushDate}_${Username}_${CommitHash}"
 $ComputedDropPathSegment = "${BranchPath}\${YearAndMonth}${BuildIdentifier}"
-$BinariesDirectory = "${Env:BUILD_SOURCESDIRECTORY}\Build\VcBuild"
 $ObjectDirectory = "${BinariesDirectory}\obj\${BuildPlatform}_${BuildConfiguration}"
 
 # Create a sentinel file for each build flavor to track whether the build is complete.
@@ -128,9 +175,9 @@ This could mean that the build is in progress, or that it was unable to run to c
 The contents of this directory should not be relied on until the build completes.
 "@
 
-$DropPath = Join-Path $Env:DROP_ROOT $ComputedDropPathSegment
+$DropPath = Join-Path $DropRoot $ComputedDropPathSegment
 New-Item -ItemType Directory -Force -Path $DropPath
-New-Item -ItemType Directory -Force -Path (Join-Path $Env:BUILD_SOURCESDIRECTORY "test\logs")
+New-Item -ItemType Directory -Force -Path (Join-Path $SourcesDirectory "test\logs")
 New-Item -ItemType Directory -Force -Path (Join-Path $BinariesDirectory "buildlogs")
 New-Item -ItemType Directory -Force -Path (Join-Path $BinariesDirectory "logs")
 
@@ -138,26 +185,33 @@ $FlavorBuildIncompleteFile = Join-Path $DropPath "${BuildType}.incomplete"
 
 if (-not (Test-Path $FlavorBuildIncompleteFile)) {
     ($buildIncompleteFileContentsString -f "Build of ${BuildType}") `
-        | Out-File $FlavorBuildIncompleteFile -Encoding Ascii
+        | Out-File $FlavorBuildIncompleteFile -Encoding utf8
 }
 
-$PogoConfig = $supportedPogoBuildTypes -contains "${Env:BuildPlatform}_${Env:BuildConfiguration}"
+$PogoConfig = $supportedPogoBuildTypes -contains "${BuildPlatform}_${BuildConfiguration}"
 
 # Write the $envConfigScript
 
 @"
 set BranchName=${BranchName}
+set ShortBranch=${ShortBranch}
 set BranchPath=${BranchPath}
 set YearAndMonth=${YearAndMonth}
 set BuildIdentifier=${BuildIdentifier}
 
-set buildPushIdString=${buildPushIdString}
+set VersionMajor=${VersionMajor}
+set VersionMinor=${VersionMinor}
+set VersionPatch=${VersionPatch}
+set VersionSecurity=${VersionSecurity}
+
+set BuildPushIdString=${buildPushIdString}
 set VersionString=${VersionString}
 set PreviewVersionString=${PreviewVersionString}
 set PushDate=${PushDate}
 set CommitTime=${CommitTime}
 set Username=${Username}
 set CommitHash=${CommitHash}
+set CommitShortHash=${CommitShortHash}
 
 set ComputedDropPathSegment=${ComputedDropPathSegment}
 set BinariesDirectory=${BinariesDirectory}
@@ -174,24 +228,30 @@ set FlavorBuildIncompleteFile=${FlavorBuildIncompleteFile}
 set PogoConfig=${PogoConfig}
 
 "@ `
-    | Out-File $envConfigScript -Encoding Ascii
+    | Out-File $envConfigScript -Encoding ASCII
 
 # Use the VSTS environment vars to construct a backwards-compatible VSO build environment
 # for the sake of reusing the pre-build and post-build scripts as they are.
 
 @"
-set TF_BUILD_SOURCEGETVERSION=LG:${branch}:${CommitHash}
+set TF_BUILD_SOURCEGETVERSION=LG:${branchFullName}:${CommitHash}
 set TF_BUILD_DROPLOCATION=${BinariesDirectory}
 
-set TF_BUILD_SOURCESDIRECTORY=${Env:BUILD_SOURCESDIRECTORY}
+set TF_BUILD_SOURCESDIRECTORY=${SourcesDirectory}
 set TF_BUILD_BUILDDIRECTORY=${ObjectDirectory}
 set TF_BUILD_BINARIESDIRECTORY=${BinariesDirectory}
 
+REM The following variables are only used for logging build metadata.
 set TF_BUILD_BUILDDEFINITIONNAME=${Env:BUILD_DEFINITIONNAME}
 set TF_BUILD_BUILDNUMBER=${Env:BUILD_BUILDNUMBER}
 set TF_BUILD_BUILDURI=${Env:BUILD_BUILDURI}
 "@ `
-    | Out-File $envConfigScript -Encoding Ascii -Append
+    | Out-File $envConfigScript -Encoding ASCII -Append
+
+# Print contents of $envConfigScript as a sanity check
+Write-Output ""
+Get-Content $envConfigScript | Write-Output
+Write-Output ""
 
 # Export VSO variables that can be consumed by other VSO tasks where the task
 # definition in VSO itself needs to know the value of the variable.
@@ -216,10 +276,10 @@ Write-Output "Setting VSO variable VSO_VersionString = ${VersionString}"
 Write-Output "##vso[task.setvariable variable=VSO_VersionString;]${VersionString}"
 
 #
-# Clean up files that might have been left behind from a previous build.
+# Optionally ($cleanBinDir): clean up files that might have been left behind from a previous build.
 #
 
-if ((Test-Path Env:\BUILD_BINARIESDIRECTORY) -and (Test-Path "$Env:BUILD_BINARIESDIRECTORY"))
+if ($BinariesDirectory -and (Test-Path "$BinariesDirectory") -and $cleanBinDir)
 {
-    Remove-Item -Verbose "${Env:BUILD_BINARIESDIRECTORY}\*" -Recurse
+    Remove-Item -Verbose "${BinariesDirectory}\*" -Recurse
 }

Build/scripts/pogo_build.ps1

@@ -1,143 +0,0 @@
-#-------------------------------------------------------------------------------------------------------
-# Copyright (C) Microsoft. All rights reserved.
-# Licensed under the MIT license. See LICENSE.txt file in the project root for full license information.
-#-------------------------------------------------------------------------------------------------------
-
-# Use this script to run a POGO build for the given BuildType (arch, flavor, subtype)
-
-param (
-    [ValidateSet("x86", "x64", "arm")]
-    [Parameter(Mandatory=$True)]
-    [string]$arch,
-
-    # We do not use ValidateSet here because this $flavor param is used to name the BuildConfuration
-    # from the solution file. MsBuild will determine whether it is valid.
-    [Parameter(Mandatory=$True)]
-    [string]$flavor,
-
-    [ValidateSet("default", "codecoverage", "pogo")]
-    [string]$subtype = "pogo",
-
-    [Parameter(Mandatory=$True)]
-    [string]$solutionFile,
-
-    [switch]$clean,
-
-    # $binDir will be inferred if not provided.
-    [string]$binDir = "",
-    [string]$buildlogsSubdir = "buildlogs",
-
-    # Assume NuGet is on the path, otherwise the caller must specify an explicit path.
-    [string]$nugetExe = "NuGet.exe",
-
-    [string]$logFile = "",
-
-    #
-    # POGO training parameters
-    #
-
-    [string[]]$scenarios = @(),
-
-    [Parameter(Mandatory=$True)]
-    [string]$binpath,
-
-    [string]$binaryName = "ch.exe"
-)
-
-#
-# Configure logging
-#
-
-$OuterScriptRoot = $PSScriptRoot
-. $PSScriptRoot\pre_post_util.ps1
-
-$buildName = ConstructBuildName -arch $arch -flavor $flavor -subtype $subtype
-
-if (($logFile -eq "") -and (Test-Path Env:\TF_BUILD_BINARIESDIRECTORY)) {
-    $logFile = "${Env:TF_BUILD_BINARIESDIRECTORY}\logs\pogo_build.${buildName}.log"
-}
-
-if (($logFile -ne "") -and (Test-Path $logFile)) {
-    Remove-Item $logFile -Force
-}
-
-#
-# Only continue with this build if it is a valid pogo build configuration
-#
-
-if ($subtype -ne "pogo") {
-    WriteMessage "This build's subtype is not pogo (subtype: $subtype). Skipping build."
-    exit 0
-}
-
-if ($scenarios.Length -eq 0) {
-    WriteMessage "No training scenarios selected. Please specify training scenarios using the -scenarios parameter."
-    exit 0
-}
-
-#
-# NuGet restore
-#
-
-ExecuteCommand "& $nugetExe restore $solutionFile -NonInteractive"
-
-#
-# Setup
-#
-
-$msbuildExe = Locate-MSBuild
-if (-not $msbuildExe) {
-    WriteErrorMessage "Error: Could not find msbuild.exe -- exiting (1)..."
-    exit 1
-}
-
-$binDir = UseValueOrDefault "$binDir" "${Env:BinariesDirectory}" "${Env:BUILD_SOURCESDIRECTORY}\Build\VcBuild"
-$buildlogsPath = Join-Path $binDir $buildlogsSubdir
-
-$defaultParams = "$solutionFile /nologo /m /nr:false /p:platform=`"${arch}`" /p:configuration=`"${flavor}`""
-$loggingParams = @(
-    "/fl1 `"/flp1:logfile=${buildlogsPath}\build.${buildName}.log;verbosity=normal`"",
-    "/fl2 `"/flp2:logfile=${buildlogsPath}\build.${buildName}.err;errorsonly`"",
-    "/fl3 `"/flp3:logfile=${buildlogsPath}\build.${buildName}.wrn;warningsonly`"",
-    "/verbosity:normal"
-    ) -join " "
-
-$targets = ""
-if ($clean) {
-    $targets += "`"/t:Clean,Rebuild`""
-}
-
-$binary = Join-Path $binpath $binaryName
-
-#
-# Build
-#
-
-function Build($targets="", $pogoParams) {
-    $buildCommand = "& `"$msbuildExe`" $targets $defaultParams $loggingParams $pogoParams"
-    ExecuteCommand "$buildCommand"
-    if ($global:LastExitCode -ne 0) {
-        WriteErrorMessage "Failed msbuild command:`n$buildCommand`n"
-        WriteErrorMessage "Build failed. Exiting..."
-        exit 1
-    }
-}
-
-Build -pogoParams "`"/p:POGO_TYPE=PGI`"" -targets "$targets"
-
-$scenariosParamValue = $scenarios -join ','
-$pogoTrainingCommand = "& `"${PSScriptRoot}\pgo\pogo_training.ps1`" -arch $arch -flavor $flavor -subtype $subtype -binary $binary -scenarios $scenariosParamValue"
-ExecuteCommand "$pogoTrainingCommand"
-
-Build -pogoParams "`"/p:POGO_TYPE=PGO`""
-
-#
-# Clean up
-#
-
-if (("$binpath" -ne "") -and (Test-Path $binpath)) {
-    # remove *.pgc, *.pgd, and pgort*
-    Get-ChildItem -Recurse -Path $binpath "*" `
-        | ? { $_.Name -match "(.*\.pg[cd]|pgort.*)" } `
-        | % { Remove-Item -Force $_.FullName }
-}

Build/scripts/post_build.ps1

@@ -37,9 +37,15 @@ param (
     [string[]]$pogo = @(),
     [string]$pogoscript = "",
 
-    [switch]$noaction
+    #
+    # Skip flags
+    #
+
+    [switch]$skipTests # or set $Env:SKIP_TESTS before invoking build
 )
 
+$skipTests = $skipTests -or (Test-Path Env:\SKIP_TESTS)
+
 $global:exitcode = 0
 
 if ($arch -eq "*") {
@@ -105,7 +111,9 @@ if ($arch -eq "*") {
         }
 
         # run tests
-        ExecuteCommand("$bvtcmdpath -$arch$flavor")
+        if (-not $skipTests) {
+            ExecuteCommand("$bvtcmdpath -$arch$flavor")
+        }
     }
 
     # check prefast

Build/scripts/pre_build.ps1

@@ -20,7 +20,7 @@
 #   $Env:TF_BUILD_BUILDDIRECTORY    (a.k.a. $objpath)
 #   $Env:TF_BUILD_BINARIESDIRECTORY (a.k.a. $binpath)
 #
-# Optional information:
+# Optional information (metadata only):
 #   $Env:TF_BUILD_BUILDDEFINITIONNAME
 #   $Env:TF_BUILD_BUILDNUMBER
 #   $Env:TF_BUILD_BUILDURI
@@ -32,7 +32,6 @@ param (
     [Parameter(Mandatory=$True)]
     [ValidateSet("debug", "release", "test", "codecoverage")]
     [string]$flavor,
-
     [ValidateSet("default", "codecoverage", "pogo")]
     [string]$subtype = "default",
 
@@ -43,7 +42,6 @@ param (
 
     [string]$corePath = "core",
 
-    [Parameter(Mandatory=$True)]
     [string]$oauth
 )
 
@@ -108,12 +106,12 @@ if (Test-Path Env:\TF_BUILD_SOURCEGETVERSION)
 
     $info = GetBuildInfo $oauth $commitHash
 
-    $BuildDate = ([datetime]$info.push.date).toString("yyMMdd-HHmm")
+    $BuildDate = ([DateTime]$info.push.date).toString("yyMMdd-HHmm")
 
     $buildPushId, $buildPushIdPart1, $buildPushIdPart2, $buildPushIdString = GetBuildPushId $info
 
     # commit message
-    $command = "$gitExe log -1 --name-status -p $commitHash"
+    $command = "$gitExe log -1 --name-only -p $commitHash"
     $CommitMessageLines = iex $command
     $CommitMessage = $CommitMessageLines -join "`r`n"
 
@@ -157,12 +155,12 @@ $CommitMessage
     $changeJson | Add-Member -type NoteProperty -name PushIdPart2 -value $BuildPushIdPart2
     $changeJson | Add-Member -type NoteProperty -name PushIdString -value $BuildPushIdString
     $changeJson | Add-Member -type NoteProperty -name Username -value $Env:Username
-    $changeJson | Add-Member -type NoteProperty -name CommitMessage -value $CommitMessage
+    $changeJson | Add-Member -type NoteProperty -name CommitMessage -value $CommitMessageLines
 
     Write-Output "-----"
     Write-Output $outputJsonFile
     $changeJson | ConvertTo-Json | Write-Output
-    $changeJson | ConvertTo-Json | Out-File $outputJsonFile -Encoding Ascii
+    $changeJson | ConvertTo-Json | Out-File $outputJsonFile -Encoding utf8
 
     $buildInfoOutputDir = $objpath
     if (-not(Test-Path -Path $buildInfoOutputDir)) {

Build/scripts/pre_post_util.ps1

@@ -4,7 +4,6 @@
 #-------------------------------------------------------------------------------------------------------
 
 . "$PSScriptRoot\util.ps1"
-. "$PSScriptRoot\locate_msbuild.ps1"
 
 function WriteCommonArguments() {
     WriteMessage "  Source Path: $srcpath"
@@ -12,6 +11,17 @@ function WriteCommonArguments() {
     WriteMessage "Binaries Path: $binpath"
 }
 
+function GetVersionField($fieldname) {
+    $gitExe = GetGitPath
+    $query = "#define ${fieldname} (\d+)"
+    $line = (iex "${gitExe} grep -P ""${query}"" :/")
+    $matches = $line | Select-String $query
+    if ($matches) {
+        return $matches[0].Matches.Groups[1].Value
+    }
+    return ""
+}
+
 function GetBuildInfo($oauth, $commitHash) {
     # Get the git remote path and construct the REST API URI
     $gitExe = GetGitPath
@@ -21,7 +31,7 @@ function GetBuildInfo($oauth, $commitHash) {
     # Get the pushId and push date time to use that for build number and build date time
     $uri = ("{0}/commits/{1}?api-version=1.0" -f $remote, $commitHash)
     $oauthToken = Get-Content $oauth
-    $header = @{Authorization=("Basic {0}" -f $oauthToken) }
+    $header = @{ Authorization=("Basic {0}" -f $oauthToken) }
     $info = Invoke-RestMethod -Headers $header -Uri $uri -Method GET
 
     return $info

Build/scripts/run_build.ps1

@@ -3,54 +3,75 @@
 # Licensed under the MIT license. See LICENSE.txt file in the project root for full license information.
 #-------------------------------------------------------------------------------------------------------
 
-# Use this script to run a build command for the given BuildType (arch, flavor, subtype)
+# Use this script to run a build for the given BuildType (arch, flavor, subtype)
 
 param (
     [ValidateSet("x86", "x64", "arm")]
     [Parameter(Mandatory=$True)]
     [string]$arch,
-
     # We do not use ValidateSet here because this $flavor param is used to name the BuildConfuration
     # from the solution file. MsBuild will determine whether it is valid.
     [Parameter(Mandatory=$True)]
     [string]$flavor,
-
     [ValidateSet("default", "codecoverage", "pogo")]
     [string]$subtype = "default",
 
     [Parameter(Mandatory=$True)]
-    [string]$solutionFile = "",
+    [string]$solutionFile,
 
     [switch]$clean,
 
-    # $binDir will be inferred if not provided.
-    [string]$binDir = "",
+    [string]$binDir = "", # will be inferred if not provided.
     [string]$buildlogsSubdir = "buildlogs",
 
     # assume NuGet is on the path, otherwise the caller must specify an explicit path
     [string]$nugetExe = "NuGet.exe",
 
-    [string]$logFile = ""
+    [string]$logFile = "",
+
+    #
+    # Skip flags
+    #
+
+    [switch]$skipPogo, # or set $Env:SKIP_POGO before invoking build
+
+    #
+    # POGO training parameters
+    #
+
+    [string[]]$scenarios = @(),
+    [string]$binpath, # will be inferred if not provided
+    [string]$binaryName = "ch.exe"
 )
 
+#
+# Configure logging
+#
+
 $OuterScriptRoot = $PSScriptRoot
 . $PSScriptRoot\pre_post_util.ps1
+. $PSScriptRoot\locate_msbuild.ps1
+
+$buildName = ConstructBuildName -arch $arch -flavor $flavor -subtype $subtype
 
 if (($logFile -eq "") -and (Test-Path Env:\TF_BUILD_BINARIESDIRECTORY)) {
     $logFile = "${Env:TF_BUILD_BINARIESDIRECTORY}\logs\run_build.${Env:BuildName}.log"
-    if (Test-Path -Path $logFile) {
-        Remove-Item $logFile -Force
-    }
+}
+
+if (($logFile -ne "") -and (Test-Path $logFile)) {
+    Remove-Item $logFile -Force
 }
 
 #
 # NuGet restore
 #
 
-ExecuteCommand "& $nugetExe restore $solutionFile -NonInteractive"
+if (-not (Get-Command $nugetExe -ErrorAction SilentlyContinue)) {
+    ExecuteCommand "& $nugetExe restore $solutionFile -NonInteractive"
+}
 
 #
-# Setup and build
+# Setup
 #
 
 $msbuildExe = Locate-MSBuild
@@ -62,6 +83,12 @@ if (-not $msbuildExe) {
 $binDir = UseValueOrDefault "$binDir" "${Env:BinariesDirectory}" "${Env:BUILD_SOURCESDIRECTORY}\Build\VcBuild"
 $buildlogsPath = Join-Path $binDir $buildlogsSubdir
 
+$skipPogo = $skipPogo -or (Test-Path Env:\SKIP_POGO)
+
+if (("$binpath" -ne "") -or (-not (Test-Path $binpath))) {
+    $binpath = Join-Path $binDir "bin\${buildName}"
+}
+
 $defaultParams = "$solutionFile /nologo /m /nr:false /p:platform=`"${arch}`" /p:configuration=`"${flavor}`""
 $loggingParams = @(
     "/fl1 `"/flp1:logfile=${buildlogsPath}\build.${Env:BuildName}.log;verbosity=normal`"",
@@ -75,11 +102,59 @@ if ($clean) {
     $targets += "`"/t:Clean,Rebuild`""
 }
 
-if ($subtype -eq "codecoverage") {
-    $subtypeParams = "/p:ENABLE_CODECOVERAGE=true"
+#
+# Build
+#
+
+function Build($targets="", $extraParams) {
+    $buildCommand = "& `"$msbuildExe`" $targets $defaultParams $loggingParams $extraParams"
+    ExecuteCommand "$buildCommand"
+    if ($global:LastExitCode -ne 0) {
+        WriteErrorMessage "Failed msbuild command:`n$buildCommand`n"
+        WriteErrorMessage "Build failed. Exiting..."
+        exit 1
+    }
 }
 
-$buildCommand = "& `"$msbuildExe`" $targets $defaultParams $loggingParams $subtypeParams"
-ExecuteCommand "$buildCommand"
+if ($subtype -eq "pogo") {
+    if ($scenarios.Length -eq 0) {
+        WriteMessage "No training scenarios selected for pogo build. Please specify training scenarios using the -scenarios parameter."
+        exit 1
+    }
+
+    Build -extraParams "`"/p:POGO_TYPE=PGI`"" -targets "$targets"
+
+    if (-not $skipPogo) {
+        $scenariosParamValue = $scenarios -join ','
+        $binary = Join-Path $binpath $binaryName
+        if (($binary -ne "") -and (Test-Path $binary)) {
+            $pogoTrainingCommand = "& `"${PSScriptRoot}\pgo\pogo_training.ps1`" -arch $arch -flavor $flavor -subtype $subtype -binary $binary -scenarios $scenariosParamValue"
+            ExecuteCommand "$pogoTrainingCommand"
+        } else {
+            WriteMessage "Binary not found at `"$binary`". Exiting..."
+            exit 1
+        }
+
+        Build -extraParams "`"/p:POGO_TYPE=PGO`""
+    }
+} else {
+    $subtypeParams = ""
+    if ($subtype -eq "codecoverage") {
+        $subtypeParams = "/p:ENABLE_CODECOVERAGE=true"
+    }
+
+    Build -extraParams $subTypeParams -targets $targets
+}
+
+#
+# Clean up
+#
+
+if (("$binpath" -ne "") -and (Test-Path $binpath)) {
+    # remove *.pgc, *.pgd, and pgort*
+    Get-ChildItem -Recurse -Path $binpath "*" `
+        | ? { $_.Name -match "(.*\.pg[cd]|pgort.*)" } `
+        | % { Remove-Item -Force $_.FullName }
+}
 
 exit $global:lastexitcode

Build/scripts/util.ps1

@@ -3,14 +3,13 @@
 # Licensed under the MIT license. See LICENSE.txt file in the project root for full license information.
 #-------------------------------------------------------------------------------------------------------
 
-function UseValueOrDefault($value, $defaultvalue, $defaultvalue2) {
-    if ($value -ne "") {
-        return $value
-    } elseif ($defaultvalue -ne "") {
-        return $defaultvalue
-    } else {
-        return $defaultvalue2
+function UseValueOrDefault() {
+    foreach ($value in $args) {
+        if ($value) {
+            return $value
+        }
     }
+    return ""
 }
 
 function GetGitPath() {
@@ -26,6 +25,11 @@ function GetGitPath() {
     return $gitExe
 }
 
+function GetRepoRoot() {
+    $gitExe = GetGitPath
+    return iex "$gitExe rev-parse --show-toplevel"
+}
+
 function WriteMessage($str) {
     Write-Output $str
     if ($logFile -ne "") {

bin/CoreCommon.ver

@@ -18,8 +18,8 @@
 ///         CHAKRA_VERSION_BUILD_QFE
 ///
 /// Binary version string is composed from the following values:
-///     CHAKRA_CORE_MAJOR_VERSION . CHAKRA_CORE_MINOR_VERSION . VER_PRODUCTBUILD . CHAKRA_CORE_VERSION_RELEASE_QFE
-///     e.g. 1.2.0.0
+///     CHAKRA_CORE_MAJOR_VERSION . CHAKRA_CORE_MINOR_VERSION . CHAKRA_CORE_PATCH_VERSION . CHAKRA_CORE_VERSION_RELEASE_QFE
+///     e.g. 1.2.1.0
 /// These fields must be explicitly set in the source.
 /// Note: VER_PRODUCTBUILD is always set to 0.
 
@@ -30,6 +30,7 @@
 
 #define VER_PRODUCTMAJORVERSION     CHAKRA_CORE_MAJOR_VERSION
 #define VER_PRODUCTMINORVERSION     CHAKRA_CORE_MINOR_VERSION
+#define VER_PRODUCTPATCHVERSION     CHAKRA_CORE_PATCH_VERSION
 
 // File Flags
 #if DBG
@@ -126,7 +127,7 @@
 
 #endif // defined(CHAKRA_VERSION_BUILD_NUMBER) && defined(CHAKRA_VERSION_BUILD_QFE)
 
-#define VER_PRODUCTVERSION          VER_PRODUCTMAJORVERSION,VER_PRODUCTMINORVERSION,VER_PRODUCTBUILD,VER_PRODUCTBUILD_QFE
+#define VER_PRODUCTVERSION          VER_PRODUCTMAJORVERSION,VER_PRODUCTMINORVERSION,VER_PRODUCTPATCHVERSION,VER_PRODUCTBUILD_QFE
 
 #define VER_PRODUCTVERSION_MAJORMINOR2(x,y) #x "." #y
 #define VER_PRODUCTVERSION_MAJORMINOR1(x,y) VER_PRODUCTVERSION_MAJORMINOR2(x, y)
@@ -136,7 +137,7 @@
 #define VER_PRODUCTVERSION_STR3(x)   VER_PRODUCTVERSION_STR4(x)
 #define VER_PRODUCTVERSION_STR2(x,y) VER_PRODUCTVERSION_STRING "." #x "." #y
 #define VER_PRODUCTVERSION_STR1(x,y) VER_PRODUCTVERSION_STR2(x, y)
-#define VER_PRODUCTVERSION_STR       VER_PRODUCTVERSION_STR1(VER_PRODUCTBUILD, VER_PRODUCTBUILD_QFE)
+#define VER_PRODUCTVERSION_STR       VER_PRODUCTVERSION_STR1(VER_PRODUCTPATCHVERSION, VER_PRODUCTBUILD_QFE)
 
 #ifdef VER_FILEDESCRIPTION_SUFFIX1_STR
 #ifdef VER_FILEDESCRIPTION_SUFFIX2_STR

lib/Common/CommonDefines.h

@@ -12,9 +12,10 @@
 //----------------------------------------------------------------------------------------------------
 #define CHAKRA_CORE_MAJOR_VERSION 1
 #define CHAKRA_CORE_MINOR_VERSION 2
+#define CHAKRA_CORE_PATCH_VERSION 1
 #define CHAKRA_CORE_VERSION_RELEASE 1
 #define CHAKRA_CORE_VERSION_PRERELEASE 0
-#define CHAKRA_CORE_VERSION_RELEASE_QFE 0
+#define CHAKRA_CORE_VERSION_RELEASE_QFE 0 // Redundant with PATCH_VERSION. Keep this value set to 0.
 
 #define CHAKRA_VERSION_RELEASE 1