Using PowerShell to Generate TFS Changed File List for Build Artifact Delivery

Delivering Artifacts for Deployment

In many enterprise-software development environments, delivering release-ready code to an Operations or Release team for deployment, as opposed to deploying the code directly, is common practice. A developer ‘kicks off’ a build of project using a build automation system like Hudson, Jenkins, CruiseControl, TeamCity, or Bamboo. The result is a set of build artifacts that are delivered and deployed as part of the release cycle. Build artifacts are logical collections of deployable code and other files, which form the application. Artifacts are often segregated by type, such as database, web code, services, configuration files, and so forth. Each type of artifact may require a different deployment methods.

There are two approaches to delivering artifacts for deployment. Some organizations deliver all the artifacts from each build for deployment. Alternately, others follow a partial delivery and release model, delivering only the artifacts that contain changes since the last delivery. The entire application is not re-deployed, only what changed. This is considered by many to be a quicker and safer method of software release.

The challenge of partial delivery is knowing precisely what changed since the last delivery. Almost all source control systems keep a history of changes (‘changesets’). Based on the time of the last build, a developer can check the history and decide which artifacts to deliver based on the changes. If you have daily releases, changes between deliveries are likely few. However, if your development cycle spans a few weeks or you have multiple developers working on the same project, there will likely be many changesets to examine. Figuring out what artifacts to deliver is tedious and error prone. Missing one small change out of hundreds of changes can jeopardize a whole release. Having to perform this laborious task ever few weeks myself, I was eager to automate this process!

Microsoft Team Foundation PowerShell Snap-In

The solution is of course PowerShell and the Microsoft Team Foundation PowerShell Snap-In. Using these two tools, I was able to write a very simple script that does the work for me. If you are unfamiliar with the Team Foundation Server (TFS) snap-in, review my earlier post, Automating Task Creation in Team Foundation Server with PowerShell. That post discusses the snap-in and explains how to install on your Windows computer.

The PowerShell script begins with a series of variables. The first two are based on your specific TFS environment. Variables include:

1. Team Project Collection path;
2. Source location within the collection to search for changes;
3. Date and time range to search for changes;
4. Location of text file that will contain a list of changed files;
5. Option to open the text file when the script is complete.

Given the Team Project Collection path, source location, and the date range, the script returns a sorted list of all files that changed. Making sure the list is distinct is important. File may change many times over the course of a development cycle. You only want to know if the file changed. How many times the file changed, or when it changed, is irrelevant. The file list is saved to a text file, a manifest, for review. The values of the script’s variables are also included in the manifest.

Excluding Certain Changes

Testing the initial script, I found it returned to much information. There were three main reasons:

1.  Unrelated Changes – Not every file that changes within the location selected is directly associated the project being deployed. There may be multiple, related projects in that location’s sub directories (child nodes).
2. Secondary Project Files – Not every file that changes is deployed. For example, build definitions files, database publishing profiles, and manual test documents, are important parts of any project, but are not directly part of the applications within the project being deployed. These are often files in the project used by the build system or required by TFS.
3. Certain Change Types – Changes in TFS include several types (Microsoft.TeamFoundation.VersionControl.Client.ChangeType) that you may not want to include on the list. For example, you may not care about deleted or renamed files. See the post script about how to get a list of all ChangeTypes using PowerShell.

To solve the problem of too much information, we can filter the results of the Get-TfsItemHistory command, using the Where-Object command with the Select-Object command, in the Get-TfsItemHistory command pipeline. Using the -notlike property of the Where-Object command, which accepts wildcards, we exclude certain ChangeTypes, we exclude files by name and size, and we exclude groups of files based on file path. You will obviously need to change the example’s exclusions to meet your own project’s needs.

Below is the PowerShell script, along with some sample contents of file change manifest text file, based on an earlier post’s SSDT database Solution:

###############################################################
#                                                             
# Search for all unique file changes in TFS 
# for a given date/time range and collection location. 
# Write results to a manifest file.                                              
#                                                             
# Author:  Gary A. Stafford
# Created: 2012-04-18
# Revised: 2012-08-11                          
#                                                             
###############################################################

# Clear Output Pane
clear

# Enforce coding rules
Set-StrictMode -version 2.0

# Loads Windows PowerShell snap-in if not already loaded
if ( (Get-PSSnapin -Name Microsoft.TeamFoundation.PowerShell -ErrorAction SilentlyContinue) -eq $null )
{
    Add-PSSnapin Microsoft.TeamFoundation.PowerShell
}

# Variables - CHECK EACH TIME
[string] $tfsCollectionPath = "http://tfs2010/tfsCollection"
[string] $locationToSearch = "$/Development/AdventureWorks/"
[string] $outputFile = "c:\ChangesToTFS.txt"
[string] $dateRange = "D2012-07-08 00:00:00Z~"
[bool]   $openOutputFile = $true # Accepts $false or $true

# For a date/time range: 'D2012-08-06 00:00:00Z~D2012-08-09 23:59:59Z'
# For everything including and after a date/time: 'D2012-07-21 00:00:00Z~'

[Microsoft.TeamFoundation.Client.TfsTeamProjectCollection] $tfs = get-tfsserver $tfsCollectionPath

# Add informational header to file manifest
[string] $outputHeader =
    "TFS Collection: " + $tfsCollectionPath + "`r`n" + 
    "Source Location: " + $locationToSearch + "`r`n" + 
    "Date Range: " + $dateRange + "`r`n" +
    "Created: " + (Get-Date).ToString() + "`r`n" +
    "======================================================================"

$outputHeader | Out-File $outputFile

Get-TfsItemHistory $locationToSearch -Server $tfs -Version $dateRange `
-Recurse -IncludeItems | 

Select-Object -Expand "Changes" | 
    Where-Object { $_.ChangeType -notlike '*Delete*'} | 
    Where-Object { $_.ChangeType -notlike '*Rename*'} | 

Select-Object -Expand "Item" | 
    Where-Object { $_.ContentLength -gt 0} | 

    Where-Object { $_.ServerItem -notlike '*/sql/*' } | 
    Where-Object { $_.ServerItem -notlike '*/documentation/*' } | 
    Where-Object { $_.ServerItem -notlike '*/buildtargets/*' } | 

    Where-Object { $_.ServerItem -notlike 'build.xml'} | 
    Where-Object { $_.ServerItem -notlike '*.proj'} | 
    Where-Object { $_.ServerItem -notlike '*.publish.xml'} | 

Select -Unique ServerItem | Sort ServerItem | 
Format-Table -Property * -AutoSize | Out-String -Width 4096 | 
Out-File $outputFile -append

Write-Host `n`r**** Script complete and file written ****

If ($openOutputFile) { Invoke-Item $outputFile }

Contents of file change manifest text file, based on my previous post’s SSDT database Visual Studio Solution:

TFS Collection: http://tfs2010/tfsCollection
Source Location: $/Development/AdventureWorks2008/
Date Range: D2012-08-02 00:00:00Z~
Created: 8/10/2012 10:28:46 AM
======================================================================

ServerItem
----------
$/Development/AdventureWorks2008/AdventureWorks2008.sln $/Development/AdventureWorks2008/Development/Development.sln
$/Development/AdventureWorks2008/Development/Development.sqlproj
$/Development/AdventureWorks2008/Development/Schema Objects/Server LevelObjects/Security/Logins/aw_dev.login.sql
$/Development/AdventureWorks2008/AdventureWorksSSDT/AdventureWorksSSDT.sqlproj
$/Development/AdventureWorks2008/AdventureWorksSSDT/dbo/StoredProcedures/uspGetBillOfMaterials.sql
$/Development/AdventureWorks2008/AdventureWorksSSDT/dbo/Stored Procedures/uspLogError.sql
$/Development/AdventureWorks2008/AdventureWorksSSDT/HumanResources/Tables/EmployeePayHistory.sql $/Development/AdventureWorks2008/AdventureWorksSSDT/Purchasing/Tables/ShipMethod.sql $/Development/AdventureWorks2008/AdventureWorksSSDT/Purchasing/Views/vVendorWithContacts.sql $/Development/AdventureWorks2008/AdventureWorksSSDT/Security/aw_dev.sql $/Development/AdventureWorks2008/AdventureWorksSSDT/Security/jenkins.sql

Conclusion

This script saves considerable time, especially for longer release cycles, and eliminates potential errors from missing changes. To take this script a step further, I would like to have it determine which artifacts to deliver based on the files that changed, not leaving it up to the developer to figure out. A further step, I would also have it generate an artifact manifest that would be passed to the build. The build would use the manifest to deliver those artifacts to the release team. This would really make it an end-to-end solution. Challenge accepted…

Post Script, PowerShell Enumeration

Assume you couldn’t find a resource on the web that listed all the ChangeType values? How would you use PowerShell to get a list of all the enumerated ChangeType values (Microsoft.TeamFoundation.VersionControl.Client.ChangeType)? It only takes one line of code, once the TFS plugin and assembly are loaded.

# Loads Windows PowerShell snap-in if not already loaded
if ( (Get-PSSnapin -Name Microsoft.TeamFoundation.PowerShell -ErrorAction SilentlyContinue) -eq $null )
{
    Add-PSSnapin Microsoft.TeamFoundation.PowerShell
}

[Void][Reflection.Assembly]::LoadWithPartialName("Microsoft.TeamFoundation.VersionControl.Client")
[Enum]::GetNames( [Microsoft.TeamFoundation.VersionControl.Client.ChangeType] )