Many times we have the need to store our project in GitHub because we want (or need) to be open source but still want to keep the capabilities offered by TFS/VSTS. Other times we need to give access to our repository to a person who can’t have (or that we don’t want to give) access to our TFS/VSTS server.
In these cases we can resolve the problem synchronizing our TFS/VSTS repository with our GitHub (public or private) repository. At first glance it may sound like a complicated process but it’s easier than you expect.
You can perform the synchronization manually (using pull/push) or automatically. For automate the process you need to create two builds for each branch, one to send the pushed changes in TFS/VSTS to GitHub and another one to send the changes pushed to GitHub to TFS/VSTS.
This method allows you to synchronize specific branches and decide if the synchronization is in both directions or in a single direction. However it must be taken into account that the synchronization can create conflicts if isn’t done in the right way.
The synchronization from GitHub to TFS on premise is the most complicated case. In TFS 2015 on premise the option to use a GitHub repository as a source repository in a build definition is not available. Also there are problems using the service hooks to connect with GitHub as external Git Repository. Nevertheless you can create a build with schedule as a trigger to run periodically a script that perform the synchronization (pull from GitHub repository and push to TFS) or use a manual trigger and run it each time that you push changes to GitHub.
If you are using TFS on premise my recommendation is to try to avoid the two way synchronization. The best approach is to remove the permissions of all users in the GitHub repository (except for the user that run the sync build) to avoid direct changes in GitHub and works against TFS only. Even so, I recommend to create the both builds (Sometimes you may want to include changes from GitHub, for example pull requests). What is extremely important is to run de sync from GitHub to TFS before push another changes in TFS.
Another option to avoid conflicts (only for TFS on premise) is create a server side plugin to run the sync build (from GitHub to TFS) before receive each push in TFS. (More info about server side plugin for Git repositories here)
- The CI build is trigged only when a new commit is pushed to the repository (no when a new tag is pushed)
- In this example I will synchronize the tags when they are added but not when there are removed (however you can create a script to compare the tags in the both repositories and delete the tags that doesn’t match)
Here step by step guides for:
Sync from TFS/VSTS to GitHub
1) Create an “Empty” VNext build
2) Select the Git TFS repository, the branch to synchronize and check the box “Continuous Integration”
3) In the section “variables” create a new variable with the GitHub credentials (user:password). Press the lock icon to hide the details.
4) Add 4 “Command Line” build steps
5) Set the first one to run “git pull <GitHubRepoUrl> master”
“git pull https://$(credentials)@github.com/leonjalfon1/TFSTeamsMigratorTool.git master”
6) Set the second one to run “git pull –tags <GitHubUrl>”
“git pull –tags https://$(credentials)@github.com/leonjalfon1/TFSTeamsMigratorTool.git”
7) Set the third one to run “git push <GitHubUrl> head:master”
“git push https://$(credentials)@github.com/leonjalfon1/TFSTeamsMigratorTool.git head:master”
8) Set the fourth one to run “git push –tags <GitHubUrl>”
“git push –tags https://$(credentials)@github.com/leonjalfon1/TFSTeamsMigratorTool.git”
9) Save the build
Sync from GitHub to VSTS
1) First of all you need create a personal token. For this, click the user icon and select “Security”
2) In the “Personal Access Token” section, click “Add”
3) Write a description for your token (to remember for what the token is being used), the duration, the affected accounts and the authorized scopes.
4) Copy your token (will be used in next steps) and store it in a safe place (optional).
5) Create a new “Empty” VNext build:
6) Select GitHub as source repository and check the continuous integration box
7) Go to the variables section and add the created token as a new build variable called “token” (optional, lock to hide)
8) Add 4 “Run command line” build steps
9) Configure the first one to run “git pull <VSTSRepoUrl> master”
“git pull https://$(token)@leonj.visualstudio.com/DefaultCollection/MyProject/_git/TFSTeamsMigratorTool master”
10) Configure the second one to run “git pull –tags <VSTSRepoUrl>”
“git pull –tags https://$(token)@leonj.visualstudio.com/DefaultCollection/MyProject/_git/TFSTeamsMigratorTool”
11) Configure the third one to run “git push <VSTSRepoUrl> head:master”
“git push https://$(token)@leonj.visualstudio.com/DefaultCollection/MyProject/_git/TFSTeamsMigratorTool head:master”
12) Configure the third one to run “git push –tags <VSTSRepoUrl>”
“git push –tags https://$(token)@leonj.visualstudio.com/DefaultCollection/MyProject/_git/TFSTeamsMigratorTool”
13) In the “Repository” section click “Manage” to add a new connection
14) Click “New Service Endpoint” and select “GitHub”
15) Give a connection name and authenticate using a GitHub personal token or authenticating by the web browser and click “OK”
16) Go back to the previous tab and click the refresh button
17) Select a connection, repository and branch (master in this example)
18) Save the Build
Note that after configuring a continuous integration build that is linked to GitHub repository, VSTS will automatically create a webhook in GitHub to trigger the VSTS build.
Sync from GitHub to TFS
1) Create a new “Empty” VNext build
2) Select your team project as repository source and select the repository and branch (master in this example)
3) In the variables section add the GitHub credentials (for private repos) and the TFS credentials
4) Add 4 “Run command line” build steps
5) Configure the first one to run “git pull <GitHubRepoUrl> master”
“git pull https://$(GitHubCredentials)@github.com/leonjalfon1/TeamsMigratorTool.git master”
6) Configure the second one to run “git pull –tags <GitHubRepoUrl >”
“git pull –tags https://$(GitHubCredentials)@github.com/leonjalfon1/TeamsMigratorTool.git”
7) Configure the third one to run “git push <TFSRepoUrl> master:master”
“git push http://$(tfscredentials)leonjalfon:8080/tfs/DefaultCollection/MyProject/_git/TFSTeamsMigratorTool master:master”
8) Configure it to run “git push –tags <TFSRepoUrl>”
“git push –tags http://$(tfscredentials)leonjalfon:8080/tfs/DefaultCollection/MyProject/_git/TFSTeamsMigratorTool”
9) Go to “Triggers” section, select “Scheduled” and configure when the build runs
10) Save the build