Sin categoría – Luis Fraile

Phased deployments with Release Management gates

23 diciembre, 201722 enero, 2018 Luis Fraile Deja un comentario

When we are enabling continuous deployment in development teams, there are a lot of things we must take care. First of all, enabling continuous deployment is not about throwing new code or features to the users, nevertheless the quality or the value it gives, this is about enabling a continuous value flow from development to the users.

For this we must ensure the quality and the impact of any new code we are going to deploy. Apart from the usual automated tests during the builds and deploys, there are something, a lot of companies does, which is a phased deployment, in which you are deploying new changes to a particular subset of users, until the new code has been “real life” tested enough to impact 100% of the users of your applications. This is something you probably has already experienced with for example Windows Insiders, Twitter which deploys features to particular subset of users, Facebook does the same, and even VSTS you can opt-in for new features until they are Generally Available for all users.

But for this one of the most important questions is, how do you decide when to deploy to a broader set of users? and also, which mechanism are you going to implement to automate this?.

This two questions can be resolved with the new (in preview at the moment of writing this) feature of Release Management Gates. These are automated approvals we can set, for any particular environment of a Release Management definition, which will be automatically evaluated, prior or post any environment deployment, also these gates are evaluated every specific period of time, until they pass, or until they timeout if they fail every check.

Gates can be set, out of the box, on a variety of things, but new ones can be created, like this example based on twitter sentiments:

Azure Functions: The gate will call a particular Azure Function, sent the function a pre-defined message (defined in the gate definition), and wait for the response, even being able to parse the response to check everything went ok.
Invoke API Rest: is similar to the previous one, but calling any particular Rest API.
Work Items Query: Check if a particular Work Item Query has grown its count of items. Think of this for example a Bugs Query, in which you decide the Gate has failed if the bug count grows over a particular threshold.
Azure Monitor: The one I wanted to explain in this article. If checks for any particular (one or n) Azure defined alert, to check if the alert has been thrown. Think for example an alert on performance degradation or number of errors to check in a particular environment, prior to deploy to new broader set of users environments.

Show me the code boxes

First of all, we must enable Gates on our preview Features, so on your Visual Studio Team Services account, click on your profile, and select Preview Features.

And on the preview features for the account (or just for you) enable Gates.

We start with a normal Release Management definition, with two environments, one dependent of another in sequence, and let’s say first one is for early adopters and the next one is the general available, I agree this is a great simplification of any real environment, but is enough for this example.

Now, lets inspect the GeneralAvailability environment pre-deployment approvals clicking on And enable Gates as a pre-deployment approval, clicking on add we can see the different choices we have, for this example we will go with Azure Monitor.

We have also there the delay before start evaluating this gate, this is the time needed to pass after previous environment deployment and the moment the first check for the gate is done. Once we add the new gate, we have to fill all the information for the gate.

Display Name for the gate.
Azure subscription connection, if we don’t have it already we need to setup an Azure connection via services
The name of the resource group in which the resource exists in the Azure subscription.
The type of the Azure resource, we can choose between Application Insights, App Service, Storage Account, Virtual Machines. In this demo we go with Application Insights resource.
Name of the Resource
Alert or alerts we want to be monitoring. The alerts must already exists in the chosen Azure Resource, but we will see this later on this post.

When we have this information filled, if we continue going down on that same screen, we can fill several options for all the gates.

First we have the timeout which is the time after which, we finish the deployment for this environment as failed if any of the gates has not passed, so we can’t go on with the environment. Sampling interval is the period of time between each check of the gate. This times by default are, respectively, 15 and 5 minutes, but they can be longer, even days, so you have time enough to go on with the early adopters (for example) environment before going ahead.

Also, for the case there are manual approvals before the deployment, you can select between three different options, like (as seen in the image) manual approvals must be done before start checking the gates, manual approval needed only after all gates has passed, or manual approval after each gate.

With this we would have the gate, but for the case you are not familiar with Azure alerts, just one thing more. For this example we chosen Azure monitor against an Application Insights Alert, so what I created before going on with all of this, I had my Application Insights resource created and configured for my selected app or Azure environment for the, in this case, EarlyAdopters environment (for the case you are not familiar with Application Insights check it here).

And if e click on Alerts, we can see the configured alerts, and go to specific alert configuration, the one we selected on the Azure Monitor Gate.

In this case the alert will raise if in the last 5 minutes, more than 2 errors has occurred in the application.

When we finish this configuration, we will start deploying the application to the early adopters environment, hopefully users will start using the new features or version of the application, after the configured time for the gate, VSTS will check for the alert, if the alert has not been raised, it will continue with the next environment deployment, if the gate has been raised, it will continue waiting for the next check of the gate, until timeout configured occurs or the gate passes.

I hope you liked gates as much as I like them, as conclusion, when working with phased continuous deployment it is important to establish which are your gates defining how to move from one phase to another, and afterwards, configure them as you need, and configure your phased deployment with VSTS Release Management.

Work Items bulk edit with templates on VSTS

15 agosto, 2017 Luis Fraile Deja un comentario

There are some occasions in which you need to apply the same changes to multiple Work Items, not only once, but several times during a project, for sure most of you already know the edit selected items feature, with several Work Items selected, just right click and select edit (sorry I had to protect the innocent on the captures) :

This allows you to edit all the selected Work Items and make the changes to the fields you selected. The only point with this, is when you need to do it several times, and always apply the same values to the same fields, as it is a little bit tedious.

So we have Work Item templates, we start from the same point, select several Work Items, in this case all of them must be from the same type, but I will explain this later, right click, and select Templates/Manage:
This bring us to the template editor screen, in this screen you will see we can define as much templates as we want for each type of Work Item, that’s the reason I said in the previous point all selected Work Items must be from the same type, as you will apply a template for a particular type:
When clicking on New template button, it will open a screen for stating which are the values for the different fields for this template:
When you save it, and go back to the list of Work Items (you will need to refresh the browser window), select the Work Items you want, right click, and now, under Templates option, you will have this newly created template, and once applied to the selected Work Items, it will apply the values to the fields selected in the template:

As you can see, templates can simplify our editing a lot when moving work across teams, organizing backlogs, bugs, etc. so go and check which would be your needed templates and go create them. Just remember, they must be defined per Work Item Type, maybe it is just a small “cons” for this, as when we used to do the bulk editing, we can select different Work Item Types, but for repetitive editing, templates are far more powerful.

Creating custom tasks for VSTS Team Build and Release Management slides

30 octubre, 2016 Luis Fraile Deja un comentario

Recently I gave a talk a out creating custom tasks for Team Build and Release Management for VSTS and also for Team Foundation Server, as also I’m starting again to write here (with lots of articles at first with ideas I had in my mind) I thought it also could be a good idea to leave links to content here.

It is in Spanish, but well, there is a bunch of useful links in the PPTX. It was mainly based on creating tasks with Javascript but I cover also Typescript (well both of them are Javascript executed with NodeJS at the end of the day) and PowerShell.

So here are the slides:

Creación de tareas personalizadas para Team Build y Release Management de Luis Fraile

And also the source code used in the demos, it was a very basic set of tasks for .NET Core projects to restore packages, build and publish:

https://github.com/lfraile/DotNetCoreTasks

Have fun.

Code Search extension for VSTS and Team Foundation Server “15”

29 octubre, 2016 Luis Fraile Deja un comentario

Recently Microsoft put as Generally Available a very interesting extension in the Visual Studio Marketplace, the Code Search extension. Install it on your VSTS is as simple as go to the previous link and click install, then select the Visual Studio Team Services account in which you want to install the extension, of course you need to be an Administrator to install it.

To install on Team Foundation Server “15”, is just as simple as install it during the installation phase of TFS.

But, what enables this extension? Once you install it, on your team projects you will have, in the menu bar, a search box in which you can select to search code:

When you select code, you will be presented some of the main options to search for code.

But there are even more options you can check in the help page.

The interesting thing of this Code Search extension, is it not only look for text inside the code files, that would be easy, it searches across all projects or just the ones you want.

But it also allows you to put filters like for example look only for classes named like the term you are looking for, or comments, references and a lot of more, I’m really impressed on how rich it is. Of course you can refine your queries with AND, OR, NOT terms.

Also it integrates with history, so when you find what you are looking for, you can see its history, compare with previous versions, and you can even see annotations within the code.

Just as a conclusion a pretty nice extension you can start installing and using on your VSTS to search for code, but far beyond the usual look in files functionality.

As technology it uses in the behind, if you look here you will see it uses:

Elasticsearch
Oracle Server JRE, yes for TFS you will need to install it on the server, but you can install Code Search on a separate server. Of course for VSTS you don’t need to care about this.
Mardowndeep.
Roslyn (hype increasing on this one)
ANTLR

And about languages it supports, currently it supports C#, C++, C, VB.NET, and recently they added support for Java also, and it is ok to think they will be updating the list of languages.

So go ahead, install it and try it, you can find more options and documentation here.

I’m back! Executing Entity Framework migrations from VSTS Release Management

29 octubre, 2016 Luis Fraile Deja un comentario

Uff there has been a long, long time since last time I wrote here … but, several people have lately asked me about my blog, so there I go, and better than write a bla bla article about the last time and why I haven’t wrote so much, let’s get technical.

In this article I assume you have the basic knowledge of creating Team Build definitions, and Release Management definitions. I’m not covering this topics in this article, as it would make it so long to read. If you are not familiar with this, I would recommend you to read about it here https://www.visualstudio.com/en-us/docs/release/overview

When we are deploying applications with a database, there are several ways we can do, usually we will deploy differential scripts to update the DB, or DACPAC and other technologies, but we can also use Entity Framework migrations, although I still have to think if it is the best way to do it …

Usually migrations are executed Entity Framework initializers, but if we need to execute them before deployment, so we can be sure we updated the DB even before deploy our application, there is a tool named “migrate.exe” included in the NuGet package of Entity Framework.

There is a couple of steps we need to take to be able to execute this tool from Release Manage:

Build our migrations Assembly
Copy the tool migrate.exe from the tools folder in the EF Nuget package folder to the same directory as the migrations assembly
Execute migrations

Let’s go with the first two steps, which we will do in a Team Build definition which publishes the results, as artifacts, to the Release Management definition. The build step is easy, usually our migrations assembly will be included in the Solution we are building to deploy our application, if not, well include it in the solution or build it in a separate build step in the same build definition, no tricks in this one.

Copy the migrate.exe tool includes a couple more of steps. First I would copy the result binaries from building the migration project to a separate folder we will publish as an artifact. This is done with a Copy task in the build steps, which we will configure this way:

The parameters:

Source folder: we point to the binaries result of our migration assembly, notice I have oversimplified the directory, with /MigrationAssembly/ be sure to include the full path to it. And I have used a couple of variables $(build.sourcesdirectory) a system variable which points to the root of sources downloaded by the agent, and a custom variable $(buildConfiguration) which points to the current build configuration (i.e.: Debug, Release or whatever you use).
Contents: ** so we copy all results.
Target folder: I’m copying to a new folder automatically created in the artifacts staging directory, as configured with the system variable $(build.artifactstagingdirectory), you don’t need to create a complex folder structure under this one, but be sure to at least create a folder structure which allows you to separate different results and artifacts.

Next step, copy the migrate.exe file, again we use a copy task:

With the parameters:

Source folder: we point to NuGet packages folder, which is usually at the same level of the solution we are building, but be sure to check this correctly, probably this path is one of the trickiest paths of this configuration.
Contents: “migrate.exe” well, no comments …
Target folder: I’m copying to a the same folder we copied the results of the migration assembly in the previous step. This is very important for all of this work, so be sure to check it twice.

And the last step in the build, publish the artifacts, usually as simple as this one, which will publish all the folder structure we have in the artifacts directory to a server artifact:

Final steps will be seen like this:

Once we have done this, we can queue this build definition, and once finished, in the resulting artifacts just check you have your binaries from the assembly migration along with the migrate.exe tool in the same folder within the artifacts.

For the Release to execute the migrate.exe file it is just a simple task of execute a command line, of course one gotcha of this is to link the build definition with the Release Management definition (again, I assume you are already familiar with this).

So within the desired environment of our Release definition, we will just add a Run on agent task of type Run script, one important point, remember this tasks runs on the agent, so you need to ensure your agent can communicate with your SQL Server or SQL Azure.

We will configure this task in this way, before the deploying task for the application:

The parameters we are using:

Path: here we configured the path to migrate.exe within the build artifacts we are using, you can take advantage of the “…” button to look for it, again, remember: you must have linked your Release Management definition to your Build definition for this to be available.
Arguments: there is different arguments you can use here, even just point to a *.config file with all the values (check full documentation), in this case I just pointed to a custom variable containing the connection string (be sure to make it secret to protect it hehe), and as I pointed to a connection string, it is mandatory to configure the parameter of “Connectionprovidername”, which in my case is just SQL Server.

Some important gotchas here, be sure to test thoroughly your migrations, and be sure to enable the appropriate backups of databases for the rollback cases, this is not easy, and you have to really take care about it, so have different environments of Release Management for test all the deployments and migrations.

Once you have this, next time you run this Build and Release definition, your database will be (hopefully if you have done it correctly) updated to last migration from Entity Framework.

And hopefully, see you later around here with more articles Smile