Variable library groups and Azure Key Vault

In my previous post, I explained how to use Azure Key Vault values in a particular Build/Release, but as Vinicius Moura posted in twitter, there is another option using Libraries in your Team Project, and I wanted to just give my opinion or how I use one or another option.

Just to introduce it, Libraries is a method to create Task Variables which can be used across different Builds/Releases in a particular Team Project. To create a Library go to the Build and Release hub in VSTS, click on Library  and then click, on the upper right corner + Variable Group.image

Here we can define a new Library and define its variables and values, just giving the Library a name to then use it in Builds/Releases, and create variables.

image

Also we have the option to link secrets from Azure Key Vault, as Vinicius pointed. Just one thing, when you link a library to Azure Key Vault secrets, you can’t add new manual variables for that library. In the process of linking to an Azure Key Vault, you will need to select an Azure Subscription, and authorize VSTS for both access the subscription and the desired Azure Key Vault, so be sure you are using an account with the proper rights.

image

Then you can just click on  + Add  to add new variables from the Azure Key Vault, as, differently from the Azure Key Vault task, it does not adds the contents from the Vault automatically.

image

Once you create the Library with your custom variables or the Azure Key Vault connection, you can just use it in a Build or Release, in the variables section when editing a build, clicking on Link variable group and selecting the variable group.

2018-02-24_17-52-58

When you link the Library group you can start using the variables with the usual syntax on your Build/Release: $(myvariable)

So, now that I explained Libraries, why and when choose this or the task option? Well for me it is just a question on how you are going to use it, and also the related security. By default, if I need values which must be shared across different builds/releases I would go for libraries for sure, but when it comes to Azure Key Vault, they are used to keep things protected and secret, if you create a library with its default permissions, you are sharing the possibility of using this secrets (and potentially leak them) to any VSTS user with contributor permissions in that particular Team Project, so if you do not pretend that, you must take care of setting the appropriate permissions on the Library, this is not a problem or anything, it is something you are suppose to do and take care of.

When you use the task, only people allowed to edit that particular build will be able to use this secrets, of course, you have to take care of this permissions too.

So, in my opinion, if you don’t need to share the variables across several builds/releases, or you just want to keep them as restricted as possible, go with the tasks. If you want to protect a set of values that you want to re-use across builds and releases, go with the libraries.

Just remember, this is not a rule, this is just my opinion, and is always subject to change a lot Smile

Debugging deployed Azure Web Apps with VSTS Symbol Server

Obviously, debugging an already deployed application, doesn’t matter if Azure or any other environment, is something we must use as a last resort, I would always prefer to be able to reproduce some kind of situation with a local test and correct it. But sometimes we would need to debug an already deployed application, and it is never as easy as click F5 in our Visual Studio. We must connect to remote processes, make sure we have the right version of the code, and more important, be able to match the binaries with the code we have, because surely you always build your applications as Release to deploy them (and if not … run, run and do it).

Having the right version of the application is easy we have branches, tags, and other tools which allows us to locate the right code version. Attach to remote processes is something maybe a little bit more complex, but fortunately we have Remote Debugging tools, and even more, in Azure Web Apps, we can directly enable it from Visual Studio or the portal, and connect automatically, we will see this later.

To make sure we match the binaries with the code we have symbols, but we need to have the right symbols for the binaries, something we can get via a Symbol Server, this is something you can setup by yourself, but luckily now we have a Symbol Server included in VSTS, well it is still in preview mode, but it is something rather interesting to setup and worth it. Let’s start on how to set it up.

Disclaimer: I’m not digging into how to create a build definition, or deploy an application to Azure Web App using Release Management, so if you are not familiar with this kind of tasks first make familiar with these, or just leave me a comment if you find it interesting for next blog posts.

Disclaimer II: You will need Visual Studio 2017 updated to the last update for doing this.

Enabling the feature

As a preview feature, we must first turn it on for our account, or ask something with the needed privileges to turn it on, we just click on the top right, in our account icon, select Preview features and enabled it for the account.image

But in this case, this is not a stand-alone feature, this feature needs the Package Management extension from the Visual Studio Marketplace, which, remember, is not free, but the symbol server added to the package management extension is worth the price.

Publishing symbols

Once set-up the account we can start publishing our symbols. Usually (if not … again … run) we have one build, n deployments. So this is something we are doing during the build. So let’s go to edit the build definition used to generate artifacts for deploy, and add a new step after the build step, and add the publish symbols task.

image

By default, when adding the task, it is added with the version 1.*, but we will select version 2.* (preview) version of the task, and in the Symbol server type parameter, we select Symbol server in this account … I will remind you the need of Package Management extension.

image

This is the only change we need to do in our build or releases. So let’s go to next step.

Configuring Visual Studio to consume VSTS Symbol Server

We must configure Visual Studio for a couple of things: instruct it to debug using symbol servers, which symbol servers to use. We will do both of them from the Tools / Options screen.

For the first one we go to Debugging section of tools, and disable Enable just my code, yes, disable it, so Visual Studio is instructed to debug external code.

image

Now under Debugging / Symbols, click on the icon I show you in the next picture, which will bring the add new Symbol Server from VSTS. You can leave the rest of the parameters with the default values.

image

On the next screen just select the VSTS account in which you set-up the symbol server. And after that close the Options screen.

image

Debugging the Azure Web App

If we came this far, we have everything ready to start debugging, so with the version of the code we published the symbols and deployed open into Visual Studio, we will attach to the Web App process for debugging. Be sure to enable the breakpoints you need for the debugging and also notice you will impact any user of the application so better do it in a slot or any environment with no real users.

On Visual Studio 2017 Server Explorer, make sure you are connected to the Azure Subscription, and in the App Service list, locate the resource group containing your app, locate the app (and the slot if you have it), right click it, and select Attach debugger.

image

This can take a while, but after finish the attach, it will automatically connect to the Symbol Server, get the debug Symbols and you are ready to go and start debugging your web app with the breakpoints and the debugging features of Visual Studio and Azure.

If you receive an alert like this.

image

Remember to disable the Enable Just my code mentioned earlier in this same post.

Keep it clean

After debugging, this is something I like to do, and is go to the Azure Web App Application Settings, and disable the remote debugging check. Next time you need to debug, following previous steps, Visual Studio will re-enable it for you, but I just like to keep it off … just for the case …

image

Hope you enjoyed this feature as much as I do.

I’m back! Executing Entity Framework migrations from VSTS Release Management

Uff there has been a long, long time since last time I wrote here … but, several people have lately asked me about my blog, so there I go, and better than write a bla bla article about the last time and why I haven’t wrote so much, let’s get technical.

In this article I assume you have the basic knowledge of creating Team Build definitions, and Release Management definitions. I’m not covering this topics in this article, as it would make it so long to read. If you are not familiar with this, I would recommend you to read about it here https://www.visualstudio.com/en-us/docs/release/overview

When we are deploying applications with a database, there are several ways we can do, usually we will deploy differential scripts to update the DB, or DACPAC and other technologies, but we can also use Entity Framework migrations, although I still have to think if it is the best way to do it …

Usually migrations are executed Entity Framework initializers, but if we need to execute them before deployment, so we can be sure we updated the DB even before deploy our application, there is a tool named “migrate.exe” included in the NuGet package of Entity Framework.

There is a couple of steps we need to take to be able to execute this tool from Release Manage:

  1. Build our migrations Assembly
  2. Copy the tool migrate.exe from the tools folder in the EF Nuget package folder to the same directory as the migrations assembly
  3. Execute migrations

Let’s go with the first two steps, which we will do in a Team Build definition which publishes the results, as artifacts, to the Release Management definition. The build step is easy, usually our migrations assembly will be included in the Solution we are building to deploy our application, if not, well include it in the solution or build it in a separate build step in the same build definition, no tricks in this one.

Copy the migrate.exe tool includes a couple more of steps. First I would copy the result binaries from building the migration project to a separate folder we will publish as an artifact. This is done with a Copy task in the build steps, which we will configure this way:

image

The parameters:

  • Source folder: we point to the binaries result of our migration assembly, notice I have oversimplified the directory, with /MigrationAssembly/ be sure to include the full path to it. And I have used a couple of variables $(build.sourcesdirectory) a system variable which points to the root of sources downloaded by the agent, and a custom variable $(buildConfiguration) which points to the current build configuration (i.e.: Debug, Release or whatever you use).
  • Contents: ** so we copy all results.
  • Target folder: I’m copying to a new folder automatically created in the artifacts staging directory, as configured with the system variable $(build.artifactstagingdirectory), you don’t need to create a complex folder structure under this one, but be sure to at least create a folder structure which allows you to separate different results and artifacts.

Next step, copy the migrate.exe file, again we use a copy task:

image

With the parameters:

  • Source folder: we point to NuGet packages folder, which is usually at the same level of the solution we are building, but be sure to check this correctly, probably this path is one of the trickiest paths of this configuration.
  • Contents: “migrate.exe” well, no comments …
  • Target folder: I’m copying to a the same folder we copied the results of the migration assembly in the previous step. This is very important for all of this work, so be sure to check it twice.

And the last step in the build, publish the artifacts, usually as simple as this one, which will publish all the folder structure we have in the artifacts directory to a server artifact:

image

Final steps will be seen like this:

image

Once we have done this, we can queue this build definition, and once finished, in the resulting artifacts just check you have your binaries from the assembly migration along with the migrate.exe tool in the same folder within the artifacts.

For the Release to execute the migrate.exe file it is just a simple task of execute a command line, of course one gotcha of this is to link the build definition with the Release Management definition (again, I assume you are already familiar with this).

So within the desired environment of our Release definition, we will just add a Run on agent task of type Run script, one important point, remember this tasks runs on the agent, so you need to ensure your agent can communicate with your SQL Server or SQL Azure.

We will configure this task in this way, before the deploying task for the application:

image

The parameters we are using:

  • Path: here we configured the path to migrate.exe within the build artifacts we are using, you can take advantage of the “…” button to look for it, again, remember: you must have linked your Release Management definition to your Build definition for this to be available.
  • Arguments: there is different arguments you can use here, even just point to a *.config file with all the values (check full documentation), in this case I just pointed to a custom variable containing the connection string (be sure to make it secret to protect it hehe), and as I pointed to a connection string, it is mandatory to configure the parameter of “Connectionprovidername”, which in my case is just SQL Server.

Some important gotchas here, be sure to test thoroughly your migrations, and be sure to enable the appropriate backups of databases for the rollback cases, this is not easy, and you have to really take care about it, so have different environments of Release Management for test all the deployments and migrations.

Once you have this, next  time you run this Build and Release definition, your database will be (hopefully if you have done it correctly) updated to last migration from Entity Framework.

And hopefully, see you later around here with more articles Smile