DNN Redis Caching Provider 3.0

Hi again! Following up with the DNN module updates, I have done some modifications on the DNN Redis Caching provider so it can be configured now through the DNN Persona bar.

The summary of changes of this release is:

  • Changed the minimum required DNN version to 9.0.1
  • Refreshed nuget packages including the latest version of StackExchange.Redis 1.2.6
  • Added configuration UI in the Persona Bar. Now the provider is not automatically enabled during installation, you need to use the new UI.
  • The Redis client automatically reconnects after a Redis connection failure

RedisCaching

If you have any interesting idea to add on the settings area, please let me know. I have thought on implementing a redis-cli command line interface, but I believe that would be better to wait for DNN 9.2 and implement that as a “DNN Prompt” command (see http://www.dnnsoftware.com/community-blog/cid/155417/dnn-prompt-making-dnn-admins-power-admins-via-the-command-line)

Hope this helps.

Un saludo and happy coding!

Building an Evoq Liquid Content chatbot with Azure Bot Service

dnnrecipes2A few months back we spent some days working in an internal hackathon at DNN Corp. called DNN Developer Days that allowed to explore the power of Evoq Liquid Content APIs. The result was an awesome set of project examples giving a glimpse of what you can do when integrating the APIs to publish and reuse the content through different channels, such as Amazon Echo, Azure Bot Service or a smart TV, including AI and machine learning capabilities such as tagging your site images automatically. You can explore all of them at http://builtwithdnn.com website, I hope you find them interesting.

I personally worked on an Azure Bot Service chatbot, exposing the contents of a recipes website through different channels like Skype, Facebook Messenger, Telegram or Teams. You can quickly play with the example at http://builtwithdnn.com/recipesbot/ 

IMG_0365 IMG_0366 IMG_0367

All the documentation and source code for each project is available on a GitHub repository at https://github.com/dnnsoftware/Dnn.Evoq.LiquidContent.Samples.Public/ 

Chatbot as a Channel: Integrating Liquid Content with Azure Bot Service

Integrating Artificial Intelligence (AI) into your site or application is easier than you think. Companies like Google, Microsoft and Amazon are making big bets on AI and machine learning. All three companies provide freely available toolkits and services that developers can use to integrate AI into our applications.

The recipes bot integrates with messaging apps like Skype, Telegram or Facebook Messenger. As a Skype user, for example, you can provide the bot with a list of ingredients. The bot connects to Liquid Content to retrieve recipes that contain those ingredients, then provides an answer to the Skype user.

This is just the beginning. I plan to connect more third-party services to the recipe bot, such as LUIS, Cortana, Vision API and Apple Pay.

Bookmark this page and check back here to get updates. I’ll update the page each time there’s news to share. You can also follow me on Twitter @davidjrh and I will mention the new additions there.

Building the Recipes Bot

In this tutorial, I show you how I built the recipe bot. The bot uses the Azure Bot Service, the intelligent, serverless bot service that scales on demand. With this service, you will be able to publish your bot through multiple channels without managing or patching any server. It can connect to additional messaging apps without writing or adding new code. The service is free, and you will only pay for the resources you consume.

ArchitectureGetting Started

Prerequisites

To follow the tutorial, you will need:

  • An active Azure subscription. You will need to login as owner of the subscription to host the bot service. You can sign up for a free subscription here.
  • An Evoq Content or Engage site with Evoq Liquid Content enabled

Also, to debug your C# bot locally using breakpoints in Visual Studio you will need:

For more information about how to setup the debugging environment, visit Debug with the emulator and Debug an Azure Bot Service C# bot on the bot framework documentation.

Content Index

  1. Setting up the recipes
  2. Creating the basic bot
  3. Setup continuous integration
  4. Debugging the bot on your local environment
  5. Customizing the basic bot
  6. Test your recipes bot
  7. Adding a webchat in your site
  8. Known issues

Hope this helps.

Un saludo and happy coding!

DNN Application Insights v3.0

Hi all. After a while working with React/Redux and the new DNN PersonaBar model introduced on DNN Platform 9.0, I have started to update all my modules to avoid the use of the disappeared Host menu on DNN.

The first module updated has been the DNN Application Insights now available for download on GitHub. This includes:

  • Previous package updates to support version 2.4.0 of Application Insights, including enhanced live metrics streaming (committed by Mitchel Sellers)
  • New UI on the persona bar to setup the module settings
  • Requiring DNN Platform 9.0.1 or later

AppInsights_InstrumentationKey3

If you have any interesting idea to add on the settings area, please let me know. I have been checking all the configuration settings that can be done through the Application Insights configuration system and I will start adding them for the next release.

Updates for the Redis Caching Provider and Azure AD provider are the next ones.

Providing a GUID function in Azure Resource Manager templates with Azure Functions

arm-functionsSome time back while preparing the Global Azure Bootcamp Science Lab, I face the lack of some functions that are not available when authoring Azure Resource Manager templates. When creating some type of resources such as Batch jobs or RBAC related resources, you need to pass a GUID (Universally unique identifier) but there is no function to create them inside the template, so you need to pass them as template parameters making the result awful.

There is already a Feedback item describing the issue (please vote on https://feedback.azure.com/forums/281804-azure-resource-manager/suggestions/13067952-provide-guid-function-in-azure-resource-manager-te) and there are other similar issues when you need to use more sophisticated functions such as date related values.

When trying to provide a workaround, I finally found that could be solved with nested templates, so I started by building a simple ARM Guid template that could be referenced on your primary one. You can check that repo here https://github.com/davidjrh/azurerm-newguid

But when testing the GAB lab with millions of guids, found that from time to time that template was generating duplicated guids, so finally ended by implementing the lab ARM Guid templates by using a WebAPI. You can check the how the guid template was being used at https://github.com/intelequia/GAB2017ScienceLab/blob/master/lab/assets/GABClient.json#L100 (check lines 100 and 398), and the template API implementation available at https://github.com/intelequia/GAB2017ScienceLab/blob/master/src/GABBatchServer/src/GAB.BatchServer.API/Controllers/TemplatesController.cs

Building a GUID function for ARM templates with Azure Functions

Revisiting this today, I thought that could be more cost effective of having that dynamically generated template by using an Azure Function. Here is the step by step so you can deploy your own, and increase your ARM functions arsenal with the same approach.

Remember that the basic idea is to reference this function in your deployment, that will generate a GUID for later usage on your template.

  1. Create an Azure Function App through the Azure Portal.
  2. Add a function triggered by a HTTP request in C#
  3. Copy and paste the following code in the function body

using System.Net;
using System.Net.Http.Headers;


public static async Task<HttpResponseMessage> Run(HttpRequestMessage req, TraceWriter log)
{
     log.Info(“C# HTTP trigger function processed a request.”);


    // parse query parameter
     string numberOfGuids = req.GetQueryNameValuePairs()
         .FirstOrDefault(q => string.Compare(q.Key, “numberOfGuids”, true) == 0)
         .Value;
     if (string.IsNullOrEmpty(numberOfGuids)) {
         numberOfGuids = “1”;
     }


    // validate the input
     int guids;
     if (!int.TryParse(numberOfGuids, out guids)) {
         return req.CreateResponse(HttpStatusCode.BadRequest, “Please pass a valid number on the query parameter ‘numberOfGuids'”);
     }
     if (guids <= 0 || guids > 1000) {
         return req.CreateResponse(HttpStatusCode.BadRequest, $”Invalid number of guids {guids}. Must be a number between 1 and 1000″);
     }


     // prepare the template
     var template = @”{
   “”$schema””: “”
https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#””,
   “”contentVersion””: “”1.0.0.0″”, “”parameters””: {}, “”variables””: {}, “”resources””: [],
   “”outputs””: {[OUTPUTS]}
}”;
     var outputs = await Task.Run<List<string>>(() =>
     {
         var o = new List<string>();
         for (var i = 0; i < guids; i++)
         {
             o.Add(@”””guid” + i + @”””: { “”type””: “”string””, “”value””: “”” + Guid.NewGuid() + @””” }”);
         }
         return o;
     });
     var result = template.Replace(“[OUTPUTS]”, string.Join(“,”, outputs.ToArray()));


// return the response
return new HttpResponseMessage() {
     Content = new System.Net.Http.StringContent(
         result,
         System.Text.Encoding.UTF8,
         “application/json”
     )
};


}

AzurePortalFunction

Testing the function

Once you have saved the function, you can test it by doing a webrequest. In my case, when I call the URL https://armtemplates.azurewebsites.net/api/NewGuid?numberOfGuids=3 I get this answer:

NewGuidWebYou can pass the number of guids to generate as parameter.

Consuming the GUID template by using a nested template

As I documented on the initial GitHub repo, you can just follow these examples to consume then in different ways:

Example 1. Getting a GUID and using it later

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {},
  "variables": {},
  "resources": [ 
    { 
        "apiVersion": "2015-01-01", 
        "name": "MyGuid", 
        "type": "Microsoft.Resources/deployments", 
        "properties": { 
          "mode": "incremental", 
          "templateLink": {
            "uri": "https://armtemplates.azurewebsites.net/api/NewGuid",
            "contentVersion": "1.0.0.0"
          }
        } 
    } 
  ],
  "outputs": {
    "result": {
      "type": "string",
      "value": "[reference('MyGuid').outputs.guid0.value]"
    }
  }
}

Example 2. Getting 2 guids and using them later

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {},
  "variables": {},
  "resources": [ 
    { 
        "apiVersion": "2015-01-01", 
        "name": "MyGuids", 
        "type": "Microsoft.Resources/deployments", 
        "properties": { 
          "mode": "incremental", 
          "templateLink": {
            "uri": https://armtemplates.azurewebsites.net/api/NewGuid?numberOfGuids=2,
            "contentVersion": "1.0.0.0"
          }
        } 
    }  
  ],
  "outputs": {
    "result0": {
      "type": "string",
      "value": "[reference('MyGuids').outputs.guid0.value]"
    },
    "result1": {
      "type": "string",
      "value": "[reference('MyGuids').outputs.guid1.value]"
    }
  }
}

If you still have any doubt on how to consume the outputs, check the GAB Science Lab template on line 398 https://github.com/intelequia/GAB2017ScienceLab/blob/master/lab/assets/GABClient.json#L398

Hope this helps! Un saludo y happy coding!

[Video] Introducción a Application Insights

Se me había pasado de publicar en el blog el vídeo sobre Application Insights que edité el pasado mes de Mayo, pero mejor tarde que nunca Smile

Application Insights es un servicio de Application Performance Management (APM) extensible para desarrolladores web en varias plataformas, que sirve para supervisar aplicaciones web en directo. Se pueden detectar automáticamente anomalías de rendimiento además de proveer de herramientas de análisis que ayudan a diagnosticar incidencias comprendiendo lo que los usuarios están haciendo con la aplicación. En esta sesión veremos una introducción rápida al servicio viendo cómo se implementa y configura el servicio, tanto para aplicaciones web nuevas partiendo desde cero, como sitios web basados en CMS.

The storage account already exists error when redeploying an ARM template

Update 06 Jul 2016

Seems this is resolved on the latest Storage Resource Provider API 2016-01-01. The schema documented at https://azure.microsoft.com/en-us/documentation/articles/resource-manager-template-storage/ is for 2015-06-15, and that version didn’t support PUT operations. The new 2016-01-01 schema can be found at:

So this new template now works as expected and tags and other settings can be changed:

{

    “$schema”: “https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#”,

    “contentVersion”: “1.0.0.0”,

    “resources”: [

      {

        “name”: “dnntest20160705”,

        “type”: “Microsoft.Storage/storageAccounts”,

        “location”: “[resourceGroup().location]”,

        “apiVersion”: “2016-01-01”,

        “dependsOn”: [ ],

        “tags”: {

          “displayName”: “MyStorageAccount”

        },

        “sku”: {

          “name”: “Standard_LRS”

        },

        “kind”: “Storage”

      }

    ]

}

 

Notes:

  • Ensure you install the latest Azure PowerShell and SDKs or the latest 2016-01-01 version won’t be recognized
  • With the latest Azure SDK 2.9.1 available, I have syntax errors on the template, seems the schema was not included in the latest release. While you have syntax errors, you can deploy from Visual Studio with no problems. I suppose this will be fixed on the next Azure SDK release.

Thanks to Tom FitzMacken for the indications.

_____________________________________________________________________________

Original issue description

Seems that when deploying an storage account using an ARM template like the one below:

{

    “$schema”: “https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#”,

    “contentVersion”: “1.0.0.0”,

    “resources”: [

      {

        “name”: “dnntest20160705”,

        “type”: “Microsoft.Storage/storageAccounts”,

        “location”: “[resourceGroup().location]”,

        “apiVersion”: “2015-06-15”,

        “dependsOn”: [ ],

        “tags”: {

          “displayName”: “MyStorageAccount”

        },

        “properties”: {

          “accountType”: “Standard_LRS”

        }

      }

    ]

}

 

If you go and change the “displayName” tag value for something else or try to add a new tag, when you deploy the update I get the following deployment exception:

22:44:07 – [ERROR] New-AzureRmResourceGroupDeployment : 22:44:07 – Resource

22:44:07 – [ERROR] Microsoft.Storage/storageAccounts ‘dnntest20160705’ failed with message ‘{

22:44:07 – [ERROR]   “error”: {

22:44:07 – [ERROR]     “code”: “StorageAccountAlreadyExists”,

22:44:07 – [ERROR]     “message”: “The storage account named dnntest20160705 already exists under

22:44:07 – [ERROR] the subscription.”

22:44:07 – [ERROR]   }

22:44:07 – [ERROR] }’

22:44:07 – [ERROR] At D:\temp\azureresourcegroup2\Scripts\Deploy-AzureResourceGroup.ps1:98 char:1

Workaround

Change the tag value through the portal/PowerShell, so new deployment updates work.

 

This sounds like a bug, that will probably be resolved in a future release.

Password is expired when using Visual Studio Release Management

Today I was investigating an issue on Visual Studio Online Release Management, getting a deployment error related to the Azure credentials used for the deployment operation

Password is Expired summary

When going to the log details, the error happens on a resource manager task. The logs show that the password of the user account used to connect to Azure has expired.

Password is Expired

And here comes something to highlight, because the configuration of the service connection between VS Online and Azure has evolved in the latest months, to support both Azure Classic and Resource Manager models. Also note that the tasks you can configure on Visual Studio Release Management can work with one of these models or both. Let me show with an example.

Configuring Azure service connection on VS Online Release Management

To setup the Azure connection on Release Management, you need to click on the “Manage Project”

ManageProject

Once there, go the “Services” tab and when clicking on the “New Service Endpoint” you will see two ways to connect to Azure: Azure Classic and the new Azure Resource Manager. A few months back there was an only option to support Classic and RM scenarios, but this changed later. The main difference now is the way each connection authenticates with Azure

SNAGHTMLd1dcd0b

So a big difference here is the use of a service principal on Azure Resource Manager connections instead of using a user principal when using Azure Classic connections. This is important for our case, because the “Password is expired” error message we got refers to a user principal, not to a service principal where the “password” and “expiration” concepts are different.

Note that depending on the task used on Release Management, you can use one or other connection, or only the Classic connection. For example:

Fixing the “password is expired” issue

Once we understand these concepts, we have just to fix the password expiration issue. The solution has two steps:

  1. Change the password of the user principal and then update the Azure Classic connections with the new password. Note: You should use a long/very strong password for these user principals because of the second step; service principals come to play to stop using user principals in the future;
  2. To avoid this to happen again, change the password expiration policy for this account to don’t expire.

The first step can be easily done manually. For the second step, we need the help of Azure AD PowerShell module.

Install Azure AD PowerShell module

On this MSDN article you can find all the information related to managing Azure AD via PowerShell: https://msdn.microsoft.com/en-us/library/jj151815.aspx The Azure AD module is supported on the following Windows operating systems with the default version of Microsoft .NET Framework and Windows PowerShell: Windows 8.1, Windows 8, Windows 7, Windows Server 2012 R2, Windows Server 2012, or Windows Server 2008 R2. You need to install two utilities:

Connecting to Azure AD

Once you have installed both utilities, to start using all the Azure AD cmdlets just open a PowerShell command prompt.

$msolcred = get-credential
connect-msolservice -credential $msolcred

Obviously you need to introduce admin credentials if you want to use the administrative cmdlets later, like changing the password expiration policy.

Change the user principal password expiration policy

Once logged in, we can just change the password expiration policy for the user with this script:

# Gets the current policiy value
Get-MsolUser -UserPrincipalName “releasemanagement@mytenant.onmicrosoft.com” | select PasswordNeverExpires

# Changes the policy to never expire
Get-MsolUser -UserPrincipalName “releasemanagement@mytenant.onmicrosoft.com” | Set-MsolUser -PasswordNeverExpires $true

There is a good blog post about this at https://azure.microsoft.com/en-us/documentation/articles/active-directory-passwords-set-expiration-policy/

What happens with service principals? Passwords never expire?

Service principals works in a different way. When you create a service principal, you can specify the StartDate and EndDate for the security principal credential (by default, StartDate=Now; EndDate = Now + 1 Year). You can change the EndDate in a similar way (is not a boolean flag, you need to set the EndDate).

For more information, visit the MSDN article https://msdn.microsoft.com/en-us/library/dn194091.aspx

Azure Bootcamp 2016: cierre y hasta el año que viene!

Después de la resaca que nos ha dejado el Global Azure Bootcamp 2016, no nos queda más que dar las gracias a todos los que habéis puesto vuestro granito de arena en lo que ha sido la mejor edición de todos los bootcamps que hemos realizado durante los últimos años. Comenzamos en 2014 con cerca de 140 asistentes, el año pasado pasamos de 200, y este año, las cifras se han disparado a lo siguiente:
· 450 registrados 1 mes antes de la fecha del evento, llegando al límite de capacidad de las instalaciones. Al final sobre 390 asistentes en total
· 3 tracks de sesiones haciendo un total de 21 sesiones entre una pre-selección de 60 charlas. Todos los recursos de las mismas ya están enlazados desde la agenda en http://azurebootcamp.es
· Emisión en directo a través de Channel9, quedando publicadas en el mismo canal http://aka.ms/azurebootcampstreaming
· 13 patrocinadores duplicando el número de la última edición
· 1 Lab Global (RacingLab con backend en Azure) con 365 participantes simultáneos
· 187 localizaciones en 62 países alrededor del mundo
Hemos recibido feedback muy positivo y hemos tomado nota de algunas incidencias a mejorar en la próxima edición, estoy ya no hay quien lo pare. También hemos subido algunas fotos del evento que ya están enlazadas también desde la propia web del Azure Bootcamp, incluyendo la foto finish, de los que se quedaron hasta más de las 19:00 horas para el sorteo final (tenemos grabado El Preguntón en Channel 9!).
Una vez más, gracias a todos por vuestra asistencia, colaboración y sobretodo, por habérnoslo hecho pasar en grande. Nos vemos en 2017.
clip_image002

Azure Bootcamp 2016: a una semana del gran evento

bootcamp2016-150x101Parece mentira, pero ya ha pasado un año desde la última vez que pasamos un sábado estupendo aprendiendo sobre los últimos avances y nuevas características de Azure, un día que nos dejó a todos un buen sabor de boca compartiendo una experiencia global y con ganas de repetir de nuevo en 2016.
¡Y ese momento por fin ya llegó! El próximo sábado 16 de Abril nos vemos de nuevo y las cifras se han disparado.

Un evento global

Como ya sabéis, el Azure Bootcamp es un evento creado por y para la comunidad, donde voluntarios alrededor del globo se ofrecen para compartir conocimiento a través de sesiones técnicas, resolviendo dudas y contando las últimas novedades sobre la plataforma de nube pública de Microsoft. Este año se celebra en casi 180 ciudades alrededor del globo, por lo que si tienes una cerca, no pierdas la oportunidad. En este enlace tienes más información de todas las ciudades donde se celebra el evento: http://global.azurebootcamp.net/locations/
EventoGlobal
Para el evento que tenemos preparado para Madrid no van a faltar las sorpresas, y siguiendo la misma tónica del año pasado tendremos:

  • 3 tracks de sesiones técnicas, desde nivel para principiantes hasta para los más avanzados.
  • Un área “Azk the expert” para preguntar de primera mano a los que más saben sobre la plataforma y así resolver tus dudas
  • Racing Lab: ver detalles más abajo
  • Novedad: Hands-on Labs con premios directos por completarlos
  • Sorteos de licencias de software relacionado con Azure para hacerte la vida más fácil
  • Presencia de las empresas que en la actualidad más en contacto están con la plataforma, por lo que si tienes un proyecto que no sabes cómo meterle mano o buscas una nueva carrera profesional en el mundo cloud, puede ser tu gran oportunidad
  • Ah! Café y bocatas también habrá 🙂

Toda esta expectación de un evento que no ha hecho más que crecer a lo largo de los años, ha provocado que a un mes de la fecha llegáramos a completar el aforo que inicialmente habíamos calculado, después de conseguir más patrocinadores y así poder ampliar al número de participantes (al doble de la última edición), tuviéramos que colgar el cartel de “todo vendido”. Hemos hecho lo que ha estado en nuestra mano para liberar plazas que se han ido reservando de nuevo casi inmediatamente. Pero no os preocupéis, que ya tenemos algunas ideas para que podáis participar de algún modo en la experiencia, tanto en los hand-on labs como en algunos de los premios. Estad atentos al Twitter de @gwab_es para más información.

Racing Lab

Los que ya habéis estado en ediciones anteriores, recordaréis que se ha implementado un lab científico, que desafortunadamente este año la organización global no ha podido encontrar el algoritmo que cumpliera las expectativas. Pero una vez más, sí que estará disponible el Global Azure Bootcamp Racing Game Lab, un juego que ha sido diseñado para ejecutarse sobre Azure y conocer los detalles de cómo funciona para así aprender cómo se millones de datos de telemetría y finalmente explotarlos directamente desde una simple página web con Javascript.
El lab permite a los asistentes de todo el mundo a competir para conseguir las vueltas más rápidas en un juego de conducción 3D. Los servicios de back-end del juego están alojados en Azure y procesarán los tiempos por vuelta, telemetría y logros desbloqueados en el juego.
Ahora mismo los organizadores de todo el mundo hemos comenzado a testearlo para que esté a punto para el día del evento con más de 10.000 jugadores alrededor del globo, y por ahora, dejando el podio bien alto 🙂
Highscores1
Para conocer más detalles del juego puedes echarle un vistazo al siguiente video de Alan Smith, quien ha creado e implementado el mismo:

 

Patrocinadores locales

Por supuesto, la realización de este evento con los líderes de la comunidad de Microsoft Azure en España manteniendo la asistencia gratuita no sería posible sin la dedicación y duro trabajo de los presentadores, organizadores y contribuciones económicas de otras organizaciones para ayudar a financiar la logística del mismo.
patrocinadoreslocales

Patrocinadores globales

Agradecemos también a los patrocinadores globales por ofrecer esa guinda al pastel con las licencias de software que nos harán la vida más fácil a todos los que trabajamos con Azure.

¡Y no olvides ponerte en contacto con nosotros en http://azurebootcamp.es o a través de la cuenta de Twitter @gwab_es para cualquier duda que tengas!

Troubleshooting DNN based websites with Application Insights Analytics

Hi folks! I am really excited to finally be able to talk about one of the areas in which we have been working here at DNN Software over the last months. If you haven’t heard, some years back we started to deliver Evoq products (previously known as DotNetNuke Professional) as a Service. This is a cloud based Software as a Service solution called Evoq OnDemand, which runs on Microsoft Azure. We have been continuously evolving our solution, adapting to new Azure Platform changes and improvements such as improvements on Azure App Service, SQL Database v12, Azure Active Directory or Redis cache to name a few. We advantage our customers by leveraging new tools as they appear.
image001
To share some numbers, we have already delivered more than 50,000 websites including Evoq trials and production environments, backed up around 400 terabytes of site information. While the log data size isn’t huge, managing 160GB of site logs per month is not easy from the operational point of view, especially when we need to troubleshoot performance issues on one of our customer properties and try to find the root cause.
When an incident happens, our DevOps team need to figure out in minutes what is the cause, and in a cloud connected world the problem sources grow at the same rhythm that new service offerings appear: is an underlying infrastructure issue? Is a recent DNN update the cause? Is a 3rd party module? Is a 3rd party connected service? We needed to add telemetry and instrumentation to every single part of our cloud infrastructure, and not only customer properties but also our backend automatic provisioning systems.
We have been covering our needs with NewRelic, successfully allowing us to dig into the problems and solving operational issues and, meanwhile, keeping an eye on Microsoft Application Insights’ evolution. Our monitoring needs kept growing, looking for aggregate views (i.e. how many websites are experiencing the issue we discovered on a customer log entry? How many websites are using this 3rd party module and experiencing performance issues?). So we continued trying other application insights tools like NewRelic Insights and Splunk for more advanced scenarios. And during Q4 last year we saw a demo of what Microsoft was doing in this field to improve the current Application Insights service. On the first demo we saw 70 terabytes of data filtered in almost real time, an advanced web tool for complex lightning fast queries, a desktop tool for multiple account aggregate queries, and the ability to consume the queries from a PowerBI dashboard. It sounded like the foundation of what we were looking for.

Preparing the Application Insights Analytics onboarding

As I mentioned before, our cloud infrastructure is not only serving customer websites or trials. Some years back I presented at Cloudburst our set of cloud services for tasks such as automated backup and restores, Evoq product updates, order processing and account management. These initial services continued growing over the years and we have now new services for pageviews calculation (Evoq OnDemand is available in page view tiers), IFilter index offloading for Azure App service environment and others for background tasks. We also have continuous integration implemented for the nightly builds of Evoq and DNN Platform being deployed on Azure App Service. Having the ability to automatically send the Application Insights information to the Analytics store was the next requirement.
image003
The easiest path to have all the data available on Application Insights Analytics was to instrument each cloud service and website with Application Insights to start sending all the telemetry data. Once on Application Insights, all the information would be available for querying from Analytics using AQL (Analytics Query Language). So we finally worked on two areas:
1) Modifying all the worker roles (cloud services) to start sending the telemetry data to Application Insights. During the Connect(); event last year the Azure Diagnostics integration with Application Insights was announced, available with the Azure SDK 2.8. This was really easy to implement just by following the steps mentioned on the blog post and deploying a new version of each worker. In just a few minutes we started to have all the telemetry available on the Azure Portal. Kudos to the team for making this so easy;
image005
2) Create a new Application Insights monitoring provider to automate the Application Insights account provisioning and deployment on each website under our control. When we initially designed our backend monitoring services, we implemented a “monitoring provider” approach starting with Pingdom and NewRelic implementations. A monitoring provider is just an integration point in our platform, that supports methods like “install, uninstall, pause and resume monitoring” helping us, for example, to pause all the alerts on a website during maintenance or update operations. Our internal Application Insights monitoring provider implements this interface, automatically provisioning the account and alerts as well as pushing a web deploy package using the Resource Manager API. We can also run these operations manually by using our backend systems, through a web UI or by using our custom PowerShell cmdlets to provision and configure hundreds of Application Insights accounts with just a few lines of code:
image007
We can then visit the Azure Portal and check what is going on with each website or service and find performance issues and what is causing them: if the problem is on server side, a dependency, or is just a new skin that the customer has applied to the website that is performing badly as we can see in the graph below where the server response was consistent but the Page Load time skewed upwards, indicating client-side problems.
image009
For every single web application, we are able to search not only by page views, requests, traces or exceptions. Since we have implemented a custom logger for DNN, we are able to search by DNN Eventlog records or the typical log4net data being stored under /portals/_default/logs. We finally have one place where we can query for all the parameters.

Advanced search using Application Insights Analytics

And once we have all the telemetry data sent to Application Insights, is now when we can start running advanced queries by using the new Analytics feature.
Application Insights Analytics is a powerful query engine for your Application Insights telemetry that uses a query language named AQL. The language instead of nesting statements like in SQL, allows to pipe the data from one elementary operation to the next. We can filter all the raw telemetry data sent from each website by any field including DNN Eventlog records, execute statistical aggregations and immediately show the raw text results or with powerful visualizations.
As part of our automatic Application Insights provisioning, we create alerts for each resource being monitored. When we receive an alert we start using the tool to start digging into the problems to find patterns by using AQL. The UI allow us to save predefined queries and load them for later use.
image011
image013

Side Benefits of Machine Learning running on background: Proactive Detection

One thing that is amazing and is getting better day by day, is the Application Insights Proactive Detection. This feature notifies you about potential performance problems in your app, by using “Near Real Time Proactive Diagnostics”. What you get are alerts on abnormal rise in the failed request rate, and no configuration is required! It just works.
As example, check this alert we received today. I was shocked on the information provided by the service and how fast we go to the root of a problem.
On this case was a bot requesting bad formatted URLs and causing an abnormal rise in failed request rate. We detected the problem thanks to the stack trace provided on the alert that arrived 15 minutes after the proactive analysis, found the problem, created a patch and problem gone.
Do you love it? Me too!
image015

Application Insights module for DNN Platform

If you also have a DNN based website and want to get started with Application Insights and Analytics I have published on GitHub an open source module that allows to start sending all your website telemetry to Application Insights: pageviews, web requests, trace information (log4net log file contents), exceptions (including client side browser exceptions) and DNN Eventlog records.

Getting Started

The module is a DNN Platorm extension to integrate Visual Studio Application Insights to monitor your DNN installation. To setup the module on your installation, follow these steps:
1. Provision a new Application Insights service following the guide at https://azure.microsoft.com/en-us/documentation/articles/app-insights-overview/ Ensure you choose “ASP.net web application” on the “Application Type” parameter
2. Once provisioned, copy the “Instrumentation Key” available on the resource Essentials properties
image017
3. Now from the Releases folder https://github.com/davidjrh/dnn.appinsights/tree/master/Releases download the latest module package version ending on “…Install.zip” (the Source.zip package contains the source code that is not needed for production websites).
4. Install the extension package in your DNN instance from the “Host>Extensions” menu like any other module
5. Once installed, a new menu under “Host (Advanced menu)>Application Insights” will allow you to paste the instrumentation key obtained on step 2. After applying the changes, you will start receiving data on Application Insights after a few minutes.
image019
Un saludo and happy coding!