DNN Azure AD Provider 3.0

Hi! After another bunch of work with React and the DNN Persona Bar, and with the special collaboration of Microsoft Azure MVP Cesar Abreu (@cesabreu), the new version of the DNN Azure AD Provider has been published.

Download DNN Azure AD Provider 3.0 from GitHub

There are several features on this release:

  • New Persona Bar integration: the Azure AD provider now has an area on the persona bar to easily setup the provider without having to dig into the authentication providers submenus. This area was also needed to have an starting point for the new features coming on the next release such as role sync and claims/profile mapping);
  • Auto-redirect: when you are only using the Azure AD provider, you would probably like to directly go to the AD login page without doing a previous stop on the DNN login page. This option just make this. If you still need, as an admin, to login with DNN credentials when this option is enabled, you can pass “legacy=1” on the query string of your login page in your browser so you can login as a regular DNN user. In any case, remember you can make a Azure AD user as super user of your site;
  • Setup simplification: on the previous releases, two applications needed to be setup in your Azure AD. This has now been simplified and only one App registration is needed. The other parameters have been simplified as well, so you only need to specify the Azure Tenant ID, the App ID and secret. Check the setup instructions on GitHub.
  • Logout: when I initially created the provider, followed the other DNN authentication providers patterns (Twitter, Google, etc.) where the logout process is not actually implemented, so when logging out, the user is logged out from DNN but the OAuth token is not expired on the OAuth provider. On this release, the logout process is fully implemented, so both the DNN cookie and the OAuth token are correctly expired. Check the video below:

Apart of these new features, other bugs have been fixed.

The project is available on GitHub as always:

Un saludo y happy coding!

DNN Redis Caching Provider 3.0

Hi again! Following up with the DNN module updates, I have done some modifications on the DNN Redis Caching provider so it can be configured now through the DNN Persona bar.

The summary of changes of this release is:

  • Changed the minimum required DNN version to 9.0.1
  • Refreshed nuget packages including the latest version of StackExchange.Redis 1.2.6
  • Added configuration UI in the Persona Bar. Now the provider is not automatically enabled during installation, you need to use the new UI.
  • The Redis client automatically reconnects after a Redis connection failure

RedisCaching

If you have any interesting idea to add on the settings area, please let me know. I have thought on implementing a redis-cli command line interface, but I believe that would be better to wait for DNN 9.2 and implement that as a “DNN Prompt” command (see http://www.dnnsoftware.com/community-blog/cid/155417/dnn-prompt-making-dnn-admins-power-admins-via-the-command-line)

Hope this helps.

Un saludo and happy coding!

Building an Evoq Liquid Content chatbot with Azure Bot Service

dnnrecipes2A few months back we spent some days working in an internal hackathon at DNN Corp. called DNN Developer Days that allowed to explore the power of Evoq Liquid Content APIs. The result was an awesome set of project examples giving a glimpse of what you can do when integrating the APIs to publish and reuse the content through different channels, such as Amazon Echo, Azure Bot Service or a smart TV, including AI and machine learning capabilities such as tagging your site images automatically. You can explore all of them at http://builtwithdnn.com website, I hope you find them interesting.

I personally worked on an Azure Bot Service chatbot, exposing the contents of a recipes website through different channels like Skype, Facebook Messenger, Telegram or Teams. You can quickly play with the example at http://builtwithdnn.com/recipesbot/ 

IMG_0365 IMG_0366 IMG_0367

All the documentation and source code for each project is available on a GitHub repository at https://github.com/dnnsoftware/Dnn.Evoq.LiquidContent.Samples.Public/ 

Chatbot as a Channel: Integrating Liquid Content with Azure Bot Service

Integrating Artificial Intelligence (AI) into your site or application is easier than you think. Companies like Google, Microsoft and Amazon are making big bets on AI and machine learning. All three companies provide freely available toolkits and services that developers can use to integrate AI into our applications.

The recipes bot integrates with messaging apps like Skype, Telegram or Facebook Messenger. As a Skype user, for example, you can provide the bot with a list of ingredients. The bot connects to Liquid Content to retrieve recipes that contain those ingredients, then provides an answer to the Skype user.

This is just the beginning. I plan to connect more third-party services to the recipe bot, such as LUIS, Cortana, Vision API and Apple Pay.

Bookmark this page and check back here to get updates. I’ll update the page each time there’s news to share. You can also follow me on Twitter @davidjrh and I will mention the new additions there.

Building the Recipes Bot

In this tutorial, I show you how I built the recipe bot. The bot uses the Azure Bot Service, the intelligent, serverless bot service that scales on demand. With this service, you will be able to publish your bot through multiple channels without managing or patching any server. It can connect to additional messaging apps without writing or adding new code. The service is free, and you will only pay for the resources you consume.

ArchitectureGetting Started

Prerequisites

To follow the tutorial, you will need:

  • An active Azure subscription. You will need to login as owner of the subscription to host the bot service. You can sign up for a free subscription here.
  • An Evoq Content or Engage site with Evoq Liquid Content enabled

Also, to debug your C# bot locally using breakpoints in Visual Studio you will need:

For more information about how to setup the debugging environment, visit Debug with the emulator and Debug an Azure Bot Service C# bot on the bot framework documentation.

Content Index

  1. Setting up the recipes
  2. Creating the basic bot
  3. Setup continuous integration
  4. Debugging the bot on your local environment
  5. Customizing the basic bot
  6. Test your recipes bot
  7. Adding a webchat in your site
  8. Known issues

Hope this helps.

Un saludo and happy coding!

DNN Application Insights v3.0

Hi all. After a while working with React/Redux and the new DNN PersonaBar model introduced on DNN Platform 9.0, I have started to update all my modules to avoid the use of the disappeared Host menu on DNN.

The first module updated has been the DNN Application Insights now available for download on GitHub. This includes:

  • Previous package updates to support version 2.4.0 of Application Insights, including enhanced live metrics streaming (committed by Mitchel Sellers)
  • New UI on the persona bar to setup the module settings
  • Requiring DNN Platform 9.0.1 or later

AppInsights_InstrumentationKey3

If you have any interesting idea to add on the settings area, please let me know. I have been checking all the configuration settings that can be done through the Application Insights configuration system and I will start adding them for the next release.

Updates for the Redis Caching Provider and Azure AD provider are the next ones.

Troubleshooting DNN based websites with Application Insights Analytics

Hi folks! I am really excited to finally be able to talk about one of the areas in which we have been working here at DNN Software over the last months. If you haven’t heard, some years back we started to deliver Evoq products (previously known as DotNetNuke Professional) as a Service. This is a cloud based Software as a Service solution called Evoq OnDemand, which runs on Microsoft Azure. We have been continuously evolving our solution, adapting to new Azure Platform changes and improvements such as improvements on Azure App Service, SQL Database v12, Azure Active Directory or Redis cache to name a few. We advantage our customers by leveraging new tools as they appear.
image001
To share some numbers, we have already delivered more than 50,000 websites including Evoq trials and production environments, backed up around 400 terabytes of site information. While the log data size isn’t huge, managing 160GB of site logs per month is not easy from the operational point of view, especially when we need to troubleshoot performance issues on one of our customer properties and try to find the root cause.
When an incident happens, our DevOps team need to figure out in minutes what is the cause, and in a cloud connected world the problem sources grow at the same rhythm that new service offerings appear: is an underlying infrastructure issue? Is a recent DNN update the cause? Is a 3rd party module? Is a 3rd party connected service? We needed to add telemetry and instrumentation to every single part of our cloud infrastructure, and not only customer properties but also our backend automatic provisioning systems.
We have been covering our needs with NewRelic, successfully allowing us to dig into the problems and solving operational issues and, meanwhile, keeping an eye on Microsoft Application Insights’ evolution. Our monitoring needs kept growing, looking for aggregate views (i.e. how many websites are experiencing the issue we discovered on a customer log entry? How many websites are using this 3rd party module and experiencing performance issues?). So we continued trying other application insights tools like NewRelic Insights and Splunk for more advanced scenarios. And during Q4 last year we saw a demo of what Microsoft was doing in this field to improve the current Application Insights service. On the first demo we saw 70 terabytes of data filtered in almost real time, an advanced web tool for complex lightning fast queries, a desktop tool for multiple account aggregate queries, and the ability to consume the queries from a PowerBI dashboard. It sounded like the foundation of what we were looking for.

Preparing the Application Insights Analytics onboarding

As I mentioned before, our cloud infrastructure is not only serving customer websites or trials. Some years back I presented at Cloudburst our set of cloud services for tasks such as automated backup and restores, Evoq product updates, order processing and account management. These initial services continued growing over the years and we have now new services for pageviews calculation (Evoq OnDemand is available in page view tiers), IFilter index offloading for Azure App service environment and others for background tasks. We also have continuous integration implemented for the nightly builds of Evoq and DNN Platform being deployed on Azure App Service. Having the ability to automatically send the Application Insights information to the Analytics store was the next requirement.
image003
The easiest path to have all the data available on Application Insights Analytics was to instrument each cloud service and website with Application Insights to start sending all the telemetry data. Once on Application Insights, all the information would be available for querying from Analytics using AQL (Analytics Query Language). So we finally worked on two areas:
1) Modifying all the worker roles (cloud services) to start sending the telemetry data to Application Insights. During the Connect(); event last year the Azure Diagnostics integration with Application Insights was announced, available with the Azure SDK 2.8. This was really easy to implement just by following the steps mentioned on the blog post and deploying a new version of each worker. In just a few minutes we started to have all the telemetry available on the Azure Portal. Kudos to the team for making this so easy;
image005
2) Create a new Application Insights monitoring provider to automate the Application Insights account provisioning and deployment on each website under our control. When we initially designed our backend monitoring services, we implemented a “monitoring provider” approach starting with Pingdom and NewRelic implementations. A monitoring provider is just an integration point in our platform, that supports methods like “install, uninstall, pause and resume monitoring” helping us, for example, to pause all the alerts on a website during maintenance or update operations. Our internal Application Insights monitoring provider implements this interface, automatically provisioning the account and alerts as well as pushing a web deploy package using the Resource Manager API. We can also run these operations manually by using our backend systems, through a web UI or by using our custom PowerShell cmdlets to provision and configure hundreds of Application Insights accounts with just a few lines of code:
image007
We can then visit the Azure Portal and check what is going on with each website or service and find performance issues and what is causing them: if the problem is on server side, a dependency, or is just a new skin that the customer has applied to the website that is performing badly as we can see in the graph below where the server response was consistent but the Page Load time skewed upwards, indicating client-side problems.
image009
For every single web application, we are able to search not only by page views, requests, traces or exceptions. Since we have implemented a custom logger for DNN, we are able to search by DNN Eventlog records or the typical log4net data being stored under /portals/_default/logs. We finally have one place where we can query for all the parameters.

Advanced search using Application Insights Analytics

And once we have all the telemetry data sent to Application Insights, is now when we can start running advanced queries by using the new Analytics feature.
Application Insights Analytics is a powerful query engine for your Application Insights telemetry that uses a query language named AQL. The language instead of nesting statements like in SQL, allows to pipe the data from one elementary operation to the next. We can filter all the raw telemetry data sent from each website by any field including DNN Eventlog records, execute statistical aggregations and immediately show the raw text results or with powerful visualizations.
As part of our automatic Application Insights provisioning, we create alerts for each resource being monitored. When we receive an alert we start using the tool to start digging into the problems to find patterns by using AQL. The UI allow us to save predefined queries and load them for later use.
image011
image013

Side Benefits of Machine Learning running on background: Proactive Detection

One thing that is amazing and is getting better day by day, is the Application Insights Proactive Detection. This feature notifies you about potential performance problems in your app, by using “Near Real Time Proactive Diagnostics”. What you get are alerts on abnormal rise in the failed request rate, and no configuration is required! It just works.
As example, check this alert we received today. I was shocked on the information provided by the service and how fast we go to the root of a problem.
On this case was a bot requesting bad formatted URLs and causing an abnormal rise in failed request rate. We detected the problem thanks to the stack trace provided on the alert that arrived 15 minutes after the proactive analysis, found the problem, created a patch and problem gone.
Do you love it? Me too!
image015

Application Insights module for DNN Platform

If you also have a DNN based website and want to get started with Application Insights and Analytics I have published on GitHub an open source module that allows to start sending all your website telemetry to Application Insights: pageviews, web requests, trace information (log4net log file contents), exceptions (including client side browser exceptions) and DNN Eventlog records.

Getting Started

The module is a DNN Platorm extension to integrate Visual Studio Application Insights to monitor your DNN installation. To setup the module on your installation, follow these steps:
1. Provision a new Application Insights service following the guide at https://azure.microsoft.com/en-us/documentation/articles/app-insights-overview/ Ensure you choose «ASP.net web application» on the «Application Type» parameter
2. Once provisioned, copy the «Instrumentation Key» available on the resource Essentials properties
image017
3. Now from the Releases folder https://github.com/davidjrh/dnn.appinsights/tree/master/Releases download the latest module package version ending on «…Install.zip» (the Source.zip package contains the source code that is not needed for production websites).
4. Install the extension package in your DNN instance from the «Host>Extensions» menu like any other module
5. Once installed, a new menu under «Host (Advanced menu)>Application Insights» will allow you to paste the instrumentation key obtained on step 2. After applying the changes, you will start receiving data on Application Insights after a few minutes.
image019
Un saludo and happy coding!

Visual Studio AppInsights module for DNN

DNNLovesAppInsightsAs a website developer or operator, I always need to know if my site goes down and get an alert, verify if the site is performing well or if it is under an attack. There are lot of tools today to give you an insight on what is happening on your web deployment, and one I’m using more and more is Visual Studio Application Insights.

While AppInsights is still a “Preview” service (note that preview means that is not generally available, so no SLA is offered yet), you can start from a Free tier that probably fits the majority of small websites, and optionally start paying depending on the amount of telemetry data you send to the store. This gives to you a powerful insight and tools to operate, diagnose and fix issues immediately.

“With great power comes…great number of alerts!” – David Rodriguez

I’m not going to start selling you all the benefits of using AppInsights or another service such as NewRelic. I’ve been personally using them both for a long time, and while NR has been on the first position on my insights tools list, now I’m getting very excited on what AppInsights is offering today. Here a few interesting highlights for DNN website developers and owners:

  • Monitor the usage and performance of live apps
  • Get immediate alerts on performance or availability issues
  • Get telemetry for existing web apps without redeploying
  • Use for a wide range of app types on devices, servers, or desktops
  • Monitor ASP.NET web apps hosted anywhere: on Azure, other cloud services, or on-premises servers
  • Search traces and exception logs for failure diagnoses (including DNN Event logs and log4net logs!!)
  • Track events, metrics, page views, users, crashes, dependencies, perf counters, and response times

AppInsights module for DNN

The only thing that perhaps is not easy and not documented at all, is how to setup AppInsights on a DNN instance. While I’ve been doing the task manually for a while, I have finally created a simple module available at https://github.com/davidjrh/dnn.appinsights that allows, on this initial version:

  • Easily setup AppInsights on a DNN Platform or Evoq installation as a Host user
  • Automatically send telemetry data to AppInsights:
    • Http requests information
    • Page views
    • Server and browser exceptions
    • Trace information including the log4net logs information
    • DNN event log entries with a new logging provider
    • Performance counters

There is still space for improvement, like adding UI to specify which performance counters you want to add (currently you have to manually edit the ~/ApplicationInsights.config file for this task). Feedback is welcome! Pull requests are welcome!

And that is not all since more features are continuously being added to AppInsights. Can you imagine a service that automatically learns how your site is being used and alert you if an abnormal pattern is detected? You would like to contact AppInsightsML@microsoft.com to try it out!

ProactiveDetection

Getting started

This module is a DNN Platform extension to integrate Visual Studio Application Insights to monitor your DNN installation. To setup the module on your installation, follow these steps:

  1. Provision a new AppInsights service following the guide at https://azure.microsoft.com/en-us/documentation/articles/app-insights-overview/. Ensure you choose "ASP.net web application" on the "Application Type" parameter
    CreateAppInsights
  2. Once provisioned, copy the "Instrumentation Key" available on the resource Essentials properties
    InstrumentationKey
  3. Now from the Releases folder https://github.com/davidjrh/dnn.appinsights/tree/master/Releases, download the latest module package version ending on "…Install.zip" (the Source.zip package contains the source code that is not needed for production websites).
  4. Install the extension package in your DNN instance from the "Host>Extensions" menu like any other module
  5. Once installed, a new menu under "Host (Advanced menu)>Application Insights" will allow you to paste the instrumentation key obtained on step 2. After applying the changes, you will start receiving data on AppInsights after a few minutes.
    ModuleSetup

What changes are done on my site?

Some changes are done during the installation and other when Enabling the Application Insights module. Note that by default, until you enable the module and specify an instrumentation key, any AppInsights module or assembly will be loaded and no telemetry data will be sent.

During the install, the following assemblies will be added to the ~/bin folder:

  • DotNetNuke.Monitoring.AppInsights.dll
  • Microsoft.AI.Agent.Intercept.dll
  • Microsoft.AI.DependencyCollector.dll
  • Microsoft.AI.PerfCounterCollector.dll
  • Microsoft.AI.ServerTelemetryChannel.dll
  • Microsoft.AI.Web.dll
  • Microsoft.AI.WindowsServer.dll
  • Microsoft.ApplicationInsights.dll
  • Microsoft.ApplicationInsights.TraceListener.dll
  • Microsoft.Web.XmlTransform.dll

After enabling the module, the following configuration files are changed, in order to send telemetry, dot4net logs tracing and DNN event logs to AppInsights:

  • /Web.config
  • /ApplicationInsights.config
  • /DotNetNuke.log4net.logs
  • /DesktopModules/AppInsights/js/appinsights.js

If you disable or uninstall the module, all the previous changes are reverted.

The following image illustrates the DNN event log data being sent to AppInsights, where you can search or filter by content, and why not, create some alerts based on any criteria.

EventLogProvider

 

Resources