Above The Cloud

Introduction

One of the things you have to consider in any application is configuration. In windows and web forms we have *.config files to help configure our application prior to start. They are a useful place to store things like provider configuration, IOC container configuration, connection strings, service end points, etc. Let’s face it – we use configuration files a lot.

In this article I will discuss the different types of configuration available to you, how they can be leveraged in your application, and how configuration items can be changed at runtime without causing your application roles to restart.

The Problems With Configuration in the Cloud

In Windows Azure applications, configuration can work exactly the same as standard .Net applications. If you have a web role, then you have a web.config. And if you have a worker role, you get an app.config. This allows you to provide configuration information to your role when it starts.

But what about configuration values you want to change after your app is deployed and running? It certainly is a lot harder to get in and change a few angle brackets in your web.config after it is deployed to production in the cloud. Do you really want to have to upload a whole new version of the app package with the new web.config file in it?

Or what about being able to change configuration aspects of all your running instances in one go, and not having to stop them from running to do so? Why should a configuration change necessitate a restart, such as is needed with web.config and app.config files?

In Windows Azure we have a new method of configuring our roles that gives us flexibility and consistency in our applications.

Service Configuration and Service Definition Files

In Windows Azure we can get the above benefits of flexibility and consistency in our configuration, via the service configuration file: ServiceConfiguration.cscfg.

This file contains information about your app that can be used at start-up, and can be changed at runtime without requiring a new package upload. This includes the number of running instances, certificate information, and specific application configuration settings, such as connection strings, port numbers, REST endpoints, etc.

The service configuration works in partnership with the service definition file: ServiceDefinition.csdef.

Both files are located in your main cloud project. The service definition is the “boss”. It can not be changed at run time. When you upload your Azure application, the service configuration is part of the application package, but the service definition is uploaded separately. This is because the service definition contains all the instructions about how an application should be deployed including: details about each of the roles, what their fault and upgrade domains are, TCP ports to open (for worker roles), what local storage should be available, VM size to use, level of trust, and more. After upload, the service definition is shipped off to the fabric controller who will parse it and work out what to do with your cloud package.

We’ll focus on one more thing you will find in the service definition file – information about configuration items. Essentially the service definition states what configuration values will be available in the service configuration file, for each role. It looks something like this:

   1: <WorkerRole name="ImageSearch.Cloud.Overlord">

   2:   <ConfigurationSettings>

   3:     <Setting name="ImageSearchSettings" />

   4:     <Setting name="ImageSearchDBC" />

   5:     <Setting name="SearchMultiplier" />

   6:     <Setting name="MatchTolerance" />

   7:   </ConfigurationSettings>

   8: </WorkerRole>

If a configuration setting name is not specified in the service definition, it can not be used in the service configuration file to configure your app. Think of it as an instruction to the fabric controller: “My app can be configured with these 4 values”. Also, once you’ve uploaded your application, you can’t add or remove settings, since the service definition file is not editable; you can only change the configuration values. I’ll show you how later on.

In the service configuration file, our entry would look like this:

   1: <Role name="ImageSearch.Cloud.Overlord">

   2:   <Instances count="1" />

   3:   <ConfigurationSettings>

   4:     <Setting name="ImageSearchSettings" value="UseDevelopmentStorage=true" />

   5:     <Setting name="ImageSearchDBC" value="Data Source=..." />

   6:     <Setting name="SearchMultiplier" value="3" />

   7:     <Setting name="MatchTolerance" value="50" />

   8:   </ConfigurationSettings>

   9: </Role>

Its pretty similar to the <appSettings> element you are probably familiar with. There is an element for every role in your cloud project, and the <ConfigurationSettings> child element contains all the settings of your application as “name/value” pairs.

Setting Configuration Values

When setting values in the service configuration and definition files directly, you will get complete intellisense, making it very easy to work out what the appropriate values are. It is also possible to set configuration values via the tooling.

In your main cloud project (which contains the links to all your roles) you can bring up the properties window for a role. Under the ‘Settings’ tab you can add new configuration values.

When adding a new setting, the IDE will insert the relevant placeholder in the service definition, and insert the value in the service configuration. This makes the Visual Studio IDE approach a bit of a time saver and helps ensure your definition and configuration files are in sync. In the above screen shot you can see we have two configuration items: the first is a cloud storage connection string, while the second is a database connection string. Essentially, all configuration items are strings, but the IDE gives you some shortcuts when creating configuration values for storage accounts via this connection string type. Clicking the ellipses […] delivers a new modal window where you can specify the details of your storage account.

In the end this just creates another string configuration value that can be used by the app.

<Setting

  name="ImageSearchSettings"

  value="DefaultEndpointsProtocol=https;AccountName=ImageSearch;AccountKey=3ob...UY=="

/>

Changing Configuration Values At Runtime In Azure

To change a configuration value of an app that is already deployed in Azure, you click the Configure button from your project, and this will take you to a screen with a textbox where you can edit all your values directly, or can optionally upload another configuration file. Once saved, that new configuration will take affect. The default behaviour of Windows Azure is to restart your instances now, using the new configuration. However this isn’t ideal, and we’ll look at ways to handle that differently.

Simple Handling Of Configuration Values In Code

The simplest way to leverage configuration items in your code is to use the API that comes with the Cloud tools. This provides a bunch of useful assemblies that give you strongly typed access to all the things you would want to do with your application. In the Microsoft.WindowsAzure.ServiceRuntime assembly (and namespace) we have a sealed class called RoleEnvironment which affords us a static method for accessing configuration values at runtime:

var tolerance = RoleEnvironment.GetConfigurationSettingValue("MatchTolerance”);

Handling Storage Connection Strings In Code

   1: public class StorageAccountFactory

   2: {

   3:     public static CloudStorageAccount Create()

   4:     {

   5:         return CloudStorageAccount.FromConfigurationSetting("ImageSearchSettings");

   6:     }

   7: }

The Catch With Using Configuration Methods In The Storage API

SetConfigurationSettingPublisher needs to be called before FromConfigurationSetting can be used

What this is trying to tell you is that you need to instruct the CloudStorageAccount to use the standard configuration definition file when loading configuration settings. That might seem a little silly at first.. where else could your configuration settings be coming from? Haven’t you just wasted half an hour reading about the Azure configuration files?

Well when running your app in Azure, your configuration will certainly come from the Azure service configuration file. But the tooling guys at Microsoft wanted you to create apps that could still be easily switched back to running on private infrastructure without having to remove all the Azure related integration points; and unfortunately the service configuration file is an Azure only integration point. We need to be able to factor that out into abstraction and make our apps transparent to the source of configuration settings.

Let me explain further with an example. Consider this scenario: you have a web role that puts items into a queue, and a worker role that grabs items out of the queue and processes them. You have your web role setup to use CloudStorageAccount.FromConfigurationSetting to load your storage client from the service configuration file. However later on you decide to move your web role on premise, and leave the other bits running in Azure. You detach your web role from the cloud service project; it now stands on its own two feet and can be run on IIS. But that also means you no longer have a service configuration file; you’re stuck with plain old web.config again. Ideally you should still be able to use the same static method to load your storage account from configuration settings, its just that those configuration settings are now in a different place.

The SetConfigurationSettingsPublisher method that the error refers to is another static method on the CloudStorageAccount class that lets us specify where configuration should be loaded from. According to the MSDN documentation:

This method should be called once to set up the environment. The environment could be the Windows Azure runtime, in which case the publisher is a simple wrapper of the configuration reading and change event. The environment could also be a .NET environment, in which case the developer can hook up a custom configuration reader and change notification.

As stated in the documentation, you should only setup the configuration publisher once, so this ideally should happen in your role’s OnStart event. The code to tell CloudStorageAccount to use the configuration definition uses a delegate which is stored and called every time the configuration is requested:

   1: CloudStorageAccount.SetConfigurationSettingPublisher(

   2:     (configName, configSetter) =>

   3:         configSetter(RoleEnvironment.GetConfigurationSettingValue(configName))

   4:     );

In essence, a configuration setting publisher is just an Action – a delegate that is called when attempting to get the value of a configuration setting name. In the code above we are telling the storage account client that whenever it needs to get a configuration item (configName) then call the standard RoleEnvironment.GetConfigurationSettingValue static method with that name to find and return the value.

How Can I Make My Application Utilise This Abstraction

Great question! Lets say we actually wanted to make our code more flexible and be able to switch between the configuration definition when running in Azure, over to web.config appSettings when running in plain old IIS on our own server. We would like our app to detect that it is no longer running in the world of Azure, and there is a property we can use that does just that: RoleEnvironment.IsAvailable. We can use it to customise who our setting publisher is. Consider the following static method on a factory class that I have created called StorageAccountFactory:

   1: public static Action<string, Func<string,bool>> GetConfigurationSettingPublisher()

   2: {

   3:   if (RoleEnvironment.IsAvailable)

   4:     return (configName, configSetter)

   5:       => configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));

   6:

   7:   return (configName, configSetter)

   8:     => configSetter(ConfigurationManager.AppSettings[configName]);

   9: }

You would then call the new method like this:

   1: CloudStorageAccount.SetConfigurationSettingPublisher(

   2:     StorageAccountFactory.GetConfigurationSettingPublisher()

   3: );

Walla! Our app is now aware of when it is and is not running in Azure and can switch its settings provider as needed, while still giving us the benefit of using the CloudStorageAccount API to load our storage account from configuration.

Handling Configuration Changes At Runtime

As I mentioned earlier, you can change configuration values at runtime while your app is already strolling along happily in production. The reasons for needing to make a change can vary; perhaps you want to tweak a performance setting or change the format of log file output.

Most importantly, you don’t want your app to always stop and restart on a configuration change. The default behaviour of Azure roles is to restart on any configuration change so if you want to prevent this behaviour, read on!

You can detect changes to the role environment by handling the 2 following static evenst, available on the RoleEnvironment class:

RoleEnvironment.Changing += RoleEnvironmentChanging;

RoleEnvironment.Changed += RoleEnvironmentChanged;

When you create a new web or worker role, the first event is automatically inserted for you, along with the event handler code:

   1: private void RoleEnvironmentChanging(object sender, RoleEnvironmentChangingEventArgs e)

   2: {

   3:     if (e.Changes.Any(change => change is RoleEnvironmentConfigurationSettingChange))

   4:     {

   5:         e.Cancel = true;

   6:     }

   7: }

The default behaviour of this code is to ‘cancel’ the event on any configuration change. Cancelling the event is a bit misleading; what this actually means is that the role will be recycled, configuration changes will be applied, and the role will start up again. So “e.Cancel” actually means “e.Reboot”.

Naturally we need to make changes to this default code. Its up to you to decide which changes require the role to restart and which ones don’t. Personally, I like to make a static array of names of configuration items that I don’t want to cause a restart, like so:

private static readonly string[] ExemptConfigurationItems =     new[] { "MatchTolerance", "SearchMultiplier" };

Then in the RoleEnvironmentChanging event handler we can decide whether or not to reboot, and if we don’t need to reboot, apply the new configuration changes in the RoleEnvironmentChanged event:

   1: private void RoleEnvironmentChanging(object sender, RoleEnvironmentChangingEventArgs e)

   2: {

   3:    Func<RoleEnvironmentConfigurationSettingChange, bool> changeIsExempt =

   4:       x => !ExemptConfigurationItems.Contains(x.ConfigurationSettingName);

   5:

   6:    var environmentChanges = e.Changes

   7:        .OfType<RoleEnvironmentConfigurationSettingChange>();

   8:    e.Cancel = environmentChanges.Any(changeIsExempt);

   9:

  10:    if (!e.Cancel)

  11:    {

  12:        var oldMultiplier = RoleEnvironment

  13:            .GetConfigurationSettingValue("SearchMultiplier");

  14:    }

  15: }

The RoleEnvironmentChangingEventArgs supplies a Changes property which is a list of all changes that caused this event to fire. Typically these will be a change of the number of instances that the role is running, or a change to configuration items. We request all the changes that are of type RoleEnvironmentConfigurationSettingChange which represents just setting changes. We then compare the changed items to our static list of configuration names, and if any of the changes aren’t in the list, we’ll do a reboot. If we don’t do a reboot (which means all changes belonged to our exempt list) we will apply those new changes as necessary.

Summary

And that’s all there is to know about configuration at this point in time. Here’s a couple of points summarising what we’ve learnt:

• Configuration values can be changed while your app is still running
• Default behaviour in such cases is to reboot the role
• Configuration is just XML but the Visual Studio IDE gives nice design time experience
• Connection strings are a special kind of configuration string that store details for a Windows Azure Storage account
• Using the API we can abstract our settings publisher to create flexibility of usage in Azure and non-Azure applications
• Runtime configuration changes can be checked and instance recycling can be avoided if necessary

Please keep in mind that this was valid at the time of writing and that the Azure platform is continually evolving, so there might already be a better technique for what you want to do.

Additional Links

• Overview of Service Architecture
• Changing The Windows Azure Service Configuration when running on the DevFabric
• Windows Azure Services – Exercise 2: Configuration, Logging, and Debugging

Last Thursday evening (18th Feb 2010) I did a presentation to the Gold Coast .Net SIG about developing applications for the Windows Azure Platform. The talk went for almost 2 hours (for that I apologise!) and seemed well received.

I started with an overview of the platform, and then dived into Windows Azure, SQL Azure and .Net Services. I used a demo application that performed searches for images based on colour content, using Azure to do the grunt work of analysing images.

You can find the files from that presentation here:

• Slide Deck PPTX (1.9Mb)
• ImageSearch Demo Application (2.9Mb)

The night seemed to go well, except for the fact that my SQL Azure account expired on that very day, mere hours before my presentation. I received no warning email and was pleasantly surprised by the little ‘disabled’ flag against my SQL Azure account.

I will now spend the next few days trying to work out what I have done to anger the demo gods. In the meantime, stay tuned, as future blog posts will break down that demo application to talk about some of the techniques used to share code, use configuration settings, and utilise database access, all in a fashion that is agnostic to whether we are using the developer fabric, or the production Azure fabric.

If you attended on the night, I’m more than happy to hear any feedback about the presentation you may have. I am trying to continuously improve as a presenter and appreciate any constructive criticism you can provide.

Foreword: Apologies for the title, I’m still not sure (after completing the entry) what it should be called.

Why would I use the Windows Azure Platform? Its a good question, one that I’ve had a lot of internal discussions on lately (fellow consultants from Readify). For most its quite a paradigm shift and therefore difficult to grok. Many believe that you can only build Azure apps from scratch and that it would be a lot of work to convert an existing product to use the Azure platform. I don’t want to say this is a false statement, but Azure is pretty big, bigger than most people realise.

I Can’t Use Azure Because…

Most of us are only familiar with the Windows Azure component which allows you to host either services or web pages. Most custom applications require a data store and for Microsoft developers, this tends to be SQL Server. Unfortunately there is no one-for-one mapping to easily port your database over to the data stores in the cloud (Windows Azure Storage, SQL Azure, etc). This means people feel resistance when they do consider the Azure services and write-off the whole platform.

When SQL Azure is presented as an option, people tend to pick on the little features that are missing, expecting a direct cloud equivalent for their data. But some things just don’t make sense for SQL Azure. People lose context: SQL Azure is a service. It should be treated like any other service within your architecture. You don’t need to be implementing the next service related buzzword (SOA, SaaS, S+S, etc) to get the benefits of the abstractions provided by services. If you build your services in an autonomous fashion, SQL Azure will fit right in with your story.

What about off vs on-premise data storage? I hear this one a lot as well. I want to control my data, I have no need for Azure. This is an easy answer for me, and I defer back to my last statement. Its a service driven world, your data should be the same. Where that service is located should be irrelevant. This is where the .Net Services ‘Service Bus’ steps up. It can route your data to your other services/applications extremely easily, with top level security all the way, so not even Microsoft can see what is being transmitted. What about my firewall, it will stop people from seeing my service right? No problem – the service bindings use outbound connections to connect to the Service Bus, meaning that you can host your own services on-premise, behind your firewall, and still get flawless connection to/from the cloud.

The Next Step In Developer Evolution

Modularity, loose-coupling, single-responsibility.. any of these terms ring familiar for you? As a developer I enjoy building applications that realise these design qualities. I use Inversion of Control and Dependency Injection wherever possible, and have realised that my code classes have somewhat turned into services. Not like ASMX or WCF services; just classes that have very defined purposes, are abstracted by a contract (interface), and provide a service to the rest of the application.

I won’t lie, since being introduced to IoC my programming style has changed, and I think that this happens for most developers who use it as well (regardless of implementation technology or language). You start viewing everything as services, even your Views (since we all love MVP, MVC, and MVVM). Quite often when I’m with a client discussing aspects of an application I might say “.. so we’ll just create a service for blah…” not realising that it might have a different meaning to them (however its not long before they agree with me and start referring to things the same way!).

So I honestly believe that services are the next level of developer evolution (“devolution” anyone?), whether we mean actual web services or just classes that service our application. For me this even brings about new meaning for Service Oriented Architecture – design your systems to be service based whether those services are across the wire or not.

<Disclaimer> I am an advocate of smart clients and believe everything should be services and that web pages are evil and should only exist to help find smart clients</Disclaimer> 

So I Have To Redesign My App As Services?

Yes. Well no, but you should certainly think about it for all new applications, regardless of whether you are considering Azure or not. I honest believe it is just good design. As for existing apps, consider the key dependencies. These might be file storage, relational database, data mining, 3rd party components, etc. Write a list and detail each one individually. Is their a mechanism you can use to abstract the calls to that dependency? Perhaps IoC will work for you, or perhaps the ASP.Net Provider model will help you. Or perhaps you just need to wrap the calls up in a new class in a separate assembly.

The next step is how those dependencies will work in the cloud. Do you need to replace any? Can they also be moved to the cloud? Do they need to be rewritten as services? Do you need to expose their functionality from within your organisational boundaries? This is where a lot or a little work may need to happen. Those applications that already have looser coupling will find this part easy. For example, you may already have a class that handles your database calls. A customer class might have a create, update, and delete methods, which wraps calls to LINQ to SQL (L2S). Well its very easy to indentify your customer class as the service boundary, and move the L2S code to a new service. All you need now is to think about the security and you’re done!

Once you have a firm plan for dealing with those dependencies, the rest should be a piece of cake. Obviously what happens next will differ for web and smart client apps, but the dependencies are really the most difficult thing. When you have hard wired dependencies to 3rd party solutions (eg. K2) you might find that it is very difficult making the transition to Azure.

What About Products?

Well the same rules apply, only that you might get an added benefit from creating products that can easily by run on Azure. For example, check out SplendidCRM – a CRM product recently refactored to run on Azure. Why? Well what if you want a CRM product but don’t want to host it yourself? Kinda seems smart to me… allow the purchaser of your software to run it anywhere they choose, on or off premise. And then of course there is the multi-tenant applications designed to only run in the cloud, like Saasu and Salesforce. If you want to go viral, you need to be prepared to scale quickly!

Summary

Its hard to say exactly when you should consider Azure, but I guess the point of this post is not so much to answer that question, but rather to provide you with some insight into good design techniques, that when followed properly, will provide you with an application that can easily be deployed into Azure, or any other cloud platform for that matter. If we’re doing things right at the bottom level, then we can be flexible with the higher level architectural choices.

So please check out the links for any of the concepts I’ve mentioned above (or listed below) to ensure you and your team are familiar with them. And if you have any questions, please feel free to email me.

Some more links:

• Introducing the Windows Azure platform
• Introducing Windows Azure
• Windows Azure and ISVs – A Guide for Decision Makers
• Overview of Microsoft SQL Azure
• Windows Azure Platform Training Kit
• An Introduction to Microsoft .NET Services for Developers
• A Developer’s Guide to the Microsoft® .NET Access Control Service
• A Developer’s Guide to the Microsoft® .NET Service Bus

To answer that we first have to look at the term ‘fabric’.

You can think of a fabric (or ‘switched fabric’) as a network of interconnected nodes. The connection is facilitated by high-speed switches through fibre-optic connections, while nodes are usually servers and other technical infrastructure.

If the network is large enough and viewable from a distance, it could look like a ‘weave’, or a ‘fabric’. This terminology is also used at the processor level describing the network of buses on the chip.

The Azure Fabric is not much different. It is made up of nodes, which are servers running Windows Server 2008 and load balancers, while the edges are the power, Ethernet, and serial communications. Each computer node might be a dedicated server, or a server petitioned for multiple virtualised environments.

When we write our Azure applications locally, there is a simulation of this fabric running on our machines, called the ‘Development Fabric’. This lets us simulate load balancing and multiple nodes for our web and worker roles.

The Development Fabric is facilitated via a series of running processes that simulate load balancing and hosting your application.

When you run a project of type ‘Cloud Service’ (the project that contains a service definition and Role links out to other projects), the development fabric is initiated (as well as a simulation of local development storage). In the diagrams below, we see the process tree as initiated by Visual Studio and the location of the files. 

It is actually possible to spin these services up yourself, rather than needing Visual Studio to do it. Although I haven’t actually tried, it should be possible to copy these files to a testing server and run them up so that you can hand your application over to your testers.

Another thing you might be able to do (I have yet to try this as well) is spin up the load balancer and various services inside unit tests. Most of these executables are in managed code and trusty old Reflector can bust them open for us. Looking at the code for DFloadbalancer, we see that it is simply simulating a load balancer through usage of WCF, binding via a Named Pipe. We are given the load balancer classes in the LoadBalancer.dll so I’m confident there is some cool unit test automation to be achieved via directly calling the necessary classes. If you do go down this track, please let me know how far you get.

In a future post I will be discussing the Azure Fabric Controller.