Above The Cloud

I’m not asking out of my own self interest, I’m simply forwarding a question from David Aiken.

The Azure offering is very large; there is lots to consider. Live Services is huge just on its own, and the possibilities are endless, especially if you start thinking about Virtual Earth and Mesh integration. But once you understand the capability of Azure, its your own ideas and inventions that will unleash the true power. I don’t think it is the tools team responsibility to provide the ideas. No doubt there is capability there that we don’t know about, perhaps that could be a focus?

I’ve been pondering the question but nothing jumps to mind! I was thinking a little bit about extensibility. For example, how would you go about creating your own role? I think this would be quite easy though if you were happy just working from the existing role templates (web and worker). Or perhaps a breakdown of the different command line tools that come in the SDK, and examples of automating your development fabric deployment?

If you you’d like to see something specific in the Azure Services Training Kit, why not hit up David on his blog and ask him?

The February update was just released. There’s a full write up of it on David Aiken’s blog so I won’t just repeat those same details. However it is worth noting that this version has videos, and labs for using Live Services.

I still have the original kit on my machine which was about 50Mb but the new one is 150Mb ! I’m guessing its all those videos.

If you are starting with Azure, the training kit is the best resource available, and David’s blog is pretty good too.

One of the services of Amazon Web Services (AWS) is the Amazon CloudFront network. This is a pay-as-you-go service for distributing content world wide. The great thing about this service is that it operates under a pattern known as ‘Edge Services’, which means that your data is distributed to clients from a decentralized location. In other words, each client will hit a data store closer to them, much the same way as ‘mirrors’ work. These ‘edges’ serve your content even though you store it in one place. Amazon CloudFront takes care of the propagation. In this case, you store your content in Amazon Simple Storage Service (Amazon S3).

AWS has recently dropped pricing on their CloudFront services, as follows in a recent media release:

Effective February 1st, AWS has announced new pricing tiers for Amazon CloudFront, our high-performance, pay-as-you-go content delivery service. The new pricing tiers decrease the price of delivering content to as low as $0.05 per gigabyte delivered for high volume users. Find the new pricing tiers on the CloudFront detail page.

As of the date of this post, when you are working with Windows Azure Services, you get the option to create either a web role or a worker role (or both) for your Cloud Services project. But what are these roles exactly and what do they do?

Well you can think of each role instance as its own project. I tend to liken the two roles to existing Visual Studio project templates.

The Web Role is similar to a ‘Web Application’ – it has aspx pages and code behinds, but can also server anything that uses the http protocol, such as a WCF service using basicHttpBinding. The Web Role is driven by UI – the user interacts with a web page or service and this causes some processing to happen. As far as I can tell, the http pipeline is very similar to standard ASP.NET requests. Just think of it as a good old ASP.NET web application.

The Worker Role is similar to a windows service. It starts up and is running all the time. Instead of a timer, it uses a simple while(true) loop and a sleep statement. When it ‘ticks’ it performs some kind of maintenance work. This is great for background processing.

What the worker is working on is up to you of course, but usually it needs some data to work with. This data can come from a number of places. For example, the AzureServicesKit has examples that show you how to communicate between a worker and a web role via the use of a queue. The idea here is that the worker role doesn’t care how stuff got into the queue: it just does its job of processing items in the queue. It is the web role that is user driven and causes data to go into the queue. The web role can then monitor the queue (via some nicely placed Ajax) and show the results when they have been processed.

So you might think: I’ll always need a web role if I have a worker role. However, this is not true.

Think for a second about Live Mesh. Here you have a folder of all your holiday photos and you’ve modified the actual photos with metadata containing the name of the town or city you took the photo. Live Mesh is built on the same services as the rest of Azure, so communicating between the two is quite easy. You could potentially create a Cloud Service with just a worker role that monitors a particular Live Mesh folder for photos, and when a new one is added, it checks for that metadata with the city name, and inserts latitude/longitude coordinates as additional metadata.

Lets run with that example for a moment. Imagine that the Live Mesh folder is actually a shared folder with thousands of people having access to it (you can do that in Mesh). People are uploading their photos from cities all around the world. Your single worker role can’t keep up with the demand! That’s where Azure steps up: this is what it was truly built for. You can very easily add another instance of your Worker Role. All of a sudden, your processing time is halved!

So there you go. Currently there is a limitation of 2 instances per role. And you can only have one worker role and one web role in your solution at the most. This will seem very restricting once you start to really get into Azure but I suppose there has to be some limitations while Microsoft builds its datacenters around the world.

Steve Marx stated about a month ago (see the comment at the bottom of that link):

… we’re absolutely interested in giving the ability to have multiple worker roles, and multiple web roles, and some other kinds of roles, and direct synchronous communication between them …

He goes on to mention that Microsoft are careful about releasing functionality in a staged manner. I’m predicting that some of the roles we see will more closely match some of the existing projects in Visual Studio, in particular an ‘MVC Web Role’.

Oh before I wrap up, you can also write your worker roles in F# now! Check out the Microsoft site for the download.

If you are new to Azure you might be overwhelmed by what the various services do and don’t do. I don’t think the Azure site has done a good job of explaining how each service is used separate unto others.

Then today when I was searching the Azure Forums for something else, I came across this thread where Yi-Lun Luo explains what each of the four services is used for and some basic information about each.

Quoted:

Windows Azure:

• Write, read, and delete resources (in the form of Blob/Queue/Table) containing from 1 byte to 50 gigabytes of data each. The number of files you can store is unlimited.
• Each resource is stored in a container and retrieved via a URI in the form of <http|https>://<account-name>.<service-name>.core.windows.net/<resource-path>.
• Currently all data is stored in the United States, but can be accessed from anywhere. We’re building more and more data centers around the world.
• A container can either be public (the requests do not need to be authenticated) or private. Currently you can’t specify access control on a specific resource.
• Uses standards-based REST interfaces designed to work with any Internet-development toolkit. We’ve also built a wrapper library on desktop .NET platform (not Silverlight or .NETCF) for easier working with the APIs. In the future, maybe SOAP APIs will also be provided.
• Currently only http/https are supported. Additional protocols will be supported in the future.
• A resource is at least duplicated for three times to prevent data loss.
• Ability to host server side partial trust web applications in the cloud. Currently the only supported technology is .NET. Additional technologies, such as Java and PHP, will be supported in the future. Also less security limitations will be forced in the future. Client side applications can either be embedded in a web page (Silverlight/Flex) in a web role, or deployed on the user’s desktop and work with the storage APIs.

Live Services (AKA Live Mesh):

• Synchronize any kind of file on any device a user owns. The data will be stored in the Mesh as well as on the devices. Up to 5 GB data can be stored in the Mesh. Larger space will be provided in the future. Each file doesn’t have a size limitation.
• The device can be PC running Windows, Macs, Windows Mobile Devices, and Mesh Web Desktop. More devices will be supported in the future.
• All devices, files, applications, and other kinds of data stored in the Mesh are considered to be resources, and can be accessed in any authenticated applications.
• The end user controls which resource to be private, and which to be shared with friends.
• Resources can be accessed and updated through Atom Feeds in the form of REST APIs. A .NET library, a Silverlight library, and a JavaScript library are also provided for easier working with resources.
• Provides mechanisms to run web applications (Silverlight or AJAX) offline, outside of the browser. Modification of the data will be synchronized the next time this device goes online.
• Built upon Windows Azure. Can be taken as a case study of services hosted in Windows Azure.
• Part of the Live Platform which contains a lot of other services, such as Live Search and Virtual Earth. But I don’t know much about them…

.NET Services:

• Provides Service Bus to connect existing services behind firewall and NAT, and integrate applications.
• Provides Workflow Service to automate business process which involves calling a lot of services.
• Provides Access Control to determine which application/service can call a specific service or participate in the workflow.
• A desktop .NET SDK and a Java SDK have already been provided. More technologies will be supported in the future.
• Service Bus supports the http/https/tcp protocols. Services can communicate either one way or two way in any format (xml, binary, etc). It also provides NetEventRelayBinding to support multicasting.
• Access Control supports username/passwrod, CardSpace, and X509 authentication. More will come in the future.

SQL Data Services:

• Provides a cloud database to store data in the relational architecture.
• Supports CRUD operations over entities through standard-based REST and SOAP interfaces designed to work with any Internet-development toolkit.
• Each entity belongs to a container. Each container belongs to an authority. Entities do not have a specific schema. Two entities in the same container can contain different properties. This is called flexible schema. In the future, strict schema (similar to columns in traditional databases) will also be supported.
• Supports cross entity join queries in the same container. Cross container queries will be supported in the future.
• Provides blobs to store large data up to 100MB. Larger data will be supported in the future.
• More features, such as data warehouse and data mining, may come in the future.

Thanks for such a great explanation Yi-Lun!