Managing assets across multiple storage accounts In Windows Azure Media Services and defining load balancing strategy
In Windows Azure Media Services 2.2 we introduced concept of associating multiple windows azure storage accounts with media service account.
Main purpose of this feature is to enable users to scale their assets across multiple storage accounts to overcome azure storage max limit of 200TB.
In this article i will demonstrate how to create simple load balancing strategy for asset creation to place asset into storage account with least occupied space. Most of examples will cover Windows Azure Media Services .NET SDK which can be installed as Nuget Package. See http://www.nuget.org/packages?q=windowsazure.mediaservices for package details.
Implemented below solution is available as sample in Azure Media Services GitHub repository. http://bit.ly/1chUG2O
Before we jump into code let’s explore few concepts:
When you creating Windows Azure Media Service through portal or management api you specifying your default storage account:
Once your account is created you will be able to see information about your default storage account associated with media service account in .NET client SDK and list through all associated accounts:
As you can see interface exposes just name and IsDefaultProperty. You can use azure storage sdk to get more information about account itself.
In order to link additional accounts you can utilize azure management rest api : http://msdn.microsoft.com/en-us/library/windowsazure/dn313102.aspx
When you creating your asset you can pass IStorageAccount as parameter to asset create method and system will create asset within selected storage account:
context.Assets.Create("MyAsset", "storageAccountName", AssetCreationOptions.None);
Implementing asset load balancing strategy across storage accounts
Since multiple storage accounts feature in Media Services has been introduced to overcome limit of 2TB storage there are few potential ways you can select storage accounts:
- Red Robin selection
- Select account with least occupied space
- Once first account reaches certain space threshold, start creating asset in the next account from list
Let’s pick up second strategy from a list to select account with least amount of occupied space.
Internally you need to map IStorageAccount from Media Services SDK to appropriate CloudStorageAccount from Azure Storage SDK:
Then implement selection strategy we need to make sure that we have storage account metrics data. By default storage analytics data is disabled and you have to enable it for a given storage account.
It can be done by calling storage sdk:
Once analytic feature has been enable Windows Azure will create table $MetricsCapacityBlob in your storage account and will start writing daily metrics there. Please not that this data is available within 24 hours.
To read data from table you can use following snippet:
Once you get data Metrics for your account it just a matter of applying required logic:
If you looked into AccountLoadBalancing /LeastCapacityOrDefaultAccountStrategy.cs file you will find that for a given list of accounts i am reading available analytics data and performing following algorithm:
- If account doesn’t have analytics data for last 2 days then it is not participating in selection
- If account has analytics data and storage capacity of account is least among other accounts then account is selected.
You can find sample solution by going to http://bit.ly/1chUG2O
Your final usage will look similar to this:
Tags: .net media services sdk, azure storage account, media services
Pingback: Media Services のための Storage | Microsoft Azure Japan Team Blog (ブログ)()