Using spatial data in the Windows Azure Blob Storage

[Infragistics] Mihail Mateev / Sunday, April 17, 2011

When talking about spatial data in the Windows Azure understand SQL Azure Spatial. Windows Azure presents other options when you file your sources for spatial data is to use Windows Azure Blob Storage. Due to the lack of good examples of this I decided to write this article.

Demo application is a Silverlight application for visualization of spatial data that uses data from the Windows Azure Blob Storage. This application uses Infragistics Silverlight XamMap component for visualization of spatial data.

Unlike my other articles about Windows Azure that will use in Windows Azure Emulator, which is installed with Windows Azure Tools and you can use to test your Windows Azure applications.

Windows Azure Blob Storage Concepts

Accessing the Windows Azure Storage is done via a storage account. A storage account can have many blob containers.
A container is a user-defined set of blobs that has only properties, which include a list of the blobs it contains. Containers don’t store data directly.

Defining the blob

The word blob stands for binary large objects. Blobs store binary data, such as images, XML documents, compressed (zipped or gzipped) files, and other content as an arbitrary array of bytes within a container that’s associated with a storage account.

Blob URL

http://<Account>.blob.core.windows.net/<Container>/<BlobName>

Example:

Account – sally
Container – music
BlobName – rock/rush/xanadu.mp3
URL: http://sally.blob.core.windows.net/music/rock/rush/xanadu.mp3

Types of Blobs

Block Blob

A Block blobs is a collection of blocks, where each block has a block ID (of 64 bytes). A Block Blob can be up to 200GB. You can upload a maximum of 64MB of data in a Put Blob operation to create a blob. If it is larger than 64MB it must be sent in multiple chunks of no more than 4MB, with Put Block operations.

Page Blob

A Page Blob can be up to 1TB is size and is organized into 512 byte pages within the block. Any point in the blob can be accessed for read or write operations by using the offside from the start of the blob.

Requirements to build a sample application:

Visual Studio 2010
SQL Server 2008 R2 Express or higher license. (to use a Windows Azure emulator)
NetAdvantage for Silverlight Data Visualization 2010 vol.3
The most recent version of Windows Azure Tools for Visual Studio
Windows Azure account

Steps to create the sample application

Create a Silverlight Application, hosted in an ASP.Net Web Application.
Add references to Microsoft.WindowsAzure.Diagnostics, Microsoft.WindowsAzure.ServiceRuntime and Microsoft.WindowsAzure.StorageClient from Windows Azure SDK.
Implement a WCF to upload and download files, using Windows Azure Blob Storage.
Add in the client Silverlight application XamMap component.
Implement shapefile loading from Isolated Storage to XamMap.
Create a new Windows Azure Storage Account and a Hosted Service (if you are using a Windows Azure Account).
Add a Web Role project in Solution
Edit settings in ServiceReferences.ClientConfig .
Publish the Windows Azure Cloud Service (if you are using a Windows Azure Account). :
Run the demo application

Create a Silverlight Application, hosted in an ASP.Net Web Application.

Add references to Microsoft.WindowsAzure.Diagnostics, Microsoft.WindowsAzure.ServiceRuntime and Microsoft.WindowsAzure.StorageClient from Windows Azure SDK.

Add in the client Silverlight application XamMap component.

<ig:XamMap  

            HorizontalAlignment="Stretch" 

            Margin="30" 

            x:Name="xamMap" 

            VerticalAlignment="Top" 

            ElementClick="XamMapElementClick"

            MapProjectionType="Equirectangular"

            Loaded="XamMapLoaded" >

    <ig:MapNavigationPane Name="navPane" Width="100" Height="300" Margin="0,0,0,0" 

            HorizontalAlignment="Left" VerticalAlignment="Top"/>

  

    <ig:XamMap.Layers>

            <ig:MapLayer 

            x:Name="World" Brushes="#4C2C42C0 #4C218B93 #4CDC1C1C #4C3D7835 #4C701B51" 

            FillMode="Choropleth">

                

                <ig:MapLayer.ValueScale>

                    <ig:LinearScale IsAutoRange="True"/>

                </ig:MapLayer.ValueScale>

            </ig:MapLayer>

        </ig:XamMap.Layers>

</ig:XamMap>

Implement a WCF to upload and download files, using Windows Azure Blob Storage.

Create a method UploadFile that receives file name and file content as byte array. This method saves a file in the blob storage.

#region UploadFile

/// <summary>

/// Uploads a file with specified file name

/// and content like a byte array.

/// </summary>

/// <param name="fileName">Name of the file.</param>

/// <param name="data">The data.</param>

[OperationContract]

public void UploadFile(string fileName, byte[] data)

{

  

    this.EnsureContainerExists();

  

    var newId = Guid.NewGuid();

  

    SaveShapeFile(newId.ToString(), null, fileName, @"application/octet-stream", data);

  

}

#endregion //UploadFile

#region EnsureContainerExists

 private void EnsureContainerExists()

 {

     var container = GetContainer();

     container.CreateIfNotExist();

  

     var permissions = container.GetPermissions();

     permissions.PublicAccess = BlobContainerPublicAccessType.Container;

     container.SetPermissions(permissions);

 }

 #endregion //EnsureContainerExists

SaveShapeFile writes file in the blob block isolated storage.

#region SaveShapeFile

 private void SaveShapeFile(string id, string description, string fileName, string contentType, byte[] data)

 {

     var blob = GetContainer().GetBlobReference(fileName);

  

     blob.Properties.ContentType = contentType;

     var metadata = new NameValueCollection();

     metadata["Id"] = id;

     metadata["Filename"] = fileName;

     metadata["Description"] = String.IsNullOrEmpty(description) ? "unknown" : description;

  

     blob.Metadata.Add(metadata);

     blob.UploadByteArray(data);

 }

 #endregion //SaveShapeFile

Create a method DownloadFile that receives a content as byte array for specified file name.

#region DownloadFile

/// <summary>

/// Downloads the file with specified file name.

/// </summary>

/// <param name="fileName">Name of the file.</param>

/// <returns></returns>

[OperationContract]

public byte[] DownloadFile(string fileName)

{

    try

    {

        var srcBlob = this.GetContainer().GetBlobReference(fileName);

        byte[] data = srcBlob.DownloadByteArray();

        return data;

    }

    catch (Exception)

    {

        return null;

    }

}

#endregion //DownloadFile

private void BtnUploadFile_Click(object sender, RoutedEventArgs e)

{

    //create an object for file open dialog

    _openDialog = new

    OpenFileDialog();

    _openDialog.Multiselect = false;

    _openDialog.Filter = "All Files|*.*";

    //show the file browser dialog to select file

    _openDialog.ShowDialog();

  

    if (_openDialog.File != null && _openDialog.File.Name != "")

    {

        string fileName = _openDialog.File.Name;

        Stream strm = _openDialog.File.OpenRead();

        UploadFile(strm, fileName);

    

    

    }

}

  

#region UploadFile

private void UploadFile(Stream strm, string fileName)

{

    byte[] buffer = new byte[strm.Length];

    strm.Read(buffer, 0, (int)strm.Length);

    strm.Close();

  

    UploadFile(buffer, fileName);

}

  

private void UploadFile(byte[] buffer, string fileName)

{

    UploadServiceClient client = new UploadServiceClient();

    client.UploadFileCompleted += ClientUploadFileCompleted;

  

    client.UploadFileAsync(fileName, buffer);

}

#endregion //UploadFile

  

  

  

#region ClientUploadFileCompleted

private void ClientUploadFileCompleted(object sender, System.ComponentModel.AsyncCompletedEventArgs e)

{

    if (e.Error == null)

    {

        MessageBox.Show(_openDialog.File.Name + ": Successfully Saved at ServerLocation");

  

        RefreshFilesList();

    }

    else

    {

        MessageBox.Show(e.Error.Message);

    }

}

#endregion //ClientUploadFileCompleted

  

  

#region RefreshFilesList

//Updates a list with file names from the server

private void RefreshFilesList()

{

    UploadServiceClient client = new UploadServiceClient();

  

    client.RefreshFileListCompleted -= ClientRefreshFileListCompleted;

    client.RefreshFileListCompleted += ClientRefreshFileListCompleted;

    client.RefreshFileListAsync();

    client.CloseAsync();

}

  

void ClientRefreshFileListCompleted(object sender, RefreshFileListCompletedEventArgs e)

{

  

    ObservableCollection<string> items = e.Result;

  

    this.FilesListBox.ItemsSource = items;

}

#endregion //RefreshFilesList

When download file from blob storage you need to save it in the Silverlight application Isolated Storage.

#region BtnDownloadClick

private void BtnDownloadClick(object sender, RoutedEventArgs e)

{ 

  

    if (this.FilesListBox.SelectedItem == null)

    {

        return;

    }

  

    string fileName = this.FilesListBox.SelectedItem.ToString();

  

    UploadServiceClient client = new UploadServiceClient();

  

    client.DownloadCompleted -= ClientDownloadCompleted;

  

    client.DownloadCompleted += ClientDownloadCompleted;

  

    client.DownloadAsync(fileName, fileName);

  

}

  

void ClientDownloadCompleted(object sender, DownloadCompletedEventArgs e)

{

    byte[] res = e.Result;

  

    string fileName = e.UserState.ToString();

  

    if (fileName != null)

    {

        WriteDataToStore(res, fileName);

    }

    

}

#endregion //BtnDownloadClick

#region WriteDataToStore

// Writes a file to an Isolated IStrorage, specified via its name 

// and content like a byte array

private void WriteDataToStore(byte[] data, string fileName)

{

    if (data == null || data.Length == 0)

        return;

    using (var store = IsolatedStorageFile.GetUserStoreForApplication())

    {

  

        IsolatedStorageFileStream fileStream = store.CreateFile(fileName);

        fileStream.Write(data, 0, data.Length);

        fileStream.Close();

  

    }

  

    RefreshIsoFilesList();

}

  

#endregion //WriteDataToStore

Implement shapefile loading from Isolated Storage to XamMap.

Shapefile is one of the first file formats, used to keep a spatial data.

More information about shapefile format you could find here:
http://en.wikipedia.org/wiki/Shapefile

Load the shape files from isolated storage using the ShapeFileReader:

You could redirect URI requests by registering custom prefixes in Silverlight.
The important line here being:

WebRequest.RegisterPrefix("iso",  

                    new IsoStorageRequestCreator());  

Which will register the iso:// prefix with our custom request creator.

More detailed information could be found in this post:

http://blogs.infragistics.com/forums/p/39754/224780.aspx

Create a new Windows Azure Storage Account and a Hosted Service

This sample will use Windows Azure simulator. Details, related with creating a Windows Azure Storage Account you could find in the article Migrating Spatial Data from SQL Server 2008 to SQL Azure

Add a Web Role project in Solution

When use a Windows Azure Emulator just set for Web Role settings for ConnectionString and DataConnectionString: “UseDevelopmentStorage=true”.

Edit settings in ServiceReferences.ClientConfig

<configuration>

    <system.serviceModel>

        <bindings>

            <customBinding>

                <binding name="CustomBinding_UploadService">

                    <binaryMessageEncoding />

                    <httpTransport maxReceivedMessageSize="2147483647" maxBufferSize="2147483647" />

                </binding>

            </customBinding>

        </bindings>

        <client>

                   

            <endpoint address="http://127.0.0.1:81/UploadService.svc"

                binding="customBinding" bindingConfiguration="CustomBinding_UploadService"

                contract="UploadService.UploadService" name="CustomBinding_UploadService" />

  

          <!--<endpoint address="http://localhost:37804/UploadService.svc"

                binding="customBinding" bindingConfiguration="CustomBinding_UploadService"

                contract="UploadService.UploadService" name="CustomBinding_UploadService" />-->

        </client>

    </system.serviceModel>

</configuration>

Run the demo application

Start the application

Upload files “cntry00.shp”, “cntry00.dbf” and “cntry00.shx” from local directory

Now parts of the shapefile are loaded in the blob storage.

Select each of the files from the blob storage and select “Download” button.
Now all files are saved in the Isolated Storage of the Silverlight application.

Select “Load from Iso” button to load shapefile.
Now data is loaded in the Infragistics XamMap component.

Navigate with the map. Enjoy! You have a working application that use a spatial data from Windows Azure blob storage.