The XAML Data Chart with High Density Scatter Series

Damyan Petev / Monday, June 4, 2012

It’s an undeniable fact that charts are irreplaceable tools for visualizing large amounts of data for ‘at-a-glance’ tend spotting and analysis. While our XAML tools offer quite the range of chart types, you can say that the XamDataChart is the heavy lifter control. It has the blazing speed and it has an array of features to satisfy almost any need! We’ve talked high-performance charting plenty of times so far and you can even read the performance whitepapers and with charts it’s all about ‘squishing’ all those values in a representation that would fit limited space and make sense and draw conclusions from. We’ve also talked analysing scattered data by calculating correlation. But how about when you have some really extreme amount of data to crunch? Would it even be clearly visible when displayed? Would it impact performance too much? To put it that way - what if you data is big enough to speak for itself and just needs the proper presentation? It’s a very possible case and the getting equipped to handle just that!

Introduction

With this release a very intriguing new series type were added to the XamDataChart – a High density scatter series. Mind you still a CTP feature, but already up there on the impressive scale. This new type is specialized and optimized for displaying huge data sets in the range of hundreds of thousands to millions! For such ambitious task the series display the scatter data as tiny dots rather than full size markers due to the sheer numbers. However as you might imagine that is simply not enough as with so many, values as bound to come too close or overlap and that’s a hard thing to display when you want to also scale them down within the chart’s bounds. For that reason the series offer additional rendering for those and will represent and color-code such points in ‘clusters’ – pretty much in a heat map style. That makes it crystal clear where the most points are. That’s the data speaking for itself I meant and I’ll let it do just that:

The XAML data Chart with high density scatter series displaying 300 000 data points.

And that is “just” 300 000 points! The red coloring indicates higher point density in the area and as that goes up the red gets brighter. Of course that isn’t actual data, more of a ‘random’ generated collection like:

/// <summary>
/// The data collection we would use with a contructor to manipulate the data size.
/// </summary>
public class ScatterDataCollection : ObservableCollection<ScatterDataPoint>
{
    public ScatterDataCollection(int count)
    {
        Random random = new Random();
        this.Clear();
        for (int i = 0; i < count; i++)
        {
            double x = random.Next(random.Next(1000));
            double y = random.Next(random.Next(1000));
            this.Add(new ScatterDataPoint { X = x, Y = y });
        }
    }
}
 
/// <summary>
/// A very basic (lightweight) point class
/// </summary>
public class ScatterDataPoint
{
    public double X { get; set; }
    public double Y { get; set; }
}

I’m only listing this block to show that I’ve restricted the randomness of the values by putting a random max value less than 1000 and that tends to generate more and more points closer to the 0 and that is why the ‘gradient’ effect on top is observed. Then again i haven’t looked at the actual numbers generated to tell you this – I was looking at the chart!

Getting Started

Using the new series is very easy. Assuming you are already aware of how to add a chart to your project ( Silverlight guide / WPF Guide ) the more data-packed series inherit all the major properties from the scatter series so adding them looks pretty much the same:

<ig:XamDataChart x:Name="xamDataChart"
                 HorizontalZoomable="True"
                 VerticalZoomable="True">
    <ig:XamDataChart.Axes>
        <ig:NumericXAxis x:Name="xAxis"/>
        <ig:NumericYAxis x:Name="yAxis" />
    </ig:XamDataChart.Axes>
    <ig:XamDataChart.Series>
        <ig:HighDensityScatterSeries ItemsSource="{Binding}"
                                     XAxis="{Binding ElementName=xAxis}"
                                     YAxis="{Binding ElementName=yAxis}"
                                     XMemberPath="X"
                                     YMemberPath="Y">
        </ig:HighDensityScatterSeries>
    </ig:XamDataChart.Series>
</ig:XamDataChart>

The generated collection is set to the data context of the chart in code-behind:

var data = new ScatterDataCollection(300000);
this.xamDataChart.DataContext = data;

With performance in mind

The series provide a number of dependency properties (so you can bind and manipulate in run time easily) that can affect the charts looks and performance.

Resolution

One important feature that comes with the chart representing higher density heat-map-style is the option to control the resolution of that representation. The series expose a resolution property on which you assign the actual pixel size of the building blocks for the representation. That means the the lower you set it, the closer to the actual data it would be and more points will be rendered. However, setting higher value would mean less rendering for the chart and therefore, performance will be much better. The whole point is that you can adjust the chart to your needs – based on the size of data and the performance required. The first screenshot was with resolution 2 and setting it to 4 would result in the following:

The high density scatter series with heat map resolution set to 4.

With high amount of points going for the millions you probably won’t be able to analyse each, so this way the chart will be much more responsive and you can still clearly tell where the majority of values are.

Progressive loading

Another feature that is here to help visualize oversized data to the end-user while still providing the best experience. The chart can render the series progressively – meaning they will load in pieces and for the whole period of loading the UI will remain responsive. That is quite frankly a huge benefit if you have a million points or so and it would probably take a few seconds ( on my humble machine a million takes about 5 seconds). Freezing the UI for as long in not a good experience at all and while it takes a second or two longer, rendering the chart in bits is a massive improvement and there’s even a bonus – the series offer an event that will be fired repeatedly to help you monitor an relay the loading status:

<!-- Progressive loading snippet -->
<ig:HighDensityScatterSeries ItemsSource="{Binding}"
                             ProgressiveLoad="True"
                             ProgressiveLoadStatusChanged="HighDensityScatterSeries_ProgressiveLoadStatusChanged"
                             />

Then if you have a favourite progress monitoring UI control (the default progress bar or custom ones) you can use them to show the advance to the users. In my case I’ll have this super simple rectangle and manipulate its width:

<Border Height="15" x:Name="border" Background="#502E1C1C" Margin="2">
     <StackPanel Orientation="Horizontal">
         <Rectangle x:Name="bar" Height="15" Width="0" HorizontalAlignment="Left" Fill="DodgerBlue"></Rectangle>
         <TextBlock x:Name="status" HorizontalAlignment="Left" Margin="5,0,0,0"></TextBlock>
     </StackPanel>
 </Border>

The event handler:

private void HighDensityScatterSeries_ProgressiveLoadStatusChanged(object sender, Infragistics.Controls.Charts.ProgressiveLoadStatusEventArgs e)
{
    this.bar.Width = (this.border.ActualWidth / 100) * e.CurrentStatus;
    this.status.Text = e.CurrentStatus + " %";
}

All of this results in a very pleasing loading experience – the chart animating the series as it advances, the UI is responsive and the user is kept in the loop as to just how much work is completed:

The high density scatter series pregressive loading in action.

Square cut-off

Another performance booster for the heat-map representation, besides it’s resolution, is the style used to truncate its areas. More specifically enabling this feature will cause the chart to use squares when halting a render traversal rather than the shape of the coalesced area. Best explained with a visual:

The high density series using the default cut off render style.

The above is with Resolution set to 8 to make it clearer. The same with square cut-off looks like:

The hige density scatter series using the square cut off style.

Controlling the mouse over

The series also provide a property that can be used to control whether the mouse over event would be fired and in most cases that means the tooltips won’t be rendered as well. That should be able to squeeze just a little bit more speed from the chart.

Conclusion

The new High density scatter series and set to deliver the optimal performance and visualization when dealing with extreme amounts of data points. Designed to perform, it will approximate data to display a heat map representation of it and allows you to control the size and style it is rendered. The series also provide pleasing experience with loading by rendering in parts, keeping the UI responsive and constantly providing status information. It can really help you make sense from an overwhelming millions of points. In this community preview stage, you are welcome to give it a try and provide your feedback and suggestions. These XamDataChart series are available in both the NetAdvantage for Silverlight and WPF.

Download and try the demo for this blog – Silverlight | WPF (keep in mind at least a Trial version of the respective platform product is required for those). Also visit the Online Samples for Silverlight or you can download the WPF samples browser. As always, you can follow us on Twitter @DamyanPetev and @Infragistics and stay in touch on Facebook, Google+ and LinkedIn!