Heat map

General

Heat maps are typically used to indicate density of data points geographically using color coding. They can be used to visualise large amounts of point data in 2D.

Examples include:

• Historical positions
• Population densities
• Frequency of events

Global AIS activity heat map

POI density, different coloring

https://en.wikipedia.org/wiki/Heat_map

Typically, bright of "hot" colors are used to indicate high level of activity or denser areas while "cooler" colors are used to indicate lower levels of activity. Several different palettes can be used for heatmaps. Some examples can be found here: https://www.andrewnoske.com/wiki/Code_-_heatmaps_and_color_gradients

Heat map generation

Geographical heat map generation follows the following general steps:

• For a given view, a set of possibly weighted positions are extracted
• Weights/counts are accumulated in a raster
• Some heat map kernel is applied to smooth and spread measurements
• Color is applied

Heat maps in GDK

Maria GDK provides an optimised engine for generating heatmaps for large amounts of geographical data, up to around 100 million points can be processed with acceptable performance. Parameters can be changed interactively with the heat map display changing nearly instantly.

IHeatMapLayerViewModel exposes the heat map API:

namespace TPG.GeoFramework.HeatMapLayer
{
    /// <summary>
    /// View model for general heat maps
    /// </summary>
    public interface IHeatMapLayerViewModel:IGeoLayerViewModel
    {
        /// <summary>
        /// Heat map data is organized in HeatMapGroups. Each group contains clusters of heat map data.
        /// Separate settings and color palettes can be set for each group.
        /// Note that the heat map property setters are applied to both default settings and to all groups.
        /// For detailed, individual group settings control, access elements in the HeatMapGroups directly
        /// </summary>
        ObservableCollection<HeatMapGroup> HeatMapGroups { get; }
        
        /// <summary>
        /// HeatMapSettings contain global settings and a default group setting that
        /// is used when no settings are available for a heat map group.
        /// </summary>
        HeatMapSettings HeatMapSettings { get; }
        
        /// <summary>
        /// Sets kernel radius unit (meters or pixels)
        /// </summary>
        KernelRadiusUnitEnum KernelRadiusUnit { get; set; }

        /// <summary>
        /// Kernel radius, unit controlled by KernelRadiusUnit
        /// </summary>
        double KernelRadius { get; set; }

        /// <summary>
        /// Sets kernel shape. The kernel shape controls how values are smoothed in the HeatMap.
        /// See https://en.wikipedia.org/wiki/Kernel_(statistics)#Kernel_functions_in_common_use
        /// </summary>
        KernelShapeEnum KernelShape { get; set; }

        /// <summary>
        /// Max pixel radius used when applying kernel. This is relevant when setting the kernel unit to "meters"
        /// and is used to limit demanding calculations
        /// </summary>
        double MaxPixelRadius { get; set; }
        
        /// <summary>
        /// Multiplier is applied to all values prior to applying the kernel
        /// </summary>
        double ValueScale { get; set; }
        
        /// <summary>
        /// Smaller grid cell sizes yield higher quality heat maps. Larger grid cell sizes are quicker. Typical range is 1-10. 
        /// </summary>
        int GridCellSize { get; set; }
    }
}

Adding positions

Positions are addud using IHeatMapLayerViewModel.HeatMapGroups. The sample below contains code from NGTester used to load heat map data from file. Note that using PositionGroupCollection below it is very easy to load large amounts of position data into the TPG NGTester heat map.

public static HeatMapGroup CreateTestGroup(string fileName)
{
    PositionGroupCollection testData;
    using (var stream = File.Open(fileName, FileMode.Open))
        testData = PositionGroupCollection.Parser.ParseFrom(stream);

    var result = new HeatMapGroup
    {
        Settings = new HeatMapGroupSettings {KernelShape = KernelShapeEnum.Triweight},
        Name = !string.IsNullOrWhiteSpace(testData.Name) ? testData.Name : NameFromFileName(fileName)
    };

    foreach (var item in testData.PositionGroups)
    {
        var hdi = new HeatMapItemData
        {
            Id = item.Name,
            GeoData = new HeatMapGeoData(item.Positions.Select(a => new GeoPos(a.Lat, a.Lon)).ToArray())
        };
        result.Add(hdi.Id, hdi);
    }
    return result;
}

heatMapVm.HeatMapGroups.Add(CreateTestGroup("some_test_data.geo");

Where PositionGroupCollection and "some_test_data.geo" is defined as a protobuf message:

message Position
{
	double lat=1;
	double lon=2;
}

message PositionGroup
{
	string Name=1;
	repeated Position Positions=2;
}
message PositionGroupCollection
{
	string Name=1;
	repeated PositionGroup PositionGroups=2;
}

Heat map settings

Heat map settings affects how the heat map is generated and displayed. IHeatMapLayerViewModel.HeatMapSettings control global settings and defaults. In addition, each HeatMapGroup contains an optional HeatMapGroupSettings

Item	Description	Value type
HeatMapSettings.GridCellSize	Controls quality of grid display. Higher values will greatly boost performance, while sacrificing heatmap quality. A value of "1" creates the best heatmap visualisation. Default is "2", this creates high resolution heatmaps while preserving good performance. Typical range is 1-8.	int (Pixels)
HeatmapGroupSettings.KernelRadius	Controls the size (radius) of the heatmap measured from every data point	double (Value in Pixels or Meters, controlled by HeatmapGroupSettings.KernelRadiusUnit)
HeatmapGroupSettings.KernelRadiusUnit	Controls the unit type for HeatmapGroupSettings.KernelRadius	enum (Pixels or Meters)
HeatmapGroupSettings.KernelShape	Controls how point values are distributed across the heatmap radius. See https://en.wikipedia.org/wiki/Kernel_(statistics)#Kernel_functions_in_common_use	enum (Quartic, Triangular, Uniform, Triweight, Epanechnikov or Gaussian)
HeatmapGroupSettings.ValueScale	Can be used to compress or expand the values in the heatmap. Manipulating ValueScale can be used interactively to alter visuals without editing the color tables	double (Factor)
HeatmapGroupSettings.MaxPixelRadius	Limits the maximum number of pixels in the heatmap radius. When selecting RadiusType "Meters", setting this parameter will limit the maximum heatmap size when zooming in. This may prevent sluggish performance.	double (Pixels)

Heat map colors

When defining heat map colors, each scalar value in the heat map raster must be associated with a color value. This is done by defining color groups that contain a list of colors and heatmap bounds, together defining color gradients for heat map values. Each HeatMapGroup contains a list of ColorGroup:

    public class HeatMapGroupSettings
    {
        ...              
        /// <summary>
        /// The color groups control how color is applied to the heat map by associating heat map values with colors.
        /// The elements must be ordered by lowest values first
        /// </summary>
        public List<ColorGroup> Colors { get; set; }
        ...
     }

Note that the ColorGroup items in Colors must be ordered so that the first element contains the lowest boundary value, with increasing boundary values.

    /// <summary>
    /// Color groups are used to map from heat map values to colors. They are typically chained in a
    /// list of values, when the heat map value falls between the lower bound of one group and the
    /// lower bound of the next higher group, the color value is interpolated
    /// </summary>
    public class ColorGroup
    {
        /// <summary>
        /// Color group is valid if heat map value is between LowerBound and LowerBound of next group
        /// If the heat map value is larger than the last/largest color group, the last group is valid
        /// </summary>
        public double LowerBound { get; set; }
        
        /// <summary>
        /// Lower bound color. If the heat map value is equal to or slightly above LowerBound,
        /// ArgbLowerBoundColor is used. The format matches low level pixel format, use ToRaw to create
        /// color values
        /// </summary>
        public UInt32 ArgbLowerBoundColor { get; set; }
        
        /// <summary>
        /// Optional upper bound color. If not set, the lower bound color from the next group is used
        /// Note: By setting this value to the same as ArgbLowerBoundColor, it is possible to create non-smoothed
        /// heatmaps. Otherwise, colors will be automatically smoothed
        /// </summary>
        public UInt32? ArgbUpperBoundColor { get; set; }

        /// <summary>
        /// Utility function to create raw ARGB-values used in ColorGroup
        /// </summary>
        /// <param name="r">Red component (0-255)</param>
        /// <param name="g">Green component (0-255)</param>
        /// <param name="b">Blue component (0-255)</param>
        /// <param name="a">Alpha/transparency (0-255) 0 is fully transparent, 255 is fully opaque</param>
        /// <returns></returns>
        public static UInt32 ToRaw(byte r, byte g, byte b, byte a)
        {
            if (a < 255)
                return ColorUtils.PremultiplyAlpha((UInt32) (a << 24 | r << 16 | g << 8 | b));
            return (UInt32) (a << 24 | r << 16 | g << 8 | b);
        }
    }

Standard colors

Maria GDK defines several default heat map colorings. They can be defined like this:

Settings.Colors = HeatMapColors.GetStandardColors(alpha, 0.1, 10, 50, 100, 200, 400, 8000);
Settings.Colors = HeatMapColors.GetGreyscaleColors(alpha, 0.1, 200, 8000);
Settings.Colors = HeatMapColors.GetBlueGreenColors(alpha, 0.1, 10, 100, 8000);

The values in the color Get-helpers are the heat map lower boundary values. The colors are defined here: http://www.andrewnoske.com/wiki/Code_-_heatmaps_and_color_gradients

Note that by default, GetStandardColors is used to generate heat map colors.

Custom colors

Any legal colors can be used for coloring heat maps. Care should be taken to create a palette and boundary values that are suited for visualising the data that are being processed. It is not possible to create universal defaults as data ranges, dynamics, values of interest and so on can vary extremely much between data sets and data domains.

The example below illustrates how the default grey scale palette is created:

     public static List<ColorGroup> GetGreyscaleColors(byte alpha, double v1, double v2, double v3)
     {
         // http://www.andrewnoske.com/wiki/Code_-_heatmaps_and_color_gradients
         var colors = new List<ColorGroup>();
         colors.Add(new ColorGroup
         {
             ArgbLowerBoundColor = ColorGroup.ToRaw(0, 0, 0, alpha),
             LowerBound = v1,
         });
         colors.Add(new ColorGroup
         {
             ArgbLowerBoundColor = ColorGroup.ToRaw(128, 128, 128, alpha),
             LowerBound = v2,
         });
         colors.Add(new ColorGroup
         {
             ArgbLowerBoundColor = ColorGroup.ToRaw(255, 255, 255, alpha),
             LowerBound = v3,
         });
         return colors;
     }

Custom grey scale palettes can add more color groups than the default in order to emphasize different data ranges. While linear interpolation is used when resolving colors from the heat map, it is possible to emulate more complex functions by adding piecewise linear boundary ranges colors.