Preparing FKB data to MBTiles: Difference between revisions

From Maria GDK Wiki
Jump to navigation Jump to search
()
No edit summary
Line 1: Line 1:
This guide shows how to process FKB and related data into vector MBTiles datasets using FME 2021 or newer. The result are split into counties. Each county will consist of 3 MBTiles datasets:  
This guide shows how to process FKB and related data into vector MBTiles datasets using the Python module fkb_processor, and FME 2021 or newer. The result are split into counties. Each county will consist of 3 MBTiles datasets:  
* FKB
* FKB
* Road names (elveg)
* Road names (elveg)
Line 6: Line 6:


== FKB ==
== FKB ==
Source data must be downloaded from Geonorge with a valid '''BAAT''' user. The large amount of data makes it neccessary to use the tool '''MassivNedlasting''' from Kartverket. The following map themes should be downloaded into a single folder. Each theme should consist of one zipped ESRI file geodatabase for each county except for '''N5 Presentasjonsdata''', which as of 2022 is only available split into municipalities.
Source data must be downloaded from Geonorge with a valid '''BAAT''' user. The large amount of data makes it neccessary to use the tool '''MassivNedlasting''' from Kartverket. The following map themes should be downloaded into a single folder. Each theme should consist of one zipped ESRI file geodatabase for each county.
* FKB-AR5 FGDB-format
* FKB-AR5 FGDB-format
* FKB-Arealbruk FGDB-format
* FKB-Arealbruk FGDB-format
Line 23: Line 23:
* N5 Presentasjonsdata FGDB-format
* N5 Presentasjonsdata FGDB-format


When all files have been downloaded, use the FME workspace '''workspacerunner.fmw'''. Fill in the user parameters <code>input</code> and <code>output</code>. It is also possible to select which counties to process using the parameters <code>Fylker</code>. Runtime will vary, but all of Norway will be at least 12 hours.
When all files have been downloaded, run the Python module '''fkb_processor'''. GDAL bindings for Python are required. 
 
Parameters:
{| class="wikitable"
|+
!Parameter
!Description
|-
|input
|Positional, required. Path to a folder containing zipped geodatabases
|-
| --schema
|Optional. Path to SQLite file containing the destination schema of the MBTiles. If not provided, included schema will be used.
|-
| --output
|Path where output MBTiles will be saved
|-
| --counties
|Optional. Comma-separated list of counties to process. If not provided, all counties will be processed.
|}
Example of a run:
 
<code>python -m fkb_processor "c:\data\FKB\basis" --output "c:\data\FKB\mbtiles_output\" --counties "03,31,32,33"</code>  


When all counties are finished, the Maria GDK products NorwayTopoDetail can be updated using the new MBTiles datasets. All products should first be set to "Draft" in Maria Map Maker. Then quit the application, and replace the existing MBTiles file in the "FKB" folder in the product. Restart Maria Map Maker, and inspect the overlays.
When all counties are finished, the Maria GDK products NorwayTopoDetail can be updated using the new MBTiles datasets. All products should first be set to "Draft" in Maria Map Maker. Then quit the application, and replace the existing MBTiles file in the "FKB" folder in the product. Restart Maria Map Maker, and inspect the overlays.

Revision as of 15:22, 29 January 2024

This guide shows how to process FKB and related data into vector MBTiles datasets using the Python module fkb_processor, and FME 2021 or newer. The result are split into counties. Each county will consist of 3 MBTiles datasets:

  • FKB
  • Road names (elveg)
  • adresses (vegadresse)

These datasets will be used to update the product series NorwayTopoDetail.

FKB

Source data must be downloaded from Geonorge with a valid BAAT user. The large amount of data makes it neccessary to use the tool MassivNedlasting from Kartverket. The following map themes should be downloaded into a single folder. Each theme should consist of one zipped ESRI file geodatabase for each county.

  • FKB-AR5 FGDB-format
  • FKB-Arealbruk FGDB-format
  • FKB-Bane FGDB-format
  • FKB-BygnAnlegg FGDB-format
  • FKB-Bygning FGDB-format
  • FKB-Ledning FGDB-format
  • FKB-Lufthavn FGDB-format
  • FKB-Naturinfo FGDB-format
  • FKB-Tiltak FGDB-format
  • FKB-TraktorvegSti FGDB-format
  • FKB-Vann FGDB-format
  • FKB-Veg FGDB-format
  • Matrikkelen - Eiendomskart Teig FGDB-format
  • N5 Høydekurve FGDB-format
  • N5 Presentasjonsdata FGDB-format

When all files have been downloaded, run the Python module fkb_processor. GDAL bindings for Python are required.

Parameters:

Parameter Description
input Positional, required. Path to a folder containing zipped geodatabases
--schema Optional. Path to SQLite file containing the destination schema of the MBTiles. If not provided, included schema will be used.
--output Path where output MBTiles will be saved
--counties Optional. Comma-separated list of counties to process. If not provided, all counties will be processed.

Example of a run:

python -m fkb_processor "c:\data\FKB\basis" --output "c:\data\FKB\mbtiles_output\" --counties "03,31,32,33"

When all counties are finished, the Maria GDK products NorwayTopoDetail can be updated using the new MBTiles datasets. All products should first be set to "Draft" in Maria Map Maker. Then quit the application, and replace the existing MBTiles file in the "FKB" folder in the product. Restart Maria Map Maker, and inspect the overlays.

Troubleshooting

The workspace described here depends on the naming convention of file names, layer names and attributes in the source data. Unfortunately, these names are often changed by Kartverket from one year to the next, which usually results in errors or missing layers in the resulting datasets. Troubleshooting this is the most time-consuming part of the process.

  • When a layer is visibly missing in the result, a good starting point is to closely compare the source data with the result. The file FeatureGrouping.xml in the products contains the attribute filters for each style layer.
  • Check the FME log files, which are written next to the workspace (fmw) file.
  • Use extractmapinfo.exe to create an xml report on all the layers and attributes in the MBTiles file.

Elveg

Elveg is a dataset representing the road network in Norway. In NorwayTopoDetail we use the road center lines to label road names and numbers. The Vbase line features themselves are not visible in the map.

We create Elveg as a separate MBTiles dataset from the base data. However, it is split by fylke in the same way.

Our starting point is one zipped Elveg GML file for each fylke. We use the FME workspace gml2mbtiles_Elveg2.0.fmw.

After running the workspace, you should have one MBTiles file for each fylke, which replaces the MBTiles files in the existing NorwayTopoDetail products.

Matrikkelen - Adresse (Adress labels)

Points representing street adresses. It is placed in each NorwayTopoDetail as a separate dataset in the same way as Elveg.

The starting point should be one zipped GML file for each fylke. Use the FME workspace gml2mbtiles_matrikkel-vegadresse.fmw. After running the workspace, you should have one MBTiles file for each fylke, which replaces the MBTiles files in the existing NorwayTopoDetail products.