Friday, January 26, 2018

Modifying Vector Tile Style (Basemap series Part 4)

Please read my previous blog post about how to author vector tile.

Okay now I will continue about how to modify vector tile styling to suit custom needs.

we start from the vector tiles layer hosted on ArcGIS online.

what you must do first is copy the style using clik on layer name, and choose copy, because I use indonesian languange, it is called SALIN.

then followed by Save layer from the copied layer.

In order to edit the style of the copied layer, you can use two built in application from ArcGIS online, which is

1. Simple Editor . This is code based, so you can freely modify as long as you understand the syntax. The syntax is similar with CSS for your information.

2. Vector Basemap Style Editor. This tool is more GUI based and perform better in many cases.

Give the two apps permission to works on your arcGIS online account

Using these two applications, you can modifying the style of the tiled layer like changing colors, thicken the outline, make the layer transparent, changing font or any other modification permitted by Vector based tiled layer specifications. Dont forget to save your works.

Besides these two built in apps, you can edit the style using any other JSON supported text editor and validator (i.e notepad++, adobe brackets , psPad, or just plain NOTEPAD *lol)

Here are two screenshots of original and modified vector layer style coming from same vector tile package source.


Vector Tile (Basemap Series Part 3)

My two previous posts concerned about tile layers served as basemap in raster format. Tile layer in raster format is superb because it is deliver fast reading, great compatibility between systems and standards, Tile layer in raster format is also works for any kind of data and independent from the source data format. Either you have thematic maps in vector format or satellite imagery/DEM in raster format, Tile layer in raster could handle them well.

But technology always changing. In recent years, development of tile layer in vector format got its fruition. It is somewhat better than raster, but it is not dedicated to replace the raster based tiles. It has specific function which is mainly to serve basemap in VECTOR format.

By using vector format, you can get :
1. Smaller file size to serve, so higher performance is guaranteed.
2. Dynamic and good resolution/display in every scale or zoom level (vector ftw man).
3. Faster generation and processing time compared to raster.
4. No artifacts or jagged features due to resolution limitation like we commonly found on raster.
5. And the most important is, points are stored in points, line stored in line, polygon stored in polygon, and label stored in font (not pixels). In this way, vector tile enable the user to change the symbolization or label format of the tiles according to their needs. This is the feature that raster based tiles couldn't do.

The limitation of vector tiles is of course they can't server data sources stored in raster format like satellite imagery or Digital Elevation Model.

You can make vector tile using online cloud GIS provider like Mapbox, or ArcGIS pro.

Here is an example to authorize vector tile using ArcGIS pro and hosted under ArcGIS online.

1. Load up your map and its final symbolization and labeling in ArcGIS pro.

2. Open Geoprocessing tools > Data Management Tools > Packages , To make proper vector tile, you must make first the tile index, so open the Create Vector Tile Index Tool. Set it up according your needs, and go.

3. Now the tiles creation, load up Create Vector Tile Tool, and make your tiles, dont forget to set the tile index created from previous step as the index polygons. The output will be saved in VTPK format.

4. After it is done, upload the created data to ArcGIS Online under organization account (public account is not working). You can apply trial to ESRI if you dont have access to ArcGIS online subscription. Here is how the map looks in ArcGIS online. 

I will continue the tutorial about how to change the symbolization of vector tile in next post, stay tuned.

Btw, here is a LINK which all tools and theoretical background about vector tiles currently has been developed

Thursday, January 25, 2018

Making Mbtiles in QGIS (Basemap series part 2)

I will continue my previous blog post.

Okay, now how to make the mbtiles of our data directly from our own maps.

In ArcGIS you can make Tile Package (TPK) which can be converted into other tiling scheme using certain tool (one of them has been covered in part 1)

In QGIS (or Global Mapper) you can actually generate directly tile packages to be consumed in WebGIS, either in compressed tile ZIP or Mbtiles format.

For this task, we going to need QTILES Plugin which can be downloaded and installed from QGIS Plugins menu.

And then, just open your QGIS map project (usually stored in qgs file) or just make it from the scratch.

QTILES can be accessed from Plugins drop down menu, once it is loaded, just set all the parameters according your needs.

Here is an example of mbtiles hosting using TILESERVER.PHP running on my local XAMPP machine.

stay tuned for the part 3, part 4, part 5. lols

Wednesday, January 24, 2018

ArcGIS Tile Package Conversion to Mbtiles using tpkutils (Basemap series part 1)

Imagine if all the published basemaps over the internet is not suit to your needs, what you can do?. The answer is simply, just make your own basemaps.

And how to make them ? by convert the created map into raster tile package of course.
From the created tile package, we can feed it off to map service provider like, Mapbox, etc, or just put it on your own server and webgis application/geoportal.

Basemap can be authored using ArcGIS or QGIS (just to name the BIG TWO software these days). I am going to talk about ArcGIS first for this post.

in ESRI ArcGIS environment, the tile package is called ArcGIS Tile Layer Package, and has *.tpk file extension.

How to make them?, simply
1. Make your own map in ArcMap (symbolization, labelling, etc and save it on MXD.
2. Boot ArcToolbox, go to Data Management Tools > Packages > Create Map Tile Package
3. Set the tiling scheme (just use arcgis online/google tiling scheme), tiling format, level or details, tag, summary, extent and output location.

In order to make the maxium benefit of the tpk, load it on your arcgis online organization, or arcgis server. The question is what if we dont have access to them? and what if we want to use the package to our own custom webgis application?. We could go to MBTILES. MBTILES can be made directly on Global Mapper or may be other software, but not in the case if we use ArcGIS (there is only TPK man).

There are few ways to convert TPK to MBTILES, one of them is using combination of PBS and MOBAC, but it is tedious. As alternative we can use TPKUTILS available on github in this LINK.

How to do it?

1. Clone the tool on Github using, PyPI, Github desktop or any other tool you familiarize with. Here is an example using PyPI by my friend Muhammad Iqnaul Siregar

2. Download Phyton 3 or newer release, and install it.
3. Check if PIP has been installed together with other python lib by typing PIP command in scripts directory of Python installation. Here is an example.

3. If it succeeded, Install the TPKUTILS using PIP INSTALL in phyton command line interface. Here is an example.
Note that I placed the downloaded TPKUTILS on my python installation directory, you can place it anywhere, just make sure windows has access r/w to the directory,

4. To keep it simple, copy your TPK file to your python installation directory, the run the python CLI. 
5. Call the tpkutils using command:
from tpkutils import TPK
6. continue with this command:
tpk = TPK('yourtpk.tpk')
7. followed by this command and you are done.
8. Here is an example of the complete code and the result

9. And here is how it looks when I put the converted MBTILES to TILESERVER running on my local XAMPP machine.

Credit to the GIS Jedi master Marhensa Aditya Hadi and Muhammad Iqnaul Siregar
Also thanks to Benny Istanto which introduce me to Vector and Raster Tile Package back then in 2011.


Sunday, January 21, 2018

download extract data from non hosted arcgis online public account

how to download arcgis online data from non hosted or public account

For Your Information

This extraction method is purely dedicated only to extract data stored in public

Account or non hosted ArcGIS online service

Non hosted data is pretty much different from hosted data, because it doesnt

have REST endpoint (Map Services) like commonly we found at ArcGIS

Server/online hosted data.

To be successfully extract the data, you must be the account owner (i.e the data

belongs to you), or the webmap has been set to public access by the account


if you still confused about non hosted data, try to make arcgis online public

account (not organizational account), and upload some data to it, then make a

webmap. The data stored in those webmap that we called non hosted data.