Saturday, February 17, 2018

Using Mapproxy to cache and host both online and offline basemaps (Basemap series Part 5)

Here we go again

In this post now I will continue about how to simultaneous host and cache basemaps, either from online or offline sources.

First, If you want to just use online basemaps to be available or hosted on your favorite protocols (like serving google Street Map to WMS or ArcGIS REST service) you could use Portable Basemap Server. I have previously covered it, check this LINK.

If you want to only cache the tiles to be used for offline needs, you can use MOBAC or SasPlanet.

Now if you want to have capabilities like both serving basemaps directly to your user using client server services protocols (either online or local online or even pure offline situation) while maintain online access, plus adding caching capabilities (useful if you want your application to be as fast as possible), you can use MAPPROXY.

It can do what PBS can do plus it has caching ability, so you wont have to worry if your internet networks get troubled, because the generated caches will serve as backup renderer. Basically it is combine PBS, and offline cacher like MOBAC or Sasplanet.

Now about the procedures:

1. Install mapproxy using python PIP from their official page.
2. You are python blind, dont worry this SITE provide you offline installer (install it as windows services for your convenient). Because the program will have their own webserver, beware of Port Conflict, make sure you choose unused HTTP PORT on your server/computer. 
3. Now deep down to settings, make YAML configuration file. The structure of the code will looks like this
services:
  demo:
  tms:
  wms:
    md: 
      title: Google MapProxy WMS Server
  wmts:
    kvp: false
    restful: true

sources:
  google_image:
    type: tile
    url: http://mt1.google.com/vt/lyrs=s&x=%(x)s&y=%(y)s&z=%(z)s&s=Galileo
    grid: grid_webmerc

grids:
  grid_webmerc:
    base: GLOBAL_WEBMERCATOR
  grid_merc:
    base: GLOBAL_MERCATOR

caches:
  cache_google_image:
    grids: [grid_merc]
    sources: [google_image]
    cache:
      type: mbtiles

layers:
  - name: layer_google_image
    title: Google Satellite Map
    sources: [cache_google_image]

globals:
  image:
    resampling_method: bilinear
    jpeg_quality: 90

4. The above YAML config file is dedicated to serve google image to your clients. Refer to this LINK for documentation. The most important things is, you must know the exact tile query URL of your target basemap providers.

5. Now save the YAML to directory C:\ProgramData\Mapproxy

6. Next, open the mapproxy_app.py or mapproxy_srv.py in C:\ProgramFiles(x86)\Mapproxy, and change the app_config line to your created YAML file.



7. Load up the mapproxy by launch it from CMD from the program files directory, like this (dont forget to run as administrator to avoid denied access error).


8. or from Windows Services (when it installed as services). Load the windows services from Control Panel or windows search tool at the taskbar and type "services",


9. Open browser and check if the webserver has been running, the typical URL is localhost:port/mapproxy/demo. In this page, you will see all the capabilities and protocols and endpoints of the map services. From this point, you can starts to seed the services to your GIS desktop, WebGIS, mobileGIS whatever.


10. A sneak peek.




Basemap is not so myths after all,

Stay Tuned


Friday, January 26, 2018

Modifying Vector Tile Style (Basemap series Part 4)

Please read my previous blog post about how to author vector tile.

Okay now I will continue about how to modify vector tile styling to suit custom needs.

we start from the vector tiles layer hosted on ArcGIS online.



what you must do first is copy the style using clik on layer name, and choose copy, because I use indonesian languange, it is called SALIN.


then followed by Save layer from the copied layer.


In order to edit the style of the copied layer, you can use two built in application from ArcGIS online, which is

1. Simple Editor . This is code based, so you can freely modify as long as you understand the syntax. The syntax is similar with CSS for your information.


2. Vector Basemap Style Editor. This tool is more GUI based and perform better in many cases.


Give the two apps permission to works on your arcGIS online account

Using these two applications, you can modifying the style of the tiled layer like changing colors, thicken the outline, make the layer transparent, changing font or any other modification permitted by Vector based tiled layer specifications. Dont forget to save your works.

Besides these two built in apps, you can edit the style using any other JSON supported text editor and validator (i.e notepad++, adobe brackets , psPad, or just plain NOTEPAD *lol)

Here are two screenshots of original and modified vector layer style coming from same vector tile package source.




Cheers!

Vector Tile (Basemap Series Part 3)

My two previous posts concerned about tile layers served as basemap in raster format. Tile layer in raster format is superb because it is deliver fast reading, great compatibility between systems and standards, Tile layer in raster format is also works for any kind of data and independent from the source data format. Either you have thematic maps in vector format or satellite imagery/DEM in raster format, Tile layer in raster could handle them well.

But technology always changing. In recent years, development of tile layer in vector format got its fruition. It is somewhat better than raster, but it is not dedicated to replace the raster based tiles. It has specific function which is mainly to serve basemap in VECTOR format.

By using vector format, you can get :
1. Smaller file size to serve, so higher performance is guaranteed.
2. Dynamic and good resolution/display in every scale or zoom level (vector ftw man).
3. Faster generation and processing time compared to raster.
4. No artifacts or jagged features due to resolution limitation like we commonly found on raster.
5. And the most important is, points are stored in points, line stored in line, polygon stored in polygon, and label stored in font (not pixels). In this way, vector tile enable the user to change the symbolization or label format of the tiles according to their needs. This is the feature that raster based tiles couldn't do.

The limitation of vector tiles is of course they can't server data sources stored in raster format like satellite imagery or Digital Elevation Model.

You can make vector tile using online cloud GIS provider like Mapbox, or ArcGIS pro.

Here is an example to authorize vector tile using ArcGIS pro and hosted under ArcGIS online.

1. Load up your map and its final symbolization and labeling in ArcGIS pro.


2. Open Geoprocessing tools > Data Management Tools > Packages , To make proper vector tile, you must make first the tile index, so open the Create Vector Tile Index Tool. Set it up according your needs, and go.



3. Now the tiles creation, load up Create Vector Tile Tool, and make your tiles, dont forget to set the tile index created from previous step as the index polygons. The output will be saved in VTPK format.


4. After it is done, upload the created data to ArcGIS Online under organization account (public account is not working). You can apply trial to ESRI if you dont have access to ArcGIS online subscription. Here is how the map looks in ArcGIS online. 





I will continue the tutorial about how to change the symbolization of vector tile in next post, stay tuned.


Btw, here is a LINK which all tools and theoretical background about vector tiles currently has been developed
https://github.com/mapbox/awesome-vector-tiles/



Thursday, January 25, 2018

Making Mbtiles in QGIS (Basemap series part 2)

I will continue my previous blog post.

Okay, now how to make the mbtiles of our data directly from our own maps.

In ArcGIS you can make Tile Package (TPK) which can be converted into other tiling scheme using certain tool (one of them has been covered in part 1)

In QGIS (or Global Mapper) you can actually generate directly tile packages to be consumed in WebGIS, either in compressed tile ZIP or Mbtiles format.

For this task, we going to need QTILES Plugin which can be downloaded and installed from QGIS Plugins menu.


And then, just open your QGIS map project (usually stored in qgs file) or just make it from the scratch.

QTILES can be accessed from Plugins drop down menu, once it is loaded, just set all the parameters according your needs.





Here is an example of mbtiles hosting using TILESERVER.PHP running on my local XAMPP machine.





stay tuned for the part 3, part 4, part 5. lols