GNOME Maps and the tile problem

The GNOME project's Maps application provides access to an array of mapping features (trip routing, address lookup, zoomable maps, etc.) from the desktop. Implementing that feature set requires hooking into a number of online services, but none of them is as prominent as the map tiles—the background images on top of which everything else is added in overlays. Recently, the tile provider that had served GNOME Maps well for several years ended its free service, suddenly cutting off all of GNOME Maps's users and forcing developers to consider new approaches for the future.

Since its initial public release, GNOME Maps had used the open tile API offered by commercial map vendor MapQuest to fetch its map tiles. Like a lot of online mapping providers, MapQuest incorporates OpenStreetMap (OSM) data into its own product offerings, a usage fully allowed under OSM's licensing (and surely a significant cost-saver for the company). In order to give back, MapQuest announced its OSM-based service in 2010.

GNOME Maps, in turn, made use of the MapQuest open tile data because it is end-user-ready—unlike the raw data provided by OSM itself. Although OSM does have a publicly accessible tile server, in practice that server acts as more of a technology demonstration or development aid than it does a full-blown public web service, and access to the tile server is limited. OSM's primary focus remains on creating and editing the map data itself.

To put it another way, ultimately OSM produces a data set; it does not run a public service. And the limitation is a technical one, too, not mere policy. Any end-user service built on top of the OSM database must, at the very least, perform its own filtering and other transformations (selecting which road and path types are of interest, dropping authorship and update information, perhaps selectively choosing land-usage type, points-of-interest and other metadata, and so on). On top of that, its must render the map data into images and apply appropriate labels and markers, plus undertake all of the tasks of running a large-scale public web service.

For years, MapQuest's free OSM-based service did just such processing, supplying ready-to-use PNG image tiles. But, in June, MapQuest abruptly changed its usage policy, making the existing free service unusable and steering existing users toward a new, paid service. The new MapQuest service did advertise a free tier, but with an API cap far too low to work for GNOME Maps.

The official shutdown occurred on July 11, causing GNOME Maps (and many other services and applications) to display a repeating display of MapQuest's shutdown announcement in place of map tiles. On July 13, Maps co-maintainer Jonas Danielsson wrote to the development list, calling attention to the service shutdown and warning that the project has "no clear alternative" for a map-tile source.

He noted that there were patches available that could switch Maps over to OSM as a direct tile source, but added that "we would probably violate the terms of service" unless GNOME could convince OSM to grant it an exception. He asked for help contacting OSM, but concluded: "I think we are going to need our own tiles.gnome.org for a map application/platform to be feasible."

GNOME could perhaps run a server that fetched and cached OSM tiles for Maps users without raising the OSM project's ire, although it is not clear whether or not such a caching server would run afoul of OSM's rate-limiting rules and require asking for special permission. And, certainly, a caching server could act as a temporary solution, something that could be swapped out in the future for a GNOME server that generated its own image tiles directly. But, as Emmanuele Bassi noted in a Reddit discussion thread, the computing power for the server may be doable, but the bandwidth could rapidly become a problem.

PNG image tiles are typically 256 pixels square but, for any given area on the map, they must be generated at multiple resolutions—OSM supports 20 zoom levels, and users like to zoom in and out at will, expecting the image tiles to be served up nearly instantaneously. Multiplied by all of the GNOME Maps users, that adds up to a lot of network traffic.

One hypothetical alternative would be to render the tiles locally. The map server could send vector data for the region of interest and have the user's CPU or GPU render the data directly to screen. Danielsson noted that such client-side rendering is an item on the Maps roadmap, but it is several steps from being practical. OSM's vector-tile API is still in the experimental stage (as there are additional challenges to solve, such as clipping paths to bounding boxes), for one thing.

But users may not like vector-tile rendering either. Local rendering would cost battery power, and it could introduce lag. The Android app OsmAnd already uses client-side rendering, and OsmAnd users will no doubt be familiar with the delay experienced after panning or scrolling to a new area on the map, waiting for the roads and buildings to render on screen.

Another approach suggested on the list was having the application fetch map tiles in advance and store them locally. GNOME Maps can already run in local-tile mode, if one downloads tiles for the area of interest ahead of time. But this was also regarded as suboptimal; it requires the user to dedicate considerable disk space to store map tiles even in the best of circumstances, and the space required to store the entire OSM global map is hundreds of gigabytes.

In the long run, GNOME may have to run its own tile server. There would certainly be advantages to having control over that bit of critical project infrastructure. In the interim, however, the project seems to have come across a viable substitute. Przemo Firszt suggested reaching out to any of the several OSM-based service providers geared toward developers. One of those service providers, Mapbox, responded a few days later with an offer to give GNOME Maps free access to its tile server for a year.

Danielsson and Mattias Bengtsson then discussed the offer with Mapbox and agreed to the switch, with Bengtsson adding: "Personally I'm interested in exploring how we could continue with Mapbox in even in the future. It might mean crowdfunding in some way."

The MapQuest bug is listed as a blocker for the release of GNOME 3.22, so implementing the fix via Mapbox is a high priority. But it could still be a while before the patched application reaches end users; Ubuntu was already forced to pull Maps from its installation images for the most recent stable update to Ubuntu 16.04 (although the package could easily be restored in the next update). The Maps team has tentatively planned to discuss a long-term solution a few weeks from now at GUADEC.

The series of events is frustrating for Maps users, of course, but it is worth remembering that the root of the trouble is MapQuest's sudden reversal of its original pledge to support the open-source community. Reliance on the external service led to the disruption, which would be the case regardless of whether that service was free or proprietary.

But, at present, reliance on third-party resources for map applications seems to be unavoidable due to the size and complexity of global mapping. GNOME Maps already relies on quite a few other services to provide functionality on top of the tile layer, including Nominatim, which converts addresses to map coordinates, GraphHopper, which calculates routes, and so on. That makes it susceptible to interference on a number of fronts—even if outages like the current one remain a rare occurrence.