Imagico.de

blog

On the discussion on OSMF supported vector tiles maps

| 7 Comments

After my initial report on the SotM conference in Milano there are a few things i would like to discuss in more depth. The first one is the vector tiles on osm.org topic.

A bit of background first: The term vector tiles has in the OSM community kind of been a white elephant for the past couple of years. I think over the years vector tiles have been proposed as the magic solution for just about every problem in community map production that exists.

When used with actual meaning and sense and not just as an empty buzzword the term is used for two fairly different things:

  • for the idea of caching the results of queries in a tiled map rendering framework. In a classical rendering system like OSM-Carto the map tiles are generated based on a number of queries of the data from a spatial database (usually postgis) for the area of the tile rendered. The results of these queries are thrown away right after the tile has been rendered. By caching the query result you can much more efficiently render different styling and tile variants – like different labeling languages, different color schemes or different resolution tiles.
  • for the idea of tiled rendering of the map on the client (web browser) instead of the server based on tiled vector data. This has similar advantages as in the first concept but in particular it allows the map provider to outsource rendering and supplying the computational capacity for it to the map user.

As you can probably imagine there are technological similarities between implementations of these two approaches so use of the same term for both of them is not without basis. But it is always important when you talk about vector tiles to make clear which of these two ideas you are talking about.

The first approach described above is fairly non-problematic. It is widely used in maps produced today without this necessarily being visible for the user. Of the maps available on openstreetmap.org several are using these methods and there also exists a port of OSM-Carto that uses server side vector tiles. The second approach however is more tricky because for this to work without serious performance issues the vector tiles transferred to the client must not be much larger than raster tiles. And this is extremely hard and practically it is almost never achieved. For one specific rendering task, the plain color rendering of polygons, i discussed this in more depth some time ago. What map producers currently do to work around this problem is using massive lossy vector data compression. And compared to lossy raster image compression where decades of research went into the methods used and we have have specialized methods for all kinds of specific applications (like raw photo compression) these methods are relatively crude. You can see that in most maps based on client side rendering where the appearance of the map is usually primarily defined by the data volume reduction needs and not by cartographic requirements and considerations and a significant part of the information communicated to the user by the map is compression artefacts rather than geographic information.

So much for general background. What i now want to discuss and comment on a bit here is the proposal from Richard Fairhurst to initiate a project to provide vector tiles for client side rendering (the second approach above) on OSMF infrastructure. There are a few blog posts and messages related to that and there was a breakout session at the SotM conference about the topic.

The alternative-colors style – more eccentric and less purple than OSM-Carto i guess

In general and as i said before i support the initiative to increase the diversity in community created maps available to OpenStreetMap users. But due to the white elephant nature of the topic there is a lot of blind enthusiasm surrounding this that can easily lead to ignoring important things. I want to point out a few of them here – some of them i have already mentioned at the SotM meeting, others are new here.

1) It is currently fundamentally impossible to fulfill all of the functions of the current standard map style with a framework based on vector tiles for client side rendering. This might not seem overly relevant because OSM-Carto at the moment unfortunately all too frequently ignores the requirements of these functions as well (see here) but the fundamental difference is that with OSM-Carto this is a (bad) choice that can easily be changed, with vector tiles for client side rendering this would be much harder.

Most at the meeting at SotM were aware of this limitation but i fear that quite a few people ultimately have a strong desire to push this approach at all costs and just agree to this in an attempt to pacify potential opposition while still believing the white elephant to be the solution to any and all problems. So i will repeat in all clarity here: The client side rendered vector tiles approach is currently fundamentally incapable of generating maps that fulfill all of the core functions of the OSM standard style and there are no technological innovations visible at the horizon that are likely to change that.

2) It would be important IMO to consider and discuss the question if the OSMF should actually become active in this field at all. If you look at the OSMF mission you can see that providing maps for all kinds of purposes is not part of that mission. The fact that the current standard map style fulfills important functions for the OSM community and for the public image of OpenStreetMap, is generally accepted (although you can of course argue how well it actually fulfills these functions). So i think a new, additional map rendering project would – to justify receiving significant support from the OSMF in the form of computer infrastructure – have to state what important functions it aims to fulfill and demonstrate that it is capable to do so within the spirit of the OSMF mission.

There seems to me a significant likelihood that such a project from a user perspective could end up being more or less a knock-off of the OpenMapTiles project or similar and in that case it would IMO be fair to ask if this would warrant the OSMF investing resources in such a project.

3) Paul Norman in the discussion mentioned an important point: The number of developers in the OSM community capable and interested in productively working on map rendering and map design projects in general is fairly limited. This means people will ultimately vote with their feet. This is similar to what i wrote about OSM-Carto last year: The question that will most likely decide on the success of such a community project (no matter if run on OSMF infrastructure or not) is if it can successfully attract competent and committed developers capable of actually achieving whatever goals the project started with.

Not that it matters that much (since i am not very qualified to help kicking off such a project anyway) but i have at the moment rather limited interest in this project myself because my interest in community maps is primarily in those areas for which as said the client side rendering approach is currently unsuitable for. But this is not set in stone and it is quite possible that there are aspects of such a map project that could turn out to be interesting for me as well.

So far for the specific plans for an additional map rendering project on osm.org which as said as an increase in diversity in community maps i would support. But i also want to put a bit of a different perspective on the topic of the future of OSM community maps in general:

Richard’s blog post starts with an accurate analysis of what makes OpenStreetMap different from the big corporate players. The visualization or map rendering part of OpenStreetMap – the standard style – has historically been an important part of OSM becoming a very distinct player in the field. OpenStreetMap would probably not have developed into what it is today if its main visualization platform would have aimed to be a Google Maps lookalike. Instead the standard style as i pointed out before has been pushing the boundaries of what is possible technologically and cartographically in quite a lot of things for a large part of its history and in many aspect in a very different direction than where Google and other commercial map providers are going. And it seems to me that this has a significant part in OSM being able to concentrate on its core values and not following short term trends that seem to promise a quick way to success. That this has diminished more recently in OSM-Carto development is something most people more intensively involved with OSM feel and which certainly is a huge part of what motivates people now betting on and pushing for the white elephant so to speak.

But trying to address this problem now by crawling under the wings of Mapbox or other big corporations and making OSMs public image fully dependent on technology developed and controlled by corporate players would in my opinion be a big mistake. Open Source or not – the past experience with Mapnik and Carto has quite well demonstrated i think that OSM currently lacks the resources and expertise to develop or to organize development of a map rendering framework for the specific needs of the project independent of either the big corporate OSM users or the broader FOSS community. OSM will either need to invest into improving capabilities in this field within the project (which is not that feasible because OSM as a project does not have the level of organization or resources for that) or it would need to reach out more to the FOSS community to foster independent development of map rendering tools for OSMs current and future technological and cartographic needs. Projects in that field already exist (like Mapserver for example) but they are currently mostly developed for smaller commercial users and public institutions. Getting these projects to include OpenStreetMap community maps as an important use case or initiating new projects for OSMs needs together with the independent FOSS development community would be a practical approach that could ensure an independent future for OSM in terms of community maps.

And yes, vector tiles (in the generic sense described above, less in the sense of a specific file format the specifications of which are under control of Mapbox) could likely be a part of such developments. But they are not the white elephant many hope for.

7 Comments

  1. A thought-provoking posting – thank you.

    On “It is currently fundamentally impossible to fulfill all of the functions of the current standard map style with a framework based on vector tiles for client side rendering”, that doesn’t worry me at all, because openstreetmap-carto is a pretty bad example of cartography. But that’s fine – every webmap I’ve ever seen, including the ones I’ve designed, have also been bad examples of cartography. (The ones that have been least bad have also been the least scalable, and osm-carto scales well.) Web mapping technology is still horribly primitive compared to the state of the printed art: everyone’s still using nasty uniform halos, to take one example, and let’s not even start to talk about label or symbol placement.

    So a vector tile-based map will also be pretty bad. It’ll just be bad in different ways. And that’s fine. No-one is suggesting turning osm-carto off tomorrow or at any point in the foreseeable future. A vector tile-based map doesn’t _have_ to do the same things that osm-carto does. The “core functions of the OSM standard style” are things that osm-carto and its XML-native predecessor have defined for themselves, not anything intrinsic to OSM.

    On the number of people interested in working on such a project: my sense (backed up with conversations with various people) is that there are a number of developers who are interested in working on map rendering but not on osm-carto. That’s not through any fault of osm-carto’s – different technologies attract different people.

    And I think that speaks to your wider point. Sure – it might not work; for community reasons, technical reasons, or whatever. That shouldn’t stop us experimenting and trying. In 2018 it would be near-criminal for us _not_ to be experimenting with vector tiles on osm.org. So let’s see what those who are interested can do.

    (Two incidental postscripts. First, don’t get too hung-up on the technology being controlled by Mapbox – there are some interesting possibilities that could give us some independence from the particular encoding/rendering stack. Second, I think you probably mean “great white hope” rather than “white elephant”. A white elephant is something that is already recognised to have failed; a great white hope is a search for a magic solution, but without any knowledge that it does or can actually exist. But I may be reading you wrong. 🙂 )

    • Regarding the white elephant metaphor – i think i mean both what you see implied by white elephant (belief and investment in a savior which obviously cannot live up to these beliefs) and also what is often the result of this, the escalation of commitment problems. This is why i consider it important to formulate clear goals and to insist that technical solutions have to compete in fulfillment of these goals. This should apply to OSM-Carto and to any future project that is privileged by using OSMF resources.

      But i also meant it indeed as something that is hoped to be a magic solution to all problems, in particular also to those you don’t understand enough to even imagine a concrete and real solution for. This is what i usually call cargo cult and it is definitely something that is widespread w.r.t. vector tiles.

      Below the threshold of using OSMF resources as a pure grassroots community project this is not really an issue of course and i would encourage anyone to really try whatever you think looks promising, no matter how crazy or radical it is. But since your initiative specifically aims for vector tiles on osm.org using OSMF infrastructure from the start i think formulating these requirements early on is a good idea.

      Regarding cartographic quality – i would always be careful trying to define quality in an absolute way independent of the purpose of the map. Certainly there are quality issues that are universally bad like overlapping labels or similar faux-pas. But specifically with OSM-Carto where there are as mentioned very specific purposes and goals and very specific constraints like the need to have minutely updates. I am probably one of the strongest and harshest critics of OSM-Carto in quality, pretense and diligence in fulfillment of its purposes but holding it up next to a hand crafted paper map and dismissing it as bad cartography would be somewhat besides the point. But this also means some future vector tiles on osm.org map design project could not in the future justify possible deficits of their map by pointing to OSM-Carto and arguing they are not any better.

      The real challenges of future digital cartography will IMO be both outside the scope of OSM-Carto (because of its specific purposes) and probably outside the scope of a client side vector tiles project (because of its technical constraints).

  2. > and there also exists a port of OSM-Carto that uses server side vector tiles

    Do you have a link to the code or is it not public?

    You also mention «the core functions of the OSM map» a couple of times. Out of simple ignorance, where are they listed? Or, could you clarify what are those?

    Maybe an intermediary solution would be to provide the vector tiles but not the styles, and let users use those tiles to provide maps in their own sites, combining OSMF vector tiles with their own style.

  3. I am a beliver in the “white elephant” and would like to contibute to a OSMF project. Mainly to a Client, may be even adding some 3D rendering. I would start with a Progressive-Web-App and use WebAssembly for extensive calculations.

    Vector tiles are to big? We are just starting. We may find special compressings. And the tiles bounding boxes should fit to the needs of the renderer. The content to. We should adapt the content after/while we use them for a while. If a user wants to see details of ground zoom in a high above overview, the client has to download the detailed vector tiles too.

    Where may i sign in?

  4. Interesting discussion, I’m more focused on the advantages client-side rendering could bring to the OSM.org website. Now we have 4 different map-styles showing off the OSM data, imagine what we could do with client-side rendering. This has the potential to *really* show what OSM is all about instead of just a few views on the data. I think this could potentially grow our community and get more people interested.

    Also contributing to something like osm-carto is not easy at all. You need to setup a huge stack of different components to get things working and then, on top of that, you need to know about cartography. I think we will have more contributions when we make it easier for everyone to experiment.

    • Your comment articulates a lot of the blind enthusiasm i was referring to in my post. You suggest to imagine what we could but you don’t support this with specific and well founded ideas so it is just that: imagining things.

      Having been involved in designing maps for quite some time and having critically reviewed map design work for even longer i can tell you: Client side rendered vector tiles are not the solution to all problems of digital cartography many imagine it to be. I don’t think it is bad if people play around with the possibilities of this with OpenMapTiles, using a free Mapbox account or any of the other possibilities available. You can learn quite a lot this way – how various design elements work in a map, line styles, colors etc. But ultimately this is mostly kids stuff. Useful for training but at one point your skills and ambitions will likely outgrow this and you will realize the toy car severely limits your cartographic expression.

      There are only very few examples of maps created based on client side vector tiles technology that bring anything substantially new to the world of cartography. Google with their dynamic projection change at small scales is one example. But most in this field at the moment are just kids playing in the sandbox. Which is kind of sad because we have plenty of big problems that could use competent people working on them.

      All of this does not mean i am against the idea brought forward by Richard – which i already said. But if this is to be using OSMF infrastructure the specific plans and their goals need to be critically evaluated without any of the wishful thinking and vague imagination often surrounding this topic.

Leave a Reply

Required fields are marked *.



By submitting your comment you agree to the privacy policy and agree to the information you provide (except for the email address) to be published on this blog.