In this post i want to write a bit about the OpenStreetMap data model. This has obviously been influenced by the discussion in the OSMF to make changes to that model i already mentioned in a previous post.

I am a bit reluctant to write about this here and i want to explain the reasons for that first. I am writing about this out of intrinsic motivation. On the one hand for the intellectual challenge of discussing a highly complex interdisciplinary topic. It involves engineering aspects obviously but also natural sciences (in the form of the physical geography that a large portion of the OSM data represents) and social problems (through the role the data model plays in social interaction in the OSM community). On the other hand i also believe that my thoughts on the matter can be valuable considerations for the OSM community so sharing them publicly could be of benefit for the project.

However, the OSMF has started the public discussion of their plans to make changes by moving the whole matter on the commercial level (by commissioning a paid study on the subject). That means taking part in the discussion with the OSMF on the matter will inevitably involve engaging in discussion with people who have economic motives for representing their views. While i do not categorically reject doing that (doing so can still provide better insights into the subject and can be of benefit for the OSM community at large), we are in the field of diminishing returns here. Pro bono fighting an uphill battle against economic interests, defending my views not against arguments and reason but against people who have an economic interest not to change their view, is not a sustainable strategy in any way and is typically not in support of what i described above as what motivates me to write about this matter here.

Long story short: I am writing this not to engage in a discussion with the OSMF of their commercial project to change the OSM data model but to have an open intellectual exchange with anyone interested in the hope to advance our collective understanding of the subject and to educate people who are interested in aspects and context of the topic they might not be aware of.

Writing this while knowing that many of the people who have the most influence on how the OSM data model will develop are – based on past observations – not very likely to be open to arguments and reasoning that challenge their views on the matter is painful – hence my reluctance of doing so. I still do this because i know that quite a few people in the OSM community are interested in diverse views of topics like this and value being confronted with ideas and perspectives especially also if they are different from their own.

So much for introduction – let’s get on the subject.

What is the OSM data model?

I want to start by defining what i want to actually talk about – because that seems to be quite a bit different from what is being discussed in the OSMF.

The OSM data model i want to discuss here is the form in which mappers engage in the act of mapping in OpenStreetMap. Technically it is the data format of the OpenStreetMap API. That is the interface through which mappers receive data from the central OpenStreetMap database and through which they submit back their changes of that data to the project.

On the social level the OSM data model is the language in which mappers in OpenStreetMap from all over the world perform the act of cooperative mapping. No matter what kind of tools a mapper uses or in what human language the UI of that tool is labeled, the OSM data model is the underlying common standard based on which mappers communicate through the act of mapping itself. That should give a bit of an idea of how fundamental it is for the functioning of the project on a very basic level.

The OSM data model is neither necessarily the format in which OpenStreetMap data is distributed to data users (which at the moment happens to be the same) nor is it necessarily the same format in which data is stored in the central OSM database (which at the moment is close to the API data model – though there are smaller differences – like the form in which coordinates are represented).

How does the OSM data model currently look like?

The OSM data model is – in its basic paradigm and compared to other common forms in which geo-data is represented – a very generic, low level format. If i try to describe that in relatively simple terms i get:

• Geographic locations (positions on the surface of the earth) as the fundamental components of geographic information are represented with so called nodes with coordinate pairs (latitude, longitude) as attributes.
• Relationships between different objects and concepts that are modeled with more than a single geographic location are represented by so called relations – which contain references to other objects, potentially in a certain order and potentially with a certain role.
• Sequences of geographic locations in a certain order can also be (and are widely) represented by so called ways which contain references to nodes.
• All of these fundamental objects can have any number of attributes in the form of free form key-value pairs – so called tags.

Most of this data model is – as mentioned – very generic. That means it makes only very few assumptions about the way geographic information is represented beyond the basic paradigms of geography itself. It is beyond doubt that this characteristic has largely contributed to the success of OpenStreetMap over the past >15 years. The positive effect of the generic nature is usually seen primarily in the free form tagging system. But that is mostly because the OpenStreetMap mapper community has concentrated on that in their activities and is mostly using tags to develop innovative ways to represent geographic information. Many of the other advantages of this model are so far severely underused and i will get to why that is the case further down.

What are the issues with the current model?

I think i already indicated in the way i presented the current OSM data model above that the concept of the ways is kind of an outlier in the model. With the plans of the OSMF in mind i would go a step further and speak of the ancestral sin of the OSM data model.

First of all ways as a concept are superfluous in the OSM data model, a relation can be used to represent what is represented with a way just as well. Or you can look at it the other way round, a way is just a hard-coded and technically more restrictive type of relation:

• it can only have nodes as members
• it can not have more than 2000 of them
• the members cannot have different roles

Beyond that ways are severely under-defined. Ways are usually interpreted to represent a sequence of straight line segments between its member nodes – but nowhere is it defined what that actually means. The most natural interpretation would be to consider the straight line between two nodes to be the shorter segment of the great circle running through those two nodes. Most tools processing ways however interpret a straight line to be a straight line in equirectangular projection (that is geographic coordinates interpreted as cartesian) or in Mercator projection – but that is not defined or documented in any way. If ways were just a type of relation defined on top of the basic OSM data model through consensus in the OSM community and could equally be changed or amended in their meaning through revised consensus, that would be different – we have plenty of similarly under-defined concepts in tagging schemes and relation types in OpenStreetMap. But as a hard-coded element of the low level data model this is highly problematic.

The other main technical or formal issue of the model is that nodes can have tags. Discussing this could get us quite a bit into an abstract philosophical domain. I will try to keep it brief but be aware that this is going to selectively only present some of the arguments.

There is a continuum (or an infinite number of) geographic locations on the planet obviously. That act of mapping consists essentially of two parts:

• identifying locations which have meaning.
• documenting what meaning these locations have.

These two activities are not necessarily identical and the same location can have meaning in different contexts – for example a location at the corner of a building has both meaning as a point on the corner of the building as well as one end of the artwork painted on the side of the building and as a (concave) corner of the pedestrian area surrounding the building. That the OSM data model separates these two activities by separately recording the geographic location (the node) and the meanings it has (by being a member of ways/relations representing building, artwork and pedestrian area in the mentioned example) is one of the huge advantages of it. But this is only the case when the meaning is represented through ways or relations. When you have for example a path (way with highway=path) crossing a fence (way with barrier=fence) with a gate (node with barrier=gate) the node doubles as a location and as a carrier of meaning. That is not ideal because the two mapping activities described above are then not separately represented in the data.

Both of these issues are things you can hardly blame anyone for retroactively because they are the result of the historic development of the data model.

The OSM data model would improve a lot in its inner consistency as well as in practical handling if these two issues were fixed. That means the new data model would look like this:

• Nodes would just contain coordinate pairs and have no tags, Real world features that are to be modeled geometrically with a single coordinate pair would be represented with a relation with the single node as member.
• Ways would be eliminated and converted to a type of relation.

I am of course not the first one to identify these two things as key points where the OSM data model can be improved. Ilya for example mentioned these ideas recently independent of me. Most who analyze the OSM data model from the perspective of mapping will likely come to the same conclusion (or have done so in the past already).

Are these the only significant issues with the OSM data model that are worth addressing? Probably not. There are quite a few further things, mostly related to the way changes in the data are being represented and managed (with versions and changesets – things i kept out of the description above). But to keep things simple i intend to limit this post to matters of representation of geographic knowledge itself.

The struggles of working with a generic data model.

Of course the structure of the low level data model is only half of the story.

The other side of the topic are the difficulties resulting from having such a generic and low level data model as we have in OpenStreetMap for the process of building consensus on how we practically document geographic knowledge with that data model.

Again this topic is historically mostly discussed in the context of the free form tagging system in OpenStreetMap – which is the practically most visible unique characteristic of the OSM data model compared to ways to represent geographic information elsewhere. There is consensus that the free form tagging system was and is an important basis for OpenStreetMap having developed the way it did (in the positive sense). But, as i already discussed in a different post, this has also resulted in problems, and as OpenStreetMap grows it is going to be of fundamental importance to have a meaningful discussion on how the development of tagging in OpenStreetMap can scale and continue to function in a growing and increasingly diverse OSM community. I presented Tagdoc as a sketch for what i think would be a valuable component to facilitate better understanding and as a result more qualified decision making in tagging and tag development.

There is a subset of tagging that is of fundamental importance so that OpenStreetMap is able to use its innovative data model to its true potential and the OSM community’s struggles to actually develop a meaningful discussion and consensus building process on that has significantly kept back OpenStreetMap for more than a decade.

This subset of tagging i am talking about are the relation types. As i outlined above, relations are the core feature of the generic OSM data model to implement higher levels of abstraction to represent concrete geographic knowledge.

Technically relation types are just tags. But the social dynamics around them in the OSM community are very different from normal tags.

For normal tags the conventions how they are used and interpreted are influenced primarily by the following actors:

• mappers through the use of the tags.
• wiki activists (people engaged in editing of tag documentation on the OSM wiki and participating in proposal processes).
• editor developers through decisions they make with tagging presets.
• map style developers through decisions what tags to interpret and how to interpret them.
• other data users through their tag interpretation practice.
• QA tool developers.

For relation types the primary influences are very different:

• editor developers through decisions for what types of relations they offer an editing interface to.
• developers of data interpretation tools through decisions which types of relations they support and how they interpret them.
• to some limited extent QA tool developers.

As you can see there is a fundamental difference. The practical use and meaning of tags is influenced by a wide range of actors with different backgrounds. This sometimes leads to chaotic situations – which some people despise and call for more authoritarian tagging paradigms. But overall it has served OpenStreetMap quite well – even if there are and continue to be problematic concentrations of power and influence in that system as well.

For relation types however there is a clear gatekeeper role of a very small set of people who practically decide (though they might not actually be aware of that) which relation types are ‘permitted’ (and as a result are practically used in significant volume) and what the mapping conventions are for those. And all of these people are software developers.

As a result of this we essentially have only a handful of relation types in OpenStreetMap that are (a) practically used in significant volume and that are (b) used with a significant level of consistency that allows data users to interpret them in a meaningful way. And all of those have been around already for more than ten years now.

This is the big practical problem in my eyes. In contrast to tags where the power and influence on conventions is widely distributed and grassroots inventions of new tags have a reasonable chance to be successful (and where separating good and bad ideas by mappers voting with their feet usually works) this is not a working mechanism for relation types. This cripples the OSM community’s ability to develop the way it maps the world wide geography in an innovative fashion and prevents it from making full use of the potential of the data model. This is not a problem of the data model itself but stems from the low level nature of it, making it necessary to develop higher level conventions on top of it – which the OSM community is, one a social level, currently unable to do.

This is a hard problem to solve and I have no solution for this to present here. But i know what is no solution: To give up and to move from the generic and low level data model we have to a constrained higher level model of points, linestrings and polygon geometries.

There are quite a few secondary problems that have turned up as a result of the OSM community essentially having been unable to advance on the development of relation types – or of other higher level conventions for representing geographic relations. One prominently visible problem is the overuse of multipolygon relations for representing concepts that could be represented much more efficiently and easier to maintain otherwise. As a simple (though maybe somewhat tongue-in-cheek) example: Think of large islands (like Greenland or Madagascar). These are currently represented in OpenStreetMap with very large multipolygon relations with thousands of member ways. These multipolygon relations however contain almost no information at all that is not already in the database otherwise. If you’d transfer the tags of the relation to a node – either anywhere within the island or on its coastline – that would contain all the information already in a much more compact form. I am not necessarily saying that this is a change that should be made specifically for islands, arguments could be made for keeping the relation here, even from a mapper perspective. My point is that the OSM community currently lacks the fundamental ability to make such a change (or any substantial change in higher level mapping conventions beyond the atomic tagging of individual features). And this is essentially a different side of the same problem. Technically fiddling with the low level data model will not help with that.

What about the other problems?

If you have read the report on changing the OSM data model commissioned by the OSMF you might wonder why so little of what is identified there as problems is discussed here and why most of what i discuss here is not of much importance in that study. That is because i look at the OSM data model purely as the language in which mappers communicate and exchange their work while that study:

• is almost exclusively concerned with the needs and the difficulties of global scale data processing on the data user side,
• makes the up-front assumption that the mapping data model and the format of distribution of OSM data en bulk to the data user necessarily need to be identical and
• essentially proposes to eliminate the function of the OSM data model as the language of exchange between mappers. Instead it suggests that the data editing tools create an abstraction layer between the data format of the API and the paradigms of representing geographic information used by the mappers.

This is the wrong approach in my eyes. The solution to issues on the data user side in processing OSM data on a global scale is simple: Distribute the data in a form that avoids these problems. Doing so would probably be simplified quite a bit if the changes to the OSM data model i sketched above (elimination of ways and of tags on nodes) would be implemented. Also the problem of the two different means of representing polygons with closed ways and multipolygon relations would be eliminated by that.

Where we need to work on solutions is the social problem of developing and maintaining higher level conventions on top of the low level data model OpenStreetMap is based on. That is not an engineering problem of course. While technical means and tools are likely to be useful in implementing solutions to that once we have identified and agreed on them, people with technical skills and qualifications should resist the urge to focus on the hope of finding primarily technical solutions to this. That is not going to work.

Conclusions

That was – despite having grown to a lengthy text again, as you have become quite used to from me probably – a very quick run through a rather complex interdisciplinary topic that only scratches the surface obviously. That means it is going to be easy for people who want to selectively maintain a different perspective on the matter to dismiss my thoughts as too superficial. And that is fine. But make no mistake: That i try to present the topic in a brief and condensed (and hopefully not too cryptic) way does not mean i have not thought it through more in depth. Those who have discussed the OSM data model with me in the past know that i have always approached the topic with an open mind. And i still do. I present no solution that i claim is the right one, i merely present an analysis of the problem.

The main points you as a reader should probably take away from this post are:

• If you look at the matter from the most important side to consider – that is from the perspective of mapping – it looks very different from how the OSMF seems to so far have looked at it.
• The main problems are not with the technical structure of the OSM data model (though there are some quite obvious things where this could be improved) but in the ability of a massively growing and increasingly diverse OSM community to productively use the possibilities of the data model to be innovative and efficient in cooperatively mapping the world wide geography.