Imagico.de

blog

November 23, 2022
by chris
0 comments

The OpenStreetMap Foundation in 2022 – Trends and Outlook

In this third part of this year’s discussion of the OSMF i am going to look at the overall trends in the organization as i see them. For the first two parts see here and here.

It is hard to make reliable predictions regarding the OSMF board with four of seven board members being new next year. What is clear is that if there is a substantial change in direction or work style pursued by the new board members, that is going to be a hard uphill battle for them against the inertia and established work culture within the organization. Two of the board members who are going to continue have made it clear that they are going to want things to mostly stay as they are and the third one has shown no indications of different interests so far. Some of the candidates in their statements and answers have also made it clear that they pursue a conservative agenda.

So, based on the assumption that this conservative desire to keep things going in the present direction will prevail, i am going to describe a bit what i perceive to be the current trends in the OSMF as they are likely to continue in the future. I don’t want to be too repetitive to what i wrote in previous years so this is just a few notes on the more recent changes of direction and new tendencies. I am also – for every section – constructively going to provide a suggestion for the OSMF how to turn the development in a more positive direction. Based on the experience from past years i don’t have high hopes that these are going to be followed, but i think it is important to outline that concrete better alternatives exist and that they are, for the most part, not difficult to pursue. And i want to give the four new board members, whoever they might be, the benefit of the doubt.

Clouds and seagull

Centralization of communication and cultural homogenization

With the definitive roll-out of the behavior control framework and the new communication platform, the trends of previous years for more explicit centralized control and management of community interaction have started to substantially manifest this year. There are mainly two trends i expect to come out of this:

  • An increasing encapsulation of people on the new platform from the rest of the OSM community. In a nutshell: that new platform and how it is managed is modeled after Facebook rather than Usenet. The strongly hierarchical elements and the prominent display of participation and engagement scores in the design of the platform’s user interface and the clear distinction between the logged in user tracked in their activities at every moment and the not logged in anonymous outsider with very limited options all support this by underlining and affirming an exclusive sense of ‘we’ on it. A desire to exclude people not using the platform has already been articulated on various occasions by users there. How fast and how far this will go is not sure yet, because at the same time there is
  • a trend of fragmentation of the OSM community into closed social circles. Due to the exclusive nature of the new platform, requiring participants to significantly adjust to the social standards and expectations that are imposed centrally, quite a few local communities have decidedly not adopted the new platform as a place of exchange. Since the OSMF has expressed no commitment to continue to provide the more traditional infrastructure like the mailing lists and because of trends in communication styles, many local communities have moved to proprietary platforms like facebook, telegram or dicord instead. These have the advantage for the local communities that the commercial operators give them more freedom than the OSMF on their platform to self manage the channel in their own style. But many of these channels are semi-public (so you can only read them after having actively signed up to them – so there is no permanent public record) which often leads to significant radicalization in particular in smaller groups.

To be clear: It is perfectly expected and necessary that a diverse and multi-cultural project like OpenStreetMap socially structures into different local groups and communities with different local cultures. The key for this to work is that these groups need to interact with each other on equal levels based on tolerance and respect for each other on the foundation of the basic universal core values of the project. This can only work by accepting and tolerating that they might differ fundamentally in their cultural values, social conventions and collective and individual goals otherwise. This way different local communities form a loose but stable and inclusive sense of ‘we’ overall across culture boundaries, unconditionally including anyone who likes to participate under the basic premise of the project of cooperative mapping. What does not work and where unfortunately the current trend seems to go, is, that one part of the global community tries to impose their cultural standards and values on others and by doing that attempts to colonialize the project while others increasingly encapsulate themselves, partly in reaction to the colonial tendencies, to protect their cultural values. This is what we see happening right now. Unfortunately, the most likely reaction from the OSMF is going to be to increase the pressure and extend the radius of cultural homogenization and cultural imperialism.

What could the OSMF do to avoid this? Go back to the roots and provide communication infrastructure openly to any group within the project to manage under their own responsibility. Offer support and guidance when self managed communities struggle dealing with difficult individuals or outside interference but have trust in people to be socially responsible at large, even if they don’t abide by and subscribe to your cultural values.

Clouds

Decreasing diversity within the OSMF

In addition to these trends in the OSM community at large we also see an increasing commitment in the OSMF to the currently dominant culture in the organization. I mentioned in the first part that the first board committee with non-board members involved has been staffed exclusively with Americans. It is likely that fundraising in the future will therefore perpetuate the dominance of US corporations and organizations among the financiers of the foundation. And since corporate OSM data users and organizations like HOT are meanwhile the main source of new volunteers in the OSMF working groups, this will also have an influence on diversity among volunteers in the working groups that are not selected by decision from the top. And this is – as pointed out in previous years – a self emphasizing problem as the increasing presence of people with a professional interest in OpenStreetMap in the structures of the OSMF makes these less attractive for hobbyists.

As discussed in the first part, the Engineering Working Group this year took on the fairly remarkable initiative to draft a framework for open calls for tender of projects to be financed by the OSMF. While this is of course specifically designed for software development projects, it could be used as a blueprint for similar procedures being made mandatory everywhere in the OSMF where outside paid services are contracted. While this in principle would be a very positive perspective i see two problems with that:

  • It is uncertain if this framework will actually be followed for any larger projects at all. The OSMF board in particular has a long history of creating policy but then flat out ignoring it.
  • Even if procedures like this are made mandatory everywhere, there are well known techniques to work around the open competition these are meant to ensure. If those drafting the call for tender know in advance who they want to win the bid, they can design the call in a way that makes this outcome likely – even without outright cheating in the assessments.

Same applies for selection of volunteers. Many of the committees that were created by the board during the past years (specifically here, here, here, here and here) were staffed based on an open call for volunteers. But it was clear from the beginning that the main criterion for selection was the people whose work we know and enjoy paradigm. Plus occasionally some ensemble optimization for on-paper diversity. To put it very bluntly – if you were not part of this small inner circle of people whose work we know and enjoy of the OSMF board (which probably contains no more than about 30 to 40 people) and do not happen to have any desirable formal traits that look good for diversity on paper, you did not need to bother volunteering in such calls. And as a result people likely stopped volunteering – so despite the formally open call the selection becomes highly non-diverse. The end result is: The staffing of committees with volunteers by the board in an intransparent process became one of the most significant factors working towards less diversity in volunteer contributions in the OSMF.

My recommendations:

  • Put everything into the open. Transparency is the best cure against favoritism or the appearance of favoritism.
  • Stop trying to control who does volunteer work. Simply publish tasks that you think need doing and call for people to work on them in their own initiative and responsibility (and cooperating as a self organized group if necessary for the task in question). Offer support for that work but don’t try to steer it. Believe it or not – there are many people in the OSM community willing to volunteer for the project who are better managers than you are.
  • Let the local chapters do selection of political appointees. There are meanwhile sufficiently many local chapters for such a process being much more inclusive and representative than the usual self referential people whose work we know and enjoy approach of the board.
  • Introduce a meaningful subsidiarity principle to the OSMF. I have said so in the past – i repeat it here.

Clouds

Management structures

There seems to be a general sentiment among the board members proactively communicated in the past months that they largely suffer from work overload and burnout.

While there is certainly some truth to the workload of the board having in total increased over the past years, it is important to recognize that much of the workload of the board is self-imposed. Self-imposed either because

  • The board has taken on tasks and is getting involved in matters that are not in its domain as defined by the OSMF mission. As Simon also has recently pointed out the board has in the past few years gotten much more involved in operational work (with an often highly questionable outcome i might add).
  • The board has in the past made many decisions to substantially grow the operational scope of the OSMF without making sure a self sustaining infrastructure to support this scope exists. On the contrary – almost all of the structures newly created by the board in the past years – like various special committees etc. – were appointed by the board and were designed to be under direct control of the board and require regular board activity to continue working. This not only massively increases the board workload in total (both for actually doing these tasks and for keeping track of them), it also leads to awkwardness when the board does not live up to these self imposed obligations, like in case of the software dispute resolution panel, as described in the first part of this blog post series.

The solution is obvious – to (a) be more diligent in limiting the board to its core tasks and to (b) trust the community to pursue tasks in the interest of the project in an independent and self determined fashion, either in the OSMF working groups, in the local chapters or in self organized groups independent of formal organizations. Again – the OSMF mission outlines this principle quite clearly, the board would just need to follow this.

Unfortunately, the measure the current board seems to increasingly favor instead to mitigate the work overload problem as perceived seems to be to hire paid management. It is even possible that the proactive communication of the workload issues is partly a measure to pave the way for such a change among the OSMF members. Of course, such a step would break with existing conventions and policy and it would also massively change the inner dynamics of the OSMF and its relationship with the OSM community. It would most likely lead to a further exodus of volunteers who do not like to be supervised in their volunteer activity by paid management, from the working groups and the OSMF in general.

In total it is likely that this would not work (in the sense that it would add more work to the board than it would take off them). But if it does, it would mean an OSMF run by paid management and labor leasing from corporate OSM data users as a new professional–managerial class of the organization with support from a number of unpaid interns running the working groups (who are motivated to volunteer – just like unpaid interns elsewhere – by career interests). The volunteer board would still formally sit above this. But since the main interest of the board in the process is to offload work, it would lack the ability to substantially exercise oversight over the management because they don’t understand the internal processes enough any more to develop meaningful policy. And the paid management will of course in that scenario have and pursue interests that are distinct and possibly substantially differ from those of the OSM community and the project.

If and when the board in the future might make a move towards hiring paid management the claim will likely be that they believe that they will be able to avoid the negative consequences sketched while still getting a net benefit. They would, however, likely be mistaken in that belief. It would therefore be a very good idea for the OSMF membership to ask the board, if and when they make a move in that direction, what their plan B is in case their belief about how this should work out turns out to be mistaken. Because the risks in that are immense.

And of course with the current work culture of the board, paid management positions would most likely be filled with people whose work we know and enjoy of the board – further emphasizing and perpetuating existing cultural bias in the organization. It is also not unlikely that a revolving door principle will develop where paid managers are predominantly recruited from former board members.

Interestingly, the upcoming board election of the OSMF can also be considered a poll of the OSMF membership if they want the board to continue the current line of direct involvement in operational work or if they should focus more on oversight and policy development. Many of the candidates in the election have positioned themselves quite clearly in that regard.

My recommendations:

  • Subsidiarity principle, subsidiarity principle, subsidiarity principle (if i could make every board member write that a hundred times i would do so).
  • Maintain the taboo against paid management. Someone paid for their work in OSM telling a hobby volunteer what to do or how to do it is an affront against every single one of the millions of volunteers in OSM. That includes by the way people paid by a third party. Don’t take this lightly and don’t be so arrogant to assume you can manage these problems.
  • Don’t try to control everything, concentrate on setting meaningful policy and exercise meaningful oversight where necessary but don’t interfere with operational decisions – as it is mandated by the OSMF mission.
  • Move the OSMF out of the UK, and while doing that, create organizational structures that separate policy making and oversight over operational work from its management, while disallowing direct interference of oversight with the operational work. Clear separation of functions like this would massively reduce workload and stress while giving operational work the competence and freedom to organize their work independently without continuous interference by the higher-ups while maintaining a clearly defined and independently codified oversight over the operational work.

Clouds

The OSMF as HOT 2.0

In the predictions from last year i indicated that the OSMF might in the future pursue a business model similar to that of HOT – selling OpenStreetMap as a solution for the needs of others, without substantially controlling the project. I also indicated that such a scenario would most likely be fairly unstable.

With the developments of the past year, in particular with the negotiations with HOT on a trademark license and the stalling of the OSMF board on the pursuit of ODbL violation by large corporate financiers of the OSMF, quite a bit could be written on that, where it might go and what risks and opportunities result from that. But i will cut this short here because – as i already indicated in the first part – i am not very keen to provide free consulting services to an OSMF in pursuit of commercial interests.

Clouds and seagulls

Conclusions

In total, my outlook on the future of the OSMF has brightened a bit. Ironically this is partly because the economic climate is getting more difficult and hence the ability of people in the OSMF to use money in pursuit of ideas that are bad for OpenStreetMap is likely going to be rather limited. But more importantly, this year’s board election has the potential – if the OSMF membership shows wisdom in selecting four new board members – to substantially change the work culture and the direction of the OSMF. The possibilities how to do that are fairly clear now – i outlined a few in my recommendations above. A lot more can be found in past comments of me on OSMF matters. Yes, there is of course equally a potential for a change to the worse. But even if that is the case – it is clearly in the hands of the OSMF membership this year – and even more, thanks to the active contributor membership program, every mapper who cares about OpenStreetMap in principle had the opportunity to become a member and vote in the interest of the project they care about. In other words: This year much more than in previous years it is in the hand of the OSMF membership to decide on the future of the OSMF.

In addition, i observe that the centralization and cultural homogenization attempts from the OSMF have led to an increasingly broader push-back from the community. As discussed above, this push-back and diversification also takes some rather problematic forms with the radicalization of smaller groups on proprietary platforms. But, overall, this is a positive sign: Local communities all over the world developing a robust self-confidence in the way they unite OpenStreetMap’s core values with their local culture. If these local communities now manage to overcome their relative isolation (which i discussed a bit for the German community in the previous post) and successfully engage in a peer-to-peer exchange with each other, that would be even better. Interestingly, the local communities in Africa, Asia and Latin America here seem to be ahead of the European communities with regular supra-national conferences and more regular exchange across language barriers.

Clouds

November 18, 2022
by chris
3 Comments

The OpenStreetMap Foundation in 2022 – The upcoming elections

This is the second part of a series of blog posts on this year in the OSMF – you might also want to read the first part.

Before i get to my look at current trends and the outlook on where things are likely to go in the next years in the OSMF, i want to have a few words on this year’s board elections. Last year, the elections were boring, three of the four board members whose seats were up for election ran again and were re-elected as expected and the fourth seat was filled with a German. The only two other candidates were two Americans working for large corporate OSM data users and were known for their fairly radical political ideas.

This year, the situation is fundamentally different. No current board member whose seat is up for election is re-running and we have again four seats open because Amanda announced her plans to resign at the end of the first year of her second term. That means it will be the first time in the history of the OSMF since the initial election in 2007 that there will be four new board members and that – also for the first time – there will be more new board members than old-timers. This is remarkable and could indicate that we might be at a turning point in the development of the organization (or it could, of course, be just a singular outlier).

I want to express my appreciation for the decision of all four board members leaving to make room for new people to have the opportunity to positively influence the direction of the OSMF. This not clinging to your seat and being willing to put trust and confidence into others for the future reflects a substantially positive attitude.

I also want to point out that with Eugene, the first OSMF board member from outside Europe and North America is leaving the board. Like Ilya in 2017, who was the first board member from outside Western Europe and North America, he only served one term and does not re-run this year. For comparison: About half of the board members who have in the past served on the OSMF have re-run for their seat at least once. While two cases is not really a statistically significant pattern, it is noticeable. I see two likely explanations for this: (a) that the board – while in principle being open to people from other cultures – is still massively culturally biased, making it unattractive for people with other cultural backgrounds to work there or very difficult to contribute in a meaningful way and (b) that the clinging to your seat, that we characteristically observe and have observed in some board members, is a character trait more prevalent in Western Europe and North America.

clouds over water with sailboats

Board member statements

I recommend everyone to read Amanda’s departing statement on resigning from her position and i hope that at least some of the other three board members leaving will publish some reflection on their work and the reasons for leaving.

Possibly even more interesting are two statements of two of the three board members who will continue their terms after the election. One of them essentially declares the end of history in the sense that all political struggle regarding the strategic direction of the OSMF has allegedly concluded and there is no fundamental disagreement on the overall direction any more so all that remains is boring technical management work to pursue the strategic goals everyone supposedly agrees on. For those unfamiliar with the end of history metaphor – see here.

The other statement by a current and future board member is in German. On the one hand it tries to resurrect the old topos of an impeding hostile takeover of the OSMF in the board election, trying to motivate German mappers to put themselves up for election in light of this threat. I have explained in the past years why i think this kind of hostile takeover through the front door is not a likely scenario and the economic and soft power influence of large corporate interests is actually much more worrisome. On the other hand, that statement also essentially declares defeat of the German OSM community on the political level. It calls for three or four people who are willing to let themselves be elected and does so explicitly with no expectation to do anything once they are elected, just to prevent others from being elected and possibly do anything undesirable then. That is nothing less than a confession of political failure and expendability.

clouds over water with sailboat

This is a remarkable and at the same time highly problematic statement, because it claims that the German OSM community as a whole has no ideals to pursue beyond the conservation of the status quo and no positive vision for a progressive development of the project on the political level. On the other hand, it also implies that no other local communities do so either and that all progressive change pursued and fought for elsewhere in the global OSM community is inherently destructive and undesirable. In this way it ironically perfectly aligns with the end-of-history idea of the other statement. The vision of Mikel of the end of history is, of course, most likely that the economically-pragmatic idea of OSM as a collection of useful geodata – that so conveniently aligns with the big corporate interests – has triumphed over the backwards idealists envisioning OSM as a social project of egalitarian cooperation of local craft mappers. This kind of reflects Fukuyama’s thought in 1989 that western liberalism has triumphed and no substantial ideological competition remains – hence my use of the end-of-history metaphor. Roland, on the other hand, just seems to try to conserve the status quo out of a glass-half-empty attitude.

Roland’s statement can be considered to retroactively confirm Allan Mustard’s criticism of the Traditionalists in OSM (for which the stereotypical German mapper is probably often considered to be representative) as regressive conservatives clinging to the status quo and averse to any change out of principle. Therefore i want to clearly state: This is not a view and attitude representative for the German mapper community. There are, of course, the apolitical mappers who just want to continue engaging in the project the way they are used to and therefore want it to change as little as possible. But there are also a lot who are there to contribute to a progressive project, who are open to change and who are eager to contribute to making it better and who are willing to engage in the political discussions and the struggles of arguments that are necessary to determine the best way to do so but who stay far away from the OSMF at the moment. Reasons for doing that are in particular that many are deterred by either the dominance of the English language, the need to adopt an anglo-american work culture they dislike, the need to cooperate as hobby volunteers with people who are active in the OSMF out of career interests or simply because of the lack of perspective to affect meaningful change in a positive direction. Most of the latter you will not find active on the new community platform, you will, however, find them editing the map and possibly at the pub meets all over the country. And most importantly: Most of them will probably feel embarrassed by Roland’s call for people to put themselves up to be elected just to fill a seat on the board so no one else can.

On the other hand, while the German OSM community cannot be blamed to collectively have the glass-half-empty attitude Roland presents, they are to blame quite a bit for failing to communicate a different, positive vision and attitude in public. Staying away from, and not volunteering for, the OSMF is one thing, not engaging in public exchange with others, also internationally, about your ideas and thoughts on the project is another. This does not have to happen on OSMF managed communication platforms with their behavior control and community management, but it should happen somewhere. And while there are admirable exceptions, collectively the German OSM community has huge deficits in that.

But i am getting a bit side tracked here.

clouds over water with sailboats

Candidates

There are eleven candidates for the four seats open on the board this year – which is a respectable number, considering the general difficulties of the OSMF to recruit volunteers, though it would have been good to be a few more for a broader choice.

I have not formed a clear opinion on any of the candidates yet and i will be reluctant to give a recommendation even after reading their self-presentations (which are probably published by the time you read this). Various collective failures of the board i have pointed out in the past in my blog posts here (including the previous one) have left me astonished how that could have happened, considering the people on the board are largely smart people who have shown good judgement before they were elected to the board. Note i am not talking about decisions that i politically disagree with and criticize because of that here, i am talking about objectively bad actions and decisions. And i have seen in talks with other people that i am not the only one who is irritated by that. The most likely explanation i have for this is that the board in their practical work and in their decision making processes in the past few years increasingly encapsulated itself in an echo chamber where group dynamics seem to strongly influence the views of the board members on things. In light of this the robustness of a candidate against group-think and social pressure, the demonstrated ability to self-reflect critically and the ability and willingness to openly discuss their thoughts in public, are probably important things to look at. Reliably assessing that from candidates’ statements is of course rather difficult. A better solution would probably be to get the board to move their deliberation and decision making into a public setting again – which requires either a considerable change in work culture from within or substantial outside pressure from the OSMF membership.

That gets me to the trends and outlook on the future i am going to discuss in the third part.

clouds over water with sailboats

November 15, 2022
by chris
1 Comment

The OpenStreetMap Foundation in 2022 – looking back

We are approaching the end of the year 2022 and that means the annual general meeting of the OpenStreetMap Foundation is coming up and it is time again to look at what happened in the last year and to update my outlook for the future of the organization.

I am – as i indicated previously in other blog posts – increasingly reluctant to write more in depth on OSMF matters. Apart from the increasing commercialization of OSMF politics and the increasing dominance of lobbying in the political deliberation in the OSMF that i have mentioned and discussed many times before, i also feel there is a progressing decline in the extent and the standard of public discourse on OSMF politics. Outside of the commentary i provide and a few sporadic, isolated remarks of individuals on various – often volatile – channels, substantial analysis and open exchange of diverging views and of the arguments behind them has almost completely vanished from the public sphere of communication. I will get to some aspects and consequences of that development in the following discussion of the last year in more detail.

The funny thing is that i of course have my own business interests in the wider context of OpenStreetMap so i imagine i could (and maybe should?) feel at home and among like-minded people in today’s OSMF. But i don’t. I find the whole idea of political lobbying for economic interests both appalling and pointless in the context of a project like OpenStreetMap. The idea that i could use anything but arguments and reason presented in open public discussion (open in particular to be countered and refuted by anyone interested) to influence decisions made within the OSMF to my – direct or indirect – benefit is completely out of the question for me. That applies to both decisions potentially for my direct benefit (i.e. trying to get into the inner circle of people whose work we know and enjoy) and to trying to influence policy decisions in the OSMF in general in a direction that seems likely to be to my benefit.

Clouds over water with sailboat

I like to emphasize that i am not naive in that regard. Of course in the business world arguments and reason are often not the main factors in decisions that are made. My argument against this kind of lobbying is not only on the moral level but also that it is pointless. As i have outlined in my past discussion of the corporate takeover of the OSMF – if the process of making policy and money spending decisions in the OSMF is a matter of (mostly non-public) negotiations of interests rather than an open exchange of arguments and reason, it would be an illusion to believe that a small actor like me (or any SME for that matter) could successfully lobby for interests that do not align fully with those of the big corporate actors with multi-million Euro lobbying budgets at their disposal. In business, you have to pick the battles you can win. And i can hold my ground in an open struggle of arguments and reason even against a multi-billion Euro giant with a lobbying team of a hundred people, but i stand no chance if this is not about who has the best arguments for their ideas but who puts most power behind their interests.

And the question where the OSMF stands on that scale between decision making based on arguments and reason and negotiation of interests is, as usual, a significant part of what i will be looking at and discuss in the following.

Changes in the 2022 board

The past year in the OSMF (and i mean that as the political year of the organization, starting with the annual general meeting (AGM) rather than the calendar year) started with the former board chairperson, Allan Mustard, leaving the board. As i have already indicated in a previous blog post, with Allan departing the last board member with a distinctly non-technical background has left the board. And while this on its own is not such a major event (he was just one of seven board members after all), it marked the conclusion of a long term trend in the OSMF for many years.

I disagreed with Allan on a lot of questions of OSMF politics and i have articulated that disagreement on many occasions – as those of you who regularly read this blog have probably seen. But at least Allan was regularly exposing and presenting his views to the public scrutiny. As a relative outsider, he knew that while he brought in valuable perspectives that were otherwise missing on the board, he also depended on the input of people with more specific inside knowledge to make meaningful decisions, even if that meant listening to people (like for example me) who you fundamentally disagree with on many things.

That openness to actively seek and listen to diverse and diverging views and perspectives is of course only the first step in the process of making good governance decisions. The next one would be to put into question your own views and preconceptions based on the diverse arguments received and to discuss the conflicting ideas broadly and with an open mind to ultimately make good decisions based on balanced reasoning and an open struggle of arguments. This second part, in particular putting into question your own views and preconceptions and to discuss the conflicting ideas broadly, is what – according to what i observed – Allan struggled more with. But, overall and in retrospect, his public communication can be considered a shining light compared to that of the current board members.

The reason why i emphasize this here is that in the upcoming board election and in the process of four new OSMF board members starting their terms that will follow, these abilities could be something that might be good to look at.

clouds over water with sailboats

What happened in the past year

Overall quite a lot of things have happened in the OSMF during the past year and largely due to the very poor public communication of the board and due to the lack of interest from the OSMF membership, much of this has flown under the radar. The remarkable circumstances of some of the decisions (i will get to some cases in the following) led me to conclude that hardly any of the OSMF members currently feels inclined to do any substantial supervision of the work of the board these days. So a comprehensive writeup of what has happened from an independent outside perspective would be crucial probably. I, however, do not have the time or inclination to really do that so what follows is just a somewhat subjective and sparse cross section.

clouds over water

Present throughout the whole year was the never-ending story of takeover protection/membership prerequisites. In short: In the 2020 AGM the membership, on suggestion of the board, passed two resolutions: (a) One mandating the board to work on a set of proposals to require OSMF members to have contributed to the project in some meaningful way before they can be accepted as members and (b) to investigate the risk of paid voting in the OSMF membership and potential measures to safeguard against that possibility. Both resolutions mandated a completion of the tasks within a year.

The board members seemed not to be able to agree on how to proceed on these resolutions so they appointed a special committee on takeover protection to investigate the paid votes and takeover risks. Nothing came out of that committee that was publicly disclosed though. There was apparently a report prepared internally end 2021/early 2022 but no information about that was disclosed to the members.

Before the 2021 AGM, the board realized they can hardly go into the AGM without having in any way acted in substance on the resolutions of the members – which, as mentioned, ironically were originally approved by the members based on the initiative by the board. So they went into the AGM with the promise that there would be an extraordinary general meeting in April when the board wanted to provide a proposal for membership prerequisites for the members to decide on. It was already kind of foreseeable that this would be unlikely to happen and not surprisingly the plans for the April general meeting were cancelled. Now in late 2022 we are in a way in a similar situation as last year, so what the board did in the October board meeting was to rush doing a survey of the OSMF members (so not a binding vote like they had intended for April) about what the members think of the decision the board has already made in June, which however has so far apparently not been implemented. And this is also not what either of the original 2020 resolutions called for – which are proposals to be submitted to the members at an AGM.

This probably sounds to outsiders like i am making this up as a parody – but i am not.

clouds over water with sailboats

My own take on this: Adding the requirement for a voting member of the OSMF to have shown some interest in and to have gained some at least superficial familiarity with what OpenStreetMap is all about is prudent and 15 lifetime mapping days is a reasonable cutoff for that. But this could have been done a long time ago without making a fuzz and dragging it on taking everyone’s time with repeated votes and surveys for years. If the board wanted to ask the members what they think about this idea then they could have done what they were mandated to do in the 2020 resolutions and make a resolution proposal for the AGM and let them actually decide. Making a non-binding survey about a single policy question a few weeks before the AGM, where they could have presented it as a resolution proposal, is disrespectful and patronizing. By first dragging their feet for two years, then making a decision that is decidedly not what the resolutions called for but not actually implementing it and asking the members instead in a survey what they think about it the board is seriously damaging their relationship with the OSMF members i think.

And no one should for a second believe that such requirement will substantially limit corporate influence on the OSMF.

clouds

A rather significant development during the past year was the rollout of the behavior regulation system and of the centralized communication platform. I have commented on both more specifically already. As indicated in previous comments, observing the hierarchical and centralized community management ideas from the OSMF colliding with some of the more diverse and independent parts of the OSM community is rather interesting – though technically challenging because you have to either use the mailing list mode with the broken references (so without any threading as a flat stream of messages) or the inefficient, patronizing, unstructured and poorly readable web interface. While traditional communication infrastructure provided by the OSMF in the past (specifically the forum and mailing lists) have been managed strictly from the bottom in a decentralized fashion, the new platform has been rolled out with a tight centralized management and a community manager paid by HOT. Every local group of people that wants to have their space on the platform needs to undergo a formal procedure under the auspices of this central management before they get accepted and are obliged to continue to abide by the centrally imposed procedures also after that. Despite these constraints, there has been quite a bit of uptake of the new platform by some local communities (including the German one) but it is also visible that quite a lot of filtering out of people happened as a side effect, partly because of the requirement to accept the top-down imposed social norms and procedures, partly because of the lack of a truly self determined way to participate on these channels.

A lot more could be said about what can be observed there these days in terms of social dynamics, but i will leave that for another time.

clouds over water with sailboats and freight ship

The board has in May started the first board committee that includes non-board membersthe fundraising committee. The non-board members selected for that by the board are all Americans – evidently people whose work we know and enjoy of the Fundraising Committee chairperson. The Committee’s scope was later extended to also include budgeting – meaning that the OSMFs finances are now not only externally (through the main financiers of the organization), but also internally, firmly in American hands.

clouds over water

Another thing that happened in the past year was that the OSMF board has gone ahead with hiring their first employee. In doing so the board follows the pattern established with Martin Reifer (contracted as iD developer in 2021) to (a) sign the contract without a formal board decision and (b) to not publish the contract terms. This is remarkable because an employment contract creates substantial obligations for the OSMF as a whole for a longer time. Note the OSMF also has no regular financial auditing with a financial auditor appointed by the members independent of the management. So all of these contracts and long term financial obligations are done without any independent oversight. Apart from that, one of the important side effects is that this hiring decision probably has put an end to the considerations to move the OSMF outside the UK in the future, at least to do so in full (because you can hardly have an employee in a country where you have no legal presence as a business). The whole moving the OSMF idea is one of the things where nothing apparently happened during the past year – which is unfortunate, because as discussed in the past this would have been the chance to implement some really important structural changes in the OSMF that are difficult to impossible under UK corporate law.

clouds over water

Speaking of what has not happened during the last year – there is a long list of such things. And with the OSMF board being volunteers you of course have to be careful to manage expectations here. So i will only pick a few things where the board has in the past put obligations to do things on themselves to then fail to follow these or where these obligations derive from the core of the OSMF mission. The takeover protection/membership prerequisites is already discussed above in more depth.

Not much happened also on the front of implementation of the FOSS policy. The board in late 2021 had asked the FOSS special comittee to do another review of the (mostly unchanged) status quo in 2022. It is unclear if anything happened on that front but it is clear that the main uses of proprietary software and services remains unchanged, in particular the use of Google for Mail (i am kind of curious what would happen by the way if there ever was a board member elected who refuses to get a Google account…).

Nothing of substance has happened on the front of getting HOT to finally obtain a trademark license from the OSMF for use of the OpenStreetMap trademark. There seems to have been a lot of behind the scenes talks but the bottom line seems to be that HOT still refuses to acknowledge the OSMF’s exclusive right to the OpenStreetMap name and their need to license it and that the OSMF board seems to let them get away with that, maybe in exchange for a wad of money.

Equally nothing of substance has happened on the front of making the OSMF corporate members follow the Attribution Guidelines. This has been an open issue at least since the board has adopted the Guidelines in 2021 (prior the argument was: we need to finish the Guidelines first). There was quite a bit of pushing the matter back and forth between the Licensing working group and the board apparently. There have been pushes from the community for the board to finally do something – with no effect so far. This kind of confirmed my assessment from last year that the OSMF is now inherently unable/unwilling to do anything substantially against the interests of its major financiers, even if it is something at the core of the OSM community’s interests.

clouds over water

Also the whole matter of the software dispute resolution panel for the iD editor, that has been boldly introduced in 2020 and that was finalized in early 2021 came to a screeching halt soon after. None of the formal aspects of that panel as it was introduced was subsequently followed by the board and the panel:

  • The staggered two year terms of office of the panel members are not followed, the board instead has recently decided (after nearly two years) to summarily extend the terms of all the original members indefinitely.
  • No conflict of interest rules have been created for the panel so far.
  • No discussion of the panel and the decision to install it with the OSMF membership has happened so far (it was meant to happen after one year).

As a reminder: Originally, when the panel was created, the board had asked the working groups if any of them would be interested in taking over that function, the Data working group had indicated interest, but the board had rejected that and instead decided to create a dedicated panel appointed by them.

I think it is completely understandable that in light of this and of the now completely unclear governing rules of the panel, no other software project decided to opt into being supervised by it.

seagull flying

Something i have already mentioned in the past: The newly created Engineering working group has made a rather promising start developing a fairly elaborate project funding framework during the first part of the year to then start handing out money directly through single tender action without an open call for tender as mandated by the framework.

clouds over water

Final point – and i have left the most remarkable for last – the OSMF board continues to ignore the need for more meaningful anti-corruption rules and oversight. I have discussed the general problem extensively in the past and also more recently pointed out that one of the main problem seems to be the board’s inability recognize their inherent inability to reliably identify and acknowledge their conflicts of interest and appropriately act upon them. But the October non-public mid month meeting from this year moved this to a completely new level. During the meeting a board member declared a conflict of interest on a matter discussed (a funding request by a third party) because the requesting party are “good friends” of his. Then, in the circular following that discussion meant to decide to offer money, he not only participates in the vote, he also casts the deciding vote (4 to 3) in support of the funding request.

In general, anti corruption laws in European countries and the UK tend to be what we in German call toothless tigers. Even the least capable among all corrupt people will usually be able to weasel around these laws and avoid to violate them directly while doing their thing without serious constraints.

I am not claiming any board members to have any bad intention here – yet i cannot help but state that apparently the board managed to run afoul of anti-corruption rules in the most ostentatious way i can think of and none of the board members seems to have seen any problem with that. And if that is the case, how on earth can anyone seriously assume that the practical procedures followed by the board would prevent any substantial and serious corruption from happening?

clouds over water with ships

Some positive things to end with

As in past years, this first part with the analysis of past year’s events in the OSMF is going to be followed by a discussion of trends visible and likely perspectives for the next years. But i don’t want this post to end on such a drastically negative point.

Not everything in the OSMF was bad in the past year. In particular i saw some positive trends in some of the working groups. I already mentioned that the Engineering working group had a fairly good start initially and with some more diligent fiduciary responsibility and oversight the mentioned failure could have been avoided. The irony is that in the current constellation that oversight would have been the responsibility of the board – finally something where the board would have had a good reason to intervene with the work of a working group, but they did not.

I was also impressed that recently the State of the Map working group showed a remarkable amount of backbone towards the board’s attempts to intervene with their work behind the scenes. I had renewed my past criticism of the current concept of the SotM conference this year again and while i don’t know if my comments played a role, it seems that for the specific question of the location of SotM 2023, the working group members discussed in depth their options and the various arguments that should play into that choice, and made the decision that they deemed to be advised by the arguments, despite knowing that it is going to be unpopular among many loud voices – for Cameroon. This would have strongly pointed in the direction of what i had in the past suggested as a possible future concept for international SotM conferences, namely to hook onto different regional conferences in a rotating fashion all over the world with the international visitors coming in as guests visiting an event of regional design, rather than a traveling conference of ‘western’ design moving around the world to places they consider compatible with their style of conference. And when the OSMF board then intervened, they made the prudent decision to not give in and instead said: Either we do it the way we consider to be the right one – or not at all. The lesson to learn from this: You need to accept to step on peoples’ toes if you want to initiate meaningful change on a larger scale. If this bold initiative will carry fruit will of course still remain to be seen in the coming year and i am not too optimistic about that.

And finally, it was nice to observe that the Licensing working group this year has actually started to more actively reach out to OSM data users to provide proper attribution as required under the ODbL. They understandably excluded the big corporate members, some of which fail to properly attribute, from that effort and pushed the responsibility to deal with those to the board – where, as discussed above, it still sits. This is of course highly problematic because it communicates the message: If you are a small OSM data users you have to abide by the license and the OSMF might come after you if you don’t. However, if you are large enough and have paid your dues to the OSMF you get a free pass. That is not the fault of the working group of course. This activity is in particular noteworthy since with the Attribution Guidelines the board back in 2021 had not accepted the LWG draft as is but made substantial modifications to it.

This is the first part of a series of blog posts on this year in the OSMF – see also the second part with some thoughts on this year’s board election and a third part with observations on current trends and outlook.

clouds over water with sailboat

Green Marble 3 southwest China example

October 15, 2022
by chris
0 comments

Green Marble version 3

I am pleased to announce a major update to my global satellite image product Green Marble.

The Green Marble is a medium resolution whole earth image, offering an unsurpassed uniformity in quality across the whole planet surface on both water and land coloring, based on recent data, and a hundred percent cloud free.

Green Marble 3 in southwest China

Green Marble 3 in southwest China (GM 2.1 for comparison)

The newly produced version 3 provides a complete update to the land surface depiction – based now primarily on Sentinel-3 data (like the water rendering as it was already in version 2.1) – and using a completely new aggregation methodology, based on experiences derived from earlier versions of the Green Marble, as well as techniques developed for regional mosaics.

From the user perspective, version 3 is also a huge improvement in quality and i will provide some examples for that in the following.

Data sources

The first version of the Green Marble was produced exclusively from MODIS data. With both satellites carrying MODIS instruments reaching the end of their life it has however become important for a future proof update path to move to other image sources. For water surfaces i had already moved to use of Sentinel-3 data in version 2 of the image. Apart from the foreseeable end of the supply of new MODIS observations, MODIS data also comes with various issues. In addition to the various problems stemming from the age of the instrument and the fact that only one visible light spectral band (red) is available in high spatial resolution, the more recent versions of the MODIS surface reflectance data available are subject to some pretty severe systematic errors. These essentially make the data unusable for visualization applications without investing significant effort into mitigating them. This has already made production of the Green Marble from MODIS data quite difficult in version 2 and is also likely one major reason why you hardly see any newer larger area visual color mosaics made from MODIS data any more.


Cascade Range and Palouse example (large: GM 2.1/GM 3)

Sentinel-3 land surface reflectance data has its own issues (as i discussed) but most of the inaccuracies are random in nature and therefore not too troublesome when you do pixel statistics. The real problem is that Sentinel-3 Synergy data is incomplete due to the fairly silly masking of water areas. Because of that, i moved to processing Sentinel-3 images from the Level-1 data using Synergy Level-2 data for calibration of the atmosphere compensation. This requires processing a much higher volume of data of course. Overall, about 750 TB of data were downloaded and processed for the production of the Green Marble version 3 – including 120 TB of MODIS data, 320 TB of Sentinel-3 OLCI Level-1 data, 170 TB of Sentinel-3 Synergy Level-2 data and 140 TB of Sentinel-3 OLCI Level-2 water reflectance data.

MODIS data is still used primarily for the following purposes:

  • Cross calibration of colors with Sentinel-3 to improve color accuracy and reduce systematic errors in atmosphere compensation.
  • Supplementing Sentinel-3 data at high latitudes. Because Sentinel-3 records images at a lower sun position and has a stricter recording limit based on sun elevation, it provides less useful data at high latitudes.
  • Rendering of the Antarctic. Sentinel-3 data is incomplete for the Antarctic interior due to the orbit constraints, existing upstream data processing (cloud detection, atmosphere correction) is poor in this area and the ice shelves are largely not included in Synergy processing. Combined with the general high latitude constraints (see previous point) use of MODIS data was therefore much more practicable for the Antarctic.
  • Rendering of sea ice. Since sea ice is not included in either water or land data processing workflows of Sentinel-3 use of Sentinel-3 data here would have required significant additional preprocessing work.

Northern Ural Mountains example (large: GM 2.1/GM 3)

As you can see in the samples, despite the switch of the primary data source, not that much has changed about the overall appearance of the image in terms of colors at small scale compared to the previous version – which is testimony to the highly consistent and accurate color depiction. Both the differences in atmosphere compensation and the remaining systematic errors in that and the different spectral characteristics of the two sensors lead to some color shifts in the results. Ultimately, neither the MODIS nor the OLCI instruments are ideal for accurate visual color representation.


Egypt example (large: GM 2.1/GM 3)

Processing improvements

In addition to the switch in the primary data source, the land data processing methodology was completely redesigned for the version 3 mosaic. This has lead to quite significant improvements in the results, despite using a more narrow data basis in terms of number of years covered.

Apart from the improvements in quality that i will show examples of in the following, i first want to mention that the whole processing – and as a result the final product – are now available with both the illumination and shading as recorded and in a shading compensated version representing the illumination independent color of the earth surface.


Egypt example (large: original shading/shading compensated)

In the previous versions of the Green Marble i had not produced these different variants, except for the polar regions in version 2, because by combining data with a morning and afternoon observation time frame (from the Terra and Aqua satellites) most shading effects in the image data were already eliminated. With the move to using predominantly Sentinel-3 data with a constant earlier morning recording time this changed.


Scotland example (large: original shading/shading compensated)

Based on the shading compensated image variant, renderings with customized shading can be now produced in much better quality.


Kamchatka example (large: original shading/custom shading)

Substantial quality improvements are in particular visible at higher resolution because the noise levels have been substantially reduced almost everywhere.


Scotland example (large: GM 2.1/GM 3)


Chersky Range example (large: GM 2.1/GM 3)

This is the case even in desert regions where it is often not that readily visible in the standard tone mapping, but where you can see a significant difference in contrast enhanced rendering.


Ennedi Range example (contrast enhanced, large: GM 2.1/GM 3)

Conclusions

To wrap up this announcement, i want to provide some historic and market context for the Green Marble as a product.

It has now been more than eight years since i announced the first version of the Green Marble in 2014. During these years, and over the different updates and improvements i provided to the image, it has stayed unique in its market segment. Essentially all market competitors concentrate on higher spatial resolution products but with lower quality in just about every other aspect (like lack of visible clouds, color quality and consistency, completeness in coverage, noise levels). That puts me in many ways in a comfortable position but it also means that i have rather limited information about the market needs and where users and potential users of the Green Marble see deficits and room for improvements.

The improvements i developed in version 3 and before were designed based on user feedback and my own assessment of where there are deficits and where things should be improved. But i regard this to be a rather limited perspective on the product and its value for the user. So i would be highly interested in feedback of potential users of the Green Marble – either in the comments below or via mail. If the principal idea behind the Green Marble as a global satellite image mosaic is appealing to you and for your use case, especially that it does not focus on high spatial resolution but puts other aspects of quality in the foreground, what dimensions of quality are the most important for you? This would be very interesting to know, in particular if it includes aspects that i have so far not put focus on.

Mauretania in Green Marble 3

Mauritania in Green Marble 3

As usual you can find the updated specifications page for the Green Marble mosaic on services.imagico.de. Existing customers with a Green Marble license are eligible for a reduced price update to the new version. If you are interested in using the Green Marble in your application contact me through the form there or via email. An interactive map in Mercator-Projektion for further browsing can be found also on maps.imagico.de.

Patagonia in Green Marble 3

Patagonia in Green Marble 3

Nettilling Lake in early October 2022

October 5, 2022
by chris
0 comments

Autumn and spring in polar regions

In addition to the recent autumn colors images here two views of the polar region autumn and spring on the northern and southern hemisphere. The first shows the Ellsworth Mountains with a low sun position just after the end of the southern winter and the beginning of the observation season in September.

Ellsworth Mountains in late September 2022

Ellsworth Mountains in late September 2022

The second one shows southern Baffin Island with the Nettilling Lake after the first show in early October.

Nettilling Lake in early October 2022

Nettilling Lake in early October 2022

Both views are produced from Landsat 8 data and you can find them in the catalog on services.imagico.de.

September 30, 2022
by chris
0 comments

Blog moving

As you maybe have already noticed this blog has moved to a new location – from blog.imagico.de to imagico.de/blog. This is mostly to support also https connections (for which a separate subdomain is inconvenient). The move was made difficult by the blog location being not easy to change in WordPress. I now finally got around implementing that – together with an upgrade of the WordPress software, which is always quite a hassle meanwhile as well.

All links going to the old location should get redirected to the new place.

Hope everything works as before. If there are any problems please let me know in the comments.

Shiveluch, Kamchatka in Autumn 2022

September 22, 2022
by chris
0 comments

Autumn colors 2022

Autumn 2022 is starting on the northern hemisphere so here two impressions of early fall colors, the first from Kamchatka:

Shiveluch, Kamchatka in Autumn 2022

Shiveluch, Kamchatka in Autumn 2022

Shiveluch detail

Shiveluch detail

The second from Canada, from the lower end of the Great Slave lake near Fort Providence where the Yellowknife Highway crosses the Mackenzie River:

Fort Providence and Mackenzie River in Autumn 2022

Fort Providence and Mackenzie River in Autumn 2022

Fort Providence detail

Fort Providence detail

Both are based on data from Sentinel-2 and you can find them in the catalog on services.imagico.de.

Eastern Himalaya by Landsat 7 on 2022-08-12

September 21, 2022
by chris
0 comments

Landsat 7 last images

According to plans announced some time ago the USGS is going to stop recording images with Landsat 7 at the end of September, ending routine operation of the satellite after more than 20 years.

Like EO-1 where i discussed this matter in more depth, Landsat 7 has run out of fuel to maintain its orbit and has started drifting to earlier morning recording times. This drift has not progressed as much as with EO-1 in 2017 yet but it is visible in the images recorded in the form of longer shadows quite well.

Eastern Himalaya by Landsat 7 on 2022-08-12

Eastern Himalaya by Landsat 7 on 2022-08-12

Eastern Himalaya by Landsat 8 on 2022-08-13

Eastern Himalaya by Landsat 8 on 2022-08-13

In addition the USGS has lowered the orbit of the satellite some time ago to make space for Landsat 9.

Landsat 7 is in particular noteworthy historically because it is Landsat 7 imagery that has shaped the public perception of Landsat as a source of earth observation images more than any of the other Landsat satellites. That is mostly because Landsat 7 was the newest Landsat satellite and the main source of Landsat imagery when the Landsat program moved to an open data distribution policy and during much of the period of popularization of Landsat data that followed (see the history of the Landsat satellites).

The strange thing about this is that Landsat 7 was struck by a major failure in its imaging system (known as the SLC failure – standing for scan line corrector) only four years after beginning of observations in 1999. The failure massively reduced the usability of Landsat 7 data for many, in particular for visualization application. So while the public image of Landsat has largely been formed by Landsat 7 recordings, it has predominantly been images from 1999-2003 that have contributed to that. These about 20 year old images are still widely used in popular map services these days, most notably Bing Maps, but also nearly universally by almost all map services for the Antarctic (though you can now get a more up-to-date alternative meanwhile of course ;-))

Malaspina Glacier by Landsat 7 on 2022-09-05

Malaspina Glacier by Landsat 7 on 2022-09-05

Malaspina Glacier by Landsat 9 on 2022-09-06

Malaspina Glacier by Landsat 9 on 2022-09-06

Technologically and in terms of image quality Landsat 7 is a bit of a relic from another time. Even compared to other satellites launched around the same time (like Terra in 1999 and EO-1 in 2000) Landsat 7 was using conservative technology for its imaging system, with only smaller improvements compared to what had already been used for Landsat 6 in 1993, which failed on launch.

The most significant constraint of Landsat 7 data in terms of image quality is the rather limited dynamic range of the sensor and as a result the high noise levels in dark areas and frequent overexposure in bright areas. The design tried to mitigate that limitation by offering two different amplification settings of the sensor which were chosen based on the expected brightness of the earth surface in the area. As a result of this limitation the USGS stopped recording areas with particularly high contrast (that is especially polar regions) and concentrated the recording capacity of Landsat 7 on the main lower latitude land masses.

Another noteworthy aspect of Landsat 7 was that its high resolution panchromatic spectral band extended across both the visible range and the near infrared, which made its use for producing high resolution natural color images difficult and subject to artefacts in case contrast in the near infrared is significantly different from that in visible light. It shares this characteristic with many higher resolution commercial satellite sensors – both back then but even today. You can see that in the images shown here comparing the appearance in Landsat 7 and Landsat 8 images.

Despite these limitations – for the needs of most data users it seems to have been fairly sufficient for most of the operational history of the satellite. The rather slow adoption of Landsat 8 by data users after 2013 pretty strongly supports this impression. In a way that is also the business model of Landsat overall so far – continuity before innovation.

Chersky Range by Landsat 7 on 2022-09-05

Chersky Range by Landsat 7 on 2022-09-05

Chersky Range by Landsat 8 on 2022-09-06

Chersky Range by Landsat 8 on 2022-09-06

The future of Landsat

Landsat 7 is being replaced directly by Landsat 9 in its orbit and therefore recording schedule. I have yet to write about Landsat 9 in more depth which i have not yet gotten around to. That means we now have a two Landsat satellite constellation with a combined revisit time of 8 days and with both satellites having almost the same capabilities.

What comes after Landsat 9 is still unclear. The most recent publicly available update i could find is here – but this is not providing much more in substance than this and is still rather vague and at the same time it is unclear to what extent what is presented there as what will be (which in a nutshell looks very much like a Sentinel-2 clone with additional spectral bands) is actually already decided on the actual decision making level. In particular everyone should keep in mind that during the early considerations for Landsat 10/Landsat Next the idea was discussed to depart from the full open data model. And there is no clear statement so far that this is not under consideration any more. The existence of Sentinel-2 makes this somewhat unlikely (because with a free alternative there would not be much of a business case for non-free data in a very similar quality and timeliness range). The 10m resolution number that has been widely communicated as target around future Landsat systems is interesting in that regard because this is what now – thanks to Sentinel-2 – is the division line between what is available as open data and what is only commercially available. If Landsat does not go beyond that the status quo would be kind of preserved. If Landsat would move to higher spatial resolution, however, that would massively cut into the domain of commercial satellite operators. In other words: It would probably be politically unfeasible for Landsat planners to try going beyond 10m spatial resolution.

On the other hand a future Landsat with a 10m base resolution would mean an increasing gap between low and high resolution systems with nothing in sight so far to bridge that gap. Let me explain: Over the past decades (essentially since 1999) the main global open data visible light image sources we had were:

  • MODIS (since 1999/2002, near daily coverage at two times of day) at 250m resolution
  • Landsat (since 1999, every 16 days – 8 days with Landsat 5/7 and after Landsat 9 became operational) at 15m reolution (30m multispectral)

With both MODIS satellites reaching their end of life now and no direct replacement being planned we will in the future have from the US:

  • VIIRS (since 2011/2017 + another planned for 2022, daily coverage at 2(3) times of day) at 375m resolution
  • Landsat Next (probably 2030+, probably less than one week revisit interval) at 10m resolution

From the EU we have in addition:

  • Sentinel-3 OLCI with near daily coverage at 300m resolution
  • Sentinel-2 with 5 day revisit (but no complete recording of all land masses in practical operation) at 10m resolution

There is not much in between though, just the Japanese GCOM-C at 250m resolution and Amazonia-1 from Brazil at 60m resolution (which is not operated for world wide recording apparently). But neither of them offers a substantially better revisit frequency than the two satellite Sentinel-2 constellation. This means as a data user you practically have to either work with a resolution of 375m or 300m or go to a much higher resolution but deal with the disadvantage of a a low recording frequency and a large data volume to process.

September 3, 2022
by chris
0 comments

Thoughts on the OpenStreetMap data model

In this post i want to write a bit about the OpenStreetMap data model. This has obviously been influenced by the discussion in the OSMF to make changes to that model i already mentioned in a previous post.

I am a bit reluctant to write about this here and i want to explain the reasons for that first. I am writing about this out of intrinsic motivation. On the one hand for the intellectual challenge of discussing a highly complex interdisciplinary topic. It involves engineering aspects obviously but also natural sciences (in the form of the physical geography that a large portion of the OSM data represents) and social problems (through the role the data model plays in social interaction in the OSM community). On the other hand i also believe that my thoughts on the matter can be valuable considerations for the OSM community so sharing them publicly could be of benefit for the project.

However, the OSMF has started the public discussion of their plans to make changes by moving the whole matter on the commercial level (by commissioning a paid study on the subject). That means taking part in the discussion with the OSMF on the matter will inevitably involve engaging in discussion with people who have economic motives for representing their views. While i do not categorically reject doing that (doing so can still provide better insights into the subject and can be of benefit for the OSM community at large), we are in the field of diminishing returns here. Pro bono fighting an uphill battle against economic interests, defending my views not against arguments and reason but against people who have an economic interest not to change their view, is not a sustainable strategy in any way and is typically not in support of what i described above as what motivates me to write about this matter here.

Long story short: I am writing this not to engage in a discussion with the OSMF of their commercial project to change the OSM data model but to have an open intellectual exchange with anyone interested in the hope to advance our collective understanding of the subject and to educate people who are interested in aspects and context of the topic they might not be aware of.

Writing this while knowing that many of the people who have the most influence on how the OSM data model will develop are – based on past observations – not very likely to be open to arguments and reasoning that challenge their views on the matter is painful – hence my reluctance of doing so. I still do this because i know that quite a few people in the OSM community are interested in diverse views of topics like this and value being confronted with ideas and perspectives especially also if they are different from their own.

So much for introduction – let’s get on the subject.

What is the OSM data model?

I want to start by defining what i want to actually talk about – because that seems to be quite a bit different from what is being discussed in the OSMF.

The OSM data model i want to discuss here is the form in which mappers engage in the act of mapping in OpenStreetMap. Technically it is the data format of the OpenStreetMap API. That is the interface through which mappers receive data from the central OpenStreetMap database and through which they submit back their changes of that data to the project.

On the social level the OSM data model is the language in which mappers in OpenStreetMap from all over the world perform the act of cooperative mapping. No matter what kind of tools a mapper uses or in what human language the UI of that tool is labeled, the OSM data model is the underlying common standard based on which mappers communicate through the act of mapping itself. That should give a bit of an idea of how fundamental it is for the functioning of the project on a very basic level.

The OSM data model is neither necessarily the format in which OpenStreetMap data is distributed to data users (which at the moment happens to be the same) nor is it necessarily the same format in which data is stored in the central OSM database (which at the moment is close to the API data model – though there are smaller differences – like the form in which coordinates are represented).

How does the OSM data model currently look like?

The OSM data model is – in its basic paradigm and compared to other common forms in which geo-data is represented – a very generic, low level format. If i try to describe that in relatively simple terms i get:

  • Geographic locations (positions on the surface of the earth) as the fundamental components of geographic information are represented with so called nodes with coordinate pairs (latitude, longitude) as attributes.
  • Relationships between different objects and concepts that are modeled with more than a single geographic location are represented by so called relations – which contain references to other objects, potentially in a certain order and potentially with a certain role.
  • Sequences of geographic locations in a certain order can also be (and are widely) represented by so called ways which contain references to nodes.
  • All of these fundamental objects can have any number of attributes in the form of free form key-value pairs – so called tags.

Most of this data model is – as mentioned – very generic. That means it makes only very few assumptions about the way geographic information is represented beyond the basic paradigms of geography itself. It is beyond doubt that this characteristic has largely contributed to the success of OpenStreetMap over the past >15 years. The positive effect of the generic nature is usually seen primarily in the free form tagging system. But that is mostly because the OpenStreetMap mapper community has concentrated on that in their activities and is mostly using tags to develop innovative ways to represent geographic information. Many of the other advantages of this model are so far severely underused and i will get to why that is the case further down.

What are the issues with the current model?

I think i already indicated in the way i presented the current OSM data model above that the concept of the ways is kind of an outlier in the model. With the plans of the OSMF in mind i would go a step further and speak of the ancestral sin of the OSM data model.

First of all ways as a concept are superfluous in the OSM data model, a relation can be used to represent what is represented with a way just as well. Or you can look at it the other way round, a way is just a hard-coded and technically more restrictive type of relation:

  • it can only have nodes as members
  • it can not have more than 2000 of them
  • the members cannot have different roles

Beyond that ways are severely under-defined. Ways are usually interpreted to represent a sequence of straight line segments between its member nodes – but nowhere is it defined what that actually means. The most natural interpretation would be to consider the straight line between two nodes to be the shorter segment of the great circle running through those two nodes. Most tools processing ways however interpret a straight line to be a straight line in equirectangular projection (that is geographic coordinates interpreted as cartesian) or in Mercator projection – but that is not defined or documented in any way. If ways were just a type of relation defined on top of the basic OSM data model through consensus in the OSM community and could equally be changed or amended in their meaning through revised consensus, that would be different – we have plenty of similarly under-defined concepts in tagging schemes and relation types in OpenStreetMap. But as a hard-coded element of the low level data model this is highly problematic.

The other main technical or formal issue of the model is that nodes can have tags. Discussing this could get us quite a bit into an abstract philosophical domain. I will try to keep it brief but be aware that this is going to selectively only present some of the arguments.

There is a continuum (or an infinite number of) geographic locations on the planet obviously. That act of mapping consists essentially of two parts:

  • identifying locations which have meaning.
  • documenting what meaning these locations have.

These two activities are not necessarily identical and the same location can have meaning in different contexts – for example a location at the corner of a building has both meaning as a point on the corner of the building as well as one end of the artwork painted on the side of the building and as a (concave) corner of the pedestrian area surrounding the building. That the OSM data model separates these two activities by separately recording the geographic location (the node) and the meanings it has (by being a member of ways/relations representing building, artwork and pedestrian area in the mentioned example) is one of the huge advantages of it. But this is only the case when the meaning is represented through ways or relations. When you have for example a path (way with highway=path) crossing a fence (way with barrier=fence) with a gate (node with barrier=gate) the node doubles as a location and as a carrier of meaning. That is not ideal because the two mapping activities described above are then not separately represented in the data.

Both of these issues are things you can hardly blame anyone for retroactively because they are the result of the historic development of the data model.

The OSM data model would improve a lot in its inner consistency as well as in practical handling if these two issues were fixed. That means the new data model would look like this:

  • Nodes would just contain coordinate pairs and have no tags, Real world features that are to be modeled geometrically with a single coordinate pair would be represented with a relation with the single node as member.
  • Ways would be eliminated and converted to a type of relation.

I am of course not the first one to identify these two things as key points where the OSM data model can be improved. Ilya for example mentioned these ideas recently independent of me. Most who analyze the OSM data model from the perspective of mapping will likely come to the same conclusion (or have done so in the past already).

Are these the only significant issues with the OSM data model that are worth addressing? Probably not. There are quite a few further things, mostly related to the way changes in the data are being represented and managed (with versions and changesets – things i kept out of the description above). But to keep things simple i intend to limit this post to matters of representation of geographic knowledge itself.

The struggles of working with a generic data model.

Of course the structure of the low level data model is only half of the story.

The other side of the topic are the difficulties resulting from having such a generic and low level data model as we have in OpenStreetMap for the process of building consensus on how we practically document geographic knowledge with that data model.

Again this topic is historically mostly discussed in the context of the free form tagging system in OpenStreetMap – which is the practically most visible unique characteristic of the OSM data model compared to ways to represent geographic information elsewhere. There is consensus that the free form tagging system was and is an important basis for OpenStreetMap having developed the way it did (in the positive sense). But, as i already discussed in a different post, this has also resulted in problems, and as OpenStreetMap grows it is going to be of fundamental importance to have a meaningful discussion on how the development of tagging in OpenStreetMap can scale and continue to function in a growing and increasingly diverse OSM community. I presented Tagdoc as a sketch for what i think would be a valuable component to facilitate better understanding and as a result more qualified decision making in tagging and tag development.

There is a subset of tagging that is of fundamental importance so that OpenStreetMap is able to use its innovative data model to its true potential and the OSM community’s struggles to actually develop a meaningful discussion and consensus building process on that has significantly kept back OpenStreetMap for more than a decade.

This subset of tagging i am talking about are the relation types. As i outlined above, relations are the core feature of the generic OSM data model to implement higher levels of abstraction to represent concrete geographic knowledge.

Technically relation types are just tags. But the social dynamics around them in the OSM community are very different from normal tags.

For normal tags the conventions how they are used and interpreted are influenced primarily by the following actors:

  • mappers through the use of the tags.
  • wiki activists (people engaged in editing of tag documentation on the OSM wiki and participating in proposal processes).
  • editor developers through decisions they make with tagging presets.
  • map style developers through decisions what tags to interpret and how to interpret them.
  • other data users through their tag interpretation practice.
  • QA tool developers.

For relation types the primary influences are very different:

  • editor developers through decisions for what types of relations they offer an editing interface to.
  • developers of data interpretation tools through decisions which types of relations they support and how they interpret them.
  • to some limited extent QA tool developers.

As you can see there is a fundamental difference. The practical use and meaning of tags is influenced by a wide range of actors with different backgrounds. This sometimes leads to chaotic situations – which some people despise and call for more authoritarian tagging paradigms. But overall it has served OpenStreetMap quite well – even if there are and continue to be problematic concentrations of power and influence in that system as well.

For relation types however there is a clear gatekeeper role of a very small set of people who practically decide (though they might not actually be aware of that) which relation types are ‘permitted’ (and as a result are practically used in significant volume) and what the mapping conventions are for those. And all of these people are software developers.

As a result of this we essentially have only a handful of relation types in OpenStreetMap that are (a) practically used in significant volume and that are (b) used with a significant level of consistency that allows data users to interpret them in a meaningful way. And all of those have been around already for more than ten years now.

This is the big practical problem in my eyes. In contrast to tags where the power and influence on conventions is widely distributed and grassroots inventions of new tags have a reasonable chance to be successful (and where separating good and bad ideas by mappers voting with their feet usually works) this is not a working mechanism for relation types. This cripples the OSM community’s ability to develop the way it maps the world wide geography in an innovative fashion and prevents it from making full use of the potential of the data model. This is not a problem of the data model itself but stems from the low level nature of it, making it necessary to develop higher level conventions on top of it – which the OSM community is, one a social level, currently unable to do.

This is a hard problem to solve and I have no solution for this to present here. But i know what is no solution: To give up and to move from the generic and low level data model we have to a constrained higher level model of points, linestrings and polygon geometries.

There are quite a few secondary problems that have turned up as a result of the OSM community essentially having been unable to advance on the development of relation types – or of other higher level conventions for representing geographic relations. One prominently visible problem is the overuse of multipolygon relations for representing concepts that could be represented much more efficiently and easier to maintain otherwise. As a simple (though maybe somewhat tongue-in-cheek) example: Think of large islands (like Greenland or Madagascar). These are currently represented in OpenStreetMap with very large multipolygon relations with thousands of member ways. These multipolygon relations however contain almost no information at all that is not already in the database otherwise. If you’d transfer the tags of the relation to a node – either anywhere within the island or on its coastline – that would contain all the information already in a much more compact form. I am not necessarily saying that this is a change that should be made specifically for islands, arguments could be made for keeping the relation here, even from a mapper perspective. My point is that the OSM community currently lacks the fundamental ability to make such a change (or any substantial change in higher level mapping conventions beyond the atomic tagging of individual features). And this is essentially a different side of the same problem. Technically fiddling with the low level data model will not help with that.

What about the other problems?

If you have read the report on changing the OSM data model commissioned by the OSMF you might wonder why so little of what is identified there as problems is discussed here and why most of what i discuss here is not of much importance in that study. That is because i look at the OSM data model purely as the language in which mappers communicate and exchange their work while that study:

  • is almost exclusively concerned with the needs and the difficulties of global scale data processing on the data user side,
  • makes the up-front assumption that the mapping data model and the format of distribution of OSM data en bulk to the data user necessarily need to be identical and
  • essentially proposes to eliminate the function of the OSM data model as the language of exchange between mappers. Instead it suggests that the data editing tools create an abstraction layer between the data format of the API and the paradigms of representing geographic information used by the mappers.

This is the wrong approach in my eyes. The solution to issues on the data user side in processing OSM data on a global scale is simple: Distribute the data in a form that avoids these problems. Doing so would probably be simplified quite a bit if the changes to the OSM data model i sketched above (elimination of ways and of tags on nodes) would be implemented. Also the problem of the two different means of representing polygons with closed ways and multipolygon relations would be eliminated by that.

Where we need to work on solutions is the social problem of developing and maintaining higher level conventions on top of the low level data model OpenStreetMap is based on. That is not an engineering problem of course. While technical means and tools are likely to be useful in implementing solutions to that once we have identified and agreed on them, people with technical skills and qualifications should resist the urge to focus on the hope of finding primarily technical solutions to this. That is not going to work.

Conclusions

That was – despite having grown to a lengthy text again, as you have become quite used to from me probably – a very quick run through a rather complex interdisciplinary topic that only scratches the surface obviously. That means it is going to be easy for people who want to selectively maintain a different perspective on the matter to dismiss my thoughts as too superficial. And that is fine. But make no mistake: That i try to present the topic in a brief and condensed (and hopefully not too cryptic) way does not mean i have not thought it through more in depth. Those who have discussed the OSM data model with me in the past know that i have always approached the topic with an open mind. And i still do. I present no solution that i claim is the right one, i merely present an analysis of the problem.

The main points you as a reader should probably take away from this post are:

  • If you look at the matter from the most important side to consider – that is from the perspective of mapping – it looks very different from how the OSMF seems to so far have looked at it.
  • The main problems are not with the technical structure of the OSM data model (though there are some quite obvious things where this could be improved) but in the ability of a massively growing and increasingly diverse OSM community to productively use the possibilities of the data model to be innovative and efficient in cooperatively mapping the world wide geography.

August 17, 2022
by chris
8 Comments

Over engineering

In the past years i have more and more moved to post my commentary on political developments in the OpenStreetMap Foundation towards the end of the year before the OSMF board elections instead of timely commentary of things as they are happening during the year. I am going to make an exception here by writing down some observations and thoughts on more recent trends and developments i consider particularly noteworthy.

If you have followed some of my more recent OpenStreetMap related posts here you might have observed that i have put an increasing emphasis on pointing out that the value put in the OSM Community on technical work in contrast to non-technical, in particular intellectual work, is quite seriously out of balance and that this is increasingly affecting the OSM Community’s ability to handle the various challenges it faces.

The recognition of this trend and its effects on my side has been emphasized in particular also by the OSMF more recently moving strongly towards an increasing dominance of technical interests and viewpoints. On the level of the OSMF board this in particular manifested in the departure of Allan Mustard from the board end of last year. Allan was the last remaining board member with a distinctly non-technical professional background. Everyone on the board now has a technical professional background and most more specifically in the domain of IT and software development. Allan’s departure from the board is of course not the singular cause of this shift in the OSMF, this is more the conclusion of a long term trend overall which – on the board level – started much earlier with people with a broader non-technical perspective on the board increasingly resigning and the OSMF membership increasingly electing people with technical backgrounds.

At least to me it became increasingly clear in the last months how much of a paradigm shift this is and how much of an impact on actions and decisions of the OSMF this could have in the future.

Beyond the composition of the OSMF board this trend is best visible in form of the shift in expenses for paid work. Until late 2020 the OSMFs main regular expenses for paid work were administrative assistance and accounting. This has completely shifted by contracting a software developer for iD and hiring a sysadmin since then. The exact amounts of money spent here are not known (contracts are not published any more and we will probably have to try to reverse engineer this info from the financial reports at the end of the year). In addition there has also been an increasing volume of non-regular paid technical work (in particular the three hand picked projects in the aftermath of the microgrant program). The newly revived Engineering Working Group now has a EUR 50k budget for paid work – the highest of any working group after Operations if i am not mistaken.

A project of the Engineering Working Group is also what i want to discuss in more detail here. Earlier this year the EWG has decided to contract a study for changing the OSM data model.

The specific details and motives for this are unclear – the minutes do not reveal substantial information on that. What we know is that this study was contracted through single tender action (the contract terms are not disclosed, not even the exact aim of the study) and not through the EWGs project funding framework.

The study the EWG has contracted has now been published and this is what i want to discuss a bit more in depth here. I am not going to comment on the technical aspects in substance but i want to share a few thoughts on the economic and social context of the whole thing.

I am – by education – an engineer myself, with a master and a PhD in mechanical engineering. During my early years in engineering i had – like many other engineers – a tendency to look down on consulting companies doing feasibility studies and receiving quite significant amounts of money for those while having no real street credibility so to speak in the domain of engineering. Furthermore these consulting companies often did not predominantly seem to employ engineers but people with a background in economics or social sciences.

This negative view of the young engineer in me has significantly changed since then. And in case you wonder: This change has happened already before i became a consultant myself ;-). While i think many larger generic consulting companies (which is what many people have in mind when they hear the term consulting) have very questionable business practices and models, i meanwhile have a significant appreciation of the work of smaller specialized consultancies, in particular in producing feasibility studies, and consider their role in our society with its highly specialized competencies and division of labour to be quite essential.

My impression is that what the OSMF apparently did here is the approach of a naive young engineer towards doing a larger project: Contracting an experienced engineer with practical work experience in implementing this kind of project for a feasibility study with the aim to find out how they can make the project work.

There are very good reasons why this approach is not typically taken.

One obvious reason is that the contracted engineer has, if they are also a likely candidate for being contracted to implement the project, a clear conflict of interest. This is one of the main reasons why feasibility studies tend to be done by independent consultancies who have no stake in the actual implementation of the project.

The second important reason is that in most larger engineering projects the main risks and obstacles are typically not technical in nature. To truly assess the feasibility of the project, the risks involved and the resources required, you need experience outside the domain of engineering. You need to regard the broader social and economic context of the project. This is one reason why consulting companies frequently employ social scientists and humanists.

I am not sure if it is realistic to expect people in the EWG and on the OSMF board to realize they took the wrong approach here and recognize the need to revise that. But i know that there are plenty of people in the larger OSM community who have a broader perspective on the matter and who will therefore likely be interested in a more critical commentary on the approach taken to this project here. In any case i thought it is prudent to make this comment early to give everyone the chance to consider it.

What impact does this have on the actual matter of the OSM data model and its future development? I obviously have my thoughts on that too. But this is beyond the scope of this blog post.