It has been more than two weeks since SotM in Heidelberg and some are probably already wondering about my commentary. Part of the delay is due to me being busy with other things, part is due to some information i would like to have had not being available so far (i will get to that later).
The venue
Overall it was a pleasant experience for me. We had mostly luck with the weather so the choice of time of year was not too bad. I mentioned already last year that Heidelberg is conveniently close for me with only a two hours train ride to get there. Heidelberg – of all possible locations you could have chosen in Germany – is more near the upper end in terms of accomodation costs. But since a public transport ticket was included in the conference ticket for the duration of the conference for the whole regional transport area you were not limited to stay in Heidelberg actually so there was a pretty broad range of cheap options to stay – although probably not that easy to find for visitors from abroad. The information on budget accomodations on the SotM website could have been better – but that is a problem we essentially already had in previous years. Side note: I find it kind of annoying that the website has removed much of the pre-conference information after the conference – this is IMO not good style – maintaining previous information for future reference is important. Update: This has meanwhile been fixed – see comment below.
The venue itself was pretty well suited for the conference i think. All the main rooms for the talks were very close together – the rooms for the BoF sessions were a bit further away but quickly reachable as well. The most serious issue was IMO the acoustics in the large lecure hall (Großer Hörsaal) where – as you can see in the videos – the speakers frequently had trouble understanding questions from the audience. This is obviously hard to get right in such a large room and lecture halls like this are of course not really designed for dialogue between the speaker and the audience.
Conference size
Though we have no numbers and statistics so far for the conference visitors it was quite clearly the largest SotM conference so far. When the original planned number of tickets was sold the conference organizers increased the capacity beyond the original planning to allow more people interested to visit. And all of this together showed at various places during the event. Most obviously at the social event on Saturday where the catering turned out to be under-dimensioned – both in terms of amount of food and distribution capacity. Thanks to the good weather and the possibility to go outside the venue for the social event itself was fine – though a bit short in sitting opportunities outside the main room intended for eating, which turned out to be a bit stuffy and very loud – the whole venue was a former industrial building which in terms of acoustics was obviously not designed for a large number of people.
The poster session on Sunday evening was also a bit sub-optimal because the poster display and catering were separate on different levels so there was not really a natural looking at and talking about the posters while eating and drinking. This was also owed to the limited space mostly – even as it was the area where to drink and eat was crowded and there would have been no way to also appropriately display the posters directly there.
At the conference itself i did not see any problems with overcrowding or overfull lecture halls – at lunchtime at least during the first day the lines were long but there was plenty of space so it was not an issue.
In total my conclusion is that this kind of conference size exceeds the limits of what can be reliably managed the way SotM conferences have traditionally been operated. Specifically
- organization at this scale IMO either requires a well-established team with multiple years of experience with a presence on-site during most of the planning phase or help from a professional event organizer with experience with this kind of event.
- any location for a conference of this size with sufficient capacity – no matter if the main conference venue or for a social event – is going to be hard to find and organize so for a conference of this size the confirmed availability of all needed locations would usually need to be a pre-requisite for selecting a location for the conference – something that for SotM has so far not usually been a criterion (applications usually mentioned possibilities but rarely came with a full set of places with confirmed availability).
All of this is not meant to say the organizers did not do a good job – the opposite is the case IMO: Given the number of visitors it really went quite well.
The talks
I have not yet watched all the videos of the talks i did not see at the conference so here only a few select comments on the talks i recommended before the conference and others i went to.
- Introduction to OSM: How it’s made and how it’s used (video) – although i have not seen this live at the conference based on the video this really worked out fine. This is definitely a format that could be built on in the future, potentially also for more specialized topics.
- Communication and Knowledge Transfer in OSM (video) – as i hoped for this gave you a fairly broad look at the different communication methods used by the OSM community and their advantages and disadvantages. Really recommended for anyone who wants to take a look over the limits of their horizon of what channels and platform they are famiiliar with.
- Mapathon, mapathon, mapathon! (video) – This offered an important critical view on common practices in remote humanitarian mapping efforts and their historic development. In my pre-conference post i likewise mentioned the followup talk (video) which focused more on organizational aspects of local communities in countries outside Europe and North America – which i can also recommend.
- I specifically also want to mentioned a later event organized also by Nicolas and Severin – the Bilingual Breakout Session – Community building and empowerment in South: French-speaking countries in Africa+Haiti (video). I think this could nicely serve as a blueprint for future cross language communication formats at OSM related conferences. Language barriers are one of the main limitations for cross cultural communication within the OSM community and this format shows how this can be overcome with limited effort and lead to much improved communication between people speaking different languages.
- Is the OSM data model creaking? (video) – this was a pretty solid analysis of the various practical problems when working with the current data in OSM for roads and paths for navigation purposes. Unfortunately in the end the considerations on how to address these problems were primarily data user centered and not mapper centered – in other words they wondered what might be the most convenient way to represent things in data form for the data user rather than for the mapper.
- New processes to agree on tagging suggestions and their interaction with the editing software available on openstreetmap.org – considering the controvertial nature of the subject the discussion was actually pretty civil and meaningful. Roland already posted a summary of the results of the discussions and i really hope people will follow up on the ideas that have been discussed there.
- OSM Vector Tiles in custom coordinate systems (video) – this was a very disappointing talk. The only thing they actually showed was maps in equirectangular projection – which, coming from Mercator, is kind of inverse evolution. No substantial discussion of any of the actual problems and challenges when creating digital maps in projections other than Mercator.
- Board + Working Groups meeting (video) – this was rather interesting regarding the dynamics within the OSMF though ultimately not really that productive, which is how it was probably kind of expected by everyone. The format worked out quite well, there was a lot of commentary and discussion happening. There was quite a bit of what i would call essentially the OSMF circling around itself without much connection with the OSM community outside the OSMF but there were also plenty of interesting and valuable comments by various people i would encourage everyone interested in the OSMF to look up in the video (and if you were there – maybe re-contemplate them again).
Audience
I would have really liked to look at the number of visitors at the conference from different parts of the world here as i did last year but so far unfortunately no such information has been made publicly available.
My impression was that the audience composition was similar to last year in Milano with two main differences:
- there was obviously a significantly larger fraction of visitors from Germany.
- it seemed also that due to the HOT summit having taken place directly before SotM in Heidelberg there was a larger fraction of visitors with a HOT background.
One thing i noticed though is that although Germany has a very large local hobby mapper community – possibly the largest one world wide – there were relatively few pure hobby mappers at the conference. There is a huge overlap between the German visitors of this year’s SotM and the regular visitors of the FOSSGIS conference with an OSM background. Keep in mind though that FOSSGIS is not purely an OSM conference and FOSSGIS visitors with an OSM background are only a small subset of the German OSM community – despite FOSSGIS having free admission for active community members. There was some IMO quite understandable critique from the German mapper community that even the early bird community ticket price is kind of steep for a hobbyist. Given the conference was sold out quite early this understandably felt a bit like this financial barrier served at least partly to give professional visitors priority over local hobby mappers.
And i think everyone should be able to relate to a hobby mapper who has invested possibly thousands of hours into mapping their local area over the past decade and who therefore does not feel right about the need to pay EUR 75 for being able to visit a conference where others who largely have invested much less get their travel expenses paid in full.
Scholarships
Which brings me to the SotM scholarship program. I wrote about this before the conference but back then had only very limited data on just 10 scholars. Now we have a bit more information – both on the scholars and the selection process.
Some might wonder why i make so much fuzz about the scholarship program. The reason is that this is about quite a lot of money – in 2018 this was more than 20k GBP and given the larger number of scholars this is probably even more this year. While this is perfectly affordable for the OSMF that does not mean it is all right to spend all this money without proper accountability and consideration – money that could obviously also be spend for other things where it might do more good for the OpenStreetMap project.
Here is the updated map of where the scholars come from – this year as well as in the previous years. My analysis of this has not changed much – there are additional scholars from North America, Europe and (partly French speaking) West Africa. But the complete gap in Northern Africa, the Middle East and Central Asia is still there.
What we now have is – for the first time – numbers on the regional distribution of applications. These i tried to illustrate in the following maps. The first is for the total number of applications and the second is after filtering out formally incomplete applications.
I get two main observations from these:
- the bias towards English speaking countries and former British and US colonies is even larger in the applications. More generally speaking much of the bias observed in where scholars came from in the last three years seem to be already pre-defined in the applications. This however does not mean the scholarship program can’t do anything about it. If the call for applications selectively speaks to people from some countries but not from others there is a reason for that.
- there seem to have been essentially mass applications from a number of countries (Nigeria, Uganda, Ghana and Kenya with each more the 35 applications) and from none of these countries there was ultimately a scholar selected.
What we also have is more detailed documentation of the selection process – most of which however seemed to have not worked out as planned. A number of observations on the selection:
- We have a list of people involved with scoring – however the documentation also said the plans for scoring “did not work out as initially planned”.
We have no information on the scoring itself (like for example the anonymized scoring data in comparison to the final selection)See addition below. - There is a list of suggested criteria for scoring – however these are fairly vague and in parts also somewhat questionable. In particular the “unique story or experience to share” is highly prone to depend on cultural commonalities between the applicant and the reviewer.
- The list of people involved in scoring lists 13 people – 8 of which have been scholars in 2018/2017. While i mentioned in last year’s comments that involving previous scholars in application evaluation could be helpful i also mentioned that disqualifying them permanently for applying for scholarships in the future would be absolutely essential for this to work. Otherwise you’d have a high probability of a ‘revolving door’ system evolving with people switching roles from scholar to reviewer and back every year. When you look at where the former scholars who scored applications came from (Kenya, Lesotho, Philippines, Germany, Russia, Uganda, Niger, Nepal) there is no clear pattern – while scholars were accepted from the Lesotho, Philippines and Nepal there were none accepted from Kenya and Uganda despite a large number of applications. So i don’t see indications for any actual impropriety in the process here but none the less in this form the system runs a very high risk of favouritism.
- There seems to be no formal conflict of interest management of any kind. Given that quite a few of both the people involved in scoring as well as the accepted scholars have a job with some OSM connection or a formal position in an organization with OSM connections (like being HOT voting members) this is a reason for concern.
- As i analyzed before the ultimate selection of scholars seems to be based on ensemble optimization rather than independent rating of the individual application and selection of “the top 20”. And as i read the documentation this final selection was done without any oversight by a single person who essentially decided where more than 20k GBP will go. If i was an OSMF financial auditor (and i am really glad i am not) this would be something i could not accept.
Note the obvious derivation between the social structure of the OSM community and the selection of scholars has a high likeliness of becoming a self reproducing system – even without former scholars being involved in the selection. Given the de facto preference for people with a job somehow connected to OSM or a formal position in some organization local hobby mappers from all over the world without such connections will realize that their chances for actually getting a scholarship are very small and will depend on them presenting themselves as being alike and compatible to the professional OSM environment. Introverts or people from cultures with predominantly different communication styles will have almost no chances because they do not match the established ideal for a SotM-Scholar.
Overall i think independent of the future of SotM the OSMF board needs to pull the plug on the scholarship program in its current form. Even if for the moment i give everyone involved the benefit of the doubt and assume that they all tried their best to accomplish a just and unbiased selection this is just a disaster waiting to happen – either through deliberate favoritism and corruption or just through plain incompetence. For neither scenario there seem to be mechanisms in place that would prevent this.
Any kind of scholarship or financial support program (and i specifically also have in mind ideas like microgrants here – which unfortunately might be destined to be managed in a similar fashion) would in my opinion need to be managed by and with broad support from the hobby mapper community. The fact that hardly anyone from the community seems to be currently interested in helping with that should tell the OSMF board and the SotM-WG that there is something seriously wrong with how it is run at the moment. Even if (or more precisely especially if) you feel scholarships are important to have you should work towards giving it a fresh start with a proper mandate and a solid ethical and procedural framework. This would give potential volunteers the confidence that they can be comfortable in contributing to something beneficial for the project. The more detailed documentation we have this year about how the selection process actually took place is much appreciated but to me this mostly better illustrates the lack of and the serious need for a proper framework of binding rules and control mechanisms.
Addition: There was more data and information added after i started writing this post. Everyone is encouraged to have a look at this.
The future of SotM
Coming to the last part and a renewed critical view of the idea of the SotM conference in total. I have expressed my concerns about this before, in particular about the illusion of SotM being a conference for the whole OSM community. Next year’s conference is now planned to take place in Capetown, South Africa. This makes it relatively easy for me since there is very little chance i would want to go there – at least not on my own expenses. Roughly estimated for the cost of visiting SotM in Capetown i could probably visit most of the local OSM conferences in Europe during that year which would allow me to meet more people and it seems overall more attractive than a single visit in South Africa.
When i originally suggested to stop having a dedicated international SotM conference and instead having the OSMF every year give special support to a regional OSM conference most reactions i got were negative. This year talking to people at SotM about the future of the conference i heard a lot more people essentially agreeing with that idea. Overall i would say there are two potential futures for SotM:
- giving up the idea of an international SotM and instead giving rotating support to local/regional conferences fully managed and organized by the local mapper communities. The goal could for example be for the OSMF to provide financial support (through either OSMF funds or by organizing sponsorships) that allows the conference to offer free entry to local community members and thereby ensuring broad accessibility for local mappers. In addition the OSMF could organize video recording and live transmission of the events at the conference and this way facilitate broader reach and participation without the need for expensive and resource intensive travel.
- giving up the pretense of SotM actually being a community conference and concentrating it on what it mainly is right now: A meeting of professionals with OSM connection and the international OSM jet set in addition inviting some locals of the OSM community from the place they meet at.
In the discussing for next year’s place for SotM (the decision was made last minute during the conference this year) the selection was between two applications – Rapperswil and Capetown – which even without the travel costs of getting there are both on the expensive side. The documented criteria for selection essentially already make it abundantly clear that this is not a community conference but primarily targeted at the interests of business visitors and wealthy cosmopolitan hobbyists. Affordability of a visit is nowhere to be found on the list. So in a way we are already pretty far into the second option. But broader realization within the community that this is the case – and next year’s decision for Capetown further underlines that – could also create more support for the first variant.
And just in case anyone wonders – this comment probably would have been more or less the same if the decision had been made for Rapperswil. My personal travel costs for a visit to Rapperswil would obviously be less than for Capetown but that is only because of the costs of the flight. The local costs would probably be even higher in Rapperswil. For most people from outside Europe the difference would probably be fairly low.
October 11, 2019 at 01:20
Hello! Thank you so much for the insights here. What you wrote makes me think a lot about the community aspect of OSM. My thoughts/reflections:
1. We should also recognise that several of the scholars, while they have a low edit/changeset count, contribute a lot to the community by organising events, connecting people, building relationships in their home countries.
2. I really like the idea of the rotation of SoTM to various regions.
Again, thanks for your very thoughtful piece.
October 11, 2019 at 12:44
Hello David,
thanks for your comment. Although i have not looked at all of the scholars and their personal history of contributing to the project i don’t doubt they all made significant contributions and quite a few of them probably even exceptional ones. And i am pretty sure all of their visits to the conference were of benefit for the conference and the OSM community in some way. Still i think the question makes sense if there are better ways to invest the money – like for example offering a larger number of active local community members free tickets for the conference and thereby create a low barrier chance for participation for the local mapper community or to ask if the selection process and the call for applications are designed in a way that ensures a good balance and fair chances for all OSM community members.
October 15, 2019 at 13:10
> Side note: I find it kind of annoying that the website has removed much of the pre-conference information after the conference – this is IMO not good style – maintaining previous information for future reference is important.
That’s a fair point. We’ve restored the information on the website about the (expired) conference hotel offers: https://2019.stateofthemap.org/accommodation/#hotels
October 15, 2019 at 13:30
Thanks. I added a note about this.
Pingback: weeklyOSM 482 | weekly – semanario – hebdo – 週刊 – týdeník – Wochennotiz – 주간
October 23, 2019 at 08:56
Thank you for the good conference report.
It’s only my opinion.
>The future of SotM
>giving up the pretense of SotM actually being a community conference and concentrating it on what it mainly is right now:
>A meeting of professionals with OSM connection and the international OSM jet set in addition inviting some locals of the OSM community from the place they meet at.
I agree with you.
Even if we had a local conference in Tokyo, I think there are 3 types(groups) of participants that you wrote last year.
And I think there are members who think that the transportation and accommodation costs from a small town to Tokyo are high.
But I would like to have the conference in Big City.
I think it is better to hold conferences in Big cities to balance OSM professionals and business people.
> giving rotating support to local/regional conferences fully managed and organized by the local mapper communities.
That’s ideal.
Even in the country, we think we should go on a local tour when local members power are not enough. It doesn’t have to be a big conference.
November 6, 2019 at 23:17
On the scholarships, I’m glad we provided much more detail on the process this year.
A few comments on your observations.
I think we all want to see more applications from more places and in more languages. We tried to get the word out widely, but help is needed. Part of why we have fewer applications from North Africa, Middle East and Central Asia is underdeveloped or at least under-connected OSM communities in these places. Part of this is outside the scope of scholar program — we should still do more. Partly due to visas (I think some good candidates this year were disqualified by this) (several years before we in fact had visa trouble with a scholar from Palestine). I’ll note that one of the first scholars in 2009 and awesome presentation did come from Egypt (video here https://vimeo.com/5607276).
I think part of why we see more applications from Nigeria, Uganda, Ghana and Kenya is due to YouthMappers programs there, which promote the scholarships, but don’t help with application preparation or setting expectations. Would love more support directly to applicants.
Involving former scholars in scoring worked well. While we don’t have a formal rule against people applying again after two years, it did weigh pretty heavily against selection, and no former scholars were selected. I’m open to thinking about no repeat scholarships as a rule, but I also think there may be a rare exceptional circumstance at some point where we’d want to extend scholarship to someone again.
Yes final selection is made of an assessment of the whole group. Selecting the top 10 just based on score wouldn’t be right, when everyone in the top 25 is pretty close. I did have to make the choice here, though I would have preferred to do this within a group. It was not without oversight though — the rest of the committee evaluated the outcome and weighed in.
There was no preference for those with an OSM job in the selection, in fact the opposite. The final stage selection disqualified those who’s primary position in the OSM community is through a fairly established employer.
Otherwise I whole heartedly disagree with your conclusion that the program is rife for corruption and that it isn’t supported by a broad base of the community. I think we can definitely improve a lot in 2020, including more involvement from those ready to get the word out, evaluate applications and support the logistics process of bringing scholars to SotM. Doing scholarships is hard work, and I encourage anyone interested to get involved and make this work great.
November 6, 2019 at 23:24
Btw, I also agree we should examine the role of the international conference, and look to other models. I’m not ready to say cancel SotM after next year, but there’s definitely a place for a different kind of event. It’s not only expense, but the climate costs of global travel.
Closer to home, I’d love to see SotM US evolve to more of an un-conference itself, rather than replicating part of what happens at SotM; even then, the US is so big, that a national conference makes it hard to collect “local” energy. I’m not so concerned about the artificial divide between “hobby mappers” and others. I think the more we connect folks who just do OSM for a little bit of fun, and those that dedicate themselves in other ways (and I think anyone engaging heavily in the OSMF or writing about OSM is way beyond a hobby), the stronger the whole community becomes.
Also note that SotM did make free tickets available for those ready to do a volunteer shift at the conference. We possibly could advertise that better.
November 23, 2019 at 15:11
Thank you Chris,
This has been read by the (new) local SotM team. It’s spurred some discussion and thoughts on things for next year.
November 23, 2019 at 15:41
Don’t you think it would be important to have such discussions in public or at least publicly minuted? After all these considerations and decisions will ultimately decide how several tens of thousands of Euros of community money will be spent.