Looking for team members project to enable Bench-marking Map Edits

Google Maps contribution stories
1

Hi,
[EDIT: The basis of this project is not against the rules, see this post by LG Team member).

I have been walking around with the wish to track my Map Edits score and benchmark my score against regional and global statistics. The next step was thinking about a way how we could make this into a project to benefit individual Local Guides and unofficial Local Guides Community.

The Project

A collaboration research project where we invite both unofficial Local Guides communities and individuals to participate.

The aim of this research-exercise is to be able to bench-mark your own Edit Score against others.

The secondary aim is to make Local Guides more aware of their performance and the concept of making a risk assessment before they submit an edit.

Questions to readers:

1) Do you think this research project is a good idea? I am going ahead with this, no it is just a case of making it perfect and successful with your help!
2) Are you interested to be a participant ( a person who enters his score data and benchmark his/her score)?
3) Let me know if you are interested to join the actual Project Team.

Team Member Requirements:
To become a valuable member of the team, you obviously need to be able to invest time with us in executing the project. First, we have the task of setting up and testing the automated system (using Google Forms). Once it is up and running, team members need to monitor that the automation of the data collection is working correctly. In addition, there shall also be manual admin work, including corresponding with our partners (the regional unofficial Local Guides Communities) and participants (the people who share their score data).

Participants:

For those that are interested to participate in the actual research and benchmark their performance, please let us know in the comments.

This project is part of the (unofficial) #LocalGuides Community Exchange Program:

community-exchange-stamp.png
community-exchange-stamp.png150Ă—147 13.9 KB

Looking forward to hearing from you.

Jeroen

[SKIP OUTDATED DISCUSSION: don’t waste your time, jump to this post here.]

10 Likes
2

@Osaka78forTRUMP and @Briggs

Given your interest and experience in research, it would be great to have you on the team.

3 Likes
3

Hi @JeroenM

Answer of the first question is. .

Yes this is a good idea. Whoever benchmark their data score.

Just wanted to ask you how we can do it? Please spot the light on that.

For example i am a member of Malaysia local guides community so what should I do? And how I participation in this research.

Thanks

6 Likes
4

@AkmalB

Thank you for your interest in participating and your support to the project.

Without going into the technical details, this can be done with Google Sheets and Google Forms. We would create a master individual submission form that can be copied. This personal form (one for each participant) would have a link to the personal statistics. The data from each individual participant feeds into a regional score sheet so that the regional statistics can be generated. That regional score sheet, in your case the overall score sheet of the participating members of Malaysia Local Guides feeds into the global sheet. We should probably also look at the possibility to add an extra level so that the regional score sheets (Local Guides communities that represent a city or province) feed into a national score sheet.

As moderator of an unofficial Local Guides community, you would be asked to promote the research and provide your members with a special sign-up form, so that the people participating get a link to a correct Google Form that feeds into your Malaysia Local Guides score sheet.

Once the project is up and running, you get of course access to the anonymous data of your members so you can publish statistics on your community G+ page. We will probably offer participants the option to participate in a public (regional & national) scoreboard. So you could list people in your community with the best score to trigger those competitive energies :wink:

Does this make sense?

6 Likes
5

Yes understand.

Thanks for the information and details @JeroenM .

And I think its not against the terms and policies of the local guides programme?

Thanks

4 Likes
6

Dear @JeroenM , hope all’s well. Suggested edit quality is already fully quantifiable. Google’s trust system applies to all Maps contributors. Local Guides points provide basic detail of individual contributor profiles. Please explain clearly, what additional information are you seeking? Thanks and kind regards from Osaka78

3 Likes
7

Hello Jeroen,

As I share the same sentiment with Osaka, I’m actually a little bit more confused with the aim for this project. There is a backend trust system that us regular Local Guides don’t see, and all mainly computerised by the Google Maps API. Where there would be something very useful to aggregate the percentage of edits that we make, trying to quantify our “correct” edits that are not applied would skew the data severely. I know by looking at the percentage amount of edits being applied on Facebook Editor, being mainly a social experiment to get things right, people cannot understand the actual purposes or what the core standard of edits should be. So I would like a bit more of a further explanation too hehe!

5 Likes
8

Thank you, guys, @Briggs @Osaka78forTRUMP , for questioning the purpose of this project. You are proving yourselves again as analytical thinkers.

@AkmalB , with regards to the legality of this project, I would like to say this: Even if a project would be strictly within the rules of the program, but potentially damaging for the Local Guides Program or damaging to Google Maps, I would not wish to go near it. I trust that the community here on Connect is mature enough so that through peer review, bad ideas (even if well intended by the initiator) are being “objected” by either the community or the Moderators. Sometimes it requires looking at the pros and cons from different perspectives, so if you see a reason how this project would hurt any one’s interests, please don’t hesitate to let me know.

The main concern would be: would the data this project reveals benefit hackers and bad people that wish to beat the system? I don’t think the data we shall share with the public would reveal (secret) insights in how the bots evaluate edits. We probably confirm with this research that making consistent safe edits improves one’s success rate, but that is not a secret.

What (I think) the project does:

Collect data for each edit a person makes. These are the data fields:

  • URL of the location of the edit (not relevant for the statistics but great to have available when doing quality evaluations afterward).
  • What was edited in the single submission (name, address, pointer, category, website? etc.).
  • The outcome: either Accepted or Not Approved (not interested in Pending, or are we?).
  • The subjective judgment of the participant if it was a high or low-risk edit.
  • Was it the first attempt to get this changed on Maps or not?

Primary benefits from an individual point of view:

  • Monitor performance. What is the success rate of my edits (e.g. 10% get’s Not Approved)?
  • What type of edits do I score the highest and what type of edits do I score the lowest? (for example, a person does badly with editing categories and scores highly with Name Places).
  • How is my personal judgment? Do most of my “safe” edits get approved?
  • How do I perform compared to others in my area?

Answers that this project might provide:

  1. Are there areas where certain people (individuals, region, global) perform below expectation? Can this group be helped with education to improve their performance?
  2. Do people that risk assesses their edits before they submit them, do better than people that don’t?
  3. Are there trends (long term and short term). This one must interest you, Briggs, as it could tell you that perhaps after launching a new bot, many high performers like yourself do temporarily badly. On another level, we might find that very few people edit categories. Does this mean they don’t look at them when evaluating the data of a place they visit? If so, should we “campaign” for people/ members of a regional community to pay more attention…OR it could give us insight in what data is mostly “dirty” on the maps. Does that tell us anything?
  4. Is the success rate different when it is not a first attempt (should we ask if it was the second, third, fourth attempt? to be more specific?)

The additional benefits for unofficial LG Communities:

This program could greatly boost the regional communities. After all, when a Local Guide wants to participate, they get the application from their regional community. This could have a positive side effect that the regional community will get more local Local Guides becoming a member of their community. Secondly, it leverages the value the regional community has and thereby (potentially) improving the engagement by members of those participating communities.

The project is completely executed outside the Maps environment. So, I was not thinking about using the API to collect data. I am not totally up to speed with what can and cannot be done with the API, but as far as I understood one cannot call for “My Contributions” via the API. As participants, would manually submit their performance data per edit, I don’t see how this could in any way upset any system related to Maps.

@Briggs , I am leaving statistics related to valid edits that are being rejected out of the equation. After all, you as an expert would be able to have a good idea if you make correct edits, but of course, many of us don’t know when we are making a mistake (because we are not properly trained!). Rather than asking participants for their LG level (do we really think it has an effect on your trust score?), personally asking for the number of total edit contributions they have made, gives a much better indicator of their experience level when analyzing individual data. Having said that, seeing some of the poor advice given by high-level LGs here on connect, it still could mean that certain top contributors are poor in making quality edits…

Yes, we potentially change the behaviour of participants, but I was thinking in a positive way or am I missing something?

5 Likes
9

I just had another thought. For the purpose to distinguish adding new places and people editing multiple fields in one go for an existing place on Maps, perhaps we should add a field in the form where people say, I am adding a missing place.

4 Likes
10

It could be something along those lines. Essentially we’re looking at pretty much evaluating our own edits. For initial parts, our editing capability might be quite near zilch but it would start to be refined as time goes by. I have a feeling that this is quite similar to the current spreadsheet that I posted a few days ago (although that’s really out of the question now since everything is being denied by the system unfairly). Pending edits are a strange thing because half the time you’ll never know, and the essence of time has too many variables.

There is really no way to beat the system, since the Maps API itself is constantly evolving. The only mistakes that I would have is if I’ve initially marked a place as closed, only to find further down the road or deeper into the shopping centre (or even on another floor) that it actually exists, mostly due to incorrect map marker locations. LG Level doesn’t really seem to be a massive factor, although being Level 9 allowed me to find hundreds more edit suggestions to be checked and verified.

Furthermore, the advice to involve the local communities into this project might seem a little bit taxing, and most people would often question the authenticity or validity of such a project, especially when filling out a form. Most people are pretty light and casual when it comes to local guides mapping, so I wouldn’t exactly know where to start or how to get more people involved, except if they read this post. If we were to alter or at least change the behaviours and the attitudes of the way local guides suggest edits, it would be more of a transparent, open collaboration for each. I’ve seen how others deem international addresses to be formatted, and some of them are completely bogus and not even logical. Of course, it all depends on the location and the general nature/language.

6 Likes
11

@Briggs

The personal motivator to wish to monitor once own performance and get insight into trends is what should attract people to wish to participate. Yes, you were proactive and did it for yourself on the sheet you referred to and my guess is that other serious Mappers do this already privately as well.

The project team in partnership with the regional communities are the facilitators to make it easier for participants and offer them the additional benefits of at least global a benchmark. The regional benchmarks very much depend on the number of participants and if I look at my own regional community right now in the Canaries, I would expect two people to be interested…clearly not enough for useful statistic, but then, on the other hand, there are probably enough people interested to participate in Spain to get a national benchmark.

The project team in partnership with the regional communities are the facilitators to make it easier for participants and offer them the additional benefits of at least global benchmarks. The regional benchmarks very much depend of the number of participants and if I look at my own regional community right now in the Canaries, I would expect two people to be interested…clearly not enough for useful statistic, but then, on the other hand, there are probably enough people interested to participate in Spain to get a national benchmark.

There are over 300 communities and I would say that at least a 100 of them seem to be very active and serious about Local guides, add the roughly 200 very active members on this forum we should manage to get 100 people to participate to start with. Once the global benchmark is up, more people will be interested in how they are doing compared to that benchmark and slowly the project should get more people participating. All beginnings are slow and take energy. That’s why we would need the moderators of the LG communities on board to make it happen.

With regards to time investment required. We would need to set a minimum number of edits submitted per participants before they get a personal report. People may go back and add the minimum in one go and there-after use the form to add each edit as they go along in time. Or decide not to continuously monitor their performance, but simply take a one-off test.

With regards to transparency and trust (?) there are different approaches the project could be executed. My initial thought was to have participants sign up first and then simply submit each edit log one by one over time. Alternatively, we could create a tool that people download (Google Sheet) which is for personal monitoring (no link with the global data collection) that optionally auto-feeds in a separate form that people can use to voluntarily bulk feed their data to the global data-bank. I might not be clear here, but once we go ahead with this we are obviously going to all the possible ways of executing the project as a team via Hangouts.

4 Likes
12

@DavidTito

I received an email notification that you posted a reply here, yet it is not visible. Did you delete it? Or did it get filtered by the spam filter?

2 Likes
13

Dear @JeroenM , hope all’s well

That kind of information was available in Map Maker. You could look deeply into the edit history of any feature or user.

To get edit history back, send a feature request. Creating bulk feeds is prohibited.

Personally, we like Maps without edit history, as it’s simpler.

As for trust level, published/denied, correct/incorrect suggested edits: all you can control is your suggested edits.

Raising the quality of your own suggested edits is easy. It’s what Maps Help, LG Connect and Maps & Earth Forum can be used for. If you’re not sure, search or ask. Go the extra mile, help others by answering their questions.

Google probably has multiple benchmark tests already set up and running. :smiley:

Thanks again, JeroenM. Kind regards from Osaka78

5 Likes
14

@JeroenM I deleted it because I was going to add to your thread but I looked a little further into it and this requires a lot of thought from verity of affiliated owners such as Google and feedback from lots individuals as @Osaka78forTRUMP said Mapmarker closed down which was a pity this is a long shot I think, but hey I’m a master at My-maps

kind regards @DavidTito

@JeroenM wrote:

@DavidTito

I received an email notification that you posted a reply here, yet it is not visible. Did you delete it? Or did it get filtered by the spam filter?

2 Likes
15

b @DavidTito and @Osaka78forTRUMP

I must be doing a really poor job in communicating here because based on both your responses I can only conclude that you must have misunderstood the concept.

Calculating your own success rate does not require permission from Google. There is no breaking of rules with regards to Bulk Feeds, the feeds I referred to are related to the research that is disconnected electronically from Maps.

Later today, I shall have time to see if I can explain myself better. For now, I would like to ask you if you monitor your success rate. I know, @Briggs knows based on the sheet he shared the other day. Do you know yours?

So if from every 10 edits you make the only one gets the Not Applied response, your success rate would be 90%.

So simply said, if we all had a Google Sheet to monitor our success rate, the only next step that I am suggesting is to link the results and get a global (and or regional) average figure so that we have a benchmark to compare our own scores with.

@DavidTito

As an LG hero of Google My Maps, I can use your help with something else but will do so outside this thread to keep this on topic.

3 Likes
16

@JeroenM Here is one of my example Google My-maps what would like to do with it?

Example Google My-maps below only

2 Likes
17

[edited - upon request]

18

Dear @JeroenM , hope all’s well. Thanks again for your efforts to look more deeply into the behind the scenes workings of Maps. Thanks for your invitation to participate in the project, we will pass. Kind regards from Osaka78

2 Likes
19

@Osaka78forTRUMP I agree @JeroenM I’m

@Osaka78forTRUMP wrote:

Dear @JeroenM , hope all’s well. Thanks again for your efforts to look more deeply into the behind the scenes workings of Maps. Thanks for your invitation to participate in the project, we will pass. Kind regards from Osaka78

standing with @Osaka78forTRUMP take it all down ASAP

1 Like
20

@Osaka78forTRUMP and @DavidTito

Apologies for using your names for the samples. I could, of course, have said “demo participant 1” and “demo participant 2”.

It is still not clear to me how this kind of data collection would be against the rules, but I shall do my best to find out.

2 Likes