Briggs's post
cancel
Showing results for 
Search instead for 
Did you mean: 
Level 10

Overhaul of the editing moderation system

No Personal.png

 

 

It has now been a full month since I had initially posted the series of 'Forever Not Applied', which at this moment in time has become an alternative benchmark to explaining why our edits are 'not applied', with several moderators constantly moving similar posts to it. At this point, we know that it is due to an algorithm shift in an attempt to strike a balance between the spammers of bad data and the genuine local guides who really strive to make a difference. But there has been silence, and no real material solution has been found, and a community of dedicated mappers is growing weary and tiresome of making futile edits.

 

The current state of suggesting genuine, real edits by diligent local guides is extremely unhealthy. Local Guides and their communities are being discouraged to improving the accuracy of their local areas. Some people have voiced their concerns and are even desperate to see change in this process. Like a broken record, I have also complained and vented to my hearts' content, but that has been worn out and now I am actively seeking for change.

 

So in this post, I will be looking at the current moderation process. Given that now 100% of the edits that I suggest over the past week are now being instantly not applied in comparison to my mother's account, I would like to suggest a new route to follow in the form of an idea, that of which I humbly made drastic improvements to my own workplace (before I got fired, that is).

 

The current moderation process is this:

  1. Lisa Gail (LG for short) identifies a need to either add a place or correct the information in an existing listing, and therefore makes the edit.
  2. LG's edit goes through the automated algorithms, based on the trust threshold required of the addition criteria, current local information and/or listing type, and personal trust levels, and will either be approved immediately, reach a pending status or are not applied instantly.
  3. Other Google Maps users via the Check the facts feature or Google staff review LG's edit.
  4. Once approved, LG will receive a notification of a successful edit. Rejections will silently appear into LG's status of edits.

 

It's a simple process, but the billion possibilities of the algorithm behaving the way that it does, as well as the various length of time it takes for Google to review an edit or feedback isn't as efficient as the way it has thought out to be. There is also the premise of having the intermediary of Google staff reviewing the local guide's edit as well, but as Gus has mentioned here and in accordance with the Local Guides Community Guidelines, you should only add/edit/approve/deny an edit if you have first-hand knowledge of the place or you have personally checked that the information is real.

What further mystifies me is how Google staff attains such first-hand knowledge or check an edit's validity if they have not been there themselves. There is a great skew of power between local guides and Google's review process and its algorithms. This argument is however partially defeated by the Google Maps users who rampantly provide bad data, which results in the need to strike the fine balancing act of approvals and rejections. But the premise of having genuine edits immediately not applied is extremely poor form, and a sheer mistake on the algorithm's part, not even giving any sort of breathing space for any human intervention.

A few nights ago, I stumbled upon OpenStreetMap, or OSM for short. Without going into too much information OpenStreetMap is a transparent, community-driven platform for map data, giving its users almost free reign on editing its various features. While I can only imagine that this is a taste of what Google Map Maker was like, I for one actually enjoy using their editing feature and how they've managed it. The tutorial was spunky, there's a public history of edits and changesets that is publicly viewable and traceable, and I think that can efficiently monitor bad edits as well as provide a strong conjoined feeling of accountability towards a community of dedicated mappers and editors, one that Google Maps currently lacks on the client-side.

Of course privacy is a concern, and given the high volume of edits that Google Maps faces on a daily basis from 50 million local guides all across the world, as well as information from third-parties, business owners and databases alike. While it might seem like a long shot, I have therefore put together the following, to improve the moderation process based on the current status of our editing prowess and inspired from several mapping data programs and moderation.

 

The proposed moderation process is this:

  1. Lisa Gail identifies a need to either add a place or correct the information in an existing listing, and therefore makes the edit.
    However, LG can also add a description, comment or explanation of her edit, as the current edit suggestions for Google Maps do not currently provide any space for free-text, except for sending feedback.
  2. LG's edit goes through the automated algorithms, based on the trust threshold required of the addition criteria, current local information and/or listing type, and personal trust levels, and will either be approved immediately, reach a pending status or are not applied instantly.
    Our role as Local Guides is to distinguish users who are providing excellent data and spammers who are littering the maps with awful data, so we will aim for lesser iterations of not applied without an explanation.
  3. LG's edit will appear in a separate dashboard for trusted or a role similar to the legacy regional leads to keep a track of edits, either from a particular user or within a specified area. Timothy Cress (TC for short), a regional lead has access to the dashboard and can diligently keep track of edits, as well as having direct consultation and communication with Google staff to escalate and report problems if necessary. I will detail TC's powers and privileges further in another post.
  4. Other Google Maps users via the Check the facts feature, a regional lead via their dashboard or Google staff review LG's edit. In this case, moderation is slowly decentralised from Google and empowers both the regional lead instead and to an extent other Google Maps users.
  5. Once approved, LG will receive a notification of a successful edit. If TC has personally reviewed the edit and has resulted in either a rejection or revision, a comment or discussion can be optionally fostered between TC and LG along with a notification, to improve future editing through training and feedback. However, if the edit has not filtered through or been reviewed by TC, it will still end up filtering through to Google staff, and such rejections if applicable will silently appear into LG's status of edits.

 

As is it clear and evident that I am quite heavily inspired by OpenStreetMaps, and in a resolve to reduce the hierarchical need for top-heavy decisionmaking and review, I will be continually composing and working towards making Google Maps a better platform, that also encourages the rights and editing powers of genuine local guides, protects map data from corruption and bad edits, and expedites local and regional edits with boundless incentives.

14 comments
Level 10

Re: Overhaul of the editing moderation system

Hi @Briggs this remind me exactly the previous Map Maker environment.

Obviously I agree, although some RL weren't so collaborative and their choice must be done with great care.

Great post, really. One of the best post on this topic I ever read.

Level 10

Overhaul of the editing moderation system (Part 2)

Dashboard

In the proposed moderation process, I mentioned some sort of dashboard in which a hypothetical regional lead-like role such as Timothy Cress (TC) might have exclusive access to. What does this special dashboard represent, and what sort of features might it enable for TC to look at on a broader level? For someone like TC to have the following features, it is presumed that they would have been a major and top contributor of Google Maps, communicated well with the Google team and is also under a strict non-disclosure agreement. Anywho, let's take a look.

 

Possible features of the Regional Lead dashboard

  • Audit trail
    TC would have the access of viewing the entire history of edits that any local guide has made, reviewing any edits where applicable and also flagging any particular oddities or objectionable suggestions to the Google team. TC will be able to see which edits were approved, those that are currently pending, those not applied, and any additional feedback/discussions that may arise.
    For example, if TC identifies that all the spam websites previously reported were suggested by one account, TC would have the ability to either report it to Google or effectively clear all pending suggestions from needing to be checked, and vice-versa for legitimate edits.
  • Revision of edits and viewable edit history
    TC will also have the function to reconsider whether edits approved should not have been approved, and vice-versa to allow places to revert back to original information or endorse genuine edits. This will allow a simpler log of information and identify edits that have a risk of inaccuracy, and simplify the process to suggesting another edit. TC will also be able to monitor and check the edit history of each listing, and also check for any protective sanctions (such as a GMB claim or frequently targeted businesses).
  • Broader 'Check the facts' items
    It is evident that as a local guide, you will not be able to see every single edit that you or other local guides have suggested, and thus are able to check. Personally when I reached Level 9, I was able to see twice as more facts to be checked, including a few of mine I had additionally suggested a few weeks back.
  • Stronger reporting and escalation channels
    TC in direct consultation with Google staff will have quick and prompt responses in terms of objectionable content, locating and deterring spam, and taking action for investigations, especially with irrelevant photos, local requests and GMB-related queries such as fake reviews, reversion software, and other features.

 

Now there's a lot of information right now I've communicated, but how would I facilitate these activities? Of course, it first comes down to selecting the most dedicated local guides who will not defect, and who will uphold their integrity in following the required NDA. There are so many edits that are being looked at on a global scale, how will we distinguish between what we should just approve immediately and also reject immediately? Well, for TC and the algorithms to work together there needs to be further tiers of trust, and also editing difficulties levels as well. Here is what I would like to call risk assessment: the combination of the current trust level of a local guide, and the editing difficulty of their suggestion.

 

Risk Assessment

This feature may or may not already be embedded in our current algorithms, but giving local guides a bit more substance in their current trust levels and their suggestions at least some traction (instead of all of them being not applied unfairly). There are three elements which are illustrated below that make up the holistic risk assessment approach.

The trust matrix for the three elements.The trust matrix for the three elements.

  • Trust level
    The basic measurement of trust, based on the history of the LG's edits, communication with TC and perhaps a "billion" behind-the-scenes factors. This essentially states your status as to what edits you can suggest, the ease of editing you are able to make, rate of response from TC and the trust threshold you are able to edit up to.
  • Editing difficulty
    This focuses on the actual edit you are trying to make, or the place you are trying to add. Whether you are trying to update the name, category, add the operating hours, etc. each of the edits you suggest are sorted into a threshold which is then actioned accordingly depending on the third element. If your edit seems simple enough, like updating the hours then the system is more than happy to classify it as a low difficulty edit. But if your edit is thinking of changing the category of a place from a convenience store to a garbage dump, as what many spammers like to do for no apparent reason, they would be classified as a high difficulty edit.
    To give you a range of what I would classify into each of the difficulty edits, you can check out the spoiler tag below, click to collapse.
    Spoiler
    List of low difficulty edits
    Correcting a spelling mistake, e.g. Chiken V to Chicken V
    Correcting a business name, e.g. Ansell Contractors LLC to Ansell Contractors, or Pathoya - Thai Restaurant
    Changing the category of a restaurant named Maprang Thai Restaurant that's within the name from Restaurant to Thai Restaurant
    Updating the address to remove unnecessary words
    Adding hours to a previously blank listing that seems most appropriate
    Adding a website that clearly has the listing's name in it and can be verified by the host by pinging the website
    Moving a map marker within a Google-Maps defined address boundary (initial edit)

    List of medium difficulty edits
    Adding a place within the parameters of the local area and accuracies, verifiable with evidence and or Streetview data
    Correcting the location descriptor name that are not hotels, e.g. McDonald's Maroubra to McDonalds
    Changing the name of a listing to something that is also possible of having the same category, e.g. rebrands like Shelley's Coffee to Jeren's Espresso Bar
    Changing the category from a broad term to a niche term not evident in the name, e.g. a listing named Haus Lärsen from Restaurant to Bar & Grill
    Adding a website that needs closer inspection due to not being in the URL or a Facebook link, e.g. Steve's Games with a website starterupgames.co.uk
    Moving a map marker within a Google-Maps defined address boundary (subsequent edits)
    Moving a map marker slightly outside a Google-Maps defined address boundary

    List of high difficulty edits
    Adding a place that may overlap or potentially be an existing listing, does not have Streetview data or accompanying evidence
    Changing the name of a listing to something entirely irrelevant or contrasting from the original name, e.g. Carl's Jr. to Garbage Dump, or Atherton Swimming Pool to A, or even Dr. O K Lanmann to RETIREDalthough highly-trusted users shouldn't be making these mistakes
    Changing the name of a listing to something entirely irrelevant to the existing category, e.g. Galaxy Coffee Hunters to iiNet Mobile Telecommunications Business Hub
    Moving a map marker extremely far from a Google-Maps defined address boundary

    Other factors which may increase difficulty
    Reversion edits
    Editing multiple fields at once
    Frequently edited listings
    Suggesting an edit to somewhere you have not passed nearby before
  • Trust Threshold
    You're probably wondering what this element is, since I have already gave examples of editing difficulty. The trust threshold actually attached to the actual listings instead, and signifies the required trust level that LG can suggest to such. There are four thresholds that act independently of the editing difficulty above, and listings can increase or decrease their trust threshold in several ways:

    Low threshold: All users are able to suggest edits to these listings, and include those such as listings created by other users, unclaimed listings that haven't been edited much and potentially poorly-managed GMB listings that have not been either logged into or updated for more than 12 months.
    Medium threshold: Users with a minimum of a medium trust level can suggest edits to GMB listings.
    High threshold: Highly trusted users are able to suggest edits to closely-held GMB listings (who opt in for protective measures as indicated by their account), those who handle bulk updates via a datasheet, and to listings that have been protected due to a history of spam, suspicious targeting or were frequently edited by users.
    Special threshold: These listings are large multi-branch franchises, international subsidiaries, protected listings (such as government agencies) or data provided to Google by third-party organisations such as transport companies with bus stops, universities, indoor-maps owners and privacy-oriented companies, in which only highly trusted users with consultation with the regional lead (or just the regional lead) can suggest edits to.

 

There is a lot of mentioning of a 'regional lead', which we can only hope to be optimistic in the return of such. I don't know if I've just unearthed a large can of worms, but of course there are many other factors that need to be considered, such as regional familiarity, a physical Google presence, privacy legislation, Internet usage, political pressure, the list goes on.

It's getting late and I can't really think of anything else right now, so I'll continue my writings another time.

 

-

 

P.S. Thank you @LucioV! I looked back into Local Guides Connect and the product forums in posts circa late 2016 and I saw a lot of disapproval and reminscence posts about their dismay of scrapping Map Maker and the Regional Lead program, which inspired me to compose these posts. I truly am seeking the success and the improvement of Google Maps and despite what's happening now, I'm going against the urge to vent and instead harnessing it into something that might be useful for a change.

Level 10

Re: Overhaul of the editing moderation system (Part 2)

@Briggs you are amazing.

This is not a FR, this is a complete project executive summary that should land on some Google Maps Program Manager.

If you want to prepare a PDF i will be happy to forward to all PM i know; but the better way should be that @TraciC@LuisRG and @GusMoreira escalate this complete request for evaluation.

Level 10

Re: Overhaul of the editing moderation system (Part 2)

Once again, thank you @LucioV!

 

It's not complete as of yet. I initially reached the post character limit, hence the reason why the post split into two. I'm not finished, there still a lot of information I'm piecing together and considering as well (including language and translations, addresses and collaboration, and some things that staying up late is currently melting my mind right now), but I would be more than happy for any sort of escalation to reach anyone for further consideration.

 

Besides, you did put together my performance review, hehe!

Level 10
Level 10

Re: Overhaul of the editing moderation system (Part 2)

@Briggs One of the best posts i ever read on connect .

Awesome 

Level 10
Level 10

Re: Overhaul of the editing moderation system

@Briggs : Also with trusted percentage , their should be more power . Exactly what i was suggesting in each meetup . Well written buddy

Level 10

Re: Overhaul of the editing moderation system

Level 9

Отговор: Overhaul of the editing moderation system

Благодаря !
Владимир Йосифов, 
https://www.facebook.com/Vitosha.Boulevard.Sofia/posts/1511353062283188

Level 10

Re: Overhaul of the editing moderation system

Wow!!! @Briggs you've made some really good suggestions here.