Speech on tolerance -Need of the hour
Answers
Answered by
2
This morning the label over New York City was changed by a malicious contributor via one of our data sources. This hate speech attack is disgusting. Within an hour of discovering the edit, our team deleted and removed it, but the slur was live on our maps.
Hate speech online is intolerable. The internet needs to be safe from hate. Our maps must be safe. Our root cause analysis shows that this malicious attack was immediately detected and put into quarantine for human review. This is where our system failed. While our AI immediately flagged this malicious edit, we experienced a human failure. The micro-tasking system, used for human reviews of suspicious data that is detected from our AI pipeline, has been suspended until we understand why this obvious error was allowed to be overridden and pushed live onto the map.
Our investigation shows this edit to the label of New York City was part of a series of hateful attacks from the same individual. While all other edits from this user were blocked, the New York label edit was mistakenly approved and the AI quarantine was overridden. The malicious contributor responsible is already banned from the platform and we will refer supporting information we have on this and any other hate speech on our platform to the appropriate law enforcement.
Our maps are made from over 130 different sets of data, and we have an AI validation system that reviews all upstream data sources, from satellite imagery to restaurant data we buy to open communities like Wikidata, a global community of data editors; OpenAddresses and local municipalities; Foursquare and a community of local POI enthusiasts; OpenStreetMap and a community of geo-enthusiasts. Groundtruthed data from users on the map is critical.
This is a moment in time in the world where we are compelled to stand, together, against hate. Not just as one company, but together with public and private sector actors to make the internet and our maps a place free from hate and malicious content.
Below is our initial analysis of what took place and our response.
Root Cause Analysis
Mapbox runs a proprietary validation pipeline that reviews all edits from OpenStreetMap and other data sources before they reach the Mapbox map. This pipeline automatically flags suspicious edits, and every single review is approved by at least one human reviewer. On an average day, Mapbox reviews 70,000 map changes and blocks around 50 incidents of vandalism from OpenStreetMap from reaching the Mapbox map.
The process for reviewing map labels begins with an analysis from predictive models trained on known profanity or contentious terms. From here, each review is passed to a human reviewer. Our review tool did mark the label change as significant, since it was a change to a major feature on the map. The review process did not mark the term “Jew” in the edit as an additional risk, given that it is a common word in valid labels.
Mapbox Actions
Mapbox fixed the incident in our map 57 minutes after it was reported by a customer.
Mapbox has verified it is not affected by other offensive acts of vandalism by this user
Mapbox has locked our map update pipeline while we do a full audit of our map validation system, including any manual processes.
Mapbox will be strengthening our process and working directly with our customers on our next actions to prevent attacks from making it to our map.
Timeline of events
Impact
Consumers of Mapbox Streets v7 were exposed to the anti-semitic slur in NYC for a window of 4 hours, beginning at 8/30/2018 8:30 UTC. Due to caching in users devices this issue may persist longer for users that opened a map within that critical window.
Preventing Future Vandalism
Our system design allowed for a human to be a single point of failure. Our next actions are targeted at eliminating any single point of failure with redundant reviews, improving the effectiveness of our reviewers, and improving our metrics to more accurately weight customer impact when defining our internal SLAs.
We will provide concrete details on next actions in the full Root Cause Analysis once completed.
Hate speech online is intolerable. The internet needs to be safe from hate. Our maps must be safe. Our root cause analysis shows that this malicious attack was immediately detected and put into quarantine for human review. This is where our system failed. While our AI immediately flagged this malicious edit, we experienced a human failure. The micro-tasking system, used for human reviews of suspicious data that is detected from our AI pipeline, has been suspended until we understand why this obvious error was allowed to be overridden and pushed live onto the map.
Our investigation shows this edit to the label of New York City was part of a series of hateful attacks from the same individual. While all other edits from this user were blocked, the New York label edit was mistakenly approved and the AI quarantine was overridden. The malicious contributor responsible is already banned from the platform and we will refer supporting information we have on this and any other hate speech on our platform to the appropriate law enforcement.
Our maps are made from over 130 different sets of data, and we have an AI validation system that reviews all upstream data sources, from satellite imagery to restaurant data we buy to open communities like Wikidata, a global community of data editors; OpenAddresses and local municipalities; Foursquare and a community of local POI enthusiasts; OpenStreetMap and a community of geo-enthusiasts. Groundtruthed data from users on the map is critical.
This is a moment in time in the world where we are compelled to stand, together, against hate. Not just as one company, but together with public and private sector actors to make the internet and our maps a place free from hate and malicious content.
Below is our initial analysis of what took place and our response.
Root Cause Analysis
Mapbox runs a proprietary validation pipeline that reviews all edits from OpenStreetMap and other data sources before they reach the Mapbox map. This pipeline automatically flags suspicious edits, and every single review is approved by at least one human reviewer. On an average day, Mapbox reviews 70,000 map changes and blocks around 50 incidents of vandalism from OpenStreetMap from reaching the Mapbox map.
The process for reviewing map labels begins with an analysis from predictive models trained on known profanity or contentious terms. From here, each review is passed to a human reviewer. Our review tool did mark the label change as significant, since it was a change to a major feature on the map. The review process did not mark the term “Jew” in the edit as an additional risk, given that it is a common word in valid labels.
Mapbox Actions
Mapbox fixed the incident in our map 57 minutes after it was reported by a customer.
Mapbox has verified it is not affected by other offensive acts of vandalism by this user
Mapbox has locked our map update pipeline while we do a full audit of our map validation system, including any manual processes.
Mapbox will be strengthening our process and working directly with our customers on our next actions to prevent attacks from making it to our map.
Timeline of events
Impact
Consumers of Mapbox Streets v7 were exposed to the anti-semitic slur in NYC for a window of 4 hours, beginning at 8/30/2018 8:30 UTC. Due to caching in users devices this issue may persist longer for users that opened a map within that critical window.
Preventing Future Vandalism
Our system design allowed for a human to be a single point of failure. Our next actions are targeted at eliminating any single point of failure with redundant reviews, improving the effectiveness of our reviewers, and improving our metrics to more accurately weight customer impact when defining our internal SLAs.
We will provide concrete details on next actions in the full Root Cause Analysis once completed.
Similar questions