Instagram Implements An AI System To Fight Mean, Offensive And Harassing Comments

As Wired first reported, the Instagram system follows the social network’s successful use of the tech to fight spam, which began in October of last year. After training the system with human input on how to ID spam, the team liked the results, though they aren’t saying exactly how much its implementation reduced spam versus previous methods.

Based on the success of that system, the team wanted to see if it could tackle an even stickier issue: mean-spirited, hateful and harassing comments. Perhaps you’re familiar with the internet? If so, you might also be aware that it ends up providing the means of delivery for a lot of hurtful invective, hurled about with reckless abandon and little consideration for its ultimate impact.

Contractors trained up DeepText on identifying negative comments and categorizing them into broad segments like “bullying, racism, or sexual harassment,” according to Wired. The raters are said to have analyzed at least two million comments in total prior to today’s go-live for the software, and each of those has been rated at least twice to ensure correct classification.

Once the system goes live, which the report says will happen today, entering hostile comments should result in them just disappearing (though it’ll still show up for the original poster, to keep them from trying to override the filters through repeat attempts). The filter is debuting in English only at first, but all raters employed in the project are at least bilingual, and Instagram is also expanding its spam filter to other languages, which means it’s likely they’ll deploy this tool for more regions over time, too.

Instagram is already a social network a lot of friends go to because it’s friendlier than most other internet social forums; if this system proves effective, it could become even more of a refuge, which is likely good for user stickiness long term.