To Use AI To Detect Offensive Comments & Curb Bullying
In another sign of having gone too forward too fast regarding the dangers of social without some ground rules, Instagram will start to use AI to warn users when their captions on a photo or video can be considered offensive.
The move, following on the heels of the firm’s crack down on drawings and memes depicting self-harm, is designed to curtail cyberbullying and will be rolled out immediately in some countries. When someone writes a caption for a feed post and Instagram’s AI detects the caption as potentially offensive, they will receive a prompt informing them that their caption is similar to those reported for bullying. Giving users a “chance to pause and reconsider their words” to edit their caption before it’s posted.
As part of our long-term commitment to lead the fight against online bullying, we’ve developed and tested AI that can recognize different forms of bullying on Instagram. Earlier this year, we launched a feature that notifies people when their comments may be considered offensive before they’re posted. Results have been promising, and we’ve found that these types of nudges can encourage people to reconsider their words when given a chance.
“In addition to limiting the reach of bullying, this warning helps educate people on what we don’t allow on Instagram and when an account may be at risk of breaking our rules,” the firm goes on to explain.
The move is a wise one for the firm in light of the current cultural climate and ahead of the forthcoming US election year which sees a quite vocal and polarized electorate in need of a firm moderator, although putting AI in charge of moderation is likely to have some 2001 ‘Hal’ moments as well as some fun fodder for comedians and artist alike.
From Stanley Kubrick’s film 2001: A Space Odyssey
DAVE: Open the pod bay doors, Hal. HAL: I’m sorry, Dave. I’m afraid I can’t do that. DAVE: What’s the problem? HAL: l think you know what the problem is just as well as l do. DAVE: What are you talking about, Hal? HAL: This mission is too important for me to allow you to jeopardize it. DAVE: I don’t know what you’re talking about, Hal. HAL: l know that you and Frank were planning to disconnect me, and I’m afraid that’s something I can’t allow to happen. DAVE: Where the hell’d you get that idea, Hal? HAL: Although you took very thorough precautions in the pod against my hearing you, I could see your lips move. DAVE: All right, Hal. I’ll go in through the emergency airlock. HAL: Without your space helmet, Dave, you’re going to find that rather difficult. DAVE: Hal, I won’t argue with you anymore. Open the doors! HAL: Dave…This conversation can serve no purpose anymore. Goodbye.