The problem lies when we need our SFW-only model to deal with NSFWs, since these contents may cross the boundary between clean and unsafe area in too ambiguous a way. The AI systems that manage and moderate content are automatic low-level intelligent in shorts who detect NSFW (Not Safe for Work) images, violent photos or hate speech to violate community guidelines. But the complexity comes when some of these posts, for instance controversy content doesn't simply fit into predefined categories and AI must make nuanced decision whether to flag or block allow it.
How NSFW AI WorkNSFW AI powered by Machine Learning On the probative side, AI models are tasked to train on mountains of data that span millions upon millions -- comprised both by examples acceptable imagery as much as those considered unsuitable —to decode patterns and help identify harmful content. Services like Facebook and Instagram also incorporate AI to scan millions of pieces of content a day for violations of community standards. According to Facebook's Community Standards Report, in 2022 Q1 AI discovered and removed explicit content before users reported nearly all of it: 95 percent. This efficiency showcases the ability of AI to process high volumes of material in real time, however controversial content frequently needs more than just speedy detection but also context.
Context is one of the largest issues when it comes to NSFW AI. The technology is great at identifying outright nudity or gore, but fares poorly when it comes to subtler content like artwork, satire, and educational material. In 2018, Facebook's AI famously mis-identified "The Origin of the World," a painting by Gustave Courbet as inappropriate content Ironically, nudity in art is a basic concept one should have taught this AI over 5 years of development to help it avoid these kinds of blunders. This error goes to show how the lack of contextual understanding in AI can result in over-censoring.
The fact that biased training data will necessarily result in more imbalanced moderation of contentious content. A 2019 report from Georgetown University found that AI moderation systems were in fact far more likely to deem posts NSFW overall, but particularly so when it came to images of Black or plus-size women, as well as those shared by self-identified transgender individuals. These biases are a result of the data out of which AI learns, where a specific perspective or cultural norm might be over-represented. Sam Altman, the CEO of OpenAI — a company getting serious about reframing artificial intelligence with a bias-aware lens to enforce fairness responded “AI should reflect the world’s diversity, not its prejudices.”
NSFW AI is specifically challenged when it comes to the political and socially sensitive content. Twitter and YouTube have each received pushback for either censoring too much or not enough. Take, for instance YouTube during the 2020 U.S. elections — it continued to fail in adequately moderating videos peddling myths about a stolen presidential election – so-called “stolen-election lies”). That share rate was lower than last year, but AI missed over a third of tweet takedowns and 15% of the flagged videos were up long enough taht they could gather views to help spread misinformation. But the question highlights how hard moderating content can be when social and political sensitivities are at play, even for such an advanced AI system.
The processing speed and scalability is the perfect attribute of NSFW AI, especially at a time when people are passing controversial content to each other so rapidly that they go viral within minutes. This is why platforms deploy AI systems for real-time moderation in order to prevent harmful content from going viral. Although Google said YouTube has taken down 94% of content that was flagged before it hit the view count (100-) automated decision can't beat humans in a way; controversial reuse spread faster than the AI catchline context.
To overcome these limitations, companies are using an AI detection-mod protection hybrid model. Using this strategy can also help serve as an interpretational context to material that AI might wrongly understand. Human moderators may be better equipped to make more nuanced decisions, including over controversial posts that need some level of cultural, political or historical context for them. AI is good at being able to flag suspect content on a large scale from the base layer; however in terms of dealing with grey areas and murky waters having human judgment could be an answer.
So to sum up, NSFW AI models can process such controversial content with different levels of success. AI systems, of course, are great at spotting blatant violations of community standards but start to go wobbly on more nuanced or context-dependent material. Despite this, biases in training data and the fact that some of these systems are unable to catch nuances make it difficult for them when they encounter more contentious or controversial content. For more information about how NSFW AI is being used to tackle difficult content issues, see nsfw ai for a look at the growing need of AI in content moderation.