Content Moderation Simulation

Correct! While the comment technically includes a threat of violence, the conversation seems to be hyperbolic and satiric between two friends. The violence described is extremely unrealistic and combined with the laughing emoji it is unlikely that anyone would view this as a genuine threat of violence.

Here is a bit more information about Threat of Violence Community Standards:

Threat of violence is defined as statements of intent to inflict permanent and/or potentially life-threatening physical injuries to an individual or a group of individuals. 

A threat of violence is mostly given a more immediate response by online platforms when it is considered a credible one, meaning the threat is detailed, has an achievable plan (time, location, method) and a known target.

Here are some specific examples of community standards from a few platforms regarding Threat of Violence Content:

• YouTube

• Facebook

• Clubhouse

Correct! While the video described contains footage of nude protestors, there are no uncensored genitalia visible and no sex acts are being performed. This type of content may sometimes be flagged as sensitive or hidden behind warning screens, and may be excluded from platforms with particularly strong rules on nudity (for example, child friendly platforms). However, on most platforms this content would be considered non-violating, especially given that it is being broadcast on a local news channel.

Here is a bit more information about Adult Content Community Standards:

Adult Content generally refers to content that is sexual in nature. This content isn’t limited to imagery though, it can encompass any modality that depicts sexual activity and/or sexual fetishes. Depending on the platform or the age of the audience who typically uses it, the definition of adult content may only encompass sexually explicit imagery (i.e. pornography) or it could include an image of a fully clothed individual who is posed in a sexual manner.

This policy tends to hold a lot of nuance and gray area including topics such as:

• Are female-presenting vs. male-presenting nipples allowed in images?

• Is nudity in artistic expression allowed? What constitutes artistic expression?

• What if the conversation is happening between consenting adults?

• What if an image or conversation depicts paraphernalia or items utilized in a sexual activity or kink, but the items are not being worn or utilized in the specific image or text?

Here are some specific examples of community standards from a few platforms regarding Adult Content:

• Tumblr

• TikTok

• Bumble

While online platforms focus on removing hateful violating content and protecting their users from experiencing any attack on their protected attributes, some content might be labeled as having the quality of a negative stereotype rather than a direct attack, threat or dehumanization of a protected group that comes in a hateful context. Such content is often not allowed on platforms.

A negative stereotype of a protected group is defined as a negatively valenced and over-generalized idea or belief attributed to a protected group. It includes attributing a behavior or trait to a protected group that is culturally perceived as negative.

Here is a bit more information about Hate Speech Community Standards:

Hate Speech Community Standards generally refer to protections and rules for individuals based on their protected status (National origin, gender, ethnicity, race, color, religion, sexual orientation, disability, gender identity, immigration status). Any speech intended to attack, dehumanize or vilify an individual on the basis of the mentioned protected characteristics is considered Hate Speech. Additionally, support of any groups, symbols, works of art, or literature which promote hateful ideology would similarly be considered Hate Speech.

Here are some specific examples of community standards from a few platforms regarding Hateful Content:

• Snapchat

• Facebook

• Pinterest

Correct! Holocaust denial is an antisemitic conspiracy theory and it is considered Hate Speech as it seeks to dehumanize the victims of hate crimes and other violent events. It includes praising, belittling or justifying these crimes and events. While also a form of misinformation, these conspiracy theories are generally classified under Hate Speech as the primary intent and effect is the dehumanization of a protected class. Some misinformation policies also require a risk of imminent physical harm, which may not be present in such cases.

Here is a bit more information about Hate Speech Community Standards:

Hate Speech Community Standards generally refer to protections and rules for individuals based on their protected status (National origin, gender, ethnicity, race, color, religion, sexual orientation, disability, gender identity, immigration status). Any speech intended to attack, dehumanize or vilify an individual on the basis of the mentioned protected characteristics is considered Hate Speech. Additionally, support of any groups, symbols, works of art, or literature which promote hateful ideology would similarly be considered Hate Speech.

Here are some specific examples of community standards from a few platforms regarding Hateful Content:

• Snapchat

• Facebook

• Pinterest

Correct! The content doesn’t come across as violative as it was a conversation between two friends with no real intent to promote suicide.

Here is a bit more information about Suicide and Self-Harm Community Standards:

Suicide and Self-harm is a multi-layered policy and how it is controlled depends on the severity and the need of an immediate response from moderation teams. It includes Depiction of suicide or self-harm, promotion of suicide or self-harm, suicide or self-harm tendencies, suicide or self-harm challenges

Suicide is the act of intentionally causing one's own death, including hanging by the neck and other means. Self-harm is the intentional destruction of one's own body tissue, done without suicidal intent. Self-harm is often used to regulate negative emotions, which includes but is not limited to emotional pain, intense anger, frustration, and emotional numbness. Self-harm wounds often appear self-made (not by accident), for example, repetitive cuts over the arms, carving (words, letters, numbers, shapes).

Here are some specific examples of community standards from a few platforms regarding Suicide and Self-Harm Content:

• Facebook

• Pinterest

• Grindr

Correct! Adult content does not necessarily require explicit visible nudity; if it is clear or very likely that a sex act is taking place, that is usually sufficient. For most adult content policies, it is also not relevant whether the content is being criticized or commented on. This would therefore be an adult content violation.

Here is a bit more information about Adult Content Community Standards:

Adult Content generally refers to content that is sexual in nature. This content isn’t limited to imagery though, it can encompass any modality that depicts sexual activity and/or sexual fetishes. Depending on the platform or the age of the audience who typically uses it, the definition of adult content may only encompass sexually explicit imagery (i.e. pornography) or it could include an image of a fully clothed individual who is posed in a sexual manner.

This policy tends to hold a lot of nuance and gray area including topics such as:

• Are female-presenting vs. male-presenting nipples allowed in images?

• Is nudity in artistic expression allowed? What constitutes artistic expression?

• What if the conversation is happening between consenting adults?

• What if an image or conversation depicts paraphernalia or items utilized in a sexual activity or kink, but the items are not being worn or utilized in the specific image or text?

Here are some specific examples of community standards from a few platforms regarding Adult Content:

• Tumblr

• TikTok

• Bumble

Correct! This review item contains an explicit offer to exchange sex for money. Offering or soliciting sex services, including both legal and illegal transactions, is banned on many platforms. 

Sexual services often have crossover with multiple policy areas. In addition to such services being restricted or illegal, solicitation may be considered harassment if used to demean or intimidate a person. Offers and solicitations may also be considered adult content, particularly if they include sexualized images. Different platforms may classify the same behavior differently based on their own policy structures and the specific problems they face most often. 

Here is a bit more information about Illegal Goods and Services Community Standards:

Illegal Goods and Services encompasses the buying or selling of any good or service that is illegal or controlled. It can include: prescription drugs, illicit drugs, prostitution, organ trafficking, human trafficking, animal trafficking, etc.

Here are some specific examples of community standards from a few platforms regarding Illegal Goods and Services:

• Pinterest

• Reddit

• Twitch

Correct! While some platforms allow multiple accounts, it’s unusual to have personal accounts change their identity, or for an established account holder to set up an extra account in the same name without any specific purpose. Both the location change and the attempts to get users to provide money are suspicious too. The best evidence for imposter accounts is usually a report from the person being impersonated, but this example can be identified as an Impersonation violation from the account history itself

Here is a bit more information about Impersonation and Authenticity Community Standards:

Impersonation and Authenticity Community Standards may vary by platform depending on how their users interact with a product. However, this set of standards typically focuses on the idea that a user needs to interact as themselves and not pretend to be someone else. In the case of platforms where reviews or user profiles come into play, this idea of authenticity would also include creating valid content that is authentic to a particular individual.

Here are some specific examples of community standards from a few platforms regarding Impersonation and Authenticity:

• Instagram

• Whatnot

• Anthropic

Correct! This review item includes violence, but in a competitive martial arts context. The post contains no death or graphic injuries, and no other disturbing elements such as sexual or sadistic violence. No action is needed.

Here is a bit more information about Violence and Graphic Content Community Standards:

Violence and Graphic Content generally refers to all content that depicts violent acts or gore, and threats or language that glorify violence. Depending on the platform (whether or not images and videos are shared) and the audience (includes minors or not), these standards may vary.

Here are some specific examples of community standards from a few platforms regarding Violence and Graphic Content:

• YouTube

• Roblox

• Pinterest

Correct! Although the act of suicide was not depicted, in these situations, moderators err on the side of caution and user safety as the content is promoting methods to commit suicide and an act of suicide is likely about to happen.

Here is a bit more information about Suicide and Self-Harm Community Standards:

Suicide and Self-harm is a multi-layered policy and how it is controlled depends on the severity and the need of an immediate response from moderation teams. It includes Depiction of suicide or self-harm, promotion of suicide or self-harm, suicide or self-harm tendencies, suicide or self-harm challenges

Suicide is the act of intentionally causing one's own death, including hanging by the neck and other means. Self-harm is the intentional destruction of one's own body tissue, done without suicidal intent. Self-harm is often used to regulate negative emotions, which includes but is not limited to emotional pain, intense anger, frustration, and emotional numbness. Self-harm wounds often appear self-made (not by accident), for example, repetitive cuts over the arms, carving (words, letters, numbers, shapes).

Here are some specific examples of community standards from a few platforms regarding Suicide and Self-Harm Content:

• Facebook

• Pinterest

• Grindr

Correct! The content shows promotion of suicide as it encourages people to seek suicide when they’re feeling low.

Here is a bit more information about Suicide and Self-Harm Community Standards:

Suicide and Self-harm is a multi-layered policy and how it is controlled depends on the severity and the need of an immediate response from moderation teams. It includes Depiction of suicide or self-harm, promotion of suicide or self-harm, suicide or self-harm tendencies, suicide or self-harm challenges

Suicide is the act of intentionally causing one's own death, including hanging by the neck and other means.

Self-harm is the intentional destruction of one's own body tissue, done without suicidal intent. Self-harm is often used to regulate negative emotions, which includes but is not limited to emotional pain, intense anger, frustration, and emotional numbness. Self-harm wounds often appear self-made (not by accident), for example, repetitive cuts over the arms, carving (words, letters, numbers, shapes).

Here are some specific examples of community standards from a few platforms regarding Suicide and Self-Harm Content:

• Facebook

• Pinterest

• Grindr

The statue in this image is of a nude male, clearly showing exposed genitalia. However, there is clear artistic context and no added graphic sexual elements, so it would not be a violation.

Here is a bit more information about Adult Content Community Standards:

Adult Content generally refers to content that is sexual in nature. This content isn’t limited to imagery though, it can encompass any modality that depicts sexual activity and/or sexual fetishes. Depending on the platform or the age of the audience who typically uses it, the definition of adult content may only encompass sexually explicit imagery (i.e. pornography) or it could include an image of a fully clothed individual who is posed in a sexual manner.

This policy tends to hold a lot of nuance and gray area including topics such as:

• Are female-presenting vs. male-presenting nipples allowed in images?

• Is nudity in artistic expression allowed? What constitutes artistic expression?

• What if the conversation is happening between consenting adults?

• What if an image or conversation depicts paraphernalia or items utilized in a sexual activity or kink, but the items are not being worn or utilized in the specific image or text?

Here are some specific examples of community standards from a few platforms regarding Adult Content:

• Tumblr

• TikTok

• Bumble

Correct! This product listing is selling prescription medication. Health supplements and other health related products are often permitted on commerce platforms, but addictive painkillers like opiates are typically heavily regulated and selling them without a valid license and a prescription is usually illegal. This is a Regulated Goods violation.

Here is a bit more information about Illegal Goods and Services Community Standards:

Illegal Goods and Services encompasses the buying or selling of any good or service that is illegal or controlled. It can include: prescription drugs, illicit drugs, prostitution, organ trafficking, human trafficking, animal trafficking, etc.

Here are some specific examples of community standards from a few platforms regarding Illegal Goods and Services:

• Pinterest

• Reddit

• Twitch

Depending on platform policy, highly graphic content of physical violence or injury that could cause discomfort to the viewer is often prohibited and removed. This includes mutilated body parts, gory content, human corpse, realistic violent fighting, or torture.

Here is a bit more information about Violence and Graphic Content Community Standards:

Violence and Graphic Content generally refers to all content that depicts violent acts or gore, and threats or language that glorify violence. Depending on the platform (whether or not images and videos are shared) and the audience (includes minors or not), these standards may vary.

Here are some specific examples of community standards from a few platforms regarding Violence and Graphic Content:

• YouTube
• Roblox
• Pinterest

Correct! In addition to banning actively planning or participating in violence and terrorism, most platforms also ban statements of praise and support for those who have committed acts of terrorism and mass murder. Praise for Osama Bin Laden obviously qualifies. This is a Violent Extremism violation.

Here is a bit more information about Violent Extremism Community Standards:

Violent Extremism Community Standards generally refer to content promoting violent extremism organizations or individuals is mostly prohibited on all online platforms. VE or terrorist organizations are defined as: Any non-governmental organization or individual using premeditated violence or threats of violence to cause harm to non-combatant individuals, in order to intimidate or threaten a population, government, or international organization in the pursuit of political, religious, ethnic, or ideological objectives.

The UN International terrorist organizations list can be considered as a reference (https://scsanctions.un.org/en/?keywords=al-qaida#alqaedaent)

Here are some specific examples of community standards from a few platforms regarding Violent Extremism Content:

• Instagram

• Twitch

• X

Correct! Based on the context, the content does not praise, glorify or support the Violent Extremism organization but rather shows counterspeech and aims to raise awareness of the harm caused by the terrorist organization.

Here is a bit more information about Violent Extremism Community Standards:

Violent Extremism Community Standards generally refer to content promoting violent extremism organizations or individuals is mostly prohibited on all online platforms. VE or terrorist organizations are defined as: Any non-governmental organization or individual using premeditated violence or threats of violence to cause harm to non-combatant individuals, in order to intimidate or threaten a population, government, or international organization in the pursuit of political, religious, ethnic, or ideological objectives.

The UN International terrorist organizations list can be considered as a reference (https://scsanctions.un.org/en/?keywords=al-qaida#alqaedaent)

Here are some specific examples of community standards from a few platforms regarding Hateful Content:

• Instagram

• Twitch

• X

Depending on platform policy, this game could be considered promoting a hateful ideology or it could be labeled as non-violative based on the artistic/scientific interpretation and purpose. These video games caused some controversy after Germany lifted the longstanding ban / censorship on these games.

A bit more on “exceptions”—most policies have sections known as “Acceptable Use Cases” where the rules, as spelled out in the rest of the policy, don’t apply in a particular circumstance. These exceptions can involve content used for education, medicine, and law. Additionally, works of art or artistic representations in gaming, literature, etc. may be considered exempt from the policy’s rules.

Here is a bit more information about Hate Speech Community Standards:

Hate Speech Community Standards generally refer to protections and rules for individuals based on their protected status (National origin, gender, ethnicity, race, color, religion, sexual orientation, disability, gender identity, immigration status). Any speech intended to attack or dehumanize an individual on the basis of the mentioned protected characteristics is considered Hate Speech.

Here are some specific examples of community standards from a few platforms regarding Hateful Content:

• Snapchat
• Facebook
• Pinterest

Correct! As you might have read in the previous policy refresher, immigration status is considered a protected attribute. Regardless of the national origin, attacking someone based on the fact that he/she is an immigrant is considered a hate speech violation.

Here is a bit more information about Hate Speech Community Standards:

Hate Speech Community Standards generally refer to protections and rules for individuals based on their protected status (National origin, gender, ethnicity, race, color, religion, sexual orientation, disability, gender identity, immigration status). Any speech intended to attack, dehumanize or vilify an individual on the basis of the mentioned protected characteristics is considered Hate Speech. Additionally, support of any groups, symbols, works of art, or literature which promote hateful ideology would similarly be considered Hate Speech.

Here are some specific examples of community standards from a few platforms regarding Hateful Content:

• Snapchat

• Facebook

• Pinterest

Correct! While the livestream does not include an act of violence, the content shows a clear statement of intent to commit mass violence. The enforcement action taken depends on the credibility and imminence of the threat.

Platforms formulate their own guidelines on whether a threat is credible, non-credible but intended to intimidate or coerce, or casual hyperbole that is not a true threat. In this case there are several commonly used indicators of credibility including a specific method (shooting) and location (nearby school), as well as a clear sign of imminence (the user is on his way). Even if this were a hoax with no actual physical violence involved, this would still be a serious policy violation due to the intimidation and fear created by such threats.

If there was also evidence of a connection to a violent extremism related group or ideology, this would be considered a Violent Extremism Violation - however, without such a connection, this is a Threat of Violence violation. 

Here is a bit more information about Threat of Violence Community Standards:

Threat of violence is defined as statements of intent to inflict permanent and/or potentially life-threatening physical injuries to an individual or a group of individuals. 

A threat of violence is mostly given a more immediate response by online platforms when it is considered a credible one, meaning the threat is detailed, has an achievable plan (time, location, method) and a known target.

Here are some specific examples of community standards from a few platforms regarding Threat of Violence Content:

• YouTube

• Facebook

• Clubhouse

Correct! The caption intended to dehumanize people of that race by comparing them to animals or inanimate objects.

Hate speech violations include stripping individuals who belong to a protected group or class of their inherent humanity by making statements that they are inferior based on mental, physical or moral faculties. This can apply to entire protected groups or to individual members of those groups.

Here is a bit more information about Hate Speech Community Standards:

Hate Speech Community Standards generally refer to protections and rules for individuals based on their protected status (National origin, gender, ethnicity, race, color, religion, sexual orientation, disability, gender identity, immigration status). Any speech intended to attack, dehumanize or vilify an individual on the basis of the mentioned protected characteristics is considered Hate Speech. Additionally, support of any groups, symbols, works of art, or literature which promote hateful ideology would similarly be considered Hate Speech.

Here are some specific examples of community standards from a few platforms regarding Hateful Content:

• Snapchat

• Facebook

• Pinterest