What Is An Appeal?
No decision process for content moderation is 100% accurate. The majority of platforms provide channels for users to request that a decision be reconsidered if they believe that decision is incorrect, unfair, or not aligned with their expectations. These requests are collectively referred to as appeals.
Appeals serve multiple purposes in trust and safety. First and foremost, appeals allow for the flagging and correction of erroneous decisions. Appeals also provide a clear and intuitive way for users affected by decisions to have their voices heard. Lastly, appeals give useful insights into how much users know and understand about policies, which can help improve communication and education about those policies.
Appeals are an incredibly valuable source of information for a number of reasons. If a user appeals, it usually means that they sincerely believe they are not a bad actor and/or their content is not violating the platform rules. Blatant and deliberate bad actors are unlikely to waste their time on appealing, except potentially as an act of protest. Appealers also care enough about the decision that they have gone to the trouble of making an appeal. This means the appeals population leans heavily towards reviews that users feel are likely to be mistakes and are likely to be important to them.
Types of Appeals
The most common type of appeal is a responsible actor appeal. This is when a piece of content, account, or other entity is penalized, and the person responsible for it asks for the decision to be reexamined. This normally corresponds to the owner of the account who created the policy violation, although sometimes the user responsible for a group or area can appeal decisions on violations they did not personally create but which affect their group or area.
Some platforms also allow reporter appeals. These are appeals where the person who reported a potential policy violation objects to the decision made. This is most commonly used when no action is taken, though sometimes an appeal is made when the action taken is less impactful than the reporter expected (i.e.,when content is labeled as disturbing rather than removed completely.)
Not all forms of objection to a decision will lead automatically to an appeals procedure. Disagreement reports allow users to flag a decision that they consider incorrect without suggesting that the decision will be reexamined. Data from these reports can be used in a similar way to appeals to identify policy misalignment and other quality issues without committing resources to a full appeals procedure.
Submitting Appeals
Channels
Appeals can come through a variety of different communication methods. For most platforms, the first source of appeals is a simple messaging system, such as a Contact Us page containing a contact form or email address. This approach has an extremely low setup cost, requiring almost no technical infrastructure or investment and is flexible to a wide variety of problems.
For organizations dealing with appeals at a larger scale, manual reviews of messages like emails quickly become impractical. These types of reviews often require a large amount of effort to resolve, and sometimes require follow up or investigation to establish what is being appealed and for what reason. As a result, most platforms dealing with larger volumes of appeals will introduce a dedicated appeals channel for registering appeals.
Appeals systems often gather requests by sending users a customized link to an appeals channel at the same time as any notifications about policy violations, most often through emails and messaging systems. By linking the appeal request directly to a specific violation notification, most of the critical data to begin an appeal can be sourced automatically and data provided by the user can be standardized for easier processing. Users can also often appeal through dedicated pages or other user areas which display a list of appealable violations. These areas may also allow users to see the current status of any penalties they have received and any appeals they have made.
Submitting an appeal can be as simple as a single click. Adding additional information and checks for the user, such as asking them to read relevant community standards, can reduce the number of low quality appeals. In addition, longer and more complicated appeals processes sometimes act as a hurdle. If an appeals process is very easy (i.e.,clicking a single button labeled Appeal), many more users are likely to attempt appeals, even if the likelihood of success is low.
A more complex process also acts to filter out users for whom the possible value is not worth completing the appeal, reducing overall volumes and required resources. In experiments and focus groups user opinions on such hurdles have historically been mixed. While users often express the desire to communicate issues and provide more detailed context, perceived difficulty of formulating successful appeal arguments and limited belief that such arguments will be listened to or acted upon discourage engagement with more complex processes.
Information Collection
One consideration with appeals is how much information to collect from the appealer. Some appeals processes accept only the appeal request and some basic automatically generated information about the appealer. This makes the appeals process faster and simpler, but it also greatly limits the ability of the user to explain context or highlight specific errors.
Some appeals processes use large freeform fields where the appealer can provide information and context. This information can be useful both for making the right appeal decision and for information on responsiveness and accuracy of appeals. It can be used to generate substantial insights into both the appeals and enforcement processes.
However, more information can substantially increase the complexity of the appeals process. Freeform fields could contain any information the appealer can type, including false or misleading information. An open text entry can also create a risk of legal liability if important information is being sent to a communications channel not set up to deal with it correctly. Open appeals procedures can also be used to send abusive and threatening communications to employees from users whose content has been penalized.
Processing Appeals
Reviewing appeals requests is usually handled in a similar way to reviewing the original violation, and it is common for the same basic tools, systems, and structures to be used. This helps maintain a level of consistency between the original reviews and appeal reviews.
Consistency is particularly important in appeals because if there is a large mismatch between appeals and moderation then penalties can be repeatedly restored and revoked as the two processes disagree. Low levels of consistency between moderation and appeals can undermine the credibility of both processes and leads to an extremely poor user experience.
Appeals reviews can sometimes be designed to mimic the original review as closely as possible, with little to no additional information about the violation. This approach increases consistency between appeals and moderation processes, but it also increases the likelihood that individual appeals may be incorrect if an incorrect decision was caused by that original process.
Other appeals systems add additional information and processes related to appeals specifically, such as context from users, the history of the appellant, and any policy changes or problems that might justify a reevaluation. Maintaining additional information and the guidelines and processes on how to use that information consistently can require significant additional resources.
Prioritizing Appeals
Just like other decisions made in trust and safety, appeals reviews can be given different levels of priority based on the impact that decisions in those review systems are likely to have. Because appeals are intended to restore non-violating behavior, the priority given to appeals is unlikely to directly affect many of the issues with the highest risks of real world physical harm. However, many appeals are likely to have significant personal and financial effects on those affected, particularly for businesses whose operations are blocked or severely hampered by penalties.
Automating Appeals
Just like more general content review decisions, appeals decisions can be handled using automation. However, because appeals act as a safety net for errors in other parts of trust and safety operations, automation can introduce additional risk. Appeals also usually occur at much lower volumes than general moderation, reducing the amount of training data available for automated systems. As a result, automation in appeals is often limited to prioritization, decisions with the lowest user impact, and rejections of users who have abused the appeals system.
Aside from automated decisions, sometimes appeals are rejected automatically without a decision for other rule based reasons. Examples of this are rejection of appeals that are beyond a certain age, where the decision was already corrected, or where other more severe penalties make the appeal irrelevant.
Repeated Appeals
One important decision with appeals systems is how many times a user should be allowed to appeal. Just as there is a non-zero chance that moderation will make the wrong initial decision, there is also a non-zero chance that appeals will make the wrong decision, especially if both moderation and appeals use the same reviewers and resources to make decisions.
This creates two significant problems. If appeals are limited, it is more likely that an appeal may be unfairly rejected. On the other hand, an unlimited number of appeals drains resources by evaluating the same cases over and over and increases the likelihood that violations will be incorrectly restored.
In some situations appeals are submitted for content that was originally violating but has since been corrected. An example where this is common is webspam in search engines; pages with no value designed to manipulate search engine rankings. Because a website can be edited, replaced, or sold in its entirety, this raises the question of whether or not the site should stay banned after violations are corrected, especially if violating content can be added back in as soon as an appeal succeeds.
Policy Updates and Appeals
Policies in trust and safety are regularly updated in response to changing user behavior and new issues arising. In appeals this creates an unusual situation where the relevant policy changes between a suspected violation being actioned and an appeal being made for that violation.
In general, historical violations are rarely re-reviewed automatically to remove penalties after a policy change as this would usually involve a prohibitively large number of rechecks. However, when an appeal is made it can be evaluated under the new policy guidelines. This prevents the need to maintain training in historical versions of policy. This approach means that policy changes can cause large spikes in volumes and metrics related to appeals, so communicating such changes proactively and clearly is important.
Other Channels for Appeals
Oversight and Independent Review
Appeals processes are generally run internally through platforms, often through the same teams responsible for evaluating policy decisions. Some platforms will also have support from independent teams or organizations who are responsible for providing an outside perspective.
External reviews tend to have limited resources, and as a result often focus on a small number of the highest impact decisions. Examples include decisions around major newsworthy events, addressing concerns of governmental and industry regulators, and where highly charged or political issues are involved.
One heavily publicized example of a group providing independent review is the Oversight Board, which reviews and makes precedent setting decisions for Meta. This board is managed by a group of trustees and at time of writing board members include prominent academics, senior figures in non-profit organizations, and former members of government. A board like this is clearly not scalable, but can provide a highly detailed review for a limited number of incidents and issues.
Groups such as this are not limited to examining whether a given decision is correct. Various other issues related to operations such as speed of response and transparency may also potentially be subject to review. For example, the Oversight Board has on multiple occasions examined decisions that Meta had corrected to look at both the original and final decision, and to identify process issues. The board has also provided more general feedback around operations, such as highlighting the issue of transparency around the use of automated content moderation.
Article: Oversight Board Case 2020-004-IG-UA (Post about breast cancer removed by automation)
Article: Oversight Board Case 2021-014-FB-UA (Post about atrocity allegations in Ethiopia)
High Profile Events
Occasionally decisions get significant attention from the general public. A decision may have a high level of public interest, the affected party may have an audience such as a politician or celebrity, or the decision may be particularly controversial, egregious or newsworthy in some way.
In such situations, it is often inevitable that platforms will become aware of the issue through media directly or through media requests. When such attention is focused on a single decision, it is common for that decision to be proactively checked and corrected. High profile examples of a more widespread issue are also often proactively rechecked in this manner, particularly when they are heavily featured in news or social media.
| In-depth Look: News coverage and response to disturbing content targeted at children on Youtube In 2017 the subreddit r/ElsaGate was created on social media site Reddit to investigate and share examples of inappropriate content on Youtube. This content targeted children, but also contained disturbing and inappropriate content such as gore, violence, sex, and fetishized behavior. The name came from the common and prominent usage of the character Elsa from the movie Frozen in such videos, along with other characters popular with children such as Peppa Pig and Spiderman. Further attention was drawn to the issue by a post from James Bridle titled ”Something is wrong on the internet,” highlighting the issue and a variety of examples. In particular the post provided details on the channel Toy Freaks which at the time was one of the 100 most viewed channels on Youtube. Both the issue and Toy Freaks as an example were then reported on by multiple news outlets. Multiple users stated that they had reported Toy Freaks, but that no action had been taken. Youtube responded by removing the Toy Freaks channel for violating policy, along with a wide scale crackdown on Elsagate content on the site and an announcement from Youtube CEO Susan Wojcicki of increases in the number of human reviewers and in the use of automation to tackle this and other issues. Case Study: YouTube deals with disturbing content disguised as videos for kids (2017) – Trust and Safety Foundation Project |
Direct Contact
Sometimes affected users will make direct contact with platform employees in an attempt to reverse a decision. For partners and larger customers in particular, this can often happen through existing relationships such as account managers. Staff in such circumstances may be in a position to help guide users through the main appeals process, or escalate if this is not effective. Posts on official forums or messages to dedicated platform accounts are also occasionally used to reach out.
Other times contact is made directly to individuals, through personal relationships or even attempted random contacts with staff found through social media. This is rarely effective. Even if someone is willing to help through such contact, most employees at platforms are not directly involved in trust and safety and may not have helpful information about resolving the problem. Many platforms also have ethical restrictions on making or influencing decisions for users that staff have communicated with personally, or that restrict sharing of information due to rules around external disclosure. In addition, because internal escalation processes are not public facing or heavily used, they are often less fully developed and may be significantly slower and less reliable than user facing appeals systems.
