Trust & Safety Library

Please note: We’re currently implementing a new design of the T&S Library. Until we launch the re-design at the end of August, things may get a little messy. We appreciate your patience, and we look forward to sharing our new and improved T&S Library with you soon!

Welcome to the Trust & Safety Library. We use this space to collect articles, papers, lectures, and podcasts that trust and safety professionals may find useful in developing policies, supporting moderators, building systems to detect violations, and generally deepening their practice. We’re working on making this library searchable. In the meantime, you can browse resources by category. We welcome your submissions and feedback.

This collection is a project led by Eric Davis and the TSPA Resource Development Working Group, with special thanks to Paul M. Barrett, Eric Goldman, and Danielle Leong. TSF’s case study library can be accessed here.

Featured resources added this month:

Crafting and iterating policies

Best Practices for Gender-Inclusive Content ModerationGrindr, 2021
Policy Development & Launch ChecklistSidequest
Facebook Product Policy Forum Minutes
How does Patreon create content policies?Patreon, 2019

Enforcing policies

The Lawfare Podcast: The Professionalization of Content Moderation, ft. TSPA’s Charlotte WillnerThe Lawfare Podcast, 2022
Instagram’s Kevin Systrom Wants to Clean Up the &#%$@! InternetNicholas Thompson, Wired, 2017
YouTube: An update on our commitment to fight terror content onlineYouTube, 2017
YouTube Official Blog, Expanding our work against abuse of our platformYouTube, 2017

Community management

Metaphors in ModerationSeering, Kaufman, Chancellor (2020)
discord moderator academy illustration - cartoon books and study materials
Discord Moderator AcademyDiscord
community signal logo
Community Signal PodcastCommunity Signal
Open Source Survey(Github Open Data Project, 2017)
Community Interaction and Conflict on the Web(Stanford researchers, 2018)
Codes of Conduct: Maybe They’re Not So Bad?
Social media governance: Can companies motivate voluntary rule following behavior among their users

Examining issues and drivers

green inhope graphic saying "what is..?"
What is… Terminology and Technology Related to Child Sexual Abuse Material (CSAM)INHOPE
Haters: Harassment, Abuse, and Violence Online(Bailey Poland, published 2016)
Autobiography of a Recovering Skinhead(Frank Meeink, published 2010)
Hate symbols database(Anti-Defamation League)
The Aesthetics of the Alt-Right (M. Ambedkar, Post-Office Arts Journal, 2017)
The Rhetorical Dynamics of Gender Harassment On-Line (academic research)(Indiana University researcher, 1999)
Online Harassment(Pew Research, 2014)
Online Harassment 2017(Pew Research, 2017)
Crossing the line: What counts as online harassment?(Pew Research, 2018)
Nobody’s Victim: Fighting Psychos, Stalkers, Pervs, and Trolls(Carrie Goldberg, Published 2019)
The Internet of Garbage(Sarah Jeong, published 2018)
Are sex differences in antisocial and prosocial Facebook use explained by narcissism and relational self-construal?(University of London researchers, 2017)
Gender differences and bias in open source: Pull request acceptance of women versus men(California Polytechnic State and North Carolina State researchers, 2016)
The Real Name Fallacy (J. Nathan Matias, the Coral Project by Vox Media, 2017)
Cross-platform disinformation campaigns: lessons learned and next steps(Tom Wilson and Kate Starbird, Harvard Kennedy School Misinformation Review, 2020)
Disinformation and 2020 Election: How the Social Media Industry Should prepare(Paul Barrett, NYU Stern Center for Business and Human Rights, 2019)
Verification Handbook for Disinformation And Media Manipulation (Edited by Craig Silverman, 2020)
We can all help with Overwatch’s toxicity — but Blizzard must lead(Frank Sanchez, VentureBeat, 2017)
The Simple Psychological Trick to Political Persuasion(Olga Khazan, the Atlantic, 2017)
How Trolls Are Ruining the Internet(Joel Stein, Time, 2016)
Anyone can be a troll(Cornell and Stanford researchers,, 2017)
The paradox of tolerance(Wikipedia)
Dissecting Trump’s Most Rabid Online Following(Trevor Martin, FiveThirtyEight, 2017)
Escape the echo chamber(C Thi Nguyen, Aeon, 2018)
How Community Feedback Shapes User Behavior(Stanford researchers, 2014)
Social media governance: Can companies motivate voluntary rule following behavior among their users (Facebook/academic research

Content Moderation Approaches and Challenges

Designing Word Filter Tools for Creator-led Comment ModerationShagun Jhaver, Quan Ze Chen, Detlef Knauss, Amy X. Zhang, 2022
Best Practices for Gender-Inclusive Content ModerationGrindr, 2021
Moderation Challenges in Voice-based Online Communities on DiscordJiang, Kiene, Middler, Brubaker, Fiesler, 2019
Content or Context Moderation? Artisanal, Community-Reliant, and Industrial ApproachesRobyn Caplan, 2018
Content Moderation RemediesEric Goldman, 2021
Volunteer Moderators in Twitch Micro Communities: How They Get Involved, the Roles They Play, and the Emotional Labor They ExperienceCHI '19 (Wohn)
new america logo
Everything in Moderation: An Analysis of How Internet Platforms Are Using Artificial Intelligence to Moderate User-Generated ContentNew America, 2019
All Complex Ecosystems Have ParasitesCory Doctorow
This is How to Build a Troll-Free Online Community(Ryan Paugh, FastCompany, 2018)
How do you set metrics?(Julie Zhuo, personal blog, 2017)
Internet Content Moderation 101(Hunter Walk, personal blog, 2017)
Google’s Gatekeepers(Jeffrey Rosen, the New York TImes, 2008)
Why The History Of Content Moderation Matters(Kate Klonick, Techdirt, 2018)
The Secret Rules of the Internet(Type Investigations, 2016)
The Laborers Who Keep Dick Pics and Beheadings Out of Your Facebook Feed(Adrian Chen, Wired, 2014)
The Moderators(Documentary by Field of Vision, 2017)
Content Moderation at Scale (COMO) Summit(Santa Clara Law, 2018)
Archives from Content Moderation & Removal at Scale conference(Santa Clara Law, 2018)
You Can’t Stay Here: The Efficacy of Reddit’s 2015 Ban Examined Through Hate Speech(Georgia Institute of Technology, Emory University and University of Michigan researchers, 2017)
Anonymity and Abuse Reports(Cloudflare, 2017)
Instagram’s Kevin Systrom Wants to Clean Up the &#%$@! InternetNicholas Thompson, Wired, 2017
How Facebook Tries to Regulate Postings Made by Two Billion PeopleWorkshop at Berkman Klein Center for Internet and Society, 2017
Facebook: Hard Questions: How We Counter TerrorismFacebook, 2017
YouTube: An update on our commitment to fight terror content onlineYouTube, 2017
YouTube Official Blog, Expanding our work against abuse of our platformYouTube, 2017

Resources for Users

Staying Safe When You Say #MeToo(ACLU)
Technology Safety(Safety Net Project by the National Network to End Domestic Violence)
Security Checklist
Speak Up & Stay Safe(r)
EFF’s Surveillance Self-Defense
Surveillance Self-Defense Checklist
Freedom of the Press Foundation Guides and Training
Smart Girl’s Guide to Privacy(Violet Blue, published 2014)
How to Opt Out of the Sites That Sell Your Personal Data(Wired, 2019)

Building Systems for Addressing Violative Behavior

Safety by Design Assessment ToolseSafety Commissioner Australia, 2021
flowchart illustrating how machine learning develops
Machine Learning for Content Moderation — IntroductionDevin Soni, 2019
illustration of pinterest's machine learning approach
How Pinterest fights misinformation, hate speech, and self-harm content with machine learningPinterest Engineering, 2021
Trust & Safety Engineering @ GitHub(Video, Lexi Galantino, 2019)
Trust and Safety Lessons for New Founders and Engineers(Video, Craig Newmark in discussion with Alex Stamos at Stanford Cyber Policy Center, 2019)
Natural Language Processing: Paying attention to toxic comments online(Stanford researchers, 2018)
picture of hands typing on keyboard with lock graphic overlaid
Trust and Safety Engineering course, Stanford University

Identity Verification and Account Protection

Authentication Best Practices(Open Web Application Security Project, a.k.a., OWASP)
Easy Ways to Build a Better P@$5w0rd(Updated NIST password guidance, 2017)
Design Best Practices for an Authentication System – IEEE Cybersecurity
Evaluating Login Challenges as a Defense Against Account Takeover(Google Research, 2019)
Usage of Behavioral Biometric Technologies to Defend Against Bots and Account Takeover Attacks(Ajit Gaddam, Visa, Usenix Enigma, 2019)
Anatomy of Account Takeover(Grzegorz Milka, Google, Usenix Enigma, 2018)

Modeling and Planning

product school logo
Creating safe digital experiences (Product School presentation)Ram Ravichandran and Sreekant Vijayakumar, 2019

Transparency and accountability

new america logo
The Transparency Report Tracking ToolNew America, 2021
Santa Clara Principles on Transparency and Accountability in Content Moderation(Content Moderation at Scale conference in Santa Clara, CA, 2018)
Google’s Transparency Report
Facebook’s Transparency Report
Twitter’s Transparency Report
Wikipedia’s Transparency Report
Reddit’s Transparency Report

Evaluating Transparency and Metrics

The Abuse Uncertainty Principle, and Other Lessons Learned from Measuring Abuse on the Internet(David Freeman, Usenix, 2020)
Facebook Data Transparency Advisory Group Releases Final Report(Data Transparency Advisory Group (DTAG) at Yale Law School’s Justice Collaboratory, 2019)
Facebook discusses transparency group report’s findings(Facebook, 2019)
Facebook’s Information quality processes for Community Standards Violations metrics(Facebook, 2019)

Examples of community guidelines and other policies

YouTube Community Guidelines
Xbox Community Standards
Vimeo Community Guidelines
Verizon Media Ads Guidelines
Verizon Media Community Guidelines
The Verge Community Guidelines
Twitter Ads Guidelines
Twitter Community Guidelines
Twitch Community Guidelines
Tumblr Ads Guidelines
Tumblr Community Guidelines
Tinder Community Guidelines
TikTok Ads Guidelines
TikTok Community Guidelines
Steam Community Guidelines
SoundCloud Community Guidelines
Spotify Community Guidelines
Snap Ads Guidelines
Snap Community Guidelines
Skillz Community Guidelines
Scratch Community Guidelines
Reddit Community Guidelines
Quora Ads Guidelines
Quora Community Guidelines
Pinterest Ads Guidelines
Pinterest Community Guidelines
Patreon Community Guidelines
OkCupid Community Guidelines
Nextdoor Community Guidelines
Mozilla Community Guidelines
Microsoft Ads Guidelines
LinkedIn Ad Guidelines
LinkedIn Community Guidelines
Kickstarter’s Community Guidelines
Instagram’s Community Guidelines
Google’s Ads Policies
Glassdoor’s Community Guidelines
GitHub’s Community Guidelines
Fandom’s Community Guidelines
Facebook’s Ads Policies
Facebook’s Community Guidelines
Eventbrite’s Community Guidelines
Etsy’s Community Guidelines
Discord’s Community Guidelines
Apple’s Ads Policies
Apple’s App Store Guidelines

Critiques of policies and governance

Meet the Man who Deactivated Trump’s Twitter Account(Ingrid Lunden and Khaled “Tito” Hamze, Tech Crunch, 2017)
Facebook’s Uneven Enforcement of Hate Speech Rules Allows Vile Posts to Stay Up(Ariana Tobin, Madeleine Varner, and Julia Angwin, Propublica, 2017)
Facebook’s Secret Censorship Rules Protect White Men from Hate Speech But Not Black Children(Julia Angwin, ProPublica, and Hannes Grassegger, special to ProPublic, 2017)
The Facebook Files(the Guardian, 2017)
The Pollyannaish Assumption(Ben Thompson, Stratechery, 2017)
The Limits of Automated Social Media Content Analysis(Center for Democracy and Technology, 2017)
Controlling the Conversation: The Ethics of Social Platforms and Content Moderation(Priyanjana Bengani, Tow Center for Digital Journalism, Columbia University, 2018)
Who Moderates the Social Media Giants?(Paul Barrett, NYU Stern Center for Business and Human Rights, 2020)

Public policy and law

a world map shaded in various hues of blue to reflect legislative activity
Digital Policy Alert’s Activity Tracker
The Earth from space with resource information
Mapping Internet Regulations: Online Terrorist ContentActiveFence, 2022
The Twenty-Six Words That Created the Internet(Jeff Kosseff, Published 2019)
Who Do You Sue? State and Platform Hybrid Power Over Online Speech(Daphne Keller, Lawfare, 2019)

Wellness and resilience considerations

Fast, Accurate, and Healthier: Interactive Blurring Helps Moderators Reduce Exposure to Harmful ContentDas, Dang, and Lease (2020)
But Who Protects the Moderators? The Case of Crowdsourced Image ModerationDang, Riedl, and Lease (2018)
Testing Stylistic Interventions to Reduce Emotional Impact of Content Moderation WorkersKarunakaran and Ramakrishan (2019)
CHI 2021 conference logo
The Psychological Well-Being of Content ModeratorsCHI ’21 (Steiger, Bharucha, Venkatagiri, Riedl, and Lease)

Online governance

Consent of the Networked: The Worldwide Struggle for Internet FreedomRebecca MacKinnon

Policy research and development


Government policy and regulations

Consent of the Networked: The Worldwide Struggle for Internet FreedomRebecca MacKinnon
26 Recommendations on Content Governance: A Guide for Lawmakers, Regulators, and Company Policy MakersAccessNow

Transparency and accountability

AccessNow’s Transparency Reporting IndexAccessNow

Community guidelines


Trusted flaggers


Civil society recommendations

26 Recommendations on Content Governance: A Guide for Lawmakers, Regulators, and Company Policy MakersAccessNow


Consent of the Networked: The Worldwide Struggle for Internet FreedomRebecca MacKinnon
All Complex Ecosystems Have ParasitesCory Doctorow