Creating and Enforcing Policy

Developing policies to prevent, address, and remediate abusive behavior on digital platforms is a core practice within the field of Trust and Safety. Abusive behavior can take a nearly infinite variety of forms: examples include everything from theft of data from cloud services to fraud in payment processing to real-world violence communicated or facilitated through digital services. This chapter focuses on user-generated content (UGC) because it is likely most relevant to the greatest number of emerging professionals in the field; however, the practices and approaches described here will be relevant to most types of products and services. 

Notice: Due to the nature of content policy, this chapter contains references to potentially sensitive or offensive topics.

In this chapter, readers will gain:

  • An understanding of standard policy-building models
  • An overview of common user-generated content (UGC) abuse types
  • An overview of common policy enforcement methods
  • A sampling of relevant regulation to consider when developing policy