While creating and enforcing policies that define acceptable behavior for content online, internet companies will inevitably intersect with law enforcement agencies. In an increasingly digital world, harmful user interactions may come to the attention of law enforcement in two ways. First, content becomes the subject of a criminal or civil investigation; second, it can be surfaced proactively by platforms through user reporting and automated detection. In either scenario, this may require a platform provider to disclose user information to law enforcement to comply with applicable law — or to uphold company values.
For internet companies, there are typically two functional areas leading to contact with law enforcement:
- Processing and responding to legal and emergency requests.
- Investigating and disclosing threats of harm proactively detected on the platform.
Harm, Law, and Enforcement
The T&S community generally defines harmful content as online content which can cause or lead to physical injury or death, destruction of property, fraud, grave emotional distress or damage, or language intended to intimidate. Harmful content can be spread intentionally or unintentionally.
What Is Harmful Content?
Harmful content may or may not be criminal in nature. Oftentimes harmful material could be language deemed so offensive it may shock the public’s conscience and/or violate a platform’s terms of service. Establishing criteria to determine if content reaches the level for law enforcement notification and providing advanced training will work to reduce the subjectivity in the decision making process. However, making such judgements is not always clear. Supplementing training and policies with experienced T&S professionals is beneficial.
Assigning Criteria
Not all harm is the same. Classifying and prioritizing types of harm is important to manage limited resources. For example, the harm caused from fraud, while important, is likely not the same level of concern for T&S professionals as threats to life. While each company may define harm slightly differently, what matters most is that a company has both a clear definition and policies and procedures for employee training.
As a note to readers, harm is discussed across verticals in the broader T&S environment. For more information, see Creating and Enforcing Policy.
How Law Enforcement Works
Understanding some of the laws and regulations governing how law enforcement works and why interaction may occur is important for T&S professionals.
U.S. Legal Requirements
This section is intended to familiarize T&S professionals with US law and associated obligations. It also enables T&S professionals to begin to distinguish between reasonable law enforcement requests and potential overreach. This section does not replace a T&S professional’s obligation to seek guidance from their supervisor or legal counsel when the T&S professional has questions.
United States federal law enforcement is empowered under the law to use a variety of intelligence and evidence gathering techniques to identify a threat. The Attorney General Guidelines (42 U.S. Code Subchapter II – ATTORNEY GENERAL GUIDELINES | U.S. Code | US Law | LII / Legal Information Institute (cornell.edu)) provide overarching guidance that requires federal agencies to have an authorized purpose consistent with the agency’s mission and seek the least intrusive methods to gather information. Furthermore, federal agents must have a reasonable belief (an articulable fact, sets of indicators, or specific reasonable and articulable knowledge which gives sufficient reasons for them to conduct a query ) to proceed with intelligence collection. A hunch, or mere belief, that something is true or false would not qualify as a reasonable belief.
Law enforcement can collect publicly available information as collection of such information does not require a legal process. Federal law defines publicly available information as information that any member of the public could lawfully obtain by request or observation (not amounting to physical surveillance) and information, including public communications, that is lawfully accessible to any member of the public. “Publicly available” covers information that has been published or broadcast for public consumption, is available on request to the public, is accessible online or otherwise to the public, is available to the public by subscription or purchase, could be seen or heard by any casual observer, is obtained by visiting any place or attending any event that is open to the public, or is made available at a meeting open to the public. Alternatively, closed sources are defined as information environments that are not publicly available.
The Electronic Communications Privacy Act of 1986 (ECPA) extends restrictions on the government’s interception of certain communications and other forms of surveillance to include transmissions of electronic data. ECPA establishes guidelines for law enforcement’s access to information that’s in transit versus stored which is addressed by The Stored Communications Act (SCA). The SCA (18 U.S.C. Chapter 121 §§ 2701–2712) limits the ability of the government to compel an Internet or Electronic Service Provider (ISP/ESP) to disclose content information and non-content information. Specifically, the SCA lays out the process that state and federal law enforcement agencies must adhere to so they can compel disclosure of stored account information by the provider. The SCA also addresses the variety of information providers store. This can include basic subscriber information like name, address, and credit card number. Other potential information includes logs and opened, unopened, draft, and sent e-mails. How providers interpret and apply ECPA and the SCA in their law enforcement response policies and procedures is nuanced and complex. Even in recent cases, courts have been unsure of the ways they apply the laws to modern service providers.
With respect to handling personally identifiable information, United States federal and state law enforcement are governed by The Privacy Act, 5 U.S. Code § 552a. The Act governs the handling of information about individuals that agencies maintain in their “systems of records.” A system of records is a database of records that an agency holds, organized by individual identifiers. If information is in a system of records, an agency cannot disclose individual information from the system, even to other agencies, except under one of the exceptions specified in the statute. This restriction covers law-enforcement requests as well but, as stated above, there are exceptions. In some cases a law enforcement agency can request information directly from a platform and, in other cases, a request can take the form of legal process such as a subpoena or search warrant (this list is not inclusive of all forms of legal requests and is covered more fully in the “Law Enforcement Response” section of this chapter).
The First Amendment applies primarily when the government is involved in speech. The First Amendment covers five core freedoms which are also considered protected speech: religion, speech, press, assembly, and the right to petition the government. The First Amendment’s protections extend to activities on the internet inasmuch as private companies interact with federal law: “A fundamental principle of the First Amendment is that all persons have access to places where they can speak and listen, and then, after reflection, speak and listen once more” (see Packingham vs. North Carolina). The U.S. Supreme Court has recognized that digital spaces, such as social media platforms, are among the “most important places […] for the exchange of views” (see Packingham vs. North Carolina).
There are several exceptions which allow the government to lawfully collect information. One of those categories is unprotected speech. Unprotected speech includes obscenity, defamation, fraud, incitement, fighting words, true threats, speech integral to criminal conduct, and child pornography. While the government may freely regulate unprotected speech, categories of protected speech are subject to a balancing of interests. Speech that enjoys the highest level of protection includes political and ideological speech. Speech of this sort can be restricted only by federal or state governments in service of a compelling state interest, and the restriction must be narrowly tailored to achieve that interest.
While we have described categories of protected and unprotected speech it is important to recognize the examination of speech is more nuanced. Legal guidance and training are best practices to ensure employees have the skills and resources needed.
Information-gathering might fall under the First Amendment if it has an improper purpose. For example, law enforcement agencies cannot target individuals or groups for surveillance on the basis solely of activity that is protected by the First Amendment. But as a note of caution, T&S professionals will likely not know all of the details behind a law enforcement request and must consider they are not privy to all the facts when considering how to respond. This caution is important when evaluating news on a subject. The news is also unlikely to have all of the facts either, and healthy skepticism must be used when trying to determine if a legal order or request is lawful.
The Fourth Amendment is the primary constitutional protection against government intrusion on privacy interests. The Fourth Amendment’s restraints apply to law enforcement and non-law enforcement agencies differently. In this section the explanation is germane only to the Fourth Amendment and law enforcement. The Fourth Amendment is acutely meaningful for law enforcement agencies because there are several consequences for violating a person’s Fourth Amendment rights. Whether the government has violated the Fourth Amendment turns on a two-part inquiry: (i) the existence of a “search or seizure” and (ii) the reasonableness of that search. This section addresses whether the government’s collection of information on the internet rises to the level of a Fourth Amendment “search,” and under what circumstances such a search is considered “reasonable.”
The issues described above can be complicated, and it is recommended T&S professionals seek legal guidance when questions arise.
Responding to Incoming Legal and Emergency Requests
Legal Requirements
In practice, T&S professionals may receive one of the following types of requests: subpoena, 2703(d) court order, search warrant, National Security Letter, statutorily based letter for information or an emergency disclosure request. For example, law enforcement agencies may serve a search warrant on a corporate entity requiring the entity to produce user records that may assist in law enforcement’s investigation to locate a missing child, prevent a violent act, or obtain evidence in a fraud case. To process and handle incoming requests, internet companies often establish dedicated law enforcement response functions, intake channels, and external guidelines (for example, see Reddit’s Guidelines for Law Enforcement) in addition to publishing public transparency reports about key metrics associated with such requests.
When legal process is issued, companies are legally required to comply. Companies can seek recourse if they object to an authorized legal requirement via the court system. Law enforcement teams can also consider vetting the requesting agency to ensure it’s a legitimate law enforcement agency submitting a request. While not common, companies have been known to receive illegitimate requests for user data.
Additionally, as emergency disclosure requests do not need legal processes to obtain user data, it is best practice for law enforcement teams to have policies and procedures established to properly vet and handle these types of requests.