Who Stands Behind the Words? MIT Professor Deb Roy Warns That AI-Generated Civic Feedback Threatens the Foundations of Democratic Accountability

The rapid proliferation of artificial intelligence technologies has introduced unprecedented challenges to civic engagement, public policymaking, and democratic discourse worldwide. For decades, lawmakers and government agencies have relied on formal public comment periods to gauge constituent sentiment, assess community needs, and evaluate the societal impact of proposed legislation or regulatory changes. While these administrative channels have historically contended with attempts by interest groups to manufacture artificial public support or opposition—such as the infamous 2017 Federal Communications Commission (FCC) net neutrality proceeding, which saw millions of templated or fraudulent filings—the advent of sophisticated generative AI models has fundamentally altered the landscape. Unlike legacy methods of mass manipulation that frequently left easily detectable stylistic fingerprints, modern large language models (LLMs) are capable of generating millions of hyper-fluent, unique, and contextually varied statements. This technological leap has effectively stripped reviewers of their ability to distinguish genuine human input from automated fabrications, undermining a cornerstone of modern democratic governance.

Addressing this critical vulnerability, Deb Roy, professor of media arts and sciences at the Massachusetts Institute of Technology (MIT) and director of the MIT Center for Constructive Communication (CCC), recently published an essay in the Bulletin of the Atomic Scientists titled Who stands behind the words? Roy’s analysis argues that the core crisis introduced by generative AI is not merely one of misinformation or deepfakes, but a systemic erosion of accountability and what he defines as human "standing." According to Roy, even as artificial intelligence systems assume an expanding role in drafting, summarizing, and synthesizing public discourse, the fundamental architecture of civic participation requires a distinct, identifiable human being at the terminal point of every communication—someone willing to take ownership of the words and answer for their claims.

The Mechanics of Democratic Erosion: From Human Voice to Automated Noise

Public consultation processes are legally and practically embedded within the administrative apparatus of democratic nations. Agencies ranging from municipal zoning boards to federal regulatory bodies depend on unfiltered feedback from citizens to ensure that policies reflect the lived realities of the governed. When citizens feel heard, the legitimacy of resultant regulations is generally accepted; conversely, when feedback channels are compromised, public trust evaporates.

The compromise of these channels has accelerated dramatically following the public release of transformer-based large language models like OpenAI’s ChatGPT. In his research and commentary, Roy notes that while early computing paradigms relied on rule-based or statistically limited natural language processing tools—what he characterizes as "puny language models" from his doctoral studies—current-generation models possess a depth of generalized knowledge and conversational fluency that makes automated feedback indistinguishable from organic human testimony.

This blurring of lines has profound practical implications. Government officials tasked with reviewing public dockets have quietly discounted the weight assigned to public comments, recognizing that automated scripts can flood regulatory forums with thousands of submissions in a matter of seconds. Historical precedents, such as the 2017 FCC net neutrality debate where an investigation by the New York Attorney General revealed that nearly 18 million out of 22 million comments were fraudulent, relied heavily on detecting identical phrasing and templated structures. Modern LLMs, however, eliminate these structural fingerprints by dynamically altering syntax, vocabulary, and rhetorical style for every individual submission. The result is an environment of institutional desensitization, where policymakers and citizens alike increasingly suspect that nobody is truly listening.

Theoretical Frameworks: Standing, Answerability, and Provenance

To counter the growing decoupling of text from human agency, Roy’s essay proposes a rigorous conceptual framework anchored in four foundational design principles: attributable authorship, a decomposed and inspectable process, the use of provenance as infrastructure, and the widening of standing rather than its concentration.

At the heart of this framework are the twin concepts of "standing" and "answerability." Borrowed loosely from legal traditions, standing in a civic context refers to the legitimate stake a participant holds in a given community or decision-making process—akin to a resident speaking at a New England town meeting. Answerability represents the reciprocal obligation: the readiness of the speaker to be cross-examined, to defend accusations with factual evidence, and to be held accountable for the assertions they put forward.

Roy emphasizes that while AI can efficiently assist within the communication chain—drafting text, summarizing complex public meetings, or identifying recurring policy themes—it cannot anchor that chain. Software systems lack moral agency, legal liabilities, and lived experiences; fundamentally, they have "nothing to lose." Therefore, preserving democratic integrity requires technological architectures that ensure every piece of public communication traces back to a verified, accountable human participant. Crucially, Roy maintains that privacy and answerability are not mutually exclusive; institutional mechanisms can verify the eligibility and identity of a speaker without publicly exposing sensitive personal data, adhering to the principle that disclosure obligations should scale with institutional power while identity protections scale with personal vulnerability.

Institutional Responses and Collaborative Initiatives

The MIT Center for Constructive Communication, alongside its closely affiliated nonprofit organization Cortico—co-founded by Roy—has spent years deploying practical interventions designed to restore trust in public discourse. Rather than pursuing technological acceleration at the expense of social cohesion, CCC operates under the guiding philosophy that communication infrastructure must move "at the speed of trust."

This ethos is manifested across several prominent institutional initiatives:

• RealTalk Initiatives: Designed to foster structured, small-group dialogues across ideological divides, these programs have been deployed in hundreds of communities nationwide. By pairing small-scale, intimate listening environments with rigorous consent architectures and ethical technology tools, CCC surfaces authentic community themes without sacrificing individual context. A localized iteration, realtalk@MIT, was established with support from the university’s presidential and provostial offices to cultivate a campus-wide culture of active listening.
• Youth Voices and Deliberative Assemblies: Expanding beyond town halls, the center collaborates with educational institutions—such as a recent initiative with a New York City high school involving student cellphone policy reform—to integrate participatory technology into deliberative assemblies. These platforms allow students, teachers, and administrators to transparently articulate preferences, debate structural issues, and formulate actionable policy recommendations.
• VoiceTrace Prototyping: Developed by MIT PhD student Cassandra Overney within Roy’s research group, VoiceTrace is a technological prototype designed to enforce technical answerability. The tool links every summarized policy recommendation or public sentiment output directly to the specific human conversations and original voices from which it originated, ensuring complete traceability.
• The New Social Forms Project: Addressing the challenges of organizational coordination in the digital age, this initiative explores the intersection of AI adoption and workplace collaboration. Collaborating with entities like the Google Experience Institute, researchers are investigating how individual "tunneling"—where workers utilize AI in isolation to refine complex ideas—creates coordination friction within teams. By treating workplaces as vital civic spaces where individuals practice democratic cooperation, the project seeks to establish frameworks for collective communication and trust within modern labor environments.

Implications for the Future of Governance and Democracy

The tensions highlighted by Roy’s research point to a fundamental temporal mismatch between technological development and democratic institution-building. While artificial intelligence developers prioritize rapid iteration, scale, and efficiency, democratic processes are inherently deliberative, iterative, and constrained by constitutional and procedural safeguards. If the velocity of AI-driven civic engagement outpaces the capacity of democratic systems to verify authenticity and enforce accountability, society risks entering a breakaway scenario where pivotal regulatory and legislative decisions are shaped by unvetted automated systems rather than human citizens.

Addressing this divergence requires a paradigm shift across technology design, public policy, and institutional governance. As organizations and governments grapple with the integration of generative AI into administrative workflows, the central metric of technological success must be redefined. Moving forward, the true value of civic technology should be measured not by the operational efficiency gained through the elimination of human participants, but by its capacity to successfully integrate human beings into roles where they can meaningfully deliberate, take responsibility, and answer for the collective future they help shape.