Bay Area residents who emailed their officials in the past few years may be surprised to learn their correspondence could train a company’s AI for future use.
Mountain View-based company GovernmentGPT filed 90 California Public Records Act requests with multiple cities across the Bay Area for emails from residents addressed to mayors, councilmembers and city clerks from 2020 to 2023. The goal is to create an artificial intelligence tool that can easily summarize a resident’s thoughts based on public comments. AI experts have raised concerns about privacy, consent and ethics when training an AI with this data.
San José Spotlight found requests sent to San Jose, Santa Clara, Mountain View and Milpitas in late 2023 and early 2024. Three cities received a short request asking for documents, while Milpitas received a 10-page request.
Rich Tran, former Milpitas mayor and part of the GovernmentGPT team, is helping spearhead one of the company’s projects, CivicVoice, which documents and summarizes public comment at meetings.
Tran did not respond to requests for comment.
GovernmentGPT CEO and founder Raj Abhyanker, a patent lawyer, told San José Spotlight the company specializes in civic AI and would be providing a service by making public participation easier.
He said they wanted to record residents’ feelings during the COVID-19 pandemic and learn what issues were at the top of people’s minds, adding they aren’t planning to seek records outside the pandemic timeframe.
Abhyanker said the company has a handful of other AI tools, such as CivicVoice. All of these tools are meant to make government work more efficient.
“Cities operate very hatefully, bureaucratically and inefficiently with bloated budgets, and they need major reform and overhaul of how they operate,” Abhyanker told San José Spotlight. “They need less layers between citizens and decision making.”
Alarm bells are ringing
Abhyanker wrote in his request to Milpitas the goal is to create a “ChatGPT-like AI interface” to identify the most pressing issues for residents and streamline municipal operations.
Ahmed Banafa, an engineering professor at San Jose State University and expert on AI technologies, said he lives in Milpitas and was shocked that any of his communications with the city could have been used to train an AI model. Banafa wants to know if his emails are being used, especially if the company plans to make money from the AI.
“It’s freedom of information, not freedom of selling my data, not monetizing my data, that’s never been the case,” Banafa told San José Spotlight.
Abhyanker wrote in the records request the company plans to send this to every city in the state. But he told San José Spotlight the company doesn’t plan to sell the data.
Banafa added only using emails limits the amount of insight an AI model can glean because it overlooks public comments at meetings, from phone calls and online.
“I’m surprised the cities are sitting on this and not making a fuss about this. We have attorneys at the cities, we have counsels, they can question that,” Banafa said. “I’m a citizen of Milpitas, I’ve been here for 30 years, and I don’t want this to happen to my emails or my communications with the city.”
Santa Clara Councilmember Kevin Park, who has worked as an engineer in multiple AI companies for the past decade, said the proposition could be dangerous and likely inaccurate, as it would be easy to taint the training data. It would also take a lot of effort to clean the data due to the sheer number of topics governments deal with.
Park also said the lack of AI regulation means there is little to no repercussion if the product causes harm.
“If making a critical mistake is more than a hand slap, then people will be careful, but right now, what happens if AI is incorrect?” Park told San José Spotlight. “The short answer is this is a new territory, we really don’t have regulations around that.”
Not only are there a lack of regulations, but there are ethical issues with training an AI on emails from residents. Brian Green, director of technology ethics at the Markkula Center for Applied Ethics in Santa Clara University, said the company is exploiting the residents who wrote the emails and the city employees who had to manually redact the private information.
He added these types of AIs need lots of data to train off of, so it’s possible that the company is intending to sell the data rather than build the AI tool itself.
Green said this situation is likely outside the scope of the California Public Records Act and federal Freedom of Information Act, as the writers of those laws didn’t conceive of public records being used in this way.
“That’s not what those emails were intended for in the first place,” Green told San José Spotlight. “These are legally accessible but that doesn’t mean they should have access to them.”
Contact B. Sakura Cannestra at [email protected] or @SakuCannestra on X.
unedited Milpitas Government GPT PRA
