FILE - Text from the ChatGPT page of the OpenAI website is shown in this photo, in New York, Feb. 2, 2023. European lawmakers have rushed to add language on general artificial intelligence systems like ChatGPT as they put the finishing touches on the Western world's first AI rules. (AP Photo/Richard Drew, File) Richard Drew
This picture taken on Jan. 23, 2023, in Toulouse, France, shows screens displaying the logos of OpenAI and ChatGPT. Lionel Bonaventure / AFP via Getty Images / TNS
Kal McKay lives in Canterbury.
Back in May, at the behest of board members, our town tried out Otter AI to take minutes. We were faced with concerns about timeliness, expense, and accuracy with our human secretaries.
As a former software developer and cybersecurity engineer, I was reluctant to give this a try. I was worried about hallucinations, missed context, and the potential for misinformation in a small town where rumor already spread like wildfire.
What I didn’t expect was having to spend an hour trying to teach an AI that the planning board is talking about “abutters,” not sticks of butter, and not rear-ends.
After several weeks of experimentation, I found that the key problem with trying to use an AI to take minutes is sound quality. You not only need good microphones, you need every single person in the room to be mic’d, you need people to use full sentences, you need everyone to stop cross-talking, you need mumblers to speak up, you need those with medical conditions affecting their voices to get miracle cures, and you need New England accents to stop existing.
In one notable experiment, another secretary and I tried three different minute-taking methods for the same meeting: human minutes, human transcript, and AI minutes. Because AI minutes first require an accurate AI transcript, it took four times longer to get the AI version done than the human minutes.
If I had let it out into the world uncorrected, it would have either been nonsensical or worse, just sensical enough to convince people that there were PFAs in the groundwater.
Another cost that I don’t see Civic Sunlight accounting for in their $25 estimate is the administrative cost of addressing inaccuracies. In the example where the AI claimed that funding had been approved for projects that had only been mentioned, I could only think of how much time the City Council’s staff must have spent correcting the misinformation.
Each person who hears the lie and gets upset enough to call, email, or visit the office takes about 20 minutes of talking them off the ledge, explaining what AI is, and regaining trust. Trust that was broken by a private entity, but is being corrected with tax dollars.
If the technology improves, maybe I will change my mind. But as it is right now, AI is more time-consuming, more costly to the taxpayer, and worse for the environment, than just letting secretaries do their jobs.