New York City Mayor back AI chatbot that shared wrong advice to businesses

Eric Adam defends AI chatbot: New York City Mayor Eric Adams is standing by the city's newly implemented artificial intelligence chatbot, despite recent instances where it provided incorrect information or advice that could potentially lead to legal infractions.

Initially launched as a pilot program in October, the MyCity chatbot was heralded as a groundbreaking use of AI technology on a city-wide scale, aiming to furnish business owners with "trustworthy and actionable information" in response to their inquiries via an online portal.

However, its performance has not consistently lived up to these expectations. Investigations conducted by journalists at The Markup revealed inaccuracies, such as erroneously suggesting that employers could claim a portion of their workers' tips and asserting that there were no regulations mandating employers to notify staff about schedule changes.

Acknowledging these shortcomings, Mayor Adams, a member of the Democratic Party, addressed reporters on Tuesday, underscoring the nature of the program as a pilot initiative. He stressed the necessity of subjecting technology to real-world conditions to identify and rectify any deficiencies, stating, "Any time you use technology, you need to put it into the real environment to iron out the kinks."

Mayor Adams has been a vocal advocate for the adoption of untested technologies in the city, although not all endeavours have been successful. For instance, his introduction of a 400-pound robot in the Times Square subway station last year to aid law enforcement in crime deterrence was retired after approximately five months, with commuters observing its apparent inactivity and inability to navigate stairs.

Despite ongoing inaccuracies, the MyCity chatbot remains operational, continuing to dispense incorrect information on occasion. Notably, it erroneously asserts that store owners are permitted to operate cashless businesses, disregarding a 2020 city council law prohibiting establishments from refusing cash payments. Additionally, it inaccurately states the city's minimum wage as $15 per hour, failing to reflect the updated rate of $16 as of 2024.

The chatbot, powered by Microsoft's Azure AI service, seems to be susceptible to issues common among generative AI platforms like ChatGPT, occasionally fabricating or asserting falsehoods confidently. While Microsoft declined to elaborate on the underlying causes of these problems, it assured collaboration with the city to address them. The city's Office of Technology and Innovation likewise expressed commitment to resolving inaccuracies promptly, anticipating significant improvements shortly.

Amid these developments, disclaimers on the MyCity chatbot website have been updated to caution users that its responses may be inaccurate or incomplete, advising against relying on them for legal or professional guidance.

Andrew Rigie, Director of the NYC Hospitality Alliance representing numerous restaurant owners, voiced concerns over business owners' confusion stemming from the chatbot's responses. While acknowledging the city's efforts to leverage AI for business support, Rigie emphasised the imperative of ensuring its accuracy, cautioning that adherence to erroneous guidance could entail severe legal repercussions. He stressed the need for clarity, stating, "If I have to consult my lawyers to verify the accuracy of the chatbot's responses, it defeats its purpose."

(With Reuters inputs)