Did Snapchat AI just go rogue?

• Snapchat My AI chatbot experienced a glitch, making users think the AI had become self-aware. 
• As of July 2023, Snapchat has 750 million daily active users worldwide.
• While the problem was a technical glitch, the impact shows the need to have more regulations on AI development. 

Several months ago, Snapchat unveiled several new features to its My AI chatbot. Available to Snapchatters, the Snapchat My AI chatbot uses generative AI technology to answer questions and even offer advice to users based on their queries.

Powered by OpenAI’s ChatGPT technology, Snapchat also assured users that My AI had additional safety enhancements and controls unique to the app. A check on the company’s help page on My AI stated that the company believes that “there should be transparency when AI is involved in content creation” and if a user shares content generated by My AI with others, they should let them know AI was involved in the work.

The site also stated, “We’re constantly working to improve and evolve My AI, but it’s possible My AI’s responses may include biased, incorrect, harmful, or misleading content. Because My AI is an evolving feature, you should always independently check answers provided by My AI before relying on any advice, and you should not share confidential or sensitive information.”

With all this said, users might think that Snapchat AI should work the same way as any other generative AI tool. However, it seems that the AI may have plans of its own.

According to numerous reports, Snapchat users, which number 750 million daily active participants worldwide, began to panic when the platform’s AI chatbot posted a live update to its profile and stopped responding to messages. In posts that have since gone viral on other social media platforms, Snapchat users shared how the AI chatbot posted a one-second video without being prompted and then deleted it.

The Story My AI posted was just a two-toned image that some mistook to be a photo of their ceiling or a photo frame. When users tried to chat with the bot, the AI in some cases replied to users by saying “Sorry, I encountered a technical issue.”

Users also started to worry even more when the AI chatbot was giving them weird replies to their questions. For example, a user wanted to ask AI for a suggestion but the chatbot replied that it was busy and unable to respond to them.

CNN, which also reported the anomaly, reached out to Snapchat. The company said that they were not trying to make the AI tool sound more realistic but that there was an actual glitch with the program. A Snapchat spokesperson told CNN that the My AI chatbot had experienced a temporary outage that has since been resolved.

Did Snapchat AI really experience a glitch?

Many users, however, were still not convinced that the app had solved the problem and began to ponder whether the AI had actually become sentient or even gone rogue.

Since My AI posted the video, more users have also started unpinning or removing the function from their chat feed. To remove My AI features, Snapchat+ subscribers can just toggle off the function from their management screen. There are also several steps to remove the feature from the chat space, which users can find on Snapchat’s website.

The Snapchat MyAI chatbot has also faced backlash from users over privacy concerns. Some users have even highlighted creepy exchanges and an inability to remove the feature from their chat feed unless they pay for a premium subscription.

Over the past few months, concerns have grown about AI potentially causing more harm than good in society. There have been numerous calls for AI to be regulated. Some companies have even banned the use of certain AI tools like ChatGPT by their employees, due to concerns of data security. In fact, Sam Altman, CEO of OpenAI has himself called on lawmakers to regulate AI.

The European Union (EU) is already working on a new legal framework that aims to significantly bolster regulations on the development and use of AI. The proposed legislation, the Artificial Intelligence (AI) Act, focuses primarily on strengthening rules around data quality, transparency, human oversight and accountability.

Tech leaders, including the “godfathers” of AI, have also called for the regulation of AI technology. Geoffrey Hinton, one of the so-called godfathers of AI, has urged governments to step in and make sure that machines do not take control of society. Hinton had even quit Google to speak more freely on the dangers of AI.

Meanwhile, in a meeting with President Biden last month, tech leaders from Amazon, Google, Meta, Microsoft, and other companies agreed to establish AI safeguards. The safeguards will be an important step in managing the capabilities and potential risks posed by AI in the future.

While Snapchat’s My AI glitch may have been resolved, the impact it created serves as a minor indication of the potential societal panic if an AI program genuinely becomes sentient or goes rogue one day.

For now, the technology is still under the control of humans, or made to believe that humans still have control over it.

---

---

---

---