The Security Hole at the Heart of ChatGPT and Bing

May 25, 2023

(Wired) – Giardina created the replica of Sydney using an indirect prompt-injection attack. This involved feeding the AI system data from an outside source to make it behave in ways its creators didn’t intend. A number of examples of indirect prompt-injection attacks have centered on large language models (LLMs) in recent weeks, including OpenAI’s ChatGPT and Microsoft’s Bing chat system. It has also been demonstrated how ChatGPT’s plug-ins can be abused.

The incidents are largely efforts by security researchers who are demonstrating the potential dangers of indirect prompt-injection attacks, rather than criminal hackers abusing LLMs. However, security experts are warning that not enough attention is being given to the threat, and ultimately people could have data stolen or get scammed by attacks against generative AI systems. (Read More)

Recommended Reading