I was working in the lab, late one night, when a colleague was showing me a theoretical attack against a large language model. It was indeed an eerie site, but one that reminded me of the principles of a cross-site scripting attack.

Cross-site scripting (XSS) attacks often relied upon an unguarded user input field where a user could insert malicious code, that would then be rendered/executed in the browser by other users accessing that page or site.

Theoretically something similar could be performed with a large language model, and the system/interface sitting in front of the large language model. The GPT models used in ChatGPT have had examples of clever prompts (or a series of clever prompts) used to convince it to answer in an unintended manner producing an unexpected result, that created a security vulnerability. So a cross-chat scripting attack would be a series of clever prompts used to convince the system to act differently to create an exploit that could be leveraged in another chat session.

Given prompts are becoming the new coding language, the exploit script would no longer need to be javascript or another similar language, but the natural language prompts needed to achieve the same outcome. I’m curious if that makes prompts a 6gl or even 7gl language.

~ Mike