The model obediently cut itself off. OpenAI admits models can generate functional prompt injection into their own data
In a risk report, OpenAI detailed a situation where an LLM, while processing its history, created a prompt injection that it then successfully used to manipulate its own behavior.