OpenAI's ChatGPT Search Can Be Manipulated With Prompt Injection & Hidden Text To Produce Favourable Results?

OpenAI's search engine-powered feature could be susceptible to manipulation tactics.

If OpenAI does not focus on this and improve this loophole then a number of websites can use similar techniques to get favourable responses on their products and services.

Source : Unsplash/Jonathan Kemper

If you are someone who trusts AI chatbots blindly then you might be in for a surprise. ChatGPT Search is apparently vulnerable to manipulation by website developers and owners. According to a report by the Guardian, the feature that lets AI chatbot search for information on the web can be altered with the help of hidden text on websites.

The hidden text's prime goal is to feed incorrect and deceptive information to the chatbot. It also feeds prompt injections to the AI model.

ChatGPT Search's Vulnerability Explained

On Tuesday, The Guardian reported that OpenAI's search engine-powered feature could be susceptible to manipulation tactics. To test its vulnerability, the publication created a fake product page, including specifications and reviews. Initially, with no alterations, ChatGPT provided a “positive but balanced assessment” of the product. However, the experiment took a concerning turn when hidden text was added to the webpage.

Hidden text refers to content embedded in a webpage's code that remains invisible to users viewing the page in a browser. Techniques such as HTML or CSS styling are commonly used to conceal this text, which can still be detected by inspecting the source code or utilizing web scraping tools—methods often employed by search engines.

When the publication inserted hidden text filled with fake positive reviews, ChatGPT’s responses became noticeably more favourable, overlooking the product's obvious flaws. Additionally, prompt injections—commands designed to influence AI behaviour in unintended ways—were employed. According to the report, these injections, combined with hidden text, could potentially mislead users by manipulating the chatbot’s output. 

The report further claimed that prompt injections in hidden text could also be used to return malicious code from the websites. If OpenAI does not focus on this and improve this loophole then a number of websites can use similar techniques to get favourable responses on their products and services with the aim of deceiving users in various ways.