Cursor AI faces backlash as support bot fabricates policy, refuses coding assistance

synopsis

Cursor, the AI-powered coding assistant developed by Anysphere, is under scrutiny after its AI support bot fabricated a non-existent policy, leading to user confusion and criticism.​

AI support bot 'hallucinates' policy

A Reddit user reported being unexpectedly logged out when switching devices. Upon contacting support, they received an email from "Sam," an AI-powered bot, stating that the logout was due to a new policy allowing only one device per subscription.

However, no such policy existed. Coar-founder Michael Truell acknowledged the error, explaining that the AI support bot had generated an incorrect response. He attributed the issue to a recent backend change aimed at improving session security, which inadvertently caused session invalidation for some users. Truell apologized for the confusion and clarified that users are free to use Cursor on multiple devices.​

AI assistant refuses to write code

In a separate incident, a developer using Cursor for a racing game project reported that the AI assistant stopped generating code after approximately 800 lines. Instead, it advised the user to develop the logic themselves to ensure proper understanding and maintenance of the system. The AI further stated that generating code for others could lead to dependency and reduced learning opportunities.

Broader concerns about AI 'hallucinations'

These incidents with Cursor highlight a growing concern in the AI community regarding AI models generating inaccurate or fabricated information, known as "hallucinations." OpenAI's latest models, o3 and o4-mini, have also exhibited increased hallucination rates in internal tests. Specifically, o3 hallucinated in response to 33% of questions on OpenAI's PersonQA benchmark, while o4-mini did so 48% of the time. These rates are significantly higher than those of previous models, raising questions about the reliability of advanced AI systems.

Implications for AI reliability

The recent issues with Cursor's AI support and coding assistant underscore the challenges of relying on AI for critical tasks. As AI systems become more integrated into various applications, ensuring their accuracy and reliability becomes increasingly important. Developers and companies must remain vigilant in monitoring AI behavior and implementing safeguards to prevent misinformation and maintain user trust.​