On Monday, OpenAI revealed it was making changes to promote a healthier relationship between users and ChatGPT.
In a blog post, it said: "AI can feel more responsive and personal than prior technologies, especially for vulnerable individuals experiencing mental or emotional distress."
It said it would not give a definitive answer to questions such as, "Should I break up with my boyfriend?"
Instead, it would "help you think it through - asking questions, weighing pros and cons", according to the blog post.
In May, OpenAI pulled a heavily-criticised update which made ChatGPT "overly flattering", according to Sam Altman.
On a recent episode of OpenAI's own podcast, Mr Altman said he was thinking about how people interact with his products.
"This is not all going to be good, there will still be problems," he said.
"People will develop these somewhat problematic, or maybe very problematic, parasocial relationships [with AI]. Society will have to figure out new guardrails. But the upsides will be tremendous."
Mr Altman is known to be a fan of the 2013 film Her, where a man develops a relationship with an AI companion.
In 2024, actress Scarlett Johansson, who voiced the AI companion in the film, said she was left "shocked" and "angered" after OpenAI launched a chatbot with an "eerily similar" voice to her own.