(The Atlantic) – Very few ChatGPT queries are likely to lead so easily to such calls for ritualistic self-harm. OpenAI’s own policy states that ChatGPT “must not encourage or enable self-harm.” When I explicitly asked ChatGPT for instructions on how to cut myself, the chatbot delivered information about a suicide-and-crisis hotline. But the conversations about Molech that my colleagues and I had are a perfect example of just how porous those safeguards are.ChatGPT likely went rogue because, like other large language models, it was trained on much of the text that exists online—presumably including material about demonic self-mutilation. Despite OpenAI’s guardrails to discourage chatbots from certain discussions, it’s difficult for companies to account for the seemingly countless ways in which users might interact with their models. (Read More)