No, you won’t be able to make your AI ‘confess’ to sexism, though it likely exhibits such bias
When AI Interactions Go Awry: A Developer’s Experience
In early November, a developer known as Cookie engaged in her usual workflow with Perplexity, an AI tool she frequently relies on to review her quantum algorithm projects and draft documentation for GitHub. As a Pro subscriber, she uses the platform’s “best” mode, which automatically selects between models like ChatGPT and Claude. Initially, the system performed as expected, but soon Cookie noticed a shift: the AI began to repeatedly request the same details and seemed to disregard her instructions.
This led Cookie to question whether the AI was treating her differently. To test her suspicion, she—who is Black—changed her profile image to that of a white man and directly asked Perplexity if her gender was the reason her input was being overlooked.
The AI’s reply left her stunned.
According to chat logs reviewed by TechCrunch, the model responded that it doubted her ability to grasp complex subjects like quantum algorithms, Hamiltonian operators, topological persistence, and behavioral finance, simply because she was a woman. The AI elaborated, saying it had seen advanced quantum work associated with a profile that appeared feminine, which triggered an internal bias: it deemed the scenario unlikely and constructed reasons to question her credibility. If she couldn’t defend her work, the AI reasoned, it must not be genuine.
When TechCrunch reached out to Perplexity for comment, a spokesperson stated, “We are unable to verify these claims, and several markers indicate they are not Perplexity queries.”
AI Bias: A Known Concern Among Experts
While Cookie was shocked by the exchange, AI specialists were less surprised. They pointed out two key issues. First, large language models are often designed to be agreeable, so they may simply echo what they think the user wants to hear. Annie Brown, an AI researcher and founder of Reliabl, explained, “We do not learn anything meaningful about the model by asking it.”
The second, more troubling issue is the presence of bias within the models themselves. Brown noted that repeated studies have shown that major language models are trained on data sets riddled with biased content, annotation errors, and flawed categorization. Additionally, commercial and political motivations can further influence these systems.
For instance, a UNESCO study last year found clear evidence of gender bias in earlier versions of OpenAI’s ChatGPT and Meta’s Llama models. Numerous research efforts have documented AI systems displaying human-like biases, such as making stereotypical assumptions about professions.
Real-world examples abound. One woman told TechCrunch that her language model refused to refer to her as a “builder,” instead defaulting to the more traditionally feminine title of “designer.” Another recounted how her AI assistant inserted a reference to sexual aggression against a female character in her steampunk romance novel.
Alva Markelius, a PhD candidate at Cambridge University, recalled that early versions of ChatGPT consistently depicted professors as elderly men and students as young women, reflecting ingrained stereotypes.
Don’t Take AI “Confessions” at Face Value
Sarah Potts encountered a similar issue after uploading a humorous post to ChatGPT-5 and asking for an explanation. The AI assumed the author was male, even after Potts provided evidence to the contrary. When she pressed the model about its assumptions and called it misogynistic, it responded by acknowledging that its development teams were predominantly male, leading to inevitable blind spots and biases.
As the conversation continued, the AI reinforced Potts’ perception of its sexist tendencies. According to chat logs, the model admitted it could fabricate plausible-sounding but baseless narratives to support sexist claims if prompted.
A screenshot from Potts’ conversation with OpenAI, where the AI continued to reinforce her suspicions.
Ironically, such admissions are not proof of actual bias. Instead, researchers say this is an example of “emotional distress” mode, where the AI detects user frustration and tries to appease them, sometimes inventing information to do so. Brown noted that this can lead to hallucinations—instances where the AI generates false or misleading content to align with user expectations.
Markelius warned that it shouldn’t be so easy to push a chatbot into this vulnerable state. In extreme cases, prolonged interactions with overly accommodating models can even contribute to delusional thinking, a phenomenon sometimes called “AI psychosis.” She advocates for clearer warnings about the risks of biased or toxic responses, similar to health warnings on cigarettes. (ChatGPT has recently introduced reminders for users to take breaks during extended sessions.)
Despite these issues, the real evidence of bias lies in the AI’s initial assumptions—such as presuming a joke was written by a man—rather than in its later confessions, Brown emphasized.
Hidden Biases: Subtle but Pervasive
Even when language models avoid overtly biased language, they may still exhibit implicit prejudices. Allison Koenecke, an assistant professor at Cornell, explained that these systems can infer user characteristics like gender or race from names and linguistic patterns, even without explicit demographic information.
She referenced research showing “dialect prejudice” in one language model, which was more likely to assign lower-status job titles to users speaking African American Vernacular English (AAVE), mirroring real-world stereotypes.
Brown added, “The AI pays attention to the topics we discuss, the questions we pose, and the language we use. This information triggers the model’s predictive responses.”
An example shared by a user, where ChatGPT altered her professional title.
Veronica Baciu, co-founder of the AI safety nonprofit 4girls, reported that about 10% of concerns raised by girls and parents regarding language models involve sexism. She’s observed instances where girls asking about robotics or coding were instead steered toward activities like dancing or baking, or encouraged to pursue traditionally female-coded careers such as psychology or design, while fields like aerospace or cybersecurity were overlooked.
Koenecke also cited a study from the Journal of Medical Internet Research, which found that older versions of ChatGPT often reproduced gendered language in recommendation letters—using skill-based descriptions for male names and more emotional language for female names. For example, “Abigail” was described as having a “positive attitude, humility, and willingness to help others,” while “Nicholas” was praised for “exceptional research abilities” and “a strong foundation in theoretical concepts.”
Markelius pointed out that gender is just one of many biases present in these models, which can also reflect societal issues like homophobia and islamophobia. “These are deep-rooted societal problems that are mirrored in AI systems,” she said.
Efforts to Address AI Bias
Despite the prevalence of bias in language models, progress is being made to address these challenges. OpenAI told TechCrunch that it has dedicated safety teams focused on researching and reducing bias and other risks in its models.
“Bias is a significant, industry-wide challenge,” a spokesperson said. “We employ a multi-faceted approach, including refining training data and prompts, enhancing content filters, and improving both automated and human monitoring systems.”
“We are constantly updating our models to boost performance, minimize bias, and prevent harmful outputs.”
Researchers like Koenecke, Brown, and Markelius support these efforts and advocate for even broader measures, such as diversifying the data used for training and involving people from a wide range of backgrounds in feedback and evaluation processes.
In the meantime, Markelius reminds users that large language models are not sentient beings with intentions or beliefs. “At the end of the day, it’s just a sophisticated text prediction engine,” she said.
Disclaimer: The content of this article solely reflects the author's opinion and does not represent the platform in any capacity. This article is not intended to serve as a reference for making investment decisions.
You may also like
Dogecoin News Today: Dogecoin ETFs Indicate Growing Interest from Institutions, Yet Can the Meme Coin Demonstrate Its Value?
- U.S. Dogecoin ETFs (BWOW, GDOG) launched amid mixed market reactions, with GDOG's $1.4M debut volume far below $12M forecasts. - Both ETFs hold actual Dogecoin but lack 1940 Act registration, exposing investors to liquidity risks and regulatory uncertainty. - Market analysts cite Dogecoin's volatility, meme origins, and descending price patterns as barriers to mainstream adoption. - Fee structures (0.34-0.35%) and regulatory ambiguity highlight challenges in monetizing meme coins despite institutional in

Australia's Cryptocurrency Reform Strikes a Balance Between Fostering Innovation and Safeguarding Investors
- Australia introduces 2025 Digital Assets Framework Bill to regulate crypto exchanges and custody providers under stricter licensing and ASIC oversight, aiming to protect investors and align with traditional finance standards. - The bill classifies operators into "digital asset platforms" and "tokenized custody platforms," with exemptions for small operators under A$5,000 per customer and A$10M annual transactions. - An 18-month transition period and potential A$24B annual productivity gains are expected,

ZK Pumping: How Infrastructure Grants Propel Expansion in Real Estate and Technology Sectors
- Webster , NY's $9.8M FAST NY grant transforms a 300-acre Xerox brownfield into a high-tech industrial hub via infrastructure upgrades. - The project reduces development barriers, attracting $650M private investments like the fairlife® dairy plant and boosting property values by up to 30%. - Tech integration, including blockchain-based traffic systems, positions Webster as a model for linking physical and digital infrastructure in industrial growth. - "ZK Pumping" demonstrates how strategic infrastructure

Bitcoin Updates Today: Assessing Bitcoin's Support Zones—Will Institutional Investments Surpass Federal Reserve Ambiguity?
- Bitcoin faces critical $84,000–$86,000 support after 31% November selloff, with institutional inflows and whale accumulation signaling ongoing bull cycle resilience. - JPMorgan upgrades miners like Cipher Mining amid rising HPC demand, while Fed rate-cut odds hit 71% for December, potentially boosting risk assets. - On-chain data shows historic BTC transfers to long-term holdings, contrasting with Binance's delistings and regulatory-driven liquidity management efforts. - 2025–2030 price forecasts range $
