Cit Labs’ framing of what it says its model will do. Credit: Cit Labs Christy Chen, founder of Cit Labs, argues that AI chatbots fail at emotional support not because the technology is incapable but because the models were optimized for engagement, trained on text that performs support rather than delivers it, and lack physiological signal. Her startup combines EEG data with clinician-defined guardrails to build something purpose-built for the job General-purpose AI chatbots have become everyday software, used by hundreds of millions of people for work, homework and, increasingly, for talking through personal problems. As this specific use has spread, reports have accumulated of the same systems agreeing with users’ unhealthy thoughts, reinforcing delusions or simply telling people what they want to hear.
Christy Chen, founder of Cit Labs, which is prototyping a model that links noninvasive EEG brain-wave data with voice and language, reads those same failures differently. She locates them in what today’s systems were built to do, what they learned from, and the business models they inherited from the rest of the internet. What AI was (and wasn’t) built for Chen locates the failure in two inherited choices: the business models the rest of the internet runs on, and the habit of repurposing general-purpose tools for a job they weren’t necessarily trained for. Mental health adds commercial complications of its own, including reimbursement and insurance structures that shape what gets funded and who it reaches. First, general-purpose large language models are tuned for engagement, productivity and keeping a conversation going. “The LLMs we have today aren’t trained to be helpful therapists,” Chen says. “They’re optimized for other things, productivity, or simply keeping you talking to them.” Pointed at emotional support, a system rewarded that way will tend to agree, because agreement keeps the user talking. Sycophancy, the habit of flattering and validating whatever a user brings, is the predictable output of those incentives.
Then the training signal, which doesn’t exist for the act of helping. The models were never taught when to reflect something back, when to push against a user, when to stay quiet and when to hand a situation to a person. That judgment is learned from outcomes over years, and it lives with practitioners. The internet holds people performing support and describing therapy.
It does not hold the graded record of doing it well. The raw material compounds the gap. Text is plentiful for sounding supportive, which is why models are fluent at hollow empathy, and thin on the two things that make support work: reading a person’s actual state, and knowing whether what was said landed. Chen is not arguing that AI is inherently incapable of meaningful emotional support.
Her claim is that today’s general-purpose chatbots weren’t trained or incentivized to provide it safely. She separates her position from broad fears about “AI psychosis,” locating the problem instead in how these systems were built and what they were built for. Her claim is narrower: a repurposed chatbot with better incentives still will not do the job, because it has neither the data nor the training signal. “I think it’s more of an incentive problem in how the product is built than a question of whether it’s technically feasible,” she says. Those gaps are why she started a company instead of waiting for the general-purpose models to improve.
Cit Labs runs on physiological signal and clinician-informed guardrails, and she incorporated it as a Delaware public benefit corporation so the commitment sits in the company’s legal structure rather than in its intentions. The demand already exists The behavior in question isn’t hypothetical. People are already using general-purpose models for emotional support and informal therapy at meaningful scale, and Chen counts herself among them, having turned to the tools that way since the first chatbots of the current wave went public. “People are already using LLMs for this, but the answers often aren’t what people actually need,” she says. The pull toward those tools traces to a shortage. “There are a limited number of people on this planet who can unconditionally accept others and provide emotional value,” Chen says.
Doing it well takes an enormous amount of bandwidth from anyone, clinicians included, which caps supply no matter how many practitioners enter the field. Her own frustration is what turned that into a company, though not one aimed at the therapist shortage directly. Rather than configure an existing chatbot for the use case, Chen set out to build for the underlying need, and the design follows from the diagnosis. A chatbot can hear what a person says and, increasingly, how they say it.
Cit Labs is developing a system that adds a third source of evidence, aligning noninvasive EEG with voice and language on a single timeline. Physiology isn’t treated as ground truth or as a hidden reading of what someone really means. Two things supply the meaning the model doesn’t assume on its own. Clinicians define when the system should act, and the user teaches it what their own signals mean over time.
That division of labor is aimed at a problem Chen has seen both in the recommendation systems that profit from it and in her own meditation and yoga practice, where she learned to watch it arrive in herself: an internal shift starts driving behavior before the person notices it. The first product is consumer-facing and conversational, with verbal interactions meant to help people notice patterns in their own states rather than to diagnose them or hold their attention. Clinical use and direct physiological intervention are later-stage possibilities that would need separate validation and professional oversight. What building for the job would require The failure isn’t one prompt adjustments can solve.
Fixing it means changes at the system level: what a model conditions on, and what it is permitted to conclude, and just as necessary is structured input from practitioners during design. “You need the right model architecture, and you need the right input from professional therapists and medical professionals to make it genuinely helpful,” she says. Clinicians, she argues, should define when a system ought to intervene and when it should leave ordinary emotional fluctuation alone, which is the arrangement she has built into her own. Asked how any product draws that line without intruding on the normal ups and downs of a life, Chen is direct: “I think it needs professional input. We’ll work with professional therapists to draw that line.” Beneath the technical argument sits a distinction she treats as consequential, between a system that understands a person and one that only appears to know them.
Therapy makes the gap visible. A good therapist challenges a client when challenge is what the moment calls for, and a well-built system should manage the same instead of defaulting to agreement. Therapeutic usefulness, on her account, has to be engineered deliberately. It does not emerge from a general-purpose model on its own.
What responsible design would look like Chen starts further back than the therapist shortage, with the need itself. The moments her product is aimed at recur in every life (a conversation avoided, or a reaction that arrives before its reason) and the attention that could meet them, whether a clinician’s, a friend’s, or one’s own, is scarce enough that most are met by no one. “It isn’t necessarily the best tool, but it provides things that aren’t currently available,” she says. The ambition is additive: more of that capacity, reaching moments nothing currently does, which reads as a more modest claim than other consumer AI products. Boundaries, in her design, need to be set by professionals, and any system handling emotional content
Leave a Reply