AI for the principled practitioner 

The advent of AI brings a host of ethical concerns for therapists and mental health practitioners. Therapists have their fair share of chores when it comes to medical documentation, and AI-powered tools promise to increase the efficiency of these mundane, labor-intensive tasks. Yet, it is undeniable that therapy is a deeply human-centric practice. AI poses concerns over placing limitations on the empathetic aspects of therapy, and it is well known that efficacy of treatment is predicated on the ability to create genuine therapeutic connection. This raises important questions over how much of the therapeutic process can responsibly be handed over to AI while preserving the integrity of the practice.

AI’s debut role in therapy

If you’re a therapist, you’ve likely seen a surge of “AI powered note taking tools” that promise to ease or even eliminate progress note and medical documentation burden. While the allure is understandable, these tools largely divest the therapist’s decision making power and compromise client confidentiality by relying on recorded session transcripts. There are several pitfalls of this approach that muddy the therapeutic process including forcing the practitioner to bend on privacy practices and ethical considerations. 

Human IN the loop (HITL)

Human-in-the-loop is a technological principle that aims to blend human expertise with automated processes. The goal is to create a system that maximizes the strength of each party’s characteristics. In our case, this means leveraging AI to perform lower-order, memory intensive tasks such as symptom recall, pattern recognition, and data analysis while leaving the therapist to perform higher-order tasks such as treatment strategy, interventions, and relational work. 

Therapist role (HITL)

  • Provide a safe, privacy-focused environment that fosters trust and therapeutic effectiveness

  • Decide which data the AI should analyze, minimizing noise and improper reporting

  • Execute interventions while maintaining a therapeutic alliance with the client

  • Evaluate outputs of AI to inform, rather than dictate, treatment

  • Set and guide treatment direction for the client

AI role (HITL)

  • Identify trends and patterns in symptoms, behaviors, and goal progress based on therapist input, rather than relying on text-based session transcripts.

  • Format lay terms into clinically precise language

  • Provide recommendations aligned with the therapist’s treatment plan, always leaving the final decision in human hands

Human ON the loop (HOTL)

Human-on-the-loop is another approach that places human interaction into more of a supervisory role, granting AI a higher degree of autonomy and decision making power. While this approach is practical in specific use-cases, it becomes precarious when applied to a field predicated on human empathy and connection. In these types of applications, the therapist’s role is to ensure the AI’s operability, checking transcripts for errors, and reviewing AI-generated outputs after the fact—leaving the AI, rather than the therapist, to handle the bulk of the therapeutic work.

Therapist role (HOTL)

  • Maintain the AI system and ensure it can consistently capture sessions and gather data

  • Review lengthy session transcripts to try and minimize interpretation errors and transcription inaccuracies

  • Review the output of the AI application

AI role (HOTL)

  • Transcribe audio to text (often with a considerable margin of error)

  • Analyze text and synthesize data to draw therapeutic conclusions based on the data it’s been trained on

  • Produce fully formed medical documentation from session transcripts, leaving little room for therapist intervention

  • Hand off documentation to the therapist

Striking balance

Ultimately, the ethical questions surrounding AI in therapy boil down to a balance between technological efficiency and preserving the therapist’s role as a caring, empathic professional. While AI can significantly reduce paperwork and offer data-driven insights, clinicians must remain vigilant about privacy, autonomy, and the therapeutic alliance. Approaches like HITL show promise by keeping therapists firmly in control of high-level decisions and maintaining the human connection, while AI handles more mechanical tasks. By thoughtfully integrating AI without compromising the core principles of empathy, trust, and confidentiality, therapists can reap the benefits of technology while safeguarding the integrity of therapeutic practice.

Our commitment to ethical practices

At PepperChat, privacy and ethics are at the heart of everything we do. We never share or sell client information, and we’ve signed BAAs with our AI vendors to ensure that client information is never retained or used to train AI models keeping client confidentiality front and center. Our mission is to integrate world-class engineering with evidence-driven therapeutic practices, all without disrupting the fundamental human connection that makes therapy so effective.