Integration of Artificial Intelligence Into Extended Reality Debriefing in Healthcare Simulation: A Narrative Review

Introduction

Debriefing, which translates performance into learning through guided reflection and facilitator-led inquiry, is commonly acknowledged as the pedagogical core of simulation-based education (SBE).¹ Experienced facilitators employ organized debriefing methods (such as advocacy-inquiry and plus-delta) in more conventional simulation contexts (manikins, task trainers, and team-based scenarios) to encourage learners to consider their actions, mental processes, and results and to make connections to clinical practice.² New opportunities such as reproducible immersive scenarios, comprehensive telemetry (actions, gaze, physiology), and asynchronous access are brought about by the introduction of extended reality (XR), which includes virtual reality (VR), augmented reality (AR), and mixed reality (MR), in healthcare simulation. However, there are also new challenges for debriefing, as the facilitator might not be able to watch every action in real time, learners might interact asynchronously or remotely, and complex data streams could be overwhelming.^3,4

Beyond their technical capabilities, XR environments present unique pedagogical contexts for debriefing. Immersive simulations modify perceptual focus, embodiment, and emotional involvement, thereby generating learning experiences that are spatially dispersed and individually experienced via head-mounted displays, as opposed to being collectively observed within a shared physical setting.^4,5 In traditional mannequin-based simulation, facilitators can directly observe the majority of learner behaviors and team interactions in real time, facilitating debriefing that reconstructs events from a largely shared experiential perspective.² Conversely, XR interactions may transpire within partially opaque virtual environments, necessitating that facilitators depend on telemetry, replay functionalities, and learner narrative accounts to establish a shared mental model of events.^1,4 Furthermore, immersive XR experiences have been demonstrated to augment presence and cognitive engagement, potentially intensifying emotional significance and influencing how learners encode and recall events.⁵ These characteristics necessitate deliberate strategies during debriefing to scaffold reflection, reconstruct distributed experiences, and uphold psychological safety while navigating high-fidelity performance traces.^2,6

These differences extend beyond increased data volume. XR simulations frequently enable asynchronous participation, distributed teams, and remote facilitation, thereby altering the social and temporal structure of debriefing compared with traditional co-located simulation.^1,4 The availability of granular replay data, including gaze tracking, trajectory mapping, and timestamped event logs, introduces the possibility of detailed performance analysis.⁷ While such precision may enhance objective feedback, it also risks overemphasizing micro-level technical deviations at the expense of broader clinical reasoning, teamwork processes, and affective dimensions of learning.² Additionally, embodied interaction metrics, such as movement smoothness or force application, represent novel performance signals that require interpretation within pedagogically meaningful frameworks.^5,8,9 Without careful alignment to debriefing models grounded in reflective inquiry, the presence of objective telemetry may shift debriefing toward technical audit rather than reflective meaning-making.^2,6 Consequently, XR contexts demand thoughtful integration of data-rich analytics within established facilitation frameworks to preserve the core educational intent of debriefing.

XR debriefing can be understood through the lens of existing learning theories that inform simulation-based education. From the perspective of reflective practice, debriefing offers a structured approach to reflection-on-action, enabling learners to examine their reasoning, underlying assumptions, and actions within a psychologically safe environment that promotes professional growth.¹⁰ Reflective dialogue has been identified as the key process through which simulation experiences are transformed into enduring learning.¹¹ Moreover, experiential learning theory views simulation as the concrete experience phase, with debriefing functioning as the reflective observation and abstract conceptualization stages that support transfer to future clinical practice.¹² Within immersive XR environments, the increased sense of presence and environmental complexity may elevate cognitive demands.⁵ Cognitive load theory suggests that excessive sensory input or unstructured performance data can increase extraneous load and impede schema construction.^8,13 Therefore, AI systems that filter telemetry, highlight critical decision points, and scaffold structured inquiry may serve as cognitive supports that enhance, not replace, facilitator-guided reflection.^1,13 Evaluating AI-augmented XR debriefing through these theoretical frameworks ensures that technological advancements align with established principles of reflective depth, cognitive efficiency, and transfer-oriented learning.

Konzelmann et al, in their recent review, highlighted the “integration of artificial intelligence (AI) into XR debriefing” as a potentially valuable, yet insufficiently investigated, domain.¹ Considering the escalating interest in AI-driven simulation, this narrative review specifically examines AI’s function in debriefing within XR/VR simulation, deliberately excluding the wider simulation process. This review is based on peer-reviewed publications from 2015 to 2025, sourced through structured searches of PubMed/MEDLINE and Google Scholar, and supplemented by reference screening of relevant articles. Search strategies employed combinations of terms such as “extended reality”, “virtual reality”, “augmented reality”, “mixed reality”, “XR”, “healthcare simulation”, “medical education”, “debriefing”, “feedback”, “artificial intelligence”, “natural language processing”, “large language models”, “conversational agents”, and “learning analytics”. Studies were deemed eligible if they directly applied artificial intelligence to debriefing processes or provided debriefing-relevant feedback within XR/VR healthcare simulations; conversely, studies concentrating solely on scenario generation, automated grading devoid of reflective elements, or non-XR environments were excluded. Specifically, we examine: the functional roles AI can play in debriefing (feedback generation, spoken/dialogue analysis, conversational agents, multimodal fusion); synthesize empirical evidence from the past decade; discuss pedagogical alignment and technical design considerations; and offer a roadmap for education-developers and researchers.

The primary contribution of this review is a functional taxonomy of AI roles in XR debriefing to guide educators, developers, and researchers. Secondarily, we map the maturity of the empirical evidence and provide pragmatic implementation considerations aligned with debriefing standards. Where possible, findings are interpreted using commonly applied medical education outcomes hierarchies, noting that most existing studies report learner perceptions or performance metrics (lower-level outcomes), with limited evidence of sustained behavior change or patient-level impact.^1,9,14

Utility of AI in XR Debriefing

XR simulation platforms produce rich, high-fidelity datasets encompassing timestamps of learner actions, gesture and trajectory logs, audio and speech recordings, gaze tracking, and in some cases, physiological signals.⁷ Artificial intelligence (AI) enables the transformation of these raw data into meaningful debriefing artefacts through four primary mechanisms. First, automated performance metrics and analytics use AI models to analyze event logs and generate objective performance indicators, such as time-to-task completion, deviation from optimal trajectories, and error counts, thereby facilitating effective, data-driven feedback for procedural skills.^15,16 Second, Natural Language Processing (NLP) for dialogue analysis allows AI to transcribe speech, code communication for key themes like decision-making or coordination, detect tone or emotion, and identify critical “turning-point” utterances that can enrich reflective discussions during debriefing.^17,18 Third, conversational agents, embodied conversational agents (ECAs), and large language models (LLMs) can emulate virtual facilitators or participants, engaging learners in guided reflective dialogue—such as by asking “What were you thinking when…?”—and providing immediate, context-specific feedback derived from simulation logs.^19–21 Finally, multimodal fusion integrates multiple data streams, including learner actions, speech transcripts, gaze, and physiological signals, to deliver adaptive, individualized feedback, such as by highlighting issues in team-role allocation or adjusting the pacing of reflection according to learner needs.^22,23 Collectively, these AI-driven functions directly address persistent debriefing challenges, such as limited faculty availability, variability in facilitation quality, and the demand for objective feedback, while capitalizing on XR’s intrinsic advantages of rich telemetry, asynchronous accessibility, and scalability.¹ Table 1 attempts to summarize the functional roles of AI in XR debriefing keeping in mind objectives, evidence maturity, and limitations.

Table 1 Functional Roles of AI in XR Debriefing: Objectives, Evidence Maturity, and Key Considerations

Current State of Evidence

NLP and Automated Discourse Analysis

Early work in medical education applied NLP to simulate clinical reasoning and discourse coding, showing feasibility for automated coding of transcripts and reflective statements (Chary et al reviewed NLP in medical education).¹⁷ Trauma simulation research showed relationships between human rater ratings and NLP-derived metrics (eg, amount of advocacy-style utterances) by comparing pre- and post-debrief conversation and quantifying teamwork behaviors.^24,25 NLP has been employed in proof-of-concept research more recently (2024–2025) to extract sentiment patterns, decision points, and the frequency of reflecting utterances from team simulations. These studies have shown agreement with human-rater judgments.^26,27 However, limitations remain: the majority of NLP research concentrates on surface metrics (turn count, keywords) rather than more profound interpretive concepts (emotional understanding, depth of thought).^17,24 Emotion detection and multimodal alignment (eg, speech + gaze + action) are underdeveloped.

Automated Scoring and Analytics for Procedural Debriefing

XR and VR procedural simulators (eg, for surgery) capture kinematic and timing data; AI anomaly detectors have been used to compute performance dashboards that correlate with expert ratings and distinguish between competence levels.^5,8,9 For example, a haptics-enabled VR drill classified users as novice or expert based on automatic criteria of smoothness and mistakes.⁹ However, the majority of these indicators (force profiles and trajectories) are still technical, and they do not always correspond to instructional messages that are pertinent to the debrief (eg, team coordination and decision-making). The major challenge lies in translating lower-level metrics into pedagogically meaningful feedback (eg, “You applied excess force at step X, consider slower hand-off”).^5,9

Conversational Agents, LLMs, and ECAs for Debrief Dialogue

Emerging prototypes embed LLMs or ECAs in VR simulations to conduct interactive virtual patient encounters and to provide automated debriefing prompts.^19,20 Research shows that these systems have the ability to ask guided reflection questions, role-play difficult conversations before providing feedback, and summarize performance logs.¹⁴ For example, a virtual AI patient simulator (VAPS) powered by LLMs allowed learners to complete a simulated scenario, then receive immediate feedback and reflection scaffolding through a conversational agent.¹⁹ There are not many controlled studies that directly compare AI-only debriefing with hybrid or human facilitator models. Key concerns include AI’s lack of affective nuance, contextual sensitivity, and risk of generating misleading feedback if the model is improperly trained or domains are outside it’s scope.^3,14,20

AI Model Comparison Synthesis

Across studies, three broad debriefing configurations can be identified: AI-only, hybrid (AI-assisted human facilitation), and traditional human-led models. AI-only approaches offer scalability, consistency, and immediate feedback delivery, particularly in asynchronous XR environments; however, empirical support remains largely prototype-based, and concerns persist regarding affective nuance, contextual sensitivity, and the risk of misleading feedback.^14,19–21 Traditional human-led debriefing remains the gold standard for psychological safety, emotional processing, and nuanced meaning-making, yet is resource-intensive and subject to facilitator variability.² Hybrid models, where AI prepares structured artefacts such as transcripts, performance dashboards, and suggested advocacy–inquiry prompts for facilitator use, appear most aligned with current evidence and educational theory.^1,14 In this configuration, AI augments analytic capacity while preserving human oversight for high-stakes interpretation, emotional attunement, and adaptive questioning.

Pedagogical Alignment

AI-augmented debriefing needs to follow accepted debriefing pedagogy (such as the International Nursing Association for Clinical Simulation and Learning or SSH best practice guidelines) in order to be considered educationally sound.⁶ Three alignment strategies are particularly relevant:

1. Scaffolded AI prompts mimicking facilitators: AI should start reflection with neutral, curiosity-driven questions (eg, “Tell me what you were thinking when…”) rather than judgmental statements, then follow up according to learner responses.^5,20
2. Hybrid models: AI-assisted facilitator workflows: Instead of replacing human debriefers, AI can prepare documentation (transcripts, event highlights, suggested questions) freeing up facilitators to concentrate on meaningful, high-value conversations.^1,14
3. Learner-centered personalization: AI can adjust feedback and prompts based on a learner’s past performance, cognitive load (eg, shorter prompts when physiological stress is high), and competence, to maximize reflection and support germane load.^13,28 However, there is still little proof of long-term learning, behavior modification, and transfer to clinical settings (Kirkpatrick levels 3 and 4).^1,9,14

Ethical Considerations

AI-augmented debriefing raises ethical issues. Audio, video, and telemetry data are sensitive. Recording, secondary analysis, retention, and anonymization must all be covered by explicit learner consent.^3,7 NLP models may mis-transcribe non-native accents or under-represent non-western communicative styles, disadvantaging diverse learners.¹⁷ AI should not be used for decision-making, but rather as an adjunct. Human facilitators must continue to be in charge of making crucial decisions (like competency).^1,14 Feedback should be transparent and comprehensible, demonstrating the methods used to arrive at conclusions (eg, event times and transcript excerpts).²⁹

Key Gaps

Despite promising developments, major gaps remain. Few controlled trials comparing debrief modes have been conducted, as a result, very limited data exist on long-term learning or patient-care transfer.^9,14,15 Psychometric validation of AI metrics should be prioritized (eg, does “hesitation index” predict clinical decision error?).^9,29 Implementation research needs to be conducted in order to determine the kind of training required for faculty to integrate AI into debrief operations, interpret AI outcomes, and minimize over-reliance^1,6 Empirical work on bias (voice/accent), privacy risk, and learner perceptions remains scarce.^3,17

Practical Recommendations

Educators and simulation developers should adopt a hybrid model in which artificial intelligence (AI) is used to prepare debriefing artefacts (such as transcripts, event highlights, and suggested questions) while preserving the human facilitator as the central reflective coach who guides meaning-making and emotional processing.^1,14 AI-generated outputs should be deliberately designed to align with familiar debriefing frameworks by structuring automated feedback in advocacy-inquiry-compatible language and including transparent references such as quotations or timestamps to support credibility.^6,29 Before full implementation, institutions should pilot and localize AI systems by validating AI-derived scores against local expert raters and checking for transcription errors or biases, including those related to accent or language variability.^3,9 Safeguarding learner privacy is essential; programs must build opt-in consent processes, ensure telemetry data are anonymized, and retain data only for the minimum necessary duration.³ Finally, faculty should receive dedicated training to develop the skills needed to interpret AI outputs, integrate them appropriately into debriefing practice, and maintain effective facilitation when automated systems generate unexpected or ambiguous feedback.^1,6

Conclusion

XR platforms generate high-resolution performance data, and AI systems can transform these data into structured analytics and feedback artefacts, with feasibility studies demonstrating technical viability and preliminary alignment with expert ratings. Promising applications include scalable, personalized debrief preparation, and adaptive reflective prompting in immersive contexts. However, AI-only debriefing models, the predictive validity of AI-derived metrics, and demonstrable effects on behavior change or patient outcomes remain unproven.

Unlike broader XR or AI-in-simulation reviews that emphasize scenario design or automated assessment, this review centers on debriefing as the pedagogical core of simulation-based education. By proposing a functional taxonomy of AI roles in XR debriefing and synthesizing current evidence through an outcomes lens, it clarifies both the potential and the present limitations of AI-augmented reflection.

Future work should extend beyond technological development to examine how AI reshapes debriefing scholarship, particularly its effects on reflective depth, psychological safety, facilitator judgment, and human–AI co-facilitation. Robust validation and longitudinal outcome studies will be essential for translating innovation into evidence-based practice.