24/12/2025
Causal and Explainable AI Agents, Not All is done By LLM and Agents
,
Topic : Causal Inference and Explainability in RAG-Powered Clinical AI Agents.
Research Area: Causal AI, Explainable AI (XAI), Human-Computer Interaction.
Core Research Question: How can we design and evaluate a RAG-based AI agent that not only retrieves relevant clinical information but can also reason about potential causal relationships and provide transparent, evidence-backed explanations for its clinical recommendations?
Problem Statement & Motivation: A standard RAG agent might recommend a certain intervention because the retrieved documents mention it. However, it cannot distinguish between correlation and causation. It might also not be able to explain why it chose one piece of evidence over another. For a clinician to trust an AI's recommendation, it must provide a line of reasoning that is scientifically sound and grounded in evidence.
Proposed Methodology & Approach:
Integrating Causal Discovery Models: Integrate a causal discovery framework (e.g., a PC algorithm or a gradient-based causal discovery model) into the RAG pipeline. After retrieving relevant FHIR resources, the agent would use this model to generate a causal graph (e.g., Condition -> Medication -> Lab Result Improvement).
Causal RAG (CRAG) Architecture: Design a novel agent architecture where the LLM's reasoning process is guided by this causal graph. The agent's final output would be a recommendation supported by a causal chain.
Structured Explanation Generation: Develop a system that generates a dual-layer explanation: (a) A natural language summary for the clinician, and (b) a structured, machine-readable provenance report showing the exact FHIR resources used and the causal links inferred between them.
Novelty & Contribution to Knowledge: This PhD is at the absolute cutting edge of AI research, combining RAG, Causal AI, and Explainable AI. The primary contribution is a new agent architecture (CRAG) and a methodology for evaluating the trustworthiness and clinical validity of AI-generated explanations. This work would set a new standard for what it means for a clinical AI agent to be "safe" and "effective."
Potential Evaluation Plan: Use clinical case vignettes where the correct diagnosis and treatment path are known. Have the agent generate recommendations and explanations. These would then be evaluated by a panel of clinicians using a standardized rubric that scores the accuracy of the recommendation, the plausibility of the causal reasoning, and the clarity and usefulness of the explanation.