Skip to content

P5: Technology for Patient Uses of Care Networks

Context

Complementing P4, and augmenting P1, P2 and P3, we will develop patient-facing agentic, conversational AI (CAI) to support a sociotechnical model of care coordination incorporating an AI ally to help patients navigate services. Learning from Stage 1, we will focus on the musculoskeletal, pain and mental illness (including substance use disorders) MSPM phenotype – see for example, Lucy’s story.

Objectives

  1. Identify and define core tasks patients require assistance with including: a) preparing for clinical appointments; b) navigating appointments; and c) information seeking and exploration to increase health literacy.
  2. Establish methods to measure patients and carers challenges in navigating services.
  3. Co-develop and feasibility test ~2-4 prototype CAIs that: a) help patients prepare, organise and structure information/narratives describing their own health needs in readiness for clinical encounters; b) use systems automation technology to interrogate EHR data and assist with constructing personalised schedules/itineraries for attending multiple appointments for LTCs; and c) act as an‘avatar’ to support dialogue with patients and carers post-clinical encounter.

Approach

Using the expertise drawn from voluntary- and patient/public stakeholders across CHIL (MLTCs) and M-RIC (mental illness) and the Byres Community Hub will use human-centred and participatory design methods alongside agile, iterative development with embedded health equity in the delivery of innovations.

Engineering of prototype CAI agents will leverage existing natural language processing/AI expertise within CHIL, MRIC and the University of Glasgow. The sampling and involvement of stakeholders will draw on WS1, and iterative co-design/-development/-evaluation methods in WS3.

We will recruit ~5 co-designers (specifically for the MSPM phenotype) and 5 NHS stakeholders to work with WS1 engineers to co-create the prototype applications.

For evaluation, the PPIE co-developers, clinicians and engineers will use a small group deliberative activityto select the most mature and potentially impactful innovation to test in a feasibility pilot. All stakeholders will have lived experience of the MSPM phenotype and be selected to represent the diversity of health-, technology- and educational- literacy challenges across different ethnic and socio-demographic groups in Liverpool and Glasgow. Clinical stakeholders will be drawn from primary and secondary care experts in the MSPM phenotype -- borrowing from P4

Process

  1. Evidence synthesis: Existing systematic reviews for CAIs identify trials across single diseases and the literature largely describes prototypes. Reviews of barriers to navigating care for MLTCs are more coherent but they rarely abstract away from individual conditions or patient groups. To elicit patients’ unmet care navigation needs, we will review the evidence for existing instruments. For CAI solutions, given the rapid pace of development, with WS3, we will maintain a living review of literature for CAI techniques relevant to conditions in the MSPM phenotype. We will map CAI-by-LTC coverage, identify evidence gaps relevant to prototype projects including evidence quality (by adopting relevant standards from ~ 26 different reporting standards and CHEERS-AI for health economic evaluation).

  2. Solution designs: The core ‘care navigator’ design and implementation processes in primary care will be adapted for MLTCs across the healthcare system. Participatory design will be used to capture stakeholder views on necessary functionality for blended human/CAI tools across the 2-4 agentic CAI prototypes. Initial functionality (‘scripts’ of common queries and expected answers) will be co-developed with stakeholders using knowledge elicitation techniques and interview methods alongside measuring the patient group’s health literacy (people's ability to find, understand, appraise and communicate information to engage with the demands of different health contexts).

  3. Candidate solutions development: CAI agents 26 will use private large language models (LLMs) with prompt-engineering and retrieval-augmented, chain- and tree-of-thought reasoning with self-consistency techniques to improve dialogue functionality, performance and safety.

  4. Candidate solutions evaluation, iteration and selection: The performance of CAI agents will be assessed using evaluation benchmarks including automatic metrics (e.g. accuracy and trustworthiness of dialogue), human-like qualities/acceptability (cognitive, affective and compassionate domains of empathy) and the Technology Acceptance Model. The most mature and promising prototype project/innovation will then be selected for feasibility testing.

Feasibility Study

Of the prototypes, we will select one CAI considered mature enough for a 2-arm, cross-over pilot feasibility trial involving 10 patients at UoG and another 10 at UoL.

Methodology: Prospective two-site cluster cross-over feasibility study using a mixed-methods approach.  10 participants at each site will use either their usual care navigation strategies, or the candidate (prototype) CAI tool for navigating care.

  • Participant recruitment:
  • 20 patient participants (10 in Liverpool, 10 in Glasgow)

  • Adults aged 18+ with mental and physical MLTCs (the musculo-skeletal, pain and mental illness phenotype).

  • Intervention:

  • The candidate prototype CAI for care navigation used for 8 weeks by the participants.

  • Comparator:

  • 'Usual’ care navigation strategies for 8 weeks

  • Outcome Measures:

  • Primary: At baseline, we will measure each participant’s care navigation needs using the European Patients’ Forum (EPF) 5As/Es framework (including perception of solution availability, adequacy, accessibility and patient’s expertise, experience, engagement) These will be compared pre- and post- use of the prototype CAI. Note, one of the objectives described above will be to define quantitative (structured) measures of satisfaction with care navigation.

  • Process evaluation: Semi-structured interviews with participants and healthcare professionals to explore experiences, barriers, and facilitators. 

  • Analysis:

  • Quantitative: Descriptive statistics for pre- and post-test measures (pending review and adoption of appropriate measures)
  • Qualitative: Thematic analysis of participant interviews, triangulated with usage data from the CAI.