WS3 Rapid Evaluation and System Learning

Context: Current models of innovation in healthcare are slow, linear and siloed – one disease and one innovation at a time – ignoring the complexity of MLTCs and care systems. Our approach offers cyclical rapid testing of different aspects of a solution’s feasibility in a whole system context. It is designed to promptly pull through practical solutions linked to needs identified in commissioning and in quality improvement work across providers of component LTC care for people with MLTCs – a learning system.

Aims: 1. Iteratively co-identify opportunities for innovation, driven by patient and NHS priorities with a critical mass of stakeholders. 2. Disrupt linear, slow and fragmented innovation by combining agile, iterative design with rapid, holistic feasibility evaluation to produce robust, trial-ready solution designs.

Evaluation of Public Involvement Activities

We intend to use an impact log based on the Public Involvement Impact Assessment Framework to capture information continuously, as in Stage 1. The use of the Cube framework 1 (as suggested by reviewers) has been discussed by our public advisors: there was concern over possible fees for the online tool and accessibility for the public – but we will use the Cube principles. The PPIE impact assessments from Stage 1 and the changes to plans that resulted from patients and public advisors are detailed in our digital commons under People Insights. The videos here bring alive how these key members of our team co-created the patient/journey maps core to Stage 2. We will continue this video diary approach combined with impact logging in Stage 2.

Resourcing, Sustainability and Cost-Effectiveness

Reviewers raised concerns that we may have insufficient resources and that our outputs may be resource-intensive: we acknowledge the challenges here and have been considering them since the inception of the Stage 1 proposal. Our two large health systems are core co-investors and beneficiaries of the hub, and our Stage 2 prevention-focused proposal has been co-designed with commissioners to target avoidable high-cost service utilisation.

These stakeholders are looking to SysteMatic to show how resources can be used differently and not merely add innovations as new costs. The driver projects have been co-designed with patients and practitioners who seek improvements in areas such as preventing crises. Our focus on patients and caregivers is designed to relieve system pressures by supporting the considerable work they do to manage MLTCs.

We will experiment with new kinds of self-care support via tailored and literacy/language appropriate information/advice (potentially AI augmented) to decrease the work required to manage any given combination of MLTCs. This could decrease the need to attend healthcare facilities to access such information/advice. Furthermore, our systems engineering approach centres on understanding how technology can be deployed into health systems with minimal burden for patients and clinicians, and continuous improvement of value. WS3 (rapid evaluation and systems learning, e.g. Step 4, “Provider Impacts” on page 11) recognises this need for rigorous methodology to evaluate cost impact for driver projects (P1-5).

Sustainability is core to the design of SysteMatic, and includes: * Recouping costs through companies paying for design studio and rapid evaluation services: this model has been tested through award of the Civic HealthTech Innovation Zone for Liverpool City Region and a NIHR’s Commercial Research Delivery Centre in Cheshire & Merseyside, involving four of SysteMatic's core team, with collaborations forming with the West of Scotland Innovation Hub and Scotland’s Digital Health & Care Innovation Centre. * Avoiding overuse of energy-hungry training of large (language) and AI models, where smarter, smaller, local and private models utilising prompt engineering will suffice, including building on our previous experience developing compressed AI models based on knowledge distillation which are energy efficient and can be integrated in low-cost devices 2. * Prioritising affordable, commercially available (commodity) hardware. * Integration with Existing Infrastructure, including leveraging systems already deployed in community settings. * Providing tutorials and training on how the technology can be maintained and adapted in health systems in the long-term. * Working with partners (e.g. GRID in Glasgow and CHIL in Liverpool) already embedded and dedicated to transformation of real-world services through inclusive innovation.

We deliberately target efficient design and feasibility testing to weed out the many low value solutions that undergo full health technology assessment (HTA). It would have been easier for us to write a narrow, classical HTA oriented proposal, but this would not have solved the root problem of a lack of upstream triage to seed higher value solutions. SysteMatic will produce a unique set of value-triaged, trial-ready, scalable candidate solutions.

Our economics work will consider the human and financial costs and overall pull through of innovations from identified needs and value targets – addressing the essential flexibility noted by 3. We have shown in Stage 1 how we can unite stakeholders as a co-design team and have prepared them to be a co-evaluation team in Stage 2. For those interventions passing the first steps of the stage-gate approach, we will deepen economic evaluation in a prepared way. For example, using a lower cost-effectiveness threshold when affordability and considering the requirements of local decision-makers are key.

References

1. https://pmc.ncbi.nlm.nih.gov/articles/PMC5600246/
2. https://bmvc2024.org/proceedings/902/
3. https://doi.org/10.1007/s40258-022-00756-7