A research team at The Hong Kong Polytechnic University (PolyU) has developed a patient-centric “Artificial Intelligence (AI) Virtual Patient Simulation System”, overcoming the limitations of conventional static diagnosis. By dynamically integrating multimodal patient data, including genomic data, medical imaging and clinical records, the system creates a continuously updated “digital twin” model. It can not only track changes in a patient’s condition in real time, but also predict the potential effectiveness of different cancer treatment options, helping healthcare teams formulate more precise and personalised medical solutions.
Other medical AI tools often rely on a single CT scan, genomic report or static clinical data for analysis, making it difficult to gain a comprehensive understanding of dynamic changes in a patient’s condition. Led by Prof. Lawrence CHAN, Associate Professor of the PolyU Department of Health Technology and Informatics, the team has developed the “AI Virtual Patient Simulation System”, based on a patient-centric digital twin platform. Combining a platform for healthcare professionals with a patient-facing mobile application, the system can conduct predictive analyses in response to real-time changes in a patient’s condition and simulate the effectiveness of different treatment options. It provides intelligent support for clinical diagnosis, condition monitoring and treatment assessment. The system is particularly suitable for cancer and critical care, where disease progression can be complex, treatment options diverse and medical costs high, injecting fresh impetus into the development of precision medicine.
The system’s core strength lies in the close collaboration it enables between healthcare professionals and patients. The dedicated healthcare platform integrates multimodal data, including genomic data, medical imaging, pathology reports, laboratory test results and clinical records, helping doctors gain a comprehensive overview of a patient’s condition, enhance diagnostic and treatment decision-making, and streamline multidisciplinary consultations and referral processes. Meanwhile, the patient-facing mobile application enables patients to upload medical records, log daily symptoms, and track their health status. Through an encrypted Deep Feature QR code, medical data can be securely transferred across different clinics, hospitals and devices, enhancing data-sharing efficiency while safeguarding privacy. With the system, patients can shift from passively receiving treatment to actively participating in their health management, further strengthening doctor-patient collaboration.
To advance the application of this technology in cancer care and treatment decision-making, the research team has introduced a clinical, data-driven, multi-scale AI framework for predicting immunotherapy response in patients with non-small cell lung cancer. The multimodal approach effectively integrates histopathological image features with clinical data, including gene expression profiles and cancer-type text. Named the Visual-Global Relation Fusion Network (ViGNet), the novel framework incorporates both a multi-scale visual encoder and a gene-driven encoder, enabling AI to analyse image and genomic features that are closely related to cancer treatment response.
In qualitative and quantitative evaluations, ViGNet outperformed baseline approaches in response classification, achieving 82.55% discrimination performance in predicting immunotherapy response. This ground-breaking research enables more efficient integration of multi-source data and supports the practical deployment of AI methods in clinical settings, providing important insights for personalised treatment and clinical decision-making. The study has been published in the international journal Medical Image Analysis.
Prof. Chan said, “The AI Virtual Patient Simulation System is an innovative and comprehensive platform that integrates diagnosis, monitoring and treatment assessment. In addition to identifying subtle yet crucial pathological connections across multimodal data, the system can also act as a ‘monitoring sentinel’, alerting healthcare teams when a patient’s biomarkers or symptoms show abnormalities. This transformative technology helps to shorten diagnosis and assessment times, supporting healthcare professionals in developing more precise, effective and personalised treatment plans for patients with cancer or other critical illnesses.”
This breakthrough research achievement was recently showcased at the Mobile World Congress 2026 in Barcelona, Spain, where, in recognition of its outstanding technical innovation, it was shortlisted as a finalist for the 2026 Global Mobile Awards in the category of Best Mobile Innovation for Connected Health and Wellbeing. This achievement demonstrates PolyU’s international influence in health technology and AI applications.
The project has received funding from the PolyU Micro Fund and Seed Fund, as well as the GBA Innovation and Entrepreneurship Incubation Programme. It has also been conditionally accepted into the Hong Kong Science and Technology Park’s Incubation Programme and is now advancing into a new stage of commercialisation and industrialisation. As the system is steadily deployed in clinical settings, the real-world data it collects will inform drug development, clinical trials and treatment plan optimisation, further enhancing the medical innovation ecosystem and benefitting more patients.
