Multimodal Foundation Models and Agentic Frameworks for Neurology and Epilepsy Management

Neurological disorders are among the world’s greatest health burdens, affecting hundreds of millions globally. In Australia, conditions such as epilepsy, stroke, dementia, Parkinson’s disease, and traumatic brain injury are leading causes of long-term impairment. These disorders often progress or fluctuate, making timely and accurate diagnosis essential.

Managing neurological conditions is highly complex and must be tailored to the individual. Clinicians must integrate multimodal information—neuroimaging (MRI, PET), electrophysiology (EEG, ECG), patient histories, genomics, and functional assessments.

This project will deliver a nationally accessible platform for foundational neuroimaging models, with a multimodal AI use case in epilepsy care. It will provide open, reusable, and clinically validated tools, transforming how neuroimaging is accessed, interpreted, and applied across Australia.

Epilepsy use case

Epilepsy affects 4% of the population over a lifetime, with unpredictable seizures of varying type, cause, and severity. Effective management requires accurate reporting and monitoring. This project will develop tools to support:

Foundation Models for Video-EEG monitoring and seizure tracking
Agentic AI systems for decision support during VEM sessions
Multimodal Foundation Models for therapy planning, including medication and surgical recommendations

Output

Hakeem, H., Feng, W., Chen, Z., Choong, J., Brodie, M. J., Fong, S. L., ... & Kwan, P. (2022). Development and validation of a deep learning model for predicting treatment response in patients with newly diagnosed epilepsy. JAMA neurology, 79(10), 986-996.
Chen, Z., Rollo, B., Antonic-Baker, A., Anderson, A., Ma, Y., O’Brien, T. J., ... & Kwan, P. (2020). New era of personalised epilepsy management. bmj, 371.
Mehta, D., Sivathamboo, S., Simpson, H., Kwan, P., O’Brien, T., & Ge, Z. (2023, October). Privacy-preserving early detection of epileptic seizures in videos. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 210-219). Cham: Springer Nature Switzerland.
Pham, D. K., Mehta, D., Jiang, Y., Thom, D., Chang, R. S. K., Foster, E., ... & Ge, Z. (2025). Adapting Biomedical Foundation Models for Predicting Outcomes of Anti Seizure Medications. medRxiv (Miccai 2025 accepted), 2025-08.