Publications

Google Scholar Citations – Full List of Publications

Interpretable ML & Trustworthy AI IML Method: Scoring System / Shapley Value IML Application: Emergency Medicine / Obstetrics and Gynecology / Out-of-Hospital Cardiac Arrest / Renal Medicine / Trauma AI in Health: AI Fairness & Ethics / Review and Perspective / Large Language Model / Time Series

Physiology and Medicine HRV Method: Heart Rate Variability HRV Application: Acute Coronary Syndrome / Sepsis Clinical Investigation: Emergency and Critical Care / Out-of-Hospital Cardiac Arrest / Neurological Disorders / Surgery

IML Method

IML Method: Scoring System

• Xie F, Chakraborty B, Ong MEH, Goldstein B, Liu N. AutoScore: A machine learning-based automatic clinical score generator and its application to mortality prediction using electronic health records. JMIR Medical Informatics 2020; 8(10): e21798. [Code] AutoScore
• Xie F, Ning Y, Yuan H, Goldstein BA, Ong MEH, Liu N, Chakraborty B. AutoScore-Survival: developing interpretable machine learning-based time-to-event scores with right-censored survival data. Journal of Biomedical Informatics 2022 Jan; 125: 103959. [Code] AutoScore-Survival
• Yuan H, Xie F, Ong MEH, Ning Y, Chee ML, Saffari SE, Abdullah HR, Goldstein BA, Chakraborty B, Liu N. AutoScore-Imbalance: An interpretable machine learning tool for development of clinical scores with rare events data. Journal of Biomedical Informatics 2022 May; 129: 104072. [Code] AutoScore-Imbalance
• Ning Y, Li S, Ong MEH, Xie F, Chakraborty B, Ting DSW, Liu N. A novel interpretable machine learning system to generate clinical risk scores: An application for predicting early mortality or unplanned readmission in a retrospective cohort study. PLOS Digital Health 2022 Jun; 1(6): e0000062. [Code] AutoScore-ShapleyVIC
• Saffari SE, Ning Y, Xie F, Chakraborty B, Volovici V, Vaughan R, Ong MEH, Liu N. AutoScore-Ordinal: An interpretable machine learning framework for generating scoring models for ordinal outcomes. BMC Medical Research Methodology 2022 Nov; 22: 286. [Code] AutoScore-Ordinal
• Li S, Ning Y, Ong MEH, Chakraborty B, Hong C, Xie F, Yuan H, Liu M, Buckland DM, Chen Y, Liu N. FedScore: A privacy-preserving framework for federated scoring system development. Journal of Biomedical Informatics 2023. [Code] FedScore
• Xie F, Ning Y, Liu M, Li S, Saffari SE, Yuan H, Volovici V, Ting DSW, Goldstein BA, Ong MEH, Vaughan R, Chakraborty B, Liu N. A universal AutoScore framework to develop interpretable scoring systems for predicting common types of clinical outcomes. STAR Protocols 2023 Jun; 4(2): 102302. [Code] AutoScore protocol

IML Method: Shapley Value

• Ning Y, Ong MEH, Chakraborty B, Goldstein BA, Ting DSW, Vaughan R, Liu N. Shapley variable importance cloud for interpretable machine learning. Patterns 2022 Apr; 3: 100452. [Code] ShapleyVIC
• Liu M, Ning Y, Yuan H, Ong MEH, Liu N. Balanced background and explanation data are needed in explaining deep learning models with SHAP: An empirical study on clinical decision making. arXiv:2206.04050. [Code] BalanceSHAP

IML Application

IML Application: Emergency Medicine

• Xie F, Ong MEH, Liew JNMH, Tan KBK, Ho AFW, Nadarajan GD, Low LL, Kwan YH, Goldstein BA, Matchar DB, Chakraborty B, Liu N. Development and assessment of an interpretable machine learning triage tool for estimating mortality after emergency admissions. JAMA Network Open 2021 Aug; 4(8): e2118467.
• Liu N, Xie F, Siddiqui FJ, Ho AFW, Chakraborty B, Nadarajan GD, Tan KBK, Ong MEH. Leveraging large-scale electronic health records and interpretable machine learning for clinical decision making at the emergency department: Protocol for system development and validation. JMIR Research Protocols 2022 Mar; 11(3): e34201.
• Xie F, Liu N, Yan L, Ning Y, Lim KK, Gong C, Ho AFW, Low LL, Chakraborty B, Ong MEH. Development and validation of an interpretable machine learning scoring tool for estimating time to emergency readmissions. eClinicalMedicine 2022 Mar; 45: 101315.
• Xie F, Zhou J, Lee JW, Tan M, Li SQ, Rajnthern L, Chee ML, Chakraborty B, Wong AKI, Dagan A, Ong MEH, Gao F, Liu N. Benchmarking emergency department triage prediction models with machine learning and large public electronic health records. Scientific Data 2022 Oct; 9: 658. [Code] MIMIC benchmark

IML Application: Obstetrics and Gynecology

• Tan HS, Liu N, Tan CW, Sia ATH, Sng BL. Developing the BreakThrough Pain Risk Score: an interpretable machine-learning-based risk score to predict breakthrough pain with labour epidural analgesia. Canadian Journal of Anesthesia 2022 Oct; 69(10): 1315-1317.

IML Application: Out-of-Hospital Cardiac Arrest

• Wong XY, Ang YK, Li K, Chin YH, Lam SSW, Tan KBK, Chua MCH, Ong MEH, Liu N, Pourghaderi AR, Ho AFW. Development and validation of the SARICA score to predict survival after return of spontaneous circulation in out of hospital cardiac arrest using an interpretable machine learning framework. Resuscitation 2022 Jan; 170: 126-133.
• Liu N, Liu M, Chen X, Ning Y, Lee JW, Siddiqui FJ, Saffari SE, Matthew M, Shin SD, Tanaka, Ho AFW, Ong MEH. Development and validation of interpretable prehospital return of spontaneous circulation (P-ROSC) score for out-of-hospital cardiac arrest patients using machine learning. eClinicalMedicine 2022 Jun; 48: 101422.

IML Application: Renal Medicine

• Ang Y, Li S, Ong MEH, Xie F, Teo SH, Choong L, Koniman R, Chakraborty B, Ho AFW, Liu N. Development and validation of an interpretable clinical score for early identification of acute kidney injury at the emergency department. Scientific Reports 2022 May; 12: 7111.

IML Application: Trauma

• Yu JY, Heo S, Xie F, Liu N, Yoon SY, Chang HS, Kim T, Lee SU, Ong MEH, Ng YY, Shin SD, Kajino K, Cha WC. Development and Asian-wide validation of the Grade for Interpretable Field Triage (GIFT) for predicting mortality in pre-hospital patients using the Pan-Asian Trauma Outcomes Study (PATOS). The Lancet Regional Health – Western Pacific 2023.

AI in Health

AI Fairness & Ethics

• Liu M, Ning Y, Teixayavong S, Mertens M, Xu J, Ting DSW, Cheng LTE, Ong JCL, Teo ZL, Tan TF, RaviChandran N, Wang F, Celi LA, Ong MEH, Liu N. A translational perspective towards clinical AI fairness. npj Digital Medicine 2023 Sep; 6: 172.
• Ning Y, Volovici V, Ong MEH, Goldstein BA, Liu N. A roadmap to fair and trustworthy prediction model validation in healthcare. arXiv:2304.03779.

Review and Perspective

• Xie F, Yuan H, Ning Y, Ong MEH, Feng M, Hsu W, Chakraborty B, Liu N. Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies. Journal of Biomedical Informatics 2022 Feb; 126: 103980.
• Volovici V, Syn NL, Ercole A, Zhao JJ, Liu N. Steps to avoid overuse and misuse of machine learning in clinical research. Nature Medicine 2022 Oct; 28(10): 1996-1999.
• Liu M, Li S, Yuan H, Ong MEH, Ning Y, Xie F, Saffari SE, Shang Y, Volovici V, Chakraborty B, Liu N. Handling missing values in healthcare data: A systematic review of deep learning-based imputation techniques. Artificial Intelligence in Medicine 2023 Aug; 142: 102587.
• Li S, Liu P, Nascimento G, Wang X, Leite FRM, Chakraborty B, Hong C, Ning Y, Xie F, Teo ZL, Ting DSW, Haddadi H, Ong MEH, Peres MA, Liu N. Federated and distributed learning applications for electronic health records and structured medical data: A scoping review. Journal of the American Medical Informatics Association 2023.

Large Language Model

• Yang R, Tan TF, Lu W, Thirunavukarasu AJ, Ting DSW, Liu N. Large language models in health care: Development, applications, and challenges. Health Care Science 2023 Aug; 2(4): 255-263.

Time Series

• Panja M, Chakraborty T, Nadim SS, Ghosh I, Kumar U, Liu N. An ensemble neural network approach to forecast Dengue outbreak based on climatic condition. Chaos, Solitons and Fractals 2023 Feb; 167: 113124.
• Panja M, Chakraborty T, Kumar U, Liu N. Epicasting: An ensemble wavelet neural network for forecasting epidemics. Neural Networks 2023 Aug; 165: 185-212.

HRV Method

HRV Method: Heart Rate Variability

• Liu N, Guo DG, Koh ZX, Ho AFW, Xie F, Tagami T, Sakamoto JT, Pek PP, Chakraborty B, Lim SH, Tan JWC, Ong MEH. Heart rate n-variability (HRnV) with its application to risk stratification of chest pain patients in the emergency department. BMC Cardiovascular Disorders 2020; 20: 168. [Code] HRnV algorithm
• Niu C, Guo D, Ong MEH, Koh ZX, Marie-Alix GAL, Ho AFW, Lin Z, Liu C, Clifford DG, Liu N. HRnV-Calc: A software for heart rate n-variability and heart rate variability analysis. Journal of Open Source Software 2023 May; 8(85): 5391. [Code] HRnV software

HRV Application

HRV Application: Acute Coronary Syndrome

• Liu N, Lin Z, Cao J, Koh ZX, Zhang T, Huang GB, Ser W, Ong MEH. An intelligent scoring system and its application to cardiac arrest prediction. IEEE Transactions on Information Technology in Biomedicine 2012; 16(6): 1324-1331.
• Liu N, Koh ZX, Chua EC, Tan LM, Lin Z, Mirza B, Ong MEH. Risk scoring for prediction of acute cardiac complications from imbalanced clinical data. IEEE Journal of Biomedical and Health Informatics 2014; 18(6): 1894-1902.
• Liu N, Koh ZX, Goh J, Lin Z, Haaland B, Ting BP, Ong MEH. Prediction of adverse cardiac events in emergency department patients with chest pain using machine learning for variable selection. BMC Medical Informatics and Decision Making 2014; 14(1): 75.
• Liu T, Lin Z, Ong MEH, Koh ZX, Pek PP, Yeo YK, Oh B, Ho AFW, Liu N. Manifold ranking based scoring system with its application to cardiac arrest prediction: a retrospective study in emergency department patients. Computers in Biology and Medicine 2015; 67: 74-82.
• Heldeweg MLA, Liu N, Koh ZX, Fook-Chong S, Lye WK, Harms M, Ong MEH. A novel cardiovascular risk stratification model incorporating ECG and heart rate variability for patients presenting to the emergency department with chest pain. Critical Care 2016; 20(1): 179.
• Liu N, Sakamoto JT, Cao J, Koh ZX, Ho AFW, Lin Z, Ong MEH. Ensemble-based risk scoring with extreme learning machine for prediction of adverse cardiac events. Cognitive Computation 2017; 9(4): 545-554.
• Sakamoto JT, Liu N, Koh ZX, Guo DG, Heldeweg MLA, Ng JCJ, Ong MEH. Integrating heart rate variability, vital signs, electrocardiogram, and troponin to triage chest pain patients in the ED. American Journal of Emergency Medicine 2018; 36(2): 185-192.23.
• Liu N, Chee ML, Koh ZX, Leow SL, Ho AFW, Guo DG, Ong MEH. Utilizing machine learning dimensionality reduction for risk stratification of chest pain patients in the emergency department. BMC Medical Research Methodology 2021 Apr; 21: 74.

HRV Application: Sepsis

• Liu N, Chee ML, Foo M, Pong JZ, Guo DG, Koh ZX, Niu C, Chong SL, Ong MEH. Heart rate n-variability (HRnV) measures for prediction of mortality in sepsis patients presenting at the emergency department. PLOS ONE 2021 Aug; 16(8): e0249868.
• Chong SL, Ong G, Allen JC, Lee JH, Piragasam R, Koh Z, Mahajan P, Liu N, Ong MEH. Early prediction of serious infections in febrile infants incorporating heart rate variability in an emergency department: a pilot study. Emergency Medicine Journal 2021 Aug; 38(8): 607-612.
• Chong SL, Niu C, Piragasam R, Koh ZX, Guo D, Lee JH, Ong GYK, Ong MEH, Liu N. Adding heart rate n-variability (HRnV) to clinical assessment potentially improves prediction of serious bacterial infections in young febrile infants at the emergency department: a prospective observational study. Annals of Translational Medicine 2023 Jan; 11(1): 6.

Clinical Investigation

Emergency and Critical Care

• Zhang Z, Zheng B, Liu N, Ge H, Hong Y. Mechanical power normalized to predicted body weight as a predictor of mortality in patients with acute respiratory distress syndrome. Intensive Care Medicine 2019; 45(6): 856-864.
• Parker CA, Liu N, Wu SX, Shen Y, Lam SSW, Ong MEH. Predicting hospital admission at the emergency department: a novel prediction model. American Journal of Emergency Medicine 2019; 37(8): 1498-1504.
• Yeo CFC, Li H, Koh ZX, Liu N, Ong MEH. Risk stratification of patients with atrial fibrillation in the emergency department. American Journal of Emergency Medicine 2020 Sep; 38(9): 1807-1815.
• Zhang Z, Zheng B, Liu N. Individualized fluid administration for critically ill patients with sepsis with an interpretable dynamic treatment regimen model. Scientific Reports 2020 Oct; 10: 17874.
• Zhang Z, Navarese EP, Zheng B, Meng Q, Liu N, Ge H, Pan Q, Yu Y, Ma X. Analytics with artificial intelligence to advance the treatment of acute respiratory distress syndrome. Journal of Evidence-Based Medicine 2020 Nov; 13(4): 301-312.
• Hu Z, Siddiqui FJ, Fan Q, Lian SWQ, Liu N, Ong MEH. Trends of chronic illness in emergency department admissions among elderly adults in a tertiary hospital over ten years. BMC Health Services Research 2021 Dec; 21: 1305.
• Chan SL, Lee JW, Ong MEH, Siddiqui FJ, Graves N, Ho AFW, Liu N. Implementation of prediction models in the emergency department from an implementation science perspective – determinants, outcomes and real-world impact: A scoping review. Annals of Emergency Medicine 2023.

Out-of-Hospital Cardiac Arrest

• Ng YY, Wah W, Liu N, Zhou SA, Ho AFW, Pek PP, Shin SD, Tanaka H, Khunkhlai N, Lin CH, Wong KD, Cai WW, Ong MEH. Associations between gender and cardiac arrest outcomes in Pan-Asian out-of-hospital cardiac arrest patients. Resuscitation 2016; 102: 116-121.
• Liu N, Ong MEH, Ho AFW, Pek PP, Lu TC, Khruekarnchana P, Song KJ, Tanaka H, Naroo GY, Gan HN, Koh ZX, Ma MHM. Validation of the ROSC after cardiac arrest (RACA) score in Pan-Asian out-of-hospital cardiac arrest patients. Resuscitation 2020 Apr; 149: 53-59.
• Teoh SE, Masuda Y, Tan DJH, Liu N, Morrison L, Ong MEH, Blewer AL, Ho AFW. Impact of the COVID-19 pandemic on the epidemiology of out-of-hospital cardiac arrest: A systematic review and meta-analysis. Annals of Intensive Care 2021 Dec; 11: 169.
• Yeo JW, Ng ZHC, Goh AXC, Gao JF, Liu N, Lam SSW, Chia YW, Perkins GD, Ong MEH, Ho AFW. Impact of cardiac arrest centers on the survival of patients with nontraumatic out-of-hospital cardiac arrest: A systematic review and meta-analysis. Journal of the American Heart Association 2022 Jan; 11: e023806.
• Liu N, Ning Y, Ong MEH, Saffari SE, Ryu HH, Kajino K, Lin CH, Sarah AK, Rao GVR, Ho AFW, Lim SL, Siddiqui FJ. Gender disparities among adult recipients of layperson bystander cardiopulmonary resuscitation by location of cardiac arrest in Pan-Asian communities: A registry-based study. eClinicalMedicine 2022 Feb; 44: 101293.
• Goh AXC, Seow J, Lai M, Liu N, Goh YM, Ong MEH, Lim SL, Yeo JW, Ho AFW. Association of high-volume centers on survival outcomes among nontraumatic out-of-hospital cardiac arrest patients: a systematic review and meta-analysis. JAMA Network Open 2022 May; 5(5): e2214639.
• Liu N, Wnent J, Lee JW, Ning Y, Ho AFW, Siddiqui FJ, Lim SL, Chia MYC, Tiah L, Mao DRH, Gräsner JT, Ong MEH. Validation of the cardiac arrest survival score (CRASS) for predicting good neurological outcome after out-of-hospital cardiac arrest in an Asian emergency medical service system. Resuscitation 2022 Jul; 176: 42-50.
• Lam TJR, Yang J, Poh JE, Ong MEH, Liu N, Yeo JW, Gräsner JT, Masuda Y, Ho AFW. Long term risk of recurrence among survivors of sudden cardiac arrest: a systematic review and meta-analysis. Resuscitation 2022 Jul; 176: 30-41.
• Ho AFW, Ting PZY, Ho JSY, Fook-Chong S, Shahidah N, Pek PP, Liu N, En ST, Chia CH, Lim DYZ, Lim SL, Wong TH, Ong MEH. The effect of building-level socioeconomic status on bystander cardiopulmonary resuscitation: a retrospective cohort study. Prehospital Emergency Care 2023 Feb; 27(2): 205-212.

Neurological Disorders

• Deng X, Saffari SE, Liu N, Xiao B, Allen JC, Ng SYE, Chia N, Tan YJ, Choi X, Heng DL, Lo YL, Xu Z, Tay KY, Au WL, Ng A, Tan EK, Tan LCS. Biomarker characterization of clinical subtypes of Parkinson Disease. npj Parkinson’s Disease 2022 Aug; 8: 109.
• Deng X, Saffari SE, Ng SYE, Chia N, Tan JY, Choi X, Heng DL, Xu Z, Tay KY, Au WL, Liu N, Ng A, Tan EK, Tan LCS. Blood lipid biomarkers in early Parkinson disease and Parkinson disease with mild cognitive impairment. Journal of Parkinson’s Disease 2022 Sep; 12(6): 1937-1943.
• Ho AFW, Tan BYQ, Zheng H, Leow AST, Pek PP, Liu N, Raju Y, Yeo LLL, Sharma VK, Ong MEH, Aik J. Association of air pollution with acute ischemic stroke risk in Singapore: A time-stratified case-crossover study. International Journal of Stroke 2022 Oct; 17(9): 983-989.

Surgery

• Chiew CJ, Liu N, Wong TH, Sim YE, Abdullah HR. Utilizing machine learning methods for preoperative prediction of postsurgical mortality and intensive care unit admission. Annals of Surgery 2020 Dec; 272(6): 1133-1139.
• Singh M, Chiang J, Seah A, Liu N, Mathew R, Mathur S. A clinical predictive model for risk stratification of patients with severe acute lower gastrointestinal bleeding. World Journal of Emergency Surgery 2021 Nov; 16: 58.