Named entity recognition in electronic health records using transfer learning bootstrapped Neural Networks

Luka Gligic, Andrey Kormilitzin, Paul Goldberg, Alejo J Nevado-Holgado
November 2021
Cite DOI

Abstract

Neural networks (NNs) have become the state of the art in many machine learning applications, such as image, sound (LeCun et al., 2015) and natural language processing (Young et al., 2017; Linggard et al., 2012). However, the success of NNs remains dependent on the availability of large labelled datasets, such as in the case of electronic health records (EHRs). With scarce data, NNs are unlikely to be able to extract this hidden information with practical accuracy. In this study, we develop an approach that solves these problems for named entity recognition, obtaining 94.6 F1 score in I2B2 2009 Medical Extraction Challenge (Uzuner et al., 2010), 4.3 above the architecture that won the competition. To achieve this, we bootstrap our NN models through transfer learning by pretraining word embeddings on a secondary task performed on a large pool of unannotated EHRs and using the output embeddings as a foundation of a range of NN architectures. Beyond the official I2B2 challenge, we further achieve 82.4 F1 on extracting relationships between medical terms using attention-based seq2seq models bootstrapped in the same manner.

Type
Journal article
Neural Networks NLP Named entity recognition NER Electronic health records EHR Transfer learning LSTM
Andrey Kormilitzin
Andrey Kormilitzin
Senior Researcher

My research is centred around translating advances in mathematics, statistical machine learning and deep learning to address challenges involved in learning, inference and ethical decision making using complex biomedical and health data.

Alejo J Nevado-Holgado
Alejo J Nevado-Holgado
Associate Professor

I am an Associate Professor of the Department of Psychiatry and the Big Data Institute, and part of Dementia Research Oxford. I am very glad to supervise the AI team in the TNDR, formed by 10 excellent machine learners and bioinformaticians. Our focus is on the applications of machine learning and bioinformatics to mental health care. In addition, I also hold a position at the Big Data Institute, where we collaborate in the application of machine learning to genomics and target discovery. I am also consultant to a number of AI companies.

Related