The “sound” of ML implementation. A round trip in the “last mile” and its challenges.
Federico Cabitza, PhD, Associate Professor, University of Milano-Bicocca, Italy
Watch the video here: https://vimeo.com/452529854
Abstract: Achieving a pragmatic, or even an ecological validation (Cabitza and Zeitoun, 2019) of medical AI systems that nevertheless exhibit very high (statistical) accuracy has been observed to be more complicated than initially expected (Coiera et al. 2018): in fact, most of the challenges that make technically sound systems perform poorly in real-world settings lie in the so called “last mile of implementation” (Coiera, 2019). This evocative concept expresses the semantic difference between developing medical machine learning (or medical AI) and the mere application of machine learning techniques to medical data. Moreover, we will make the point that this “last mile”, although apparently short, is not a flat and regular path, but rather presents two chasms: the hiatus of human trust, and the hiatus of machine experience. The former one requires to focus on usability and causability (Holzinger et al 2019), while the latter ones requires data governance and to focus on data work, including practice of “data awareness” and “data hygiene”. I will discuss these notions, report about some researches performed in the above “last mile” and propose some ways to see how we can bridge across this “sound” spanning between development and deployment of medical AI, or at least navigate in it without too much trouble and the danger of sinking.
Federico Cabitza received his Master in IT Engineering at the Politecnico of Milan in 2001 with an experimental thesis on subsymbolic AI. In 2007, he received a PhD in Informatics from the University of Milano-Bicocca, with a thesis on Electronic Patient Record and coordination in hospital setting. Currently, he is an Associate Professor at the University of Milano-Bicocca (Milan, Italy) where he teaches Human-Computer Interaction and where he coordinates the research activities of the MUDI Lab (Modelling Uncertainty, Decisions and Interaction). Since 2016 he has also had a research appointment with the IRCCS Orthopaedics Institute Galeazzi in Milano (Italy). He is stable PC member of several international conferences and editor of the International Journal of Medical Informatics (IJMI, ISSN: 1386-5056), the Annals of Translational Medicine (Ann Transl Med; ATM; Print ISSN 2305-5839; Online ISSN 2305-5847) and the Journal of Medical Artificial Intelligence (ISSN 2617-2496). He is the author of more than 130 research publications to date, in international conference proceedings, edited books and scientific journals, including the JAMA, CSCW Journal, Computers in Biology and Medicine, Behaviour and Information Technology, International Journal of Human Computer Studies, Computers in Human Behavior, International Journal of Approximate Reasoning, and the Journal of Visual Languages and Computing.