Deep Reinforcement Learning for Medical Imaging

Why we organize this tutorial:

Reinforcement learning is a framework for learning a sequence of actions that maximizes the expected reward. Deep reinforcement learning (DRL) is the result of marrying deep learning with reinforcement learning. DRL allows reinforcement learning to scale up to problems that were previously intractable. Deep learning and reinforcement learning were selected by MIT Technology Review as one of 10 Breakthrough Technologies in 2013 and 2017, respectively. The combination of these two powerful technologies currently constitutes one of the state-of-the-art frameworks in artificial intelligence.

Recent years have witnessed rapid progresses in DRL, resulting in significant performance improvement in many areas including games, robotics, natural language processing, and computer vision. Unlike supervised learning, DRL framework can deal with sequential decisions, and learn with highly delayed supervised information (e.g., success or failure of the decision is available only after multiple time steps). DRL can also deal with non-differentiable metrics. For example, one can use DRL to search for an optimal deep network architecture or parameter settings to maximize the classification accuracy, which is clearly non-differentiable with respect to the number of layers or the choice of non-linear rectifier functions. Another use of DRL is in finding efficient search sequence for speeding up detection, or optimal transformation sequence for improving registration accuracy.

Despite its successes, application of this technology to medical image analysis remains to be fully explored. The goal of this tutorial is to bridge the gap by providing a comprehensive introduction to deep reinforcement learning methods in terms of theories, practice, and future directions. The tutorial will invite leading researchers in DRL to present its state-of-the-art and explain in-depth how DRL was applied in a selected set of topics such as neural architecture search, detection, segmentation, and controlling of surgical robots.

Date and Time: Sunday Afternoon, 16th September

What will be covered:
The tutorial will last 4 hours, consisting of two lectures, a plenary talk, and invited talks on selected topics, and a short coffee break. The structure is tentatively as follows and will be subject to changes depending on real situations:

Introduction to Reinforcement Learning [Slides]
Review of Deep Learning [Slides]
Introduction to Deep Reinforcement Learning [Slides]
Automatic View Planning with Multi-scale Deep Reinforcement Learning Agents [Slides]
Parametric Detection and Registration Using Deep Reinforcement Learning [Slides]
Reinforcement Learning for Surgical Robots Control [Slides available soon]
Face Aging Prediction Using Inverse Reinforcement Learning [Slides]

Snapshots of the Event:

About Organizers

Dr. S. Kevin Zhou is currently a Principal Key Expert of Image Analysis at Siemens Healthineers Technology. His research interest lies in machine learning and their applications to medical image analysis. Dr. Zhou has published 150+ book chapters, journals, and conference papers, 250+ patents and inventions, 2 research monographs, and 3 books. Dr. Zhou has won multiple awards, including Best Paper Awards (2010), Thomas Alva Edison Patent Award (2013) from NJ R&D Council, R&D 100 Award or Oscar of Invention (2014). He has been an associate editor for TMI and Medical Image Analysis journals, an area chair for MICCAI, IPMI, and CVPR, and an AIMBE fellow.
Dr. Hien V. Nguyen is an Assistant Professor of Electrical and Computer Engineering Department, University of Houston. His research interest lies in machine learning and medical image understanding. He has co-authored 30+ journals, conference papers, and book chapters, 10+ patents and inventions. He co-organized the first MICCAI deep learning tutorial for medical imaging, and has served as a reviewer for 20+ top-tier conferences and journals, including CVPR, ICCV, MICCAI, TPAMI.
Dr. Khoa Luu is the Research Project Director in Cylab Biometrics Center at Carnegie Mellon University. He is the editor-in-chief of International Journal of Cognitive Biometrics. His research is in biometrics and computer vision systems. He has published 60+ papers in top-tier conferences and journals and received two best paper awards in IEEE International Conference on Biometrics: Theory, Applications and Systems in 2011 and 2012.
Dr. Animesh Garg is a Postdoctoral Researcher at Stanford AI lab. Dr. Garg studies data-driven automation for human skill-augmentation in medicine and personal care. He received Best Applications Paper Award at IEEE CASE, Best Video at Hamlyn Symposium on Surgical Robotics, and Best Paper Nomination at IEEE ICRA 2015. His work was featured by New York Times, UC Health, UC CITRIS News, and BBC Click.
Dr. Hoang Ngan Le is a Researcher at Cylab Biometrics Center at Carnegie Mellon University. Her current research interests are level sets, scene understanding, semantic segmentation, facial recognition, medical analysis, deep learning, computer vision.