Bayesian Deep Learning | Hien Van Nguyen

Why we organize this tutorial:

The past few years have witnessed rapid progress in deep learning, resulting in significant performance improvement in numerous medical image analysis tasks including detection of anatomical landmarks, classification of pathological findings, semantic segmentation of multiple organs, and automatic generation of medical reports. While the majority of work in deep learning focuses on improving the final performance, understanding when a deep network does not work well is critical for many medical and healthcare systems, especially those with high safety standards. Unfortunately, most modern deep learning algorithms are unable to estimate the uncertainty of deep networks reliably. Without a fail-safe mode for when a model is highly uncertain, the system can have disastrous behaviors such as missing obvious abnormalities or containing racial discrimination.
Recently, there has been an increasing interest in combining Bayesian methods with deep neural networks to enable estimating the confidence of the model’s prediction. While the conventional approach views a deep network as a deterministic function that produces only a single output for an input. In contrast, Bayesian deep learning computes a distribution of output for each input by taking into account the randomness inherent in the training data and the modeling parameters. This distribution allows estimating the confidence level of the output. New methods based on stochastic regularization techniques like dropout or scalable Monte Carlo interference have been shown to capture meaningful uncertainty while scaling well to high-dimensional data. The revisit of Bayesian techniques in light of deep learning has created many promising results.
Despite its importance, this topic has been largely under-investigated in MICCAI community. The goal of this tutorial is to bridge the gap by providing a comprehensive introduction to Bayesian deep learning methods in terms of theories, practice, and future directions. The tutorial will invite leading researchers in Bayesian deep learning to present its state-of-the-art and explain in-depth how the techniques were applied in a selected set of topics image detection, segmentation, and radiotherapy. The recent Bayesian deep learning workshop at the 2018 Neural Information Processing Systems conference attracts a large number of paper submissions and audiences. Our tutorial is expected to generate a similar level of interest in MICCAI.

Date and Time: 12:30pm - 16:30pm, October 17 (Sunday), 2019

Location: Room Espana II

What will be covered:

The tutorial will last 3.5 hours, consisting of lectures on selected topics of Bayesian Deep Learning, practical demos on skin lesion classification and CT organ segmentation, and a short coffee break. The tentative structure is as follows:

a) Introduction to Bayesian Modeling and Variational Inference [60 minutes, Pengyu "Ben" Yuan] [Slides]

Introduction & basic Bayesian rule
Non-Bayesian machine learning methods
Variational Inference
Gaussian Process

b) Bayesian deep learning [60 minutes, Dan Nguyen] [Slides]

Uncertainty in model predictions
Bayesian deep learning with dropout
Some practical application examples

c) Coffee Break [15 minutes]

d) DropConnect Is Effective in Modeling Uncertainty of Bayesian Deep Networks [30 minutes, Pengyu "Ben" Yuan] [Slides]

e) Bayesian deep learning demos [45 minutes, Pengyu "Ben" Yuan]

Skin lesion classification demo
Organ segmentation demo

Learning Objectives:

Graduate students and researchers new to Bayesian deep learning can use this tutorial to quickly understand important concepts and techniques. The tutorial is designed to provide a solid understanding of the theory, and a concise review of recent advances in Bayesian deep learning. In addition, medical researchers will gain a better understanding of how these techniques have been applied to solve challenging medical data analysis tasks.

About Organizers:

Dr. Hien Nguyen is an Assistant Professor in the Department of Electrical and Computer Engineering, University of Houston.
Dr. S. Kevin Zhou (Fellow AIMBE) is a Professor at the Institute of Computing Technology, Chinese Academy of Sciences. He was a Principal Expert of Image Analysis and a Senior R&D director at Siemens Healthcare Technology.
Dr. Vishal M. Patel is an Assistant Professor in the Department of Electrical and Computer Engineering at Johns Hopkins University.
Dr. Ngan Le is currently a research associate in the Department of Electrical and Computer Engineering at Carnegie Mellon University (CMU).
Dr. Dan Nguyen is an Assistant Professor of the Medical Artificial Intelligence and Automation Laboratory, Department of Radiation Oncology, UT Southwestern Medical Center.
Pengyu "Ben" Yuan is currently a Ph.D. student in Houston Learning Algorithms (HULA) lab. His research interests are in meta-learning and reinforcement learning with applications in medical image analysis.