Linked e-resources
Details
Table of Contents
Front Cover
Biomedical Image Synthesis and Simulation
Copyright
Contents
Contributors
Preface
1 Introduction to medical and biomedical image synthesis
Part 1 Methods and principles
2 Parametric modeling in biomedical image synthesis
2.1 Introduction
2.2 Parametric modeling paradigm
2.2.1 Modeling of the cellular objects
2.2.1.1 Generic parameter-controlled shape modeling: random shape model for nucleus and cell body
2.2.1.2 Cell-type specific parametric shape models
2.2.1.3 Modeling appearance: texture and subcellular organelle models
2.2.1.4 Modeling spatial distribution and populations
2.2.2 Modeling microscopy and image acquisition: from object models to simulated microscope images
2.3 On learning the parameters
2.4 Use cases
2.4.1 SIMCEP: parametric modeling framework aimed for generating and understanding microscopy images of cells
2.4.2 Simulated data for benchmarking
2.5 Future directions
2.6 Summary
Acknowledgments
References
3 Monte Carlo simulations for medical and biomedical applications
3.1 Introduction
3.1.1 A brief history
3.1.2 Monte Carlo method and biomedical physics
3.2 Underlying theory and principles
3.3 Particle transport through matter
3.3.1 Photon physics effects
3.3.2 Cross-section and mean free path
3.3.3 Models
3.3.4 Particle transport
3.4 Monte Carlo simulation structure
3.4.1 Particle source model
3.4.1.1 Analytical source
3.4.1.2 Voxelized source
3.4.1.3 Cumulative density function
3.4.1.4 Time management
3.4.1.5 Phase space
3.4.2 Digitized phantom
3.4.2.1 Matter composition
3.4.2.2 Analytical geometry
3.4.2.3 Voxelized geometry
3.4.2.4 Tessellated geometry
3.4.2.5 Mixed geometry
3.4.2.6 Hierarchical geometry and space partitioning data structure
3.4.3 Particle detector.
3.5 Running a Monte Carlo simulation
3.6 Improving Monte Carlo simulation efficiency
3.6.1 Woodcock tracking
3.6.2 GPU
3.6.3 Fixed force detection
3.6.4 Angular response functions
3.7 Examples of Monte Carlo simulation applications in medical physics
3.8 Monte Carlo simulation for computational biology
3.8.1 Generalization of the Monte Carlo method
3.8.2 Examples of computational biology applications
3.9 Summary
References
4 Medical image synthesis using segmentation and registration
4.1 Introduction
4.2 Segmentation-based image synthesis
4.2.1 Segmentation approaches
4.2.1.1 Manual segmentation
4.2.1.2 Automatic segmentation
4.2.2 Intensity assignment approaches
4.2.2.1 Segmentation methods with bulk assignment
4.2.2.2 Segmentation methods with subject-specific assignment
4.3 Registration-based image synthesis
4.3.1 Single-atlas registration approaches
4.3.1.1 Direct multimodal registration
4.3.1.2 Indirect unimodal registration
4.3.2 Multi-atlas registration approaches
4.3.3 Combination of registration and regression approaches
4.4 Hybrid approaches combining segmentation and registration
4.5 Future directions and research challenges
4.6 Summary
Acknowledgments
References
5 Dictionary learning for medical image synthesis
5.1 Introduction
5.2 Sparse coding
5.2.1 Orthogonal matching pursuit
5.3 Dictionary learning
5.4 Medical image synthesis with dictionary learning
5.5 Future directions and research challenges
5.6 Summary
Acknowledgments
References
6 Convolutional neural networks for image synthesis
6.1 Convolutional neural networks for image synthesis
6.2 Neural network building blocks
6.2.1 Neuron
6.2.2 Activation function
6.2.3 Generator layer details
6.3 Training a convolutional neural network.
6.3.1 Loss functions
6.3.2 Back propagation
6.3.3 Image synthesis accuracy
6.4 Practical aspects
6.4.1 Pooling layers
6.4.2 Convolutional versus fully connected neural networks
6.4.3 Vanishing gradient
6.5 Commonly known networks
6.5.1 AlexNet
6.5.2 UNet
6.5.3 Inception network
6.6 Conclusion
References
7 Generative adversarial networks for medical image synthesis
7.1 Introduction
7.2 Generative adversarial networks
7.2.1 Network architecture
7.2.1.1 Deep convolutional GANs
7.2.2 Loss function
7.2.2.1 Discriminator loss
7.2.2.2 Adversarial loss
7.2.3 Challenges of training GANs
7.3 Conditional GANs
7.3.1 Network architecture
7.3.2 Loss function
7.3.2.1 Image distance loss
7.3.2.2 Histogram matching loss
7.3.2.3 Perceptual loss
7.3.3 Variants of cGANs
7.3.3.1 Pix2pix
7.3.3.2 InfoGAN
7.4 Cycle GAN
7.4.1 Network architecture
7.4.2 Loss function: cycle consistency loss
7.4.3 Variants of Cycle GAN
7.4.3.1 Residual Cycle-GAN
7.4.3.2 Dense Cycle-GAN
7.4.3.3 Unsupervised image-to-image translation networks (UNIT)
7.4.3.4 Bicycle-GAN
7.4.3.5 StarGAN
7.5 Practical aspects
7.5.1 Network input dimension and size
7.5.2 Pre-processing
7.5.3 Data augmentation
7.6 CGAN and Cycle-GAN applications
7.6.1 Multi-modal MRI synthesis
7.6.2 MRI-only radiation therapy treatment planning
7.6.3 Image quality improvement/enhancement
7.6.4 Cell synthesis
7.7 Summary and discussion
Disclosures
References
8 Autoencoders and variational autoencoders in medical image analysis
8.1 Introduction
8.1.1 History of the method
8.1.2 Autoencoders and variational autoencoders in biomedical image analysis and synthesis
8.1.3 Outline of this chapter and notation
8.2 Autoencoders
8.2.1 Regularized autoencoders.
8.2.1.1 Sparse autoencoders
8.2.1.2 Contractive autoencoders
8.2.1.3 Denoising autoencoders
8.2.2 Summary
8.3 Variational autoencoders
8.3.1 The evidence lower bound (ELBO)
8.3.2 Implementation and optimization of variational autoencoders
8.3.3 Advantages and challenges of variational autoencoders
8.3.3.1 Current challenges of variational autoencoders
8.3.4 Disentanglement of the latent space
8.3.5 Alternative reconstruction objectives
8.3.6 Improving the flexibility of the model
8.3.6.1 Alternative priors and auxiliary variables
8.3.6.2 Importance weighted autoencoder
8.3.6.3 Adversarial autoencoders
8.4 Example applications
8.4.1 Unsupervised pathology detection
8.4.2 Image synthesis for the explanation of black-box classifiers
8.4.3 Decoupled shape and appearance modeling for multimodal data
8.5 Future directions and research challenges
8.6 Summary
References
Part 2 Applications
9 Optimization of the MR imaging pipeline using simulation
9.1 Overview
9.2 History of MRI simulation
9.2.1 Diffusion MRI
9.3 The POSSUM simulation framework
9.3.1 POSSUM for MRI and functional MRI
9.3.1.1 Modeling artifacts
9.3.2 POSSUM for diffusion MRI
9.4 Applications
9.4.1 Motion correction algorithms for fMRI
9.4.1.1 MCFLIRT algorithm
9.4.1.2 Simulations
9.4.1.3 Results
9.4.2 Motion and eddy-current correction algorithms for diffusion MRI
9.4.3 Investigating the susceptibility-by-movement artifact
9.4.4 Investigating and optimizing image acquisition
9.4.5 Simulated data for machine learning
9.5 Future directions and research challenges
References
10 Synthesis for image analysis across modalities
10.1 General motivation
10.2 Registration
10.2.1 Background
10.2.2 Similarity metrics and their limitations.
10.2.3 Synthesis-based similarity metrics
10.2.4 Other applications of synthesis-based registration
10.3 Segmentation
10.3.1 Background
10.3.2 Domain gap and synthesis-based solutions
10.4 Other directions and perspectives
References
11 Medical image harmonization through synthesis
11.1 Introduction
11.2 Supervised techniques
11.2.1 Architecture and training
11.2.2 Using more information
11.3 Unsupervised techniques
11.3.1 Generative adversarial networks
11.3.2 Learning interpretable representations
11.3.3 One-/few-shot harmonization
11.3.4 Conclusion
References
12 Medical image super-resolution with deep networks
12.1 Introduction to super-resolution
12.1.1 Basic concepts
12.1.2 Brief history of SR methods prior to deep networks
12.1.2.1 SR through mathematical modeling
12.1.2.2 Example-based SR
12.2 SR methods with deep networks
12.2.1 Data acquisition
12.2.1.1 Fully-supervised, unsupervised, and self-supervised learning
12.2.1.2 Multiple network inputs
12.2.2 Network architectures
12.2.2.1 General frameworks
12.2.2.2 Upsampling before or within networks
12.2.2.3 Components in networks
12.2.2.4 Progressive networks
12.2.3 Loss functions
12.2.3.1 Paired losses
12.2.3.2 Unpaired losses
12.3 Applications of super-resolution in medical images
12.3.1 Super-resolution in different image modalities
12.3.1.1 Super-resolution in CT
12.3.1.2 Super-resolution in MRI
12.3.1.3 Super-resolution in optical coherence tomography
12.3.1.4 Super-resolution in microscopy
12.3.2 Super-resolution used for different tasks
12.3.2.1 Super-resolution for image quality enhancement
12.3.2.2 Super-resolution for diagnostic acceptability
12.3.2.3 Super-resolution for segmentation
12.3.2.4 Super-resolution for clinical abnormality detection.
Biomedical Image Synthesis and Simulation
Copyright
Contents
Contributors
Preface
1 Introduction to medical and biomedical image synthesis
Part 1 Methods and principles
2 Parametric modeling in biomedical image synthesis
2.1 Introduction
2.2 Parametric modeling paradigm
2.2.1 Modeling of the cellular objects
2.2.1.1 Generic parameter-controlled shape modeling: random shape model for nucleus and cell body
2.2.1.2 Cell-type specific parametric shape models
2.2.1.3 Modeling appearance: texture and subcellular organelle models
2.2.1.4 Modeling spatial distribution and populations
2.2.2 Modeling microscopy and image acquisition: from object models to simulated microscope images
2.3 On learning the parameters
2.4 Use cases
2.4.1 SIMCEP: parametric modeling framework aimed for generating and understanding microscopy images of cells
2.4.2 Simulated data for benchmarking
2.5 Future directions
2.6 Summary
Acknowledgments
References
3 Monte Carlo simulations for medical and biomedical applications
3.1 Introduction
3.1.1 A brief history
3.1.2 Monte Carlo method and biomedical physics
3.2 Underlying theory and principles
3.3 Particle transport through matter
3.3.1 Photon physics effects
3.3.2 Cross-section and mean free path
3.3.3 Models
3.3.4 Particle transport
3.4 Monte Carlo simulation structure
3.4.1 Particle source model
3.4.1.1 Analytical source
3.4.1.2 Voxelized source
3.4.1.3 Cumulative density function
3.4.1.4 Time management
3.4.1.5 Phase space
3.4.2 Digitized phantom
3.4.2.1 Matter composition
3.4.2.2 Analytical geometry
3.4.2.3 Voxelized geometry
3.4.2.4 Tessellated geometry
3.4.2.5 Mixed geometry
3.4.2.6 Hierarchical geometry and space partitioning data structure
3.4.3 Particle detector.
3.5 Running a Monte Carlo simulation
3.6 Improving Monte Carlo simulation efficiency
3.6.1 Woodcock tracking
3.6.2 GPU
3.6.3 Fixed force detection
3.6.4 Angular response functions
3.7 Examples of Monte Carlo simulation applications in medical physics
3.8 Monte Carlo simulation for computational biology
3.8.1 Generalization of the Monte Carlo method
3.8.2 Examples of computational biology applications
3.9 Summary
References
4 Medical image synthesis using segmentation and registration
4.1 Introduction
4.2 Segmentation-based image synthesis
4.2.1 Segmentation approaches
4.2.1.1 Manual segmentation
4.2.1.2 Automatic segmentation
4.2.2 Intensity assignment approaches
4.2.2.1 Segmentation methods with bulk assignment
4.2.2.2 Segmentation methods with subject-specific assignment
4.3 Registration-based image synthesis
4.3.1 Single-atlas registration approaches
4.3.1.1 Direct multimodal registration
4.3.1.2 Indirect unimodal registration
4.3.2 Multi-atlas registration approaches
4.3.3 Combination of registration and regression approaches
4.4 Hybrid approaches combining segmentation and registration
4.5 Future directions and research challenges
4.6 Summary
Acknowledgments
References
5 Dictionary learning for medical image synthesis
5.1 Introduction
5.2 Sparse coding
5.2.1 Orthogonal matching pursuit
5.3 Dictionary learning
5.4 Medical image synthesis with dictionary learning
5.5 Future directions and research challenges
5.6 Summary
Acknowledgments
References
6 Convolutional neural networks for image synthesis
6.1 Convolutional neural networks for image synthesis
6.2 Neural network building blocks
6.2.1 Neuron
6.2.2 Activation function
6.2.3 Generator layer details
6.3 Training a convolutional neural network.
6.3.1 Loss functions
6.3.2 Back propagation
6.3.3 Image synthesis accuracy
6.4 Practical aspects
6.4.1 Pooling layers
6.4.2 Convolutional versus fully connected neural networks
6.4.3 Vanishing gradient
6.5 Commonly known networks
6.5.1 AlexNet
6.5.2 UNet
6.5.3 Inception network
6.6 Conclusion
References
7 Generative adversarial networks for medical image synthesis
7.1 Introduction
7.2 Generative adversarial networks
7.2.1 Network architecture
7.2.1.1 Deep convolutional GANs
7.2.2 Loss function
7.2.2.1 Discriminator loss
7.2.2.2 Adversarial loss
7.2.3 Challenges of training GANs
7.3 Conditional GANs
7.3.1 Network architecture
7.3.2 Loss function
7.3.2.1 Image distance loss
7.3.2.2 Histogram matching loss
7.3.2.3 Perceptual loss
7.3.3 Variants of cGANs
7.3.3.1 Pix2pix
7.3.3.2 InfoGAN
7.4 Cycle GAN
7.4.1 Network architecture
7.4.2 Loss function: cycle consistency loss
7.4.3 Variants of Cycle GAN
7.4.3.1 Residual Cycle-GAN
7.4.3.2 Dense Cycle-GAN
7.4.3.3 Unsupervised image-to-image translation networks (UNIT)
7.4.3.4 Bicycle-GAN
7.4.3.5 StarGAN
7.5 Practical aspects
7.5.1 Network input dimension and size
7.5.2 Pre-processing
7.5.3 Data augmentation
7.6 CGAN and Cycle-GAN applications
7.6.1 Multi-modal MRI synthesis
7.6.2 MRI-only radiation therapy treatment planning
7.6.3 Image quality improvement/enhancement
7.6.4 Cell synthesis
7.7 Summary and discussion
Disclosures
References
8 Autoencoders and variational autoencoders in medical image analysis
8.1 Introduction
8.1.1 History of the method
8.1.2 Autoencoders and variational autoencoders in biomedical image analysis and synthesis
8.1.3 Outline of this chapter and notation
8.2 Autoencoders
8.2.1 Regularized autoencoders.
8.2.1.1 Sparse autoencoders
8.2.1.2 Contractive autoencoders
8.2.1.3 Denoising autoencoders
8.2.2 Summary
8.3 Variational autoencoders
8.3.1 The evidence lower bound (ELBO)
8.3.2 Implementation and optimization of variational autoencoders
8.3.3 Advantages and challenges of variational autoencoders
8.3.3.1 Current challenges of variational autoencoders
8.3.4 Disentanglement of the latent space
8.3.5 Alternative reconstruction objectives
8.3.6 Improving the flexibility of the model
8.3.6.1 Alternative priors and auxiliary variables
8.3.6.2 Importance weighted autoencoder
8.3.6.3 Adversarial autoencoders
8.4 Example applications
8.4.1 Unsupervised pathology detection
8.4.2 Image synthesis for the explanation of black-box classifiers
8.4.3 Decoupled shape and appearance modeling for multimodal data
8.5 Future directions and research challenges
8.6 Summary
References
Part 2 Applications
9 Optimization of the MR imaging pipeline using simulation
9.1 Overview
9.2 History of MRI simulation
9.2.1 Diffusion MRI
9.3 The POSSUM simulation framework
9.3.1 POSSUM for MRI and functional MRI
9.3.1.1 Modeling artifacts
9.3.2 POSSUM for diffusion MRI
9.4 Applications
9.4.1 Motion correction algorithms for fMRI
9.4.1.1 MCFLIRT algorithm
9.4.1.2 Simulations
9.4.1.3 Results
9.4.2 Motion and eddy-current correction algorithms for diffusion MRI
9.4.3 Investigating the susceptibility-by-movement artifact
9.4.4 Investigating and optimizing image acquisition
9.4.5 Simulated data for machine learning
9.5 Future directions and research challenges
References
10 Synthesis for image analysis across modalities
10.1 General motivation
10.2 Registration
10.2.1 Background
10.2.2 Similarity metrics and their limitations.
10.2.3 Synthesis-based similarity metrics
10.2.4 Other applications of synthesis-based registration
10.3 Segmentation
10.3.1 Background
10.3.2 Domain gap and synthesis-based solutions
10.4 Other directions and perspectives
References
11 Medical image harmonization through synthesis
11.1 Introduction
11.2 Supervised techniques
11.2.1 Architecture and training
11.2.2 Using more information
11.3 Unsupervised techniques
11.3.1 Generative adversarial networks
11.3.2 Learning interpretable representations
11.3.3 One-/few-shot harmonization
11.3.4 Conclusion
References
12 Medical image super-resolution with deep networks
12.1 Introduction to super-resolution
12.1.1 Basic concepts
12.1.2 Brief history of SR methods prior to deep networks
12.1.2.1 SR through mathematical modeling
12.1.2.2 Example-based SR
12.2 SR methods with deep networks
12.2.1 Data acquisition
12.2.1.1 Fully-supervised, unsupervised, and self-supervised learning
12.2.1.2 Multiple network inputs
12.2.2 Network architectures
12.2.2.1 General frameworks
12.2.2.2 Upsampling before or within networks
12.2.2.3 Components in networks
12.2.2.4 Progressive networks
12.2.3 Loss functions
12.2.3.1 Paired losses
12.2.3.2 Unpaired losses
12.3 Applications of super-resolution in medical images
12.3.1 Super-resolution in different image modalities
12.3.1.1 Super-resolution in CT
12.3.1.2 Super-resolution in MRI
12.3.1.3 Super-resolution in optical coherence tomography
12.3.1.4 Super-resolution in microscopy
12.3.2 Super-resolution used for different tasks
12.3.2.1 Super-resolution for image quality enhancement
12.3.2.2 Super-resolution for diagnostic acceptability
12.3.2.3 Super-resolution for segmentation
12.3.2.4 Super-resolution for clinical abnormality detection.