Intro
Preface
References
Contents
Acronyms
Part I Fundamentals and Backgrounds
1 Evolutionary Computation
1.1 Genetic Algorithms (GAs)
1.2 Particle Swarm Optimization (PSO)
1.3 Differential Evolution (DE)
1.4 Genetic Programming (GP)
1.5 Chapter Summary
References
2 Deep Neural Networks
2.1 Deep Belief Networks
2.2 Stacked Auto-Encoders
2.2.1 Sparse Auto-Encoders
2.2.2 Weight Decay Auto-Encoders
2.2.3 Denoising Auto-Encoders (DAEs)
2.2.4 Contractive Auto-Encoders
2.2.5 Convolutional Auto-Encoders (CAEs)

2.2.6 Variational Auto-Encoders (VAEs)
2.3 Convolutional Neural Networks (CNNs)
2.3.1 CNN Skeleton
2.3.2 Convolution
2.3.3 Pooling
2.3.4 Reflect Padding
2.3.5 Batch Normalization (BN)
2.3.6 ResNet Blocks (RBs) and DenseNet Blocks (DBs)
2.4 Benchmarks for Deep Neural Networks
2.5 Chapter Summary
References
Part II Evolutionary Deep Neural Architecture Search for Unsupervised DNNs
3 Architecture Design for Stacked AEs and DBNs
3.1 Introduction
3.2 Related Work and Motivations
3.2.1 Unsupervised Deep Learning

3.2.2 Evolutionary Algorithms for Evolving Neural Networks
3.3 Algorithm Details
3.3.1 Framework of EUDNN
3.3.2 Evolving Connection Weights and Activation Functions
3.3.3 Fine-Tuning Connection Weights
3.3.4 Discussion
3.4 Experimental Design
3.4.1 Performance Metric
3.4.2 Peer Competitors
3.4.3 Parameter Settings
3.5 Experimental Results and Analysis
3.5.1 Performance of EUDNN
3.5.2 Analysis on Pre-training of EUDNN
3.5.3 Analysis on Fine-Tuning of EUDNN
3.5.4 Representation Visualizations
3.6 Chapter Summary
References

4 Architecture Design for Convolutional Auto-Encoders
4.1 Introduction
4.2 Motivation of FCAE
4.3 Algorithm Details
4.3.1 Algorithm Overview
4.3.2 Encoding Strategy
4.3.3 Particle Initialization
4.3.4 Fitness Evaluation
4.3.5 Velocity Calculation and Position Update
4.3.6 Deep Training on Global Best Particle
4.4 Experimental Design
4.4.1 Peer Competitors
4.4.2 Parameter Settings
4.5 Experimental Results and Analysis
4.5.1 Overview Performance
4.5.2 Evolution Trajectory of PSOAO
4.5.3 Performance on Different Numbers of Training Examples

4.5.4 Investigation on x-Reference Velocity Calculation
4.6 Chapter Summary
References
5 Architecture Design for Variational Auto-Encoders
5.1 Introduction
5.2 Algorithm Details
5.2.1 Algorithm Overview
5.2.2 Strategy of Gene Encoding
5.2.3 Initialization of Population
5.2.4 Evaluation
5.2.5 Crossover Operator and Mutation Operator
5.2.6 Environmental Selection
5.3 Experimental Design
5.3.1 Parameter Setting
5.3.2 Peer Competitors
5.3.3 Performance Evaluation
5.4 Experimental Results and Analysis
5.4.1 Overall Performance