Linked e-resources
Details
Table of Contents
Intro
Table of Contents
About the Author
About the Technical Reviewer
Introduction
Part I: What Is an Anomaly?
Chapter 1: The Importance of Anomalies and Anomaly Detection
Defining Anomalies
Outlier
Noise vs. Anomalies
Diagnosing an Example
What If We're Wrong?
Anomalies in the Wild
Finance
Medicine
Sports Analytics
A 23 Million Mistake
A Persistent Anomaly
Web Analytics
And Many More
Classes of Anomaly Detection
Statistical Anomaly Detection
Clustering Anomaly Detection
Model-Based Anomaly Detection
Building an Anomaly Detector
Key Goals
How Do Humans Handle Anomalies?
Known Unknowns
Conclusion
Chapter 2: Humans Are Pattern Matchers
A Primer on the Gestalt School
Key Findings of the Gestalt School
Emergence
Reification
Invariance
Multistability
Principles Implied in the Key Findings
Meaningfulness
Conciseness
Closure
Similarity
Good Continuation
Figure and Ground
Proximity
Connectedness
Common Region
Symmetry
Common Fate
Synchrony
Helping People Find Anomalies
Use Color As a Signal
Limit Nonmeaningful Information
Enable "Connecting the Dots"
Conclusion
Chapter 3: Formalizing Anomaly Detection
The Importance of Formalization
"I'll Know It When I See It" Isn't Enough
Human Fallibility
Marginal Outliers
The Limits of Visualization
The First Formal Tool: Univariate Analysis
Distributions and Histograms
The Normal Distribution
Mean, Variance, and Standard Deviation
Additional Distributions
Log-Normal
Uniform
Cauchy
Robustness and the Mean
The Susceptibility of Outliers
The Median and "Robust" Statistics
Beyond the Median: Calculating Percentiles
Control Charts
Conclusion
Chapter 4: Laying Out the Framework
Tools of the Trade
Choosing a Programming Language
Making Plumbing Choices
Reducing Architectural Variables
Developing an Initial Framework
Battlespace Preparation
Framing the API
Input and Output Signatures
Defining a Common Signature
Defining an Outlier
Sensitivity and Fraction of Anomalies
Single Solution
Combined Arms
Framing the Solution
Containerizing the Solution
Conclusion
Chapter 5: Building a Test Suite
Tools of the Trade
Unit Test Library
Integration Testing
Writing Testable Code
Keep Methods Separated
Emphasize Use Cases
Functional or Clean: Your Choice
Creating the Initial Tests
Unit Tests
Integration Tests
Conclusion
Chapter 6: Implementing the First Methods
A Motivating Example
Ensembling As a Technique
Sequential Ensembling
Independent Ensembling
Choosing Between Sequential and Independent Ensembling
Implementing the First Checks
Standard Deviations from the Mean
Median Absolute Deviations from the Median
Distance from the Interquartile Range
Table of Contents
About the Author
About the Technical Reviewer
Introduction
Part I: What Is an Anomaly?
Chapter 1: The Importance of Anomalies and Anomaly Detection
Defining Anomalies
Outlier
Noise vs. Anomalies
Diagnosing an Example
What If We're Wrong?
Anomalies in the Wild
Finance
Medicine
Sports Analytics
A 23 Million Mistake
A Persistent Anomaly
Web Analytics
And Many More
Classes of Anomaly Detection
Statistical Anomaly Detection
Clustering Anomaly Detection
Model-Based Anomaly Detection
Building an Anomaly Detector
Key Goals
How Do Humans Handle Anomalies?
Known Unknowns
Conclusion
Chapter 2: Humans Are Pattern Matchers
A Primer on the Gestalt School
Key Findings of the Gestalt School
Emergence
Reification
Invariance
Multistability
Principles Implied in the Key Findings
Meaningfulness
Conciseness
Closure
Similarity
Good Continuation
Figure and Ground
Proximity
Connectedness
Common Region
Symmetry
Common Fate
Synchrony
Helping People Find Anomalies
Use Color As a Signal
Limit Nonmeaningful Information
Enable "Connecting the Dots"
Conclusion
Chapter 3: Formalizing Anomaly Detection
The Importance of Formalization
"I'll Know It When I See It" Isn't Enough
Human Fallibility
Marginal Outliers
The Limits of Visualization
The First Formal Tool: Univariate Analysis
Distributions and Histograms
The Normal Distribution
Mean, Variance, and Standard Deviation
Additional Distributions
Log-Normal
Uniform
Cauchy
Robustness and the Mean
The Susceptibility of Outliers
The Median and "Robust" Statistics
Beyond the Median: Calculating Percentiles
Control Charts
Conclusion
Chapter 4: Laying Out the Framework
Tools of the Trade
Choosing a Programming Language
Making Plumbing Choices
Reducing Architectural Variables
Developing an Initial Framework
Battlespace Preparation
Framing the API
Input and Output Signatures
Defining a Common Signature
Defining an Outlier
Sensitivity and Fraction of Anomalies
Single Solution
Combined Arms
Framing the Solution
Containerizing the Solution
Conclusion
Chapter 5: Building a Test Suite
Tools of the Trade
Unit Test Library
Integration Testing
Writing Testable Code
Keep Methods Separated
Emphasize Use Cases
Functional or Clean: Your Choice
Creating the Initial Tests
Unit Tests
Integration Tests
Conclusion
Chapter 6: Implementing the First Methods
A Motivating Example
Ensembling As a Technique
Sequential Ensembling
Independent Ensembling
Choosing Between Sequential and Independent Ensembling
Implementing the First Checks
Standard Deviations from the Mean
Median Absolute Deviations from the Median
Distance from the Interquartile Range