Machine Learning Evaluation

A practical guide to robust performance evaluation methods machine learning models for modern industrial-strength applications.

This accessible, comprehensive guide is aimed at students, practitioners, engineers, and users. The emphasis is on building robust, responsible machine learning products incorporating meaningful metrics, rigorous statistical analysis, fair training sets, and explainability. Implementations in Python and sklearn are available on the book's website.As machine learning applications gain widespread adoption and integration in a variety of applications, including safety and mission-critical systems, the need for robust evaluation methods grows more urgent. This book compiles scattered information on the topic from research papers and blogs to provide a centralized resource that is accessible to students, practitioners, and researchers across the sciences. The book examines meaningful metrics for diverse types of learning paradigms and applications, unbiased estimation methods, rigorous statistical analysis, fair training sets, and meaningful explainability, all of which are essential to building robust and reliable machine learning products. In addition to standard classification, the book discusses unsupervised learning, regression, image segmentation, and anomaly detection. The book also covers topics such as industry-strength evaluation, fairness, and responsible AI. Implementations using Python and scikit-learn are available on the book's website.