How to Efficiently Test ML Models & Data

Automatic testing for ML pipelines is hard. Part of the executed code is a model that was dynamically trained on a fresh batch of data, and silent failures are common. Therefore, it’s problematic to use known methodologies such as automating tests for predefined edge cases and tracking code coverage.

We’ll demonstrate common pitfalls with ML models, and cover best practices for automatically validating them: What should be tested in these pipelines? How can we verify that they’ll behave as we expect once in production?

We’ll discuss how to automate tests for these scenarios, introduce the deepchecks open source package for testing ML models and data, and demonstrate how it can aid the process.