Farhad Pourkamali-Anaraki, Department of Mathematical and Statistical Sciences, University of Colorado Denver
Evaluation of Classification Models in Limited Data Scenarios with Application to Additive Manufacturing
Scientific observations and experiments provide valuable data to build machine learning (ML) models that reveal links between input variables and quantities of interest. Specifically, adopting machine ML-based surrogate models in scientific and engineering applications can accelerate design space exploration and optimization where closed-form analytical models for complex systems are unattainable. In this talk, we present recent work on developing a novel framework that enables the generation of accurate and unbiased test loss estimates using fewer labeled samples, effectively evaluating the predictive performance of classification models in data-limited applications. Central to the framework's innovation is designing an adaptive sampling distribution, which identifies pivotal testing samples based on the dynamic interplay between learner and evaluator models. A noteworthy aspect of this adaptive distribution lies in its ability to continually recalibrate the supervisory role of the evaluator model by prioritizing inputs that exhibit disparities. Comprehensive experimental analyses on two sparse data sets from material extrusion additive manufacturing problems concerning filament and printer selection validate the framework's superiority over uniform and fixed sampling distributions.
More information about this speaker may be found at听
听