Benchmarks and performance metrics for lab autonomy

[sgbaird]

Xref:

Benchmarking acceleration in materials discovery with AI

I’ve been discussing with a lot of people recently about the best approaches to benchmarking “acceleration” in materials discovery with AI. There are few examples of this done in literature. Baselines are often either easy to establish but not highly relevant, such as a human with random sampling—or more relevant but requires intensive effort, such as comparing a human with design-of-experiments vs. human with AI. We have some ideas internally at the Acceleration Consortium about how assess th…