None
RO
NIST Launches New Testbed Program to Evaluate AI Model Performance
[]
Executive Gov
AITE offers volunteer-based testing to data providers and AI model providers looking to compare their technologies against blind data.
Data providers may contribute an original dataset from their domain, along with a defined task to be performed using that data, while keeping the dataset inaccessible to other participants.
In exchange, data providers receive detailed performance measurements showing how top-performing models handle their specific data and task.
Model providers, meanwhile, submit AI models for evaluation against the available datasets and tasks.
NIST said this approach is designed to improve comparability across models while ensuring that evaluation data is never used to train any submitted model.