Model evaluation and the human on the other side

For months some developers have been trying to one-up each other using performance evaluation metrics. But some companies may look to use a softer touch.

A small pixar - styled robot trying to pull several large computer servers up a giant hill — midjourney

Each new open source model released these days isn’t complete without a suite of scores to evaluate its performance.

But as time goes on and companies begin to find ways to implement language models (APIs or otherwise), there’s a growing recognition amo…