Which model for the first case?
Options
- 1The model the business team knows from daily work
- 2The strongest model in the catalogue
- 3Two candidates on the test set, decided by result and place
Our recommendation: Two candidates on the test set. For classification, extraction and summarisation a small model competes against a medium one, and often the small one is enough, deployed regionally; for long contracts, analyses with references and coding a large GPT model competes against Claude. The winner is the one that scores better on the test set and may compute in the place the data class demands.
What we advise against: Choosing the model by ranking or headline. A vendor's benchmarks measure someone else's tasks, your test set measures yours.