# A new AI model has launched: should your business switch? New AI models are announced every few weeks. Here is how to decide whether yours is worth changing, how to test it on your own work, and what to ask your supplier first. - Page: https://www.cognivise.co.uk/blog/should-you-switch-ai-models - Topic: AI development - Published: 2026-10-29 - Updated: 2026-10-29 - Author: Cognivise ## Key takeaways - Switch to a new AI model only when a test on your own tasks shows a clear gain in quality, cost or speed. - Build a small set of real examples with known good answers, and run the old and new model on it. - Check data terms, cost per correct answer and how you would switch back before you change anything in production. - Keep the model behind a thin layer in your software so the next change is a setting, not a rewrite. Should your business switch to a new AI model when one is announced? Not automatically. Switch when a test on your own tasks shows a clear gain in quality, cost or speed, and only after you have checked how your data is handled and how you would go back. A model that tops a public leaderboard may do no better on your invoices, support emails or product data. ## Why is it tempting to switch straight away? Announcements lead with benchmark scores, price cuts and new abilities. They are written to be read by everyone, not by a business with one particular job for the model to do. Early coverage is also often inconsistent. In the first days, outlets can disagree on names, versions and availability. Wait for the provider's own documentation before you decide anything. ## What can go wrong when you change model? - **Behaviour changes.** The same instructions can give different answers, a different format or a different tone. Something that was reliable can become unreliable. - **Cost moves in both directions.** A lower price per unit does not help if the new model writes longer answers or needs more attempts. - **Speed changes.** This matters most when a customer is waiting for the reply. - **Data terms differ.** Where data is processed, how long it is kept and whether it is used for training are set by the provider's terms. Check them before sending customer data to a new model or a new supplier. - **Other code depends on the output.** If a system reads the answer, for example as structured data, a small change in format can break it. - **Models are retired.** Providers withdraw older models on their own schedule, so the one you use today will not last forever. ## How do you test a new model fairly? 1. **Collect real examples.** Gather a few dozen typical tasks from your own work, with personal data removed, and write down what a good answer looks like. 2. **Run both models on the same set.** Use the same instructions and the same inputs for the current model and the new one. 3. **Score what matters to you.** Accuracy, format, tone, refusals, time taken and cost per task. 4. **Have someone who knows the work read the results.** Look at the worst answers, not only the average. 5. **Compare cost per correct answer.** Not the price list. If you cannot do this, you do not yet have a way to know whether a new model is better for you. Building that test set is the most useful thing to do first, and it can be reused every time a new model appears. ## Should you pin a version? Where the provider offers it, yes. Name a specific model version in production so behaviour does not change underneath you, and set a date to review it. Pinning is a way to control when you change, not a way to avoid changing. ## What should you ask your supplier or developers? - **Which model and version do we use today?** - **What test set do we have, and when was it last run?** - **What happens if the provider changes or retires the model?** - **Where is our data processed, and for how long is it kept?** - **Can we switch back within a day if the new model misbehaves?** ## When is switching worth it? Switching is usually worth the effort when: - the new model passes a task that matters and the current one fails it; - the cost per task falls by an amount that is worth the work of changing; - speed matters and the new model is clearly faster; - the current model is about to be retired. It is usually not worth it when the current setup works, nobody is complaining and there is no test to show what would improve. ## How do you make future changes easier? Keep the model behind a thin layer in your software, so changing it is a settings change and not a rewrite. Store your instructions and your test examples with the code, and write down which model each feature uses. Avoid building features that only work with one provider's special options unless the gain is clear. If you are weighing up where AI fits in your business at all, read [when a business actually needs AI development](https://www.cognivise.co.uk/blog/when-does-a-business-need-ai-development) and [what AI chatbots can and cannot do](https://www.cognivise.co.uk/blog/ai-chatbots-for-business). If you want help testing models on your own work or building something you can change later, see our [AI development services](https://www.cognivise.co.uk/services/ai-development-services) or [contact us](https://www.cognivise.co.uk/contact). ## Related - [AI development services](https://www.cognivise.co.uk/services/ai-development-services) - [When does a business actually need AI development?](https://www.cognivise.co.uk/blog/when-does-a-business-need-ai-development) - [AI chatbots for business: what they can and cannot do](https://www.cognivise.co.uk/blog/ai-chatbots-for-business)