The State of AI in Insurance: The Power of Fine-Tuning (Vol. IV)

The Report in Brief:

• While prompt engineering is often thought of first, fine-tuning shows incredible promise in boosting the performance of LLMs
• Fine-tuning can deliver “big model” performance at “small model” prices despite incurring some additional costs  
• Fine-tuning on complex data sets appears to yield the best overall results
• Fine-tuning is just one strategy to achieve the best performance in an LLM strategy

Fine-Tuning LLMs for Improved Performance

As reported in Vol. III of our comparison of LLMs, performance gains across various available models and their variants are quickly reaching a point where current LLM performance differences are negligible depending on the use case. With performance equalizing, the cost differences of LLMs become even more critical to the decision making process.

At the same time, there are proven methods for improving LLM performance in production environments. For example, prompt engineering is a well documented approach to achieving incremental performance. The more precise the initial prompt, the more accurate the results.

Another approach generating interest is the concept of fine-tuning. In the case of LLMs, fine-tuning is defined as the process of further training a pre-trained model on smaller, more specific data sets to achieve greater performance. To better understand the kind of performance increases that are possible via fine-tuning, the Shift Data Science (DS) and Research Teams compared a fine-tuned GPT4o-Mini against a standard GPT4o and a standard GPT4o-Mini on an information extraction task.

Interested in learning more about what they discovered?

Download now to read the full report: