# Is Fine-Tuning Still Needed? LLMs, RAG, & LoRA

## Executive summary

While early successes demonstrated that fine-tuning custom LLMs could outperform general models (e.g., legal AI in 2023), the landscape has shifted significantly. Modern frontier models are rapidly closing the gap due to massive context windows and improved reasoning capabilities. The current architectural best practice suggests prioritizing non-weight modification techniques like Retrieval Augmented Generation (RAG), Context Engineering, and Agent Skills before resorting to fine-tuning. Fine-tuning remains valuable for specific bottlenecks, such as achieving low latency or when using parameter-efficient methods like LoRA.

## Key takeaways

- Fine-Tuning vs. General Models: Historically, custom fine-tuned models outperformed off-the-shelf leaders (e.g., legal AI over GPT-4 in 2023). However, general models are catching up due to massive context windows and improved inference reasoning.
- Modern Customization Stack: The preferred order for customization is: Base Model $\rightarrow$ Prompt/Context Engineering $\rightarrow$ RAG (for proprietary knowledge) $\rightarrow$ Agent Skills (for procedural know-how). Fine-tuning should be the last resort.
- Cost and Complexity of Customization: Fine-tuning is costly, involving not only training runs but also data collection, evaluation, regression avoidance, and continuous maintenance as frontier models advance.

## Technical details

- LLM Fine-Tuning Process: Fine-tuning takes a base LLM (trained on general internet data) and customizes it by continuing training on a focused dataset (e.g., legal contracts), resulting in a model optimized for narrow tasks.
- RAG (Retrieval Augmented Generation): A technique that avoids modifying model weights. Instead, the application retrieves relevant documents at query time and feeds them directly into the prompt context.
- Agent Skills: Procedural knowledge packaged in files/folders that allow a general-purpose model to load tools (like an SQL agent skill) on demand, rather than requiring weight modification.
- LoRA (Low Rank Adaptation): A modern technique for fine-tuning that trains only a small adapter layer on top of the existing base model weights, keeping most original weights locked and reducing computational overhead.
- Distillation & RFT: Distillation involves training a smaller model on high-quality outputs generated by a large 'teacher' model. Reinforcement Fine-Tuning (RFT) uses a grader to score candidate answers, updating the model toward higher scores.

## Practical implications

- For build engineers designing AI systems, the focus should shift from model weight customization to robust data retrieval and orchestration layers (RAG).
- When latency is a critical constraint (e.g., voice agents), small fine-tuned models or distilled versions may still be necessary.
- The architectural decision framework must account for the operational cost of maintaining custom models against rapidly advancing frontier general models.

## Topics

LLMs, Generative AI, Fine-Tuning, RAG, LoRA, Machine Learning, IBM Technology, https://ibm.biz/~c5OKNxoyO, https://ibm.biz/~zn4aDcdla

Source: https://www.youtube.com/watch?v=-W2JdSl1v48