In our previous article, we elucidated the use of techniques like Retrieval-Augmented Generation (RAG), fine-tuning, and prompt-engineering through an HR-specific use case. The goal was to develop an HR-specialized assistant to support HR teams with compensation questions, evaluated through the key question: "What is a good salary for a candidate based on the CV?" The article included screenshots comparing the answers from standard ChatGPT-4o, which utilized prompt-engineering, and a custom GPT employing the RAG technique to enhance contextual understanding.
Today, we aim to demonstrate how fine-tuning can further refine the responses from a generative AI assistant. Specifically, we used fine-tuning to enhance the style, tone, format, and quality of the answers, as well as to ensure reliability in producing the desired output. For an HR-specific assistant, it's crucial to grasp HR nuances and ground its answers in the company's compensation reasoning.
%20(2).jpg)
To illustrate the benefits of fine-tuning quickly, we used the GPT-3.5-Turbo-0125 model and fine-tuned it via the OpenAI platform. The training was streamlined, using a dataset of only 100 high-quality examples, which is generally a good starting point.
The evaluation was done through zero-shot prompting, meaning the prompt used didn’t contain examples or demonstrations of how to answer, with the only context provided being the CV summary of the potential new hire.
The non-fine-tuned model offered a broad salary range of $130,000 to $160,000, with generalized justifications based on the candidate's qualifications, such as technical skills, leadership, and certifications. While it suggested that the candidate might command a higher salary due to these factors, its response remained generic and lacked specific insights into the company’s compensation process.
In contrast, the fine-tuned model proposed a specific salary of $150,000, supported by detailed justifications grounded in HR nuances and a thorough understanding of the company's compensation strategy. This model highlighted the candidate’s decade-long experience, Python and data science expertise, project management skills, leadership abilities, and relevant certifications, explicitly connecting these attributes to the company's needs and success.
The fine-tuned model better understood the company’s compensation reasoning by emphasizing how the candidate’s skills and achievements could drive organizational success. It pointed out her contributions to operational efficiency and data consistency, showcasing her potential impact on the company. The model's tailored approach reflected a deeper comprehension of the specific metrics and strategic goals guiding the company’s compensation decisions.Below are the screenshots that compare the responses from both models:
This practical case study demonstrates that fine-tuning was a highly effective way to enhance our HR assistant's ability to generate desired outputs focused on business-relevant data points. By fine-tuning the model, we were able to teach the assistant to produce more precise, nuanced, and contextually grounded responses, reflecting a deeper understanding of the specific metrics and strategic goals guiding the company’s compensation decisions.
.jpg)
Fine-tuning not only improved the quality of the answers but also ensured that the assistant could provide insights that are directly aligned with our company’s compensation strategy, ultimately aiding HR teams in making more informed and strategic decisions.
