Fine-Tune Open-Source LLMs Using Lamini - Analytics Vidhya

Recently, with the rise of large language models and AI, we have seen innumerable advancements in natural language processing. Models in domains like text, code, and image/video generation have archived human-like reasoning and performance.These models perform exceptionally well in general knowledge-based questions. Models like GPT-4o, Llama 2, Claude, and Gemini are trained on publicly available datasets. They fail to answer domain or subject-specific questions that may be more useful for various organizational tasks.

Fine-tuning helps developers and businesses adapt and train pre-trained models to a domain-specific dataset that archives high accuracy and coherency on domain-related queries. Fine-tuning enhances the model’s performance without requiring extensive computing resources because pre-trained models have already learned the general text from the vast public data.

This blog will examine why we must fine-tune pre-trained models using the Lamini platform. This allows us to fine-tune and evaluate models without using much computational resources.

So, let’s get started!

Learning Objectives

• To explore the need toFine-Tune Open-Source LLMs UsingLamini
• To find out the use of Lamini and under instructions on fine-tuned models
• To get a hands-on understanding of the end-to-end process of fine-tuning models.

This article was published as a part of theData Science Blogathon.

Table of contents

• Learning Objectives
• Why Should One Fine-Tune Large Language Models?
• How to Fine-Tune Open-Source LLMs Using Lamini?
  • Data Preparation
  • Tokenize the Dataset
  • Fine-Tuning Process
  • Setting up an Environment
  • Load Dataset
  • Setup Training to Fine-Tune, the Model
• Conclusion
• Frequently Asked Questions

Why Should One Fine-Tune Large Language Models?

Pre-trained models are primarily trained on vast general data with a high chance of lack of context or domain-specific knowledge. Pre-trained models can also result in hallucinations and inaccurate and incoherent outputs. Most popular large language models based on chatbots like ChatGPT, Gemini, and BingChat have repeatedly shown that pre-trained models are prone to such inaccuracies. This is where fine-tuning comes to the rescue, which can help to adapt pre-trained LLMs to subject-specific tasks and questions effectively. Other ways to align models to your objectives include prompt engineering and few-shot prompt engineering.

Still, fine-tuning remains an outperformer when it comes to performance metrics. Methods such as Parameter efficient fine-tuning and Low adaptive ranking fine-tuning have further improved the model fine-tuning and helped developers generate better models. Let’s look at how fine-tuning fits in a large language model context.

# Load the fine-tuning dataset
filename = "lamini_docs.json"
instruction_dataset_df = pd.read_json(filename, lines=True)
instruction_dataset_df

# Load it into a python's dictionary
examples = instruction_dataset_df.to_dict()

# prepare a samples for a fine-tuning 
if "question" in examples and "answer" in examples:
  text = examples["question"][0]   examples["answer"][0]
elif "instruction" in examples and "response" in examples:
  text = examples["instruction"][0]   examples["response"][0]
elif "input" in examples and "output" in examples:
  text = examples["input"][0]   examples["output"][0]
else:
  text = examples["text"][0]

# Using a prompt template to create instruct tuned dataset for fine-tuning
prompt_template_qa = """### Question:
{question}

### Answer:
{answer}"""

The above code shows that instruction tuning uses a prompt template to prepare a dataset for instruction tuning and fine-tune a model for a specific dataset. We can fine-tune the pre-trained model to a specific use case using such a custom dataset.

The next section will examine how Lamini can help fine-tune large language models (LLMs) for custom datasets.

How to Fine-Tune Open-Source LLMs UsingLamini?

The Lamini platform enables users to fine-tune and deploy models seamlessly without much cost and hardware setup requirements. Lamini provides an end-to-end stack to develop, train, tune,e, and deploy models at user convenience and model requirements. Lamini provides its own hosted GPU computing network to train models cost-effectively.

Lamini memory tuning tools and compute optimization help train and tune models with high accuracy while controlling costs. Models can be hosted anywhere, on a private cloud or through Lamini’s GPU network. Next, we will see a step-by-step guide to prepare data to fine-tune large language models (LLMs) using the Lamini platform.

Data Preparation

Generally, we need to select a domain-specific dataset for data cleaning, promotion, tokenization, and storage to prepare data for any fine-tuning task. After loading the dataset, we preprocess it to convert it into an instruction-tuned dataset. We format each sample from the dataset into an instruction, question, and answer format to better fine-tune it for our use cases. Check out the source of the dataset using the link given here. Let’s look at the code example instructions on tuning with tokenization for training using the Lamini platform.

import pandas as pd

# load the dataset and store it as an instruction dataset
filename = "lamini_docs.json"
instruction_dataset_df = pd.read_json(filename, lines=True)
examples = instruction_dataset_df.to_dict()

if "question" in examples and "answer" in examples:
  text = examples["question"][0]   examples["answer"][0]
elif "instruction" in examples and "response" in examples:
  text = examples["instruction"][0]   examples["response"][0]
elif "input" in examples and "output" in examples:
  text = examples["input"][0]   examples["output"][0]
else:
  text = examples["text"][0]

prompt_template = """### Question:
{question}

### Answer:"""

# Store fine-tuning examples as an instruction format
num_examples = len(examples["question"])
finetuning_dataset = []
for i in range(num_examples):
  question = examples["question"][i]
  answer = examples["answer"][i]
  text_with_prompt_template = prompt_template.format(question=question)
  finetuning_dataset.append({"question": text_with_prompt_template, 
                             "answer": answer})

In the above example, we have formatted “questions” and “answers” in a prompt template and stored them in a separate file for tokenization and padding before training the LLM.

Tokenize the Dataset

# Tokenization of the dataset with padding and truncation
def tokenize_function(examples):
    if "question" in examples and "answer" in examples:
      text = examples["question"][0]   examples["answer"][0]
    elif "input" in examples and "output" in examples:
      text = examples["input"][0]   examples["output"][0]
    else:
      text = examples["text"][0]
    
    # padding
    tokenizer.pad_token = tokenizer.eos_token
    tokenized_inputs = tokenizer(
        text,
        return_tensors="np",
        padding=True,
    )

    max_length = min(
        tokenized_inputs["input_ids"].shape[1],
        2048
    )
    # truncation of the text
    tokenizer.truncation_side = "left"
    tokenized_inputs = tokenizer(
        text,
        return_tensors="np",
        truncation=True,
        max_length=max_length
    )

    return tokenized_inputs

The above code takes the dataset samples as input for padding and truncation with tokenization to generate preprocessed tokenized dataset samples, which can be used for fine-tuning pre-trained models. Now that the dataset is ready, we will look into the training and evaluation of models using the Lamini platform.

Fine-Tuning Process

Now that we have a dataset prepared in an instruction-tuning format, we will load the dataset into the environment and fine-tune the pre-trained LLM model using Lamini’s easy-to-use training techniques.

Setting up an Environment

To begin the fine-tuning open-sourceLLMs UsingLamini, we must first ensure that our code environment has suitable resources and libraries installed. We must ensure you have a suitable machine with sufficient GPU resources and install necessary libraries such as transformers, datasets, torches, and pandas. You must securely load environment variables like api_url and api_key, typically from environment files. You can use packages like dotenv to load these variables. After preparing the environment, load the dataset and models for training.

import os
from lamini import Lamini

lamini.api_url = os.getenv("POWERML__PRODUCTION__URL")
lamini.api_key = os.getenv("POWERML__PRODUCTION__KEY")

# import necessary library and load the environment files
import datasets
import tempfile
import logging
import random
import config
import os
import yaml
import time
import torch
import transformers
import pandas as pd
import jsonlines

# Loading transformer architecture and [[
from utilities import *
from transformers import AutoTokenizer
from transformers import AutoModelForCausalLM
from transformers import TrainingArguments
from transformers import AutoModelForCausalLM
from llama import BasicModelRunner

logger = logging.getLogger(__name__)
global_config = None

Load Dataset

After setting up logging for monitoring and debugging, prepare your dataset using datasets or other data handling libraries like jsonlines and pandas. After loading the dataset, we will set up a tokenizer and model with training configurations for the training process.

# load the dataset from you local system or HF cloud
dataset_name = "lamini_docs.jsonl"
dataset_path = f"/content/{dataset_name}"
use_hf = False

# dataset path
dataset_path = "lamini/lamini_docs"

Set up model, training config, and tokenizer

Next, we select the model for fine-tuning open-sourceLLMs UsingLamini, “EleutherAI/pythia-70m,” and define its configuration under training_config, specifying the pre-trained model name and dataset path. We initialize the AutoTokenizer with the model’s tokenizer and set padding to the end-of-sequence token. Then, we tokenize the data and split it into training and testing datasets using a custom function, tokenize_and_split_data. Finally, we instantiate the base model using AutoModelForCausalLM, enabling it to perform causal language modeling tasks. Also, the below code sets up compute requirements for our model fine-tuning process.

# model name
model_name = "EleutherAI/pythia-70m"

# training config
training_config = {
    "model": {
        "pretrained_name": model_name,
        "max_length" : 2048
    },
    "datasets": {
        "use_hf": use_hf,
        "path": dataset_path
    },
    "verbose": True
}

# setting up auto tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)
tokenizer.pad_token = tokenizer.eos_token
train_dataset, test_dataset = tokenize_and_split_data(training_config, tokenizer)

# set up a baseline model from lamini
base_model = Lamini(model_name)

# gpu parallization
device_count = torch.cuda.device_count()
if device_count > 0:
    logger.debug("Select GPU device")
    device = torch.device("cuda")
else:
    logger.debug("Select CPU device")
    device = torch.device("cpu")

Setup Training to Fine-Tune, the Model

Finally, we set up training argument parameters with hyperparameters. It includes learning rate, epochs, batch size, output directory, eval steps, sav, warmup steps, evaluation and logging strategy, etc., to fine-tune the custom training dataset.

max_steps = 3

# trained model name
trained_model_name = f"lamini_docs_{max_steps}_steps"
output_dir = trained_model_name

training_args = TrainingArguments(
  # Learning rate
  learning_rate=1.0e-5,
  # Number of training epochs
  num_train_epochs=1,

  # Max steps to train for (each step is a batch of data)
  # Overrides num_train_epochs, if not -1
  max_steps=max_steps,

  # Batch size for training
  per_device_train_batch_size=1,

  # Directory to save model checkpoints
  output_dir=output_dir,

  # Other arguments
  overwrite_output_dir=False, # Overwrite the content of the output directory
  disable_tqdm=False, # Disable progress bars
  eval_steps=120, # Number of update steps between two evaluations
  save_steps=120, # After # steps model is saved
  warmup_steps=1, # Number of warmup steps for learning rate scheduler
  per_device_eval_batch_size=1, # Batch size for evaluation
  evaluation_strategy="steps",
  logging_strategy="steps",
  logging_steps=1,
  optim="adafactor",
  gradient_accumulation_steps = 4,
  gradient_checkpointing=False,

  # Parameters for early stopping
  load_best_model_at_end=True,
  save_total_limit=1,
  metric_for_best_model="eval_loss",
  greater_is_better=False
)

After setting the training arguments, the system calculates the model’s floating-point operations per second (FLOPs) based on the input size and gradient accumulation steps. Thus giving insight into the computational load. It also assesses memory usage, estimating the model’s footprint in gigabytes. Once these calculations are complete, a Trainer initializes the base model, FLOPs, total training steps, and the prepared datasets for training and evaluation. This setup optimizes the training process and enables resource utilization monitoring, critical for efficiently handling large-scale model fine-tuning. At the end of training, the fine-tuned model is ready for deployment on the cloud to serve users as an API.

# model parameters
model_flops = (
  base_model.floating_point_ops(
    {
       "input_ids": torch.zeros(
           (1, training_config["model"]["max_length"])
      )
    }
  )
  * training_args.gradient_accumulation_steps
)

print(base_model)
print("Memory footprint", base_model.get_memory_footprint() / 1e9, "GB")
print("Flops", model_flops / 1e9, "GFLOPs")

# Set up a trainer
trainer = Trainer(
    model=base_model,
    model_flops=model_flops,
    total_steps=max_steps,
    args=training_args,
    train_dataset=train_dataset,
    eval_dataset=test_dataset,
)

Conclusion

In conclusion, this article provides an in-depth guide to understanding the need to fine-tune LLMs using the Lamini platform. It gives a comprehensive overview of why we must fine-tune the model for custom datasets and business use cases and the benefits of using Lamini tools. We also saw a step-by-step guide to fine-tuning the model using a custom dataset and LLM with tools from Lamini. Let’s summarise critical takeaways from the blog.

Key Takeaways

1. Learning is needed for fine-tuning models against prompt engineering and retrieval augmented generation methods.
2. UUtilizationof platforms like Lamini for easy-to-use hardware setup and deployment techniques for fine-tuned models to serve the user requirements
3. We are preparing data for the fine-tuning task and setting up a pipeline to train a base model using a wide range of hyperparameters.

Explore the code behind this article on GitHub.

The media shown in this article are not owned by Analytics Vidhya and is used at the Author’s discretion.

Frequently Asked Questions

Q1. How to fine-tune my models?

A. The fine-tuning process starts with understanding context-specific requirements, dataset preparation, tokenization, and setting up training setups like hardware requirements, training configs, and training arguments. Eventually, a training job for model development is run.

Q2. What does fine-tuning of LLMs mean?

A. Fine-tuning an LLM means training a base model on a specific custom dataset. This generates accurate and context-relevant outputs for specific queries per the use case.

Q3. What is Lamini in LLM fine-tuning?

A. Lamini provides integrated language model fine-tuning, inference, and GPU setup for LLMs’ seamless, efficient, and cost-effective development.

The above is the detailed content of Fine-Tune Open-Source LLMs Using Lamini - Analytics Vidhya. For more information, please follow other related articles on the PHP Chinese website!