Comprehensive Guide to Zephyr-7B: Features, Usage, and Fine-tuning

Explore Zephyr-7B: A Powerful Open-Source LLM

The OpenAI LLM Leaderboard is buzzing with new open-source models aiming to rival GPT-4, and Zephyr-7B is a standout contender. This tutorial explores this cutting-edge language model from WebPilot.AI, demonstrating its use with the Transformers pipeline and fine-tuning on an Agent-Instruct dataset. New to AI? The AI Fundamentals skill track is a great starting point.

Understanding Zephyr-7B

Zephyr-7B, part of the Zephyr series, is trained to function as a helpful assistant. Its strengths lie in generating coherent text, translating languages, summarizing information, sentiment analysis, and context-aware question answering.

Zephyr-7B-β: A Fine-Tuned Marvel

Zephyr-7B-β, the second model in the series, is a fine-tuned Mistral-7B model. Trained using Direct Preference Optimization (DPO) on a blend of public and synthetic datasets, it excels at interpreting complex queries and summarizing lengthy texts. At its release, it held the top spot among 7B chat models on MT-Bench and AlpacaEval benchmarks. Test its capabilities with the free demo on Zephyr Chat.

Image from Zephyr Chat

Accessing Zephyr-7B with Hugging Face Transformers

This tutorial uses Hugging Face Transformers for easy access. (If you encounter loading issues, consult the Inference Kaggle Notebook.)

1. Install Libraries: Ensure you have the latest versions:

!pip install -q -U transformers
!pip install -q -U accelerate
!pip install -q -U bitsandbytes

2. Import Libraries:

import torch
from transformers import pipeline

3. Create Pipeline: The device_map="auto" utilizes multiple GPUs for faster generation. torch.bfloat16 offers faster computation and reduced memory usage (but with slightly lower precision).

model_name = "HuggingFaceH4/zephyr-7b-beta"

pipe = pipeline(
    "text-generation",
    model=model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

4. Generate Text: The example below demonstrates generating Python code.

prompt = "Write a Python function that can clean the HTML tags from the file:"

outputs = pipe(
    prompt,
    max_new_tokens=300,
    do_sample=True,
    temperature=0.7,
    top_k=50,
    top_p=0.95,
)
print(outputs[0]["generated_text"])

5. System Prompts: Customize responses with Zephyr-7B style system prompts:

messages = [
    {
        "role": "system",
        "content": "You are a skilled software engineer who consistently produces high-quality Python code.",
    },
    {
        "role": "user",
        "content": "Write a Python code to display text in a star pattern.",
    },
]

prompt = pipe.tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

outputs = pipe(
    prompt,
    max_new_tokens=300,
    do_sample=True,
    temperature=0.7,
    top_k=50,
    top_p=0.95,
)
print(outputs[0]["generated_text"])

Fine-tuning Zephyr-7B on a Custom Dataset

This section guides you through fine-tuning Zephyr-7B-beta on a custom dataset using Kaggle's free GPUs (approximately 2 hours). (See the Fine-tuning Kaggle Notebook for troubleshooting.)

Setting Up and Preparing the Environment

1. Install Libraries:

!pip install -q -U transformers
!pip install -q -U accelerate
!pip install -q -U bitsandbytes

2. Import Modules:

import torch
from transformers import pipeline

3. Kaggle Secrets (for Kaggle notebooks): Retrieve Hugging Face and Weights & Biases API keys.

4. Hugging Face and Weights & Biases Login:

model_name = "HuggingFaceH4/zephyr-7b-beta"

pipe = pipeline(
    "text-generation",
    model=model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
)

5. Define Model and Dataset Names:

prompt = "Write a Python function that can clean the HTML tags from the file:"

outputs = pipe(
    prompt,
    max_new_tokens=300,
    do_sample=True,
    temperature=0.7,
    top_k=50,
    top_p=0.95,
)
print(outputs[0]["generated_text"])

AgentInstruct Dataset Processing

The format_prompt function adapts the dataset to Zephyr-7B's prompt style.

messages = [
    {
        "role": "system",
        "content": "You are a skilled software engineer who consistently produces high-quality Python code.",
    },
    {
        "role": "user",
        "content": "Write a Python code to display text in a star pattern.",
    },
]

prompt = pipe.tokenizer.apply_chat_template(
    messages, tokenize=False, add_generation_prompt=True
)

outputs = pipe(
    prompt,
    max_new_tokens=300,
    do_sample=True,
    temperature=0.7,
    top_k=50,
    top_p=0.95,
)
print(outputs[0]["generated_text"])

Loading and Preparing the Model

1. Load Model with 4-bit Precision: This is crucial for efficient training on GPUs with limited VRAM.

%%capture
%pip install -U bitsandbytes
%pip install -U transformers
%pip install -U peft
%pip install -U accelerate
%pip install -U trl

2. Load Tokenizer:

# ... (Import statements as in original tutorial) ...

3. Add Adapter Layer (PEFT): This allows for efficient fine-tuning by only updating parameters in the adapter layer.

!huggingface-cli login --token $secret_hf
# ... (wandb login as in original tutorial) ...

Training the Model

1. Training Arguments: Configure hyperparameters (refer to the Fine-Tuning LLaMA 2 tutorial for details).

base_model = "HuggingFaceH4/zephyr-7b-beta"
dataset_name = "THUDM/AgentInstruct"
new_model = "zephyr-7b-beta-Agent-Instruct"

2. SFT Trainer: Use Hugging Face's TRL library to create the trainer.

# ... (format_prompt function and dataset loading as in original tutorial) ...

3. Start Training:

# ... (bnb_config and model loading as in original tutorial) ...

Saving and Deploying the Fine-Tuned Model

1. Save the Model:

# ... (tokenizer loading and configuration as in original tutorial) ...

2. Push to Hugging Face Hub:

# ... (peft_config and model preparation as in original tutorial) ...

Testing the Fine-Tuned Model

Test the model's performance with various prompts. Examples are provided in the original tutorial.

Conclusion

Zephyr-7B-beta demonstrates impressive capabilities. This tutorial provides a comprehensive guide to utilizing and fine-tuning this powerful LLM, even on resource-constrained GPUs. Consider the Master Large Language Models (LLMs) Concepts course for deeper LLM knowledge.

The above is the detailed content of Comprehensive Guide to Zephyr-7B: Features, Usage, and Fine-tuning. For more information, please follow other related articles on the PHP Chinese website!