DeepSeek的开创性AI模型挑战Openai的主导地位。 这些先进的推理模型是免费的，可以使获得强大AI的访问民主化。 了解如何通过我们的视频教程微调DeepSeek：
该教程微调使用拥抱脸部医疗链数据集使用DeepSeek-r1-Distill-lalama-8b型号。 这种蒸馏型型号衍生自Llama 3.1 8b，提供了与原始DeepSeek-R1相当的推理能力。 LLM和微调的新手？ 考虑我们在Python课程中对LLM的介绍。
>由作者 图像
介绍DeepSeek R1模型> deepSeek-r1-Zero
> deepSeek-r1
DeepSeek-R1解决DeepSeek-R1-Zero的局限性，在RL之前包含了冷启动数据。这种多阶段的训练可实现最先进的性能，匹配OpenAI-O1，同时提高输出清晰度。
来源：DeepSeek-ai/deepSeek-r1
>在我们的博客文章中了解更多有关DeepSeek-R1的功能，开发，蒸馏模型，访问，定价和OpenAi O1比较的信息：“ DeepSeek-R1：功能，O1比较，蒸发模型及更多”。
>微调DeepSeek R1：实用指南
按照以下步骤微调您的DeepSeek R1型号：
>我们利用Kaggle的免费GPU访问权限。创建一个Kaggle笔记本电脑，将您的拥抱脸和偏见令牌添加为秘密。安装
<code>%%capture !pip install unsloth !pip install --force-reinstall --no-cache-dir --no-deps git+https://github.com/unslothai/unsloth.git</code>
>用拥抱的面部CLI和重量和偏见（WANDB）进行身份验证：
<code>from huggingface_hub import login
from kaggle_secrets import UserSecretsClient
user_secrets = UserSecretsClient()
hf_token = user_secrets.get_secret("HUGGINGFACE_TOKEN")
login(hf_token)
import wandb
wb_token = user_secrets.get_secret("wandb")
wandb.login(key=wb_token)
run = wandb.init(
project='Fine-tune-DeepSeek-R1-Distill-Llama-8B on Medical COT Dataset',
job_type="training",
anonymous="allow"
)</code>3。预先调节推理
<code>from unsloth import FastLanguageModel
max_seq_length = 2048
dtype = None
load_in_4bit = True
model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "unsloth/DeepSeek-R1-Distill-Llama-8B",
max_seq_length = max_seq_length,
dtype = dtype,
load_in_4bit = load_in_4bit,
token = hf_token,
)</code>用样本医学问题测试模型：
<code>prompt_style = """Below is an instruction that describes a task, paired with an input that provides further context.
Write a response that appropriately completes the request.
Before answering, think carefully about the question and create a step-by-step chain of thoughts to ensure a logical and accurate response.
### Instruction:
You are a medical expert with advanced knowledge in clinical reasoning, diagnostics, and treatment planning.
Please answer the following medical question.
### Question:
{}
### Response:
<think>{}"""</think></code>>观察模型的预先调整推理，并通过微调来确定改进的领域。
<code>question = "A 61-year-old woman with a long history of involuntary urine loss during activities like coughing or sneezing but no leakage at night undergoes a gynecological exam and Q-tip test. Based on these findings, what would cystometry most likely reveal about her residual volume and detrusor contractions?"
FastLanguageModel.for_inference(model)
inputs = tokenizer([prompt_style.format(question, "")], return_tensors="pt").to("cuda")
outputs = model.generate(
input_ids=inputs.input_ids,
attention_mask=inputs.attention_mask,
max_new_tokens=1200,
use_cache=True,
)
response = tokenizer.batch_decode(outputs)
print(response[0].split("### Response:")[1])</code>4。加载和处理数据集
创建一个函数以格式化数据集：
<code>train_prompt_style = """Below is an instruction that describes a task, paired with an input that provides further context.
Write a response that appropriately completes the request.
Before answering, think carefully about the question and create a step-by-step chain of thoughts to ensure a logical and accurate response.
### Instruction:
You are a medical expert with advanced knowledge in clinical reasoning, diagnostics, and treatment planning.
Please answer the following medical question.
### Question:
{}
### Response:
<think>
{}
</think>
{}"""</code>加载并处理数据集：
<code>EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN
def formatting_prompts_func(examples):
inputs = examples["Question"]
cots = examples["Complex_CoT"]
outputs = examples["Response"]
texts = []
for input, cot, output in zip(inputs, cots, outputs):
text = train_prompt_style.format(input, cot, output) + EOS_TOKEN
texts.append(text)
return {
"text": texts,
}</code>5。设置模型
<code>from datasets import load_dataset
dataset = load_dataset("FreedomIntelligence/medical-o1-reasoning-SFT","en", split = "train[0:500]",trust_remote_code=True)
dataset = dataset.map(formatting_prompts_func, batched = True,)
dataset["text"][0]</code>设置教练：
<code>model = FastLanguageModel.get_peft_model(
model,
r=16,
target_modules=[
"q_proj",
"k_proj",
"v_proj",
"o_proj",
"gate_proj",
"up_proj",
"down_proj",
],
lora_alpha=16,
lora_dropout=0,
bias="none",
use_gradient_checkpointing="unsloth", # True or "unsloth" for very long context
random_state=3407,
use_rslora=False,
loftq_config=None,
)</code>6。模型培训
<code>from trl import SFTTrainer
from transformers import TrainingArguments
from unsloth import is_bfloat16_supported
trainer = SFTTrainer(
model=model,
tokenizer=tokenizer,
train_dataset=dataset,
dataset_text_field="text",
max_seq_length=max_seq_length,
dataset_num_proc=2,
args=TrainingArguments(
per_device_train_batch_size=2,
gradient_accumulation_steps=4,
# Use num_train_epochs = 1, warmup_ratio for full training runs!
warmup_steps=5,
max_steps=60,
learning_rate=2e-4,
fp16=not is_bfloat16_supported(),
bf16=is_bfloat16_supported(),
logging_steps=10,
optim="adamw_8bit",
weight_decay=0.01,
lr_scheduler_type="linear",
seed=3407,
output_,
),
)</code>（注意：原始响应包括训练进度的图像；此处省略了这些图像，因为不可能进行图像复制。
7。邮政调节推理<code>trainer_stats = trainer.train()</code>
通过与以前相同的问题查询微调模型来比较结果。 观察推理和响应简洁性的改善。
>在本地保存模型，然后将其推到拥抱的脸部集线器：
（注意：原始响应包括显示成功的模型保存和推动的图像；此处省略了这些。）
>教程结束时，建议使用Bentoml或本地转换为GGEF格式提出部署选项。 它强调了开源LLM的重要性，并强调了O3和操作员AI的OpenAI柜台。 保留了这些资源的链接。
<code>new_model_local = "DeepSeek-R1-Medical-COT" model.save_pretrained(new_model_local) tokenizer.save_pretrained(new_model_local) model.save_pretrained_merged(new_model_local, tokenizer, save_method = "merged_16bit",) new_model_online = "kingabzpro/DeepSeek-R1-Medical-COT" model.push_to_hub(new_model_online) tokenizer.push_to_hub(new_model_online) model.push_to_hub_merged(new_model_online, tokenizer, save_method = "merged_16bit")</code>
>重写的响应在简化结构并删除不必要的重复时维护核心信息。 保留代码块以进行完整。 图像被引用但不复制。
>以上是微调DeepSeek R1（推理模型）的详细内容。更多信息请关注PHP中文网其他相关文章！
