-
Notifications
You must be signed in to change notification settings - Fork 830
Closed as not planned
Labels
Description
Describe the bug
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图)
kto训练完成后,使用lora和merge权重后预测效果差异大
一、lora启动方式:
swift deploy
--model Qwen2.5-7B-Instruct
--adapters kto_checkpoint
--infer_backend lmdeploy
二、merge方式:
merge导出脚本:
swift export
--model Qwen2.5-7B-Instruct
--adapters kto_checkpoint
--merge_lora true
merge启动:
swift deploy
--model kto_checkpoint-merged
--infer_backend lmdeploy
Additional context
Add any other context about the problem here(在这里补充其他信息)
lora的相对符合预期,但是merge后,效果变的很差