Skip to content

kto训练后,使用lora和merge权重后预测效果差异很大 #4215

@xiaozhu1106

Description

@xiaozhu1106

Describe the bug
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图)
kto训练完成后,使用lora和merge权重后预测效果差异大
一、lora启动方式:
swift deploy
--model Qwen2.5-7B-Instruct
--adapters kto_checkpoint
--infer_backend lmdeploy
二、merge方式:
merge导出脚本:
swift export
--model Qwen2.5-7B-Instruct
--adapters kto_checkpoint
--merge_lora true
merge启动:
swift deploy
--model kto_checkpoint-merged
--infer_backend lmdeploy

Additional context
Add any other context about the problem here(在这里补充其他信息)
lora的相对符合预期,但是merge后,效果变的很差

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions