Skip to content

Packing and lazy_tokenize are incompatible in v3.8.0 #5402

@alanayu

Description

@alanayu

Describe the bug
What the bug is, and how to reproduce, better with screenshots(描述bug以及复现过程,最好有截图)
I found that in v3.8.0.dev, the packing and lazy_tokenize can not be set true at the same time. But the setting is OK in v3.7.0.

How to enjoy the two features simultaneously?

Your hardware and system info
Write your system info like CUDA version/system/GPU/torch version here(在这里给出硬件信息和系统信息,如CUDA版本,系统,GPU型号和torch版本等)

Additional context
Add any other context about the problem here(在这里补充其他信息)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions