Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Finite lorax support #153

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Finite lorax support #153

wants to merge 1 commit into from
+1,044 −12

Conversation

quic-jouachen
Copy link

  • Use case: Users can activate multiple LoRA adapters and compile them with base model. At runtime, they can specify which prompt should utilize which adapter, allowing for mixed adapter usage within the same batch. (refer to examples/lora_models.py for more details)
  • These changes are built for continuous batching inference scenario.

@quic-jouachen quic-jouachen force-pushed the finiteloras branch 3 times, most recently from 3c3615e to 3c33e99 Compare October 10, 2024 19:20
quic-mohitgupta
quic-mohitgupta reviewed Oct 10, 2024
View reviewed changes
QEfficient/exporter/export_hf_to_cloud_ai_100.py Outdated Show resolved Hide resolved
QEfficient/exporter/export_hf_to_cloud_ai_100.py Show resolved Hide resolved
QEfficient/exporter/export_utils.py Show resolved Hide resolved
QEfficient/lora/auto.py Outdated Show resolved Hide resolved
QEfficient/lora/auto.py Outdated Show resolved Hide resolved
QEfficient/lora/auto.py Show resolved Hide resolved
QEfficient/lora/auto.py Show resolved Hide resolved
QEfficient/lora/auto.py Show resolved Hide resolved
tests/lora/test_lora_model.py Show resolved Hide resolved
tests/lora/test_lora_model.py Outdated Show resolved Hide resolved
@quic-jouachen quic-jouachen force-pushed the finiteloras branch 2 times, most recently from 24ddfc6 to 5885208 Compare October 11, 2024 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@quic-mohitgupta quic-mohitgupta quic-mohitgupta left review comments

@quic-rishinr quic-rishinr Awaiting requested review from quic-rishinr quic-rishinr is a code owner

@ochougul ochougul Awaiting requested review from ochougul ochougul is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@quic-jouachen @quic-mohitgupta