Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Fused AWQ modules #27896

Merged
merged 1 commit into from
Dec 11, 2023
Merged

[docs] Fused AWQ modules #27896

merged 1 commit into from
Dec 11, 2023

Conversation

stevhliu
Copy link
Member

@stevhliu stevhliu commented Dec 8, 2023

Streamlines the quantization docs a bit with the latest AWQ-fused module update :)

  • I think the reintroduced Benchmarks in the AWQ section is a duplicate because it's already currently at the end of the doc. I moved the Benchmarks to the end because I think comparing the results of all the quantization schemes makes more sense after having read through them. Looking at the benchmarks now in the AWQ section, you don't have as much context about the other quantization schemes (AutoGPTQ, bitsandbytes), making it a bit more difficult to compare IMO.
  • condense the fused/unfused code with the code-switching feature in the docs
  • try to improve the table titles of the fused/unfused benchmarks

Copy link
Contributor

@younesbelkada younesbelkada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, thanks @stevhliu !

Copy link
Collaborator

@ArthurZucker ArthurZucker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 🔥

@stevhliu stevhliu merged commit 3547818 into huggingface:main Dec 11, 2023
8 checks passed
@stevhliu stevhliu deleted the quant-fix branch December 11, 2023 18:41
iantbutler01 pushed a commit to BismuthCloud/transformers that referenced this pull request Dec 16, 2023
staghado pushed a commit to staghado/transformers that referenced this pull request Jan 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants