Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(v3) Draft PR: add Pyright static typing to hook_points.py #590 #607

Merged
merged 28 commits into from
May 24, 2024

Conversation

starship006
Copy link
Contributor

Description

This is a copy of PR #590 and #464 , but just remade so that @bryce13950 can close it.


This draft PR adds Pyright static typing to hook_points.py. See Alan's message on the slack for motivation/background. Specifically, this PR:

  • modifies the HookPoint and HookedRootModule to support a baseline amount of pyright typing
  • adds a [tool.pyright] configuration to pyproject.toml to configure the typing for this project

It's not entirely complete, but is in a state to where it would benefit from some review/assistance. Before being PR ready, we should:

  • Address some TODO's I've marked on lines of code that I'm ucnertain about
  • Clean up remaining type warnings on hook.py
  • If possible, adjust the pyproject.toml file so the type warnings quit applying to all files

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

starship006 and others added 27 commits November 21, 2023 02:58
Cumulative changes made to transformer lens recently
)

* update activation function to tanh approximation

* keep RMSNorm calcs in float32 and match cfg dtype for embedding scaling

* formatting

* keep mypy happy

* formatting
@bryce13950
Copy link
Collaborator

Alright, there is one more issue left to resolve. I am done for the day, but I am happy to take care of it tomorrow. If you want to look at it feel free, but I don't mind taking care of it.

@starship006
Copy link
Contributor Author

starship006 commented May 23, 2024

okay feel free to knock it out tomorrow if i don't get to it, thanks for the support

@bryce13950
Copy link
Collaborator

All set! Thank you so much for your help and patience on this! If you are interested in contributing more, that would be much appreciated. If you aren't quite sure what you should or could do next, feel free to reach out to me on Slack as "Bryce Meyer" I would be happy to have a quick call to discuss what you are interested in, and to see if we can get you something fun to do.

@bryce13950 bryce13950 merged commit 295aca5 into TransformerLensOrg:dev May 24, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants