Use `Jupyter` mode while parsing Notebook files #5552

dhruvmanila · 2023-07-06T05:05:46Z

Summary

Enable using the new Mode::Jupyter for the tokenizer/parser to parse Jupyter line magic tokens.

The individual call to the lexer i.e., lex_starts_at done by various rules should consider the context of the source code (is this content from a Jupyter Notebook?). Thus, a new field source_type (of type PySourceType) is added to Checker which is being passed around as an argument to the relevant functions. This is then used to determine the Mode for the lexer.

Test Plan

Add new test cases to make sure that the magic statement is considered while generating the diagnostic and autofix:

For I001, if there's a magic statement in between two import blocks, they should be sorted independently

fixes: #6090

dhruvmanila · 2023-07-06T05:05:58Z

Current dependencies on/for this PR:

main
- PR Integrate the new Jupyter AST nodes in Ruff #6086
  - PR Use Jupyter mode while parsing Notebook files #5552 👈
    - PR Update F841 autofix to not remove line magic expr #6141
    - PR Update F841 autofix to not remove line magic expr #6117

This comment was auto-generated by Graphite.

dhruvmanila · 2023-07-18T17:09:31Z

crates/ruff_rustpython/src/lib.rs

 pub fn parse_program_tokens(
    lxr: Vec<LexResult>,
    source_path: &str,
+    is_jupyter_notebook: bool,
 ) -> anyhow::Result<Suite, ParseError> {


The reason to use a bool here instead of Mode is that the function is expected to return a Module node which is only possible for 2 (Module, Jupyter) out of the 4 modes.

dhruvmanila · 2023-07-18T17:10:31Z

crates/ruff_rustpython/src/lib.rs

@@ -4,9 +4,9 @@ use rustpython_parser::lexer::LexResult;
 use rustpython_parser::{lexer, Mode, ParseError};

 /// Collect tokens up to and including the first error.
-pub fn tokenize(contents: &str) -> Vec<LexResult> {
+pub fn tokenize(contents: &str, mode: Mode) -> Vec<LexResult> {


We could use a bool here instead of Mode similar to parse_program_tokens but I think this is a general function so this is ok.

github-actions · 2023-07-18T17:27:55Z

PR Check Results

Ecosystem

✅ ecosystem check detected no changes.

Benchmark

Linux

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.00      8.2±0.10ms     5.0 MB/sec    1.00      8.2±0.13ms     5.0 MB/sec
formatter/numpy/ctypeslib.py               1.00  1622.8±28.63µs    10.3 MB/sec    1.00  1626.5±26.19µs    10.2 MB/sec
formatter/numpy/globals.py                 1.01    183.2±5.44µs    16.1 MB/sec    1.00    181.3±1.14µs    16.3 MB/sec
formatter/pydantic/types.py                1.00      3.4±0.07ms     7.4 MB/sec    1.00      3.4±0.06ms     7.4 MB/sec
linter/all-rules/large/dataset.py          1.00     10.6±0.04ms     3.8 MB/sec    1.01     10.8±0.05ms     3.8 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      2.7±0.01ms     6.1 MB/sec    1.00      2.7±0.00ms     6.1 MB/sec
linter/all-rules/numpy/globals.py          1.00    379.3±0.48µs     7.8 MB/sec    1.00    380.5±0.69µs     7.8 MB/sec
linter/all-rules/pydantic/types.py         1.00      4.9±0.07ms     5.2 MB/sec    1.00      4.9±0.03ms     5.2 MB/sec
linter/default-rules/large/dataset.py      1.00      5.2±0.01ms     7.8 MB/sec    1.00      5.2±0.02ms     7.8 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00   1124.2±3.18µs    14.8 MB/sec    1.00   1123.5±3.99µs    14.8 MB/sec
linter/default-rules/numpy/globals.py      1.00    126.8±0.25µs    23.3 MB/sec    1.00    126.5±0.56µs    23.3 MB/sec
linter/default-rules/pydantic/types.py     1.00      2.4±0.04ms    10.8 MB/sec    1.00      2.4±0.03ms    10.8 MB/sec

Windows

group                                      main                                   pr
-----                                      ----                                   --
formatter/large/dataset.py                 1.00      9.8±0.08ms     4.2 MB/sec    1.01      9.8±0.08ms     4.1 MB/sec
formatter/numpy/ctypeslib.py               1.00  1897.3±19.84µs     8.8 MB/sec    1.02  1931.3±21.46µs     8.6 MB/sec
formatter/numpy/globals.py                 1.00    213.2±4.93µs    13.8 MB/sec    1.02    217.1±7.11µs    13.6 MB/sec
formatter/pydantic/types.py                1.00      4.1±0.05ms     6.2 MB/sec    1.02      4.2±0.05ms     6.1 MB/sec
linter/all-rules/large/dataset.py          1.00     13.2±0.10ms     3.1 MB/sec    1.01     13.4±0.22ms     3.0 MB/sec
linter/all-rules/numpy/ctypeslib.py        1.00      3.5±0.03ms     4.8 MB/sec    1.00      3.5±0.04ms     4.8 MB/sec
linter/all-rules/numpy/globals.py          1.00    426.2±6.15µs     6.9 MB/sec    1.01    430.8±7.79µs     6.8 MB/sec
linter/all-rules/pydantic/types.py         1.00      6.0±0.08ms     4.2 MB/sec    1.02      6.1±0.10ms     4.2 MB/sec
linter/default-rules/large/dataset.py      1.00      6.7±0.05ms     6.0 MB/sec    1.00      6.7±0.06ms     6.1 MB/sec
linter/default-rules/numpy/ctypeslib.py    1.00  1383.4±15.61µs    12.0 MB/sec    1.00  1386.2±16.23µs    12.0 MB/sec
linter/default-rules/numpy/globals.py      1.00    159.6±2.89µs    18.5 MB/sec    1.00    159.0±2.55µs    18.6 MB/sec
linter/default-rules/pydantic/types.py     1.00      3.0±0.03ms     8.5 MB/sec    1.00      3.0±0.06ms     8.5 MB/sec

MichaReiser · 2023-07-24T14:48:07Z

crates/ruff/src/importer/insertion.rs

@@ -313,7 +313,7 @@ mod tests {
    fn start_of_file() -> Result<()> {
        fn insert(contents: &str) -> Result<Insertion> {
            let program = Suite::parse(contents, "<filename>")?;
-            let tokens: Vec<LexResult> = ruff_rustpython::tokenize(contents);
+            let tokens: Vec<LexResult> = ruff_rustpython::tokenize(contents, Mode::Module);


Unrelated to this line. Ruff calls into lex_starts_at in various places. Do we need to change these calls to use the right mode?

Yes. Most of the calls are made in the rule implementation so they'll require the correct context.

MichaReiser · 2023-07-24T14:50:40Z

MichaReiser · 2023-07-26T06:02:43Z

@dhruvmanila would it be possible to split out the necessary changes to get Ruff compile? I wanted to pull in my cursor changes but am unable to do so because of the new AST nodes. The alternative is that I put todo!()s everywhere

crates/ruff/src/jupyter/snapshots/ruff__jupyter__notebook__tests__import_sorting.snap

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com) This PR contains the following updates: | Package | Change | Age | Adoption | Passing | Confidence | |---|---|---|---|---|---| | [docs/MarkDown](https://togithub.com/Python-Markdown/markdown) ([changelog](https://togithub.com/Python-Markdown/markdown/blob/master/docs/change_log/index.md)) | `==3.3.7` -> `==3.4.4` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/docs%2fMarkDown/3.4.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/docs%2fMarkDown/3.4.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/docs%2fMarkDown/3.3.7/3.4.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/docs%2fMarkDown/3.3.7/3.4.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | | [docs/mkdocs](https://togithub.com/mkdocs/mkdocs) ([changelog](https://www.mkdocs.org/about/release-notes/)) | `==1.4.3` -> `==1.5.2` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/docs%2fmkdocs/1.5.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/docs%2fmkdocs/1.5.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/docs%2fmkdocs/1.4.3/1.5.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/docs%2fmkdocs/1.4.3/1.5.2?slim=true)](https://docs.renovatebot.com/merge-confidence/) | | [docs/mkdocs-macros-plugin](https://togithub.com/fralau/mkdocs_macros_plugin) | `==1.0.2` -> `==1.0.4` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/docs%2fmkdocs-macros-plugin/1.0.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/docs%2fmkdocs-macros-plugin/1.0.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/docs%2fmkdocs-macros-plugin/1.0.2/1.0.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/docs%2fmkdocs-macros-plugin/1.0.2/1.0.4?slim=true)](https://docs.renovatebot.com/merge-confidence/) | | [docs/mkdocs-material](https://togithub.com/squidfunk/mkdocs-material) | `==9.1.18` -> `==9.1.21` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/docs%2fmkdocs-material/9.1.21?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/docs%2fmkdocs-material/9.1.21?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/docs%2fmkdocs-material/9.1.18/9.1.21?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/docs%2fmkdocs-material/9.1.18/9.1.21?slim=true)](https://docs.renovatebot.com/merge-confidence/) | | [misc-lint/ruff](https://beta.ruff.rs/docs) ([source](https://togithub.com/astral-sh/ruff), [changelog](https://togithub.com/astral-sh/ruff/releases)) | `==0.0.280` -> `==0.0.283` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/misc-lint%2fruff/0.0.283?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/misc-lint%2fruff/0.0.283?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/misc-lint%2fruff/0.0.280/0.0.283?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/misc-lint%2fruff/0.0.280/0.0.283?slim=true)](https://docs.renovatebot.com/merge-confidence/) | | [typecheck/types-Markdown](https://togithub.com/python/typeshed) ([changelog](https://togithub.com/typeshed-internal/stub_uploader/blob/main/data/changelogs/Markdown.md)) | `==3.4.2.9` -> `==3.4.2.10` | [![age](https://developer.mend.io/api/mc/badges/age/pypi/typecheck%2ftypes-Markdown/3.4.2.10?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://developer.mend.io/api/mc/badges/adoption/pypi/typecheck%2ftypes-Markdown/3.4.2.10?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://developer.mend.io/api/mc/badges/compatibility/pypi/typecheck%2ftypes-Markdown/3.4.2.9/3.4.2.10?slim=true)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://developer.mend.io/api/mc/badges/confidence/pypi/typecheck%2ftypes-Markdown/3.4.2.9/3.4.2.10?slim=true)](https://docs.renovatebot.com/merge-confidence/) | --- ### Release Notes <details> <summary>Python-Markdown/markdown (docs/MarkDown)</summary> ### [`v3.4.4`](https://togithub.com/Python-Markdown/markdown/compare/3.4.3...3.4.4) [Compare Source](https://togithub.com/Python-Markdown/markdown/compare/3.4.3...3.4.4) ### [`v3.4.3`](https://togithub.com/Python-Markdown/markdown/compare/3.4.2...3.4.3) [Compare Source](https://togithub.com/Python-Markdown/markdown/compare/3.4.2...3.4.3) ### [`v3.4.2`](https://togithub.com/Python-Markdown/markdown/compare/3.4.1...3.4.2) [Compare Source](https://togithub.com/Python-Markdown/markdown/compare/3.4.1...3.4.2) ### [`v3.4.1`](https://togithub.com/Python-Markdown/markdown/compare/3.4...3.4.1) [Compare Source](https://togithub.com/Python-Markdown/markdown/compare/3.4...3.4.1) ### [`v3.4`](https://togithub.com/Python-Markdown/markdown/compare/3.3.7...3.4) [Compare Source](https://togithub.com/Python-Markdown/markdown/compare/3.3.7...3.4) </details> <details> <summary>mkdocs/mkdocs (docs/mkdocs)</summary> ### [`v1.5.2`](https://togithub.com/mkdocs/mkdocs/releases/tag/1.5.2) [Compare Source](https://togithub.com/mkdocs/mkdocs/compare/1.5.1...1.5.2) - Bugfix (regression in 1.5.0): Restore functionality of `--no-livereload`. ([#3320](https://togithub.com/mkdocs/mkdocs/issues/3320)) - Bugfix (regression in 1.5.0): The new page title detection would sometimes be unable to drop anchorlinks - fix that. ([#3325](https://togithub.com/mkdocs/mkdocs/issues/3325)) - Partly bring back pre-1.5 API: `extra_javascript` items will once again be mostly strings, and only sometimes `ExtraStringValue` (when the extra `script` functionality is used). Plugins should be free to append strings to `config.extra_javascript`, but when reading the values, they must still make sure to read it as `str(value)` in case it is an `ExtraScriptValue` item. For querying the attributes such as `.type` you need to check `isinstance` first. Static type checking will guide you in that. ([#3324](https://togithub.com/mkdocs/mkdocs/issues/3324)) See [commit log](https://togithub.com/mkdocs/mkdocs/compare/1.5.1...1.5.2). ### [`v1.5.1`](https://togithub.com/mkdocs/mkdocs/releases/tag/1.5.1) [Compare Source](https://togithub.com/mkdocs/mkdocs/compare/1.5.0...1.5.1) - Bugfix (regression in 1.5.0): Make it possible to treat `ExtraScriptValue` as a path. This lets some plugins still work despite the breaking change. - Bugfix (regression in 1.5.0): Prevent errors for special setups that have 3 conflicting files, such as `index.html`, `index.md` *and* `README.md` ([#3314](https://togithub.com/mkdocs/mkdocs/issues/3314)) See [commit log](https://togithub.com/mkdocs/mkdocs/compare/1.5.0...1.5.1). ### [`v1.5.0`](https://togithub.com/mkdocs/mkdocs/releases/tag/1.5.0) [Compare Source](https://togithub.com/mkdocs/mkdocs/compare/1.4.3...1.5.0) **New: MkDocs now accepts donations.** Please consider supporting the current maintainer at my new GitHub sponsorship page. MkDocs has been a totally free project since the beginning and wasn't accepting funds. MkDocs will remain free of paywalls, but now you can show your support with donations (one-time and/or recurring). [**Donate for MkDocs - @oprypin sponsors page**](https://togithub.com/sponsors/oprypin) And please also consider these other individuals who have been contributing to the ecosystem for a long time and check out their donations pages: [@facelessuser](https://togithub.com/sponsors/facelessuser) [@pawamoy](https://togithub.com/sponsors/pawamoy) [@Ultrabug](https://togithub.com/sponsors/Ultrabug) *** ##### Release 1.5.0 #### New command `mkdocs get-deps` This command guesses the Python dependencies that a MkDocs site requires in order to build. It simply prints the PyPI packages that need to be installed. In the terminal it can be combined directly with an installation command as follows: ```bash pip install $(mkdocs get-deps) ``` The idea is that right after running this command, you can directly follow it up with `mkdocs build` and it will almost always "just work", without needing to think which dependencies to install. The way it works is by scanning `mkdocs.yml` for `themes:`, `plugins:`, `markdown_extensions:` items and doing a reverse lookup based on a large list of known projects (catalog, see below). Of course, you're welcome to use a "virtualenv" with such a command. Also note that for environments that require stability (for example CI) directly installing deps in this way is not a very reliable approach as it precludes dependency pinning. The command allows overriding which config file is used (instead of `mkdocs.yml` in the current directory) as well as which catalog of projects is used (instead of downloading it from the default location). See [`mkdocs get-deps --help`](https://www.mkdocs.org/user-guide/cli/#mkdocs-get-deps). Context: [#3205](https://togithub.com/mkdocs/mkdocs/issues/3205) #### MkDocs has an official catalog of plugins Check out <https://github.com/mkdocs/catalog> and add all your general-purpose plugins, themes and extensions there, so that they can be looked up through `mkdocs get-deps`. This was renamed from "best-of-mkdocs" and received significant updates. In addition to `pip` installation commands, the page now shows the config boilerplate needed to add a plugin. #### Expanded validation of links ##### Validated links in Markdown > As you may know, within Markdown, MkDocs really only recognizes **relative** links that lead to another physical `*.md` document (or media file). This is a good convention to follow because then the source pages are also freely browsable without MkDocs, for example on GitHub. MkDocs knows that in the output it should turn those `*.md` links into `*.html` as appropriate, and it would also always tell you if such a link doesn't actually lead to an existing file. However, the checks for links were really loose and had many concessions. For example, links that started with `/` ("absolute") and links that *ended* with `/` were left as is and no warning was shown, which allowed such very fragile links to sneak into site sources: links that happen to work right now but get no validation and links that confusingly need an extra level of `..` with `use_directory_urls` enabled. Now, in addition to validating relative links, MkDocs will print `INFO` messages for unrecognized types of links (including absolute links). They look like this: ```text INFO - Doc file 'example.md' contains an absolute link '/foo/bar/', it was left as is. Did you mean 'foo/bar.md'? ``` If you don't want any changes, not even the `INFO` messages, and wish to revert to the silence from MkDocs 1.4, add the following configs to `mkdocs.yml` (**not** recommended): ```yaml validation: absolute_links: ignore unrecognized_links: ignore ``` If, on the opposite end, you want these to print `WARNING` messages and cause `mkdocs build --strict` to fail, you are recommended to configure these to `warn` instead. See [**documentation**](https://www.mkdocs.org/user-guide/configuration/#validation) for actual recommended settings and more details. Context: [#3283](https://togithub.com/mkdocs/mkdocs/issues/3283) ##### Validated links in the nav Links to documents in the [`nav` configuration](https://www.mkdocs.org/user-guide/configuration/#nav) now also have configurable validation, though with no changes to the defaults. You are welcomed to turn on validation for files that were forgotten and excluded from the nav. Example: ```yaml validation: nav: omitted_files: warn absolute_links: warn ``` This can make the following message appear with the `WARNING` level (as opposed to `INFO` as the only option previously), thus being caught by `mkdocs --strict`: ```text INFO - The following pages exist in the docs directory, but are not included in the "nav" configuration: ... ``` See [**documentation**](https://www.mkdocs.org/user-guide/configuration/#validation). Context: [#3283](https://togithub.com/mkdocs/mkdocs/issues/3283), [#1755](https://togithub.com/mkdocs/mkdocs/issues/1755) ##### Mark docs as intentionally "not in nav" There is a new config `not_in_nav`. With it, you can mark particular patterns of files as exempt from the above `omitted_files` warning type; no messages will be printed for them anymore. (As a corollary, setting this config to `*` is the same as ignoring `omitted_files` altogether.) This is useful if you generally like these warnings about files that were forgotten from the nav, but still have some pages that you knowingly excluded from the nav and just want to build and copy them. The `not_in_nav` config is a set of gitignore-like patterns. See the next section for an explanation of another such config. See [**documentation**](https://www.mkdocs.org/user-guide/configuration/#not_in_nav). Context: [#3224](https://togithub.com/mkdocs/mkdocs/issues/3224), [#1888](https://togithub.com/mkdocs/mkdocs/issues/1888) #### Excluded doc files There is a new config `exclude_docs` that tells MkDocs to ignore certain files under `docs_dir` and *not* copy them to the built `site` as part of the build. Historically MkDocs would always ignore file names starting with a dot, and that's all. Now this is all configurable: you can un-ignore these and/or ignore more patterns of files. The `exclude_docs` config follows the [.gitignore pattern format](https://git-scm.com/docs/gitignore#\_pattern_format) and is specified as a multiline YAML string. For example: ```yaml exclude_docs: | *.py # Excludes e.g. docs/hooks/foo.py /drafts # Excludes e.g. docs/drafts/hello.md /requirements.txt # Excludes docs/requirements.txt ``` Validation of links (described above) is also affected by `exclude_docs`. During `mkdocs serve` the messages explain the interaction, whereas during `mkdocs build` excluded files are as good as nonexistent. As an additional related change, if you have a need to have both `README.md` and `index.md` files in a directory but publish only one of them, you can now use this feature to explicitly ignore one of them and avoid warnings. See [**documentation**](https://www.mkdocs.org/user-guide/configuration/#exclude_docs). Context: [#3224](https://togithub.com/mkdocs/mkdocs/issues/3224) ##### Drafts The `exclude_docs` config has another behavior: all excluded Markdown pages will still be previewable in `mkdocs serve` only, just with a "DRAFT" marker on top. Then they will of course be excluded from `mkdocs build` or `gh-deploy`. If you don't want `mkdocs serve` to have any special behaviors and instead want it to perform completely normal builds, use the new flag `mkdocs serve --clean`. See [**documentation**](https://www.mkdocs.org/user-guide/configuration/#exclude_docs). Context: [#3224](https://togithub.com/mkdocs/mkdocs/issues/3224) #### `mkdocs serve` no longer exits after build errors If there was an error (from the config or a plugin) during a site re-build, `mkdocs serve` used to exit after printing a stack trace. Now it will simply freeze the server until the author edits the files to fix the problem, and then will keep reloading. But errors on the *first* build still cause `mkdocs serve` to exit, as before. Context: [#3255](https://togithub.com/mkdocs/mkdocs/issues/3255) #### Page titles will be deduced from any style of heading MkDocs always had the ability to infer the title of a page (if it's not specified in the `nav`) based on the first line of the document, if it had a `<h1>` heading that had to written starting with the exact character `#`. Now any style of Markdown heading is understood ([#1886](https://togithub.com/mkdocs/mkdocs/issues/1886)). Due to the previous simplistic parsing, it was also impossible to use `attr_list` attributes in that first heading ([#3136](https://togithub.com/mkdocs/mkdocs/issues/3136)). Now that is also fixed. #### Markdown extensions can use paths relative to the current document This is aimed at extensions such as `pymdownx.snippets` or `markdown_include.include`: you can now specify their include paths to be relative to the currently rendered Markdown document, or relative to the `docs_dir`. Any other extension can of course also make use of the new `!relative` YAML tag. ```yaml markdown_extensions: - pymdownx.snippets: base_path: !relative ``` See [**documentation**](https://www.mkdocs.org/user-guide/configuration/#paths-relative-to-the-current-file-or-site). Context: [#2154](https://togithub.com/mkdocs/mkdocs/issues/2154), [#3258](https://togithub.com/mkdocs/mkdocs/issues/3258) #### `<script>` tags can specify `type="module"` and other attributes In `extra_javascript`, if you use the `.mjs` file extension or explicitly specify a `type: module` key, the script will be added with the `type="module"` attribute. `defer: true` and `async: true` keys are also available. See [updated **documentation** for `extra_javascript`](https://www.mkdocs.org/user-guide/configuration/#extra_javascript). **At first this is only supported in built-in themes, other themes need to follow up, see below.** Context: [#3237](https://togithub.com/mkdocs/mkdocs/issues/3237) #### Changes for theme developers (action required!) Using the construct `{% for script in extra_javascript %}` is now fully obsolete because it cannot allow customizing the attributes of the `<script>` tag. It will keep working but blocks some of MkDocs' features. Instead, you now need to use `config.extra_javascript` (which was already the case for a while) and couple it with the new `script_tag` filter: ```django {%- for script in config.extra_javascript %} {{ script | script_tag }} {%- endfor %} ``` See [**documentation**](https://www.mkdocs.org/dev-guide/themes/#picking-up-css-and-javascript-from-the-config). #### Upgrades for plugin developers - Breaking change: `config.extra_javascript` is no longer a plain list of strings, but instead a list of `ExtraScriptValue` items. So you can no longer treat the list values as strings. If you want to keep compatibility with old versions, just always reference the items as `str(item)` instead. And you can still append plain strings to the list if you wish. See information about `<script>` tags above. Context: [#3237](https://togithub.com/mkdocs/mkdocs/issues/3237) - `File` has a new attribute `inclusion`. Its value is calculated from both the `exclude_docs` and `not_in_nav` configs, and implements their behavior. Plugins can read this value or write to it. New `File` instances by default follow whatever the configs say, but plugins can choose to make this decision explicitly, per file. - When creating a `File`, one can now set a `dest_uri` directly, rather than having to update it (and other dependent attributes) after creation. [Context](https://togithub.com/mkdocs/mkdocs/commit/d5af6426c52421f1113f6dcc591de1e01bea48bd) - A new config option was added - `DictOfItems`. Similarly to `ListOfItems`, it validates a mapping of config options that all have the same type. Keys are arbitrary but always strings. Context: [#3242](https://togithub.com/mkdocs/mkdocs/issues/3242) - A new function `get_plugin_logger` was added. In order to opt into a standardized way for plugins to log messages, please use the idiom: ```python log = mkdocs.plugins.get_plugin_logger(__name__) ... log.info("Hello, world") ``` Context: [#3245](https://togithub.com/mkdocs/mkdocs/issues/3245) - `SubConfig` config option can be conveniently subclassed with a particular type of config specified. For example, `class ExtraScript(SubConfig[ExtraScriptValue]):`. To see how this is useful, search for this class in code. [Context](https://togithub.com/mkdocs/mkdocs/commit/73e503990e3e3504bfe1cb627d41a7e97970687e) - Bugfix: `SubConfig` had a bug where paths (from `FilesystemObject` options) were not made relative to the main config file as intended, because `config_file_path` was not properly inherited to it. This is now fixed. Context: [#3265](https://togithub.com/mkdocs/mkdocs/issues/3265) - `Config` members now have a way to avoid clashing with Python's reserved words. This is achieved by stripping a trailing underscore from each member's name. Example of adding an `async` boolean option that can be set by the user as `async: true` and read programmatically as `config.async_`: ```python class ExampleConfig(Config): async_ = Type(bool, default=False) ``` Previously making a config key with a reserved name was impossible with new-style schemas. [Context](https://togithub.com/mkdocs/mkdocs/commit/1db8e884fa7135a49adf7740add5d875a16a18bc) - `Theme` has its attributes properly declared and gained new attributes `theme.locale`, `theme.custom_dir`. - Some type annotations were made more precise. For example: - The `context` parameter has gained the type `TemplateContext` (`TypedDict`). [Context](https://togithub.com/mkdocs/mkdocs/commit/0f793b9984c7e6a1d53ce874e7d17b6d27ebf4b2) - The classes `Page`, `Section`, `Link` now have a common base class `StructureItem`. [Context](https://togithub.com/mkdocs/mkdocs/commit/01be507e30b05db0a4c44ef05ba62b2098010653) - Some methods stopped accepting `Config` and only accept `MkDocsConfig` as was originally intended. [Context](https://togithub.com/mkdocs/mkdocs/commit/c459cd24fc0320333f51525e9cf681d4a8370f50) - `config.mdx_configs` got a proper type. Context: [#3229](https://togithub.com/mkdocs/mkdocs/issues/3229) #### Theme updates - Built-in themes mostly stopped relying on `<script defer>`. This may affect some usages of `extra_javascript`, mainly remove the need for custom handling of "has the page fully loaded yet". Context: [#3237](https://togithub.com/mkdocs/mkdocs/issues/3237) - "mkdocs" theme now has a styling for `>` blockquotes, previously they were not distinguished at all. Context: [#3291](https://togithub.com/mkdocs/mkdocs/issues/3291) - "readthedocs" theme was updated to v1.2.0 according to upstream, with improved styles for `<kbd>` and breadcrumb navigation. Context: [#3058](https://togithub.com/mkdocs/mkdocs/issues/3058) - Both built-in themes had their version of highlight.js updated to 11.8.0, and jQuery updated to 3.6.0. #### Bug fixes ##### Relative paths in the nav can traverse above the root Regression in 1.2 - relative paths in the nav could no longer traverse above the site's root and were truncated to the root. Although such traversal is discouraged and produces a warning, this was a documented behavior. The behavior is now restored. Context: [#2752](https://togithub.com/mkdocs/mkdocs/issues/2752), [#3010](https://togithub.com/mkdocs/mkdocs/issues/3010) ##### MkDocs can accept the config from stdin This can be used for config overrides on the fly. See updated section at the bottom of [Configuration Inheritance](https://www.mkdocs.org/user-guide/configuration/#configuration-inheritance). The command to use this is `mkdocs build -f -`. In previous versions doing this led to an error. [Context](https://togithub.com/mkdocs/mkdocs/commit/d5bb15fa108da86a8e53fb7d84109d8f8d9d6453) #### New command line flags - `mkdocs --no-color build` disables color output and line wrapping. This option is also available through an environment variable `NO_COLOR=true`. Context: [#3282](https://togithub.com/mkdocs/mkdocs/issues/3282) - `mkdocs build --no-strict` overrides the `strict` config to `false`. Context: [#3254](https://togithub.com/mkdocs/mkdocs/issues/3254) - `mkdocs build -f -` (described directly above). - `mkdocs serve --clean` (described above). - `mkdocs serve --dirty` is the new name of `mkdocs serve --dirtyreload`. #### Deprecations - `extra_javascript` underwent a change that can break plugins in rare cases, and it requires attention from theme developers. See respective entries above. - Python-Markdown was unpinned from `<3.4`. That version is known to remove functionality. If you are affected by those removals, you can still choose to pin the version for yourself: `Markdown <3.4`. Context: [#3222](https://togithub.com/mkdocs/mkdocs/issues/3222), [#2892](https://togithub.com/mkdocs/mkdocs/issues/2892) - `mkdocs.utils.warning_filter` now shows a warning about being deprecated. It does nothing since MkDocs 1.2. Consider `get_plugin_logger` or just logging under `mkdocs.plugins.*` instead. Context: [#3008](https://togithub.com/mkdocs/mkdocs/issues/3008) - Accessing the `_vars` attribute of a `Theme` is deprecated - just access the keys directly. - Accessing the `user_configs` attribute of a `Config` is deprecated. Note: instead of `config.user_configs[*]['theme']['custom_dir']`, please use the new attribute `config.theme.custom_dir`. Other small improvements; see [commit log](https://togithub.com/mkdocs/mkdocs/compare/1.4.3...1.5.0). </details> <details> <summary>fralau/mkdocs_macros_plugin (docs/mkdocs-macros-plugin)</summary> ### [`v1.0.4`](https://togithub.com/fralau/mkdocs_macros_plugin/blob/HEAD/CHANGELOG.md#104-2023-08-07) [Compare Source](https://togithub.com/fralau/mkdocs_macros_plugin/compare/v1.0.2...v1.0.4) - Fixed: Warning due to filter issue with mkdocs >= 1.5 - Fixed: Debug html tables (including for `macro_info()`) are now readable also in dark mode. </details> <details> <summary>squidfunk/mkdocs-material (docs/mkdocs-material)</summary> ### [`v9.1.21`](https://togithub.com/squidfunk/mkdocs-material/releases/tag/9.1.21): mkdocs-material-9.1.21 [Compare Source](https://togithub.com/squidfunk/mkdocs-material/compare/9.1.20...9.1.21) - Fixed MkDocs 1.4 compat issue in social plugin (9.1.20 regression) ### [`v9.1.20`](https://togithub.com/squidfunk/mkdocs-material/releases/tag/9.1.20): mkdocs-material-9.1.20 [Compare Source](https://togithub.com/squidfunk/mkdocs-material/compare/9.1.19...9.1.20) - Updated Sanskrit translations - Fixed deprecation warnings for social plugin ### [`v9.1.19`](https://togithub.com/squidfunk/mkdocs-material/releases/tag/9.1.19): mkdocs-material-9.1.19 [Compare Source](https://togithub.com/squidfunk/mkdocs-material/compare/9.1.18...9.1.19) - Added support for MkDocs 1.5+ - Fixed [#5699](https://togithub.com/squidfunk/mkdocs-material/issues/5699): Improve error reporting in social plugin </details> <details> <summary>astral-sh/ruff (misc-lint/ruff)</summary> ### [`v0.0.283`](https://togithub.com/astral-sh/ruff/releases/tag/v0.0.283) [Compare Source](https://togithub.com/astral-sh/ruff/compare/v0.0.282...v0.0.283)  #### What's Changed ##### Breaking Changes - Assume Python 3.8 instead of 3.10 for target version by [@zanieb](https://togithub.com/zanieb) in [https://github.com/astral-sh/ruff/pull/6397](https://togithub.com/astral-sh/ruff/pull/6397) ##### Rules - \[`flake8-pyi`] `PYI019`: Detects if a type variable is used instead of `Self` in return annotations by [@qdegraaf](https://togithub.com/qdegraaf) in [https://github.com/astral-sh/ruff/pull/6204](https://togithub.com/astral-sh/ruff/pull/6204) - \[`flake8-pyi`] `PYI051`: Detects unions of `Literal` types by [@LaBatata101](https://togithub.com/LaBatata101) in [https://github.com/astral-sh/ruff/pull/6215](https://togithub.com/astral-sh/ruff/pull/6215) - \[`flake8-pyi`] `PYI055`: Detects unions of `type`s by [@LaBatata101](https://togithub.com/LaBatata101) in [https://github.com/astral-sh/ruff/pull/6316](https://togithub.com/astral-sh/ruff/pull/6316) - \[`pylint`] `E1300`: Detects invalid string format characters by [@silvanocerza](https://togithub.com/silvanocerza) in [https://github.com/astral-sh/ruff/pull/6171](https://togithub.com/astral-sh/ruff/pull/6171) - \[`pyupgrade`] `UP040`: Upgrades type alias annotations to use PEP-695 syntax by [@zanieb](https://togithub.com/zanieb) in [https://github.com/astral-sh/ruff/pull/6289](https://togithub.com/astral-sh/ruff/pull/6289) ##### Rule Changes - \[`flake8-boolean-trap`] `FBT003`: Add `is_` and `is_not` to excluded functions by [@zanieb](https://togithub.com/zanieb) in [https://github.com/astral-sh/ruff/pull/6307](https://togithub.com/astral-sh/ruff/pull/6307) - \[`flake8-logging-format`] Allow capitalized names for logger candidate heuristic match by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6356](https://togithub.com/astral-sh/ruff/pull/6356) - \[`flake8-pyi`] Applicable rules are now checked non-stub code by [@andersk](https://togithub.com/andersk) in [https://github.com/astral-sh/ruff/pull/6297](https://togithub.com/astral-sh/ruff/pull/6297) - `PYI013`: [`ellipsis-in-non-empty-class-body`](https://beta.ruff.rs/docs/rules/ellipsis-in-non-empty-class-body) - `PYI016`: [`duplicate-union-member`](https://beta.ruff.rs/docs/rules/duplicate-union-member) - `PYI018`: [`unused-private-type-var`](https://beta.ruff.rs/docs/rules/unused-private-type-var) - `PYI019`: [`custom-type-var-return-type`](https://beta.ruff.rs/docs/rules/custom-type-var-return-type) - `PYI024`: [`collections-named-tuple`](https://beta.ruff.rs/docs/rules/collections-named-tuple) - `PYI025`: [`unaliased-collections-abc-set-import`](https://beta.ruff.rs/docs/rules/unaliased-collections-abc-set-import) - `PYI030`: [`unnecessary-literal-union`](https://beta.ruff.rs/docs/rules/unnecessary-literal-union) - `PYI032`: [`any-eq-ne-annotation`](https://beta.ruff.rs/docs/rules/any-eq-ne-annotation) - `PYI034`: [`non-self-return-type`](https://beta.ruff.rs/docs/rules/non-self-return-type) - `PYI036`: [`bad-exit-annotation`](https://beta.ruff.rs/docs/rules/bad-exit-annotation) - `PYI041`: [`redundant-numeric-union`](https://beta.ruff.rs/docs/rules/redundant-numeric-union) - `PYI042`: [`snake-case-type-alias`](https://beta.ruff.rs/docs/rules/snake-case-type-alias) - `PYI043`: [`t-suffixed-type-alias`](https://beta.ruff.rs/docs/rules/t-suffixed-type-alias) - `PYI045`: [`iter-method-return-iterable`](https://beta.ruff.rs/docs/rules/iter-method-return-iterable) - `PYI046`: [`unused-private-protocol`](https://beta.ruff.rs/docs/rules/unused-private-protocol) - `PYI047`: [`unused-private-type-alias`](https://beta.ruff.rs/docs/rules/unused-private-type-alias) - `PYI049`: [`unused-private-typed-dict`](https://beta.ruff.rs/docs/rules/unused-private-typed-dict) - `PYI050`: [`no-return-argument-annotation-in-stub`](https://beta.ruff.rs/docs/rules/no-return-argument-annotation-in-stub) (Python ≥ 3.11) - `PYI051`: [`redundant-literal-union`](https://beta.ruff.rs/docs/rules/redundant-literal-union) - `PYI056`: [`unsupported-method-call-on-all`](https://beta.ruff.rs/docs/rules/unsupported-method-call-on-all) - \[`flake8-pyi`] `PYI027` is being replaced by `PYI022` / `UP035` by [@LaBatata101](https://togithub.com/LaBatata101) in [https://github.com/astral-sh/ruff/pull/6354](https://togithub.com/astral-sh/ruff/pull/6354) - \[`pydocstyle`] `D103`: Don't require docstrings in `.pyi` files by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6239](https://togithub.com/astral-sh/ruff/pull/6239) - \[`pydocstyle`] `D203`: Ignore same-line docstrings for lines-before and lines-after rules by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6344](https://togithub.com/astral-sh/ruff/pull/6344) - \[`pylint`] `PLE0605`: Allow generic tuple and list calls in `__all__` by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6247](https://togithub.com/astral-sh/ruff/pull/6247) - \[`pylint`] `PLR0124`: Add detection of comparisons with built-in calls by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6324](https://togithub.com/astral-sh/ruff/pull/6324) - \[`pyupgrade`] `UP032`: Add support for `await` expressions in f-strings by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6304](https://togithub.com/astral-sh/ruff/pull/6304) - \[`pyupgrade`] `UP032`: Add support for implicitly concatenated strings by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6263](https://togithub.com/astral-sh/ruff/pull/6263) - \[`pyupgrade`] `UP032`: Add support for repeated format fields by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6266](https://togithub.com/astral-sh/ruff/pull/6266) - \[`ruff`] `RUF012`: Permit `ClassVar` and `Final` without subscript by [@bluetech](https://togithub.com/bluetech) in [https://github.com/astral-sh/ruff/pull/6273](https://togithub.com/astral-sh/ruff/pull/6273) ##### Bug Fixes - \[`flake8-bugbear`] `B006`: Respect `typing_extensions` imports of `Annotated` by [@PIG208](https://togithub.com/PIG208) in [https://github.com/astral-sh/ruff/pull/6361](https://togithub.com/astral-sh/ruff/pull/6361) - \[`flake8-pyi`] `PYI019`: Fix panic with positional-only arguments by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6350](https://togithub.com/astral-sh/ruff/pull/6350) - \[`flake8-use-pathlib`] Avoid raising `PTH206` with `maxsplit` by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6283](https://togithub.com/astral-sh/ruff/pull/6283) - \[`flake8`] `F841`: Update autofix to not remove Jupyer magic expressions by [@dhruvmanila](https://togithub.com/dhruvmanila) in [https://github.com/astral-sh/ruff/pull/6141](https://togithub.com/astral-sh/ruff/pull/6141) - \[`pycodestyle`] `E721`: Include comparisons to builtin types by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6325](https://togithub.com/astral-sh/ruff/pull/6325) - \[`pycodestyle`] `E721`: Match left-hand side `types()` call in `types-comparison` by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6326](https://togithub.com/astral-sh/ruff/pull/6326) - \[`pyupgrade`] `UP031`: Avoid auto-fixing if there are comments within the right-hand side by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6364](https://togithub.com/astral-sh/ruff/pull/6364) - \[`pyupgrade`] `UP032`: Avoid auto-fixing if comments are present around format call arguments by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6342](https://togithub.com/astral-sh/ruff/pull/6342) - \[`pyupgrade`] `UP032`: Improve invalid expression check by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6308](https://togithub.com/astral-sh/ruff/pull/6308) - Avoid attempting to fix `.format(...)` calls with too-few-arguments by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6401](https://togithub.com/astral-sh/ruff/pull/6401) - Fix bug where `.gitignore` files in parent directories were incorrectly used by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6368](https://togithub.com/astral-sh/ruff/pull/6368) - Fix duplicate violations raised on nested bitwise or `Union` expressions by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6399](https://togithub.com/astral-sh/ruff/pull/6399) ##### Playground - Add a simple tooltip to the sidebar by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6295](https://togithub.com/astral-sh/ruff/pull/6295) - Add an icon for FIR by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6292](https://togithub.com/astral-sh/ruff/pull/6292) - Increase icon opacity on-hover by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6291](https://togithub.com/astral-sh/ruff/pull/6291) - Tweak background on theme button by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6290](https://togithub.com/astral-sh/ruff/pull/6290) ##### Other Changes - Improve handling of violations around Jupyter magic expressions by [@dhruvmanila](https://togithub.com/dhruvmanila) in [https://github.com/astral-sh/ruff/pull/5552](https://togithub.com/astral-sh/ruff/pull/5552) - Reduce memory usage by boxing type params and arguments fields on the class definition node by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6275](https://togithub.com/astral-sh/ruff/pull/6275) - Upgrade Rust to 1.71 by [@zanieb](https://togithub.com/zanieb) in [https://github.com/astral-sh/ruff/pull/6323](https://togithub.com/astral-sh/ruff/pull/6323) #### New Contributors - [@silvanocerza](https://togithub.com/silvanocerza) made their first contribution in [https://github.com/astral-sh/ruff/pull/6171](https://togithub.com/astral-sh/ruff/pull/6171) - [@PIG208](https://togithub.com/PIG208) made their first contribution in [https://github.com/astral-sh/ruff/pull/6361](https://togithub.com/astral-sh/ruff/pull/6361) **Full Changelog**: https://github.com/astral-sh/ruff/compare/v0.0.282...v0.0.283 ### [`v0.0.282`](https://togithub.com/astral-sh/ruff/releases/tag/v0.0.282) [Compare Source](https://togithub.com/astral-sh/ruff/compare/v0.0.281...v0.0.282)  #### What's Changed ##### Bug Fixes - Reset model state when exiting deferred visitors by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6208](https://togithub.com/astral-sh/ruff/pull/6208) - Skip trivia when searching for named exception by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6218](https://togithub.com/astral-sh/ruff/pull/6218) - Avoid PERF401 false positive on list access in loop by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6220](https://togithub.com/astral-sh/ruff/pull/6220) - Avoid detecting continuations at non-start-of-line by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6219](https://togithub.com/astral-sh/ruff/pull/6219) - Revert "Expand scope of `quoted-annotation` rule ([#5766](https://togithub.com/astral-sh/ruff/issues/5766))" by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6237](https://togithub.com/astral-sh/ruff/pull/6237) #### New Contributors - [@klistwan](https://togithub.com/klistwan) made their first contribution in [https://github.com/astral-sh/ruff/pull/6146](https://togithub.com/astral-sh/ruff/pull/6146) **Full Changelog**: https://github.com/astral-sh/ruff/compare/v0.0.281...v0.0.282 ### [`v0.0.281`](https://togithub.com/astral-sh/ruff/releases/tag/v0.0.281) [Compare Source](https://togithub.com/astral-sh/ruff/compare/v0.0.280...v0.0.281)  #### What's Changed See the [release blog post](https://astral.sh/blog/ruff-v0.0.281) for more, including detailed descriptions of any newly added rules. ##### New rules - \[`flake8-pyi`] Implement `PYI018` by [@LaBatata101](https://togithub.com/LaBatata101) in [https://github.com/astral-sh/ruff/pull/6018](https://togithub.com/astral-sh/ruff/pull/6018) - \[`flake8-pyi`] Implement `PYI047` by [@LaBatata101](https://togithub.com/LaBatata101) in [https://github.com/astral-sh/ruff/pull/6134](https://togithub.com/astral-sh/ruff/pull/6134) - \[`flake8-pyi`] Implement `PYI049` by [@LaBatata101](https://togithub.com/LaBatata101) in [https://github.com/astral-sh/ruff/pull/6136](https://togithub.com/astral-sh/ruff/pull/6136) - \[`flake8-pyi`] Implement `PYI056` by [@LaBatata101](https://togithub.com/LaBatata101) in [https://github.com/astral-sh/ruff/pull/5959](https://togithub.com/astral-sh/ruff/pull/5959) - \[`flake8-pyi`] Implement `PYI046` by [@LaBatata101](https://togithub.com/LaBatata101) in [https://github.com/astral-sh/ruff/pull/6098](https://togithub.com/astral-sh/ruff/pull/6098) - \[`flake8-use-pathlib`] Implement `glob` (`PTH207`) by [@sbrugman](https://togithub.com/sbrugman) in [https://github.com/astral-sh/ruff/pull/5939](https://togithub.com/astral-sh/ruff/pull/5939) - \[`flake8-use-pathlib`] Implement `os-sep-split` (`PTH206`) by [@sbrugman](https://togithub.com/sbrugman) in [https://github.com/astral-sh/ruff/pull/5936](https://togithub.com/astral-sh/ruff/pull/5936) - \[`pycodestyle`] Implement `E241` and `E242` (tab/multiple ws after commas) by [@akx](https://togithub.com/akx) in [https://github.com/astral-sh/ruff/pull/6094](https://togithub.com/astral-sh/ruff/pull/6094) - \[`pylint`] Implement `self-assigning-variable` (`W0127`) by [@tjkuson](https://togithub.com/tjkuson) in [https://github.com/astral-sh/ruff/pull/6015](https://togithub.com/astral-sh/ruff/pull/6015) - \[`pylint`] Implement `eq-without-hash` rule (PLW1641) by [@jelly](https://togithub.com/jelly) in [https://github.com/astral-sh/ruff/pull/5955](https://togithub.com/astral-sh/ruff/pull/5955) - \[`pylint`] Implement `subprocess-popen-preexec-fn` (`W1509`) by [@tjkuson](https://togithub.com/tjkuson) in [https://github.com/astral-sh/ruff/pull/5978](https://togithub.com/astral-sh/ruff/pull/5978) ##### Rule changes - \[`flake8-bugbear`] Extends `B002` to detect unary prefix decrement operators by [@tjkuson](https://togithub.com/tjkuson) in [https://github.com/astral-sh/ruff/pull/5998](https://togithub.com/astral-sh/ruff/pull/5998) - \[`flake8-pytest-style`] Avoid raising PT012 for simple `with` statements by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6081](https://togithub.com/astral-sh/ruff/pull/6081) - \[`flake8-pytest-style`] Allow pytest.raises body to contain a single func or class definition by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6083](https://togithub.com/astral-sh/ruff/pull/6083) - \[`flake8-simplify`] Extend SIM118 with `not in` by [@sbrugman](https://togithub.com/sbrugman) in [https://github.com/astral-sh/ruff/pull/5995](https://togithub.com/astral-sh/ruff/pull/5995) - \[`perflint`] Skip `PERF203` violations for multi-statement loops by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6145](https://togithub.com/astral-sh/ruff/pull/6145) - \[`pyupgrade`] Add support for `int`, `float`, `bool` in `UP018` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [https://github.com/astral-sh/ruff/pull/6013](https://togithub.com/astral-sh/ruff/pull/6013) - \[`tryceratops`] Add suggested fix for `TRY201` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [https://github.com/astral-sh/ruff/pull/6008](https://togithub.com/astral-sh/ruff/pull/6008) ##### Settings - Allow specification of `logging.Logger` re-exports via `logger-objects` by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/5750](https://togithub.com/astral-sh/ruff/pull/5750) - Set default `max-complexity` to 10 for empty McCabe settings by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6073](https://togithub.com/astral-sh/ruff/pull/6073) - \[`pep8-naming`]: New config option `extend-ignore-names` by [@dhruvmanila](https://togithub.com/dhruvmanila) in [https://github.com/astral-sh/ruff/pull/6169](https://togithub.com/astral-sh/ruff/pull/6169) ##### Bug Fixes - Avoid raising `UP032` if `format` call arguments contain multiline expressions by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/5971](https://togithub.com/astral-sh/ruff/pull/5971) - Fix `F507` false positive by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/5986](https://togithub.com/astral-sh/ruff/pull/5986) - Fix logging rules with whitespace around dot by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6022](https://togithub.com/astral-sh/ruff/pull/6022) - Ignore end-of-line comments when dirtying if-with-same-arms branches by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6031](https://togithub.com/astral-sh/ruff/pull/6031) - Avoid treating `Literal` members as expressions with `__future__` by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6032](https://togithub.com/astral-sh/ruff/pull/6032) - Avoid refactoring `x[:1]`-like slices in RUF015 by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6150](https://togithub.com/astral-sh/ruff/pull/6150) - Avoid key-in-dict violations for `self` accesses by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6165](https://togithub.com/astral-sh/ruff/pull/6165) - Avoid falsely marking non-submodules as submodule aliases by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6182](https://togithub.com/astral-sh/ruff/pull/6182) - Flag `[` as an invalid noqa suffix by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/5982](https://togithub.com/astral-sh/ruff/pull/5982) - Ignore some common builtin overrides on standard library subclasses by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6074](https://togithub.com/astral-sh/ruff/pull/6074) - Avoid A003 violations for explicitly overridden methods by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6076](https://togithub.com/astral-sh/ruff/pull/6076) - Include file permissions in key for cached files by [@zanieb](https://togithub.com/zanieb) in [https://github.com/astral-sh/ruff/pull/5901](https://togithub.com/astral-sh/ruff/pull/5901) - Ignore `explicit-string-concatenation` on single line by [@tjkuson](https://togithub.com/tjkuson) in [https://github.com/astral-sh/ruff/pull/6028](https://togithub.com/astral-sh/ruff/pull/6028) - Fix `SIM102` to handle indented `elif` by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6072](https://togithub.com/astral-sh/ruff/pull/6072) - Raise `PTH201` for `Path("")` by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/6095](https://togithub.com/astral-sh/ruff/pull/6095) - Update `SIM115` to cover `pathlib.Path.open` by [@harupy](https://togithub.com/harupy) in [https://github.com/astral-sh/ruff/pull/611](https://togithub.com/astral-sh/ruff/pull/611) - Ignore end-of-line file exemption comments by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6160](https://togithub.com/astral-sh/ruff/pull/6160) - Implement `--diff` for Jupyter Notebooks by [@dhruvmanila](https://togithub.com/dhruvmanila) in [https://github.com/astral-sh/ruff/pull/6149](https://togithub.com/astral-sh/ruff/pull/6149) - Skip BOM when determining Locator's line starts by [@charliermarsh](https://togithub.com/charliermarsh) in [https://github.com/astral-sh/ruff/pull/6159](https://togithub.com/astral-sh/ruff/pull/6159) - Avoid parenthesizing comprehension element by [@dhruvmanila](https://togithub.com/dhruvmanila) in [https://github.com/astral-sh/ruff/pull/6198](https://togithub.com/astral-sh/ruff/pull/6198) - Do not raise `SIM105` for non-exceptions by [@sbrugman](https://togithub.com/sbrugman) in [https://github.com/astral-sh/ruff/pull/5985](https://togithub.com/astral-sh/ruff/pull/5985) #### New Contributors - [@AlexWaygood](https://togithub.com/AlexWaygood) made their first contribution in [https://github.com/astral-sh/ruff/pull/5973](https://togithub.com/astral-sh/ruff/pull/5973) - [@RMPR](https://togithub.com/RMPR) made their first contribution in [https://github.com/astral-sh/ruff/pull/6048](https://togithub.com/astral-sh/ruff/pull/6048) - [@arembridge](https://togithub.com/arembridge) made their first contribution in [https://github.com/astral-sh/ruff/pull/6079](https://togithub.com/astral-sh/ruff/pull/6079) - [@Eutropios](https://togithub.com/Eutropios) made their first contribution in [https://github.com/astral-sh/ruff/pull/6082](https://togithub.com/astral-sh/ruff/pull/6082) - [@jelly](https://togithub.com/jelly) made their first contribution in [https://github.com/astral-sh/ruff/pull/5955](https://togithub.com/astral-sh/ruff/pull/5955) **Full Changelog**: https://github.com/astral-sh/ruff/compare/v0.0.280...v0.0.281 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "every 3 months on the first day of the month" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 👻 **Immortal**: This PR will be recreated if closed unmerged. Get [config help](https://togithub.com/renovatebot/renovate/discussions) if that's undesired. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/AlexWaygood/typeshed-stats).  --------- Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com> Co-authored-by: Alex Waygood <[email protected]>

dhruvmanila force-pushed the dhruv/jupyter-cell-magic branch from 9e7eab2 to cee79c1 Compare July 10, 2023 04:58

dhruvmanila force-pushed the dhruv/jupyter-cell-magic branch from cee79c1 to 28c310e Compare July 18, 2023 16:14

dhruvmanila changed the title ~~Allow line magics for Jupyter Notebook~~ Use Jupyter mode while parsing Notebook files Jul 18, 2023

dhruvmanila commented Jul 18, 2023

View reviewed changes

dhruvmanila force-pushed the dhruv/jupyter-cell-magic branch 2 times, most recently from 133624b to fe7b2c2 Compare July 19, 2023 02:27

dhruvmanila changed the base branch from main to dhruv/jupyter-lint-only July 19, 2023 02:27

dhruvmanila mentioned this pull request Jul 19, 2023

Make lint_only aware of the source kind #5876

Merged

Base automatically changed from dhruv/jupyter-lint-only to main July 19, 2023 03:59

dhruvmanila force-pushed the dhruv/jupyter-cell-magic branch from fe7b2c2 to 59244ee Compare July 24, 2023 13:12

MichaReiser reviewed Jul 24, 2023

View reviewed changes

dhruvmanila force-pushed the dhruv/jupyter-cell-magic branch 2 times, most recently from cd8dfb9 to b412795 Compare July 26, 2023 07:24

dhruvmanila changed the base branch from main to dhruv/jupyter-ast-impl July 26, 2023 07:24

dhruvmanila mentioned this pull request Jul 26, 2023

Integrate the new Jupyter AST nodes in Ruff #6086

Merged

dhruvmanila force-pushed the dhruv/jupyter-ast-impl branch from 00c27b8 to 758ba0b Compare July 26, 2023 07:32

dhruvmanila force-pushed the dhruv/jupyter-cell-magic branch 2 times, most recently from d109bfc to a049dd8 Compare July 26, 2023 07:38

Base automatically changed from dhruv/jupyter-ast-impl to main July 26, 2023 08:20

dhruvmanila force-pushed the dhruv/jupyter-cell-magic branch 3 times, most recently from 29c62a9 to 0bcbc95 Compare July 26, 2023 12:32

dhruvmanila commented Jul 26, 2023

View reviewed changes

crates/ruff/src/jupyter/snapshots/ruff__jupyter__notebook__tests__import_sorting.snap Outdated Show resolved Hide resolved

dhruvmanila force-pushed the dhruv/jupyter-cell-magic branch 2 times, most recently from 69ee5e4 to b122d14 Compare July 27, 2023 02:29

dhruvmanila added 10 commits August 2, 2023 11:27

Fix imports

9d14de7

Actually resolve merge conflicts

4c5dd4e

Use Jupyter mode for lexing

96dd177

Use is_jupyter_notebook param

adb790a

Return both original and updated SourceKind in tests

0b958d6

Update test case to include cell magic

2e953cd

Remove leftover imports from rebase

9bc3764

Use PySourceType for detecting lexing/parsing mode

0d27e56

Run cargo fmt

9d94cd6

Fix clippy warnings

bc9ec50

dhruvmanila force-pushed the dhruv/jupyter-cell-magic branch from d9e84c5 to bc9ec50 Compare August 2, 2023 05:59

dhruvmanila added 4 commits August 2, 2023 11:32

Use source_type (leftover from rebase)

689267b

Merge branch 'main' into dhruv/jupyter-cell-magic

f453a7c

Fix CI (missed in the merge)

65fe5cd

Run cargo fmt

66edc7f

MichaReiser mentioned this pull request Aug 4, 2023

Formatter: Add SourceType to context to enable special formatting for stub files #6331

Merged

charliermarsh approved these changes Aug 4, 2023

View reviewed changes

dhruvmanila added 4 commits August 4, 2023 23:27

Merge branch 'main' into dhruv/jupyter-cell-magic

c822cb2

Merge branch 'main' into dhruv/jupyter-cell-magic

8a925c7

Add AsMode trait for PySourceType

ba1fa98

Run cargo fmt

851767e

dhruvmanila enabled auto-merge (squash) August 5, 2023 00:30

dhruvmanila merged commit 32fa057 into main Aug 5, 2023
16 checks passed

dhruvmanila deleted the dhruv/jupyter-cell-magic branch August 5, 2023 00:32

chenrui333 mentioned this pull request Aug 9, 2023

ruff 0.0.283 Homebrew/homebrew-core#139033

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use `Jupyter` mode while parsing Notebook files #5552

Use `Jupyter` mode while parsing Notebook files #5552

dhruvmanila commented Jul 6, 2023 •

edited

Loading

dhruvmanila commented Jul 6, 2023 •

edited

Loading

dhruvmanila Jul 18, 2023

dhruvmanila Jul 18, 2023

github-actions bot commented Jul 18, 2023 •

edited

Loading

MichaReiser Jul 24, 2023

dhruvmanila Jul 26, 2023

MichaReiser commented Jul 24, 2023

MichaReiser commented Jul 26, 2023 •

edited

Loading

Use Jupyter mode while parsing Notebook files #5552

Use Jupyter mode while parsing Notebook files #5552

Conversation

dhruvmanila commented Jul 6, 2023 • edited Loading

Summary

Test Plan

dhruvmanila commented Jul 6, 2023 • edited Loading

dhruvmanila Jul 18, 2023

Choose a reason for hiding this comment

dhruvmanila Jul 18, 2023

Choose a reason for hiding this comment

github-actions bot commented Jul 18, 2023 • edited Loading

PR Check Results

Ecosystem

Benchmark

Linux

Windows

MichaReiser Jul 24, 2023

Choose a reason for hiding this comment

dhruvmanila Jul 26, 2023

Choose a reason for hiding this comment

MichaReiser commented Jul 24, 2023

MichaReiser commented Jul 26, 2023 • edited Loading

Use `Jupyter` mode while parsing Notebook files #5552

Use `Jupyter` mode while parsing Notebook files #5552

dhruvmanila commented Jul 6, 2023 •

edited

Loading

dhruvmanila commented Jul 6, 2023 •

edited

Loading

github-actions bot commented Jul 18, 2023 •

edited

Loading

MichaReiser commented Jul 26, 2023 •

edited

Loading