Add basic metadata importing to init command #265

cnpryer · 2023-05-30T18:03:04Z

Summary of changes

Add configparser for INI file parsing
Add monotrail-utils for RequirementsTxt requirements file parsing.
rye init by default will attempt an import from setup.cfg and setup.py files (setup.py is prioritized)
Add --no-import argument
Add -r/--requirements and --dev-requirements arguments for requirements files

How it works

If --no-import is not used:

Attempt to read metadata from setup.py
Attempt to read from setup.cfg
Attempt to read requirements from -r/--requirements/--dev-requirements

RequirementsTxt is used from monotrail-utils for requirements file parsing.
- Add monotrail-utils crate konstin/poc-monotrail#53
  - See unsupported features.
  - No license.
If setup.py/setup.cfg is found or requirements are provided rye attempts an import.
- Can skip with --no-import

Tag #191

TODO:

import setup.cfg first-pass
import setup.py first-pass
import requirements first-pass (see unsupported features)

Does not include default file name fallback (requirements.txt,requirements-dev.txt,etc.)

mitsuhiko · 2023-05-30T21:45:37Z

One thought I have is that if --import is not passed, but there is something importable it should probably not do something stupid but let the user know that they can import instead.

cnpryer · 2023-05-31T00:44:45Z

One thought I have is that if --import is not passed, but there is something importable it should probably not do something stupid but let the user know that they can import instead.

e18f12d adds an assumed import. If you'd prefer waiting for user input or something similar/more explicit I'll modify it.

rye/src/cli/init.rs

cnpryer · 2023-06-08T16:00:19Z

I'm changing this PR to only tag #191 instead of closing. I think we'd want to close it when/if requirements parsing is fully supported, when we're able to import from poetry-managed projects (or other tooling), and after we'd decided if we see a future where Rye fully supports setup file managed projects.

cnpryer · 2023-06-12T20:25:24Z

Marking this as ready for a review, but I'd like to do more testing later. Here's where it's at so far:

See PR description for callouts.

We use the monotrail-utils crate for requirements parsing.
We perform the import when rye init is called and a setup.py or setup.cfg is found
You cant import all supported requirements formats (see this link).

cnpryer · 2023-06-12T20:48:32Z

rye/src/cli/init.rs

+        }
+        if let Some(Some(reqs)) = section.get("install_requires") {
+            reqs.lines()
+                .filter_map(|x| Requirement::from_str(x).ok())


This will silently fail

cnpryer · 2023-06-12T20:48:49Z

rye/src/cli/init.rs

+        reqs.iter()
+            .map(ToString::to_string)
+            .map(escape_string)
+            .filter_map(|x| Requirement::from_str(&x).ok())


This will silently fail

rye/src/cli/init.rs

cnpryer · 2023-06-13T19:35:52Z

rye/src/cli/init.rs

+    if let Some(Value::String(name)) = json.get("name") {
+        if metadata.name.is_none() {


Suggested change

if let Some(Value::String(name)) = json.get("name") {

if metadata.name.is_none() {

if metadata.name.is_none() {

if let Some(Value::String(name)) = json.get("name") {

Might flip these

mitsuhiko · 2023-06-15T21:09:39Z

This looks pretty good but I will need to do some tests still. Will try to get to it on the weekend.

cnpryer · 2023-06-15T21:17:28Z

Have you thought about how we want to do testing in general? I'm happy to introduce some here for this change, but I want to align with broader testing of rye.

ischaojie · 2023-06-16T02:41:28Z

I've been concerned about the lack of extensive testing for rye, and if someone can add some tests as a reference (I lack experience in this area of rust), I'd be happy to contribute.

mitsuhiko · 2023-06-16T08:52:35Z

Rye definitely has reached the point where it needs tests. I have started something a while back based on insta-cmd but it was all quite dependent on external sources and felt super fragile.

I made a meta issue for the testing here: #322

mitsuhiko · 2023-06-21T17:29:45Z

I will merge this now. I did not make progress on tests yet, but that should not hold this up.

cnpryer added 8 commits May 30, 2023 13:43

Add rust-ini v0.19.0

0cf1bfc

First-pass init --import w/o requirement file parsing

38f82ab

Add init --import to docs

b47ce76

Merge branch 'main' into init-import

0a9dc4c

Merge branch 'main' into init-import

c1759a6

Remove unused setup.py proxy script imports

ad28ba2

Update init requirements flags value names

3a9b145

First-pass requirement files parse

3d56e62

Does not include default file name fallback (requirements.txt,requirements-dev.txt,etc.)

cnpryer mentioned this pull request May 30, 2023

Support array like string zonyitoo/rust-ini#91

Open

cnpryer added 5 commits May 30, 2023 19:30

Move escape_string into utils module

3b205ad

Import setup.cfg data if Some and parse json with escape_string

67b4b88

Implicitly import metadata from setup files if available

e18f12d

Small chore

fe14f3a

Use json::Value::String match for setup.py parse

538ba89

cnpryer added 4 commits May 31, 2023 09:43

Use configparser v3.0.2

8be6d08

Don't use expect

89e610b

Handle multi-line ini values missing first line

da814a9

Merge branch 'main' into init-import

ca83732

mitsuhiko reviewed May 31, 2023

View reviewed changes

rye/src/cli/init.rs Outdated Show resolved Hide resolved

cnpryer added 4 commits May 31, 2023 13:36

Use lazy boolean operator

083de34

Merge branch 'main' into init-import

fda80c2

Add basic requirements parsing

cf678ed

Update docs

db623f1

cnpryer force-pushed the init-import branch from 02bc4ff to db623f1 Compare May 31, 2023 23:13

Don't unwrap

8ee1e33

cnpryer mentioned this pull request Jun 1, 2023

Re-exporting RequirementsTxt as public konstin/poc-monotrail#51

Closed

cnpryer added 2 commits June 1, 2023 20:02

Use RequirementsTxt for requirements file parsing

b432445

Fix setup.py proxy and use --no-import

56cbb96

cnpryer mentioned this pull request Jun 7, 2023

Add monotrail-utils crate konstin/poc-monotrail#53

Merged

Merge branch 'main' into init-import

4d4ba50

cnpryer added 6 commits June 12, 2023 08:24

Import setup.py first

e0865ff

Ignore .git and .tox dirs

0c5fa3d

Use monotrail-utils fork temporarily

1af86e0

Format Requirement strings

cb950b1

Fix metadata import

b5bb68c

Merge branch 'main' into init-import

2ddc7aa

cnpryer changed the title ~~WIP: Add metadata importing to init command~~ Add metadata importing to init command Jun 12, 2023

cnpryer added 3 commits June 12, 2023 15:11

Use monotrail-utils v0.0.1

f61382b

Fix requirement importing

9051f5f

Use python_requires from setup.py and setup.cfg

a95059b

cnpryer marked this pull request as ready for review June 12, 2023 20:25

cnpryer commented Jun 12, 2023

View reviewed changes

ischaojie reviewed Jun 13, 2023

View reviewed changes

rye/src/cli/init.rs Outdated Show resolved Hide resolved

Fix import_requirements_file comment

8382d29

cnpryer commented Jun 13, 2023

View reviewed changes

cnpryer changed the title ~~Add metadata importing to init command~~ Add basic metadata importing to init command Jun 16, 2023

cnpryer mentioned this pull request Jun 17, 2023

Strategy for Tests #322

Open

mitsuhiko merged commit 4a477d1 into astral-sh:main Jun 21, 2023

mitsuhiko added a commit that referenced this pull request Jun 21, 2023

Added changelog entry for #265

0bde3ba

cnpryer deleted the init-import branch June 21, 2023 17:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add basic metadata importing to init command #265

Add basic metadata importing to init command #265

cnpryer commented May 30, 2023 •

edited

Loading

mitsuhiko commented May 30, 2023

cnpryer commented May 31, 2023 •

edited

Loading

cnpryer commented Jun 8, 2023 •

edited

Loading

cnpryer commented Jun 12, 2023

cnpryer Jun 12, 2023

cnpryer Jun 12, 2023

cnpryer Jun 13, 2023

mitsuhiko commented Jun 15, 2023

cnpryer commented Jun 15, 2023

ischaojie commented Jun 16, 2023

mitsuhiko commented Jun 16, 2023 •

edited

Loading

mitsuhiko commented Jun 21, 2023

		if let Some(Value::String(name)) = json.get("name") {
		if metadata.name.is_none() {

Add basic metadata importing to init command #265

Add basic metadata importing to init command #265

Conversation

cnpryer commented May 30, 2023 • edited Loading

Summary of changes

How it works

mitsuhiko commented May 30, 2023

cnpryer commented May 31, 2023 • edited Loading

cnpryer commented Jun 8, 2023 • edited Loading

cnpryer commented Jun 12, 2023

cnpryer Jun 12, 2023

Choose a reason for hiding this comment

cnpryer Jun 12, 2023

Choose a reason for hiding this comment

cnpryer Jun 13, 2023

Choose a reason for hiding this comment

mitsuhiko commented Jun 15, 2023

cnpryer commented Jun 15, 2023

ischaojie commented Jun 16, 2023

mitsuhiko commented Jun 16, 2023 • edited Loading

mitsuhiko commented Jun 21, 2023

cnpryer commented May 30, 2023 •

edited

Loading

cnpryer commented May 31, 2023 •

edited

Loading

cnpryer commented Jun 8, 2023 •

edited

Loading

mitsuhiko commented Jun 16, 2023 •

edited

Loading