Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Propagate module and display name of the operator to backend #5344

Merged
merged 16 commits into from
Mar 7, 2024

Conversation

klecki
Copy link
Contributor

@klecki klecki commented Feb 29, 2024

Category: New feature, Refactoring

Description:

Move module and op name partitioning to backend.
The "__" delimiter splitting of schema name into a module path and operator name is moved to backend.

Python APIs are switched over to query the schema.

ops.Compose is removed from registry of operators - it is mainly used to generate documentation and stubs, Compose is not really an operator but a special Python wrapper. It will be a standalone entity in the ops module and the documentation for it is now listed separately.

Add new _module and _display_name arguments that are updated by the fn and ops API bindings.

OpSpec name() is adjusted to SchemaName() to differentiate that from the name/display name
of the operator and avoid the confusion.

New module that formats module path and operator display name is added in the backend.

Tests are added on Python level, to see if the bindings correctly propagate the snake_case
and CamelCase names (with the proper API specified).

The tests is based on the origin stack trace tests, both are generalized into simple common utility.

In a followup those utilities will be used to report better error messages.

Additional information:

Affected modules and functionalities:

API bindings generation - schema processing,
OpSpec and OpSchema APIs are adjusted and extended
New module for processing names in backend.
ops.Compose.

Key points relevant for the review:

Tests:

  • Existing tests apply
  • New tests added
    • Python tests
    • GTests
    • Benchmark
    • Other
  • N/A

Checklist

Documentation

  • Existing documentation applies
  • Documentation updated
    • Docstring
    • Doxygen
    • RST
    • Jupyter
    • Other
  • N/A

DALI team only

Requirements

  • Implements new requirements
  • Affects existing requirements
  • N/A

REQ IDs: N/A

JIRA TASK: N/A

@klecki klecki changed the title Propagate operator name and instantiation API to backend Propagate API and display name of the operator to backend Feb 29, 2024
The "__" delimiter splitting of schema name into a module path and
operator name is moved to backend.

Python APIs are switched over to query the schema.

ops.Compose is removed from registry of operators - it is mainly used to
generate documentation and stubs, Compose is not really an operator but
a special Python wrapper. It will be a standalone entity in the ops
module and the documentation for it is now listed separately.

Signed-off-by: Krzysztof Lecki <[email protected]>
Signed-off-by: Krzysztof Lecki <[email protected]>
Signed-off-by: Krzysztof Lecki <[email protected]>
Signed-off-by: Krzysztof Lecki <[email protected]>
Signed-off-by: Krzysztof Lecki <[email protected]>
Signed-off-by: Krzysztof Lecki <[email protected]>
Signed-off-by: Krzysztof Lecki <[email protected]>
@klecki klecki marked this pull request as ready for review March 5, 2024 20:00
Signed-off-by: Krzysztof Lecki <[email protected]>
Signed-off-by: Krzysztof Lecki <[email protected]>
@klecki
Copy link
Contributor Author

klecki commented Mar 6, 2024

!build

Signed-off-by: Krzysztof Lecki <[email protected]>
@klecki
Copy link
Contributor Author

klecki commented Mar 6, 2024

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13311633]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13311633]: BUILD FAILED

Signed-off-by: Krzysztof Lecki <[email protected]>
Signed-off-by: Krzysztof Lecki <[email protected]>
@klecki
Copy link
Contributor Author

klecki commented Mar 6, 2024

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13315046]: BUILD STARTED

klecki added a commit to klecki/DALI that referenced this pull request Mar 6, 2024
Move module and op name partitioning to backend

The "__" delimiter splitting of schema name into a module path and
operator name is moved to backend.

Python APIs are switched over to query the schema.

ops.Compose is removed from registry of operators - it is mainly used to
generate documentation and stubs, Compose is not really an operator but
a special Python wrapper. It will be a standalone entity in the ops
module and the documentation for it is now listed separately.

Signed-off-by: Krzysztof Lecki <[email protected]>

Propagate API and display name of the operator to backend

Signed-off-by: Krzysztof Lecki <[email protected]>

Some improvements

Signed-off-by: Krzysztof Lecki <[email protected]>

black

Signed-off-by: Krzysztof Lecki <[email protected]>

Flake

Signed-off-by: Krzysztof Lecki <[email protected]>

C++ lint

Signed-off-by: Krzysztof Lecki <[email protected]>

OpSpec::name() -> OpSpec::SchemaName()

Signed-off-by: Krzysztof Lecki <[email protected]>

Add naming utils to operators

Signed-off-by: Krzysztof Lecki <[email protected]>

Test stub

Signed-off-by: Krzysztof Lecki <[email protected]>

Finish the tests

Signed-off-by: Krzysztof Lecki <[email protected]>

Linting

Signed-off-by: Krzysztof Lecki <[email protected]>

Remove dead code

Signed-off-by: Krzysztof Lecki <[email protected]>

Add missing files

Signed-off-by: Krzysztof Lecki <[email protected]>

Cleanup

Signed-off-by: Krzysztof Lecki <[email protected]>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13315046]: BUILD FAILED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13315046]: BUILD PASSED

Signed-off-by: Krzysztof Lecki <[email protected]>
@klecki
Copy link
Contributor Author

klecki commented Mar 7, 2024

!build

klecki added a commit to klecki/DALI that referenced this pull request Mar 7, 2024
Move module and op name partitioning to backend

The "__" delimiter splitting of schema name into a module path and
operator name is moved to backend.

Python APIs are switched over to query the schema.

ops.Compose is removed from registry of operators - it is mainly used to
generate documentation and stubs, Compose is not really an operator but
a special Python wrapper. It will be a standalone entity in the ops
module and the documentation for it is now listed separately.

Signed-off-by: Krzysztof Lecki <[email protected]>

Propagate API and display name of the operator to backend

Signed-off-by: Krzysztof Lecki <[email protected]>

Some improvements

Signed-off-by: Krzysztof Lecki <[email protected]>

black

Signed-off-by: Krzysztof Lecki <[email protected]>

Flake

Signed-off-by: Krzysztof Lecki <[email protected]>

C++ lint

Signed-off-by: Krzysztof Lecki <[email protected]>

OpSpec::name() -> OpSpec::SchemaName()

Signed-off-by: Krzysztof Lecki <[email protected]>

Add naming utils to operators

Signed-off-by: Krzysztof Lecki <[email protected]>

Test stub

Signed-off-by: Krzysztof Lecki <[email protected]>

Finish the tests

Signed-off-by: Krzysztof Lecki <[email protected]>

Linting

Signed-off-by: Krzysztof Lecki <[email protected]>

Remove dead code

Signed-off-by: Krzysztof Lecki <[email protected]>

Add missing files

Signed-off-by: Krzysztof Lecki <[email protected]>

Cleanup

Signed-off-by: Krzysztof Lecki <[email protected]>

Rework the _api into _module

Signed-off-by: Krzysztof Lecki <[email protected]>
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13344178]: BUILD STARTED

@klecki klecki changed the title Propagate API and display name of the operator to backend Propagate module and display name of the operator to backend Mar 7, 2024
@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13344178]: BUILD PASSED

Signed-off-by: Krzysztof Lecki <[email protected]>
@mzient mzient assigned mzient and unassigned szalpal Mar 7, 2024
@klecki
Copy link
Contributor Author

klecki commented Mar 7, 2024

!build

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13347837]: BUILD STARTED

@dali-automaton
Copy link
Collaborator

CI MESSAGE: [13347837]: BUILD PASSED

@klecki klecki merged commit 6f89fb3 into NVIDIA:main Mar 7, 2024
7 checks passed
@klecki klecki deleted the propagate-op-name branch March 7, 2024 20:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants