Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allow pytorch stream model loading #729

Merged
merged 2 commits into from
Mar 9, 2021

Conversation

lanking520
Copy link
Contributor

Description

Add a Stream model loading option specifically for PyTorch

@codecov-io
Copy link

codecov-io commented Mar 6, 2021

Codecov Report

Merging #729 (3662376) into master (a6a2232) will increase coverage by 0.02%.
The diff coverage is 15.00%.

Impacted file tree graph

@@             Coverage Diff              @@
##             master     #729      +/-   ##
============================================
+ Coverage     69.15%   69.18%   +0.02%     
- Complexity     3991     3998       +7     
============================================
  Files           462      462              
  Lines         18640    18654      +14     
  Branches       1999     2004       +5     
============================================
+ Hits          12891    12906      +15     
+ Misses         4728     4725       -3     
- Partials       1021     1023       +2     
Impacted Files Coverage Δ Complexity Δ
api/src/main/java/ai/djl/ndarray/NDArray.java 74.25% <ø> (ø) 89.00 <0.00> (ø)
...i/src/main/java/ai/djl/ndarray/NDArrayAdapter.java 0.00% <0.00%> (ø) 0.00 <0.00> (ø)
...ine/src/main/java/ai/djl/dlr/engine/DlrEngine.java 34.48% <0.00%> (-2.56%) 7.00 <1.00> (ø)
...e/src/main/java/ai/djl/mxnet/engine/MxNDArray.java 86.10% <0.00%> (-0.16%) 208.00 <0.00> (ø)
.../main/java/ai/djl/onnxruntime/engine/OrtUtils.java 52.94% <0.00%> (-1.61%) 6.00 <0.00> (ø)
...main/java/ai/djl/paddlepaddle/engine/PpEngine.java 56.66% <0.00%> (-4.05%) 11.00 <3.00> (ø)
...c/main/java/ai/djl/tflite/engine/TfLiteEngine.java 37.93% <0.00%> (-2.81%) 8.00 <1.00> (ø)
...ain/java/ai/djl/onnxruntime/engine/OrtNDArray.java 35.00% <12.50%> (+13.37%) 6.00 <1.00> (+3.00)
...main/java/ai/djl/onnxruntime/engine/OrtEngine.java 58.62% <100.00%> (+10.47%) 11.00 <3.00> (+2.00)
...ain/java/ai/djl/modality/cv/util/NDImageUtils.java 64.47% <0.00%> (+1.31%) 21.00% <0.00%> (+1.00%)
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a6a2232...3662376. Read the comment docs.

@@ -73,6 +73,7 @@ public void testOrt() throws TranslateException, ModelException, IOException {
public void testStringTensor()
throws MalformedModelException, ModelNotFoundException, IOException,
TranslateException {
System.setProperty("ai.djl.onnx.disable_alternative", "true");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about putting this functionality into Criteria?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We cannot simply added to criteria, it is usually scoped into a series of operations

Qing Lan added 2 commits March 8, 2021 16:38
Change-Id: Ibc26261b90de673712e90de0d640a8f32f23763e
@lanking520 lanking520 merged commit a90129e into deepjavalibrary:master Mar 9, 2021
zachgk added a commit that referenced this pull request Mar 11, 2021
* This creates the component which will populate the Download Tab with Download Buttons.

* Making a place for the download buttons.

* Adding the Model Download Handler allowing the backend to feed the links into the Model View and making slight changes for readablity.

* Getting rid of some of the test code.

* Improve Block usability (#712)

* Use builder pattern for Parameter (#661)

* Make XavierInitializer default value & Improve setInitializer (#664)

* Refactor initialize (#675)

* Remove NDManager on getOutputShapes (#710)

* Removing unnecessary logging messages.

* block factory init commit (#697)

* [DOCS] Fixing TrainingListener documentation (#718)

* Fixing TrainingListener documentation

* Fixing PR reviews

* Fix DJL serving flaky test for mac (#721)

Change-Id: I9eccc84b0c34652e50c5fe5a4fe42f2b82d65a3d

* Fixing all of the nits.

* Getting rid of unnecessary methods.

* update onnxruntime along with String tensor (#724)

* Add profiler doc (#722)

* Resolving some comments.

* Using a better criteria incase multiple models have the same name.

* Fixing the java doc.

* Configure verbose of mxnet extra libraries (#728)

Change-Id: I66d54aa496cccbb9e8c0a89eeaa458605958d9c6

* Added a TODO for using the artifact repo to get the base uri.

* paddlepaddle CN notebook (#730)

* paddlepaddle CN notebook

* install font

Change-Id: I2d749e617b0bf78ecbcd168b82c53a1fab49a2c0

* refactor on name

Change-Id: I9e379eee51ceae16391850b3ba9782acb04c4021

* Refine the text

Co-authored-by: gstu1130 <[email protected]>

* add EI documentation (#733)

* add EI documentation

* fix pmd rules

Change-Id: Ieee5577c26f6df2843781f8f9180de35069a5de3

* allow pytorch stream model loading (#729)

* allow pytorch stream model loading

* updates

Change-Id: Ibc26261b90de673712e90de0d640a8f32f23763e

* add NDList decode from inputStream (#734)

Change-Id: I6a31d8b0b955f2dbb762220b101e3928a34699c1

* Remove memory scope and improve memory management (#695)

The MemoryScope reveals a number of shortcomings within the DJL memory
management. While the MemoryScope is deleted, many of them are fixed as part of
this PR.

First, the NDManager.{attach, detach} were renamed to xxxInternal. This is to
differentiate them from the attach and detach methods that are intended to be used.

There are two new concepts in memory management. An NDResource interface was
created to combine the concepts of managed memory that was used in NDArray and
NDList. It could also be used in more classes in the future. This includes the
getManager, attach, and detach.

Within the NDManager, it gains a second "management convention". The first
convention of normal resources are added to the manager and then closed when the
manager closes. This works for small numbers of things on the NDArray, but not
when operations transitively create. So, the second convention is a
tempResource. Instead of freeing them when the manager is closed, they are
returned to their original manager. This is used to create a temporary scope, do
operations within it, and then the inputs and return value are returned to the
parent while the intermediate work is cleaned. This also matches the concepts of
ownership/borrowing as well.

Using these, a few additional helper methods were created. There is
`NDManager.from(resource)` to ease creation of managers based on a resource.
There is also `scopeManager.ret(returnValue)` to help with returning values
outside of the scopeManager. Lastly, there is a `scopeManager.{temp,}AttachAll`
to attach a number of resources to a manager within a single call.

Using these improvements, the new method were applied to the old locations where
MemoryScope was used as well as an additional case in NDManagerEx.

Also, the old attach methods were altered to be `void`. Because the return
values are no longer used anywhere and are not as necessary in the current
scheme, I figured it would simplify things. It also helps for things like
`NDList.attach` which does not have a single original NDManager when attaching.

Change-Id: I91d109cd14d70fa64fd8fffa0b50d88ab053013e

* Remove erroneous random forest application (#726)

The application was changed to the more accurate softmax_regression (matching
the terminology from the D2L book).

Change-Id: I1f69f005bbe38b125f2709c2988d06c14eebb765

* Minor fixes on duplicated code (#736)

* remove methods that already defined in the NDArrayAdapter

Change-Id: I01cc03a7f5b427bf31c6b3fd8d2136f2a27fe93b

* refactor toString

Change-Id: Iea22b16e1daa9f759b55c1a8b8b85536482e551a

* remove sparse NDArray

Change-Id: Icb44096519775f54cb32cc768c14f49e33dc7ea5

* fix test

Change-Id: Icef580ed77e7bba22864ce44577de3cba51e3e41

Co-authored-by: Jake Lee <[email protected]>
Co-authored-by: Lanking <[email protected]>
Co-authored-by: aksrajvanshi <[email protected]>
Co-authored-by: Frank Liu <[email protected]>
Co-authored-by: Zach Kimberg <[email protected]>
@lanking520 lanking520 deleted the pytorch branch May 6, 2021 22:03
Lokiiiiii pushed a commit to Lokiiiiii/djl that referenced this pull request Oct 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants