Update stable diffusion signature with more parameters #362
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
FYI @outtanames this might break pixeltable SD integration since defaults are not relayed via the |
spillai
force-pushed
the
spillai/0.0.11-sd-parameter-signature
branch
3 times, most recently
from
eba719d
to
5fa07cc
Compare
- parameter validation only checks if parameter subsets are provided (not fully specified) - updated model spec to allow '.' and '_' in addition to `/` and `-` in parameter names - no need to update SD tests to reflect new parameter signature - fix client-side message length option default `GRPC_MAX_MESSAGE_LENGTH` to 32MB
- packages include torch_tensorrt, torch_neuron, torch_neuronx
…ckage_available()`.
- moved `NOS_RAY_*` constants to `nos.constants` - moved stable diffusion neuron model to `autonomi.nos._internal`
spillai
force-pushed
the
spillai/0.0.11-sd-parameter-signature
branch
from
a8f0847
to
a39c105
Compare
4 tasks
spillai
added a commit
that referenced
this pull request
Oct 16, 2023
## Summary
This PR makes a significant update to the core API that simplifies the
need to specify TaskType's in every `Run()` call and also supports
multiple methods for any wrapped Model.
**Key Features:**
- Updated `Run()` to remove the need to specify `TaskType`s
```python
# Old `Run()` implementation
client.Run(task=TaskType.IMAGE_EMBEDDING, model_name="openai/clip")
# New `Run()` implementation (wraps `__call__`)
client.Run("yolox/yolox-medium", inputs={...})
# New `Run()` implementation with custom methods (wraps `encode_image`)
client.Run("openai/clip", inputs={...}, method="encode_image")
```
- Patch `ModelHandle` with additional methods that may be implemented
(`forward`, `encode_text`, `encode_image` ). This allows us remotely
execute model methods with a familiar API. In addition, we support
`ModelHandlePartial` objects to also allow for async submissions of
specific methods.
submisions for a specific method.
```python
spec: ModelSpec = hub.load_spec("openai/clip")
handle: ModelHandle = manager.load(spec)
# New methods automatically patched
img_embeddings = handle.encode_image(...)
txt_embeddings = handle.encode_image(...)
# ModelHandlePartials allow us to make async `submit()` calls via:
# Async submission of image encoding
handle.encode_image.submit(...)
```
- `ModelHandlePartial` now enables the following
- `handle.encode_image` is actually a `ModelHandlePartial` that wraps
`handle.__call__(..., _method="encode_image")` through it's `__call__`
method.
- `handle.encode_image.submit` wraps the `handle.submit(...,
_method="encode_image")` method.
- `Module` interface now supports multiple methods that are dynamically
patched that is automatically inspected.
- `FunctionSignature` now supports multiple signature methods and
metadata (tasks, model resources now are defined here).
**Tests**
- added checks in `test_hub.py` to test for model info and metadata
Fully functional ModelHandlePartial impl with multiple method signatures
- adds tests for `ModelHandle` and `ModelHandlePartial`s to call custom
methods including `__call__` and `submit()`
- Added tests for multiple signatures and default signatures/methods.
- Added / updated and working tests for `hub`, `common`, `executor` and
`manager`
## Related issues
Resolves #366 #368 #369 #370 #362
## Checks
- [x] `make lint`: I've run `make lint` to lint the changes in this PR.
- [x] `make test`: I've made sure the tests (`make test-cpu` or `make
test`) are passing.
- Additional tests:
- [ ] Benchmark tests (when contributing new models)
- [ ] GPU/HW tests
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
/and-in parameter namesGRPC_MAX_MESSAGE_LENGTHto 32MBSummary
Related issues
Checks
make lint: I've runmake lintto lint the changes in this PR.make test: I've made sure the tests (make test-cpuormake test) are passing.