Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[core worker] Python core worker normal task submission #5566

Merged
merged 188 commits into from
Sep 14, 2019
Merged

[core worker] Python core worker normal task submission #5566

merged 188 commits into from
Sep 14, 2019

Conversation

edoakes
Copy link
Contributor

@edoakes edoakes commented Aug 28, 2019

Why are these changes needed?

Part of the migration of the Python worker to use the C++ core worker. This builds on top of #5272, submitting normal tasks through the core worker.

Related issue number

Linter

  • I've run scripts/format.sh to lint the changes in this PR.

@edoakes
Python core worker put
d16c9f0
@edoakes
Remove unused imports
a936886
@edoakes
Remove unused buffer methods
065b8ce
@edoakes
get() working
e11fcfa
@edoakes
bugfixes
7cc3d9d
@edoakes
reorder pyarrow and _raylet imports
afc9cc4
@edoakes
remove non-core-worker code
caea63a
@edoakes
Remove debug prints
f22ce76
@edoakes
don't dedup in core worker
d6f34f7
@edoakes
handle duplicates and implement wait() and free()
e0d96fd
@edoakes
const reference to metadata
3c45f76
@edoakes
re-enable TaskExecutor in constructor
cc20396
@edoakes
Merge branch 'master' into python-core-worker-put
8e95f8a
@edoakes
Cleanup buffer.pxi
7716bfa
@edoakes
clean up _raylet.pyx
766d43c
@edoakes
more cleanup
1656f0e
@edoakes
Refactor plasma store provider Get()
a5e0be7
@edoakes
Specify nullptr return in comment
6ea29c0
@edoakes
Remove unused methods
6bf23be
@edoakes
Merge branch 'master' into python-core-worker-put
b53c05a
@edoakes
get_objects -> get_object
a3974e1
@edoakes
address comments
8caa78e
@edoakes
Merge branch 'master' into python-core-worker-put
6579512
@edoakes
Add gcs client init
ac6231a
@edoakes
create & seal comments
b68e496
@edoakes
Comments and pointer
bd72754
@edoakes
FetchOrReconstruct in batches
0e48a31
@edoakes
fix java build
fbaf8c6
@edoakes
import error message
b245e2e
@edoakes
some formatting
8c7188f
@edoakes edoakes marked this pull request as ready for review September 12, 2019 19:06
@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/16997/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/17002/
Test PASSed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/17011/
Test FAILed.

@edoakes edoakes changed the title [WIP] [core worker] Python core worker normal task submission [core worker] Python core worker normal task submission Sep 13, 2019
pcmoritz
pcmoritz reviewed Sep 14, 2019
View reviewed changes
python/ray/_raylet.pyx Outdated Show resolved Hide resolved
pcmoritz
pcmoritz reviewed Sep 14, 2019
View reviewed changes
python/ray/_raylet.pyx Outdated Show resolved Hide resolved
pcmoritz
pcmoritz approved these changes Sep 14, 2019
View reviewed changes
Copy link
Contributor

@pcmoritz pcmoritz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please address the comments :)

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/17024/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/17022/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/17025/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/17032/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/17041/
Test PASSed.

@ericl
Copy link
Contributor

ericl commented Sep 14, 2019

Lint seems to be from @simon-mo ?

@ericl ericl merged commit a5d7de6 into ray-project:master Sep 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@pcmoritz pcmoritz pcmoritz approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@edoakes @AmplabJenkins @ericl @pcmoritz