Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI is broken #137

Closed
astrojuanlu opened this issue Jun 26, 2023 · 3 comments
Closed

CI is broken #137

astrojuanlu opened this issue Jun 26, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@astrojuanlu
Copy link
Member

astrojuanlu commented Jun 26, 2023

#136 made a one character change in the README but there were some test failures.

https://app.circleci.com/pipelines/github/kedro-org/kedro-starters/618/workflows/687e92b1-d6d7-41e1-9348-dc8566bb9c39/jobs/6650

Failing scenarios:
  features/run.feature:26  Run a Kedro project created from pyspark-iris

0 features passed, 1 failed, 0 skipped
4 scenarios passed, 1 failed, 0 skipped
24 steps passed, 1 failed, 0 skipped, 0 undefined
Took 3m56.385s

Exited with code exit status 1

CircleCI received exit code 1

This is the error:

DatasetError: An exception occurred when parsing config for dataset 
'example_classifier':
Dataset type 'kedro.io.memory_dataset.MemoryDataSet' is invalid: all data set 
types must extend 'AbstractDataSet'.
@astrojuanlu
Copy link
Member Author

Root cause:

# This is an example how to use `MemoryDataSet` with Spark objects that aren't `DataFrame`'s.
# In particular, the `assign` copy mode ensures that the `MemoryDataSet` will be assigned
# the Spark object itself, not a deepcopy version of it, since deepcopy doesn't work with
# Spark object generally.
example_classifier:
type: MemoryDataSet
copy_mode: assign

and

https://github.com/kedro-org/kedro/blob/fd8162d5bf384ef666c01ef2c529d01fd9fa8354/kedro/io/core.py#L427-L431

@astrojuanlu
Copy link
Member Author

In progress at kedro-org/kedro#2724

@astrojuanlu
Copy link
Member Author

Test: gh-138

astrojuanlu pushed a commit to kedro-org/kedro that referenced this issue Jun 27, 2023
astrojuanlu pushed a commit to kedro-org/kedro that referenced this issue Jun 27, 2023
astrojuanlu pushed a commit to kedro-org/kedro that referenced this issue Jun 28, 2023
astrojuanlu pushed a commit to kedro-org/kedro that referenced this issue Jun 28, 2023
debugger24 pushed a commit to debugger24/kedro that referenced this issue Jun 28, 2023
* Complete build requirements

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Add timeout for e2e process waiting

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Remove DeprecatedClassMeta in favor of getattr

Fix kedro-org/kedro-starters#137

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Ignore deprecation warnings in our own code

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

---------

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Deepyaman Datta <[email protected]>
Signed-off-by: debugger24 <[email protected]>
astrojuanlu added a commit to kedro-org/kedro that referenced this issue Jun 28, 2023
* FIX: Typo in cli help

Signed-off-by: debugger24 <[email protected]>

* Make GitPod works (#2688)

* update base image

Signed-off-by: Nok <[email protected]>

* update gitpod image and apt-get

Signed-off-by: Nok <[email protected]>

* update docker

Signed-off-by: Nok <[email protected]>
---------

Signed-off-by: Nok <[email protected]>
Signed-off-by: debugger24 <[email protected]>

* Remove `DeprecatedClassMeta` in favor of `getattr` (#2724)

* Complete build requirements

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Add timeout for e2e process waiting

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Remove DeprecatedClassMeta in favor of getattr

Fix kedro-org/kedro-starters#137

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

* Ignore deprecation warnings in our own code

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>

---------

Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Deepyaman Datta <[email protected]>
Signed-off-by: debugger24 <[email protected]>

---------

Signed-off-by: debugger24 <[email protected]>
Signed-off-by: Nok <[email protected]>
Signed-off-by: Juan Luis Cano Rodríguez <[email protected]>
Co-authored-by: Nok Lam Chan 陳諾林 <[email protected]>
Co-authored-by: Deepyaman Datta <[email protected]>
Co-authored-by: Juan Luis Cano Rodríguez <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Archived in project
Development

No branches or pull requests

1 participant