Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Task launch failed: failed to create workflow in propeller namespaces “flytetester-development” not found #1173

Closed
mayitbeegh opened this issue Jun 23, 2021 · 2 comments
Assignees
Labels
bug Something isn't working
Milestone

Comments

@mayitbeegh
Copy link
Contributor

mayitbeegh commented Jun 23, 2021

Describe the bug
Launching an example flytekit-java task registered by jflyte results in the following error: Request failed with status code 500. In the browser dev tools, the response payload says:
Ufailed to create workflow in propeller namespaces “flytetester-development” not found
Ufailed to create workflow in propeller namespaces “flytetester-development” not found`
Potentially caused by syncresources cron job not running.

Expected behavior
The task executes successfully.

[Optional] Additional context
To Reproduce
Steps to reproduce the behavior:

  1. [flytekit-java] Follow the example guide in the [README]($ mvn package
    $ scripts/jflyte register workflows
    -d=development
    -p=flytetester
    -v=$(git describe --always)
    -cp=flytekit-examples/target/lib
    ) to register example tasks and workflows with local sandbox.
  2. [flyte-snack]Find flytetester/development/org.flyte.examples.HelloWorldTask in the console and launch the task.
  3. The error message shows up in the same modal.

To Remedy
Follow the instruction in #162 and run kubectl create job --from=cronjob/syncresources <job name> -n flyte

Screenshots
If applicable, add screenshots to help explain your problem.

Screen Shot 2021-06-23 at 09 58 57

@mayitbeegh mayitbeegh added bug Something isn't working untriaged This issues has not yet been looked at by the Maintainers labels Jun 23, 2021
@welcome
Copy link

welcome bot commented Jun 23, 2021

Thank you for opening your first issue here! 🛠

@kumare3 kumare3 changed the title [BUG] Task launch failed: Ufailed to create workflow in propeller namespaces “flytetester-development” not found [BUG] Task launch failed: failed to create workflow in propeller namespaces “flytetester-development” not found Jul 2, 2021
@kumare3 kumare3 removed the untriaged This issues has not yet been looked at by the Maintainers label Jul 2, 2021
@kumare3 kumare3 added this to the 0.16.0 milestone Jul 2, 2021
@EngHabu EngHabu modified the milestones: 0.16.0, 0.17.0 Aug 2, 2021
@EngHabu EngHabu modified the milestones: 0.17.0, 0.18.0 Aug 31, 2021
@pmahindrakar-oss pmahindrakar-oss self-assigned this Sep 13, 2021
@pmahindrakar-oss
Copy link
Contributor

I ran the registration using the latest flyte components and flytekit-java and i wasn't able to reporduce this issue.

scripts/jflyte register workflows \
  -d=development \
  -p=flytetester \
  -v=$(git describe --always) \
  -cp=flytekit-examples/target/lib
[main] DEBUG org.flyte.jflyte.ClassLoaders - Loading jars from [/jflyte/modules/jflyte-aws]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/ion-java-1.0.2.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/jackson-core-2.6.7.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/aws-java-sdk-core-1.11.327.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/commons-codec-1.10.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/jackson-dataformat-cbor-2.6.7.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/aws-java-sdk-s3-1.11.327.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/jackson-annotations-2.6.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/commons-logging-1.2.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/jmespath-java-1.11.327.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/httpcore-4.4.9.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/jflyte-aws-0.3.1-SNAPSHOT.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/jackson-databind-2.6.7.1.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/joda-time-2.8.1.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/httpclient-4.5.5.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-aws/aws-java-sdk-kms-1.11.327.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Loading jars from [/jflyte/modules/jflyte-google-cloud]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/gax-1.51.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-http-client-jackson2-1.33.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-cloud-core-1.91.3.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/proto-google-iam-v1-0.13.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/api-common-1.8.1.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-oauth-client-1.30.3.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/javax.annotation-api-1.3.2.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/jflyte-google-cloud-0.3.1-SNAPSHOT.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/checker-compat-qual-2.5.5.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/jackson-core-2.10.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/opencensus-contrib-http-util-0.24.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-http-client-1.33.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/error_prone_annotations-2.3.4.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/protobuf-java-util-3.11.1.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/opencensus-api-0.24.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/httpcore-4.4.12.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/protobuf-java-3.11.4.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-auth-library-oauth2-http-0.18.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/proto-google-common-protos-1.17.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/gson-2.8.6.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-auth-library-credentials-0.18.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/animal-sniffer-annotations-1.18.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-api-services-storage-v1-rev20190910-1.30.3.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/auto-value-annotations-1.7.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/commons-logging-1.2.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/commons-codec-1.11.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/j2objc-annotations-1.3.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-cloud-storage-1.101.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-http-client-appengine-1.33.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/failureaccess-1.0.1.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/threetenbp-1.3.3.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/gax-httpjson-0.68.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/grpc-context-1.25.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-cloud-core-http-1.91.3.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/google-api-client-1.30.4.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/httpclient-4.5.10.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/jflyte/modules/jflyte-google-cloud/guava-28.1-android.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Loading jars from [/workdir/flytekit-examples/target/lib]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/workdir/flytekit-examples/target/lib/flytekit-examples-0.3.1-SNAPSHOT.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/workdir/flytekit-examples/target/lib/flytekit-jackson-0.3.1-SNAPSHOT.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/workdir/flytekit-examples/target/lib/flytekit-java-0.3.1-SNAPSHOT.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/workdir/flytekit-examples/target/lib/jackson-annotations-2.11.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/workdir/flytekit-examples/target/lib/jackson-core-2.11.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/workdir/flytekit-examples/target/lib/jackson-databind-2.11.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/workdir/flytekit-examples/target/lib/jackson-datatype-jsr310-2.11.0.jar]
[main] DEBUG org.flyte.jflyte.ClassLoaders - Discovered [file:/workdir/flytekit-examples/target/lib/jackson-module-parameter-names-2.11.0.jar]
[main] DEBUG org.flyte.jflyte.Registrars - Discovering RunnableTaskRegistrar
[main] DEBUG org.flyte.jflyte.Registrars - Discovered [org.flyte.flytekit.SdkRunnableTaskRegistrar]
[main] DEBUG org.flyte.flytekit.SdkRunnableTaskRegistrar - Discovering SdkRunnableTask
[main] DEBUG org.flyte.flytekit.SdkRunnableTaskRegistrar - Discovered [org.flyte.examples.AddQuestionTask]
[main] DEBUG org.flyte.flytekit.SdkRunnableTaskRegistrar - Discovered [org.flyte.examples.BatchLookUpTask]
[main] DEBUG org.flyte.flytekit.SdkRunnableTaskRegistrar - Discovered [org.flyte.examples.GreetTask]
[main] DEBUG org.flyte.flytekit.SdkRunnableTaskRegistrar - Discovered [org.flyte.examples.HelloWorldTask]
[main] DEBUG org.flyte.flytekit.SdkRunnableTaskRegistrar - Discovered [org.flyte.examples.PrintMessageTask]
[main] DEBUG org.flyte.flytekit.SdkRunnableTaskRegistrar - Discovered [org.flyte.examples.SumTask]
[main] DEBUG org.flyte.jflyte.Registrars - Discovering WorkflowTemplateRegistrar
[main] DEBUG org.flyte.jflyte.Registrars - Discovered [org.flyte.flytekit.SdkWorkflowTemplateRegistrar]
[main] DEBUG org.flyte.flytekit.SdkWorkflowTemplateRegistrar - Discovering SdkWorkflow
[main] DEBUG org.flyte.flytekit.SdkWorkflowTemplateRegistrar - Discovered [org.flyte.examples.FibonacciWorkflow]
[main] DEBUG org.flyte.flytekit.SdkWorkflowTemplateRegistrar - Discovered [org.flyte.examples.PhoneBookWorkflow]
[main] DEBUG org.flyte.flytekit.SdkWorkflowTemplateRegistrar - Discovered [org.flyte.examples.WelcomeWorkflow]
[main] DEBUG org.flyte.jflyte.Registrars - Discovering LaunchPlanRegistrar
[main] DEBUG org.flyte.jflyte.Registrars - Discovered [org.flyte.flytekit.SdkLaunchPlanRegistrar]
[main] DEBUG org.flyte.flytekit.SdkLaunchPlanRegistrar - Discovering SdkLaunchPlans
[main] DEBUG org.flyte.flytekit.SdkLaunchPlanRegistrar - Discovered [org.flyte.examples.FibonacciWorkflow]
[main] DEBUG org.flyte.flytekit.SdkLaunchPlanRegistrar - Discovered [org.flyte.examples.PhoneBookWorkflow]
[main] DEBUG org.flyte.flytekit.SdkLaunchPlanRegistrar - Discovered [org.flyte.examples.WelcomeWorkflow]
[main] DEBUG org.flyte.flytekit.SdkLaunchPlanRegistrar - Discovered [FibonacciWorkflowLaunchPlan]
[main] DEBUG org.flyte.flytekit.SdkLaunchPlanRegistrar - Discovered [FibonacciWorkflowLaunchPlan2]
[main] DEBUG org.flyte.flytekit.SdkLaunchPlanRegistrar - Discovered [FibonacciWorkflowLaunchPlan3]
[main] DEBUG org.flyte.jflyte.FileSystemLoader - Discovering FileSystemRegistrar
[main] DEBUG org.flyte.jflyte.FileSystemLoader - Discovered FileSystem [org.flyte.jflyte.gcp.GcsFileSystem]
[main] DEBUG org.flyte.jflyte.FileSystemLoader - Discovering FileSystemRegistrar
[main] DEBUG org.flyte.jflyte.aws.S3FileSystem - Using FLYTE_AWS_ACCESS_KEY_ID [minio]
[main] DEBUG org.flyte.jflyte.aws.S3FileSystem - Using FLYTE_AWS_ENDPOINT [http://localhost:30084]
[main] DEBUG org.flyte.jflyte.FileSystemLoader - Discovered FileSystem [org.flyte.jflyte.aws.S3FileSystem]
[main] DEBUG org.flyte.jflyte.ArtifactStager - Staging [flytekit-examples-0.3.1-SNAPSHOT.jar] to [s3://my-s3-bucket/flytekit-examples-0.3.1-SNAPSHOT-z7e0Ezb60Xq9FScjMDQEdA.jar]
[main] DEBUG org.flyte.jflyte.ArtifactStager - Staging [flytekit-jackson-0.3.1-SNAPSHOT.jar] to [s3://my-s3-bucket/flytekit-jackson-0.3.1-SNAPSHOT-yn8FdQBMdcYfptFAYMSr-w.jar]
[main] DEBUG org.flyte.jflyte.ArtifactStager - Staging [flytekit-java-0.3.1-SNAPSHOT.jar] to [s3://my-s3-bucket/flytekit-java-0.3.1-SNAPSHOT-KCCQ2LJIGsKBqieFOqOmVA.jar]
[main] DEBUG org.flyte.jflyte.ArtifactStager - Staging [jackson-annotations-2.11.0.jar] to [s3://my-s3-bucket/jackson-annotations-2.11.0-qJMo0V6eEZfF2IeKhxUTOg.jar]
[main] DEBUG org.flyte.jflyte.ArtifactStager - Staging [jackson-core-2.11.0.jar] to [s3://my-s3-bucket/jackson-core-2.11.0-8DipMXEncZXS0jr3wo7aRA.jar]
[main] DEBUG org.flyte.jflyte.ArtifactStager - Staging [jackson-databind-2.11.0.jar] to [s3://my-s3-bucket/jackson-databind-2.11.0-_aybv0Uf2tFPeU5uo8emIw.jar]
[main] DEBUG org.flyte.jflyte.ArtifactStager - Staging [jackson-datatype-jsr310-2.11.0.jar] to [s3://my-s3-bucket/jackson-datatype-jsr310-2.11.0-fcoXquwxBcEImGYT90SUAw.jar]
[main] DEBUG org.flyte.jflyte.ArtifactStager - Staging [jackson-module-parameter-names-2.11.0.jar] to [s3://my-s3-bucket/jackson-module-parameter-names-2.11.0-OZFD_mvBkwP421k_G5Muuw.jar]
[main] DEBUG org.flyte.jflyte.ArtifactStager - Staging [classpath] to [s3://my-s3-bucket/classpath-RNuE1zhqUb3LeYHVrANHGw.]
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createTask TaskIdentifier{name=org.flyte.examples.BatchLookUpTask, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createTask TaskIdentifier{name=org.flyte.examples.GreetTask, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createTask TaskIdentifier{name=org.flyte.examples.HelloWorldTask, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createTask TaskIdentifier{name=org.flyte.examples.AddQuestionTask, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createTask TaskIdentifier{name=org.flyte.examples.PrintMessageTask, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createTask TaskIdentifier{name=org.flyte.examples.SumTask, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createWorkflow WorkflowIdentifier{name=org.flyte.examples.FibonacciWorkflow, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createWorkflow WorkflowIdentifier{name=org.flyte.examples.PhoneBookWorkflow, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createWorkflow WorkflowIdentifier{name=org.flyte.examples.WelcomeWorkflow, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createLaunchPlan LaunchPlanIdentifier{name=FibonacciWorkflowLaunchPlan3, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createLaunchPlan LaunchPlanIdentifier{name=org.flyte.examples.FibonacciWorkflow, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createLaunchPlan LaunchPlanIdentifier{name=FibonacciWorkflowLaunchPlan2, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createLaunchPlan LaunchPlanIdentifier{name=org.flyte.examples.PhoneBookWorkflow, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createLaunchPlan LaunchPlanIdentifier{name=org.flyte.examples.WelcomeWorkflow, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}
[main] DEBUG org.flyte.jflyte.FlyteAdminClient - createLaunchPlan LaunchPlanIdentifier{name=FibonacciWorkflowLaunchPlan, domain=development, project=flytetester, version=0.3.0-1-g36ffde3}

Screenshot 2021-09-14 at 4 45 39 PM

Screenshot 2021-09-14 at 4 44 09 PM

I am closing this issue for now and we can reopen it if we encounter this again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants