Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flink improvements rocksdb state pr #406

Merged
merged 5 commits into from
Jul 26, 2023
Merged

Flink improvements rocksdb state pr #406

merged 5 commits into from
Jul 26, 2023

Commits on Jul 26, 2023

  1. Updated setup-local-ingress script to avoid destroying the coredns co… 

    …nfigs when there where access right problems earlier

Signed-off-by: marcel <[email protected]>
    @wagmarcel
    wagmarcel committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    1540d77 View commit details
    Browse the repository at this point in the history

  2. Update flink helm configmap and deployments to allow high availabilit… 

    …y and RocksDB based state backend

Signed-off-by: marcel <[email protected]>
    @wagmarcel
    wagmarcel committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    b43acff View commit details
    Browse the repository at this point in the history

  3. Add sqlsettings to beamsqlstatementsets CR 

    Statements do not only contain SQL INSERTs, TABLES and VIEWS, they can also contain
SETTINGS, like ttl, pipeline.name, etc. In order to make it more applicable to
the Operator and SQL-Gateway, the sqlsettings must be provided as list of key/value pairs.

Signed-off-by: marcel <[email protected]>
    @wagmarcel
    wagmarcel committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    afd58f2 View commit details
    Browse the repository at this point in the history

  4. Split KafkaTopic CR creation from other CRs and allow makefile to rem… 

    …ovee kafkatopics before deploying flink

Strimzi is great. But one annoying behaviour is that it creates KafkaTopic CR for every topic it finds
on Kafka. This is in principle great, because KafkaTopic CRDs allow configuring all details of topics, like
replication. However, when an application creates a Kafka Topic, Strimzi creates a KafkaTopic CR which conflicts
with the CRs from the helm deployments. In order to be able to delete the KafkaTopics CR which are relevant, the
KafkaTopic CRs are created in separate files to be able to explicitly delete them before deploying the shacl job.

Closes IndustryFusion#402

Signed-off-by: marcel <[email protected]>
    @wagmarcel
    wagmarcel committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    3457cce View commit details
    Browse the repository at this point in the history

  5. Remove unneccesary test file creation by create_udfs.py 

    Signed-off-by: marcel <[email protected]>
    @wagmarcel
    wagmarcel committed Jul 26, 2023
    Configuration menu
    Copy the full SHA
    b2c3bb4 View commit details
    Browse the repository at this point in the history