Skip to content

Commit

Permalink
RDART-771 EJSON support (#1295)
Browse files Browse the repository at this point in the history
* Worthy of first commit

* Add actual generator

* Tweak pubspec.yaml for bette PANA score

* move ejson project into ejson folder

* Rudimentary generator inplace

* Custom decoding tests

* Fix realm generator tests

* Update pubspec repo ref

* wip

* Add copyright headers

* Fix multi parameter ctor bug. More tests

* Add bad input test, and cheat a bit for dynamic tests

* A few compile tests in place

* Split in multiple project and move to melos

* Basic source error reporting

* add qa script to melos.yaml (combines format:check, analyze, and test)

* testCompile supports skip

* Enable melos qa on github actions

* Use utc in test

* Add LICENSE file + repo link in pubspecs

* More melos magic

* Make ejson package public

* Add internal dep

* Use >- over | to work around windows line ending issue

* Force \n in DartFormatter due to windows line ending issue

* Drop qa:static scripts (qa:full calls qa)

* Fix bug in testCompile

* format and check coverage

* Use ejson in realm

* Lint rules (WIP)

* Report lints as errors instead of infos

* Use links for LICENSE file

* Link READMEs

* Expand melos:qa to include test:lints and analyze:deps

* Fix tests after rebase

* Support ObjectId and Uuid out-of-the-box

* Update analyzer dependency for ejson

* Update lint related deps

* Fix lints and tests

* Run builder runner

* Fix realm generator tests

* Flatten package structure

* Avoid path deps in public packages and remove publish_to:

* Add melos support

* Add lints package on bootstrap

Ensure lints is added to all packages during bootstrap, if missing

* Simplify CI a bit with melos

* Update root .gitignore

* build_native.dart (wip)

* Split bootstrap pre-hook into separate setup script (for speed when setup not needed)

* Align SDK requirement (handled by melos bootstrap)

* melos bootstrap needed

This is because some packages are not published yet

* fixup: bad rebase

* Reorder steps

* missing deps in example

* Cleanup .gitignore hierarchy

* Align analysis_options.yaml files with symlinks

* realm/example is a flutter project

* Remove last remnants of toplevel ejson folder

* Tweak melos.yaml

* TMP: workaround

* Coverage (wip)

* Use combine_coverage package instead of lcov

* Only report on lib folder

* Prune coverage a bit

* Don't run tests twice

* Skip redundant step

* Update checkout action to v4

* Update upload-artifact action to v4

* Update download-artifact action to v4

* Update dorny/test-reporter  action to v1.8.0

* Skip redundant step

* Update actions/setup-java action to v4

* Update gradle/gradle-build-action action to v3

* Don't use random github action to install ninja-build. It is not maintained

* Update futureware-tech/simulator-action action to v3

* Update geekyeggo/delete-artifact action to v4

* Fix symlink blunder

* Tighten analysis rules

* add upgrade script to melos.yaml

* bump custom_lint_builder

* tweak publish-release.yml

* Update format and lint:format to not touch generated files (workaround for dart-lang/dart_style#864)

* Implicit casts made explicit, as mandated by stricter analysis_options.yaml

* melos run format

* tweak melos.yaml

* Fix lints and increase coverage

* switch to melos coverage:groom

* strong-mode has been superceeded by strict

* enable custom_lint tools

* Add example to package ejson

* Fix missing quotes on int and double values in canonical mode

* Relax Array type from LIst to Iterable

* Update a ejson_lint/example README.md

* Fix missing quotes on DateTime in canonical mode

* Fix ejson_generator tests

* Only install ninja on linux builds

* Tweak a test

* Force a new native build..

* Tweak condition for Ninja (android- & linux-)

* Fix linux.. and simplify

* Upgrade realm-core to v14.0.1

* testing hypothesis

* Rework install command

* Drop ejson_serialization_setup.g.dart (for now)

* Support Uint8List

* Get rid of to<T> extension (use fromJson<T> instead)

* Update ejson codecs

* Fix "generator" tests

* Fix realm_value_test.dart

* Convert coverage, despite test failure

* More encoding tests

* Fix blunder regarding canonical vs. relaxed

* Don't forget windows *sigh*

* Split test and coverage handling

* Small formatting error

* Use super.<x> syntax

* Drop some imports

* Don't use _ prefix on already local stuff

* DateTime codec use local time in relaxed mode, but millisecondSinceEpoch (utc) in canonical mode

* export ejson.dart from realm.dart to avoid extra import in user code

* export ejson_annotation.dart from ejson.dart to avoid extra import in user code

* Update copyright

* Remove redundant "library;"

* Remove the now unnecesary imports

* Remove deprecated exports

* Add unsynced to CSpell

* Don't use ejson_generator for realm objects, but have realm_generator create the codec pair

* Update .expected files

* Drop dep on ejson_annotation and ejson_generator

* Rerun generator (.g.dart files no longer generated)

* format

* Absolute symlinks should be relative

* Upgrade simulator to iPhone SE (3rd generation)

* Update example with to/From-EJson

* Update CHANGELOG

* Dart doc updates

* Missing headers

* Missing headers .. unrelated to ejson

* Implement PR feedback (part 1)

* Refactor int.toEJson to take an optional forcedFormat argument, and otherwise infer format from size of value

* Add toEJsonString and fromEJsonString<T> convinience functions

* Missing tests for Uint8List aka binary with subtype '00'

* Fix Uint8List decoding bug

* Generate registerX function

* Refactor realm_generator EJSON support a bit

* fix realm example

* Update CONTRIBUTING.md

* No need to build realm_dart for maccatalyst by default, as flutter doesn't currentlys

* More work on build.dart

* Update build:native script to use tool/build.dart

* A bit of gold plating..

* .. and some bugfixes

* Important to drain both stdout and stderr
  • Loading branch information
nielsenko authored Mar 6, 2024
1 parent d02dda2 commit 26e7b30
Show file tree
Hide file tree
Showing 219 changed files with 6,113 additions and 2,468 deletions.
15 changes: 8 additions & 7 deletions .github/workflows/binary-combine-android.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,38 +10,39 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Fetch x86 build
uses: actions/download-artifact@9782bd6a9848b53b110e712e20e42d89988822b7
uses: actions/download-artifact@v4
with:
name: librealm-android-x86
path: packages/realm_dart/binary/android
- name: Fetch x86_64 build
uses: actions/download-artifact@9782bd6a9848b53b110e712e20e42d89988822b7
uses: actions/download-artifact@v4
with:
name: librealm-android-x86_64
path: packages/realm_dart/binary/android
- name: Fetch armeabi-v7a build
uses: actions/download-artifact@9782bd6a9848b53b110e712e20e42d89988822b7
uses: actions/download-artifact@v4
with:
name: librealm-android-armeabi-v7a
path: packages/realm_dart/binary/android
- name: Fetch arm64-v8a build
uses: actions/download-artifact@9782bd6a9848b53b110e712e20e42d89988822b7
uses: actions/download-artifact@v4
with:
name: librealm-android-arm64-v8a
path: packages/realm_dart/binary/android

- name: Store combined artifact
uses: actions/upload-artifact@83fd05a356d7e2593de66fc9913b3002723633cb
uses: actions/upload-artifact@v4
with:
name: librealm-android
path: packages/realm_dart/binary/android
retention-days: 1

- name: Delete individual build artifacts
uses: geekyeggo/delete-artifact@54ab544f12cdb7b71613a16a2b5a37a9ade990af
uses: geekyeggo/delete-artifact@v4
with:
name: |
librealm-android-x86
librealm-android-x86_64
librealm-android-armeabi-v7a
librealm-android-arm64-v8a
librealm-android-arm64-v8a
failOnError: false
11 changes: 6 additions & 5 deletions .github/workflows/binary-combine-ios.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,17 +11,17 @@ jobs:

steps:
- name: Fetch device build
uses: actions/download-artifact@9782bd6a9848b53b110e712e20e42d89988822b7
uses: actions/download-artifact@v4
with:
name: librealm-ios-device
path: binary/ios
- name: Fetch simulator build
uses: actions/download-artifact@9782bd6a9848b53b110e712e20e42d89988822b7
uses: actions/download-artifact@v4
with:
name: librealm-ios-simulator
path: binary/ios
- name: Fetch catalyst build
uses: actions/download-artifact@9782bd6a9848b53b110e712e20e42d89988822b7
uses: actions/download-artifact@v4
with:
name: librealm-ios-catalyst
path: binary/ios
Expand All @@ -36,16 +36,17 @@ jobs:
rm -rf ./binary/ios/Release-*
- name: Store .xcframework artifact
uses: actions/upload-artifact@83fd05a356d7e2593de66fc9913b3002723633cb
uses: actions/upload-artifact@v4
with:
name: librealm-ios
path: binary/ios
retention-days: 1

- name: Delete individual framework artifacts
uses: geekyeggo/delete-artifact@54ab544f12cdb7b71613a16a2b5a37a9ade990af
uses: geekyeggo/delete-artifact@v4
with:
name: |
librealm-ios-device
librealm-ios-simulator
librealm-ios-catalyst
failOnError: false
15 changes: 8 additions & 7 deletions .github/workflows/build-native.yml
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
build: ${{ fromJSON(inputs.build) }}
steps:
- name: Checkout
uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
uses: actions/checkout@v4
with:
submodules: 'recursive'

Expand All @@ -41,17 +41,18 @@ jobs:
path: ./packages/realm_dart/binary/**
key: binaries-${{ matrix.build }}-${{ inputs.runner }}-${{hashFiles('./packages/realm_dart/src/**')}}


- name: Setup Ninja
if: steps.check-cache.outputs.cache-hit != 'true'
uses: seanmiddleditch/gha-setup-ninja@1815f2d05c2cd60c2d900f89843139b8dde09f4c
if: steps.check-cache.outputs.cache-hit != 'true' && (startsWith(matrix.build, 'android') || startsWith(matrix.build, 'linux'))
run: |
sudo apt-get update -y
sudo apt-get install -y ninja-build
- name: Setup Android NDK
if: startsWith(matrix.build, 'android-')
if: startsWith(matrix.build, 'android')
run: echo "ANDROID_NDK=$ANDROID_NDK_LATEST_HOME" >> $GITHUB_ENV

- name: Downgrade XCode for MacOS
if: ${{ matrix.build == 'macos' }}
if: matrix.build == 'macos'
run: sudo xcode-select -s /Applications/Xcode_14.0.1.app

- name: Build
Expand All @@ -61,7 +62,7 @@ jobs:
cmake --build --preset ${{ matrix.build }} --config Release ${{ startsWith(matrix.build, 'android-') && '--target strip' || '' }}
- name: Store artifacts
uses: actions/upload-artifact@83fd05a356d7e2593de66fc9913b3002723633cb
uses: actions/upload-artifact@v4
with:
name: librealm-${{ matrix.build }}
path: packages/realm_dart/binary/${{ inputs.binary }}/**
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/check-changelog.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@8230315d06ad95c617244d2f265d237a1682d445
uses: actions/checkout@v4
with:
submodules: false
- name: Enforce Changelog
Expand Down
77 changes: 30 additions & 47 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -294,15 +294,15 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
uses: actions/checkout@v4
with:
submodules: 'recursive'

- name: Enable ccache
run: echo "PATH=/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH" >> $GITHUB_ENV

- name: Fetch artifacts
uses: actions/download-artifact@9782bd6a9848b53b110e712e20e42d89988822b7
uses: actions/download-artifact@v4
with:
name: librealm-ios
path: packages/realm_dart/binary/ios
Expand All @@ -312,13 +312,18 @@ jobs:
with:
channel: 'stable'

- name: Setup Melos
run: |
dart pub global activate melos
dart pub global run melos bootstrap
- name: Install dependencies
run: dart pub get

- name: Launch Simulator
uses: futureware-tech/simulator-action@v2
uses: futureware-tech/simulator-action@v3
with:
model: 'iPhone 8'
model: 'iPhone SE (3rd generation)'
os: 'iOS'
os_version: '>= 14.0'

Expand All @@ -331,7 +336,7 @@ jobs:
--suppress-analytics
- name: Publish Test Report
uses: dorny/test-reporter@v1.7.0
uses: dorny/test-reporter@v1.8.0
if: success() || failure()
with:
name: Test Results Flutter iOS
Expand Down Expand Up @@ -373,7 +378,7 @@ jobs:

steps:
- name: Checkout
uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
uses: actions/checkout@v4
with:
submodules: 'recursive'

Expand All @@ -384,16 +389,16 @@ jobs:
sudo udevadm trigger --name-match=kvm
- name: Gradle cache
uses: gradle/gradle-build-action@v2
uses: gradle/gradle-build-action@v3

- name: Set up Java
uses: actions/setup-java@v3
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: 11

- name: Fetch artifacts
uses: actions/download-artifact@9782bd6a9848b53b110e712e20e42d89988822b7
uses: actions/download-artifact@v4
with:
name: librealm-android
path: packages/realm_dart/binary/android
Expand All @@ -403,6 +408,11 @@ jobs:
with:
channel: 'stable'

- name: Setup Melos
run: |
dart pub global activate melos
dart pub global run melos bootstrap
- name: Install dependencies
run: dart pub get

Expand Down Expand Up @@ -447,7 +457,7 @@ jobs:
script: cd packages/realm/tests && flutter test integration_test/all_tests.dart --dart-define=BAAS_BAASAAS_API_KEY=$BAAS_BAASAAS_API_KEY --dart-define=BAAS_DIFFERENTIATOR=$BAAS_DIFFERENTIATOR --file-reporter=json:test-results.json --suppress-analytics

- name: Publish Test Report
uses: dorny/test-reporter@v1.7.0
uses: dorny/test-reporter@v1.8.0
if: success() || failure()
with:
name: Test Results Flutter Android
Expand Down Expand Up @@ -481,10 +491,11 @@ jobs:

runs-on: ${{ matrix.os }}-latest
name: Generator Tests

timeout-minutes: 30
steps:
- name: Checkout
uses: actions/checkout@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
uses: actions/checkout@v4
with:
submodules: 'recursive'

Expand All @@ -493,8 +504,15 @@ jobs:
with:
channel: 'stable'

- name: Setup Melos
run: |
dart pub global activate melos
dart pub global run melos bootstrap
- name: Delete generated files in realm_dart
run: find . -name "*.realm.dart" -delete
run: |
find . -name "*.g.dart" -delete
find . -name "*.realm.dart" -delete
working-directory: packages/realm_dart

- name: Run generator in realm_dart
Expand Down Expand Up @@ -533,53 +551,18 @@ jobs:
exit 1
fi
- name: Run generator tests
run: |
dart pub get
dart test -r expanded --coverage ./coverage/ --test-randomize-ordering-seed random
- name: Generate generator coverage report
if: matrix.os == 'ubuntu'
run: |
dart run coverage:format_coverage \
--in coverage/ \
--out ./coverage/lcov.info \
--check-ignore \
--lcov \
--packages .dart_tool/package_config.json \
--report-on lib
- name: Publish Generator Coverage
if: matrix.os == 'ubuntu'
id: publish-coverage
uses: coverallsapp/github-action@f350da2c033043742f89e8c0b7b5145a1616da6d
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
flag-name: generator
path-to-lcov: packages/realm_generator/coverage/lcov.info
parallel: true

- name: Output Coveralls response
if: matrix.os == 'ubuntu'
run: echo ${{ steps.publish-coverage.outputs.coveralls-api-result }}

coverage-finished:
needs:
- generator
- dart-tests-linux
runs-on: ubuntu-latest
steps:

- name: Coveralls Finished
id: publish-coverage
uses: coverallsapp/github-action@f350da2c033043742f89e8c0b7b5145a1616da6d
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
parallel-finished: true

- name: Output Coveralls response
run: echo ${{ steps.publish-coverage.outputs.coveralls-api-result }}

slack-on-failure:
name: Report failure in main branch
needs:
Expand Down
Loading

0 comments on commit 26e7b30

Please sign in to comment.