Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds mediapipe_core package #11

Merged
merged 17 commits into from
Nov 13, 2023
Merged

Adds mediapipe_core package #11

merged 17 commits into from
Nov 13, 2023

Conversation

craiglabenz
Copy link
Collaborator

@craiglabenz craiglabenz commented Oct 13, 2023

Description

Adds the mediapipe_core package.

mediapipe_core is designed to be a shared dependency of all other MediaPipe packages (mediapipe_text, mediapipe_vision, and mediapipe_audio). It contains:

  • Shared header files from google/mediapipe and their Dart complements
    • These include Category, TaskOptions, BaseOptions, and ClassificationResult.
  • Shared ffigen output which will be imported by the other packages
  • Common FFI utilities

TODOs:

  • Update README
  • Improve documentation

@@ -0,0 +1,39 @@
<!--
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note to self: Come back and review this once filled out

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

'tis filled out 🎉

packages/mediapipe-core/analysis_options.yaml Outdated Show resolved Hide resolved
packages/mediapipe-core/analysis_options.yaml Outdated Show resolved Hide resolved
packages/mediapipe-core/analysis_options.yaml Show resolved Hide resolved
packages/mediapipe-core/lib/src/task_options.dart Outdated Show resolved Hide resolved
packages/mediapipe-core/lib/src/task_options.dart Outdated Show resolved Hide resolved
packages/mediapipe-core/lib/src/task_options.dart Outdated Show resolved Hide resolved
packages/mediapipe-core/pubspec.yaml Outdated Show resolved Hide resolved
Makefile Show resolved Hide resolved
Makefile Show resolved Hide resolved
this.categoryDenylist,
});

/// The locale to use for display names specified through the TFLite Model
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is TFLite Model Metadata? Should this have a link leading to some info?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I copied this info from some MediaPipe documentation somewhere. You're right that that link would be useful - presumably on the class itself.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PaulTR Are there any other useful resources / links we could include here?

packages/mediapipe-core/lib/src/task_options.dart Outdated Show resolved Hide resolved
packages/mediapipe-core/test/task_options_test.dart Outdated Show resolved Hide resolved
Copy link
Member

@Piinks Piinks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're just waiting for confirmation on license bits before we can land this, otherwise just nits left below!

Makefile Show resolved Hide resolved
packages/mediapipe-core/lib/src/containers.dart Outdated Show resolved Hide resolved
packages/mediapipe-core/lib/src/containers.dart Outdated Show resolved Hide resolved
/// Converts a list of Dart strings into their C memory equivalent.
///
/// See also:
/// * [prepareString]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

///
/// See also:
/// * [prepareListOfStrings]
Pointer<Char> prepareString(String val) => val.toNativeUtf8().cast<Char>();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need both methods to be public since prepareListOfStrings calls this? Maybe prepareString -> _prepareString?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both functions are called from elsewhere - the list format method just makes use of the single-string variant in its loop.

///
/// See also:
/// * [toDartStrings]
String? toDartString(Pointer<Char> val) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here maybe? toDartString --> _toDartString

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above - this function is also necessarily public in the PRs for other tasks.

/// are ignored. Mutually exclusive with `categoryDenylist`.
///
/// See also:
/// * [Category.categoryName]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, why would they want to see this too? We usually add a brief blurb of what or why the recommendation is listed

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's frivolous, I'm not sure. It's THE field whose values will appear in this list; but that'd probably be worth clarifying.

packages/mediapipe-core/lib/src/task_options.dart Outdated Show resolved Hide resolved
@@ -0,0 +1,19 @@
name: mediapipe_core
description: Shared logic and utilities required by other MediaPipe Task packages.
version: 0.0.1-wip
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a publish_to: none for now? Or nah?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linter votes yes, but I haven't personally been worried about accidentally typing pub publish 😂

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right. We automate publishing in flutter/packages

packages/mediapipe-core/test/task_options_test.dart Outdated Show resolved Hide resolved
craiglabenz and others added 7 commits October 23, 2023 14:59
* adds CI/CD for mediapipe_core

* add newlines

* moves file into workflows dir

* uncomment flutter doctor

* testing PR config to run this now

* added master and beta CI scripts

* add executable permissions to CI scripts
Copy link
Collaborator Author

@craiglabenz craiglabenz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is ready for final eyes, nits, and maybe an LGTM-and-merge, @Piinks and @PaulTR.

Copy link
Member

@Piinks Piinks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple of to do's throughout still, but this is the first of many changes. I assume we will return to the remaining ones as the other changes get in. Soooooo...

Flutter_LGTM

@craiglabenz craiglabenz merged commit 10990cf into ffi-wrapper Nov 13, 2023
4 checks passed
@craiglabenz craiglabenz deleted the ffi-wrapper-core-pkg branch November 13, 2023 20:05
craiglabenz added a commit that referenced this pull request Apr 16, 2024
* Adds mediapipe_core package (#11)

* adds mediapipe_core package

* adds makefile for all packages

* fixes typo in Makefile

* Apply suggestions from code review

* Update Makefile

* Apply suggestions from code review

* updated generated code's location and license (for 3P status)

* code review responses

including:
  * comments
  * licensing
  * resolving testing nits

* updated README

* setup sharing of base analysis_options

* Update packages/mediapipe-core/lib/src/containers.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-core/lib/src/containers.dart

Co-authored-by: Kate Lovett <[email protected]>

* add free methods to core structs

* code review updates to options

* adds publish blocker

* Add GitHub actions for CI/CD (#14)

* adds CI/CD for mediapipe_core

* add newlines

* moves file into workflows dir

* uncomment flutter doctor

* testing PR config to run this now

* added master and beta CI scripts

* add executable permissions to CI scripts

* adds ffiwrapper to ci/cd

---------

Co-authored-by: Kate Lovett <[email protected]>

* Add utility to collect headers from google/mediapipe (#10)

* adds cmd to pull header files from google/mediapipe

* polish and missing parts from git surgery

* More comments and touch ups

* Apply suggestions from code review

* moves build command into `tool/` directory

and renames folder `build_cmd` -> `builder`

* complete build_cmd -> builder rename

* Update readme

* Added licenses

* Adds DownloadModelCommand

---------

Co-authored-by: Kate Lovett <[email protected]>

* [FFI] MediaPipe SDKs finder automation (#16)

* adds sdks_finder command to builder utility

* propagates changes to existing commands

* adds manifest files generated by new sdks_finder command

* updates in response to code review

* Adds mediapipe_text package (#12)

* adds mediapipe_text package

* Update .vscode/settings.json

added newline

* resync headers

* regenerated core bindings

* Apply suggestions from code review

* Adds example to `mediapipe-task-text` (#15)

* initial commit of example

* build file changes from `flutter pub get`

* update main.dart

* removes commented code

* updates example for isolates design

* Use `native-assets` to vendor MediaPipe SDK (#9)

* adding bare structure for native assets

* add MVP / first draft of build.dart

* build.dart updates

* update build.dart

TODO: stream file

* model memory troubleshooting

* vendoring script tweak

* remove development logging

* removes pointless build method

* Add utility to collect headers from google/mediapipe (#10)

* adds cmd to pull header files from google/mediapipe

* polish and missing parts from git surgery

* More comments and touch ups

* Apply suggestions from code review

* moves build command into `tool/` directory

and renames folder `build_cmd` -> `builder`

* complete build_cmd -> builder rename

* Update readme

* Added licenses

* Adds DownloadModelCommand

---------

Co-authored-by: Kate Lovett <[email protected]>

* adds mediapipe_text package

* Update .vscode/settings.json

added newline

* resync headers

* regenerated core bindings

* native assets troubleshooting

this commit is broken

* Removes redundant count field

* update build.dart for correct bindings path

* download text classification model for CI

* better memory freeing in executor

* added SafeArea to example

* added CI to PRs into text package

* ci tooling change

* remove accidentally commited model

* more CI shenanigans

* lowers minimum Dart version for builder

* added smoke test for text example

* Added CI/CD for examples

* More CI tweaks

* entering "please work" territory

* d'oh

* trying more random stuff

* one more time

* it'd be funny if this helped

* more print statements

* enable reaching new print statements

* more logging

* see what's in build dir

* another test

* adding flutter config list

* turn off fail-fast for beta and master

* moar logs

* way moar prints

* moare things

* commit rest of rename

* moar whatevers

* adds manifest files generated by new sdks_finder command

* adds sdks_finder command to builder utility

* propagates changes to existing commands

* updates in response to code review

* updates to build.dart and tests

* add Android runtime

* sdks_finder logging improvement for when build folders change names

* refreshed symbols from google/mediapipe

* cleanup

* loosens closeness thresholds in integration tests

* separate build commands for macos architectures

* restores fail-fast setting to CI

* removed stale logging statements from CI

* removes accidentally committed lines

* add formatting of sdk_downloads.dart for CI

* fixes broken example test

* code touch ups from @Piinks code review

---------

Co-authored-by: Kate Lovett <[email protected]>

* added base Dart class ClassificationResult to consolidate results logic in `mediapipe_core`

in doing so, removed meaningless TextClassifierResult.timestamp field

* Update Makefile

* Update Makefile

* Update packages/mediapipe-task-text/build.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/lib/src/tasks/text_classification/text_classification_executor.dart

Co-authored-by: Kate Lovett <[email protected]>

* code review responses

* formatting

* removes stale comment

* adds Dart to Native converters, with tests

* changes from code review

* updates mediapipe-core to prepare for IO/web split

* Improves memory management in core tests

* updated ffigen / bindings

* refactors text package for better memory management and eventual web/io split

* removed stale test

* cleanup on aisle COMMENTS

* Comments and documentation improvements

* Moved log statement

* Removed native memory management helpers in favor of `free` extensions on pointers

* Renamed abstract classes to have Base prefix

* sorted out class constructors

* moved `fake` constructor to default unnamed constructor
* leaning on the fact that the native constructors will be hidden by conditional exports, reducing confusion

* Improved docstrings explaining memory ownership

* Convert lists to lazy iterable / generators

* added missing licenses

* Update packages/mediapipe-task-text/example/test/widgets_test.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/example/lib/main.dart

Co-authored-by: Kate Lovett <[email protected]>

* completed return style change

* CI troubleshooting

* moved around debugging code

* logging tweak

* cat native-assets.yaml

* moar logs

* removed bad echo

* fixed native-assets typo

should be underscore!

* Removes CI debugging statements

---------

Co-authored-by: Kate Lovett <[email protected]>

* Native Assets CI fix (#20)

* adds native assets debugging statements

* Try only downloading target arch

* Revert "adds native assets debugging statements"

This reverts commit b2bc215.

---------

Co-authored-by: Daco Harkes <[email protected]>

* Text Embedding task (#21)

* updated and re-ran generators

* added embedding concepts to mediapipe-core

* fixed embedding header file and bindings

* adds text embedding classes to text pkg

* updates example with text embedding

* removed dead file

* added more embedding tests

* added embedding model download to CI script

* touch ups

* Update packages/mediapipe-core/lib/src/io/containers.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/example/.gitignore

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/example/lib/text_embedding_demo.dart

Co-authored-by: Kate Lovett <[email protected]>

* moved worker dispose method to base class

* docstring & comment improvements

* throw exceptions in impossible code paths instead of returning null

* class hierarchy improvements

* fixed outdates tests

* cleaned up dispose methods

* various tidying

* fixed deprecation warning

* moves repeated widgets into helper method

---------

Co-authored-by: Kate Lovett <[email protected]>

---------

Co-authored-by: Craig Labenz <[email protected]>
Co-authored-by: Kate Lovett <[email protected]>
Co-authored-by: Daco Harkes <[email protected]>
craiglabenz added a commit that referenced this pull request May 12, 2024
* Adds mediapipe_core package (#11)

* adds mediapipe_core package

* adds makefile for all packages

* fixes typo in Makefile

* Apply suggestions from code review

* Update Makefile

* Apply suggestions from code review

* updated generated code's location and license (for 3P status)

* code review responses

including:
  * comments
  * licensing
  * resolving testing nits

* updated README

* setup sharing of base analysis_options

* Update packages/mediapipe-core/lib/src/containers.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-core/lib/src/containers.dart

Co-authored-by: Kate Lovett <[email protected]>

* add free methods to core structs

* code review updates to options

* adds publish blocker

* Add GitHub actions for CI/CD (#14)

* adds CI/CD for mediapipe_core

* add newlines

* moves file into workflows dir

* uncomment flutter doctor

* testing PR config to run this now

* added master and beta CI scripts

* add executable permissions to CI scripts

* adds ffiwrapper to ci/cd

---------

Co-authored-by: Kate Lovett <[email protected]>

* Add utility to collect headers from google/mediapipe (#10)

* adds cmd to pull header files from google/mediapipe

* polish and missing parts from git surgery

* More comments and touch ups

* Apply suggestions from code review

* moves build command into `tool/` directory

and renames folder `build_cmd` -> `builder`

* complete build_cmd -> builder rename

* Update readme

* Added licenses

* Adds DownloadModelCommand

---------

Co-authored-by: Kate Lovett <[email protected]>

* [FFI] MediaPipe SDKs finder automation (#16)

* adds sdks_finder command to builder utility

* propagates changes to existing commands

* adds manifest files generated by new sdks_finder command

* updates in response to code review

* Adds mediapipe_text package (#12)

* adds mediapipe_text package

* Update .vscode/settings.json

added newline

* resync headers

* regenerated core bindings

* Apply suggestions from code review

* Adds example to `mediapipe-task-text` (#15)

* initial commit of example

* build file changes from `flutter pub get`

* update main.dart

* removes commented code

* updates example for isolates design

* Use `native-assets` to vendor MediaPipe SDK (#9)

* adding bare structure for native assets

* add MVP / first draft of build.dart

* build.dart updates

* update build.dart

TODO: stream file

* model memory troubleshooting

* vendoring script tweak

* remove development logging

* removes pointless build method

* Add utility to collect headers from google/mediapipe (#10)

* adds cmd to pull header files from google/mediapipe

* polish and missing parts from git surgery

* More comments and touch ups

* Apply suggestions from code review

* moves build command into `tool/` directory

and renames folder `build_cmd` -> `builder`

* complete build_cmd -> builder rename

* Update readme

* Added licenses

* Adds DownloadModelCommand

---------

Co-authored-by: Kate Lovett <[email protected]>

* adds mediapipe_text package

* Update .vscode/settings.json

added newline

* resync headers

* regenerated core bindings

* native assets troubleshooting

this commit is broken

* Removes redundant count field

* update build.dart for correct bindings path

* download text classification model for CI

* better memory freeing in executor

* added SafeArea to example

* added CI to PRs into text package

* ci tooling change

* remove accidentally commited model

* more CI shenanigans

* lowers minimum Dart version for builder

* added smoke test for text example

* Added CI/CD for examples

* More CI tweaks

* entering "please work" territory

* d'oh

* trying more random stuff

* one more time

* it'd be funny if this helped

* more print statements

* enable reaching new print statements

* more logging

* see what's in build dir

* another test

* adding flutter config list

* turn off fail-fast for beta and master

* moar logs

* way moar prints

* moare things

* commit rest of rename

* moar whatevers

* adds manifest files generated by new sdks_finder command

* adds sdks_finder command to builder utility

* propagates changes to existing commands

* updates in response to code review

* updates to build.dart and tests

* add Android runtime

* sdks_finder logging improvement for when build folders change names

* refreshed symbols from google/mediapipe

* cleanup

* loosens closeness thresholds in integration tests

* separate build commands for macos architectures

* restores fail-fast setting to CI

* removed stale logging statements from CI

* removes accidentally committed lines

* add formatting of sdk_downloads.dart for CI

* fixes broken example test

* code touch ups from @Piinks code review

---------

Co-authored-by: Kate Lovett <[email protected]>

* added base Dart class ClassificationResult to consolidate results logic in `mediapipe_core`

in doing so, removed meaningless TextClassifierResult.timestamp field

* Update Makefile

* Update Makefile

* Update packages/mediapipe-task-text/build.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/lib/src/tasks/text_classification/text_classification_executor.dart

Co-authored-by: Kate Lovett <[email protected]>

* code review responses

* formatting

* removes stale comment

* adds Dart to Native converters, with tests

* changes from code review

* updates mediapipe-core to prepare for IO/web split

* Improves memory management in core tests

* updated ffigen / bindings

* refactors text package for better memory management and eventual web/io split

* removed stale test

* cleanup on aisle COMMENTS

* Comments and documentation improvements

* Moved log statement

* Removed native memory management helpers in favor of `free` extensions on pointers

* Renamed abstract classes to have Base prefix

* sorted out class constructors

* moved `fake` constructor to default unnamed constructor
* leaning on the fact that the native constructors will be hidden by conditional exports, reducing confusion

* Improved docstrings explaining memory ownership

* Convert lists to lazy iterable / generators

* added missing licenses

* Update packages/mediapipe-task-text/example/test/widgets_test.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/example/lib/main.dart

Co-authored-by: Kate Lovett <[email protected]>

* completed return style change

* CI troubleshooting

* moved around debugging code

* logging tweak

* cat native-assets.yaml

* moar logs

* removed bad echo

* fixed native-assets typo

should be underscore!

* Removes CI debugging statements

---------

Co-authored-by: Kate Lovett <[email protected]>

* Native Assets CI fix (#20)

* adds native assets debugging statements

* Try only downloading target arch

* Revert "adds native assets debugging statements"

This reverts commit b2bc215.

---------

Co-authored-by: Daco Harkes <[email protected]>

* Text Embedding task (#21)

* updated and re-ran generators

* added embedding concepts to mediapipe-core

* fixed embedding header file and bindings

* adds text embedding classes to text pkg

* updates example with text embedding

* removed dead file

* added more embedding tests

* added embedding model download to CI script

* touch ups

* Update packages/mediapipe-core/lib/src/io/containers.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/example/.gitignore

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/example/lib/text_embedding_demo.dart

Co-authored-by: Kate Lovett <[email protected]>

* moved worker dispose method to base class

* docstring & comment improvements

* throw exceptions in impossible code paths instead of returning null

* class hierarchy improvements

* fixed outdates tests

* cleaned up dispose methods

* various tidying

* fixed deprecation warning

* moves repeated widgets into helper method

---------

Co-authored-by: Kate Lovett <[email protected]>

* updated and re-ran generators

* fixed embedding header file and bindings

* adds text embedding classes to text pkg

* moved worker dispose method to base class

* class hierarchy improvements

* cleaned up dispose methods

* initial commit of language detection task

* finishes language detection impl

* adds language detection demo

* backfilling improvements to classification and embedding

* adds language detection tests

* add new model download to CI script

* fixes stale classification widget test, adds language detection widget test

* copied latest headers from mediapipe

* Update packages/mediapipe-task-text/lib/src/io/tasks/language_detection/language_detector_executor.dart

Co-authored-by: Kate Lovett <[email protected]>

* comments / documentation improvements

---------

Co-authored-by: Kate Lovett <[email protected]>
Co-authored-by: Daco Harkes <[email protected]>
craiglabenz added a commit that referenced this pull request May 12, 2024
* Adds mediapipe_core package (#11)

* adds mediapipe_core package

* adds makefile for all packages

* fixes typo in Makefile

* Apply suggestions from code review

* Update Makefile

* Apply suggestions from code review

* updated generated code's location and license (for 3P status)

* code review responses

including:
  * comments
  * licensing
  * resolving testing nits

* updated README

* setup sharing of base analysis_options

* Update packages/mediapipe-core/lib/src/containers.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-core/lib/src/containers.dart

Co-authored-by: Kate Lovett <[email protected]>

* add free methods to core structs

* code review updates to options

* adds publish blocker

* Add GitHub actions for CI/CD (#14)

* adds CI/CD for mediapipe_core

* add newlines

* moves file into workflows dir

* uncomment flutter doctor

* testing PR config to run this now

* added master and beta CI scripts

* add executable permissions to CI scripts

* adds ffiwrapper to ci/cd

---------

Co-authored-by: Kate Lovett <[email protected]>

* Add utility to collect headers from google/mediapipe (#10)

* adds cmd to pull header files from google/mediapipe

* polish and missing parts from git surgery

* More comments and touch ups

* Apply suggestions from code review

* moves build command into `tool/` directory

and renames folder `build_cmd` -> `builder`

* complete build_cmd -> builder rename

* Update readme

* Added licenses

* Adds DownloadModelCommand

---------

Co-authored-by: Kate Lovett <[email protected]>

* [FFI] MediaPipe SDKs finder automation (#16)

* adds sdks_finder command to builder utility

* propagates changes to existing commands

* adds manifest files generated by new sdks_finder command

* updates in response to code review

* Adds mediapipe_text package (#12)

* adds mediapipe_text package

* Update .vscode/settings.json

added newline

* resync headers

* regenerated core bindings

* Apply suggestions from code review

* Adds example to `mediapipe-task-text` (#15)

* initial commit of example

* build file changes from `flutter pub get`

* update main.dart

* removes commented code

* updates example for isolates design

* Use `native-assets` to vendor MediaPipe SDK (#9)

* adding bare structure for native assets

* add MVP / first draft of build.dart

* build.dart updates

* update build.dart

TODO: stream file

* model memory troubleshooting

* vendoring script tweak

* remove development logging

* removes pointless build method

* Add utility to collect headers from google/mediapipe (#10)

* adds cmd to pull header files from google/mediapipe

* polish and missing parts from git surgery

* More comments and touch ups

* Apply suggestions from code review

* moves build command into `tool/` directory

and renames folder `build_cmd` -> `builder`

* complete build_cmd -> builder rename

* Update readme

* Added licenses

* Adds DownloadModelCommand

---------

Co-authored-by: Kate Lovett <[email protected]>

* adds mediapipe_text package

* Update .vscode/settings.json

added newline

* resync headers

* regenerated core bindings

* native assets troubleshooting

this commit is broken

* Removes redundant count field

* update build.dart for correct bindings path

* download text classification model for CI

* better memory freeing in executor

* added SafeArea to example

* added CI to PRs into text package

* ci tooling change

* remove accidentally commited model

* more CI shenanigans

* lowers minimum Dart version for builder

* added smoke test for text example

* Added CI/CD for examples

* More CI tweaks

* entering "please work" territory

* d'oh

* trying more random stuff

* one more time

* it'd be funny if this helped

* more print statements

* enable reaching new print statements

* more logging

* see what's in build dir

* another test

* adding flutter config list

* turn off fail-fast for beta and master

* moar logs

* way moar prints

* moare things

* commit rest of rename

* moar whatevers

* adds manifest files generated by new sdks_finder command

* adds sdks_finder command to builder utility

* propagates changes to existing commands

* updates in response to code review

* updates to build.dart and tests

* add Android runtime

* sdks_finder logging improvement for when build folders change names

* refreshed symbols from google/mediapipe

* cleanup

* loosens closeness thresholds in integration tests

* separate build commands for macos architectures

* restores fail-fast setting to CI

* removed stale logging statements from CI

* removes accidentally committed lines

* add formatting of sdk_downloads.dart for CI

* fixes broken example test

* code touch ups from @Piinks code review

---------

Co-authored-by: Kate Lovett <[email protected]>

* added base Dart class ClassificationResult to consolidate results logic in `mediapipe_core`

in doing so, removed meaningless TextClassifierResult.timestamp field

* Update Makefile

* Update Makefile

* Update packages/mediapipe-task-text/build.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/lib/src/tasks/text_classification/text_classification_executor.dart

Co-authored-by: Kate Lovett <[email protected]>

* code review responses

* formatting

* removes stale comment

* adds Dart to Native converters, with tests

* changes from code review

* updates mediapipe-core to prepare for IO/web split

* Improves memory management in core tests

* updated ffigen / bindings

* refactors text package for better memory management and eventual web/io split

* removed stale test

* cleanup on aisle COMMENTS

* Comments and documentation improvements

* Moved log statement

* Removed native memory management helpers in favor of `free` extensions on pointers

* Renamed abstract classes to have Base prefix

* sorted out class constructors

* moved `fake` constructor to default unnamed constructor
* leaning on the fact that the native constructors will be hidden by conditional exports, reducing confusion

* Improved docstrings explaining memory ownership

* Convert lists to lazy iterable / generators

* added missing licenses

* Update packages/mediapipe-task-text/example/test/widgets_test.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/example/lib/main.dart

Co-authored-by: Kate Lovett <[email protected]>

* completed return style change

* CI troubleshooting

* moved around debugging code

* logging tweak

* cat native-assets.yaml

* moar logs

* removed bad echo

* fixed native-assets typo

should be underscore!

* Removes CI debugging statements

---------

Co-authored-by: Kate Lovett <[email protected]>

* Native Assets CI fix (#20)

* adds native assets debugging statements

* Try only downloading target arch

* Revert "adds native assets debugging statements"

This reverts commit b2bc215.

---------

Co-authored-by: Daco Harkes <[email protected]>

* Text Embedding task (#21)

* updated and re-ran generators

* added embedding concepts to mediapipe-core

* fixed embedding header file and bindings

* adds text embedding classes to text pkg

* updates example with text embedding

* removed dead file

* added more embedding tests

* added embedding model download to CI script

* touch ups

* Update packages/mediapipe-core/lib/src/io/containers.dart

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/example/.gitignore

Co-authored-by: Kate Lovett <[email protected]>

* Update packages/mediapipe-task-text/example/lib/text_embedding_demo.dart

Co-authored-by: Kate Lovett <[email protected]>

* moved worker dispose method to base class

* docstring & comment improvements

* throw exceptions in impossible code paths instead of returning null

* class hierarchy improvements

* fixed outdates tests

* cleaned up dispose methods

* various tidying

* fixed deprecation warning

* moves repeated widgets into helper method

---------

Co-authored-by: Kate Lovett <[email protected]>

* updated and re-ran generators

* fixed embedding header file and bindings

* adds text embedding classes to text pkg

* moved worker dispose method to base class

* class hierarchy improvements

* cleaned up dispose methods

* initial commit of language detection task

* finishes language detection impl

* adds language detection demo

* backfilling improvements to classification and embedding

* adds language detection tests

* add new model download to CI script

* fixes stale classification widget test, adds language detection widget test

* initial inference commit of flutter create -t package

* rename inference folder

* adds build tooling for inference headers and ffigen

* rename "inference" to "genai"

* adds initial inference impl

* flutter creat example for genai

* sdks crawling update

* adds genai
* adds model name to output file
* looks backwards through time until a build is good instead of betting the house that the most recent build was good

* adds android project to genai example

* adds ios project to genai example

* updates to macos project in genai example

* inference impl improvements

* inference example - nearly working

* various cleanup

* inference demo updates

* model path debugging

* inference demo improvements

* improvements to inference demo

* genai inference example improvements

* updates to inference and example

* fixed issue with chat screens sometimes not refreshing while the LLM was responding

problem came from state.transcript changes not busting the hashCode cache

* updated SDK manifests

* removed unused ffi utils copy param

---------

Co-authored-by: Kate Lovett <[email protected]>
Co-authored-by: Daco Harkes <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants