Some tweaks and improvements

[luiscib3r]

1. Updated the cupertino_icons package in the text_classification example (it did not allow build) [9930d47]
2. Use ffi for binding to iOS dependencies. Use the ios/tflite_flutter.podspec file to specify tensorflow lite library version. Dependencies are automatically downloaded without user intervention (no need for releases/download folder) [909fcd8]
3. Use ffi for binding with Android dependencies. Use the android/build.gradle file to specify tensorflow lite library version. Dependencies are automatically downloaded without user intervention (no need for releases/download folder) [3ef861e]
4. Use ffigen to generate the binding code in dart. [0291c5c]
5. Enable delegates in text_classification example [6ff4ef4].
6. Add conversion for tensors of type uint8 [eef0c19]
7. Add image classification example using mobilenet [3f764de]
8. Add super resolution example using esrgan [5ab2ca9]
9. Add style transfer example [227ce2b]
10. IsolateInterpreter to run inference in an isolate. Related to Use isolates to run inference #52 [972decf]

[PaulTR]

Nice! So we're on the same page for timelines, I'll run through the apps at some point this week. Catching up on some stuff that I pushed aside for I/O last week :)

[PaulTR]

So I'm going to show my ignorance a bit since this is actually one of the only times I've ever needed iOS - is there anything special you had to do to get the ios samples working on your machine? I went and added my personal icloud id, then ran flutter build ios and flutter install ios, but it doesn't look like any of the inference is working (using an ipad 16.4.1 I think? I just updated). Here's what I'm seeing:

https://photos.app.goo.gl/SByfzE1wtqVpm6s38

[PaulTR]

Just ran everything on Android though and damn that's cool. Good job!

https://photos.app.goo.gl/wkztJD4QMtzKNaEL8

[luiscib3r]

I will carry out some tests with a physical device. thanks for checking it out

[luiscib3r]

@PaulTR. I made some adjustments. The problem was that when compiling in release mode the TensorFlowLiteC/Metal symbols were not being linked. I changed the dependencies to use TensorFlowLiteSwift instead of the C version and it was resolved.

[denny61302]

Thank you @luiscib3r for sharing this critical improvement.

I faced an issues when running IsolateInterpreter with GpuDelegateV2 enabled on my Google Pixel 6.

This is the message I got
"TfLiteGpuDelegate Invoke: GpuDelegate must run on the same thread where it was initialized. "
and the inference won't work.

Everything works fine when I use
XNNPackDelegate with IsolateInterpreter or
GpuDelegate without IsolateInterpreter

Are these normal behaviours?

Cheers,
Denny

[luiscib3r]

@denny61302 Thanks for your review. I'm dealing with the same problem, the solution would be to build the interpreter and delegate in a separate isolate. But I'm still thinking of a way to integrate it with the package. I don't want to modify the Interpreter base class. All ideas are welcome.

[PaulTR]

Yeah the GPU thread thing doesn't surprise me - it does require that the interpreter setup happen on the same thread where inference happens. It's something that I'll need to look into for fixing in this plugin since I had to do some obnoxious stuff in Android samples to make delegate switching work.

One quick request: Can you look over the CI build for the needed fixes in the samples?

Thanks!

[luiscib3r]

@PaulTR In the CI workflow, when you do dart pub get it downloads the package dependencies but not the example dependencies. For this reason, when passing the analyzer, errors of the type Target of URI doesn't exist and those that follow are shown. One solution would be to exclude the example folder in the analysis_options.yaml but then it becomes difficult to work with an IDE as it doesn't catch errors for you. The other variant can be to use a tool like melos, with the melos bootstrap command you can download the main package dependencies and examples in one go and then like this run flutter analyzer. If it's okay with you I can include melos in the PR and modify the CI workflow.

[PaulTR]

Ah got it. I've never used melos, but given that it is in FlutterFire, that seems like a fair solution. If you're OK adding that, I think it'd be a great addition :) Thanks!

[luiscib3r]

@PaulTR CI workflow is done. I did some tests on the fork: https://github.com/luiscib3r/flutter-tflite/actions. It's ready.

[PaulTR]

So I am running into this issue when trying to build via

flutter build ios

Error (Xcode): ../../lib/src/bindings/tensorflow_lite_bindings_generated.dart:2060:7: Error: The type 'TfLiteModel' must be 'base', 'final' or 'sealed' because the supertype 'Opaque' is 'base'.

[PaulTR]

Alright so hopefully this helps :) I don't know how FFI binding works yet, but I just added 'base' before all of the 'class' objects at the bottom of the generated file and that does it, so it just needs to be done from wherever those classes are created.

I'm guessing there just needs to be modifiers in front of the objects in the dart files under lib/src?

Edit: I pinged someone on the Flutter team to figure this out, too, because I want to know how to run the ffi script (currently getting a message about flutter_test not existing, but that's something I'll sort out on my end). If I get that working, I can merge this and do a separate PR for the new files.

[luiscib3r]

@PaulTR maybe you need to do flutter clean and run flutter pub get again, the problem is that we returned the sdk version to support dart 2, so you may still have linked the latest version of ffi which only supports dart 3

Using the latest version of ffigen and ffi class modifiers are added automatically but it is only supported in dart 3, we would remove support for everything still in dart 2

[PaulTR]

Fair enough - ran all of that and still seeing the issue, but will talk with Craig Labenz in the morning :)

ptruiz-macbookpro:flutter-tflite ptruiz$ flutter clean
Deleting .dart_tool...                                               2ms
ptruiz-macbookpro:flutter-tflite ptruiz$ dart --version
Dart SDK version: 2.19.5 (stable) (Mon Mar 20 17:09:37 2023 +0000) on "macos_arm64"
ptruiz-macbookpro:flutter-tflite ptruiz$ flutter pub get
Resolving dependencies...
! cli_util 0.4.0 (overridden)
  collection 1.17.1 (1.17.2 available)
  file 6.1.4 (7.0.0 available)
  matcher 0.12.15 (0.12.16 available)
  material_color_utilities 0.2.0 (0.5.0 available)
  pub_updater 0.2.4 (0.3.0 available)
  source_span 1.9.1 (1.10.0 available)
  test 1.24.1 (1.24.3 available)
  test_api 0.5.1 (0.6.0 available)
  test_core 0.5.1 (0.5.3 available)
Got dependencies!

ptruiz-macbookpro:flutter-tflite ptruiz$ dart run ffi
Resolving dependencies in /Users/ptruiz/Documents/code/flutter/flutter-tflite...
Because tflite_flutter depends on flutter_test from sdk which doesn't exist (the Flutter SDK is not available), version solving failed.

Flutter users should run `flutter pub get` instead of `dart pub get`.

[luiscib3r]

@PaulTR Good morning. I remembered that I had also increased the SDK version in the samples and forgot to update them when I made the change in the package. Maybe that's why it didn't allow you to compile. I made the adjustment, try again and let me know if it was resolved.

[PaulTR]

Sweet, that did the trick. Thanks for putting this together! It's definitely a lot of work and something that'll be great for everyone, and I really appreciate all of your effort here.

I'm going to merge this and publish.

Also please send up another PR to add your name to the contributor file.

[PaulTR]

Looks like there's one more issue with publishing this, but I'll dig into it:

Package validation found the following hint:
* Non-dev dependencies are overridden in pubspec.yaml.

  This indicates you are not testing your package against the same versions of its
  dependencies that users will have when they use it.

  This might be necessary for packages with cyclic dependencies.

  Please be extra careful when publishing.
Sorry, your package is missing a requirement and can't be published yet.
For more information, see: https://dart.dev/tools/pub/cmd/pub-lish.

Looks like it has to do with the dependency override for melos (cli_util). Working through it :) Looks like invertase runs that, so I'll reach out to them (might be a day or two before I have a response - that team is based out of London and its afternoon for them)

[PaulTR]

Alright should be fixed in #56