Question 385895 in Flutter Question posted in
The official Flutter documentation can be found here.

Flutter – Is there a way to move AI image processing to separate thread?

Need help improving the performance of this feature. Is there a way to move AI processing logic to separate thread because the application feels junky once I startImageStream.

class SessionRecordingScreen extends StatefulWidget {
  const SessionRecordingScreen({super.key});

  @override
  State<SessionRecordingScreen> createState() => _SessionRecordingScreenState();
}

class _SessionRecordingScreenState extends State<SessionRecordingScreen> {
  ObjectDetector? _objectDetector;
  bool _canProcess = false;
  bool _isBusy = false;
  bool _isInitialized = false;
  final _model = 'assets/ml/yolov8_small.tflite';

  int catches = 0;
  int droppedBalls = 0;

  void _initializeDetector() async {
    _objectDetector?.close();
    _objectDetector = null;
    final modelPath = await getAssetPath(_model);
    final options = LocalObjectDetectorOptions(
      mode: DetectionMode.stream,
      modelPath: modelPath,
      classifyObjects: true,
      multipleObjects: true,
    );
    _objectDetector = ObjectDetector(options: options);
    _canProcess = true;
  }

  @override
  Widget build(BuildContext context) {
    return PopScope(
      canPop: false,
      child: Material(
        child: CameraViewWidget(
          onImage: _processImage,
          onCameraFeedReady: () {
            if (!_isInitialized) {
              _initializeDetector();
              _isInitialized = true;
            }
          },
          catches: catches,
          droppedBalls: droppedBalls,
        ),
      ),
    );
  }

  Future<void> _processImage(InputImage inputImage) async {
    if (!_canProcess || _isBusy) return;
    _isBusy = true;

    final List<Object> results =
        await _objectDetector!.processImage(inputImage);

    if (results.isEmpty) {
      _isBusy = false;
      return;
    }
    _isBusy = false;
  }
}

CameraViewWidget is just a CameraWidget with a parser to InputImage as I need it for google_ml_kit.

I tried using compute and isolates but was unsuccessful. Thank you for your help

Tags:

2

Answers


  1. 0

    You can adapt your implementation to remove these async/awaits to adopt a concurrency approach to your solution.

    In flutter/dart if you want to use a concurrency solution, you will need to understand how to work with the Isolate class. In this LinkedIn post (not mine) there’s an example how to work with it and I think that this is the perfect solution to your case.

    And in this link there’s the official documentation about Concurrency on Flutter/Dart.

    Login or Signup to reply.
  2. 0
    1. Use tflite_flutter instead of flutter_tflite which is updated.
    2. Use isolateInterpreter with tflite_flutter. Or you can move preprocessing and detection to a new isolate. For example, you can refer to this example in tflite_flutter: link
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search