How to convert flutter CameraImage to png or jpg with proper colors and rotation?

KristiJorgji
March 17, 2023
222 views
0 votes
2 Answers

I am using the camera flutter lib and startImageStream method of the controller
Every X ms I get one CameraImage input and want to save this in file as jpg/png

I am using some code found online but it has 2 issues

It is grayscaled and without any colors
It is rotated 90 degree anti-clockwise

The code that I am currently using is

import 'package:camera/camera.dart';
import 'package:image/image.dart' as imglib;

Future<List<int>> convertImageToPng(CameraImage image) async {
  imglib.Image? img;
  if (image.format.group == ImageFormatGroup.yuv420) {
    img = _convertYUV420(image);
  } else if (image.format.group == ImageFormatGroup.bgra8888) {
    img = _convertBGRA8888(image);
  }

  imglib.PngEncoder pngEncoder = imglib.PngEncoder();

  // Convert to png
  List<int> png = pngEncoder.encodeImage(img!);
  return png;
}

// CameraImage BGRA8888 -> PNG
// Color
imglib.Image _convertBGRA8888(CameraImage image) {
  return imglib.Image.fromBytes(
    image.width,
    image.height,
    image.planes[0].bytes,
    format: imglib.Format.bgra,
  );
}

// CameraImage YUV420_888 -> PNG -> Image (compression:0, filter: none)
// Black
imglib.Image _convertYUV420(CameraImage image) {
  var img = imglib.Image(image.width, image.height); // Create Image buffer

  Plane plane = image.planes[0];
  const int shift = (0xFF << 24);

  // Fill image buffer with plane[0] from YUV420_888
  for (int x = 0; x < image.width; x++) {
    for (int planeOffset = 0;
        planeOffset < image.height * image.width;
        planeOffset += image.width) {
      final pixelColor = plane.bytes[planeOffset + x];
      // color: 0x FF  FF  FF  FF
      //           A   B   G   R
      // Calculate pixel color
      var newVal = shift | (pixelColor << 16) | (pixelColor << 8) | pixelColor;

      img.data[planeOffset + x] = newVal;
    }
  }

  return img;
}

method convertImageToPng

I need a cross-platform ios+android method to convert CameraImage to jpg/png with proper colors and without rotating.

    _controller?.startImageStream(
      (image) {
        final asPng = await convertImageToPng(image);
        // todo then I save this on disk or uploa
      },
    );

Any help will be appreciated.

Tags: flutter

Answers

Chosen as BEST ANSWER

After working for like one week I managed to got a function that converts Yuv420 CameraImage to png with colors, BUT the image has bad rotation, is inverted upside down (I think -180 degrees)

My improved COLORFUL function is. I still need to understand how to rotate properly the image

imglib.Image _convertYUV420toImageColor(CameraImage image) {
  const shift = (0xFF << 24);

  final int width = image.width;
  final int height = image.height;
  final int uvRowStride = image.planes[1].bytesPerRow;
  final int uvPixelStride = image.planes[1].bytesPerPixel!;

  final img = imglib.Image(height, width); // Create Image buffer

  // Fill image buffer with plane[0] from YUV420_888
  for (int x = 0; x < width; x++) {
    for (int y = 0; y < height; y++) {
      final int uvIndex =
          uvPixelStride * (x / 2).floor() + uvRowStride * (y / 2).floor();
      final int index = y * width + x;

      final yp = image.planes[0].bytes[index];
      final up = image.planes[1].bytes[uvIndex];
      final vp = image.planes[2].bytes[uvIndex];
      // Calculate pixel color
      int r = (yp + vp * 1436 / 1024 - 179).round().clamp(0, 255);
      int g = (yp - up * 46549 / 131072 + 44 - vp * 93604 / 131072 + 91)
          .round()
          .clamp(0, 255);
      int b = (yp + up * 1814 / 1024 - 227).round().clamp(0, 255);
      // color: 0x FF  FF  FF  FF
      //           A   B   G   R
      if (img.boundsSafe(height - y, x)) {
        img.setPixelRgba(height - y, x, r, g, b, shift);
      }
    }
  }

  return img;
}

(Edit)

To capture and save images you can use as follows.

import 'package:camera/camera.dart';
import 'package:image/image.dart' as imglib;

Then by your method:

_controller?.startImageStream((CameraImage image){
  Uint8List imageBytes = image.planes[0].bytes;
  //convert bytedata to image
  imglib.Image bitmap = imglib.decodeImage(imageBytes.toList());

  //then save on your directories use path_provider package to get all directories you are able to save in a device
  File("pathToSave").writeAsBytesSync(imglib.encodeJpg(bitmap));

  // or as png
  File("pathToSave").writeAsBytesSync(imglib.encodePng(bitmap));
});

Or as an alternative:

// get picture as cross platform file as an alternative more easy (without passin by _controller.startImageStream)
XFile xfile = await _controller?.takePicture();

// get bytedata of it
Uint8List imageBytes = await xfile.readAsBytes();

//convert bytedata to image
imglib.Image bitmap = imglib.decodeImage(imageBytes.toList());

//then save on your directories use path_provider package to get all directories you are able to save in a device
File("pathToSave").writeAsBytesSync(imglib.encodeJpg(bitmap));

// or as png
File("pathToSave").writeAsBytesSync(imglib.encodePng(bitmap));

To rotate by the right side you can use this, passing bytedata’s As UintList:

void saveImageFromUint8List(Uint8List data, int width, int height, String pathToSave) async {
  
    try {
      // Here you can use read_exif package
      final Map<String, IfdTag> exifData = await readExifFromBytes(data.toList());
      imglib.Image  source = imglib.decodeImage(data);
      if (source != null) {
        if (exifData.length > 0) {
          final IfdTag? imgOrientation = exifData['Image Orientation'];
          if (imgOrientation != null) {
            int rotation = 0;
            switch (imgOrientation.values.firstAsInt()) {
              case 6: // Make constants for it read about EXIF metadata
                rotation = 90;
                break;
              case 3: // Make constants for it read about EXIF metadata
                rotation = 180;
                break;
              case 8: // Make constants for it read about EXIF metadata
                rotation = -90;
                break;
            }
            if (rotation != 0) source = imglib.copyRotate(source, rotation); //or specify this parameter for newest version of  , angle: rotation);
          }
        }
        if (source.width != width || source.height != height) {
          File("pathToSave").writeAsBytesSync(imglib.encodeJpg(source));
        }
      }
    } catch (e) {
      // log(e.toString());
    }

  }

And to get rotated image by passing the original you can use:

Future<imglib.Image> getRotatedByTheRightSide(imglib.Image image) async {
    imglib.Image result = image;
    try {
     
      if (image != null) {
        // Here you can use read_exif package
        final Map<String, IfdTag> exifData = await readExifFromBytes(image.getBytes());
        
        if (exifData.length > 0) {
          final IfdTag? imgOrientation = exifData['Image Orientation'];
          if (imgOrientation != null) {
            int rotation = 0;
            switch (imgOrientation.values.firstAsInt()) {
              case 6: // Make constants for it read about EXIF metadata
                rotation = 90;
                break;
              case 3: // Make constants for it read about EXIF metadata
                rotation = 180;
                break;
              case 8: // Make constants for it read about EXIF metadata
                rotation = -90;
                break;
            }
            if (rotation != 0) result = imglib.copyRotate(image, angle: rotation);
          }
        }
      }
    
    } catch (e) {
      // log(e.toString());
    }
    
    return result;
  }

Please signup or login to give your own answer.

Click here to cancel reply.