Question 72216 in IOS Swift Question posted in
The official documentation can be found here.

How to convert CMSampleBuffer to OpenCV's Mat instance in swift – Ios swift

I am working on an OpenCV project where I am taking input from iPhone native camera as CMSampleBuffer now I wanted to create Mat instance that is required in OpenCV for further process

I have found some old post related with it but all are not working in current swift as those are pretty old.

Raw image data from camera like "645 PRO"

How to convert CMSampleBufferRef to IplImage (iOS)

Tags:

2

Answers


  1. 0

    As far as I know openCV is written in C++ and you will have to interact with it using an Objc++ or an Objc wrapper.
    Hence the resources you found are solid and there are many others on working with OpenCV on iOS. A simple swifty wrapper I found online is – https://github.com/Legoless/LegoCV.

    There is a full interoperability between Objc/Objc++ and swift and it is fairly easy to implement. See these two resources for more info – https://developer.apple.com/documentation/swift/imported_c_and_objective-c_apis/importing_objective-c_into_swift
    https://rderik.com/blog/understanding-objective-c-and-swift-interoperability/

    Login or Signup to reply.
  2. 0

    First, convert CMSampleBuffer To UIImage.

    extension CMSampleBuffer {
    func asUIImage()-> UIImage? {
        guard let imageBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(self) else {
            return nil
        }
        let ciImage = CIImage(cvPixelBuffer: imageBuffer)
        return convertToUiImage(ciImage: ciImage)
    }
    
    func convertToUiImage(ciImage: CIImage) -> UIImage? {
        let context = CIContext(options: nil)
        context.clearCaches()
        guard let cgImage = context.createCGImage(ciImage, from: ciImage.extent) else {
            return nil
        }
        let image = UIImage(cgImage: cgImage)
        return image
    }
}

    Then you can easily convert UIImage to Mat and return UIImage with/without doing something.
    OpenCVWrapper.h file

    #import <Foundation/Foundation.h>
#import <UIKit/UIKit.h>

NS_ASSUME_NONNULL_BEGIN

@interface OpenCVWrapper : NSObject

+ (UIImage *) someOperation : (UIImage *) uiImage;

@end

NS_ASSUME_NONNULL_END

    OpenCVWrapper.mm file

    #import <opencv2/opencv.hpp>
#import <opencv2/imgcodecs/ios.h>
#import <opencv2/core/types_c.h>
#import "OpenCVWrapper.h"
#import <opencv2/Mat.h>

#include <iostream>
#include <random>
#include <vector>
#include <chrono>
#include <stdint.h>

@implementation OpenCVWrapper

+ (UIImage *) someOperation : (UIImage *) uiImage {
    cv::Mat sourceImage;
    UIImageToMat(uiImage, sourceImage);
    // Do whatever you need
    return MatToUIImage(sourceImage);
}

@end
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search