skip to Main Content

While trying to make an object detection app in react native using torchvison models encountered the following error –
error –

Possible Unhandled Promise Rejection (id: 1):
Object {
  "message": "Format error
Exception raised from _load_for_mobile at /data/users/atalman/pytorch/torch/csrc/jit/mobile/import.cpp:623 (most recent call first):
(no backtrace available)",
}

reinstalling pytorch did not solve the problem.
followed documentation – > https://playtorch.dev/docs/tutorials/snacks/object-detection/
for version 0.2.4

code App.js –

import {StyleSheet, Text, View} from 'react-native';
import React from 'react';
import {
  Camera,
  MobileModel,
  torch,
  torchvision,
  media,
} from 'react-native-pytorch-core';
let model = null;
const T = torchvision.transforms;
const App = () => {
  async function handleImage(image) {
    console.log('Image Taken!!');
    const width = image.getWidth();
    const height = image.getHeight();
    // 3.ii. Convert image to blob, which is a byte representation of the image
    // in the format height (H), width (W), and channels (C), or HWC for short
    const blob = media.toBlob(image);
    // 3.iii. Get a tensor from image the blob and also define in what format
    // the image blob is.
    let tensor = torch.fromBlob(blob, [height, width, 3]);
    // 3.iv. Rearrange the tensor shape to be [CHW]
    tensor = tensor.permute([2, 0, 1]);
    // 3.v. Divide the tensor values by 255 to get values between [0, 1]
    tensor = tensor.div(255);
    // 3.vi. Crop the image in the center to be a squared image
    const centerCrop = T.centerCrop(Math.min(width, height));
    tensor = centerCrop(tensor);
    // 3.vii. Resize the image tensor to 3 x 224 x 224
    const resize = T.resize(224);
    tensor = resize(tensor);
    // 3.viii. Normalize the tensor image with mean and standard deviation
    const normalize = T.normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]);
    tensor = normalize(tensor);
    // 3.ix. Unsqueeze adds 1 leading dimension to the tensor
    tensor = tensor.unsqueeze(0);
    // console.log(tensor);
    // 3.x. Return the tensor shape [1, 3, 224, 224]
    const result = tensor.shape;
    console.log('result:', result);
    if (model == null) {
      console.log('Loading model...');
      const filePath = await MobileModel.download('detr_resnet50.ptl');
      model = await torch.jit._loadForMobile(filePath);
      console.log('Model successfully loaded');
    }
    console.log('Forward propogation !!');
    const output = await model.forward(tensor);
    console.log(output);
  }
  return (
    <View style={styles.container}>
      <Text style={styles.label}>Class: </Text>
      <Camera style={styles.camera} onCapture={handleImage} />
    </View>
  );
};

export default App;

const styles = StyleSheet.create({
  container: {
    flexGrow: 1,
    backgroundColor: '#ffff',
    padding: 20,
    alignItems: 'center',
  },
  label: {
    marginBottom: 10,
    color: 'black',
    fontSize: 20,
  },
  camera: {
    flexGrow: 1,
    width: '100%',
    marginTop: 70,
  },
});

project dependencies –

  "devDependencies": {
    "@babel/core": "^7.12.9",
    "@babel/runtime": "^7.12.5",
    "@react-native-community/eslint-config": "^2.0.0",
    "babel-jest": "^26.6.3",
    "eslint": "^7.32.0",
    "jest": "^26.6.3",
    "metro-react-native-babel-preset": "0.72.3",
    "react-test-renderer": "18.1.0"
  }

2

Answers


  1. you can try:

    const filePath = await MobileModel.download(require('./detr_resnet50.ptl'));
    
    Login or Signup to reply.
  2. I believe the issue here is that you must specify the full path of the model based on where it is stored in the cloud. Following this tutorial you linked, it looks like the full url for the model is

    const MODEL_URL =
      'https://github.com/facebookresearch/playtorch/releases/download/v0.1.0/detr_resnet50.ptl';
    
    

    whereas in your code you only specify the file name:

        if (model == null) {
          console.log('Loading model...');
          const filePath = await MobileModel.download('detr_resnet50.ptl'); <-- filename here
          model = await torch.jit._loadForMobile(filePath);
          console.log('Model successfully loaded');
        }
    

    try to create a varialbe named MODEL_URL as in the turorial and pass that into the MobileModel.download function like this:

       const MODEL_URL ='https://github.com/facebookresearch/playtorch/releases/download/v0.1.0/detr_resnet50.ptl';
        
        if (model == null) {
          console.log('Loading model...');
          const filePath = await MobileModel.download(MODEL_URL); <-- full path here
          model = await torch.jit._loadForMobile(filePath);
          console.log('Model successfully loaded');
        }
    
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search