While trying to make an object detection app in react native using torchvison models encountered the following error –
error –
Possible Unhandled Promise Rejection (id: 1):
Object {
"message": "Format error
Exception raised from _load_for_mobile at /data/users/atalman/pytorch/torch/csrc/jit/mobile/import.cpp:623 (most recent call first):
(no backtrace available)",
}
reinstalling pytorch did not solve the problem.
followed documentation – > https://playtorch.dev/docs/tutorials/snacks/object-detection/
for version 0.2.4
code App.js –
import {StyleSheet, Text, View} from 'react-native';
import React from 'react';
import {
Camera,
MobileModel,
torch,
torchvision,
media,
} from 'react-native-pytorch-core';
let model = null;
const T = torchvision.transforms;
const App = () => {
async function handleImage(image) {
console.log('Image Taken!!');
const width = image.getWidth();
const height = image.getHeight();
// 3.ii. Convert image to blob, which is a byte representation of the image
// in the format height (H), width (W), and channels (C), or HWC for short
const blob = media.toBlob(image);
// 3.iii. Get a tensor from image the blob and also define in what format
// the image blob is.
let tensor = torch.fromBlob(blob, [height, width, 3]);
// 3.iv. Rearrange the tensor shape to be [CHW]
tensor = tensor.permute([2, 0, 1]);
// 3.v. Divide the tensor values by 255 to get values between [0, 1]
tensor = tensor.div(255);
// 3.vi. Crop the image in the center to be a squared image
const centerCrop = T.centerCrop(Math.min(width, height));
tensor = centerCrop(tensor);
// 3.vii. Resize the image tensor to 3 x 224 x 224
const resize = T.resize(224);
tensor = resize(tensor);
// 3.viii. Normalize the tensor image with mean and standard deviation
const normalize = T.normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225]);
tensor = normalize(tensor);
// 3.ix. Unsqueeze adds 1 leading dimension to the tensor
tensor = tensor.unsqueeze(0);
// console.log(tensor);
// 3.x. Return the tensor shape [1, 3, 224, 224]
const result = tensor.shape;
console.log('result:', result);
if (model == null) {
console.log('Loading model...');
const filePath = await MobileModel.download('detr_resnet50.ptl');
model = await torch.jit._loadForMobile(filePath);
console.log('Model successfully loaded');
}
console.log('Forward propogation !!');
const output = await model.forward(tensor);
console.log(output);
}
return (
<View style={styles.container}>
<Text style={styles.label}>Class: </Text>
<Camera style={styles.camera} onCapture={handleImage} />
</View>
);
};
export default App;
const styles = StyleSheet.create({
container: {
flexGrow: 1,
backgroundColor: '#ffff',
padding: 20,
alignItems: 'center',
},
label: {
marginBottom: 10,
color: 'black',
fontSize: 20,
},
camera: {
flexGrow: 1,
width: '100%',
marginTop: 70,
},
});
project dependencies –
"devDependencies": {
"@babel/core": "^7.12.9",
"@babel/runtime": "^7.12.5",
"@react-native-community/eslint-config": "^2.0.0",
"babel-jest": "^26.6.3",
"eslint": "^7.32.0",
"jest": "^26.6.3",
"metro-react-native-babel-preset": "0.72.3",
"react-test-renderer": "18.1.0"
}
2
Answers
you can try:
I believe the issue here is that you must specify the full path of the model based on where it is stored in the cloud. Following this tutorial you linked, it looks like the full url for the model is
whereas in your code you only specify the file name:
try to create a varialbe named MODEL_URL as in the turorial and pass that into the MobileModel.download function like this: