Javascript Question posted in
A very good W3school tutorial can be found here.

Javascript – Teachable Machine from Google with tensorflow.js model – jpg files for clasification instead of video stream from webcam

I would like to ask you if somebody have experience with Google Teachable Machine model of tenworflow.js . The complete code to use is generated on Google Teachalbe machine, that is not a problem. But I tried to modify it to clasify image in form of jpg file instead of clasifying of webcam stream. I did not succeed with canvas and also not with img elements. Do you have any experiences with it or any advice? I will appreciate it. Here is how the standard javascript from Teachable Machine looks like and where I would like to modify the part which is clasifying the webcam stream to clasify the static jpg file which I would provide (with the webcam stream it works perfect). Thanks in advance for your help and comments:

<button type="button" onclick="init()">Start</button>
<div id="webcam-container"></div>
<div id="label-container"></div>
<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@latest/dist/tf.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@teachablemachine/image@latest/dist/teachablemachine-image.min.js"></script>
<script type="text/javascript">
    // More API functions here:
    // https://github.com/googlecreativelab/teachablemachine-community/tree/master/libraries/image

    // the link to your model provided by Teachable Machine export panel
    const URL = "myModel";

    let model, webcam, labelContainer, maxPredictions;

    // Load the image model and setup the webcam
    async function init() {
        const modelURL = URL + "model.json";
        const metadataURL = URL + "metadata.json";

        // load the model and metadata
        // Refer to tmImage.loadFromFiles() in the API to support files from a file picker
        // or files from your local hard drive
        // Note: the pose library adds "tmImage" object to your window (window.tmImage)
        model = await tmImage.load(modelURL, metadataURL);
        maxPredictions = model.getTotalClasses();

        // Convenience function to setup a webcam
        const flip = true; // whether to flip the webcam
        webcam = new tmImage.Webcam(200, 200, flip); // width, height, flip
        await webcam.setup(); // request access to the webcam
        await webcam.play();
        window.requestAnimationFrame(loop);

        // append elements to the DOM
        document.getElementById("webcam-container").appendChild(webcam.canvas);
        labelContainer = document.getElementById("label-container");
        for (let i = 0; i < maxPredictions; i++) { // and class labels
            labelContainer.appendChild(document.createElement("div"));
        }
    }

    async function loop() {
        webcam.update(); // update the webcam frame
        await predict();
        window.requestAnimationFrame(loop);
    }

    // run the webcam image through the image model
    async function predict() {
        // predict can take in an image, video or canvas html element
        const prediction = await model.predict(webcam.canvas);
        for (let i = 0; i < maxPredictions; i++) {
            const classPrediction =
                prediction[i].className + ": " + prediction[i].probability.toFixed(2);
            labelContainer.childNodes[i].innerHTML = classPrediction;
        }
    }
</script>

One of the modificaion I have tried but with no success, as it is returning wrong predictin and still the exactely same prediction results for any .jpg file, no difference when I feed completely different pictures, still exactelly the same numbers on prediction):

        // predict can take in an image, video or canvas html element

    const canvas = document.getElementById('myCanvas');
    const ctx = canvas.getContext('2d');

    const image = new Image();
    image.onload = function() {
        ctx.drawImage(image, 0, 0);
    };
    image.src = '1.jpg';


        const prediction = await model.predict(canvas); //webcam.canvas
        for (let i = 0; i < maxPredictions; i++) {
            const classPrediction =
                prediction[i].className + ": " + prediction[i].probability.toFixed(2);
            labelContainer.childNodes[i].innerHTML = classPrediction;
        }
    }````
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I have tried to modify the code to avoid problem with the timing of not loaded picture. To make sure it is ok I created another button for initialization and then for prediction. Anyway I am still not able to make right prediction. What is strange that in case I draw to canvas only a line or something then I am getting the prediction with the canvas. But when I draw a picture to the canvas then I am not getting any result from the predict function and it stuck. Here is the modified code I used: (So I guess there is some problem in the canvas data representation when red from the jpg file)

    <div>Teachable Machine Image Model</div>
<button type="button" onclick="init()">Init</button>
<button type="button2" onclick="predict()">Start predict</button>
<div id="labels">aaa</div>
<canvas id="myCanvas" width="200" height="200"></canvas>
<img id="picture1" src="1.jpg" alt="Popisek obrázku">


<script src="https://cdn.jsdelivr.net/npm/@tensorflow/tfjs@latest/dist/tf.min.js"></script>
<script src="https://cdn.jsdelivr.net/npm/@teachablemachine/image@latest/dist/teachablemachine-image.min.js"></script>
<script type="text/javascript">

labels = document.getElementById("labels");
picture1 = document.getElementById("picture1");
canvas1 = document.getElementById("myCanvas");

//With this I am able to run the predict(canvas1) and will get some results
const ctx = canvas1.getContext('2d');
      ctx.strokeStyle = 'red';
      ctx.beginPath();
      ctx.moveTo(0, 0);
      ctx.lineTo(100, 100);
      ctx.stroke();

//But with this I am NOT able to run the predict(canvas1) and script stuck
  const image = new Image();
    image.onload = function() {
      ctx.drawImage(image, 0, 0);
    };
  image.src = "1.jpg";


let model, webcam, labelContainer, maxPredictions;


const URL = "https://teachablemachine.withgoogle.com/models/lVXdg0qSw/";

async function init() {
        const modelURL = URL + "model.json";
        const metadataURL = URL + "metadata.json";

        model = await tmImage.load(modelURL, metadataURL);
        maxPredictions = model.getTotalClasses();
}


    async function predict() {
        // predict can take in an image, video or canvas html element
        const prediction = await model.predict(canvas1); //
        for (let i = 0; i < maxPredictions; i++) {
            const classPrediction =
                prediction[i].className + ": " + prediction[i].probability.toFixed(2);
            labels.innerHTML = classPrediction;
        }

    }

</script>


  2. 0

    Your problem here might be a so called "race condition". The code is not executed in the order you are expecting.

    async function predict() {    
    const canvas = document.getElementById('myCanvas');
    const ctx = canvas.getContext('2d');

    const image = new Image();
    // the loading here takes some time
    image.onload = function() {
        // this is executed after the foor loop below
        ctx.drawImage(image, 0, 0);
    };
    image.src = '1.jpg';

    // this is executed before the image has loaded and was drawn on the canvas
    // so, the canvas is probably still blank at this point
    const prediction = await model.predict(canvas); //webcam.canvas
    for (let i = 0; i < maxPredictions; i++) {
        const classPrediction =
            prediction[i].className + ": " + prediction[i].probability.toFixed(2);
        labelContainer.childNodes[i].innerHTML = classPrediction;
    }
}

    You would need to make sure to call predict only when the image has fully loaded. This is what the onload is for, call predict from there.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search