Question 334608 in Javascript Question posted in
A very good W3school tutorial can be found here.

Javascript – Web Development – Seeking a Solution for Video Slicing to Upload Segments to Server

In the course of a web development project, I encountered a challenge with video slicing for efficient uploading to a server. Initially, I developed a service that successfully sliced MP4 files by extracting bytes from one point to another. However, I’ve encountered an issue during the merging process after uploading the segmented videos. The videos fail to merge successfully, and I’m seeking insights and solutions to overcome this hurdle.

In my attempts to address the issue, I’ve reviewed the video slicing process and implemented various approaches. Despite these efforts, the merging of the segmented videos still poses a problem. I expected a seamless merging process after uploading the segmented parts, but unfortunately, that hasn’t been the case. However, I’ve come to realize that video slicing involves more complexity, as outlined in this insightful Stack Overflow post.

If anyone has experience with video slicing, especially in the context of web development, and has successfully tackled similar merging challenges, I would greatly appreciate your guidance. Additionally, if you have recommendations for packages or libraries that handle video slicing and merging effectively, please share your insights.

EDIT:

Simplified client code

export class Chunks {

  chunkSize: number
  private file: File

  constructor({chunkSize}) {
    this.chunkSize = chunkSize * 1024
  }

  get totalChunks () {
    return Math.ceil(this.file.size / this.chunkSize);
  }

  setFile(file: File) {
    this.file = file
  }

  // Function to read a chunk of the file
  readChunk(start: number, end: number) {
    const file =  this.file
    return file.slice(start, end)
  }


  async getChunks(i: number) {
    i--
    if(i < this.totalChunks) {
      const start = i * this.chunkSize;
      const end = Math.min(start + this.chunkSize, this.file.size);
      const chunk = await this.readChunk(start, end);

      return chunk
    }
  }

}


  const chunks = new Chunks({chunkSize: 1024})
  chunks.setFile(File)

  for (let index = 1;  index <= chunks.totalChunks; index++) {

    const blob = await chunks.getChunks(index)
    const blobFile = new File([blob], "test.mp4", { type: blob.type });
    const formData = new FormData();

    formData.append("blobFile", blobFile);
    formData.append("length", chunks.totalChunks);
    formData.append("index", index);
    return this.http.post(`${geturl}`, formData, options)
  }

Server side code

enter image description here

The error arises within the concatMedia function, and in the exception block, nothing is caught.

Test

I conducted some tests, and I can play the first chunk as a video. However, the rest cannot be recognized as a video; it simply doesn’t recognize it.

const blob = await chunks.getChunks(1)
const blobFile = new File([blob], "test.mp4", { type: blob.type });
// Create a URL for the blobFile
const blobUrl = URL.createObjectURL(blobFile);

// Get the video element from the DOM
const videoElement: any = document.getElementById("yourVideoElementId");

// Set the src attribute of the video element to the blob URL
videoElement.src = blobUrl;

// Optionally, you can also set other attributes or play the video
videoElement.controls = true; // Add controls for play, pause, etc.
videoElement.play(); // Auto-play the vid

Report

The backend guy conducted some tests as well. He split the video using an online tool, then uploaded the first video and the second. He successfully merged them.

I attempted not to slice the video and sent the full video in one chunk. After uploading, the concatMedia function did not raise any errors.

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Thanks to traktor, I was able to come up with the client and server code

    Client code

    <!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Chunked Upload</title>
</head>
<body>
  <input type="file" id="fileInput" />
  <button onclick="uploadChunks()">Upload</button>

  <script>
    async function uploadChunks() {
      const fileInput = document.getElementById('fileInput');
      const file = fileInput.files[0];

      const chunkSize = 1024 * 1024; // 1MB chunks
      const totalChunks = Math.ceil(file.size / chunkSize);

      for (let chunkNumber = 1; chunkNumber <= totalChunks; chunkNumber++) {
        const start = (chunkNumber - 1) * chunkSize;
        const end = Math.min(chunkNumber * chunkSize, file.size);
        const chunkData = await readChunk(file, start, end);

        // Send the chunk to the server
        await sendChunk(file.name, chunkNumber, totalChunks, chunkData);
      }
    }

    async function readChunk(file, start, end) {
      return new Promise((resolve, reject) => {
        const reader = new FileReader();
        reader.onload = (event) => resolve(event.target.result.split(',')[1]);
        reader.onerror = (error) => reject(error);
        reader.readAsDataURL(file.slice(start, end));
      });
    }

    async function sendChunk(fileName, chunkNumber, totalChunks, data) {
      const url = 'http://localhost:3005/upload';

      const response = await fetch(url, {
        method: 'POST',
        headers: {
          'Content-Type': 'application/json',
        },
        body: JSON.stringify({
          fileName,
          chunkNumber,
          totalChunks,
          data,
        }),
      });

      const result = await response.json();
    }
  </script>
</body>
</html>

    Server code

    const multer = require('multer');
const express = require('express');
const fs = require('fs');
const path = require('path');
const bodyParser = require('body-parser');
const cors = require('cors');
const app = express();
const formData = require('express-form-data');
app.use(express.json({ limit: '50mb' })); // Adjust the limit as needed
app.use(express.urlencoded({ limit: '50mb', extended: true })); // Adjust the limit as needed
const port = 3005;

app.use(formData.parse());
const corsOptions = {
  origin: '*',
  methods: 'GET,HEAD,PUT,PATCH,POST,DELETE',
  allowedHeaders: 'Content-Type, Authorization',
};
// Use body-parser middleware to parse form data
app.use(bodyParser.urlencoded({ extended: true }));
app.use(cors(corsOptions));

app.use(express.json());
app.use(express.urlencoded({ extended: true }));

app.post('/upload', async (req, res) => {
  const { fileName, chunkNumber, totalChunks, data } = req.body;

  const filePath = path.join(__dirname, 'uploads', fileName);

  try {
    await appendToFile(filePath, Buffer.from(data, 'base64'));

    if (chunkNumber === totalChunks) {
      res.json({ success: true, message: 'File uploaded successfully!' });
    } else {
      res.json({ success: true, message: 'Chunk received successfully.' });
    }
  } catch (error) {
    console.error(error);
    res.status(500).json({ success: false, message: 'Error uploading chunk.' });
  }
});

app.listen(port, () => {
  console.log(`Server is running on http://localhost:${port}`);
});

async function appendToFile(filePath, data) {
  return new Promise((resolve, reject) => {
    fs.writeFile(filePath, data, { flag: 'a' }, (err) => {
      if (err) {
        reject(err);
      } else {
        resolve();
      }
    });
  });
}


  2. 0

    I’ll skip over the actual client side code for now given your assurances that it is in fact working.

    The most suspicious issue that I would not expect to work, but would account being able to play the first chunk but not anything afterward, is trying to use ffmpeg to concatenate chunks of binary data that are not in themselves valid mp4 video files or, if split in the wrong place of a video stream, disrupt decoding without ffmpeg being able to recover.

    On the server I would suggest discarding the use of ffmpeg to concatenate uploaded chunks and instead:

    1. Create an empty output file named after the uploaded mp4 file.

    2. Append chunks of the uploaded file content to the output file, making sure that chunk data is treated as binary data in both read/get and write/append operations.

    3. Close and flush the output file after the last chunk has been appended to it. The output file should be the uploaded video file.

    If the output file plays as expected, problem solved. If not it may provide clues in its file size compared with the uploaded file, and may require revisiting client side upload code in more detail.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search