Question posted in
The official documentation can be found here.

ffmpeg pipe images to redis

I need to find a way to save ffmpeg frames into redis variable.

Currently I’m piping png images from ffmpeg to redis but it concatenate all images:

ffmpeg -re -i udp://127.0.0.1:5545 -c:v png -r 1 -update 1 – | redis-cli -x HSET sb4up

Is it possible to update the redis variable with a new frame every second?

Tags:

2

Answers


  1. 0

    I have been having a think about this. I am no expert in ffmpeg and there may be something much simpler, so if someone makes me aware of a better method I will likely delete this.

    For the moment, my thought is that if ffmpeg writes JPEG or PNG as output, you need a tool that is able to parse a concatenated stream and search for START-OF-IMAGE/END-OF-IMAGE or (JPEG SOI/EOI or PNG IHDR/IEND) markers and understand chunks and sections. So, I thought it might be simpler to force ffmpeg to write fixed length images, then you can just read N bytes knowing it corresponds to a frame and you don’t need a bunch of "glue" code to write and maintain – just standard tools.

    I don’t have your UDP stream, so I just used a .mov file. I then forced RGB24 encoding and a fixed size, resized frame of 640×480. I then split that into chunks of 640x480x3 RGB bytes and ask ImageMagick to create a PNG from the raw video and whack it into Redis as a PNG:

    #!/bin/bash

# Set height and width of frame, and size in bytes
h=480
w=640
bytes=$((h*w*3))

ffmpeg -i video.mov -r 1 -s ${w}x${h} -update 1 -pix_fmt rgb24 -f rawvideo - 2> /dev/null | 
   gsplit -b $bytes --filter="magick -depth 8 -size ${w}x${h} rgb:- png:- | redis-cli -x SET sb4up"

    I then use OpenCV to grab the image every 0.5 seconds from Redis and display the result.

    #!/usr/bin/env python3

import redis
import cv2
import numpy as np

host='localhost'
port=6379
        
r = redis.Redis(host,port)

while True:
   png=r.get('sb4up')
   frame = cv2.imdecode(np.frombuffer(png,np.uint8), cv2.IMREAD_COLOR)
   cv2.imshow("Viewer", frame)
   cv2.waitKey(500)

    Note: I used gsplit for GNU split because I am on a Mac. On Linux, just use split.

    Note: Obviously you can use a differently sized frame as long as you update it in both parts of the code – i.e. both in gsplit and ImageMagick.

    Note: You don’t necessarily need to install ImageMagick and PNG-encode each image, because you will only have the one, single, latest image in Redis so space will not be an issue as there are not thousands of frames, so you could omit ImageMagick and just write the raw RGB24 data into Redis without ImageMagick. Then grab the raw data from Redis and use np.reshape() to make it into a correctly-shaped image.

    I am still thinking and may come up with something better… Perl might be good for splitting a binary stream on a pattern…

    Another idea is to have ffmpeg write PNG files with a sequential number like frame-%03d.png and run fswatch/inotify to grab the individual frames and stuff them into Redis.

    Login or Signup to reply.
  2. 0

    For some unknown reason. I got the urge to write some Python to split a continuous stream of concatenated PNG files and send each one to Redis as it arrives.

    The program reads a stream of PNG files in PNG chunks, rather than in 1MB blocks and looking for IEND markers which would result in a 30 image delay if your individual PNG images were 33kB each in size. Each PNG chunk has a "chunk size" (4 bytes in network byte order) and a "chunk type" (4 bytes). The first chunk is the 8-byte signature and the last one in an image is the IEND chunk. As chunks arrive they are read and gathered in a bytearray called wholePNG. When the end of the PNG arrives, I send this to Redis:

    #!/usr/bin/env python3
################################################################################
# pngsplit - split a concatenated stream of PNG files
# Mark Setchell
################################################################################

import redis
import os, sys
from struct import unpack

# Ensure stdin is binary
fp = os.fdopen(sys.stdin.fileno(), 'rb')

# Connect to Redis
host='localhost'
port=6379
r = redis.Redis(host,port)

# Define a valid PNG signature
validSig = bytes.fromhex('89504e470d0a1a0a')
expectingSig = True

while True:
   # We expect an 8-byte PNG signature at the start of each PNG file
   if expectingSig:
      # Get signature and ensure it is a PNG signature
      rawSig = fp.read(8)
      if len(rawSig) != 8:
         break
      if rawSig != validSig:
         print('ERROR: Not a valid PNG signature', file=sys.stderr)
         sys.exit(1)
      expectingSig = False
      # Start building a new PNG to send to Redis
      wholePNG = bytearray(validSig)

   # Read 4 bytes and extract PNG chunk length
   rawChunkLength = fp.read(4)
   chunkLen  = unpack('!I', rawChunkLength)[0]
   wholePNG += rawChunkLength

   # Read 4 bytes and extract PNG chunk type
   rawChunkType = fp.read(4)
   b0,b1,b2,b3 = unpack('BBBB', rawChunkType)
   chunkType = chr(b0) + chr(b1) + chr(b2) + chr(b3)
   print(f'DEBUG: {chunkType} PNG chunk, length={chunkLen}')
   wholePNG += rawChunkType

   # Read the chunk data
   if chunkLen > 0:
      rawData = fp.read(chunkLen)
      wholePNG += rawData

   # Read the CRC
   rawCRC = fp.read(4)
   wholePNG += rawCRC

   # If IEND chunk
   if chunkType == 'IEND':
      print('DEBUG: Received IEND chunk - sending to Redis')
      r.set('sb4up', bytes(wholePNG))
      # I guess we are now expecting a new PNG
      expectingSig = True

    You can run it like this:

    ffmpeg command producing PNG stream | ./pngsplit.py

    and it will send each PNG to Redis as sb4up as it arrives.

    I tested it like this in a directory with a load of random PNG images:

    while : ; do cat *.png; done | ./pngsplit.py

    Keywords: Python, image processing, ffmpeg, video, stream, split, PNG chunk, concatenate.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search