Question 376641 in Ubuntu Question posted in
The official Ubuntu documentation can be found here.

Ubuntu – FFmpeg dynamic cropping using sendcmd correct syntax?

I’m trying to dynamically crop a video using FFmpeg’s sendcmd filter based on coordinates specified in a text file, but the crop commands do not seem to be taking effect. Here’s the format of the commands I’ve tried and the corresponding FFmpeg command I’m using.

Following the documentation https://ffmpeg.org/ffmpeg-filters.html#sendcmd_002c-asendcmd, commands in the text file (coordinates.txt) like this:

0.05 [enter] crop w=607:h=1080:x=0:y=0;
0.11 [enter] crop w=607:h=1080:x=0:y=0;
...

Ffmpeg command:

ffmpeg -i '10s.mp4' -filter_complex "[0:v]sendcmd=f=coordinates.txt" -c:v libx264 -c:a copy -r 30 output.mp4

This doesn’t seem to do anything.

And with the commands in the text file (coordinates.txt) like this:

0.05    crop w 607, crop h 1080, crop x 0, crop y 0;
0.11    crop w 607, crop h 1080, crop x 0, crop y 0;
...

Ffmpeg command:

ffmpeg -i '10s.mp4' -filter_complex "[0:v]sendcmd=f=coordinates.txt,crop" -c:v libx264 -c:a copy -r 30 output.mp4

(following this answer https://stackoverflow.com/a/67508233/1967110)

This one does something, but something very messy. It looks like it crops at the correct x, but does not take into account the y, w or h, and it puts the crop at the right side of the input video.

Edit: what I’m trying to do is create a 607×1080 (portrait format, 9:16) video from a 1920×1080 video, with the x parameter varying across time (imagine sliding horizontally a 9:16 frame over a 16:9 video). So fixed w, h and y, just x varying.

I’m using this ffmpeg version:

ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
configuration: --prefix=/home/ffmpeg-builder/release --pkg-config-flags=--static --extra-libs=-lm --disable-doc --disable-debug --disable-shared --disable-ffprobe --enable-static --enable-gpl --enable-version3 --enable-runtime-cpudetect --enable-avfilter --enable-filters --enable-nvenc --enable-nvdec --enable-cuvid --toolchain=hardened --disable-stripping --enable-opengl --pkgconfigdir=/home/ffmpeg-builder/release/lib/pkgconfig --extra-cflags='-I/home/ffmpeg-builder/release/include -static-libstdc++ -static-libgcc ' --extra-ldflags='-L/home/ffmpeg-builder/release/lib -fstack-protector -static-libstdc++ -static-libgcc ' --extra-cxxflags=' -static-libstdc++ -static-libgcc ' --extra-libs='-ldl -lrt -lpthread' --enable-ffnvcodec --enable-gmp --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libkvazaar --enable-libmp3lame --enable-libopus --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libshine --enable-libsoxr --enable-libsrt --enable-libsvtav1 --enable-libtheora --enable-libvidstab --ld=g++ --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-libzimg --enable-openssl --enable-zlib --enable-nonfree --extra-libs=-lpthread --enable-pthreads --extra-libs=-lgomp
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    Ok so the only command I got working so far, taken from https://video.stackexchange.com/posts/19403/revisions , is this:

    ffmpeg -i in.mp4 -filter_complex_script file.txt -map "[out]" output.mp4

    with file.txt like this

    nullsrc=WxH:r=FPS[cv];
[cv][0]overlay=-X0:-Y0:shortest=1:enable='eq(n,0)'[b0];
[b0][0]overlay=-X1:-Y1:shortest=1:enable='eq(n,1)'[b1];
[b1][0]overlay=-X2:-Y2:shortest=1:enable='eq(n,2)'[b2];
...
[bm-1][0]overlay=-Xm:-Ym:shortest=1:enable='eq(n,m)'[out]

    this takes about 40s to crop a 1920x1080 5s 30fps video to a 606x1080 video (on google colab, so low-level cpu, and without gpu).

    I also tried to first decompose the video into each of its frames, crop each frame, and reassemble the video. Here is the code:

    def decompose_video(video_path, frames_dir):
    if not os.path.exists(frames_dir):
        os.makedirs(frames_dir)
    command = [
        'ffmpeg', '-i', video_path, 
        os.path.join(frames_dir, 'frame_%04d.png')
    ]
    subprocess.run(command, check=True)

def read_crop_values(crop_file):
    with open(crop_file, 'r') as file:
        crops = [line.strip().split(',') for line in file]
    return crops

def apply_crop(frames_dir, crops):
    cropped_frames_dir = frames_dir + '_cropped'
    if not os.path.exists(cropped_frames_dir):
        os.makedirs(cropped_frames_dir)
    
    for i, crop in enumerate(crops, 1):
        x, y, w, h = crop
        input_frame = os.path.join(frames_dir, f'frame_{i:04d}.png')
        output_frame = os.path.join(cropped_frames_dir, f'frame_{i:04d}.png')
        command = [
            'ffmpeg', '-i', input_frame,
            '-filter:v', f'crop={w}:{h}:{x}:{y}',
            output_frame
        ]
        subprocess.run(command, check=True)

def recreate_video(cropped_frames_dir, output_video_path):
    command = [
        'ffmpeg', '-y', '-loglevel', 'verbose', '-r', '29.97', '-f', 'image2', '-i', 
        os.path.join(cropped_frames_dir, 'frame_%04d.png'),
        '-vcodec', 'libx264', '-crf', '23', '-pix_fmt', 'yuv420p',
        output_video_path
    ]
    try:
        subprocess.run(command, check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    except subprocess.CalledProcessError as e:
        print("FFmpeg failed with error:")
        print(e.stderr.decode()) 

video_path = 'input.mp4'
frames_dir = 'frames'
crop_file = 'crop_values.txt'
output_video_path = 'output.mp4'

decompose_video(video_path, frames_dir)
crops = read_crop_values(crop_file)
apply_crop(frames_dir, crops)
recreate_video(frames_dir + '_cropped', output_video_path)

    and in crop_values.txt you just put x, y, w, h:

    0,0,606,1080
8,0,606,1080
17,0,606,1080

    It also works but is quite a bit slower (1m40 for the same video).


  2. 0

    The only syntax I got working (but with messy cropping) was:

    0.0-3.0 [enter] crop w 807,h 720,x 0,y 0;
3.0-6.0 [enter] crop w 407,h 720,x 0,y 0;
6.0-9.0 [enter] crop w 307,h 720,x 0,y 0;
9.0-10.0 [enter] crop w 207,h 720,x 0,y 0;

    And:

    ffmpeg -i 10s.mp4 -filter_complex "[0:v]sendcmd=f=coordinates.txt,crop" -c:v libx264 -c:a copy -r 30 output.mp4 -y

    As mentioned, the crop would work with w and h identical on each line.
    But successive crops do not work well.

    Following this answer, I seem to see more cropping with:

    swaprect=900:500:0:0:100:100:enable='between(t,0,3)',
swaprect=900:500:0:0:200:100:enable='between(t,3,6)',
swaprect=900:500:0:0:400:100:enable='between(t,6,9)',
swaprect=900:500:0:0:600:100:enable='between(t,9,10)',
crop=900:500:0:0;

    And: ffmpeg -i 10s.mp4 -filter_script:v:0 coordinates2.txt output.mp4 -y

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search