I have an input file with 9095 lines of text. I need to take this list of filenames and use them as input to an ffmpeg command. I really would like to avoid running it all at once (line by line until end of file).
Is there a way in a bash script to take the first 25 lines, execute the ffmpeg command on them, then continue to the next 25 lines and execute the same command, until the end of the input file?
Input file (example.txt) has list of 9095 filenames to be processed.
I have tried the following:
#!/bin/bash
while mapfile -n 25 convert < example.txt do
while read -r line || [[ -n "$line" ]]; do fn_out="w$line"; (/path/to/file/ffmpeg -report -nostdin -i /path/to/file/watermark.png -i /path/to/file/"$line" -filter_complex "scale2ref[a][b];[b][a]overlay=x=(main_w-overlay_w)/2:y=(main_h-overlay_h)/2" /path/to/file/"$fn_out" -c:v libx265); done < $convert >> script.txt
done
This only gets me the first 25 files. How to account for all 9095 files in example.txt?
Also, is there any way to "pause" the script in between each set of 25 filenames? I am not familiar enough with mapfile and bash scripting to know how to handle this kind of iteration.
Basically, the output needs to be as follows:
- Read first 25 lines of example.txt
- Run the ffmpeg command on those 25 files (some files are over 2GB. Will take time to do the ffmpeg command.
- Pause for about a minute or two. Let server rest/catch up/recover resources.
- Get the next 25 names from example.txt
- Run the ffmpeg conversion loop again in step 2
- Continue until EOF of example.txt
Any direction on this would be greatly appreciated!
Running CentOS 7 with bash 5.0.17.
3
Answers
You need to put the redirection around the entire loop, not just the
mapfileline. Just like when you usewhile readto iterate through a file one line at a time.Then each time through the loop it will get the next 25 lines of the file.
With
mapfileakareadarray, something like:A simple test run.
Output:
The
(( ${#files[*]} == num ))will execute the loop if it is true, if that is not desired just test if the arrayfileshas a value (regardless if it is less than 25 elements), change it to(( ${#files[*]} ))wc -l < file.txtto get the total lines in a file, to save that in a variable you could do,total_lines=$(wc -l < file.txt)Update:
Instead of using
wc -l < file.txt(not to hardcode the total number of lines in a file) to save the number of lines in a variable, something like:Adapting to @mark-mapuso’s modulus approach, something like:
Assumptions/understandings:
One
bashapproach using a simple counter (and no need for an array):Setup a small demo:
file.txtRun the demo: