FFMPEG Programming: Overlay Audio with Pictures

[Eli]

This post details using FFmpeg to overlay audio with images, creating both short and long videos. We'll cover techniques for single image/audio combinations and multiple images across longer audio tracks.

I. Short Videos: One Frame per Second

For quickly displaying images as snapshots, use this command:

Code: [Select all] [Expand/Collapse]

1. ffmpeg -r 1 -f concat -safe 0 -i list_of_frames -i AUDIO -vf fps=1 VIDEO

This produces a video with one frame per second, its length determined by the number of frames in the input list.

II. Long Videos: Overlay Audio with Multiple Images

This section explains creating a video with images displayed continuously across a longer audio track. We use a 5.57-minute (357-second) audio file and 20 images as an example.

A. Creating list_of_frames.txt

To ensure continuous image display throughout the audio's duration, we create a text file listing images and their display durations. Each image will be shown for approximately 18 seconds. The total image duration (18 seconds/image * 20 images = 360 seconds) slightly exceeds the audio length. This is advantageous as FFmpeg's -shortest option will stop the video when the audio ends.

The list_of_frames.txt file will contain:

file Image1.jpg
duration 18
file Image2.jpg
duration 18
file Image3.jpg
duration 18
...
file Image20.jpg
duration 18
file Image20.jpg

Note: The last image is duplicated to extend the video slightly beyond the audio length.

B. FFmpeg Command for Long Videos

The following command generates the final video:

Code: [Select all] [Expand/Collapse]

1. ffmpeg -f concat -safe 0 -i list_of_frames.txt -i Audio.wav -vf format=yuv420p -c:v libx264 -c:a aac -shortest Video.mp4

This resulted in a high-quality 180 MB video, roughly equivalent to the combined size of the images and audio.

C. Command Explanation:

-f concat -safe 0 -i list_of_frames.txt: Reads image list and durations from list_of_frames.txt.
-i Audio.wav: Adds the audio file.
-vf format=yuv420p: Sets pixel format for broad player compatibility.
-c:v libx264: Encodes video using the H.264 codec.
-c:a aac: Encodes audio using the AAC codec.
-shortest: Stops the video when the audio ends.

III. Verifying Audio Length

Use this command to determine the exact duration of your audio file in seconds:

Code: [Select all] [Expand/Collapse]

1. ffprobe -i Audio.wav -show_entries format=duration -v quiet -of csv="p=0"

IV. Single Image/Audio Video Creation

For combining a single image and audio file, tested commands include:

Code: [Select all] [Expand/Collapse]

1. ffmpeg -loop 1 -i image.jpg -i input_audio.wav -c:v libx264 -tune stillimage -c:a aac -b:a 192k -pix_fmt yuv420p -shortest output.mp4

This command, adapted from the FFmpeg wiki via a Superuser link, may produce larger files.

A simpler (but potentially less flexible) alternative:

Code: [Select all] [Expand/Collapse]

1. ffmpeg -loop 1 -i Image.png -i Audio.wav -shortest Video.mp4

(Note: This worked with .png but not .jpg in the original testing). The quality of output can be enhanced with optional parameters.

V. Compressing Large Videos

A. CRF Compression:

Instead of directly setting bitrate, use Constant Rate Factor (CRF) for compression. Start with a higher CRF (lower bitrate, smaller file size) and adjust accordingly:

Code: [Select all] [Expand/Collapse]

1. ffmpeg -i input.mp4 -c:v libx264 -crf 28 -preset medium output.mp4

-c:v libx264: Specifies the x264 video codec.
-crf 28: Sets the CRF value (adjust between 23-28). Lower values mean higher quality.
-preset medium: Controls encoding speed (adjust to slow or fast).

This is an iterative process: Encode, check the size, adjust CRF, and repeat until satisfied.

B. Alternative Compression using libx265:

Another option, suggested here, may also offer potentially better compression:

Code: [Select all] [Expand/Collapse]

1. ffmpeg -i input.mp4 -vcodec libx265 -crf 28 output.mp4

This uses the libx265 codec. You may also consider specifying a libx265 preset for further optimization.

VI. Recommendations

• Begin with CRF-based compression, adjusting CRF to balance quality and file size.
• Use bitrate-based commands (see here) only if precise file size control is crucial.
• Employ two-pass encoding for optimal compression if time permits.

This comprehensive guide should assist you in effectively using FFmpeg for audio-image video creation and compression. Remember to replace placeholder filenames with your actual file names.

[Eli]

How to Loop a Short Video to Match a Long Audio Track Using FFmpeg on Ubuntu

If you’ve ever needed to overlay a long audio track onto a short video—causing the video to loop until the audio finishes—this guide is for you. Using FFmpeg, a powerful multimedia processing tool, you can achieve this with a single command:

Code: [Select all] [Expand/Collapse]

1. ffmpeg -stream_loop -1 -i input_video.mp4 -i input_audio.mp3 -map 0:v -map 1:a -c:v copy -c:a aac -shortest output_video.mp4

Here is another command that you can use on the Ubuntu command line to overlay an audio file with a video file for the entire duration of the video:

Code: [Select all] [Expand/Collapse]

1. ffmpeg -i input_video.mp4 -i input_audio.mp3 -map 0:v -map 1:a -c:v copy -c:a aac -shortest output_video.mp4