A video artist approached me with the following problem: she wanted to play back a recorded performance art piece in "double-wide" format, i.e. two 16:9 videos next to each other.
The video was recorded with a 4k camera (3840x2160 resolution) and then edited and cropped to produce a 4k video file, in which only the "middle vertical slice" contains information (letterbox). After discussing several options, such as using a single 4k projector with a cropped image, we settled on using two FullHD projectors next to each other. Financial considerations played a significant role as well.
Playing back a 32:9 format video on 2 projectors was deemed as too fickle / difficult without using specialized hardware (video wall controllers) or software (e.g. Multi-View HD Pro 2. 2 older mac minis that were lying around seemd good candidates for the job.
The tasks at hand were roughly the following:
- Cropping Videos with FFMPEG: Produce 2 videos from the source material for both the left and the right projector. Each video would have a FullHD resolution (
1920x1080), thus covering exactly the "payload" of the source material. - Synchronized Playback with Mplayer Find a way to play back both video halves in a (sufficiently) synchronized manner on the 2 mac minis.
- Make the setup robust and suitable for an installation in a museum - i.e. easily operable by non-technical people with minimal effort and with an automated restart / loop.
- Additional wish from the artist: have the videos freeze/loop on the first frame for a couple of seconds for improved daramturgy.
These are the areas of interest in the 4k video source material:
┌───────────────────────-───────────────────────┐
│ │
│ X X X │
│ │
├───────────────────────┼───────────────────────┼
│ │ │
│ │ │
│ │ │
│ Left Video │ Right Video │
│ │ │
│ │ │
│ │ │
┼───────────────────────┼───────────────────────┼
│ │
│ X X X │
│ │
└───────────────────────-───────────────────────┘
Original: Resolution 3840x2160
Left Video: Resolution 1920x1080 Origin (x,y) = 0,540
Right Video: Resolution 1920x1080 Origin (x,y) = 1920,540
While the video could be simply cropped during playback (e.g. with mplayer), this requires a certain amount of processing power which the older mac minis (2012-ish) that were available could not necessarily provide. Therefore, it seemed easier to 'cut up' the videos.
FFMPEG is a fantastic tool to edit videos via the command line. On macos, it can easily be installed via brew: brew install ffmpeg. A file can be sent to a series of filters, one of which is the crop filter. These are the commmands I used:
# NOTE: Crop filter takes width:height:offset_x:offset_y ('ih' is input height, etc.)
# -crf is the "Constant Rate Factor", which for x264 has a range of
# 0-51 (0=lossless, 23=default, 51=worst possible)
ffmpeg -i original_4k.mp4 -filter:v "crop=iw/2:ih/2:0:ih/4" -c:v libx264 -crf 12 -c:a copy left.mp4
ffmpeg -i original_4k.mp4 -filter:v "crop=iw/2:ih/2:iw/2:ih/4" -c:v libx264 -crf 12 -c:a copy right.mp4The idea came up to loop the first frame of each video for 5 seconds (without any audio). This took a while and quite a bit of StackOverflowing. Particularly inserting the loop while keeping the audio in sync was not trivial. In the end, I did this by extracting the first frame, creating a videp by looping and the concatenating it with the original.
# extract first frame of a video into an image
ffmpeg -i left.mp4 -frames:v 1 firstFrameL.png
# Loop 1 image for 5s, with blank audio (adjust framerate to 29.97)
# NOTE: video timescale of 30k was set to same as source video
ffmpeg -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=48000 -loop 1 -i firstFrameL.png -r 29.97 \
-c:v libx264 -t 5 -pix_fmt yuv420p -video_track_timescale 30k -vf scale=1920:1080 freezeL.mp4
# Concatenate 2 videos without re-encoding (map 0 transfers video and audio streams 1:1)
ffmpeg -f concat -i concatL.txt -c copy -map 0 left_withFrozenStart.mp4
# concatL.txt contains:
file 'freezeL.mp4'
file 'left.mp4'
# ... + exactly same for the right side.Both HD videos would playing on one mac mini each. Mplayer can be fully controlled via command line and comes with built-in synchronized playback (via UDP) and was thus the first choice (VLC was an alternative).
The toughest nut to crack turned out to the looping (-loop 0 gets mplayer to loop indefinetely). I connected the mac minis to a switch via Ethernet, to reduce the amount of "foreign" traffic. This helped quite a bit in terms of jitter reduction, but the slave/secondary player would be stuck for the about 20 seconds at the beginning of each loop, and would then sync and resume playback correctly.
I tried many kinds of configurations and settings, even a slightly hacky (or elegant?) method of continuously querying mplayer for the playback position and then restarting playback when the playback nears its end.
In the end I turned to an 'out of the box' solution: I completely forewent mplayer's built-in looping capabilities. Instead, I handled looping on the outside, in a bash script. This resulted in the macos desktop being shown for a couple of seconds, which was deemed acceptable (wallpaper set to black), since the loop length was large enough (~10 minutes)
The mac minis were configured to restart of power loss and the scripts called from .command files and added as startup items.
These are the two (almost identical) scripts, which sync via the broadcast address .255.
#!/bin/bash
while true; do
printf "\n\nPlaying video in loop\nPress \'Q\' or ESC key twice to stop\n\n"
mplayer -udp-master -udp-ip 192.168.0.255 -fixed-vo -fs videoLeft.mp4
read -t 3 -n 1 k <&1 # -t timeouts
if [[ $k = q ]] || [[ $k = $'\e' ]] ; then
break
fi
printf "Restarting video...\n"
done#!/bin/bash
while true; do
printf "\n\nPlaying video in loop\nPress \'Q\' or ESC key twice to stop\n\n"
mplayer -udp-slave -udp-ip 192.168.0.255 -fixed-vo -fs videoRight.mp4
read -t 3 -n 1 k <&1 # -t timeouts
if [[ $k = q ]] || [[ $k = $'\e' ]] ; then
break
fi
printf "Restarting video...\n"
done┌───────────┐ ┌────────────────┐
│ │ │ master │
│projector 1│◄──────┼ ├────────────┐
│ │ │ 192.168.0.33 │ │
└───────────┘ └────────────────┘ │
│
┌────┴─────┐
│ switch │
└────┬─────┘
┌───────────┐ ┌────────────────┐ │
│ │ │ slave │ │
│projector 2│◄──────┼ ┼────────────┘
│ │ │ 192.168.0.66 │
└───────────┘ └────────────────┘