2016_Making a side-by-side 3D video out of blu-ray

I have a 3D blu-ray disc of a movie (3DBD). I want to convert side-by-side 3D video out of it, so that I can watch it on my mobile phone in the VR BOX virtual reality headset.

The First Try

The first software program that I found via the cn.bing.com search engine is called 3DBDNet (which actually 3DBD.net). It is found via the link below:

3DBD.net (later commercially 3DBD Buster, but unmaintained as of year 2016), a software program that converts a 3D blu-ray disc into a side-by-side video file

I found a guide about it from the Web. Following the instructions, I opened an SSIF file, which is the largest on the blu-ray disc (the file is about 42 GB, taking up most of the 40 GB BD space).

Unfortunately, although 3DBD.net can finish demuxing, but it can't encode the right view, rendering this one-stop approach invalid.

The Second Try

Next I did another search, and majorly referred to page http://forum.doom9.org/archive/index.php/t-155246.html (or its forum view: http://forum.doom9.org/showthread.php?t=155246; title: Guide to convert BD 3D to 3D Left+Right Stereoscopic and Anaglyph). Although there are quite a few commercial integrated solutions (such as DVDFab, EASEFab), I would like to have a try with the step-by-step self-service using only free tools, and meanwhile it can help me better understand the working mechanisms.

Freeware/ Open source software tool chain

Then I dug deeper by following the detailed manual steps mentioned in the DOOM9 forum post. The first step is to use eac3to to demux the streams. The eac3to version I use is copied from a sub-directory of 3DBD.net. As I found out, the largest SSIF file is named 00800.ssif. eac3to requires the play list file name, so I chose the one with the most similar name: 00800.mpls. The following command lists all the streams inside.

eac3to D:\BDMV\PLAYLIST\00800.mpls

1) 00800.mpls, 00800.m2ts, 2:05:43

- Chapters, 18 chapters

- h264/AVC (left eye), 1080p24 /1.001 (16:9)

- h264/AVC (right eye), 1080p24 /1.001 (16:9)

- TrueHD, English, multi-channel, 48kHz

- AC3, French, multi-channel, 48kHz

- AC3, Spanish, multi-channel, 48kHz

- AC3, Portuguese, multi-channel, 48kHz

- AC3, English, multi-channel, 48kHz

eac3to D:\BDMV\PLAYLIST\00800.mpls -demux

After the disc/disk working hard, I got a quite a few streams. One left eye video stream of 21.8 GB, with a file name like 00800*left*.h264, one right video stream of around 9 GB, with a file name like 00800*right*.h264, and several audio streams (as listed above) plus quite a few subtitle streams (*.sup files).

The next step is to encode the video. The DOOM9 post mentioned using an AviSynth script (*.avs file). However, it depends on a version of H264StereoSource.dll in the attachment of that post, but I couldn't find out how to download the attachment. I tried a version of H264StereoSource.dll included in the multiAVCHD tool mentioned in the post, but couldn't get it to work. I also saw that in SSIFsucka there is another dll it uses in an *.avs file, which is called glPipeSource.dll, but after I tried mimicking the AVS script, that doesn't bring me any luck, either.

Fortunately, I downloaded FRIM Decoder. I first tried to download the MVC Decoder tool mentioned, but I couldn't find it online. So I downloaded FRIM Decoder. But unluckily, I used it in a wrong way at first. I tried making it writing to a pipe, namely \\.\pipe\test_dec1.yuv, and let ffmpeg read and encode that file. I used the following commands and I successfully get the left eye view.

FRIMDecode -i:h264 00800_left.h264 -o \\.\pipe\test_dec1.yuv

ffmpeg.exe -f rawvideo -s 1920x1080 -r 23.976 -pix_fmt yuv420p -i \\.\pipe\test_dec1.yuv -vcodec libx264 -b 2500K -an -y frim_right.avi

However when I used the same way to get the right view, when I play it in the video player, it's mostly green, with something unclear moving on the screen.

After I failed with those tries, quite a few days passed. The decoding/encoding was quite slow as the video resolution is so high. At first, I didn't realize that the right stream is not a plain H.264 stream. After I see the right view in green, I began to realize that there must be some special encoding mechanism to merge the "small" right view into the left view to make a complete right view. It also explains why the right view file is so much smaller than the left view. I looked up this in wikipedia, and it tells me about Multiview Video Coding.

So now I do have a MVC decoder--FRIM Decoder. It is based on Intel SDK. But how do I utilize it? I ran command "frimdecode", and I got its simple explanations:

FRIM Decoder version 1.26 (build: Jan 16 2016)

-i:mpeg2|h264|mvc|vc1|jpeg InputBitstream [InputBitstream_dependent]

-o OutputFile [OutputFile_R] [options]

FRIMDecode -i:h264 00800_left.h264 00800_right.h264 -o \\.\pipe\test_dec1.yuv -sbs

ffmpeg.exe -f rawvideo -s 3840x1080 -r 23.976 -pix_fmt yuv420p -i \\.\pipe\test_dec1.yuv -vcodec libx264 -b 5000K -vf scale=1920:-1 -an -y frimtest_sbs.avi

But the output is not correct. It always only contained the left view. I did more Web search, and read more forum posts. I finally realized that I wasn't using the correct switch for FRIM Decoder. Finally I dug into the "FRIM_x86_version_1.26\doc" directory, and read the document FRIMDecode_readme.pdf. I found the correct parameters to use to decode the right view. For safety, I don't want to waste another a few days to investigate how to use frimdecode to make a side-by-side output, so I chose to generate the right view only. This time I'm more confident, as I know the "small" right view is in MVC format.

FRIMDecode -i:mvc 00800_left.h264 00800_right.h264 -o \\.\nul \\.\pipe\test_dec2.yuv

ffmpeg.exe -f rawvideo -s 1920x1080 -r 23.976 -pix_fmt yuv420p -i \\.\pipe\test_dec2.yuv -vcodec libx264 -b:v 10000K -an -y frim_right.avi

And then I got a correct right view. By the way, my choice of bitrate: 2500 kbps for 960x540, so 10000 kbps for 1920x1080.

Then I get the left view by using the "copy" codec, which is much faster and lossless. There is no need to specify a screen size for a *.h264 source file because ffmpeg will determine automatically and it will fail if you do so.

ffmpeg.exe -f h264 -r 23.976 -i 00800_left.h264 -vcodec copy -an -y frim_left.avi

The next step is to merge the left view with the right view. A AviSynth script could be a way. Many video tools, including ffmpeg and x264, have built-in AviSynth support. However, to me, it is an overkill because I only want to lay the videos side-by-side, which is a simple adjustment. Instead of using an AviSynth script, I did some Web search, and finally got this command line:

ffmpeg.exe -f h264 -r 23.976 -i 00800_left.h264 -i frim_right.avi -filter_complex "[0:v] scale=iw/2:ih/2, pad=2*iw:ih [left]; [1:v] scale=iw/2:ih/2 [right]; [left][right] overlay=main_w/2:0 [out]" -vcodec libx264 -b:v 5000k -an -y -map "[out]" test_sbs.mp4

However, I tried, and ffmpeg reports that it doesn't support this command line. Later I realized: the ffmpeg version that I used earlier was the version with 3DBD.net, which is a 2009 version and is too old for my task. So I tried a newer version, built in year 2013 and bundled with FormatFactory. It worked.

The output video seemed working, but later I found that, to my surprise, the right half is always 2 frames ahead of the left half. So I need to find out a way to adjust it. I tried this:

ffmpeg.exe -i frim_left.avi -ss 0.083416666 -i frim_right.avi -filter_complex "[0:v] scale=iw/2:ih/2, pad=2*iw:ih [left]; [1:v] scale=iw/2:ih/2 [right]; [left][right] overlay=main_w/2:0 [out]" -vcodec libx264 -b:v 5000k -an -y -map "[out]" test_sbs.mp4

The value 0.083416666 was chosen because it is the result of 1.001 / 24 * 2. I first tried encoding only 20 seconds of the video (starting from a certain position, in which there are shot transitions, and it seemed all right, so I went for a full version. After the encoding (another 2 days passed), I tried the video. Now the left frame is one frame ahead of the right. I don't know why this would happen, but again, I tried going back a little, by half a frame:

ffmpeg.exe -i frim_left.avi -ss 0.0625625 -i frim_right.avi -filter_complex "[0:v] scale=iw/2:ih/2, pad=2*iw:ih [left]; [1:v] scale=iw/2:ih/2 [right]; [left][right] overlay=main_w/2:0 [out]" -vcodec libx264 -b:v 5000k -an -y -map "[out]" test_sbs.mp4

I was not 100% certain that it will get the correct result, but after I got the result, it was perfect! Left and right frames are in sync, from the very beginning to the end. Yay!

The remaining parts: audio and subtitle

Now I need to mux the audio into it. I picked up the English AC3 track, and did it. However, after I got the final video, I found that the audio track is for visually impaired people. I like the accessibility consideration from the movie makers, but I don't need it for now. So I had to turn to the TrueHD track, which I see the first time. I searched about it on the Web, and found that it is "3D audio". Maybe it's ready for VR headsets or something, but I think for conventional home audio system, it's not that useful. Anyway, let's go:

ffmpeg.exe -i test_sbs.mp4 -i 00800_4_eng_thd.thd+ac3 -vcodec copy -acodec aac -af "aresample=matrix_encoding=dplii" -ac 2 -b:a 192K -y sbs_w_audio.mp4

Well, failed. I found that it's due to that the 2013 version of ffmpeg is still not new enough. I then downloaded a 2016 version. Note that for Windows, a binary compilation is not released on the official website, so I have to download it from a third-party website.

But ffmpeg now tells me that it can't process the TrueHD file. It deems the file unrecognizable. I searched the Web again, and tried using eac3to to convert it into a *.wav file, and eac3to can't recognize it, either. Even the delaycut software mentioned by eac3to that might recognize the file cannot understand the format.

Fortunately I have another source (secret) of the audio: another *.mkv version of the movie. Then I used the following command:

ffmpeg.exe -i Movie_720p.x264.mkv -map 0:a:0 -acodec flac -ac 2 Movie_720p.flac

ffmpeg.exe -i test_sbs.mp4 -i Movie_720p.flac -vcodec copy -acodec aac -ac 2 -b:a 192K -y sbs_w_audio.mp4

Hint: H.264 is one MPEG4-compliant encoding standard that most modern video players support (at least since year 2012 Surface RT). It provides much higher video quality than the MPEG2 standard used in DVDs. The quality is also much higher than the MPEG1 standard used in VCDs, and higher than the DivX MPEG4 encoder that appeared earlier than H.264.

Hint: AAC, also not dominant when used standalone, is the default audio format for most H.264 MPEG4 videos. Compared to MP3, it has higher audio quality with the same size. 192 kbps provides "transparent" audio compression for most users.

For the subtitle, I found *.sup files on the blu-ray disc. I used a very good tool, SubtitleEdit, that can convert it into a *.srt file. Note that *.sup file contains graphics (not sure if it is vector graphics or bitmaps) that represent lines of the subtitles, so SubtitleEdit will call an OCR (Optical Character Recognition) library trying to convert the characters into plain text, which is the need of the *.srt format. During conversion, there are some minor errors (but not blocking). I can manually correct them by using the spell checker and also by reviewing the subtitles against the movie.

Afterword