hdffxvrt: open-source tool to convert AVCHD/M2TS video into iMovie-editable QuickTime

Last year I got (occasional) access to a Sony HDR-SR8 camcorder – a lovely if rather somewhat expensive little AVCHD video camcorder… the only problem was that none of the machines I have were beefy enough to edit the material that it generated; I have an old G5 iMac, and a MacBook Air, and a few other crusty machines, but nothing was capable of coping with the format directly.

Ah – I hear you cry – but that’s not how it works in Mac land; there you plug the camcorder into iMovie and it converts the M2TS video into Apple Intermediate Codec so you can edit it!

Yeeeeessss… well that means that if I have 1Gb of video on the camera then I need to plug it into a weedy little MacBook Air (the fastest intel Mac I have got – iMovie’s AVCHD transcoding will not work on the G5) and wait for it to suck 1Gb off the camera and transcode that into TEN GIGABYTES of AIC-encoded .MOV file – a process that takes sodding hours (days?) to complete, not aided by the fact that the Air has a slow disk, a single USB port, and a whole pile of other issues hampering it for this task. The camera has to be plugged in so the transcoder can access the codec specifications, so the laptop is tied to one spot for several days, which is not acceptable.

Plus the Air disk is only 80Gb so that’s 1/8th of my storage we are talking about…

The next few weeks passed…

– I fought with a bunch of “reference” software in C that some Linux folk had lashed together with c-shell (ick!) – and the whole experience was dreadful / evil / painful.

– I bought a license for Voltaic and found it a rather lovely tool, but I could only use it on one of my Macs at a time (else paying $$$$) and it didn’t seem to make much use of the dual-core feature of the Mac.

…and I could not imagine why all my attempts to use ffmpeg to do the conversion directly, were failing with slow-mo video and truncated audio.

Eventually a thread somewhere on the Vimeo forums (iirc) gave me the hint that ffmpeg suffered with a problem in AVCHD where decoding it would generate 2 frames for every 1, thereby halving the playback speed; combining with what I had learned from Axel Olmos and the Linuxeers (above) – notwithstanding the truly horrible c-shell script – I threw everything in the trash and started from scratch, writing the tool that I would want to use.

HDFFXVRT is the result:

home page: http://code.google.com/p/hdffxvrt/
script download: http://hdffxvrt.googlecode.com/svn/trunk/hdffxvrt
subversion: svn checkout http://hdffxvrt.googlecode.com/svn/trunk/ hdffxvrt-read-only

The script works in 2 steps:

  • 1) strip the HD audio out to a temporary raw file in /var/tmp
  • 2a) strip the raw HD video out to a pipe …
  • 2b) …which an encoding process re-merges with the raw audio (above) and re-encodes to the target codec

The above uses a pipeline so that multicore machines see benefit; but in truth one core will be maxed-out while the other is mostly idle – decode-to-raw is cheap and fast – so the script is written so that you can do something crazy like:

$ hdffxvrt *.mts & # background process 1
$ hdffxvrt *.mts & # background process 2

…(or in different Terminal windows) and it will do the right thing. The script will not clobber an existing file, and uses directory-lockfiles which should (?) make it safe to use in a networked environment. I have used this script on a my MacBook Air, my G5 iMac, and my dual-core AMD OpenSolaris, to transcode a directory of 15Gb of AVCHD into QuickTime .MOV files, all cores on all machines working simultaneously flat-out and writing to ZFS-NFS filestore shared from the latter… so it’s my own little render farm.

The only dependency is “ffmpeg” (I am using SVN-r15674) with AAC and other goodness built into it; that is left as an exercise for the reader. No other compilation is required.

Caveats: the script is currently hardwired for the HDR-SR8E, which means:

– assumes a 1440×1080 interlaced input
– with rectangular pixels of a 4:3 pixel aspect ratio
– yielding a 16:9 frame (obviously)
– PAL framerate (25fps, scanning seems to happen at 50fps, see comment above)

…but hopefully you will work out what to poke for your own purposes; it is liberally commented.

Examples:

  • Convert MTS files to 720p suitable for iMovie editing:
    hdffxvrt -p edit *.mts
  • Create short ‘video thumbnails’ as an aide-memoire:
    hdffxvrt -p thumb *.mts
  • Convert stuff from another disk or server, put in current directory:
    hdffxvrt -x -p small /somewhere/else/*.mts

Pre-defined target encodings:

  • thumb # 320×180 mpeg4 + aac audio (10 second thumbnail)
  • small # 640×360 mpeg4 + aac audio
  • medium # 1280×720 mpeg4 + aac audio (default)
  • large # 1366×768 mpeg4 + aac audio
  • huge # 1920×1080 mpeg4 + aac audio (very silly)
  • edit-mp4 # 1280×720 mpeg4 keyframes + raw audio
  • edit-mj # 1280×720 mjpeg + raw audio
  • edit # shorthand for ‘edit-mj’

The preset encoding formats labelled “edit” are deemed suitable by me for video editing, the others are for human consumption; the “editing” presets are either MJPEG (very nice) or all-keyframe MPEG4, with raw audio, and are 720p which is good enough given that the camera is only 1080i.

The “huge” preset is there to expand (!) the 1440i into a 1920×1080 aspect ratio at 25p, but I don’t recommend the results unless you have a big-ass machine.

The “thumb” preset generates a tiny 10-second clip from the start of the video, so you can work out what it is.

If you find this posting and use the tool, let me know what you think in the comments below. There is a bug management/submission system on the Google Code page.

Thanks!

2 thoughts on “hdffxvrt: open-source tool to convert AVCHD/M2TS video into iMovie-editable QuickTime

  1. Bryan

    Thank you for this writeup! I recently purchased a Sony SR-12 and am having quite a bit of difficulty editing video from it. I am a network admin and have machines running each of the following OS’s at home: Ubuntu 8.10, Dual G5 running OS X 10.4, Windows XP, and a test machine that dual boots into vista (the devil) and windows 7. So far I haven’t had any luck with any of them. I ran the Axel mt2toavi on my linux machine and get the “./xporthdmv
    ldecod: Command not found” and have not found a solution anywhere. Being as I do not currently own an intel mac it limits me to what I can use with that. Being new to AppleScript, how would one go about implementing your tutorial? Thank you in advance! I have 26 days to decide if I should return the camera or not and I love how it shoots, but need to be able to edit it :)

    – Bryan

    Reply
  2. Pingback: iMovie

Leave a Reply