Blog

Following-on from my earlier post, Mobile Video Editing Hardware: Thoughts, Ideas & Dreams, where I considered an eventual migration from my laptop to a luggable PC, my thoughts veered (possibly having spotted cash-icebergs among them) towards an alternative solution:

• Use the laptop for lightweight editing & compositing.
• Use the desktop as a number-crunching RADI-attached server.

The two could be linked by:

• Remote access / remote sessions (some of which via smartphone)
• DropBox, e.g. have an active folder where I can drop Adobe Premiere XML and have it processed remotely by Adobe apps installed there.

Some links:

• http://www.theregister.co.uk/2012/01/11/virtualisation_at_home_part_2/
  • (There’s no equivalent “_part_1″ page.  I guess it’s just “Part 2″ of that guy’s story).
  • DIY virtual machines: Rigging up at home, by Trevor Pott, 11th January 2012 14:33 GMT
  • Personal Virtual Machine (PVM) (in use) for about seven years with retail boxed version of Windows XP.
  • VM has been moved from virtualization platform to virtualization platform over the years … the most recent incarnation … inside Hyper-V.
  • …nothing beats Windows Server 2008 R2. It comes with a top-notch virtualisation platform (Hyper-V), and added RemoteFX support with Service Pack 1. You can still use the desktop operating system for all your HTPC needs, and a single Server 2008 R2 Standard license allows you to run both a host copy and a single virtual instance of Server 2008 R2.
  • In my case, the host instance does little more than play movies on the projector via VLC. The virtual instance of Server runs my Plex media server, and aggregates my many storage devices into a single share using DFS.
    • Presumably “DFS” refers to Microsoft’s Distributed File System.
• Shuttle Inc (Taiwan)
  • (MBR’s suggestion)
  • http://uk.shuttle.com/products/productsDetail?productId=1477
    • Small, quiet, hardware-configurable systems.
    • May be a sensible basis for a modular luggable?
  • http://www.okobe.co.uk/ws/product/Shuttle+SH67H3+Barebones+Personal+Computer+FH67+Socket+1155+Intel+H67+Black/1000065737?ref=1q4bs
    • Shuttle SH67H3 Barebones Personal Computer (FH67 Socket 1155 Intel H67) (Black)

Want a mobile “suitcase” editing system, something more (and more expandable) than a laptop but not too expensive.  Primarily to be used for Adobe CS5.5 for media enhancement / editing / compositing etc.

Nearest I found was NextDimension’s range around $7000 I think (but just guesswork - could be way off - would need to get a quote).   That would (if true) be around £4500 at current rates.  Plus import…  NextDimension call such machines “flextops” (Maybe they coined the term? Google searches on it mostly come up with them.)

Apart from the (mil/broadcast-lite but me-heavy) price, it might possibly be undesirably heavy to lug around much.   If so (just guessing, not assuming), it would make more sense to go for a modular quick-setup system.  So, starting to “think different” in this direction:

• Standard tower, capable of taking new CUDA etc. graphics cards etc. as they emerge, but no need for more than say a couple of disks, maybe if SSD could even get away with just a single disk? (For system and media - inadvisable for traditional disks of course, what about for SSD’s?  I have much to learn about SSD’s though).
• “Laptop-Lite” to talk to it.  With robust shuttered-stereoscopic HD monitor.
• Gigabit network to NAS fast storage (SSD and/or RAID ?).

Maybe in that case it would be far more logical/affordable to use an existing laptop as a client working together with a luggable tower server, sufficiently light and robust for frequent dis/re -connection and travel.  And remote access of course (no heavy data to be exchanged, assume that’s already sync’d).  And some means to easily swap/sync applications and projects (data) between laptop and tower, giving the option to use just the (old) laptop on its own if needed.  All such options are handy for the travelling dude (working on train, social visits etc.) who also occasionally has to do heavy processing.  Then would just need a protective suitcase for the tower, plus another one for a decent monitor for grading etc.

I certainly won’t be spending anything just yet, but it’s good to have at least some kind of “radar”.

(more…)

Video-editing on-the-move (e.g. on a train) using a MacBook Pro (laptop) with Sony Vegas 9 (64-bit) as NLE (under Boot Camp / Windows 7), my practical experience was that a GRaid Mini external drive was far, far better than a 5400 rpm Western Digital “Passport” drive.   Consistent with the dual use of the MacBook, I partitioned the drive for both NFTS (Windows) and HFS+ (Mac OS), 50-50%.  Due to Boot Camp limitations (explained below), up till now I only ever used it “tethered” to its own mains-based power supply.  But now I see it can also be used mobile, powered from the MacBook - something that up till now I could only achieve under Mac OS, not under Windows.

When using Boot Camp / Windows on the MacBook, I initially tried the shirtpocket-sized Passport drive because it was small, light and powered from the laptop’s USB port.  While its data throughput wasn’t too bad, at least for single-channel HD editing (especially when only 1280720), when it came to cuts from one video clip (hence, in my case, video file) to another, there was a frustrating delay every time.

I also have a GRaid Mini drive, but it wasn’t obvious at first how to use it mobile when using Windows (on a MacBook).  That drive consists of two 7200  drives in RAID-0 configuation (striped, giving speed but no redundancy), and appears just like any single drive to the computer (no RAID management etc. needed).  The drive has not only a USB (2) port but also FireWire 800 (FW800) and eSATA ports.  While the latter two options work fine with the MacBook under Mac OS, they don’t work under Boot Camp / Windows.  I have tried many times and trawled many forums, no solution is apparent.  Under Mac OS the eSATA drive would ordinarily plug into an ExpressCard adaptor plugged into the laptop’s ExpressCard slot, but under Boot Camp / Windows, the ExpressCard slot doesn’t work, while for the FW800 port under Boot Camp / Windows, it appears to work at first but eventually crashes as a device when it attempts to communicate data (e.g. when copying files).

When connected only by USB to the MacBook under BootCamp / Windows, the GRaid Mini is not powered from that port, hence up till now I have relied on a mains power supply to that drive.  However, I discovered if, after first connecting by USB, you subsequently connect also by the FW800 lead, then the drive takes power from the FW800 yet communicates data via the USB lead.   Hooray!  I can use it on-the-move then!

The order in which the leads are connected is vital.  If by mistake the FW lead was connected first, then the drive would sense that as the data communications route, and subsequently fail in use.  It is vital that the USB connection is made first.  Likewise, on disconnecting the drive (following “ejection” by the computer’s file-system), disconnect the FW drive first.  The rule is FW lead: connect last disconnect first.

My experience of editing with the GRaid Mini is far more fluid hence more pleasurable and efficient.  Totally worth it.  None of the per-cut delay effects of the 5400 rpm Passport drive.  And now it can be used on-the-move, even with Boot Camp / Windows on a MacBook.  I just wish Apple would fix that Boot Camp isue with FireWire and ExpressCard ports!

Suggestions for mobile (laptop) multicam etc. video editing

* NAS
o Ethernet now is 1Gbps
+ Compares well to FW800’s 0.8Gbps.
+ But is there any lag/latency ?  (significantly beyond USB and FW)
o An advantage of NAS - is operating-system-agnostic
+ Hence can conduct a (greater) production project with all material in one folder structure on one device, and Mac and Windows apps can each access each others’ files with full read/write access, including the ability to add “sidecar” files e.g. for audio waveforms or in-video motion analysis results (as used by steady/stabilize effects).
* Can connect a NAS to a laptop directly via Ethernet Crossover cable.  No need for a router.
o Google: [ethernet crossover cable].
o It’s a very standard item, e.g. available at PC World
* Exists-there a small mobile NAS?  Preferably FW-powered from the laptop?
o Google:
+ [small nas raid]
+ [mobile nas]

# http://www.pcworld.com/businesscenter/article/169490/three_nas_devices_for_efficient_smalloffice_backup.html
# http://www.smallnetbuilder.com/nas/nas-reviews/31010-lg-n2b1-super-multi-nas-reviewed?showall=&start=2

This is a problem I encountered some time ago, when I was running Boot Camp v3.1 on my MacBook Pro.  Since then I upgraded to v3.2.  I know there’s a v3.3 around but before upgrading I thought it worthwhile to see whether v3.2 had fixed that problem (especially since I couldn’t rule out the possibility of v3.3 reintroducing it).   Only one thing to do: prevaricate test.

• Copy file from GRaid Mini (GRm) to Desktop:
  • 2GB fine
  • 12GB appears ok initially but then fails (to zero b/s transfer rate, then the Grm device “no longer exists”, at least until reboot)
• Reverse: 2GB fails (same way) almost immediately.

OK not good thus far…

Next tried an alternative approach: run W7 as a Virtual Machine on Mac Os via Parallel.  I have Parallels v6.  Forum search revealed that there is no FW support in either v6 or v7, though the developers seem interested in knowing why people want it.

• 2GB GRm to W7 Desktop: ok
• The reverse: ok.

Had to stop there due to other work - and a very full W7 disk.

The next workaround to consider is attaching a NAS.  Ethernet bandwidths can be 1Gbps, hence more than FW800’s 0.8 Gbps, though I wonder if there could be any issues of lag / latency in this approach.  I’ll do some research and put up another post about this idea.

• http://filmmakeriq.com/2010/07/22-filmmaking-apps-for-the-ipad-iphone/
  • 22 Filmmaking Apps for the iPad & iPhone
• http://appfinder.lisisoft.com/tag/final-cut.html
  • All Final Cut Apps for iPad-iPhone
    • (Is not only relevant to Final Cut)
• http://www.chrisjonesblog.com/2011/05/why-every-film-maker-should-have-an-iphone.html
  • Why every film maker should have an iPhone
• http://www.chrisjonesblog.com/2011/05/why-every-film-maker-should-have-an-iphone-part-2.html
  • Why every film maker should have an iPhone: Part 2
• http://www.theblackandblue.com/2010/03/22/toolkit-useful-iphone-apps-to-have-on-set/
  • Useful Cinematography iPhone Apps to Have On a Film Set
• http://www.theblackandblue.com/2010/07/08/toolkit-useful-cinematography-iphone-apps-to-have-on-set-part-2/
  • Useful Cinematography iPhone Apps to Have on Set (Part 2)
• http://www.videomaker.com/community/videonews/tag/iphone/ (December 2010)
  • All manner of software and hardware, including the OWLE bubo (steady mount for iPhone, has wide-angle lens)

Which is best?   Depends on the camera, scene and shot dynamics I guess.  The same point is queried at the following thread:

• http://www.dvinfo.net/forum/sony-xdcam-ex-cinealta/238593-steady-shot-smoothcam-both-what-2.html#post1551809

Some general advice from a computer-post-savvy author: definitely use the camera’s SteadyShot:

• http://www.gcvcc.com/March%202008%20%20Camera%20Techniques%20No2.pdf

Limitations of post

• Stabilization necessitates motion estimation and image reconstruction, which are extremely CPU-heavy, hence really slow to execute.
• Most stabilization apps (in post) can’t currently cope with motion-blurred edges or parallax effects (though both should be possible in principle, by deconvolution and 3D modelling both informed from multiple frames).
• For rolling-shutter-ed footage (e.g. CMOS sensors as in Sony Exmor as in Sony XDCAM-EX e.g. EX1 & EX3), there exist options to reduce the effect (don’t expect perfection, but may suffice):
  • Sony’s ClipBrowser 2.6 can remove this http://www.dvinfo.net/forum/sony-xdcam-ex-cinealta/467511-clipbrowser-takes-out-rolling-shutter.html.
  • Gunnar Thalin’s Deshaker plugin for VirtualDub can compensate for this.

My experiences:

• Stabilizing Tools:
  • Gunnar Thalin’s Deshaker works really well.  And it is multi-threaded, really speeds up the process.  The author says it is more intended for handheld pans etc. than fast-shaking shots from vehicles etc. (but has nevertheless seen good results in such situations).
    • The author says [http://forum.doom9.org/showthread.php?p=1418923] to try “to stabilize only on the most distant parts in the frames, since the moving inwards-effect is less there”.  And “you should probably increase the value for [discard motion of blocks that move > X pixels in wrong direction]. That’s to allow the blocks to move “freely” a little, since Deshaker can’t handle the “moving inwards”-effect.
    • Possibly equally applicable to other smooth/stabilize/deshake tools ?
  • Boris’s Optical/Motion Stabilizer (in Boris Red 4.3.3 on XP) is only single-threaded and I find it slower, clunkier and less intuitive than Deshaker.  Has a Smooth mode, which is like the others here, as well as a Stabilize mode (try to keep frame static, no good for motion then).  The other tools can be configured to do the same thing.
  • Mercalli in Sony Vegas has no mode for 720p50 but otherwise is pretty good and very intuitive and configurable.
  • FCP’s SmoothCam Effect worked best for a challenging clip for wobbly-hand-held camera tracking close past an object (a Formula-1 car) hence huge degree of moving-inwards effect.  The default settings worked straight away.  The result quality was way above that of the other tools.  On the other hand sometimes it’s not the best (sorry, forgot the exact situation).
• Cameras & Shots:
  • Historically, using a TRV33 DV HandyCam indoors (hence low-light hence long shutter time):
    • Way back in the past, using a (now ancient) TRV33 DV handy-cam (which has huge sensor margin i.e. spare pixels), when I shot big zooms to lecture audience individuals (e.g. question-time) I had the camera’s steady-shot (digital, not mirror) enabled  and used Gunnar Thalin’s Deshaker (VirtualDub plugin) also.  The result was astoundingly steady.
    • The same arrangement worked OK with hand or shoulder mounted cam for walk-throughs past nearby objects (e.g. walls, people, furniture).
    • An attempt to do the same thing without steady-shot enabled on the camera resulted in seriously motion-blurred edges.
  • Now, using a Sony EX3:
    • With camera Steady-Shot set to Medium, hand-held pans and motion past nearby objects seem to acquire a positional instability, as if the camera feedback mechanism needs greater damping. Maybe the camera’s internal mirror “suspension” has to be tighter (than the TRV33 digital equivalent) because it lacks the generous pixels margin of the TRV33?  or maybe something to do with the mirror’s inertia?  Or (real-time-constrained) processing-power?
      • Experimentation is needed with the camera’s other SteadyShot modes (High, Low).
      • In the absence of more generous sensor pixel margins, I wish it could be loosened-up e.g. to allow black borders (to crop in post) so as to permit smoother rides overall.

Problem:

• In Windows I export from Sony Vegas to AVI (CineForm).  In OS X I read the file into FCP and apply the SmoothCam effect, then export to ProRes.  In Windows, Sony Vegas, I replace the original file with the smoothed one.  The levels/gamma are wrong.

Solution (Search):

• Sony Vegas forum http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=718371
  • Use DNxHD
    •  Couple of tips re DNxHD:  709 color level assumes 16-235, and RGB assumes 0-255.
  • Force it back again: www.kenstone.net/fcp_homepage/gamma_mac_pc.html
    • But this presumably implies getting re-quantized twice (the roundtrip issue and the forcing), which for 8-bit footage I imagine could reduce the quality (banding).
• Uncertainties
  • Where and how does this gamma get applied?  In FCP I didn’t (knowingly) alter the levels (eg until it looked right), I just applied the SmoothCam filter.  So I guess it would look wrong on the (pre-SnowLeopard) Mac but I wouldn’t care.  Wouldn’t FCP then export back whatever it got but smoothed?  This one is really confusing.    Experiments needed (when I get time…) I guess.

Reading book “Final Cut Pro Workflows” by Osder & Carman, 2008.  On page 284 it relays advice that it is best to put Project Files [.fcp] on a separate drive to the Media Drive (e.g. Media Drive= XSAN), due to:

• Safety - not all on one drive
• Avoid fragmenting the media drive (project files, cache and to a lesser extent render files) are written often (transient files?)

I’m not immediately convinced by these arguments:

• Media drive should be backed-up in any case.  I use (slower) USB drives and file-synch.
• Fragmentation may be an old-school issue, less of an issue on Mac’s HFS +.
  • Example: article at [http://www.macmousecalls.com/files/1f6efbb25f58d77390499af1f984d2c7-33.html] linking-on to [http://www.kernelthread.com/mac/apme/fragmentation]
  • Example: Larry Jordan, “Final Cut Pro Power Skills” 2010, page 34, Chapter 2, “Defragmenting Your Hard Drive”: Once popular, this technique is no longer recommended.

How to view degree of fragmentation on an HFS volume:

• [http://osxbook.com/book/bonus/chapter12/hfsdebug/fragmentation.html]
  • Command-line app to report a variety of storage-volume statistics, including fragmentation.
  • After download, can check the sha1 checksum, but this is of the executable, not the download itself ([.dmg] file).  The ’sha1′ command is inbuilt to Mac OS, as: [/usr/bin/openssl sha1].  Note the last character of ‘openssl’ is a small ‘L’ niot a ‘1′.

• General
  • Show advanced tools [No] -> Yes
• Video
  • Import HD video as: [Large 960×540] -> Full - Original Size

No way I can see to define the iMovie work-area folder, where media gets imported to for example.  Maybe there’s no choice over this, which would be a shame (don’t want movies filling up my system drive).  In my case the iMovie work-area was:

• /Users/davidesp/Movies
  • iMovie Events.localized/
    • New Event/
      • iMovie Data
      • iMovie Cache/
      • iMovie Thumbnails/
      • <a source media file>
  • iMovie Projects/
    • My First Project.rcproject