Archive for the ‘collaboration’ Category

Shared Storage Options for Windows & Mac Video Editing Collaboration

Friday, October 18th, 2013

In summary:

There’s no magic option, each workstation needs a local storage volume with block-level data access (as opposed to simply file-level access) and formatted to a file system that is native (doesn’t require translation) to that workstation’s operating system.  Migration and collaboration imply file copying/synchronization, which implies read-access to the “foreign” file-system.  Mac OS can read NTFS, Winows can only read HFS+ via third-party add-on utilities.  Furthermore, for speed and responsiveness appropriate to video editing, the local storage should ideally be RAID or SSD.  In either case, it is possible to split the local storage (e.g. via partitioning) into more than one file-system.  At least, that worked on the mutiple occasions I have taken that approach, and have not been aware of any issues.

In greater detail:

Consider the challenge of setting up a shared data storage volume (e.g. RAID array or SSD) for video editing, such that either Windows or Mac computers can connect to it, and a video project started on (and saved to) on one of those operating systems (OS) can be continued on the other (and vice versa).

My current solution is to split the drive into separate volumes, one for each OS.  For example I have done this on RAIDs of various kinds and on an internal drive for Mac systems bootable to either Mac OS or (via Boot Camp) to Windows.  In the case of RAIDs I was advised against this by my system supplier, but got the impression they were just being defensive, not knowing of any definite issues, and to my knowledge I did not experience any issues.

It is is not practical to have just one volume (necessarily in that case, one file-system format), because:

  • Mac OS on its own is able to read NTFS but cannot write to it.
    • This is a show-stopper.  Some of the major video editing applications (e.g. NLEs), slightly disturbingly, may use (or for some functionality, even depend on) read/write access to source-files and the folders containing them.
      • I initially, naively, imagined that video editing systems etc. would only ever read source media files, not write to them, or to the folders containing them.  However that proved very naive indeed…
        • In Apple/Mac’s (erstwhile) Final Cut Pro 7 I regularly used their (moving) image stabilization effect, SmoothCam.  Its analysis phased was typically slow and heavy – not something one would wish to repeat.  The result was a “sidecar” file of similar forename to the analyzed source file, but a different extension, placed in the same folder as the source file.
        • I’m not certain, but got the feeling that maybe the source file (or folder) meta data, such as permissions or somekind of interpretation-change to media files in the quicktime ([.mov] mmedia format.
      • Certainly, Adobe (on Windows and Mac) could adulterate both files (by appending XMP data – being an Adobe media metadata dialect in XML) and the folders they occurred in (depending on uder-configuration) in terms of sidecar-files.
      • Sony Vegas also generates sidecar-files, e.g. for audio peaks.
  • File system translation add-ons can add Windows read/write access to HFS+ (ordinarily it could not even read it) and add Mac OS write access to NTFS (ordinarily it could only read it), but not sufficiently transparent/seamless for big real-time data access as required for demanding video editing endeavours.
    • File system translation add-ons (to operating systems) exist, such as MacDrive, to allow Windows to read/write Mac OS, or Tuxera NTFS, Paragon NTFS or Parallels for Mac to enable it to read/write NTFS, but these (reportedly, and in part of my experience) only really work well for standard “Office” type applications, not so well for heavy (big andd real-time) data applications such as video editing, where they can impede the data throughput.  Doh!
    • Some people have experienced obscure issues of application functionality, beyond data-movement speed issues.
    • {Also, I am concerned over the (unknown/imagined/potential) risk that the “alien” operating system and/or its translation utility might alter the file system in some way that upsets its appearance to the “home” operating system.}
  • FAT is universal but is a riskier option:
    • FAT is un-journaled, hence risks loss not only of individual files but of whole volume (integrity).
      • In video editing, corruption could be disastrous to a project, not only in terms of possible data-loss or time wasting and project delays on data recovery, but also in terms of “weird” effects during editing, such as poor responsiveness to commands, whose cause the user may not appreciate. or even an increased risk of unacceptable flaws in the final product.
    • FAT32 is essentially obsolete, because its maximum file size is (1 bit under) 4GB.
    • exFAT, a kind of “FAT64” is practical, and indeed a big successful corporate Mac-based production company once supplied me with many GB of footage on an exFAT-formatted external disk.
      • The largest file I have so far stored there is 40GB.  No problems.
  • NAS (Network-Attached Storage) sounds at first an easy option, but in my experience they impede big real-time data throughput (as stated earlier for “file system tyranslation” add-ons). Double-Doh!
    • Such devices only permit file-level access.  Consequently, the client systems can e.g. create or retrieve folders and files, but cannot e.g. format the device or address it in terms of lower-level data structures.
    • A likely explanation for the “impedement” of a NAS (to data responsiveness and throughput) is that such devices store in a local format (typically they run linux) that is invisible to the client, then translate to an appropriate protocol for each operating system accessing it.  They normally incorporate a bunch of such protocols.  As always, translation => overhead.
    • Other options, such as SAN and iSCSI, instead of providing file-level access to the client systems, instead offer the lower level of data block access.  Thus they appear to the client system as would any local storage device, and can be formatted as appropriate to the client system.
  • One suggestion I saw was to use a Seagate GoFlex drive, which can be used (read/write) with both Mac and Windows.  But the supplier’s FAQ (about that drive) indicates that it depends upon a translator utility for the Mac:
    •  If you would like to be able to “shuttle” data back and forth between a Mac and a PC, a special driver needs to be installed onto the Mac that allows it to access a Windows-formatted drive (i.e. NTFS). Time Machine will not work in this case, nor will Memeo Premium software for Mac. However, if you want your GoFlex solution to also work with TimeMachine, the drive will need to be reformatted to HFS+ journaled.

So I guess there is no “magic storage” option, my main work setup will have to remain based on separate volumes for each OS.

When transferring an editing project from one OS to another, the following actions will be necessary:

  • Copy any absent or updated files across.
    • e.g. via a file-synch utility such as Syncovery.
  • Allow time etc. for possible file re-linking, re-indexing, re-preview generation, re-“SmoothCam” (or equivalent).
    • This aspect is down to the editing application etc., as opposed to the operating or file systems themselves.
  • Ensure any effects used in the edit are present on both systems.
    • If so then these should presumably still work…

(more…)

NLE Adulteration of Source Media: Potential Workflow-Issues

Friday, September 13th, 2013

I highlighted in http://blog.davidesp.com/archives/598 (10 months ago) that Adobe Premiere etc. can adulterate media files, in terms of metadata and/or sidecar-files (depending on user-configurations of these applications.  I indicated that, regardless of the reasonableness of at least some of these actions, this could potentially cause problems to other applications.

Validating that concern, I note a post (2012-06-12) by Matt Davis on Philip Bloom’s website, stating (my italics):

  • …if sharing assets with FCPX and Adobe Premiere, Adobe ‘touches’ (resets the modification date) of each file without doing anything else to it, but also sprinkles sidecar files into directories of transcodable files for metadata, thus sending any returning FCPX activity into a tailspin, requiring a re-linking session. It’s oddities like these which haunt the implementation of FCPX in a wider system and make system managers wonder if FCPX is actually worth implementing in its current state.

That was over a year ago, and so the issue may or may not exist for the current version of FCPX.

As users, whether or not the actions of one application adhere to standards and another don’t, what we as users ultimately care about is workflow, which in this case translates to “does it connect up with my other tools/processes?”.  So we have to maintain a “situational awareness” of potential interoperability pitfalls.

Incidentally, I recall that FCPX’s predecessor (in history at least, if not development-line) FCP7 could adulterate source directories with its own sidecar files, produced by its SmoothCam effect.  Not knowing anything further for sure, I nevertheless wondered (at that time) what it might be doing “under the hood” of the QuickTime [.mov] wrapper.

Groove Folder Synchronization? What’s that?

Friday, August 23rd, 2013

While recovering and auditing a laptop I came across “Groove Folder Synchronization.  I have vaguely come across its name before, but that’s all.

It’s apparently a dropbox-like thing (loosely-speaking), by Microsoft.

(more…)

Adobe CS Review …No more it seems

Monday, June 4th, 2012

As a relatively new Adobe user, I was vaguely aware of an attractive-sounding Adobe Premiere collaboration feature, I think it was originally called Clip Notes (http://boardreader.com/thread/Clip_notes_alternative_for_CS5_other_tha_1yitjXfs3i.html confirms this), where one could send out reviews to people, who accessed it via Acrobat or as a pdf or something.  Having Adobe Production Premium CS5.5 I explored under Premiere’s File menu, discovering Create New Review.  I wish I had not, for it wasted several hours of scarce production time…  It seems that this feature has been discontinued, as announced at http://www.adobe.com/products/creativesuite/cslive.html and complained about at http://forums.adobe.com/message/4266469.  The only reason I discovered this, following three hours of rendering by the Create New Review command and further one hour waiting for the Share Review website to complete (black screen with rotating wait-animation) was googling for acrobat.com login problems.

How come there wasn’t a simple website message to say “Discontinued”?  Furthermore, why not an Application Update to remove this feature from the File menu or change the menu action to state that this feature was discontinued?   Just as well I had not based a commercial workflow on this feature.  I feel somewhat Apple’d….

My alternative, until I find anything better, will be good-old-fashioned highly compressed renders with burnt-in timecode, shared bia DropBox.   I am also aware of Sorenson 360, it looks like it has a great set of features, but its cost is prohibitive for my current purposes.

One item I did manage to salvage from my “wasted time” was the render – that had taken 2.5 hours – that had been generated as part of the CS Review process.  It appeared in the folder [C:\Users\David\AppData\Local\Temp] with the pseudo-random probably-unique filename of [8D4E4C20-0C00-0F8A-A501-B6B7CA2E4883.f4v]. The [f4v] extension indicates it is an Adobe Flash container, most likely containing h264-encoded media.  I moved it to my own [Renders] folder for the given project and it played fine in VLC Media Player, which confirmed h264 was the codec and indicated it had resolution 960×540 i.e. half-size in terms of length, quarter-size in terms of area, bitrate was around 1Mbps.

Mobile Video Editing Hardware: Thoughts, Ideas & Dreams (continued)

Friday, January 13th, 2012

Following-on from my earlier post, Mobile Video Editing Hardware: Thoughts, Ideas & Dreams, where I considered an eventual migration from my laptop to a luggable PC, my thoughts veered (possibly having spotted cash-icebergs among them) towards an alternative solution:

  • Use the laptop for lightweight editing & compositing.
  • Use the desktop as a number-crunching RADI-attached server.

The two could be linked by:

  • Remote access / remote sessions (some of which via smartphone)
  • DropBox, e.g. have an active folder where I can drop Adobe Premiere XML and have it processed remotely by Adobe apps installed there.

Some links:

  • http://www.theregister.co.uk/2012/01/11/virtualisation_at_home_part_2/
    • (There’s no equivalent “_part_1” page.  I guess it’s just “Part 2” of that guy’s story).
    • DIY virtual machines: Rigging up at home, by Trevor Pott, 11th January 2012 14:33 GMT
    • Personal Virtual Machine (PVM) (in use) for about seven years with retail boxed version of Windows XP.
    • VM has been moved from virtualization platform to virtualization platform over the years … the most recent incarnation … inside Hyper-V.
    • …nothing beats Windows Server 2008 R2. It comes with a top-notch virtualisation platform (Hyper-V), and added RemoteFX support with Service Pack 1. You can still use the desktop operating system for all your HTPC needs, and a single Server 2008 R2 Standard license allows you to run both a host copy and a single virtual instance of Server 2008 R2.
    • In my case, the host instance does little more than play movies on the projector via VLC. The virtual instance of Server runs my Plex media server, and aggregates my many storage devices into a single share using DFS.
  • Shuttle Inc (Taiwan)

Mobile Video Editing Hardware: Thoughts, Ideas & Dreams

Tuesday, January 10th, 2012

Want a mobile “suitcase” editing system, something more (and more expandable) than a laptop but not too expensive.  Primarily to be used for Adobe CS5.5 for media enhancement / editing / compositing etc.

Nearest I found was NextDimension’s range around $7000 I think (but just guesswork – could be way off – would need to get a quote).   That would (if true) be around £4500 at current rates.  Plus import…  NextDimension call such machines “flextops” (Maybe they coined the term? Google searches on it mostly come up with them.)

Apart from the (mil/broadcast-lite but me-heavy) price, it might possibly be undesirably heavy to lug around much.   If so (just guessing, not assuming), it would make more sense to go for a modular quick-setup system.  So, starting to “think different” in this direction:

  • Standard tower, capable of taking new CUDA etc. graphics cards etc. as they emerge, but no need for more than say a couple of disks, maybe if SSD could even get away with just a single disk? (For system and media – inadvisable for traditional disks of course, what about for SSD’s?  I have much to learn about SSD’s though).
  • “Laptop-Lite” to talk to it.  With robust shuttered-stereoscopic HD monitor.
  • Gigabit network to NAS fast storage (SSD and/or RAID ?).

Maybe in that case it would be far more logical/affordable to use an existing laptop as a client working together with a luggable tower server, sufficiently light and robust for frequent dis/re -connection and travel.  And remote access of course (no heavy data to be exchanged, assume that’s already sync’d).  And some means to easily swap/sync applications and projects (data) between laptop and tower, giving the option to use just the (old) laptop on its own if needed.  All such options are handy for the travelling dude (working on train, social visits etc.) who also occasionally has to do heavy processing.  Then would just need a protective suitcase for the tower, plus another one for a decent monitor for grading etc.

I certainly won’t be spending anything just yet, but it’s good to have at least some kind of “radar”.

(more…)

DropBox Tips for Video Production

Thursday, December 1st, 2011

Tips from Stu Maschwitz (Magic Bullet, DV Rebel’s Guide,…)

Training: Den Lennie’s “Music Video” Experience

Thursday, October 27th, 2011

I attended, working on one of the camera units.  Had a great time, learnt lots, at all sorts of levels.  Even how to make good use of the Movie Slate application on my iPhone!  Link: http://www.fstopacademy.com/

Cineform FirstLight: Explanatory & Instructional Links

Saturday, August 20th, 2011

Conclusions:

  • The main principle is great – the decoder part of the Cineform Neo codec has to do levels-mapping work etc. anyway as part of its normal function, and so getting it to do the grading at the same time just means altering its scaling factors etc., which in principle means using less CPU as compared to the grading being done in the NLE (post-decode).   Also fewer successive quantizations (hence better overall visual quality).  You can specify different sets of factors (hence grades) for different video files. There are also some “Movie Looks” presets.
  • Additionally it gives the ability to split the grading process off to another person (as the tutorial videos show) – a great extra bonus. By using DropBox (say) the two (or more) of you can work in parallel at remote locations, grading-updates appear automatically on the remote NLE. Essentially only a tiny shared grading project file is saved in DropBox, no need to exchange actual video files.

I’m currently trying it out on a client project (non-critical) in Sony Vegas.  I will post my experiences from this separately.

Sorenson 360 (and its relationship to Sorenson Squeeze)

Saturday, August 13th, 2011

Sorenson 360 is a cloud-web-based video content and distribution management system.  Superficially looks much like YouTube or Vimeo, but has multiple professional/enterprise “added value” features as well as integration with Sorenson Squeeze (naturally), WordPress and Drupal.  The cheapest price plan is “Small Business”, at $99/month, which at time of writing equates to £61/month. There is a 30-day free trial featuring 1GB bandwidth (sic) and 1GB storage.

(more…)

Avid MetaFuze – Description & Role

Saturday, August 13th, 2011

MetaFuze’s primary, if not only purpose is to prep files for Media Composer use – an “outboard importer”.  Though bundled with Media Composer, it is also available free, from www.avid.com/metafuze (which redirects to http://www.avid.com/US/industries/workflow/MetaFuze).  That means for example that preprocessing work (e.g. generation of burnt-timecode proxies and online files) can be generated (e.g. in DNxHD from an arbitrary source) by anyone whether or not they have an Avid system (apart from this app, and the free Avid codecs.  Potentially then a great option for breaking up work into collaborative / parallel workflows.

(more…)

Collaboration tools (beyond iChat)

Thursday, January 7th, 2010

Links:

  • http://lifehacker.com/315747/tech-support-and-document-collaboration-gets-easy-with-ichat
  • http://lifehacker.com/5088083/five-best-video-chat-applications

FCP iTheatre: play an FCP timeline straight into a chat

Thursday, January 7th, 2010

Geographically separated team members of a Final Cut project can collaborate via iTheatre.   Suppose an editor is discussing with a client via Apple’s iChat.  Then the editor can play out of FCP straight into the iChat session.   It’s in FCP: View > iChat Theatre Preview.

  • http://www.youtube.com/watch?v=RACPO0N4Hc4

There are some reputable video people covering this topic:

  • http://www.dvinfo.net/forum/non-linear-editing-mac/353369-ichat-theatre-preview.html
  • http://www.kenstone.net/fcp_homepage/fcp_ichat_theater_stone.html

But iChat is only available for Macs.  What if the client doesn’t have a Mac?  I did some googling:

  • iChat is an Instant Messenger.
  • It is only available for Macs but it is compatible with AIM (AOL Instant Messenger), which is available for Windows.
    • ” iChat is only available on OS X” [http://lifehacker.com/5088083/five-best-video-chat-applications]
  • .mac accounts are just AIM accounts. Enter your .mac username with @mac.com at the end while configuring an AIM account. [http://forums.macrumors.com/showthread.php?t=458194]
  • Whatever is playing in your Viewer or Canvas window is sent via iChat to your remote viewer. [http://digitalcontentproducer.com/videoedsys/revfeat/apple_ichat_theater_preview_0827/index1.html]
  • But can the iChat Theatre option be used with Windows users?
    • This person believes it is not possible to interact with Windows users (I wonder if they know of the AIM option or whether it handles iTheatre) [http://forums.creativecow.net/thread/8/1065470]
    • “If your collaborators are on Windows and don’t have access to a Mac, you’re out of luck.” [http://digitalcontentproducer.com/videoedsys/revfeat/apple_ichat_theater_preview_0827/index1.html]
  • “for iChat Theater Preview to work, both the host and remote computers much be running Mac OS X version 10.5 Leopard or later. Both computers must have iChat accounts and the remote user’s account must be added to the host computer’s buddy list (more detail from Apple here). ” [http://digitalcontentproducer.com/videoedsys/revfeat/apple_ichat_theater_preview_0827/index1.html]
  • [http://www.google.com/support/forum/p/Talk/thread?tid=3669a2bf94155d06&hl=en]
    • “… is it possible to make videoconference between a PC using google video and a MAC using ichat ?”,
      • “No, it is not possible.  Video chat is only between Gmail chat users.” and furthermore “You need … the Newer Version interface of Gmail”
  • It has screen sharing, so both parties can observe and control a single desktop

Some links about iChat itself:

  • iChat itself:
    • http://www.timsblog.net/2008/06/07/ichat-for-windows-maybe
      • Overview and context wrt other Instant Messengers (IMs).
    • http://en.wikipedia.org/wiki/IChat
      • Has great list of add-on tools e.g. encryption.
    • http://www.apple.com/macosx/what-is-macosx/ichat.html
  • How to converse with Windows users
    • [http://screencastsonline.com/forums/cgi-bin/yabb2/YaBB.pl?num=1216452140/0]
      • “I wish MSN Messenger was supported natively as hardly anyone I know in the UK uses AIM. I’ve only got 10 AIM buddies. I have to go through a Jabber server with an MSN transport to use it”
      • “I am same as the others. I use Adium as I have MSN and Google Talk friends mostly and very few on AIM.   Sure I can use iChat fo Google Talk”
      • “It seems most of us are using Adium because iChat doesn’t connect to MSN and/or Yahoo Messenger.”
      • “Skype doesn’t intereact with any other software as far as I know but for internet-based phone video calls, it is the best because so many people use it and it is on Windows as well as Mac.”
    • [http://digg.com/apple/Adium_2_0_Coming_With_video_audio_support]
      • Adium is what Pidgin should be.
      • Adium is based on Pidgin’s libpurple (libgaim), which is already used in Pidgin on all three platforms.
      • Adium is the Cocoa client of libpurple. Pidgin is the GTK client, finch is the CLI client, etc… Same IM backend, multiple GUIs.
    • http://pressf1.pcworld.co.nz/showthread.php?t=86311
      • “AIM [http://products.aim.com] is about as close to iChat as you’re gonna get on Windows. That said, a better option would be to go with something more flexible like Pidgin. [http://www.pidgin.im]”
  • AIM
    • http://products.aim.com/
    • [http://digg.com/apple/Adium_2_0_Coming_With_video_audio_support]
      • If you’re using the AIM service, make sure you have port 5190 open on your router, NAT, firewall, cable modem, or DSL modem. You’ll need it open for incoming connections to send and outgoing connections to receive (yes, that is the right way round: the recipient connects to the sender).
  • Pidgin
    • Is open-source, has sometimes debatable deadlines and capabilities.  Many consider Adium better, and it has been stated that they are both front-ends onto the same core.
    • (formerly GAIM) [http://www.tuaw.com/2007/10/28/adium-offers-first-look-at-video-integration]
    • [http://www.pidgin.im/about] “Pidgin is a chat program which lets you log in to accounts on multiple chat networks simultaneously. This means that you can be chatting with friends on MSN, talking to a friend on Google Talk, and sitting in a Yahoo chat room all at the same time.  Pidgin runs on Windows, Linux, and other UNIX operating systems”
    • There are third-party plugins for other protocols such as Skype and Twitter.
  • x