Archive for the ‘video’ Category

Shared Storage Options for Windows & Mac Video Editing Collaboration

Friday, October 18th, 2013

In summary:

There’s no magic option, each workstation needs a local storage volume with block-level data access (as opposed to simply file-level access) and formatted to a file system that is native (doesn’t require translation) to that workstation’s operating system.  Migration and collaboration imply file copying/synchronization, which implies read-access to the “foreign” file-system.  Mac OS can read NTFS, Winows can only read HFS+ via third-party add-on utilities.  Furthermore, for speed and responsiveness appropriate to video editing, the local storage should ideally be RAID or SSD.  In either case, it is possible to split the local storage (e.g. via partitioning) into more than one file-system.  At least, that worked on the mutiple occasions I have taken that approach, and have not been aware of any issues.

In greater detail:

Consider the challenge of setting up a shared data storage volume (e.g. RAID array or SSD) for video editing, such that either Windows or Mac computers can connect to it, and a video project started on (and saved to) on one of those operating systems (OS) can be continued on the other (and vice versa).

My current solution is to split the drive into separate volumes, one for each OS.  For example I have done this on RAIDs of various kinds and on an internal drive for Mac systems bootable to either Mac OS or (via Boot Camp) to Windows.  In the case of RAIDs I was advised against this by my system supplier, but got the impression they were just being defensive, not knowing of any definite issues, and to my knowledge I did not experience any issues.

It is is not practical to have just one volume (necessarily in that case, one file-system format), because:

  • Mac OS on its own is able to read NTFS but cannot write to it.
    • This is a show-stopper.  Some of the major video editing applications (e.g. NLEs), slightly disturbingly, may use (or for some functionality, even depend on) read/write access to source-files and the folders containing them.
      • I initially, naively, imagined that video editing systems etc. would only ever read source media files, not write to them, or to the folders containing them.  However that proved very naive indeed…
        • In Apple/Mac’s (erstwhile) Final Cut Pro 7 I regularly used their (moving) image stabilization effect, SmoothCam.  Its analysis phased was typically slow and heavy – not something one would wish to repeat.  The result was a “sidecar” file of similar forename to the analyzed source file, but a different extension, placed in the same folder as the source file.
        • I’m not certain, but got the feeling that maybe the source file (or folder) meta data, such as permissions or somekind of interpretation-change to media files in the quicktime ([.mov] mmedia format.
      • Certainly, Adobe (on Windows and Mac) could adulterate both files (by appending XMP data – being an Adobe media metadata dialect in XML) and the folders they occurred in (depending on uder-configuration) in terms of sidecar-files.
      • Sony Vegas also generates sidecar-files, e.g. for audio peaks.
  • File system translation add-ons can add Windows read/write access to HFS+ (ordinarily it could not even read it) and add Mac OS write access to NTFS (ordinarily it could only read it), but not sufficiently transparent/seamless for big real-time data access as required for demanding video editing endeavours.
    • File system translation add-ons (to operating systems) exist, such as MacDrive, to allow Windows to read/write Mac OS, or Tuxera NTFS, Paragon NTFS or Parallels for Mac to enable it to read/write NTFS, but these (reportedly, and in part of my experience) only really work well for standard “Office” type applications, not so well for heavy (big andd real-time) data applications such as video editing, where they can impede the data throughput.  Doh!
    • Some people have experienced obscure issues of application functionality, beyond data-movement speed issues.
    • {Also, I am concerned over the (unknown/imagined/potential) risk that the “alien” operating system and/or its translation utility might alter the file system in some way that upsets its appearance to the “home” operating system.}
  • FAT is universal but is a riskier option:
    • FAT is un-journaled, hence risks loss not only of individual files but of whole volume (integrity).
      • In video editing, corruption could be disastrous to a project, not only in terms of possible data-loss or time wasting and project delays on data recovery, but also in terms of “weird” effects during editing, such as poor responsiveness to commands, whose cause the user may not appreciate. or even an increased risk of unacceptable flaws in the final product.
    • FAT32 is essentially obsolete, because its maximum file size is (1 bit under) 4GB.
    • exFAT, a kind of “FAT64” is practical, and indeed a big successful corporate Mac-based production company once supplied me with many GB of footage on an exFAT-formatted external disk.
      • The largest file I have so far stored there is 40GB.  No problems.
  • NAS (Network-Attached Storage) sounds at first an easy option, but in my experience they impede big real-time data throughput (as stated earlier for “file system tyranslation” add-ons). Double-Doh!
    • Such devices only permit file-level access.  Consequently, the client systems can e.g. create or retrieve folders and files, but cannot e.g. format the device or address it in terms of lower-level data structures.
    • A likely explanation for the “impedement” of a NAS (to data responsiveness and throughput) is that such devices store in a local format (typically they run linux) that is invisible to the client, then translate to an appropriate protocol for each operating system accessing it.  They normally incorporate a bunch of such protocols.  As always, translation => overhead.
    • Other options, such as SAN and iSCSI, instead of providing file-level access to the client systems, instead offer the lower level of data block access.  Thus they appear to the client system as would any local storage device, and can be formatted as appropriate to the client system.
  • One suggestion I saw was to use a Seagate GoFlex drive, which can be used (read/write) with both Mac and Windows.  But the supplier’s FAQ (about that drive) indicates that it depends upon a translator utility for the Mac:
    •  If you would like to be able to “shuttle” data back and forth between a Mac and a PC, a special driver needs to be installed onto the Mac that allows it to access a Windows-formatted drive (i.e. NTFS). Time Machine will not work in this case, nor will Memeo Premium software for Mac. However, if you want your GoFlex solution to also work with TimeMachine, the drive will need to be reformatted to HFS+ journaled.

So I guess there is no “magic storage” option, my main work setup will have to remain based on separate volumes for each OS.

When transferring an editing project from one OS to another, the following actions will be necessary:

  • Copy any absent or updated files across.
    • e.g. via a file-synch utility such as Syncovery.
  • Allow time etc. for possible file re-linking, re-indexing, re-preview generation, re-“SmoothCam” (or equivalent).
    • This aspect is down to the editing application etc., as opposed to the operating or file systems themselves.
  • Ensure any effects used in the edit are present on both systems.
    • If so then these should presumably still work…

(more…)

NLE Adulteration of Source Media: Potential Workflow-Issues

Friday, September 13th, 2013

I highlighted in http://blog.davidesp.com/archives/598 (10 months ago) that Adobe Premiere etc. can adulterate media files, in terms of metadata and/or sidecar-files (depending on user-configurations of these applications.  I indicated that, regardless of the reasonableness of at least some of these actions, this could potentially cause problems to other applications.

Validating that concern, I note a post (2012-06-12) by Matt Davis on Philip Bloom’s website, stating (my italics):

  • …if sharing assets with FCPX and Adobe Premiere, Adobe ‘touches’ (resets the modification date) of each file without doing anything else to it, but also sprinkles sidecar files into directories of transcodable files for metadata, thus sending any returning FCPX activity into a tailspin, requiring a re-linking session. It’s oddities like these which haunt the implementation of FCPX in a wider system and make system managers wonder if FCPX is actually worth implementing in its current state.

That was over a year ago, and so the issue may or may not exist for the current version of FCPX.

As users, whether or not the actions of one application adhere to standards and another don’t, what we as users ultimately care about is workflow, which in this case translates to “does it connect up with my other tools/processes?”.  So we have to maintain a “situational awareness” of potential interoperability pitfalls.

Incidentally, I recall that FCPX’s predecessor (in history at least, if not development-line) FCP7 could adulterate source directories with its own sidecar files, produced by its SmoothCam effect.  Not knowing anything further for sure, I nevertheless wondered (at that time) what it might be doing “under the hood” of the QuickTime [.mov] wrapper.

HDV 50i from Sony Vegas to SD 50i Intermediate to Adobe Encore DVD

Sunday, July 14th, 2013

(This is actually an older post, from about a wek or so ago, but it was left languishing in “Draft” status.  But rather than delete it, here it is, out-of-sequence, for posterity)

Nowadays for video editing I mainly use Adobe CS6.  However I have still some old projects edited with Sony Vegas (10) which now have new clients.  One such project was shot as HDV on a Z1, giving 1440×1080 interlaced, at 50 fields/second, which I call 50i (it doesn’t really make sense to think of it as 25 fps).  The required new deliverable from this is a PAL-SD DVD, 720×5786 50i.  In addition, I want to deliver high-quality progressive HD (not V) 1920×1080 progressive.

The PAL-SD frame size of 720×576 has exactly half the width of the HDV source and just over half its height.  My naive initial thought was that the simple/cheap way to convert from the HDV source to the SD deliverable would be to merely allow each of the HDV fields to be downscaled to the equivalent SD field.  This could be performed in Sony Vegas itself, to produce an SD intermediate file as media asset to Encore to produce a DVD.

Some potential complications (or paranoia) that come to mind in this approach are:

  • Levels-changes, through processes associated with the intermediate file.  For example it might accidentally be written as 16-235 range and read at 0-255 range.  In general, uncertainty can arise over the different conventions of different NLEs and also the different settings/options that can be set for some codecs, sometimes independently for write and for read.
  • HD (Rec 709) to SD (Rec 601) conversion: I think Vegas operates only in terms of RGB levels, the 601/709 issue is only relevant to the codec stage, where codec metadata defines how a given data should be encoded/decoded.  The codec I intend to use is GoPro-Cineform, with consistent write/encode and read/decode settings.  Provided Vegas and Encore respect those, there should be no issue.  But there is the worry that either of these applications might impose their own “rules of thumb”, e.g. that small frames (like 720×576) should be interpreted as 601, overriding the codec’s other settings.
  • Interlace field order.  HDV is UFF, whereas SD 50i (PAL) is LFF.  Attention is needed to ensure the field order does not get swapped, as this would give an impression of juddery motion.

So I did some experiments…

  • Vegas (1) Project Settings:
    • Frame Size: 720×576
    • Field Order: LFF
    • PAR: 1.4568
  • Render Settings:
    • Frame Size: (as Project)
    • Field order: LFF (I think the default might have been something else)
    • PAR: 1.4568
    • Video Format: Cineform Codec

What Worked:

  • Sony Vegas (v.10) project for PAL-SD Wide, video levels adjusted to full-range (0-255) via Vegas’s Levels FX, then encoded to GoPro-Cineform.
  • Just as a test, this was initially read into an Adobe Premiere project, set for PAL-SD-Wide.  There, Premiere’s Reference Monitor’s YC Waveform revealed the levels range as 0.3 to 1 volts, which corresponds to NTSC’s 0-100% IRE on the 16-235 scale.  No levels-clipping was observed.
  • So using the 0-255 levels in Vegas was the right thing to do in this instance.
  • The Configure Cineform Codec panel in Sony Vegas (v.10) was quite simple, offering no distinction between encode and decode, allowing only for various Quality levels and for the Encoded Format to be YUV or RGB.  The latter was found to have no effect on the levels seen by Premiere, it only affected the file-size, YUV being half the size of RGB.  Very simple – I like that!
  • In Premiere, stepping forwards by frame manually, the movements looked smooth.

In Adobe Encore (DVD-Maker) CS6:

  • Imported the intermediate file as an Asset and appended it to the existing main timeline.
  • Encore by default assumed it was square-pixels.  Fixed that as follows:
    • [theClip >RtClk> Interpret Footage] to selrct the nearest equivalent to what I wanted: [Conform to SD PAL Widescreen (1.4587)].
      • Why does Encore’s [1.4587] differ from Vegas’s [1.4568] ?  Any consequence on my result?
  • Generated a “Virtual DVD” to a folder.
  • Played that “Virtual DVD” using Corel WinDVD
    • In a previous experiment, involving a badly-produced DVD having swapped field-order, I found this (unlike WMP or VLC) reproduced the juddering effect I had seen on a proper TV-attached DVD player.  So WinDVD is a good test.
  • Made a physical DVD via Encore.
  • The physical DVD played correctly on TV (no judder).

An alternative would be to deinterlace the original 50i to produce an intermediate file at 50p, ideally using best-quality motion/pixel based methods to estimate the “missing” lines in each of the original fields.  But would the difference from this more sophisticated approach be noticeable?

There also exists an AviSynth script for HD to SD conversion (and maybe HDV to SD also?).

  • It is called HD2SD, and I report my use of it elsewhere in this blog.  I found it not to be useful, producing a blurry result in comparison to that of Sony Vegas ‘s scaling (bicubic).

 

Tools/Workflow Philosophy: Best-of-Breed rather than Already-Integrated Suite ?

Sunday, July 14th, 2013

I am becoming less enthusiastic about the “Integrated Suite” philosophy or perhaps actuality of Adobe CS6, in favour of a “Best of Breed” approach, where I cherry-pick the best tool for each kind of job and then design or discover my own workflow for integrating them.

I reached this conclusion from the following experiences:

  • As regards editing itself:
    • For general A & B Roll” editing, I find Premiere is ok, though for improved usability, I’d prefer a Tag-based system (as in FCPX) to the traditional Bin-based one (as in Adobe & Avid).
    • For MultiCam editing, even in Adobe CS6, I find Premiere does the job but I find it clunky, frustrating and limited at times, like it has not yet been fully “baked” (though “getting there”)…
      • e.g. In the two such projects I have so far worked on, there has been an annoying 2-second delay from pressing the spacebar to actual playing.  Maybe some kind of buffering?
        • I found a setting for “Pre-roll” in the Preferences but altering it made no difference.
        • The following http://forums.adobe.com/thread/387405 suggested that the embedded audio (in video file) could be the issue, the solution to which was to relink to a WAV file.
      • e.g. It brings up a separate MultiCam Monitor instead of using the Source Monitor.  You have to remember to activate this each time before playing.  I find that a nuisance (and time-waster when I forget) especially because I tend to alternate multicam editing as such with tweaking the cut timings until they feel right, and sometimes that can only be done in retrospect.
      • e.g. When you stop playing in multicam mode, it places a cut (that you probably didn’t want) wherever the playhead happens to be at the time.
        • I see I am not the only one complaining about this: “ExactImage, Sep 15, 2012″at  http://forums.adobe.com/thread/1069438
          • A workaround given at that link: Before to stop the playback press the key 0 (zero) of the keyboard and then you can stop the play (with the Space bar) without the cut in the timeline.” Duh!
      • e.g Markers are really useful in multicam, but while Premiere’s are steadily improving with product version, they are way clunkier and more limited than those in Sony Vegas:
        • e.g. I put a marker at the start of an interesting section (of timeline), I select it and define its duration to be non-zero, so I can stretch it out to mark a region, then I drag the playhead to the find the end of that interest, I try to drag the marker’s right-hand end up to the playhead, but instead the playhead gets reset to the start of marker.  Duh!
        • e.g. Markers cannot be promoted from clip (media or nested Sequence) to current Sequence.
        • e.g. waveform displays (assuming you can get them to appear in the first place) go blank when sliding clips around.  Really annoying when trying to synchronise to music etc.
    • …so I will explore other options for multicam:
      • In the past (as will be apparent from the above) I have had more joy, as regards Multicam, with Sony Vegas.
      • I will check out what people think of other NLEs as potential “Best of Breed” for multicam editing.  Thus far I have heard (from web-search) good things about FCPX and LightWorks.
  • For audio enhancement, such as denoising, I find iZotope’s RX2 far superior to the one in Adobe Audition.
  • For making a DVD:
    • I find Encore to be handy in some ways but limited and clunky in others.
      • e.g. can’t replace an asset with one of a different type (e.g. [.avi] and [.mpg]).
    • The advantage of using an integrated DVD-Maker such as Encore might be limited:
      • e.g. many people are not using the direct link, but exporting from Premiere/AME, in which case any third-party DVD Builder could be used.
      • The only significant advantage I am aware of is the ability to define Scene/Chapter points in Premiere and have them recognised/used by Encore.
        • But maybe some third-party DVD Builder applications can also recognise these?  Or can be configured/helped to do so?  Worth finding out.
    • ?

Adobe Premiere – Source Media Adulteration

Saturday, November 10th, 2012

Sometimes Adobe Premiere may write to a source media file or proprietary folder-structure.  This may be considered a non-problem in most situations, but it is nevertheless worth being aware of.

This is nothing hidden, surreptitious or unheard-of, it’s explained in Adobe’s Help text and documentation.  However the potential consequences may not be obvious to a new user.  It may arise at various points of what we may regard as the greater process (workflow/manual) of ingesting media, consisting not only of Premiere’s Import of media but also subsequent manual updating of metadata or indeed automatic analysis such as speech recognition.  As of CS6 it can also occur as a result of adding Markers in Adobe Prelude.

Premiere likes to add and manage metadata for each media file.

  • The good side of this is that it value-enhances these files, making them easier to locate, navigate and use, potentially increasing workflow productivity and asset usage.
  • But there’s also a dark side – not necessarily Adobe’s fault (e.g. their approaches may well adhere to official media specifications) – but it may be that so-adulterated media files may cause difficulties to other applications (e.g. that may not fully take on board such standards).
    • In my experience, in the past, some (possibly poorly-written, but nevertheless useful) applications have refused to work with metadata-augmented files, again holding up productivity, in this case while the user figures out the issue and works out how to strip this data out, in order to progress.
    • Technically a non-problem, but potentially consequential to a workflow, backup software will (rightly, from its point of view) see the metadata-change as a file-change (e.g. as a consequent file-size change) and consider that the files have been updated.   Left to itself, the backup process (depending how it works/configured) will overwrite any previous copy of the files (e.g. the original files).   Even if the backup process prompts the user to confirm this, the naive user may be uncertain what to do,

Also, the user has the option at their discretion for Premiere to automatically store additional files (such as cache files and metadata sidecar files) alongside source media files.

  • In the case of media represented as a straightforward single file (like a .jpg or .mpg file) this does not affect that media.
  • However some media (e.g. TV-playable DVDs or XDCAM-EX video media) are stored as proprietary folder structures with defined contents, part of these contents being essence files (e.g. .vob files or .mp4 files) while other files alongside them etc. in that structure (e.g. DVD’s .IFO files or XDVCAM-EX’s .SMI files) contain metadata or index into them etc.  In this case, the consequence of adding further files into the structure will (in my experience) be acceptable to some applications and media players but not to others, which regard it as “pollution”, and may then reject such structures.  Certainly in the past I have seen this happen in some software applications and also even some (mostly old) TV DVD players.

This is a case for “situational awareness”: if one is aware of the nature and potential consequences of the adulteration (be it regarded as pollution or enhancement, depending on the workflow situation), one is then in a better position to be able to avoid or fix any asociated issues. (more…)

Using an XCDAM-EX Video Camera as a Webcam

Thursday, September 20th, 2012

If you put an XDCAM-EX camera in 1080-SP mode then it generates HDV, which is 1440×1080 but with brick-like pixels and data is at Constant Bit Rate (CBR), a requirement of HDV.  For an XDCAM-EX3, there is a small iLink/Firewire connector at the back, it can be enabled/disabled by the camera’s menu.If you then run Skype, it sees the image, scales it down to Skype-format, namely 640×480 with square pixels, hence 4:3 aspect ratio.   However it does not allow for the difference in pixel shape, with the result that the image looks squashed horizontally.  A nuisance!So I wonder, is there any  interposing software that can e.g. map the pixel shapes properly or even allow some kind of zoom/pan of a (e.g.) 640×480 frame within available (e.g.) 1920×1080 (equivalent, when pixel-shape converted) image from HD or HDV camera?

  • http://www.dvinfo.net/forum/sony-xdcam-ex-cinealta/479844-need-help-doing-webcast-shoot-ex3.html
  • http://forums.creativecow.net/thread/142/864423
    • The EX in HDV mode (SP) and iLink has worked for me. I’ve done it with Telestream Wirecast (another COW forum you’ll find my floating head) for live streaming.
  • http://www.b-roll.net/forum/showthread.php?t=26269
    • Someone having difficulties, others offering advice on XDCAM-EX settings, though I’m not convinced that all of them are necessary)
  • http://www.ehow.com/how_12174693_use-hdv-camera-skype.html
    • Debut Video Recording software (free)
    • Instructions:
      • Connect the video cable plug on the USB video-capture device to the video input on the camera cable. Video cable plugs and inputs are usually color-coded yellow.
      • Plug the camera cable into the camera.
      • Plug the USB video-capture device into an open USB port on your computer.
      • Download “Debut Video Recording Software 1.42” or a later version and install the application on your computer.
      • Launch the software.
      • Click “Device” from the toolbar menu and then go to Step 7. If the software doesn’t recognize your HDV camera, click “Options.” Click the “Video Capture Device” arrow and select your camera from the list. Confirm that the “Format” and “Device” settings are correct or make changes, if needed. Click “OK” to continue.
      • Click the “Skype” icon on your computer’s taskbar and select “Open Skype.”
      • Right-click the person you want to call under “Contacts.”
      • Navigate to “Share Your Screen” and select “Share Selection.”
      • Use your mouse and drag the black box down to the inside of the video-capture software’s display area.
      • Resize the black box to fit within the display area.
      • Click on the “Start Screen Sharing” box in the upper left corner of the video display area.
      • Click “OK” in the Screen Sharing dialogue box. A “Starting Video” message will appear.
      • Wait until the person you are calling picks up. The “Screen Sharing Active” message confirms the connection
      • Read more: How to Use an HDV Camera for Skype | eHow.com http://www.ehow.com/how_12174693_use-hdv-camera-skype.html#ixzz272JXfJGR

Adobe Production CS6 – Cache & Render Files and their Locations

Saturday, September 1st, 2012

When I start-up any application, I like to understand at least the main side-effects it’s having on my system.  In the case of Adobe’s primary video-editing apps, Premiere and After Effects, my experience (on Windows 7) is that they save intermediate preview-renders to the system volume.  This causes me the following concerns:

  • System Volume may serve poorly as a media drive.
    • Larry Jordan, at least in the recent past, advises against using the system drive for media read/write.  On the upside, such drives may have high-bandwidth to the system, but on the downside, the system can interrupt their use with highest priority, whuch may (I guess) pose a risk to smooth playback (though I am aware that buffering may possibly reduce this risk, I haven’t done or seen any such calculations).  Cache files are indeed media files that are written and read.
    • On the other hand, an informed representative of a well-known UK supplier of video editing laptops advised me that in his experience, most users of laptops with only a single internal drive (as system drive) do use that drive in this way (for portability).
  • System drive can become “clogged up”
    • System drive can become clogged-up by many or large video files of which that the user is only partially aware, their creation having happened implicitly during their use of the NLE etc.  Like temporary files only worse!
    • Ultimately the system drive can even become full, making the operating system itself sluggish or even less stable (and video playback less smooth.
    • Backup of a system drive that includes media files will typically require significantly greater archive space and will take significantly greater time (than a clean system).
  • Migrate-ability is reduced
    • I like the idea of a video project being a free-floating data-object.  That is, it should not be tied to any particular instance of a data storage volume, let alone a particular computer (system).  It should be possible for all files relevant to a project to be stored on any volume, migrated to any other volume, plugged into any computer having appropriate installed applications, and everything to work the same way as when the project was on its original volume being edited on the original system.  That includes not only the source media files etc. but also the intermediate rendered files.

So what do the Adobe editing applications provide to enable my preferred working arrangement?

  • Premiere:
    • [Edit > Preferences > Media]
      •  This defines the location of the folder [Media Cache Files], which contains pseudorandomly-named files.  Example Files:
        • [929_4372_01-125eeda9-ba0d-a8ea-4418-3480000001f0.ims]
        • [Rendered – 68721ea9-25e9-4f56-8430-4ca10101ace7-04602910-cd54-1a45-a5d7-557b000001f2.ims]
      • Default location: [c:\Users\…\AppData\roaming\Adobe\Common]
      • [Yes] Save Media Cache files next to originals when possible
        • e.g. for my XDCAM-EX files, inside a CLIPR folder (which contains the EX’s “essence” (.mp4) files, in this case [929_4491_01.MP4]), appeared the following files:
          • 929_4491_01.MP4 48000.cfa (9.2 MB)
            • Unknown, but the “48000” and the “a” on the end of “cfa” are suggestive of audio.
          • 929_4491_01.MP4 48000.pek (37KB)
            • Simply the peaks (waveform graphics data) file for the audio component of the essence-file.
        • Experience:
          • I clicked the [Browse] button and selected an area on my external media drive (a GRaid Mini) as: [H:\_App_Specific\Adobe].
          • Consequently, at my specified location the following folder appeared: [Media Cache Files]
    • Media Cache Database
      • Note:
        • When Premiere Pro imports video and audio in some formats, it processes and caches versions of these items that it can readily access when generating previews. Imported audio files are each conformed to a new .cfa file, and MPEG files are indexed to a new .mpgindex file. The media cache greatly improves performance for previews, because the video and audio items do not need to be reprocessed for each preview.
        • When you first import a file, you may experience a delay while the media is being processed and cached.
          A database retains links to each of the cached media files. This media cache database is shared with Adobe Media Encoder, After Effects, Premiere Pro, Encore, and Soundbooth, so each of these applications can each read from and write to the same set of cached media files.
      • Location: [c:\Users\…\AppData\roaming\Adobe\Common]
        • [Browse]
          • If you change the location of the database from within any of these applications, the location is updated for the other applications, too.
          • Each application can use its own cache folder, but the same database keeps track of them.
          • Example Experience:
            • I clicked the [Browse] button and selected an area on my external media drive (a GRaid Mini) as: [H:\_App_Specific\Adobe].
            • In response, a prompt came up saying “Move the existing media cache database to the new location, or delete it (Buttons: [Move] [Delete] [Cancel] ).
            • I clicked [Move]
            • Consequently, at my specified location the following folder appeared: [Media Cache]
      • Purging
        • [Clean]
          • This removes “orphan” cache files.
          • To remove conformed and indexed files from the cache and to remove their entries from the database, click [Clean]. This command only removes files associated with footage items for which the source file is no longer available.
            • Important: Before clicking the [Clean] button, make sure that any storage devices that contain your currently used source media are connected to your computer.
      • [Yes] Write XMP ID To Files On Import
        • Check this box to write ID information into XMP metadata fields.
        • e.g. hence [929_4491_01M01.XMP] sidecar-file (containing XMP metadata) got written into the CLIPR folder containing its associated media  file, here an XDCAM-EX essence file, [929_4491_01.MP4].
      • [Yes] Enable Clip And XMP Metadata Linking Check this box to link clip metadata to XMP metadata, so that changing one changes the other.
    • Intermediate-Preview Render Files
      • None of the above measures affect where intermediate/preview files get rendered to…
        • I proved this (in Windows 7) by deliberately causing a render then searching on “Huge” (>16MB) files created “Today”
        • Rendered files location was:[C:\Users\…\Documents\Adobe\Premiere Pro\5.5\Adobe Premiere Pro Preview Files\Untitled3.PRV]
      • A brief Google revealed several articles where the (sadly now obsolete) solution was a setting under [Edit > Preferences > Scratch Disks]
      • Eventually discovered that in CS5.5 these settings now resided in: [Project > Project Settings > Scratch Disks]
        • Here, all settings (affecting Capture and  Preview) were set to [Same as Project]
        • As it happened, my project location was [C:\Users\David\Documents\Adobe\Premiere Pro\5.5].
      • Experiment with a different project location:
        • Save As: [H:\_Media\_Projects\MyProject\030 Projects\Adobe\Experiments]
          • The file [Expt A v001.prproj] appeared in it, but that was all.
        • Save a Copy: [H:\_Media\_Projects\MyProject\030 Projects\Adobe\Experiments]
          • The file [Expt A v001 Copy.prproj] appeared there, but that was all.
        • (The file open within Premiere remained as the original, not the copy)
        • After a while, the project file was joined by: [Adobe Premiere Pro Auto-Save]
      • Experiment to migrate the Preview-Render files:
        • In Windows Explorer, I created a folder named [Adobe Premiere Pro Preview Files]
        • Into that folder, from the similarly-named folder on the system volume, I dragged the existing folder [Untitled3.PRV]
      • Experiment: Premiere “knows” when render-files have gone, and prompts for their possible new location.
        • I deleted the project-specific render-files folder on the (external) project-drive.
        • I re-rendered, resulting in a fresh such folder, re-populated.
        • In Premiere, I Closed the project
        • Then in Windows I renamed the project-specific render-files folder, then back in Premiere I re-opened the project.
          • Premiere prompted with Browser titled: “Where is the File ‘Rendered – 5ac…..228.mpeg’?”
          • I selected the stated file in its newly-renamed folder, Premiere then found all the others there.
        • Result: the timeline render-region went green (i.e. “probably playable”).
      • Experiment: Once render-files are regarded as “gone”, they cannot be restored.
        • I closed the project, ensuring I did not save changes (such as the new location of the render files), then re-opened it.
          • As in the previous experiment (since I did not save changes), the render files folder could not be found.
          • Premiere thus prompted with Browser titled: “Where is the File ‘Rendered – 5ac…..228.mpeg’?”
          • This time however I simply used the [Skip All] option.
        • Closed the project.
        • Renamed the rendered-files folder back to its original name.
        • In Premiere, re-opened the project.
          • The timeline region remained red, indication no render-files were associated.
      • Experiment: Tidy migration of a project to a new location.
        • Warnig: in the case of doing a Copy (which is Windows’ default drag operation between different volumes), take care to ensure the Project (file) is not simply referencing the original preview files at the old location…
        • Drag both Project and its folders (including render-file folder) to a new location (e.g. on a new disk).
        • If name and relative location of folder are unchanged (as they ought to be, in good practice) then the files will be automatically detected and used, not even a user-prompt.
          • Just be sure though that the project isn’t simply referencing the render-files in their original location, if they are still present there.  Premiere is “lazy” in this respect.
      • Experiment: The relative location of the Rendered Files folder does matter (relative to the project file).
        • Tried putting the render files in a non-standard location.
          • The “Locate/Browse” prompt appeared
          • I located the file
          • All at first appeared well, and the corresponding section of the timeline went green
          • However, the “Composer” window simply displayed “Media Pending”.  That never went away.
      • Experiment:
        • When migrating, also need to move (or copy):
          • The Media Cache directories
            • Actually I’m not so sure about this. I tried exiting Premiere, renaming these directories and opening Premiere.  It created and repopulated the same directories in their original location, which in my case was an external drive.
          • The Source Media files
    • http://forums.adobe.com/thread/784220?tstart=0
      • I suggest marking each external with the drive letter that the user assigns to it, say Z:\. Then, whenever Z:\ is plugged in, it will always be seen as Z:\. This way, the NLE can keep up with where the Assets are located, starting with the drive letter.
      • If one is migrating Projects between computers, they will repeat this exact process in the OS of each computer.
      • Note: when doing the migration, ALL Assets, Scratch Disks, and the Project file, MUST be included on that external

Work Procedure for Migrate-ability:

  • By associating cache and XMP files with the media (or its essence), Adobe projects are migratable.  However adding such files into the BPAV/CLIPR folder structure is considered by some applications to be an adulteration of that structure, requiring their deletion.   However, such deletion on an as-needed basis is not too onerous – given it is easy to do and in any case this situaion should rarely arise in practice.
  • When using different disks, remember to re-define (in Preferences) the location of cache files etc.
    • One work-around would be to -re-set the cache location before opening any individual project.
      • Might be hard to remember to do when opening a project from within the NLE, easier to remember when double-clicking a project file in Windows Explorer.still
    • I’m not 100% sure what to do about these…
  • As noted earlier:
    • When doing the migration, ALL Assets (Sources), Scratch Disks (Renders), and the Project file, MUST be included on that external.
      • I note that this says nothing about Cache Files etc. …

HDV Tape Capture on Old Laptop: Trim & Tweak

Thursday, May 31st, 2012

I have an old 1-core Athlon laptop running XP and AVG AntiVirus.   I use it mainly for capturing from tape via a similarly-old HDV camcorder.  The tape contains Mpeg2 in Transport-Stream (TS) format, as recorded by the camera.  This capture takes place in real-time, with the tape, via FireWire (FW).  The capture process must not be interrupted – otherwise packets will be lost.  Losing packets is a nuisance rather than a show-stopper, presumably a benefit of the TS format, the only consequence is lost frames (unsure whether that affects overall duration or whether blank frames are substituted for lost ones).

To minimise chance of lost packets, I reduce as many possibilities of delay and interference as I can think of. The machine is disconnected from network (and WiFi is disabled).  For storage target, I attach a GRaid two-disk storage device, necessarily via USB2 (the machine only has one FW port, and that is used for the camera).  Additionally I Exit/Quit/Suspend a number of processes.  Firstly all relevant icons in the System Tray.  Secondly via application-specific Control Panels:

  • AVG AntiVirus: Disable elements via AVG Uder-Interface:
    • Shield
    • Identity Protection
  • Java
    • QuickStarter: Disable via Java applet in Control Panel
  • HDV Tape-Capture App (HDVSplit)
    • Change its priority from the default of “High” to “RealTime”.
      • {Uncertain if this is advisable, but the tape in reality is definitely real-time!  Anyhow, overall it worked OK}

(more…)

FCP MultiCam: PreRes not always the best standard?

Friday, December 23rd, 2011

When multicam-ing in FCP 7, one user reports <<I have found it much better to convert the 5D footage to the XDCAM EX codec instead of converting the EX footage to a ProRes 422 as the file sizes are absurd and there is still a gamma shift problem)>>.  It’s the gamma-shift that would bother me.

Additionally the user explains how to convert DSLR footage to XDCAM-EX format:

  • http://mrmagicproductions.wordpress.com/2011/12/21/multi-cam-editing-with-a-sony-ex1-or-ex3-and-5dmk-ii/
    • To Convert the DSLR Footage:
      • Copy the exact file structure from the 5D card to the desired place on your hard drive.
        • Example tree should read:  5DCAM/”DCIM”  ”MISC” (both of the previous words in quotes are two separate folders as one will see in the native card structure)/100EOS5D/”MVI_0001.MOV”  ”MVI_0001.THM” (Again…multiple files in this folder)
      • Open MPEG Streamclip (Just google it to find and download the free program) and go to “File”, “Open Files” and select as many of the .MOV files from your hard drive that you need to convert for a multi-clip.
      • Go to “File”, “Export to Quicktime”
      • At the top of the dialog box where it says, “Compression” choose one of the XDCAM EX compression methods that fit with how your footage was shot.
      • Example:  I shot at 1920 x 1080 at 24 frames per second so I will choose, “XDCAM EX 1080p24 (35Mb/s VBR)” since this also matches the settings of the EX footage.
      • Make sure your frame rate in Streamclip on the lower right area is set to 23.98 if you shot at 24fps in your session
      • Click “Make Movie” and select your target destination
    •  The following will explain how to get the footage into FCP
      • After using Log and Transfer for your EX footage, simply select “Import” under the “File” menu and browse to your new media.
      • Double click your EX clip so it opens in the Source window.
      • Go to a point you would like to use as a sync point, stop playback and hit the letter “I” for “In-Point”  Repeat this exact process with your 5D clip.
      • Select both your 5D and EX clip in the Project area where your clips are listed, right click and select, “Make Multi-Clip”.
      • Select for the clips to be synced using In-Points and you now have a multi-clip.
    • Editing in Multi-Cam Mode
      • Drag the new multi-clip into the main timeline.
      • In the main timeline, click the “RT” button to the upper left of the video tracks.  Make sure that “Multi-clip Playback” is checked.
      • In the source window, look for the button with two playback heads and an “X” between them.  It is located at the top of the window directly in the center.  Click this button and choose, “Open”.  This will sync the source and canvas windows.
      • Double click your multi-clip in the main timeline; this should open both camera views in the source window.
      • Click anywhere in the main timeline and hit the space bar.  You should now see both videos in the source window playing and available for you to click on the angle you want.
      • When you’re done you should highlight everything in the main timeline, right click and select, “Collapse Multi-Clip”.  Don’t worry, you can easily turn it back on to continue multi-cam editing; this will just save on RAM.

Camera Sensor Sizes (Crop Factor, APS-C)

Wednesday, November 9th, 2011

What is a four-thirds sensor?  Is a 35mm sensor measured across the diagonal (like a TV screen) or width etc.?  What is ASPC?

The answers I found are:

  • http://www.dpreview.com/learn/?/Glossary/Camera_System/sensor_sizes_01.htm
    • Sensors are often referred to with a “type” designation using imperial fractions such as 1/1.8″ or 2/3″ which are larger than the actual sensor diameters. The type designation harks back to a set of standard sizes given to TV camera tubes in the 50’s. These sizes were typically 1/2″, 2/3″ etc. The size designation does not define the diagonal of the sensor area but rather the outer diameter of the long glass envelope of the tube.
    • There appears to be no specific mathematical relationship between the diameter of the imaging circle and the sensor size, although it is always roughly two thirds.
      • {The article includes a look-up table for the exact figures}
    • A “35” mm sensor is actually 36 mm on the width;  the height being 24 mm and the diagonal 43 mm.
  • http://en.wikipedia.org/wiki/APS-C
    • Advanced Photo System type-C (APS-C) is an image sensor format approximately equivalent in size to the Advanced Photo System “classic” size negatives. These negatives were 25.1 × 16.7 mm and had an aspect ratio 3:2.
    • Sensors meeting these approximate dimensions are used in many digital single-lens reflex cameras, in addition to a few large-sensored live-preview digital cameras and a few digital rangefinders.
    • Such sensors exist in many different variants depending on the manufacturer and camera model.  All APS-C variants are considerably smaller than 35 mm standard film which measures 36×24 mm. Sensor sizes range from 20.7×13.8 mm to 28.7×19.1 mm. Each variant results in a slightly different angle of view from lenses at the same focal length and overall a much narrower angle of view compared to 35 mm film.
    • This is why each manufacturer offers a range of lenses designed for its format.
  • http://philipbloom.net/2009/10/01/5dmkii-or-the-7d/
    • Philip includes a diagram comparing sensor sizes.