Archive for the ‘XML’ Category

Adobe Premiere – Source Media Adulteration

Saturday, November 10th, 2012

Sometimes Adobe Premiere may write to a source media file or proprietary folder-structure.  This may be considered a non-problem in most situations, but it is nevertheless worth being aware of.

This is nothing hidden, surreptitious or unheard-of, it’s explained in Adobe’s Help text and documentation.  However the potential consequences may not be obvious to a new user.  It may arise at various points of what we may regard as the greater process (workflow/manual) of ingesting media, consisting not only of Premiere’s Import of media but also subsequent manual updating of metadata or indeed automatic analysis such as speech recognition.  As of CS6 it can also occur as a result of adding Markers in Adobe Prelude.

Premiere likes to add and manage metadata for each media file.

  • The good side of this is that it value-enhances these files, making them easier to locate, navigate and use, potentially increasing workflow productivity and asset usage.
  • But there’s also a dark side – not necessarily Adobe’s fault (e.g. their approaches may well adhere to official media specifications) – but it may be that so-adulterated media files may cause difficulties to other applications (e.g. that may not fully take on board such standards).
    • In my experience, in the past, some (possibly poorly-written, but nevertheless useful) applications have refused to work with metadata-augmented files, again holding up productivity, in this case while the user figures out the issue and works out how to strip this data out, in order to progress.
    • Technically a non-problem, but potentially consequential to a workflow, backup software will (rightly, from its point of view) see the metadata-change as a file-change (e.g. as a consequent file-size change) and consider that the files have been updated.   Left to itself, the backup process (depending how it works/configured) will overwrite any previous copy of the files (e.g. the original files).   Even if the backup process prompts the user to confirm this, the naive user may be uncertain what to do,

Also, the user has the option at their discretion for Premiere to automatically store additional files (such as cache files and metadata sidecar files) alongside source media files.

  • In the case of media represented as a straightforward single file (like a .jpg or .mpg file) this does not affect that media.
  • However some media (e.g. TV-playable DVDs or XDCAM-EX video media) are stored as proprietary folder structures with defined contents, part of these contents being essence files (e.g. .vob files or .mp4 files) while other files alongside them etc. in that structure (e.g. DVD’s .IFO files or XDVCAM-EX’s .SMI files) contain metadata or index into them etc.  In this case, the consequence of adding further files into the structure will (in my experience) be acceptable to some applications and media players but not to others, which regard it as “pollution”, and may then reject such structures.  Certainly in the past I have seen this happen in some software applications and also even some (mostly old) TV DVD players.

This is a case for “situational awareness”: if one is aware of the nature and potential consequences of the adulteration (be it regarded as pollution or enhancement, depending on the workflow situation), one is then in a better position to be able to avoid or fix any asociated issues. (more…)

Mobile Video Editing Hardware: Thoughts, Ideas & Dreams (continued)

Friday, January 13th, 2012

Following-on from my earlier post, Mobile Video Editing Hardware: Thoughts, Ideas & Dreams, where I considered an eventual migration from my laptop to a luggable PC, my thoughts veered (possibly having spotted cash-icebergs among them) towards an alternative solution:

  • Use the laptop for lightweight editing & compositing.
  • Use the desktop as a number-crunching RADI-attached server.

The two could be linked by:

  • Remote access / remote sessions (some of which via smartphone)
  • DropBox, e.g. have an active folder where I can drop Adobe Premiere XML and have it processed remotely by Adobe apps installed there.

Some links:

  • http://www.theregister.co.uk/2012/01/11/virtualisation_at_home_part_2/
    • (There’s no equivalent “_part_1” page.  I guess it’s just “Part 2” of that guy’s story).
    • DIY virtual machines: Rigging up at home, by Trevor Pott, 11th January 2012 14:33 GMT
    • Personal Virtual Machine (PVM) (in use) for about seven years with retail boxed version of Windows XP.
    • VM has been moved from virtualization platform to virtualization platform over the years … the most recent incarnation … inside Hyper-V.
    • …nothing beats Windows Server 2008 R2. It comes with a top-notch virtualisation platform (Hyper-V), and added RemoteFX support with Service Pack 1. You can still use the desktop operating system for all your HTPC needs, and a single Server 2008 R2 Standard license allows you to run both a host copy and a single virtual instance of Server 2008 R2.
    • In my case, the host instance does little more than play movies on the projector via VLC. The virtual instance of Server runs my Plex media server, and aggregates my many storage devices into a single share using DFS.
  • Shuttle Inc (Taiwan)

Training: Den Lennie’s “Music Video” Experience

Thursday, October 27th, 2011

I attended, working on one of the camera units.  Had a great time, learnt lots, at all sorts of levels.  Even how to make good use of the Movie Slate application on my iPhone!  Link: http://www.fstopacademy.com/

Avid MC: Bundled Tools & Apps: Their Purpose

Thursday, September 8th, 2011

When you purchase Avid Media Composer, you also get a set of other applications, whose purpose (at least to the newbie) is not immediately obvious.  So I did some investigation and produced a summary of them, as below.  I have no experience of actually using them, I just trawled ReadMe files and (mostly) the web.  Here are my (interim) conclusions:

  • Avid TransferManager – Is e.g. for uploading to a Playback Server [http://blog.davidesp.com/archives/274]
  • AMA – the camera-specific AMA Plugins (e.g. for Sony XDCAM) are no longer bundled with MC, you have to download and install them separately. [http://blog.davidesp.com/archives/316]
  • Avid MetaSync automates the inclusion of metadata (expressed in suitable XML formats) into Avid editing systems, including synchronisation with video and audio. The metadata can be anything from subtitles / closed captioning to synchronized entertainments such as lightshows or simulator rides.   [http://blog.davidesp.com/archives/334]
  • Avid MetaFuze’s primary, if not only purpose is to prep files for Media Composer use – an “outboard importer”.  Avid’s article at http://www.avid.com/US/industries/workflow/MetaFuze summarises it nicely.  Though bundled with Media Composer, it is also available free. That means for example that preprocessing work (e.g. generation of burnt-timecode proxies and online files) can be generated (e.g. in DNxHD) by anyone whether or not they have an Avid system.  Potentially then a great option for breaking up work into collaborative / parallel workflows. [http://blog.davidesp.com/archives/335]
  • Sorenson Squeeze – a well-known compressor/encoder, bundled as part of Avid Media Composer (MC) but also an independent product in its own right. Avid MC5.5 specifies version v6.04 but further updates are available from Sorenson itself.  There is a free-to-Avid-users update from v6.x to v6.5.  The latest version is v7.0 (with CUDA).  Presumably these later versions are officially unsupported by Avid (but how much does that matter in practice?). [http://blog.davidesp.com/archives/337]
  • Avid EDL Manager imports and exports EDL (in various flavours) – from/to a bin (e.g. thumbnails storyboard layout?) (or a Sequence or MXF file?).  It can be run stand-alone or from within Avid.  EDLs are somewhat of a hangover from the past, so it’s unlikely to be of much use in my case, but worth knowing about as an option, and as such still features in other people’s current workflows. [http://blog.davidesp.com/archives/350]
  • Avid Film Scribe generates Cut Lists and Change Lists (used in transfer from video edit to film edit) in more contemporary formats than EDL, e.g. XML formats involved in VFX / DPX workflows (? I am on very unfamiliar ground here ?).  It can generate such formats from a Sequence and also it can be used to translate between some formats.[http://blog.davidesp.com/archives/352]
  • Avid Log Exchange (ALE) is an Avid log file format that has become a de facto standard in the post industry. It is a text-based metadata exchange format used in applications from telecine to standalone logging applications, and is supported by many NLEs.  The ALE format is based on a Comma or Tab -delimited file format. [http://blog.davidesp.com/archives/353]
  • Avid After Effects EMP is (not a disruptive elctronic weapon but) an Avid-supplied plugin for Adobe After Effects allowing that application to use a DNA family video output box such as Mojo (“ordinaire”) or Nitris to provide External Monitor Preview (EMP) on a monitor.  Helpful in order to make use of that Avid box for the Adobe After Effects application, both for convenience and consistency.  Unfortunately it does not work with the more recent DX family, such as the Mojo DX box. [http://blog.davidesp.com/archives/354]
  • The Avid DNA Diags application is for diagnostics on DNA family e.g. Mojo “ordinaire” (not DX) [http://blog.davidesp.com/archives/355]
  • The Avid Quicktime Codecs extend QuickTime for encoding and decoding to/from Avid codecs such as DNxHD.  Essentially they add such formats to QuickTime on your system.  The LE codecs are “Light Edition” – only needed on systems where Avid is not already installed.   [http://blog.davidesp.com/archives/356]
  • Avid Media Log is a standalone app supplied with Avid systems enabling assistants on non-Avid machines to select and log raw (as opposed to RAW) footage in a manner that can easily be transferred into an Avid session/system elsewhere, where the result appears as an Avid Project.  Apparently, Media Log is much like the digitize tool on Media Composer.  But I’ve never used that either… It can output e.g. to ALE (explained below) and hence e.g to other NLEs.  [http://blog.davidesp.com/archives/357]
  • Misc “Avid Downloads” (?) Looking at  my Avid Downloads page, there is a much larger list of items than I expected, and suspect that many of them are not relevant.  For example, what is Avid Deko?  It’s listed on my Avid Downloads page, though I don’t know if I would be able to activate it, or whether it would be worth the trouble.  It’s listed as Deko 2200.  So I googled and YouTubed about it…  Impression: that version (2200) is very obsolete. [http://blog.davidesp.com/archives/332]
  • On my web “travels”, I discovered a great article entitled “The Avid Ecosystem” at [http://digitalfilms.wordpress.com/the-avid-ecosystem/], listing many of the resources for the Avid world: links, tutorials, filters, applications, training…
  • It’s helpful to see some of the above items in the context of illustrative workflows, e.g.:

Avid MetaSync – Description & Role

Saturday, August 13th, 2011

Avid MetaSync automates the inclusion of metadata (expressed in suitable XML formats) into Avid editing systems, including synchronisation with video and audio. The metadata can be anything from subtitles / closed captioning to synchronized entertainments such as lightshows or simulator rides.  For closed captioning, it works particularly well with Final Draft and Evertz ProCap (both sold separately).  Thinks: can it be made to work with CeltX or some home-grown VBA-based XML-generator from Access or Excel ?

  • http://www.avid.com/us/partners/AvidMetaSync
    • Avid MetaSync™
      • Avid MetaSync allows users to synchronize virtually any kind of metadata with video and audio content during the postproduction process. The MetaSync feature is now standard within Windows-based versions of Media Composer and NewsCutter products.
      • Avid’s MetaSync technology enables postproduction professionals to tap into new revenue streams such as closed captioning, subtitle text insertion, digital rights management, and interactive TV content as well as “converging media” such as motion simulation rides, movie theatre effects, internet devices, and interactive toys. All of these applications can be made to react to triggers embedded within standard film, video and TV content.
      • As long as a file type or process can be represented in the appropriate XML format, it can now be imported into Avid editing systems using the MetaSync feature and synchronized with video and audio. In the timeline, pointers to the original file can be positioned, trimmed and edited just like video and audio clips. The file can then be launched in its original format from directly within the Avid system to be viewed or updated, and any changes made are instantly reflected in the timeline and bin.
    • Avid MetaSync Workflow
      • In today’s typical scenario, one team will work on a show’s narrative content while a separate team works on the metadata elements of a program in a remote location, often at a different time altogether. Using the MetaSync feature, the Avid Editor, linked to a LAN or WAN, can edit metadata directly into the program while it’s under development. This allows the Editor to provide real-time feedback to the metadata content developers during the postproduction process.
      • Using the MetaSync refresh capabilities, the Avid Editor can update the metadata content within a show as it’s being refined. This allows the Editor to suggest changes based on how well this data is working within the actual video and audio elements of the program. The end result is higher quality programming with metadata elements more finely integrated with standard video and audio content.
  • http://www.camcorderinfo.co.uk/content/avid_metasync_automates_creation_04_18_03.htm?redirected=www.CamcorderInfo.com%2Fcontent%2Favid_metasync_automates_creation_04_18_03.htm
    • The Avid MetaSync technology will work in conjunction with scriptwriting software provided by Final Draft, and ProCap authoring systems from Evertz, to create closed captioning and subtitling directly in the Avid timeline during the editing process. This practice will eliminate the separate step of incorporating this type of information into programs after finalized broadcast masters have been created.
    • In their press release they say the captioning and subtitling creation process for editors using Avid systems, is a simple as, importing the script dialogue directly from Final Draft and aligning it with the appropriate video content on the Avid timeline. Once editing is complete, both the video and the captioning information are fed through the Evertz ProCap system and caption-encoder, which insert the captions and subtitles into the final broadcast format, according to industry standards and specifications.
    • ‘‘Governments worldwide are mandating the adoption of closed captioning, and broadcasters have been looking for easier ways to streamline what has traditionally been a labor-intensive and time-consuming process,’’ said Ray Gilmartin, senior product marketing manager for Avid Technology. “Avid offers the perfect solution to meet their needs. Not only do MetaSync, Final Draft and Evertz ProCap make creating closed-captioning and subtitling almost effortless, but the process can now begin before the program is finished, saving producers valuable time as they strive to make tight broadcast deadlines.”
    • Avid MetaSync comes standard with all current versions of the Symphony, Media Composer, Avid Xpress and NewsCutter, systems, as well as the new Media Composer Adrenaline, NewsCutter Adrenaline FX, Avid Xpress Pro and NewsCutter XP systems announced today. Final Draft and Evertz ProCap are sold separately and are expected to work with Avid MetaSync to create the automated closed captioning and subtitling workflow in the second quarter. For more information about Avid MetaSync or Avid’s other products and services, please visit www.avid.com.

FCP, Spreadsheets & XML: Some web-links

Tuesday, August 17th, 2010

Wondered whether there was a way to integrate spreadsheets with media logging and importing (transferring) etc. in FCP.

Pros & cons of Device Explorer in Sony Vegas

Saturday, May 1st, 2010

[http://www.dvinfo.net/forum/sony-xdcam-ex-cinealta/235167-vegas-9-imports-native-xdcam-ex-files-2.html]

  • as of May 2009:
    • Technically, you “can” edit the .mp4 files right from the card. You’d need to drill down through the directories via the standard Vegas Explorer tool (not the new Device Explorer), find your .mp4 clip, and bring it into your project.
    • “We do not currently support shot markers from EX in the Vegas Pro 9 Device Explorer, but it is on our radar.”
    • Spanning clips does not work properly for everybody (could in principle be due to their circumstances as much as the app).  Recommended to join these together using ClipBrowser thenexport as MXF for NLEs.  … It is really the same concept as (FCP’s) XDCAM Transfer except instead of re-wrapping as [.mov] it re-wraps as [.mxf].
    • (In the case of FCP) … the metadata is part of the MOV after … re-wrapping the file for FCP.  (Possibly) Vegas had a problem with managing the metadata and their solution was just to (import the) native (essence/mp4) files.

My own experiences:

  • A long shoot gets listed as a sequence of smaller clips, corresponding to the separate [.mp4] files recorded by the camera.  This is known as a spanned clip.  Each of the smaller clips is of size no more than around 3.5 GB.
  • Device Explorer import results:
    • Clips with names like [929_1332_01_20100318_191600] i.e. having datetimes.
    • These clips consist of the following files, with main file name as per the clip:
      • XDCAM-EX:  [.mp4], [.xml]
      • AVCHD: [.mts] (but no clip info files).

Excel, ODBC Query, PHP, XML

Wednesday, January 20th, 2010

Excel, ODBC Query, PHP, XML