Blog

I want to put someone inside a virtual world (based on their own paintings), ultimately to be rendered out as a Stereographic 3D movie, but with development in progressive stages/generations, initially based on readily available standard tools and techniques, later proceeding to specialist 3D modelling apps etc.  How to proceed?  Some possibilities that come to mind are:

• Initial development using only “Planar 3D” as in After Effects
  • I believe this is possible to some extent, using native features of AE.
  • In my earlier post on Adobe Production Premium CS5.5 - Orientation, I noted the existence of Mocha LE, a tracker, included in that suite.
• Subsequent development using “Full 3D” as in Blender etc.  Some relevant previous entries in this blog are:
  • Pre-Visualization Apps (for Storyboard / Animatrix / Virtual Studio)
  • 3D Modelling / Animation / Rendering Apps

So I did a web-search: Google: [after effects 3d]

World / Object Creation:

• http://magazine.creativecow.net/article/building-a-3d-world-in-after-effects
  •  Real (green screened) rap artist and audients, where the rap artist appears on a virtual a giant stage surrounded by replicated audients.
  • Explains the conceptualization, storyboarding, and high-level aspects (no screenshots) of designing a world capable of representation in After Effects, along with building and testing experiences.
• http://www.computerarts.co.uk/tutorials/fake-3d-after-effects
  • How to make a truck out of a single flat surface with cut-out sections and folds.  Like you would make one out of a single sheet of cardboard (like a cornflake packet).
  • Links to tutorial and support-files.
• http://www.videocopilot.net/tutorial/3d_city/
  • Similar to the truck, flat sheet (”texture”, obtained from an actual photo, realigned and stretched to remove perspective effect in photo) folded into a 3D building object.
  • Video tutorial and project files are provided.
  • {VideoCopilot.net has both free stuff (tutorials etc.) and pay-stuff.  Well-worth a check-out.}

Tracking:

• DaVinci Resolve Lite - free
  • I covered this in an earlier post.
• http://forums.creativecow.net/thread/2/1003284#1003284
  • Re: After Effects Tracking (by Todd Kopriva, Adobe Tech. Support, on Jul 10, 2011)
    • If you have After Effect CS5, you have mocha—and it’s often the easiest and best way to do tracking for the purpose of compositing.
    • Here’s a movie that I made just to show how simple using mocha is:
      • http://www.video2brain.com/en/videos-4351.htm
• http://forums.creativecow.net/thread/2/1006635
  • Q: “Mocha, or Camera Tracker?” (by andrew donaldson on Sep 8, 2011)
    • {Mocha (LE) comes with CS5.5, Camera Tracker is a third-party add-on}
    • I need to create something similar to this:
      • http://www.vimeo.com/2445692
    • …. would liked to use similar techniques, especially with the web pages tracking the buildings in city shots.
    • I have tried to use mocha to track the side of a building, but its not proving ideal. Obviously the motion tracker inside AE won’t give me the depth/perspective.
    • So what’s the best way to do this?  … Is Syntheyes/Camera tracker the way forward?
  • A1: (by ben g unguren on Sep 8, 2011 )
    • The general rule of thumb is that Mocha works great if your graphics are ATTACHED to an EXISTING surface (like a logo on the side of a building, or changing the words on a sign). If you’re trying to add something in 3D space (like graphics that “hover” around the building, and seem to actually be there), then you need a 3D solution.
    • Mocha and AE’s internal trackers give you 2D solutions. Mocha’s solutions are a bit more sophisticated, producing corner-point information that mimics 3D, but [this is key] it doesn’t produce a 3D camera.
    • Syntheyes and similar apps will give you an animated 3D camera as well as target points that simulate the world you’re tracking (target points for the ground, buildings, etc — whatever you’ve managed to track and can get a 3D “solution” for). This is A LOT more information than what Mocha or AE’s internal tracking can get you.
    • One other point: when the camera is only panning and tilting (not actually changing it’s own position) then a 2D solution can (sort of) mimick a 3D camera solution. So if all you’re doing is panning and tilting, then you could track that in Mocha, then use that data to animate objects (that are given perspective, for instance). You would be able to achieve a lot of the graphics in the video you linked to using that technique, as they’re using a lot of static cameras.
  • A2: (by Tudor “Ted” Jelescu on Sep 8, 2011):
    • I agree with Ben.
    • In most of the shots from your example Mocha can be used. I suspect that some of those shots where not really video files, but still images cleverly transformed in a 2.5d comp where camera moves can be animated in AE - so no tracker there.
    • Here’s a good tutorial for what Mocha can do:
      • http://www.videocopilot.net/tutorials/magic_tracking/

Generally:

•  http://forums.creativecow.net/adobeaftereffects
  • AE forum, includes posts about tracking etc.

Questions:

• What should and does it consist of?
  • Including anything non-obvious e.g. “tucked away”, “additional Downloads”, “obscure” etc.
• ???

Answers (as far as I can tell…):

•  _Support & Communities
  • Looks at first sight like any application’s Help dialog, but on deeper inspection is also connected web-wise and answers to questions are not restricted to Adobe products.
• After Effects
  • In contrast to Premiere, which is for conventional light-touch editing, this is for heavy effects and compositing etc.
• After Effects Render Engine
  • Render farm: Network rendering with watch folders and render engines.
  • Previously, it was possible to install render engines on as many machines as wished, but not so under CS5.5, where a separate serial number must be obtained for each machine.  For small guys like me that makes it pretty useless.   It seems likely that a more flexible option will exist in future versions.
• Audition
  • Audio editor, derived (many years ago) from CoolEdit.
• Bridge
  • A combination of media file manager, media manager, metadata editor, also does some kinds of media processing.
  • Can be run standalone or from within apps e.g. Premiere: [File > Browse in Bridge…]
• Color Finesse
  • A multi-host (including Premiere & AE) plugin that goes above and beyond typical NLE color correction.
  • On my system it only appears to be available under AE.
• Device Central
  • Simulates media appearance etc. on a range of devices such as cellphones.
• DigiEffects FreeForm
  • Manipulate a flat object into almost any shape using displacement maps and meshes in 3D space.
  • Examples: flowing cloth, animated loose filmstrips, rippling fluids, terrain flyovers, welded metal, morphs, reveals.
• Encore
  • DVD authoring.  Menus can be created in Photoshop (using special layering techniques).
• Extension Manager
  • Manage extensions (broadly like plugins) associated with various Adobe programs.
• ExtendScript Toolkit:
  • Adobe workflow command-script editor
  • An IDE (along the lines of Visual Studio) for scripts in Adobe Bridge, itself serving to automate workflows involving multiple Adobe applications.
• Flash Professional
  • A multimedia authoring program to create content for Flash-enabled platforms/devices.
• Flash Catalyst
  • A designers’ tool for creating the user interface for Rich Internet Applications.  Primary function of being a GUI composer for Adobe Flex components.
  • Can import Photoshop, Illustrator, Fireworks, or Flash XML Graphics (FXG) files.
• Help
  • A help resource not only with local help-documentation but also capable of searching beyond, even beyond Adobe, to find solutions.
• Illustrator
  • Vector graphics editor e.g. useful for typesetting and logo graphics design. It is the companion product of Photoshop.
• KeyLight
  • An advanced chroma-keyer, tackling reflections, semi-transparent areas and hair.
• Media Encoder
  • Encodes audio and video media content.
• Mocha
  • Stand-alone tracking and roto tool to help solve problematic shots challenging the built in tools of After Effects and Final Cut by bringing advanced planar tracking and matte creation tools
• OnLocation
  • Direct to disk recorder / logger also acting as monitor with waveform monitor and vectorscope.  And no doubt much else.
• PhotoShop
  • Such a big noun that it became a verb…
• Premiere Pro
  • Primary editing app, with emphasis on productive cutting and smooth playback etc., minimising the need to render, leaving heavy effects and compositing etc. to AE.
• Story
  • A collaborative script development tool/service. There is an application for working alone on an offline version and a web-based service where you can sync up with an online version.
• Ultra
  • Vector-keyer (simple-to-use effective chroma-keying) that was once a standalone app by Serious Magic, now available as a plugin within Premiere (but not AE).  I get the impression it is regarded (or at least branded) as simple to use but ultimately less sophisticated/capable than KeyLight (???).
  • Serious Magic used to highlight its capabilities regarding reflections, semi-transparent areas and hair…
• Utilities
  • ExtendScript
    • An integrated development environment (IDE) for the creation and debugging of JavaScript code for Adobe Bridge, to facilitate workflow-enhancing automation of tasks between elements of Creative Suite.
  • PixelBlender
    • A simplifying basis for implementing image processing algorithms (filters or effects) in a hardware-independent manner, taking advantage of parallel processing / GPU etc.
    • Downside to commercial developers is that code using it may be more visible / understandable.

I came across some old videos I shot on a Nokia N95 and pulled these into Adobe Premiere.  However the individual video clips were each listed with a different framerate, hovering vaguely around 29 fps (27.08 up to 29.45).   Questions:

• What does that even mean?
  • From web-search, it sounds like it’s an average, and N95 framerates within a given recording can vary wildly
    • e.g. between 6 and 38 fps.
• How do various apps etc. handle such material?
  • YouTube:
    • In 2009 at least, it sounds like YouTube went for the minimum fps in any such clip.
  • Adobe Premiere
    • Seems to go for the average
      • I dragged a N95 clip on the “New Sequence” button and the resulting sequence had the clip’s average framerate.
    • Presumably just duplicates/drops frames as required to maintain the Sequence’s framerate.
  • GSpot (video analyzer):
    • For a clip reported by Adobe Premiere to be 28.81 fps, GSpot reported it to be 29.412 fps.
      • Misleading info from one or other or both…

(more…)

I’ve begun by re-creating from-scratch a project which I already have in Sony Vegas.

• Media Browser:
  • aMedia > DoubleClick
    • Goes to Source Monitor
• Source Monitor:
  • Adjust In/Out points e.g. as per [View Details] window in Sony Vegas
• Drag from Source Monitor to Timeline
• Use timecode from elements in Sony Vegas project as a starting-point.  Drag Dissolve transitions (Effects) from bottom-left hand pane in between clips.  Trim transition to preference, either by drag or numbers.  Pretty easy.
• Timeline:
  • [Clip > RtClk > Scale to Frame Size]
    • Then drag scale/pan/crop rectangle to required size/zoom level.
• [File > Export > Media]  = Ctrl-M
  • This created [C:\Users\David\Documents\Adobe\Premiere Pro\5.5\Sequence 01.avi]
• Renamed the Sequence as [Intro]
  • Now it saves to [Intro.avi] or the following time to [Intro_1.avi]

Pretty intuitive really.  Next I’d like to find the levels and color correction features, then (following the inevitablegrain created by that process) some means of running the Neat Video temporal denoiser.

I have a project shot on a Sony EX3.  SO I have XDCAM-EX footage.

Opened an Adobe Premiere project, set its Sequence settings to XDCAM EX HQ 1080i50

• Wondered immediately what that implied.  Like OK the source is XDCAM-EX which is Mpeg2 encoding inside a MP4 container, but why does the Sequence care how the source is stored?  Surely it only needs to know things like the format is 1080i50 then it can store any intermediate files in DNxHD or Cineform or whatever Adobe prefers.  I am very confused by this kind of thing, just as I was in FCP.  Maybe it’s obvious or maybe “I think to much”.
• Adobe has a thing called Import and it can (I discovered) accept MP4 files from XDCAM-EX’s BPAV folder-structures (deep down within the CLIPR subfolder).  But I know that is a stupid way to go. The MP4 files are but the “essence” that is “stitched together” (mixed metaphors or what?) by the likes of SMIL and XML files.  It’s only at the latter level that smooth continuum happens.
• Enter Adobe Premiere’s Media Browser. I “knew” there had to be something like that.  I discovered it via http://wwwimages.adobe.com/www.adobe.com/content/dam/Adobe/en/products/premiere/pdfs/cs5-premiere-pro-sonyxdcam-wfg.pdf which itself I discovered via Bing search on [sony xdcam-ex adobe premiere cs5.5 workflow].  OK to get XDCAM-EX footage into an Adobe Premiere project you do [WIndow >Media Browser] or else Shift-8, then don’t expect some window popping-up or anything, just instect the [Media Browser] tab at the lower-left of the GUI screen.  Drill-down to the required recording and double-click.  The media appears in a Source Preview window (I wonder but don’t mightily care what Adobe calls it).
  • OK I do care a bit really, and according to an Adobe video tutorial, it’s called a Source Monitor.
  • Initially it was too zoomed-in, presumably displayig at 1:1 (pixel).  ”Zoom to Fit” was but a right-click away…
• You can drag from Source Monitor to the Timeline or to other places.
• I tried that with some EX3 footage where I pan across the front of the famous Wembley Stadium, UK.  In Sony Vegas (my erstwhile “comfortable old shoe”) it snatches and drags.  In Adobe Premiere, as in Sony Clip Browser, it pans smoothly.  Guess where I’m heading…

Following-on from my earlier post, Mobile Video Editing Hardware: Thoughts, Ideas & Dreams, where I considered an eventual migration from my laptop to a luggable PC, my thoughts veered (possibly having spotted cash-icebergs among them) towards an alternative solution:

• Use the laptop for lightweight editing & compositing.
• Use the desktop as a number-crunching RADI-attached server.

The two could be linked by:

• Remote access / remote sessions (some of which via smartphone)
• DropBox, e.g. have an active folder where I can drop Adobe Premiere XML and have it processed remotely by Adobe apps installed there.

Some links:

• http://www.theregister.co.uk/2012/01/11/virtualisation_at_home_part_2/
  • (There’s no equivalent “_part_1″ page.  I guess it’s just “Part 2″ of that guy’s story).
  • DIY virtual machines: Rigging up at home, by Trevor Pott, 11th January 2012 14:33 GMT
  • Personal Virtual Machine (PVM) (in use) for about seven years with retail boxed version of Windows XP.
  • VM has been moved from virtualization platform to virtualization platform over the years … the most recent incarnation … inside Hyper-V.
  • …nothing beats Windows Server 2008 R2. It comes with a top-notch virtualisation platform (Hyper-V), and added RemoteFX support with Service Pack 1. You can still use the desktop operating system for all your HTPC needs, and a single Server 2008 R2 Standard license allows you to run both a host copy and a single virtual instance of Server 2008 R2.
  • In my case, the host instance does little more than play movies on the projector via VLC. The virtual instance of Server runs my Plex media server, and aggregates my many storage devices into a single share using DFS.
    • Presumably “DFS” refers to Microsoft’s Distributed File System.
• Shuttle Inc (Taiwan)
  • (MBR’s suggestion)
  • http://uk.shuttle.com/products/productsDetail?productId=1477
    • Small, quiet, hardware-configurable systems.
    • May be a sensible basis for a modular luggable?
  • http://www.okobe.co.uk/ws/product/Shuttle+SH67H3+Barebones+Personal+Computer+FH67+Socket+1155+Intel+H67+Black/1000065737?ref=1q4bs
    • Shuttle SH67H3 Barebones Personal Computer (FH67 Socket 1155 Intel H67) (Black)

Want a mobile “suitcase” editing system, something more (and more expandable) than a laptop but not too expensive.  Primarily to be used for Adobe CS5.5 for media enhancement / editing / compositing etc.

Nearest I found was NextDimension’s range around $7000 I think (but just guesswork - could be way off - would need to get a quote).   That would (if true) be around £4500 at current rates.  Plus import…  NextDimension call such machines “flextops” (Maybe they coined the term? Google searches on it mostly come up with them.)

Apart from the (mil/broadcast-lite but me-heavy) price, it might possibly be undesirably heavy to lug around much.   If so (just guessing, not assuming), it would make more sense to go for a modular quick-setup system.  So, starting to “think different” in this direction:

• Standard tower, capable of taking new CUDA etc. graphics cards etc. as they emerge, but no need for more than say a couple of disks, maybe if SSD could even get away with just a single disk? (For system and media - inadvisable for traditional disks of course, what about for SSD’s?  I have much to learn about SSD’s though).
• “Laptop-Lite” to talk to it.  With robust shuttered-stereoscopic HD monitor.
• Gigabit network to NAS fast storage (SSD and/or RAID ?).

Maybe in that case it would be far more logical/affordable to use an existing laptop as a client working together with a luggable tower server, sufficiently light and robust for frequent dis/re -connection and travel.  And remote access of course (no heavy data to be exchanged, assume that’s already sync’d).  And some means to easily swap/sync applications and projects (data) between laptop and tower, giving the option to use just the (old) laptop on its own if needed.  All such options are handy for the travelling dude (working on train, social visits etc.) who also occasionally has to do heavy processing.  Then would just need a protective suitcase for the tower, plus another one for a decent monitor for grading etc.

I certainly won’t be spending anything just yet, but it’s good to have at least some kind of “radar”.

(more…)

• http://magazine.creativecow.net/article/opening-my-mind-has-opened-doors
• http://magazine.creativecow.net/article/eitheror-both
• http://www.tolisgroup.com/products/

• http://forums.creativecow.net/thread/202/883396
  • i was self taught for after effects, as many were before non-linear editing became so affordable that any school could afford to start up a digital media program (even jr. high/middle schools).
  • you might look into an after effects book by chris & trish meyers called ‘creating motion graphics’ (i actually didn’t have that book when i was learning, but i’ve since heard that it’s one of the best for learning ae).
  • total training has a good series after effects of tutorials
  • lynda.com is good resource too.
  • and, of course, there are lots of ae tutorials here at the (creative) cow.
  • aharon rabinowitz has many geared towards the fundamentals of ae. look into some of his workflow tuts and other earlier ones where he covers some basic essentials.
• Book: Motion Graphics with Adobe Creative Suite 5 Studio Techniques, Richard Harrington and Ian Robinson
  • http://www.peachpit.com/articles/article.aspx?p=1660189
    • Excerpt: Repairing and Retiming Footage

What tools are available for this?  What are their relative merits / costs (in all senses)?

• First I tried Boris RED but encountered issues.
• Next I investigated AviSynth, discovering two approaches:
  • MSU’s Frame-Rate Convertor (FRC)
  • MVTools2-based scripts; the latter was the more complex but for me it worked best.
• Next I looked at MotionPerfect, which I purchased years ago from Dynapel back in Standard Definition days.
  • Nowadays sold by Gooder Video
    • They sell version 4.3.1
    • I have version 4.3.0
    • I updated it to 4.3.1
      • That shows a different icon/logo to the Dynapel version of that same product.
      • It works the same as far as I can see.
• Another day(s):
  • Boris RED (4 & 5) via NLEs
    • Adobe
    • Avid
    • VegasAfter Effect
  • Kronos ??
  • Twixtor ??.
    • Some people claim this is the fastest motion-estimation-based method.
  • Avid ???
    • e.g. locate the (years-old) Avid velocity / warp envelope demo featuring a surfer ???

(more…)