Archive for the ‘Deshaker’ Category

Run&Gun Shooting with Fast Shutter, then Deshake and Add Motion Blur in Post

Tuesday, July 16th, 2013

Like the title says (providing you have time for post-production), it is sensible to do Run&Gun Shooting with Fast Shutter, then Deshake and Add Motion Blur in Post.

In Summary:

  • For a bumpy aircraft flight, I shot with 1/50 second shutter and stabilized it in post.  The inevitable result, though more pleasing than the non-deshaken footage, exhibited shimmering effects due to motion blur on the various objects in the frame.
  • The shimmering could of course have been vastly reduced by shooting with a much faster shutter-speed.  One reason I didn’t was to avoid the staccato “Saving Private Ryan” look.  However I now realize that a convincing motion blur can (at least in theory, until I test it) be artificially introduced in post, following the de-shaking.
  • Some options exist for artificially introducing motion blur in post:
    • (Some degree of risk: Not perfect, but the imperfections might not necessarily be noticeable, or at least they may be less so than if not following this overall path).
    • RE:Vision Effects’ plug-in ReelSmart Motion Blur (RSMB).
      • Convenient, as it is a plug in for Premiere as well as After Effects (and various NLES/Tools).
    • After Effects’ Time Warp plugin (even if not warping time) has a Motion Blur function.
    • Comparison between them:
      • http://www.screenr.com/T3d
        • Time Warp works, but painfully-slowly.
        • RSMB is very-much faster.
        • Sometimes they get fooled when a motion-vector suddenly changes, leading to odd artefacts, though possibly not too noticeable in a changing/moving image.
  • Tips for reducing the problem at shoot-stage:
    • Shoot with elbows on a bean-bag.
    • If camera has rolling-shutter effect, then some cameras reduce this more when you increase the framerate than when you decrease the shutter angle by equivalent amount.
  • Potential methods for removing blur from the original footage, by deconvolution:

In Detail:

I shot a flight in an aeroplane (a Tristar) as a passenger that happened to have a camera (A Sony Z1)  as opposed to a proper production.  I used the Z1 because it had a (small) CCD sensor, thereby avoiding the rolling-shutter effect associated with most (not all) CMOS-sensor cameras.

Lazily, I left it at its default setting of 1/50 second.  I had a vague idea that I didn’t want a fast-shutter staccato “Saving Private Ryan” look, but that’s as far as my thinking went.  Instead of increasing shutter-speed, I applied the Z1’s built-in ND filters so I could keep keep its iris reasonably wide, so as to obtain the shallowest focus possible (with this small-sensor camera), especially to try to defocus as much as possible the inevitable muck on the passenger windows.  I had no particular plan, I was just being a tourist…

But of course, once I got back to the editing, grander plans (belatedly) came to mind: I would string some kind of video together as an entertaining souvenir for the (transient) passengers, staff (one of whom was changing his job) and of course myself.  I found some “little stories”, a “little drama”, a “celebratory ending”, and royalty-free music that was great accompaniment.

One of the first jobs was to stabilize it.  For that I used Gunnar Thalin’s Deshaker plugin for VirtualDub.   Following this, the pictures were smooth and drifting like I wanted them to be, but marred by occasional shimmering of individual objects (like houses below) that had motion-blurred at the sensor stage, prior to deshaking.

If only I had used a faster shutter…  but then it would have got that staccato look I didn’t want.  Or would it?  It occurred to me that some kind of motion-estimation-based post production technique might be able to substitute motion blur (at a stage after the deshaking).

Time for a Web-Search:

  • Google:[fast shutter stabilize motion blur]
    • http://www.guthspot.se/video/deshaker.htm
      • “Question”:
        • – I get some blurring/shimmering in frames where the camera shakes the most.
      • Response:
        • What you see is probably motion blur, i.e. motion occuring within a frame. This is present in the source video too, but it doesn’t become distracting until Deshaker has removed the motion between the frames.
        • To get rid of this effect you need to use a faster shutter speed while filming. How fast it needs to be depends on the camera and the amount of shake. For a camera with built-in stabilizer, I’d recommend at least 1/200 sec, or so. (Faster if it doesn’t have built-in stabilizer.)
    • http://rungunshoot.com/reel-smart-motion-blur/
      • Title: Skip the ND (Neutral Density) Filter and add your motion blur back in post
      • One way to add motion-blur in post is RE:Vision Effects’ plug-in ReelSmart Motion Blur (RSMB).
        • This plug-in, designed mainly for 3D graphics artists who need to add blur to their rendered objects, tracks vectors frame-to-frame and generates the appropriate blurs and streaks to mimic actual motion blur.
        • It can be used in two modes: RSMB (basic) and RSMBPro (advanced).  I have only used the basic mode because it has been satisfactory for my needs.
        • Simply drag the plug-in to your clips in your timeline and drop. That’s really about it.  If you want to tweak, there are only two parameters in basic mode:
          • Motion Blur Amount: simulates different shutter speeds.  Since cinema is universally shot with a 1/48 shutter, and this is the default setting of the plug-in, you should really just leave this alone unless going for a special look
          • Motion Sensitivity: adjusts how much the warping reacts to motion.  I’ve found that reducing the sensitivity helps reduce warping artifacts in scenes with intense motion, but in my tests it handled almost all motion well when set at .5 or 50%.
        • Is it flawless? No.  If you freeze-frame some shots, you can see warping where the foreground and background mesh in weird ways. But these artifacts are hardly noticeable when played back a regular speed.
        • The main caveat to using this plug-in is increased render time.  So I would recommend applying it as a final step before rendering for output.  But don’t apply it to all your clips as a compound clip, or even worse as a rendered movie file, because then it will try to warp your different shots together, resulting in some very strange artifacts.
    • http://www.reduser.net/forum/showthread.php?79306-Shooting-motorcycles-from-the-back-of-a-pickup
      • Question:
        • we are planning to shoot from the bed of a pickup truck.
        • I am aware that the rental of either a gyro or steadicam rig would be ideal, but the budget is limited and the rental situation here in Idaho is less than ideal.
        • So instead we have rigged a kind of “ghetto fabbed” large cinesadle for our tripod to be loosely ratchet strapped on top of in the truck bed (to reduce vibration), then we will stabilize in CS6 warp stabilizer.
        • My questions is:
          • What do you guys think of 4k 24 VS 3k 48?
          • Currently we have been using a shutter of 192 for 24 and a shutter of 384 for 48, any suggestions here?
      • Reply:
        • your shutter speeds are a bit high. You might get the Private Ryan strobing effect. Have you run any tests?
      • Reply:
        • We have done some testing at 4k, not yet at 3k. What would you think of as an ideal shutter? The problem we have been running into is the motion blur caused by movement/vibration, causing the shots at a more standard shutter speed to look much softer.
        • We have done some testing at 4k, not yet at 3k. What would you think of as an ideal shutter? The problem we have been running into is the motion blur caused by movement/vibration, causing the shots at a more standard shutter speed to look much softer.
      • Reply:
        • Testing testing testing!!!
        • We are doing a similar shoot, except shooting out the front window, ie, driver’s POV on some rough roads.
        • We built a bungee rig with straps through the windows and the camera hanging for the bungees, we tried a steadycam arm on a combo shorty stand, and other goofy looking gadgets.
        • So far the best was going handheld with elbows propped on a padded 2×4. Fortunatly for us, the client wanted smoother and we will be renting one of these
        • $1200 a week i think we were quoted. Maybe we can shoot some other fun stuff the rest of the week!
        • I guess I would err on the side of fast shutters than slow, the stabilize will look better with less motion blur. In our tests, it messed up on the sharp bumps where there was a lot of movement between frames, the motion blur made it a little goofy looking.
      • Reply:
        • The RE:Vision plugin works pretty good in most cases – I agree that it’s better to sacrifice motion blur if there’s ANY plan for post stabilization, and just fake it on the stabilized footage.
        • 48fps is only gonna give you the effect of slower motorcycles when played back at 24, so I would say shoot at 4k to give the stabilization/post blurring process more data to work with (and a wider shot w/less of a cropped sensor). Also, use a lens w/IS.
          • {I assume that means Image Optical Stabilization)
        • All that said, I got some great shots of some electric motorcycles six years ago from a truck bed on a dirt road using a steadicam… (actually a cheap rental Glidecam). It’s easy to operate since you’re just sitting/kneeling there, so you don’t need to hire an experienced operator.
      • Reply:
        • Increase framerate and cushion the camera with bungies or lock the camera hard to the car. Then remove frames in post and stabilize. the upped framrate will get you around rolling shutter in a way the shutter will not. So better to shoot 48fps 360 than 24fps 180.?
          • {Well I’d never have thought of that, good tip.  This is a RED camera forum, but maybe applies to some other cameras also?}
      • Reply:
        • First, partially deflate the tires on the pickup – especially the rear wheels. That will remove most of the vibration.
        • Next, keep the camera lower to the bed – the higher you are, the more “sway” introduced. If you have ever tried to shoot from the back of a truck standing up, you know that stability is lacking – it will fight you.
        • If you want to be clever and build a rig:
          • Mount a high hat in the middle of a piece of plywood.
          • Drill a series of holes at the edge of the plywood.
          • Mount some drilled rails along the top of each side panel of the bed (use the rectangular holes as gravity mounting points)
          • Install numerous high strength rubber straps / cords between the rail holes and plywood holes.
          • Result: a floating camera platform. The rubber straps will absorb a lot of the vibration.
        • Downside is it tracks with the level of the bed – so as the truck takes a corner, the platform tilts.   Then again, a tripod will do the same thing.
        • So, another approach is to construct a floating mount suspended from an overhead rail. With a bit of practice, you can keep the horizon level in a turn. Again, heavy duty rubber to take the load and absorb vibration.
        • If you are able to get a Steadicam or Tyler mount w/gyro, best place to position yourself is sitting down on the tailgate.
      • Reply:
    • http://ai.stanford.edu/~schuon/deblur.htm
      • Potential methods for removing blur from the original footage, by deconvolution:
      • Matlab algorithms (downloadable)

Adobe Premiere Pro CS6 Warp Stabilizer Slow: No GPU/CUDA, Maybe no MultiThread?

Sunday, March 10th, 2013

My current project, a live rock performance (at an offshore radio party on a ship) involved significant quantities of handheld footage.  I’m editing this one in Adobe Premiere,  A lot of the handheld footage benefited from stabilization.  The easiest stabilization to hand is the Warp Stabilizer effect within Premiere.

It is really handy, and worked for me maybe 75% of the time, other situations gave unrealistic or unusable results.  So there is still a role for keeping back-up options such as Gunnar Thalin’s excellent Deshaker (and rendering to intermediates e.g. in GoPro-Cineform).

It is SLOW, a real time-loser, the main delay being its Motion Analysis stage.

  • This stage is computationally intensive in principle, a fundamental issue for any such device.
  • Some systems employ parallel execution here, to good effect: vastly reduced analysis (waiting) time.
  • However it seems that Premiere CS6 does not employ GPU here, and from Windows’ Task Manager, I infer that it is not even using multithreading (though I don’t know that for a fact).

Other than that, it does apparently use GPU/CUDA for its subsequent stabilize/deshake stage, and indeed that stage is very quick indeed, facilitating experimentation with settings (e.g. Subspace Warp or Position mode) to obtain the desired effect.

Incidentally, I found the default Subspace Warp mode to be “fragile”, so I use Position instead:

  • It often makes things in the background flap or wobble in unrealistic manner.
  • I therefore use Position mode, the simplest mode, as my default, then only advance to “Rotation” (etc.) if there is camera rotation.
  • It didn’t work well with very noisy footage, e.g.  Sony Z1 in Hyper-Gain mode, even if when denoising was applied earlier in the effects-chain.

Lastly, it’s a shame there’s no way/settings for:

  • Defining a mask, rectangular or otherwise, for region(s) to focus on or to avoid.  For example to prevent it locking onto a singer’s head instead of the stage.
  • Telling it to definitely not try to compensate for rolling-shutter.  When I know the camera is CCD, I ought to be able to tell the software not to consider rolling-shutter.  I never fully trust “Auto”, not in any application or context…

(more…)

Mocha & AE for Tracking & Stabilising

Wednesday, December 19th, 2012

In fact there is an excellent-looking series of tutorials at http://www.mamoworld.com/index.php?option=com_content&view=category&layout=blog&id=42&Itemid=84&lang=en

Sony EX3 Noise & Bits-Resolution & Green-Screen

Sunday, November 4th, 2012

It has been said ( I believe by Alister Chapman ) that there are only marginal benefits from recording XDCAM-EX to more than 8 bits, due to the relatively high noise of this camera, as compared to more typical broadcast cameras.

In my experience, while it was a wonderful step-up from my Z1, certainly it’s recordings are noisier than I’d like, leading me to pretty-process certain footage (using Neat Video denoising plugin to my NLE).  And as a recent project with reasonably well-lit green-screen illustrated, it’s noise in shadows can be a particularly nuisance (much time in post experimenting to work around this).

So I wondered:

  • Even if marginal, to what extent is 10-bit beneficial to EX3 recording?
  • For the EX3, when recording 10-bit, it is also 4:2:2, surely a benefit to chroma keying and resizing (reframing, stabilising/deshaking/tracking).
  • Could the benefit depend on editing workflow?  For example:
    • What if subsequently de-noising (like I mentioned)?
    • Some NLE’s do bits-dithering, hiding the quantisation/banding that would otherwise be apparent from having only 8 bits.

I need to do my own experiments, but for now, here (below) are some results from web-searching…

(more…)

Effects-Order: Denoise before Deshake

Thursday, November 1st, 2012

This is my impression, from the expressed views of others:

  • If you have e.g. 60i or 50i then do any stabilization first, so it gets maximum temporal information, prior to any deinterlacing e.g. to corresponding 30p or 25p.  Presumably if double-deinterlacing e.g. to 60p or 50p then this is immaterial.
  • If you’re using noise reduction, always do that prior to stabilisation as it helps the algorithm to concentrate on wanted detail and not on random noise. But then if it’s Neat Video denoising then that works best with stable progressive material.

(more…)

Deshaker by Gunnar Thalin: Usage & Background Info

Sunday, July 25th, 2010

Some good links:

  • Doom9 forum attended by Deshaker’s author (Gunnar Thalin): http://forum.doom9.org/showthread.php?t=49597
    • Highly recommended, give it a trawl-through (allow an hour or so!).
    • Many specifics and clear explanations beyond the user manual.

Deshaker by Gunnar Thalin: Tips

Sunday, July 25th, 2010

Tips from Deshaker’s author, Gunnar Thalin [http://forum.doom9.org/showthread.php?p=1418923]:

  • Q:
    • Any recommendations specifically for vehicle, motorcycle in my case, mounted videos?
    • Are there particular settings that would work better for high speed movement? 
  • A:
    • I know a lot of people seem to be using Deshaker for those kind of videos, but it’s not really designed to handle videos where the camera “moves around”, at least not where it moves very fast. Deshaker can only handle panning, rotation and zoom. That said, I’ve seen some very good deshaked videos of that kind.
    • As always, make sure you’re using a high enough shutter speed, if possible. There shouldn’t be any motion blur in each frame.
    • Also, you would probably get *a lot* better results if the camera didn’t have a rolling shutter. But I believe most helmet mounted cameras today have a rolling shutter. Deshaker can remove rolling shutter effects but only if the shaking isn’t extreme.
    • I would also recommend trying to stabilize only on the most distant parts in the frames, since the “moving inwards”-effect is less there. You can do that by ignoring the other areas.
    • And turn off zoom smoothing by settings zoom smoothness to 0. You will also need to change the edge compensation type if you do that. Start with “None”, maybe.
    • Finally, you should probably increase the value for “discard motion of blocks that move > X pixels in wrong direction”. That’s to allow the blocks to move “freely” a little, since Deshaker can’t handle the “moving inwards”-effect.
    • But maybe you should ask someone who’s actually stabilized these kind of videos. I haven’t.

Glide-shots: Steady-Shot / Smooth-Deshake-Stabilize / SteadyCam

Sunday, July 25th, 2010

Which is best?   Depends on the camera, scene and shot dynamics I guess.  The same point is queried at the following thread:

Some general advice from a computer-post-savvy author: definitely use the camera’s SteadyShot:

Limitations of post

  • Stabilization necessitates motion estimation and image reconstruction, which are extremely CPU-heavy, hence really slow to execute.
  • Most stabilization apps (in post) can’t currently cope with motion-blurred edges or parallax effects (though both should be possible in principle, by deconvolution and 3D modelling both informed from multiple frames).
  • For rolling-shutter-ed footage (e.g. CMOS sensors as in Sony Exmor as in Sony XDCAM-EX e.g. EX1 & EX3), there exist options to reduce the effect (don’t expect perfection, but may suffice):

My experiences:

  • Stabilizing Tools:
    • Gunnar Thalin’s Deshaker works really well.  And it is multi-threaded, really speeds up the process.  The author says it is more intended for handheld pans etc. than fast-shaking shots from vehicles etc. (but has nevertheless seen good results in such situations).
      • The author says [http://forum.doom9.org/showthread.php?p=1418923] to try “to stabilize only on the most distant parts in the frames, since the moving inwards-effect is less there”.  And “you should probably increase the value for [discard motion of blocks that move > X pixels in wrong direction]. That’s to allow the blocks to move “freely” a little, since Deshaker can’t handle the “moving inwards”-effect.
      • Possibly equally applicable to other smooth/stabilize/deshake tools ?
    • Boris’s Optical/Motion Stabilizer (in Boris Red 4.3.3 on XP) is only single-threaded and I find it slower, clunkier and less intuitive than Deshaker.  Has a Smooth mode, which is like the others here, as well as a Stabilize mode (try to keep frame static, no good for motion then).  The other tools can be configured to do the same thing.
    • Mercalli in Sony Vegas has no mode for 720p50 but otherwise is pretty good and very intuitive and configurable.
    • FCP’s SmoothCam Effect worked best for a challenging clip for wobbly-hand-held camera tracking close past an object (a Formula-1 car) hence huge degree of moving-inwards effect.  The default settings worked straight away.  The result quality was way above that of the other tools.  On the other hand sometimes it’s not the best (sorry, forgot the exact situation).
  • Cameras & Shots:
    • Historically, using a TRV33 DV HandyCam indoors (hence low-light hence long shutter time):
      • Way back in the past, using a (now ancient) TRV33 DV handy-cam (which has huge sensor margin i.e. spare pixels), when I shot big zooms to lecture audience individuals (e.g. question-time) I had the camera’s steady-shot (digital, not mirror) enabled  and used Gunnar Thalin’s Deshaker (VirtualDub plugin) also.  The result was astoundingly steady.
      • The same arrangement worked OK with hand or shoulder mounted cam for walk-throughs past nearby objects (e.g. walls, people, furniture).
      • An attempt to do the same thing without steady-shot enabled on the camera resulted in seriously motion-blurred edges.
    • Now, using a Sony EX3:
      • With camera Steady-Shot set to Medium, hand-held pans and motion past nearby objects seem to acquire a positional instability, as if the camera feedback mechanism needs greater damping. Maybe the camera’s internal mirror “suspension” has to be tighter (than the TRV33 digital equivalent) because it lacks the generous pixels margin of the TRV33?  or maybe something to do with the mirror’s inertia?  Or (real-time-constrained) processing-power?
        • Experimentation is needed with the camera’s other SteadyShot modes (High, Low).
        • In the absence of more generous sensor pixel margins, I wish it could be loosened-up e.g. to allow black borders (to crop in post) so as to permit smoother rides overall.