Followed yet another awesome tutorial by BlenderGuru on a different scene, with mixed results.
Lesson learnt: Stay away from camera mapping environments with a lot of reflections, or get rid of all reflections in the original and add them in Blender (with lighting + compositing)
