Offline Registration for the Kinect
I am releasing source code for a program that supports off-line calibration and registration of raw Kinect data collected by the libfreenect record program. The interesting parts of this code were all developed by the OpenKinect community, I simply patched parts of their code together in a way that was useful for my Kinect work. I have argued in the past for collecting the raw data so that different (and better) calibrations can be applied in the future, and this code maintains this philosophy.
This code uses the internal calibration parameters that are shipped with each Kinect, as opposed to extrinsically determining the calibration (the method used by most checker-board image calibration methods). Offline means that the calibration parameters can be captured once when the Kinect is plugged in, and then used in the processing phase without the Kinect on data that has already been acquired. Calibration is defined as the act of mapping from the raw sensor digital numbers (DN) in (pixel, pixel, DN) coordinates to world (x,y,z) coordinates. Registration is defined as mapping the RGB pixels onto the depth data.
Importantly, this registration maps RGB to depth, whereas all other registration algorithms I know of map depth to RGB. Since the data of interest with the Kinect is the depth data, it does not make sense to perform any unnecessary error-increasing mathematical operations on the depth data.
The code can be downloaded from https://github.com/mankoff/libfreenect/tree/offline_register and built with the following commands, assuming all dependencies are already installed. The easiest way to verify your are able to build this code is to install libfreenect through an existing package manager system, then over-write that with this custom build.
git clone https://github.com/mankoff/libfreenect/
cd libfreenect
git checkout offline_register
mkdir build
cd build
cmake ..
make
make install
Optionally, skip the make install line and access the new program in build/bin/. The new program is called “kinect_register” and when run without any arguments it prints usage instructions:
$ kinect_register
Kinect Offline Registration
Usage:
kinect_register [-h] -s <regfile> | -a <regfile> <PGM> [<PPM>]
-h: Display this help message
-s: Save the registration parameters from a connected Kinect to <regfile>
-a: Apply the registration from <regfile> to <PGM> (from 'record')
Optionally align a PPM file with the PGM file
Data Formats (units: mm):
file.x: 640x480 double of x values
file.y: 640x480 double of y values
file.z: 640x480 integer of z values
file.xyz: ASCII file of x y z r g b. RGB only if PPM argument provided
file.reg.ppm: PPM shifted so pixels align with file.{x,y,z} data
