Kinect Go Kit: Top level

Kinect Go Kit Pelican Case

I travel with two Kinects, two power supplies, and two computers in case one gets damaged or destroyed in the field, although only one computer is in the kit.

The Netbook is a cheap $240 computer running Ubuntu and the libfreenect software stack at a minimum. I have also found it useful to have more advanced data collection software (ROS, RGBDemo, RGBDSLAM), and some analysis software (CloudCompare, points2grid, Viewpoints, etc.). This netbook works fine for raw data dumps from the libfreenect ‘record’ program. It can run the more computationally expensive scene stitching algorithms such as RGBDSLAM, but it takes about 10 seconds per stitch, while a more powerful laptop (but still a few years old) can do it at 0.5 to 1 Hz. Since ‘record’ collects about 1.5 GB of data per minute, it is good to have a lot of free space on the hard drive.

The plugs and cables are shown laid out below, and in addition, some velcro straps are stored in that compartment, used to attach the Kinect to the tripod arm.

The Pelican 1510 case supports two levels, and the lower level looks like this:

Kinect Go Kit: Bottom level

Battery #1 is a 12 V 5 Ah sealed led acid battery. It provides >5 hours of Kinect runtime, about equal to the runtime of the netbook.

Battery #2 is 8 AA batteries (12 V), and underneath is an 8 AA battery holder and a battery charger. If I need to turn the Kinect on for a short amount of time and want to travel lightly, these will do.

Kinect cable layouts

As shown above the cord to the Kinect can be cut and alligator clips or some other electrical termination can be attached. I often have wall power and have attached clips to the detached plug so I can use it as originally intended. However, when in the field, the clips can connect directly to the 12 V battery or the AA battery pack.

Kinect mounted on tripod

The Kinect Go Kit above is close to the minimum necessary for fieldwork. Things that I would like in it, but are not yet, include:

• Tape measure
• Liquid container (tupperwear) and opaque liquid (or additive) so that any scene can have a defined flat surface
• Sling for under tripod to hold battery, netbook, protecting equipment and keeping it off the ground
• Counter weight for tripod arm

Additional tools I have found handy to have with me in the field include, but are not limited to, the following:

• External hard disk for backups
• Zip-ties to complement the Velcro straps
• Multimeter
• Spare notebook, perhaps with a more powerful CPU, for scene stitching
• Rope or other ‘image noise’ for scene stitching with RGBSLAM when working in environments that have ‘self similar’ scenes (no good tie points)
• Mounting systems for long term deployment
• Trashbags for environmental protection

There are a variety software interfaces to the Kinect. One high-level tool that is easy to use (binaries provided, no need to compile source, supports ‘scene painting’) is RGB-Demo. It is a good tool to start with if you want to work with the Kinect.

However, most Kinect software and calibrations so far have been developed by the robotics and computer vision communities. I am grateful for the work they have done, but those communities have different data needs than earth scientists. For example, quadrotor obstacle avoidance (link (PDF), link) has distance measurement errors that appear to be on order cm, but it still works fine as the helicopter avoids obstacles by an amount larger than the error.

Earth scientists should aim for a better model of the world than the one currently provided by the Kinect and its primary users. I suggest recording and storing the raw digital numbers (DN) from the Kinect rather than higher-level auto-calibrated real-world coordinates. It will require more post-processing, but storing the DNs will allow the data to be re-processed as better calibrations are developed. In addition, the low level recorder operates at 30 Hz and the higher level point-cloud products currently do not record data at that rate.

The best supported low-level interface is the LibFreenect Fakenect record program. It dumps the uncalibrated RGB and depth images to a folder at 30 Hz until you kill the process. Uncalibrated means both that the depth data is in sensor units, and that the depth and RGB images are not aligned. You can easily convert the depth data to real world x,y,z coordinates using existing published algorithms (link, link, link, and many others exist on the web), but importantly the raw data is stored and can be used with better calibrations in the future.

After processing the raw ‘record’ data, you can work with the point cloud data or DEMs using a variety of standard software for pointclouds, LiDAR, etc. I have had great success with CloudCompare and Poinst2Grid, in addition to custom code in MATLAB, IDL, and Python. A good list of software is available at the NSF OpenTopography site.

To work with the depth data to we initially use the following algorithms found on the various sites dedicated to Kinect hacking. The data provided by these algorithms is sufficient for certain uses, and for testing algorithms and visualizations, while better calibrations are performed.

DN to distance (source):

k1 = 1.1863d
k2 = 2842.5d
k3 = 0.1236d
Z = k3 * tan( double( DN ) / k2 + k1 )

XYZ to world (source):

Xres = 640
Yres = 480
FovH = 1.0144686707507438 (rad)
FovV = 0.78980943449644714 (rad)
XtoZ = tan( FovH / 2 ) * 2
YtoZ = tan( FovV / 2 ) * 2
X = ( X_pixel / Xres – 0.5 ) * Z * XtoZ
Y = ( 0.5 – Y_pixel / Yres ) * Z * YtoZ

Question or comments? Post below…

If you use IDL and would like to image your data in Google Earth, be it points, lines, regions, surface images, or vertical data, you should be using my IDL interface to the KML API.

Vertical Data in Google Earth

# build NetCDF
export CFLAGS=-m32
export FFLAGS=-m32
./configure --prefix=/Users/mankoff/local/netcdf-4.1.1 \
          --disable-cxx --disable-curl  --disable-dap
make && make install
say netCDF done

# build GLIMMER
cd ~/local/src/
wget http://download.berlios.de/glimmer-cism/glimmer-1.7.0.tar.gz
tar zxvf glimmer-1.7.0.tar.gz
cd glimmer-1.7.0/

# OS X has issues with 32 and 64 bit libraries.
# The -m32 flag forces 32 bit compilation.
# The following should be one long line:
./configure --prefix=/Users/mankoff/local/glimmer-1.7.0 \
     --with-netcdf=/Users/mankoff/local/netcdf-4.0.1 \
     FC=gfortran F77=gfortran CFLAGS=-m32

make
make install
say GLIMMER done

There are a few ways to test the installation. The source folder provides a test folder:

export PATH=/Users/mankoff/local/glimmer-1.7.0/bin:$PATH
cd ~/local/src/glimmer-1.7.0/tests/shelf
python circular-shelf.py circular-shelf.PP.config
python confined-shelf.py confined-shelf.PP.config
say GLIMMER Test Done # Takes a while. Turn up your volume

Running the above command will result in NetCDF files being created in the output/ subdirectory. You can view the contents of example.nc with most any generic NetCDF viewer. While theses tests run over a given period of time, the output only has one time stored. If you want to see an evolution of the ice shelf, older test suites available from the previous code repository site should be used:

cd ~/tmp/
wget http://forge.nesc.ac.uk/download.php/200/glimmer-example-0.6.tar.gz
tar zxvf glimmer-example-0.6.tar.gz
cd glimmer-example-0.6/
~/local/glimmer-1.7.0/bin/glide_launch.py ./example.config
say done

Examine the output file example.nc to see ice sheet evolution over time. Basal melt is shown below:

@article{Rutt:2009,
  Author = {Ian C. Rutt and Nicholas R. J. Hulton and Antony J. Payne},
  Title = {{The Glimmer community ice sheet model}},
  Year = {2009}}
  Journal = {J. Geophys. Res.},
  Volume = {114},
  Number = {F2},

Google has made the tesseract OCR code they use for Google Books available. They don’t officially support OS X. Below are instructions to get tesseract-ocr running on OS X. As usual, Developer Tools (XCode) needs to be installed.

A slightly simpler installation does not use LibTIFF, but in this case you can only convert single page and uncompressed TIFF files. As PDFs are usually multiple pages, it is worth it to install and compile with LibTIFF.

# prepare dependencies
fink install libtiff libtiff-shlibs

# fetch
svn checkout http://tesseract-ocr.googlecode.com/svn/trunk/ tesseract-svn
cd tesseract-svn

# compile
./runautoconf
export CXXFLAGS=-m32 # Force 32-bit architecture
./configure --prefix=/Users/mankoff/local/tesseract/ --with-libtiff=/sw/lib
make
make install

# test run
~/local/tesseract/bin/tesseract image.tif out # run it
say Tesseract finished # might take a while. Turn on your speaker.
gn Finished Tesseract # alias gn='growlnotify -s -m'
less out.txt # check

Notes:

• Here is a sample image for testing
• The image must be in TIF format.
• The extension must be have one “f”: TIF or tif not TIFF.
• Images and complex equations are not handled

I tried to run tesseract on my handwriting and it could not decode it. I wrote a simple sentence as clearly as possible, took a photo:

The Quick Brown Fox

And the result was:

THE ®.U\(.K
[awww Fox
TUMPED oval
#IE uxzv ooé

It got “THE” and “Fox” and most of JUMPED. However, tesseract supports full training so if you need to convert your notes read the documentation and post what your learn below.

Get a MacBook Pro.

If you need the extreme computing power of a quad-core desktop then you might consider the Mac Pro, but ideally you’d use the MacBook Pro and ssh into your local supercomputer or cluster. If you travel a lot and don’t need a lot of disk space you might consider a Macbook Air.

Get the 13 inch model. This makes traveling easier and maximizes battery life. Since you’ll have an external monitor at your desktop, even the 17 inch laptop will have a small screen in comparison, so you might as well go for portability.

Other specifications:

• Minimum CPU.
• Maximum hard drive space.
• Minimum RAM from Apple.
• Go get the maximum possible RAM from http://crucial.com 8GB is nice.

• One External monitor.
• Two backup disks, equal in size to your internal disk. Ideally one would be the Apple TimeCapsule, but this is quite expensive, so USB or FireWire disks will suffice. Keep one at home and one at the office. Set a calendar reminder to plug into each one at least once a week if you don’t do it every night by habit.
• Two adapter cables (one for VGA one for DVI) so you can connect to external monitors and/or projectors.
• A USB key to quickly transfer files

• External mouse
• External keyboard
• Wacom tablet (to help with RSI and/or artwork)
• Headphones to tune out office mates
• A USB hub so you can plug in more than 2 peripherals
• A Firewire 400 to 800 adapter if you have old firewire drives not supported by the new laptop interface

The following software list should allow you to write code in almost any language you choose, and to access, display, and analyze almost any data set. The following software will also help you manage your PDF library and write papers.

You must install the Apple “Developer Tools” package that is an optional install on the OS X disk provided with your computer. You know you have installed it if you have a “/Developer” folder at the root of your disk.

The Apple iWork suite is nice. It is nicer than MS Office for OS X and provides Pages, Keynote, and Numbers as equivalents to Word, PowerPoint, and Excel. Free alternatives to iWork and Office are Google Docs and Open Office. Or LaTeX for documents, CSV for spreadsheets, and Beamer for presentations.

A partial list of useful 3rd party OS X software follows. Drop any of these words into your favorite search engine to get a download link. Software that cost money is starred*.

• Adium (For IM, Google Talk, etc.)
• AntiRSI (Reminds you to take a break from typing)
• Aquamacs (A good code editor)
• BibDesk or Mendeley (Manage your PDF library and references)
• Celestia (Fly through the universe)
• Cyberduck (FTP/SFTP)
• DoubleTake* (or Hugin) (Stitch together panoramic photos)
• Enthought Python Distribution (EPD) (One-stop shopping for all your Python needs)
• Firefox (Browse the web)
• Fluid (Make Gmail (or any other site) a stand-alone desktop application)
• GeoMapApp (Geospatial data browser)
• Gimp (Edit images)
• GoogleEarth
• GrandPerspective (Find out where disk space is getting used)
• Graphviz (Flowcharts)
• Growl (sleek uniform system notices)
• IDL*
• Jumpcut (Global clipboard buffer)
• LaTeXiT
• MATLAB*
• MacFUSE (make any computer appear like a local folder via ssh)
• MenuMeters (monitor your system status in the menubar)
• Mouse Locator (never lose your pointer even on a big screen)
• ODV (Ocean Data Viewer)
• OmniGraphSketcher* (Make pretty charts)
• PDF to Keynote (Convert PDFs to Keynote slides)
• Qgis (A free GIS application)
• Quicksilver (Use your keyboard for everything. Be efficient)
• Sim Daltonism (Simulate color-blind and color-deficiencies so you know your graphs will be readable by everyone, and even print to grayscale well)
• Skim (An enhanced PDF reader if you find Preview.app lacking)
• Stellarium (See the stars)
• SuperDuper!* (Clone a disk for backup, although Time Machine is probably good enough)
• Sysquake (For control or robotics applications)
• Vapor (or Visit) (Advanced visualization software)

From the above list, I want to stress the following:

• A good PDF manager is essential for research and writing. Use BibDesk, Mendeley, or Zotero or CiteULike.
• Firefox is a decent web browser, but what makes it very useful are the extensions and customization it allows:
  • Smart bookmarks can make searching any website a breeze. For example you could just type “gsch keywords” as a shortcut for Google Scholar.
  • Useful extensions are: DownloadHelper, DownThemAll!, Firefox PDF Plugin for Mac OS X, Flashblock, Permit Cookies, Tab Mix Plus, TabSwitcher, Tree Style Tab, Zotero.
• Jumpcut is amazing.
• Fluid will change the way you use your web browser. If you use FaceBook or Gmail or have some other website that is persistently open in your browser, make it a stand-alone application with its own icon, and free your browser for browsing.
• You might notice no mention of Adobe.
  • For basic image work, I use Gimp or GraphicsMagick instead of PhotoShop.
  • For posters I use Pages, Keynote, Beamer, or Powerpoint rather than Illustrator.
  • For PDFs I use Preview or Skim rather than Acrobat.

Much of the following unix-based software will only work if the Apple Developer tools is installed as suggested above.

The main system that provides access to a variety of unix tools is fink. (OS X is code-named Darwin, and Darwin is linked to Galapagos finches, and fink is the German word for finch, and the main developer of fink is German).

If you prefer not to use fink, you can probably get many of your HPC tools installed via double-clicking. This might be easier, but your choices will be limited compared to fink, and you won’t have one environment that lets you manage all of the software. However, if all you need is one FORTRAN compiler, or one LaTeX installation, then you might want to search through the pages listed in the Other Resources section below.

Download and install fink. Follow the default options, but then enable the “unstable” tree to get access to a whole lot of additional (perfectly stable) software.

Once fink is installed and set up, you want to use it to install your favorite unix tools. You can find things to install with the “list” command, like this: “fink list fortran”. I find the following tools quite useful, but I like to geo-tag images and write presentations in LaTeX…

fink install base-files bash bash-completion contacts coreutils daemonic ddclient detex exiftags exiv2 fuse g77 gcc44 ghostscript gmt gpsbabel graphicsmagick ispell latex-beamer launch lynx minicom mpack ncarg ncftp ncview proj pstree psutils r-base smartmontools tesseract texlive-base texlive-texmf tree unzip watch wget xtermcontrol

Due to dependency bloat, I prefer graphicsmagick to imagemagick, and avoid using “gv” altogether. Instead of gv I use the less user-friendly “gs” from ghostview, let Preview.app convert it to PDF, or manually convert it to pdf with “ps2pdf”. Again due to dependency bloat, Octave and gnuplot are two of the few “unix” tools that I install without fink, instead using the OS X HPC page listed at the bottom.

Not everything is available in Fink. I have found the following non-fink-packaged software useful:

• csv2latex (convert CSV files to LaTeX tables)
• gpicsync (geotag and produce KML files of photos)
• ViewPoints (multivariate data exploration tool)

Now that the software is installed, you’ll need to spend some time customizing it. The following key areas should be customized for efficiency:

• Your bash prompt (.bash_profile)
• Bash aliases (.bash_profile)
• Emacs/Aquamacs (.emacs)
• SSH (.ssh/config) and ssh keys

With a bit of effort adjusting the above files you can, for example, log into your remote machines with just two keystrokes, one of them being the return key.

If you want to run Windows and/or other flavors of Linux (OS X being based on BSD), you can easily do so. You have four main choices: BootCamp, VMWare, Parallels, or VirtualBox, and some minor choices: Crossover, Wine, etc.

Use VirtualBox to host your Windows installation as a local disk image. Do not use the Apple BootCamp.

If you need to run Windows by itself, and want to use some of the tools mentioned in this page in Linux in Windows, you have a few options. The most popular is a Linux Live CD (try Ubuntu). Less popular but easier (no CD required) is andLinux. Download, double-click, install like regular Windows software, reboot, and you’ll have a bash shell icon on your desktop.

• If you program hardware via serial ports you’ll need a USB to Serial adapter. Keyspan makes the best ones. You’ll also want to make sure minicom is installed (fink install minicom).
• If you use Google Earth a lot you will be grateful to invest in a 3D Connexion 6-DOF mouse.
• If you want more than 1 external monitor, the best solution is a new breed of USB-based monitors. You can have, I think, as many as you like.
• If you read a lot of papers an eBook might be a good investment. The only one big enough at the time of this writing to display PDFs as images is the Kindle DX. If you can wait 6 months to a year there should be some good competition and even laptops that have screens that switch to e-ink mode when dimmed.

UCSC provides licenses for MATLAB and ENVI (which includes IDL). You can run these for free when you are on the campus network (although I think you have to be wired, or on the “cruznet secure” network, not on the general cruznet network.).

If you want to run the software from home you can set up a VPN. This makes your computer appear as though it is on the UCSC network, even though you can be anywhere in the world. The VPN is also beneficial as it lets you access libraries and journals as though you are on campus, making it easy to download papers into BibDesk.

• http://hpc.sourceforge.net/
• http://www.atmos.washington.edu/~salathe/osx_unix/
• http://www.macresearch.org/
• http://www.osxgnu.org/
• You can use Darwin Ports instead of fink: http://darwinports.com/

I took the time this weekend to re-create a Google Earth layer showing these data. This visualization allows you to see a broad geospatial overview, and then select any particular location and see both long-term trends and even the data point for each individual month.

Download (330K)

GISTEMP.StationData

GISTEMP.StationData

One subset of the library is the KDM_KML lib for writing KML files. This code should make it fairly simple to display data sets in Google Earth from IDL.

I have currently implemented about 50% of the KML API and will get the rest implemented as time goes on. Right now layers can have overlays, points (placemarks), or linestrings (paths). Any element (that the API supports) can have a timestamp, and styles are supported too. With knowledge of the KML API you can create artistic balloons with whatever content you like that can even link to each other.