| Follow with RSS
The Kinect is a new sensor and is actively being used and researched, and new uses and information appears frequently. I cited what I could in my paper, but some of that information is already out-dated. In the comments below I attempt to catalog new projects, studies, publications, and information as it becomes available.
Tags: Code, Computers, HowTo, Kinect, ResearchA manuscript by Mankoff and Russo entitled “The Kinect: A low-cost, high-resolution, short-range, 3D camera” has been accepted with minor revisions by the journal Earth Surface Processes and Landforms (ESPL).
It can be cited as doi: 10.1002/esp.3332
I’m about to embark on my third trip underneath a glacier within the past 13 months. I’m taking the time to write up the first two trips now, so that next month when I go on the third trip there is some context and some record in case I don’t return (I will be rappelling down a dry moulin and spending time underneath a glacier that will likely have polar bears on top). I also want to share the experience here because it was interesting and special and I hope to educate others about some unique field sites and engineering feats.
The first trip occurred in September, 2011. My collaborator Jason Gulley and I drove a short distance from Longyearbyen, Svalbard, and then hiked to the nearby Rieperbreen glacier. We crawled up the snout of the glacier, and tested the Microsoft Kinect as a low-cost 3D camera to measure the roughness of the rocky floor and the scalloped ice roof. Although just a proof-of-concept, that work was written up by Wired, and led to an invitation for my second sub-glacial trip.
Unloading personnel and equipment from a hovering helicopter.
In April 2012 the Svartisen Subglacial Observatory hosted a Subglacial Workshop. Compared to hiking and then crawling through 0 C degree water into the glacier, this trip was relatively speaking a luxury. It began with a helicopter ride, but the helicopter, unable to land on the snowy mountainside, hovered with its nose touching the mountain while we climbed out of it (Note: climbing into a hovering helicopter is a bit more difficult than climbing out of one). After the helicopter departed, I noticed a door in the side of the mountain.
Entrance to the Svartisen Subglacial Observatory
Svartisen Subglacial Observatory mine shaft
The Svartisen Subglacial Observatory entrance is a door into a mountainside. Once inside, a km-long tunnel contains a living area able to house 8 scientists, a research laboratory, and several places with direct access to the bed of the glacier. The end of the tunnel that is not on the side of the mountain literally ends at the edge of the rock part of the mountain, but underneath 200 m of ice. There is no sunlight in this mine shaft, and one instantly feels disconnected from the outside world after entering and closing the outside door.
The living units are similar to those we live in when in Antarctica (when not in a tent): Shipping containers retro-fitted for humans to live in. There is a kitchen, rooms with bunk beds, and a bathroom and shower. There is electricity, but no internet. The water is fresh and cold. A 15 minute walk farther up the tunnel to the office and there are experiment benches and work space, and surprisingly, a WiFi signal with slow internet access to the world beyond. Another 5 minute walk to the end of the tunnel, and there are holes in the bedrock leading to the bed of the glacier, and a door big enough to walk through. Several times a year, scientists open the door and with a hot-water hose melt a cave, several cubic meters in size, into the bed of the glacier. This cave creeps closed quickly due to the weight of the ice above and around it. The room shrinks to about 1/2 its size every 24 hours, and the cave is completely closed within a week.
The temporarily-created subglacial cave melted in the Svartisen Subglacial Observatory. Brown bedrock with embedded pressure sensors is seen in the foreground, and the slowly creeping gray/blue ice in the background.
My third sub-glacial trip will happen in October 2012 underneath the Hornsbreen Glacier in southern Svalbard. We will rappel down an empty moulin to access the subglacial cave system.
Algorithms for the Kinect have been much improved since my first trip (for example, see Kintinuous and ReconstructMe), and we hope to make full 3D maps of the cave system. This third trip has been funded by the Svalbard Science Forum Arctic Field Grant, and I will describe it in more detail after it happens.
I know of only a few people that attempt to personally explore the subglacial environment, and am excited to partake in some of those expeditions.
Tags: Kinect, Research
Emacs org-mode has a companion MobileOrg app that lets you view and edit your org files while mobile. But, since this is emacs, unfortunately it doesn’t work ‘out-of-the-box’ as smoothly as some other applications. The documented method is to manually execute org-mobile-push to push local edits into the cloud (to your mobile device), and then org-mobile-pull to pull mobile edits onto the desktop.
The code below sets up MobileOrg, and introduces some additional functions that auto-pushes whenever a change is made, and polls the incoming file and auto-pulls whenever the mobile apps make a change. The functions work asynchronously in the background so they don’t tie up your emacs process.
First, set up MobileOrg in emacs:
;; Mobile Org (require 'org-mobile) (setq org-mobile-inbox-for-pull "~/Dropbox/org/mobile.org") (setq org-mobile-directory "~/Dropbox/MobileOrg") (define-key org-mode-map "\C-cp" 'org-mobile-pull) (define-key org-agenda-mode-map "\C-cp" 'org-mobile-pull)
The following is a modified version of this gist. Each time you save an org buffer it will wait 10 seconds and then execute org-mobile-push in the background.
;; Fork the work (async) of pushing to mobile ;; https://gist.github.com/3111823 ASYNC org mobile push... (require 'gnus-async) ;; Define a timer variable (defvar org-mobile-push-timer nil "Timer that `org-mobile-push-timer' used to reschedule itself, or nil.") ;; Push to mobile when the idle timer runs out (defun org-mobile-push-with-delay (secs) (when org-mobile-push-timer (cancel-timer org-mobile-push-timer)) (setq org-mobile-push-timer (run-with-idle-timer (* 1 secs) nil 'org-mobile-push))) ;; After saving files, start an idle timer after which we are going to push (add-hook 'after-save-hook (lambda () (if (or (eq major-mode 'org-mode) (eq major-mode 'org-agenda-mode)) (dolist (file (org-mobile-files-alist)) (if (string= (expand-file-name (car file)) (buffer-file-name)) (org-mobile-push-with-delay 10))) ))) ;; Run after midnight each day (or each morning upon wakeup?). (run-at-time "00:01" 86400 '(lambda () (org-mobile-push-with-delay 1))) ;; Run 1 minute after launch, and once a day after that. (run-at-time "1 min" 86400 '(lambda () (org-mobile-push-with-delay 1)))
Finally, poll the incoming mobileorg.org file every 30 seconds, and execute org-mobile-pull if it has been updated.
;; watch mobileorg.org for changes, and then call org-mobile-pull ;; http://stackoverflow.com/questions/3456782/emacs-lisp-how-to-monitor-changes-of-a-file-directory (defun install-monitor (file secs) (run-with-timer 0 secs (lambda (f p) (unless (< p (second (time-since (elt (file-attributes f) 5)))) (org-mobile-pull))) file secs)) (defvar monitor-timer (install-monitor (concat org-mobile-directory "/mobileorg.org") 30) "Check if file changed every 30 s.")
The only remaining improvement would be if the MobileOrg app had an option to auto-sync on launch. Perhaps the next version…
Tags: Code, Computers, HowTo
This post introduces yet another utility that might be useful for Kinect users. My previous posts have suggested using the libfreenect record utility to capture data. That is the workflow that works best with my kinect_register code. However, record does not show you what it is recording, which can be problematic.
I have patched together the OpenKinect record and glview utilities into a program called kinect_record. This program has output identical to record, but also shows the data as it captures it.
You can download the source from GitHub and compile and run it by following the text in the README.txt file.
Tags: Code, Computers, Kinect, VisualizationI am releasing source code for a program that supports off-line calibration and registration of raw Kinect data collected by the libfreenect record program. The interesting parts of this code were all developed by the OpenKinect community, I simply patched parts of their code together in a way that was useful for my Kinect work. I have argued in the past for collecting the raw data so that different (and better) calibrations can be applied in the future, and this code maintains this philosophy.
This code uses the internal calibration parameters that are shipped with each Kinect, as opposed to extrinsically determining the calibration (the method used by most checker-board image calibration methods). Offline means that the calibration parameters can be captured once when the Kinect is plugged in, and then used in the processing phase without the Kinect on data that has already been acquired. Calibration is defined as the act of mapping from the raw sensor digital numbers (DN) in (pixel, pixel, DN) coordinates to world (x,y,z) coordinates. Registration is defined as mapping the RGB pixels onto the depth data.
Importantly, this registration maps RGB to depth, whereas all other registration algorithms I know of map depth to RGB. Since the data of interest with the Kinect is the depth data, it does not make sense to perform any unnecessary error-increasing mathematical operations on the depth data.
The code can be downloaded from https://github.com/mankoff/libfreenect/tree/offline_register and built with the following commands, assuming all dependencies are already installed. The easiest way to verify your are able to build this code is to install libfreenect through an existing package manager system, then over-write that with this custom build.
git clone https://github.com/mankoff/libfreenect/
cd libfreenect
git checkout offline_register
mkdir build
cd build
cmake ..
make
make install
Optionally, skip the make install line and access the new program in build/bin/. The new program is called “kinect_register” and when run without any arguments it prints usage instructions:
$ kinect_register
Kinect Offline Registration
Usage:
kinect_register [-h] -s <regfile> | -a <regfile> <PGM> [<PPM>]
-h: Display this help message
-s: Save the registration parameters from a connected Kinect to <regfile>
-a: Apply the registration from <regfile> to <PGM> (from 'record')
Optionally align a PPM file with the PGM file
Data Formats (units: mm):
file.x: 640x480 double of x values
file.y: 640x480 double of y values
file.z: 640x480 integer of z values
file.xyz: ASCII file of x y z r g b. RGB only if PPM argument provided
file.reg.ppm: PPM shifted so pixels align with file.{x,y,z} data
The file format produced by the Kinect libfreenect record program is not well documented and suffers from endian complication issues on most computers. Below are four snippets of code that can be used to read the big-endian PGM (depth data) files produced by record. The PPM (RGB data) are a more popular image format and should be easier to work with.
// C
fp = fopen("file.pgm", "r");
while (getc(fp) != '\n'); // skip header line
uint16_t data[640*480];
fread(data, sizeof(uint16_t), 640*480, fp); // read the data
fclose(fp);
# Python
import numpy as np
infile = open('file.pgm','r')
header = next(infile)
infile.seek(len(header))
data = np.fromfile(infile, dtype=np.uint16).reshape((480, 640))
;; IDL
openr, lun, "file.pgm", /get_lun
header = {P5:BYTARR(2),width:BYTARR(4),height:BYTARR(4),maxV:BYTARR(7)}
readu, lun, header
data = intarr( string(header.width), string(header.height) )
readu, lun, data
free_lun, lun
% MATLAB
data = imread('file.pgm');
data = swapbytes(data);
If using my kinect_register program then reading a PGM is not absolutely necessary, as that program converts a single frame of depth data to an ASCII x,y,z file format. However, a single frame is noisy, and I suggest using the above PGM reader snippets to load several (or several hundred) PGMs and average all the good values of each pixel. Then, write out a new (averaged) PGM to be used as input to kinect_register. The snippets above can be used as templates for the writer function. The averaging will produce fractional numbers, but they will need to be rounded to whole numbers before writing the PGM.
If you have code to load 16-bit big-endian PGM files in another language, or a PGM writer function, feel free to share.
Tags: Code, Computers, HowTo, Kinect
The role of Pine Island Glacier ice shelf basal channels in deep-water upwelling, polynyas and ocean circulation in Pine Island Bay, Antarctica
Several hundred visible and thermal infrared satellite images of Antarctica’s southeast Amundsen Sea from 1986 to 2011, combined with aerial observations in 2009, show a strong inverse relation between prominent curvilinear surface depressions and the underlying basal morphology of the outer Pine Island Glacier ice shelf. Shipboard measurements near the calving front reveal positive temperature, salinity and current anomalies indicative of melt-laden, deep-water outflows near and above the larger channel termini. These buoyant plumes rise to the surface and are expressed as small polynyas in the sea ice and thermal signatures in the open water. The warm upwellings also trace the cyclonic surface circulation in Pine Island Bay. The satellite coverage suggests changing modes of ocean/ ice interactions, dominated by leads along the ice shelf through 1999, fast ice and polynyas from 2000 to 2007, and larger areas of open water since 2008.
@article{Mankoff:2012The-role,
Title = {{The role of Pine Island Glacier ice shelf basal
channels in deep water upwelling, polynyas, and
ocean circulation in Pine Island Bay, Antarctica}},
Author = {Kenneth D. Mankoff and Stanley S. Jacobs and
Slawek M. Tulaczyk and Sharon E. Stammerjohn},
Journal = {Annals of Glaciology},
Number = {60},
Volume = {53},
Year = {2012},
DOI = {10.3189/2012AoG60A062}}
Tags: Antarctica, map, NASA, NBP09-01, PhD, Pine Island Glacier, publication, Research 
Pine Island Glacier and Pine Island Bay
Pine Island Bay in the southeast Amundsen Sea, Antarctica, on 16 Nov 2008. Upwelling, melt-laden outflow plumes emerge from beneath the adjacent Pine Island Glacier ice shelf (top center) and mix in the bay waters. Warm red colors show sea surface temperatures more than a degree warmer than the near-freezing dark blue color. Cyclonic circulation in the bay is framed by the ice shelf, land ice and sea ice, in gray-scale with the darker shades colder. Landsat Enhanced Thematic Mapper Plus image, thermal infrared (channel 6H), subset of scene #LE72331132008321EDC00.
@article{Mankoff:2012The-role,
Title = {{The role of Pine Island Glacier ice shelf basal
channels in deep water upwelling, polynyas, and
ocean circulation in Pine Island Bay, Antarctica}},
Author = {Kenneth D. Mankoff and Stanley S. Jacobs and
Slawek M. Tulaczyk and Sharon E. Stammerjohn},
Journal = {Annals of Glaciology},
Number = {60},
Volume = {53},
Year = {2012},
DOI = {10.3189/2012AoG60A062}}
Tags: Antarctica, ice, map, PhD, Pine Island Glacier, Places, publication, Research, Visualization Last week I presented a poster at the 2011 AGU Fall Meeting. It has generated some press thanks to an article in Wired. If you are interested in the poster it is available by clicking on the image below.
AGU Poster: `Kinects as sensors in earth science: glaciological, geomorphological, and hydrological applications`
@conference{Mankoff:2011Kinects,
Author = {Kenneth D. Mankoff and Tess Alethea Russo and
Benjamin Kenneth Norris and Saffia Hossainzadeh and
Lucas H. Beem and Jacob I. Walter and
Slawek M. Tulaczyk},
Title = {{Kinects as sensors in earth science: glaciological,
geomorphological, and hydrological applications}},
Address = {San Francisco, CA},
Booktitle = {American Geophysical Union, Fall Meeting},
Month = {December 5 - 9,},
Note = {Abstract \#C41D-0442.},
Year = {2011}}
Tags: Kinect, Media, PhD, publication, Research 
Also of Interest…