KenMankoff

Vertical Data in Google Earth

Google Earth does not officially support vertical curtains of data. However, it does support buildings with images on the side. If you are willing to stretch the definition of a building, you can put any vertical data you like into KML. This implementation was inspired by the Goddard Earth Science (GES) Data and Information Service Center (DISC) sample files for CloudSat, CALIPSO, and Aqua.

If you use IDL and would like to image your data in Google Earth, be it points, lines, regions, surface images, or vertical data, you should be using my IDL interface to the KML API.

Vertical Data in Google Earth

GLIMMER Ice Shelf Modeling (OS X HowTo)

A new beta version of the Community Ice Sheet Model, Glimmer-CISM, has been released. Below are instructions to compile and run it on OS X.

# build NetCDF
export CFLAGS=-m32
export FFLAGS=-m32
./configure --prefix=/Users/mankoff/local/netcdf-4.1.1 \
          --disable-cxx --disable-curl  --disable-dap
make && make install
say netCDF done

# build GLIMMER
cd ~/local/src/
wget http://download.berlios.de/glimmer-cism/glimmer-1.7.0.tar.gz
tar zxvf glimmer-1.7.0.tar.gz
cd glimmer-1.7.0/

# OS X has issues with 32 and 64 bit libraries.
# The -m32 flag forces 32 bit compilation.
# The following should be one long line:
./configure --prefix=/Users/mankoff/local/glimmer-1.7.0 \
     --with-netcdf=/Users/mankoff/local/netcdf-4.0.1 \
     FC=gfortran F77=gfortran CFLAGS=-m32

make
make install
say GLIMMER done

There are a few ways to test the installation. The source folder provides a test folder:

export PATH=/Users/mankoff/local/glimmer-1.7.0/bin:$PATH
cd ~/local/src/glimmer-1.7.0/tests/shelf
python circular-shelf.py circular-shelf.PP.config
python confined-shelf.py confined-shelf.PP.config
say GLIMMER Test Done # Takes a while. Turn up your volume

Running the above command will result in NetCDF files being created in the output/ subdirectory. You can view the contents of example.nc with most any generic NetCDF viewer. While theses tests run over a given period of time, the output only has one time stored. If you want to see an evolution of the ice shelf, older test suites available from the previous code repository site should be used:

cd ~/tmp/
wget http://forge.nesc.ac.uk/download.php/200/glimmer-example-0.6.tar.gz
tar zxvf glimmer-example-0.6.tar.gz
cd glimmer-example-0.6/
~/local/glimmer-1.7.0/bin/glide_launch.py ./example.config
say done

Examine the output file example.nc to see ice sheet evolution over time. Basal melt is shown below:

@article{Rutt:2009,
  Author = {Ian C. Rutt and Nicholas R. J. Hulton and Antony J. Payne},
  Title = {{The Glimmer community ice sheet model}},
  Year = {2009}}
  Journal = {J. Geophys. Res.},
  Volume = {114},
  Number = {F2},

Optical Character Recognition HowTo

Optical character recognition (OCR) is useful for converting images of text into normal computer text that you can edit, copy, paste and search. With OCR software you can convert old image-based PDFs to text. About 95% of words from a cleanly scanned PDF using a modern font are correctly recognized. Of the ~ 1 in 20 that confuse the algorithm, about half are easily corrected with a spell checker, and the remaining must be manually adjusted.

Google has made the tesseract OCR code they use for Google Books available. They don’t officially support OS X. Below are instructions to get tesseract-ocr running on OS X. As usual, Developer Tools (XCode) needs to be installed.

A slightly simpler installation does not use LibTIFF, but in this case you can only convert single page and uncompressed TIFF files. As PDFs are usually multiple pages, it is worth it to install and compile with LibTIFF.

# prepare dependencies
fink install libtiff libtiff-shlibs

# fetch
svn checkout http://tesseract-ocr.googlecode.com/svn/trunk/ tesseract-svn
cd tesseract-svn

# compile
./runautoconf
export CXXFLAGS=-m32 # Force 32-bit architecture
./configure --prefix=/Users/mankoff/local/tesseract/ --with-libtiff=/sw/lib
make
make install

# test run
~/local/tesseract/bin/tesseract image.tif out # run it
say Tesseract finished # might take a while. Turn on your speaker.
gn Finished Tesseract # alias gn='growlnotify -s -m'
less out.txt # check

Notes:

• Here is a sample image for testing
• The image must be in TIF format.
• The extension must be have one “f”: TIF or tif not TIFF.
• Images and complex equations are not handled

I tried to run tesseract on my handwriting and it could not decode it. I wrote a simple sentence as clearly as possible, took a photo:

The Quick Brown Fox

And the result was:

THE ®.U\(.K
[awww Fox
TUMPED oval
#IE uxzv ooé

It got “THE” and “Fox” and most of JUMPED. However, tesseract supports full training so if you need to convert your notes read the documentation and post what your learn below.

Building a Software/Hardware Environment for Research

Sometimes I get asked what software/hardware systems I recommend for research. Below is my standard advice. I am a scientific programmer and student and my day-to-day tasks involve research, data analysis,  scientific programming, and writing. Data sets might include satellite images, model output, GPS, seismic, and a variety of other data. Data domains are ocean, atmosphere, and/or ice. I travel a lot, either from home to the office or to the field (which might mean months on a boat with no internet connection). Therefore I try to have a setup that supports high performance computing (HPC) on a laptop.

More »

GISTEMP.StationData

NASA GISS has recently announced another year of the surface temperature trends. Last year, 2009, tied for the 2nd warmest year on record, and the past decade (January 2000 to December 2009) was the warmest on record.

I took the time this weekend to re-create a Google Earth layer showing these data. This visualization allows you to see a broad geospatial overview, and then select any particular location and see both long-term trends and even the data point for each individual month.

Download (330K)

GISTEMP.StationData

GISTEMP.StationData

KDM-IDL Code Library

I have spent quite a bit of time developing IDL code in the past, and it appears I will do some more in the near-term future. I have decided to release my IDL code library to the public so that others can perhaps benefit from my work. This is a work-in-progress and access is via Subversion.

One subset of the library is the KDM_KML lib for writing KML files. This code should make it fairly simple to display data sets in Google Earth from IDL.

I have currently implemented about 50% of the KML API and will get the rest implemented as time goes on. Right now layers can have overlays, points (placemarks), or linestrings (paths). Any element (that the API supports) can have a timestamp, and styles are supported too. With knowledge of the KML API you can create artistic balloons with whatever content you like that can even link to each other.

KML Interactive Sampler

If you create Google Earth or KML layers, you’ll want to find a 2nd monitor and keep the KML Interactive Sampler open full screen. An invaluable resource for hand-building a KML or when writing code.

KML Interactive Sampler

Post Twitter Experiment

I recently mentioned I’d perform a small Twitter Experiment. It was certainly an interesting week to dive into Twitter. The Iran elections had just occurred. Searching for Iran was worthless due to non-Iranians tweeting support, but a friend mentioned an interesting person to follow (they were on the ground in Tehran) and I felt like I had front-row seats to a revolution.

Somehow @bodc followed me on Twitter after I wrote my first post, before I even tweeted. This was the one benefit of my experiment, as I am now aware of yet another oceanographic data center. Other than @bodc, I did not gain anything by tweeting my occasional work progress throughout the week, and nobody following me replied to anything so I don’t know if anybody else gained anything either. What I did learn is that the web interface is worthless, and if you want to use Twitter it helps to have the right software interface.

I hear TweetDeck and TWhirl are excellent interfaces but they are based on Adobe Air and I am morally opposed to using Adobe products when possible… which doesn’t leave much. If you know of a non-Air alternative to TweetDeck please let me know. It turns out TweetDeck does have a non-Air-based iPhone client which is the best Twitter interface I’ve found to date. TweetDeck allows me to follow just my friends in one pane, a search for the word “Antarctica” in another pane, two Tehran-based Iranians in a 3rd pane, etc. It turns the non-stop stream-of-consciousness that is Twitter into a manageable stream of data.

In conclusion, I’m still not sure what or why, but I can tell I’ll continue to follow it and contribute via it on occasion. And due to the wide software and access support it is still the easiest way I know of to send small bits of information onto the web where they can be consumed by others devices or humans.

TeX Symbols

DeTeXify is a stunning piece of software (embedded in a webpage). When writing a LaTeX document one often wants to place a symbol in the document, but does not necessarily know the TeX command for the document. The solution was to search lengthy pages of symbols. DeTeXify allows you to draw the symbol with your mouse, and then provides likely matches and their TeX code.

My Twitter Experiment

Everybody else seems to be discussing and commenting on Twitter so I figured I would too. I don’t know what it is, nor why it is popular, but I do use it from time to time, and will begin a Twitter experiment starting tomorrow morning.

My Twitter usage is limited to following some friends who have chosen to broadcast their activities there rather than on a blog. Most of these people have blogs too, and I follow those also. In addition to following others, I have posted some thoughts there myself, and I’ve set up my parents solar thermal system to tweet when it starts to overheat. I chose this simply because it was the easiest way for them and the technicians to get an SMS on their phone, although there exists an infinite number of non-twitter implementations.

My Twitter Experiment will begin tomorrow, Monday June 14 and continue to Friday, June 19. I am beginning a new project for work, and will tweet my thoughts, progress, trials and tribulations as I go along. The project is officially one week long, although it will continue in other forms for the next few years until I finish my PhD.

The work this coming week will be to write some software to access, visualize, and analyze oceanographic data from the Amundsen and Bellingshausen seas in Antarctica.