Flickr-Based Multitouch & Multiuser Collection Viewer

Our Open Exhibits educational software initiative recently released a new multitouch and multiuser Collection Viewer allowing visitors to browse large museum collections.  The software pulls images and their descriptions directly from “sets” in Flickr, making the collections easy to manage.

The software has customizable “docks” that allow it to be optimized for touch wall or multitouch table displays.  The software is free for educational use. The video below shows how the software works.

The example we developed pulls images from the Women in Science Flickr set posted by the Smithsonian. Open Exhibits is supported by the National Science Foundation.


Welcome to Ideum Labs

Ideum Labs title imageIf you’ve read Ideum Blog and you’ve been keeping up with GestureWorks and Open Exhibits, you know that we’re fast moving and productive group. The unfortunate thing about being this productive is that we often don’t get the chance to share all the cool ideas, experiments, and discoveries we’ve made along the way.

Enter Ideum Labs, our new Tumblr blog. Labs was conceived as a high output, low production value home for Ideum and GestureWorks Staff to share raw ideas, UX experiments, project snippets, random acts of science, and interesting HCI + multitouch news.

What have we shared thus far? How about wood-paneled multitouch tables, unpainted touch walls, multitouch solitaire (a la Windows XP), Windows 8 beta, and multitouch Van Gogh.

Learn more about Ideum Labs at, or read all the Labs posts on the Ideum facebook page.


AAM Session: Museum Community Response to the Gulf Oil Spill

Tomorrow afternoon I’m participating in a session entitled, Saving the Future: Museum Community Response to the Gulf Oil Spill at the American Association of Museums Annual Conference. It is ironic that we are here in Houston, the U.S. headquarters of BP.

In the session, I’m going to be talking about the multitouch Gulf Oil Spill Mashup application that we produced last summer and provided free of charge to museums and aquariums. The application was built with the GestureWorks multitouch SDK and many of the software components in the application available free on the Open Exhibits website.

In case you missed it, here’s a video of the application in action.

I will post my slides here following my presentation. Looking forward to hearing how others in the museum community responded to Gulf Coast oil spill.

Updated May 24, 2010:  It was a great session to be part of and I found the other presentations really inspiring. In particular, I really enjoyed the talk by Jerry Enzler, the Executive Director of the National Mississippi River Museum & Aquarium. Last summer, their institution opened a major exhibit on the Gulf of Mexico. Their large aquarium tanks were deliberately presented with no fish or any marine life. The tanks were covered with decals representing oil from the spill.

It was a great example of a museum (and aquarium) taking risks and making a strong statement about the environmental disaster that occurred in the gulf.  The opening of this unique exhibit was a great success and the story appeared in the national news. At the time, there were stories about the exhibit in MSNBC, USA Today, and many other media outlets.

I hope to post more about the session in the future, as all of the presentations were very interesting. For now, here are the slides from my presentation, Gulf Oil Spill Mashup Exhibit (PDF 3.7 MB)


Controlling a Gigapixel Image With Kinect

[Cross-posted from Open Exhibits Blog]

We’ve recently released two new modules on Open Exhibits. The gigapixel viewer module allows Open Exhibits and GestureWorks users to plug any gigapixel image into our Flash application and drag and zoom it using multitouch inputs. We recently demo’d this app for the first time at CES 2011 and it was a big hit.

MT-Kinect, our other new module, allows users to interface with a Kinect to manipulate multitouch applications using gesturing (like in the movie Minority Report) rather than directly touching a screen. We combined this module with a gigapixel viewer to create an application that allows you to move and zoom by waving your arms.

So how does our application convert Kinect data to multitouch-compatible input that our Flash application can read? We wrote a “directshow” source filter, a virtualized webcam device that reads data from the drivers released by OpenKinect.

After adjusting the depth data to amplify the edges – which optimizes this application for gestures from a single user centered in the Kinect’s camera – we output a simple webcam feed. We route this information to a vanilla installation of CCV (theoretically, other  trackers should work), which runs various filters, finds the blobs, and outputs the data in whatever format we would like to consume (in our case,”flosc,” which enables Flash apps to get “OSC” information ). Our gigapixel viewer software can then read this input as though it came from any multitouch device.

These modules are free to download and use; you just need to be an Open Exhibits member. The gigapixel viewer requires that you have either Open Exhibits Core or GestureWorks software. Open Exhibits Core is available free to educational users. Commercial users can try GestureWorks free or purchase a license.

And if you’re wondering about the stunning gigapixel image of El Capitán, it was taken by xRez Studio who were nice enough to let us use the image for this demo.


Open Exhibits – Free Release and New Website, November 15

>>Reposted from Open Exhibits blog.

Last month, we announced that we received funding from the National Science Foundation for Open Exhibits. Today we’re happy to announce that, on November 15th and just 60 days after being funded, Open Exhibits 1.0 will be available free to educational organizations.

The Open Exhibits core software along with six modules and one template will make up the initial release. Open Exhibits core is based on our commercial GestureWorks software package.

Open Exhibits core is a multitouch framework for Adobe Flash and Flex. The open source modules and templates will be interoperable with GestureWorks 2.0 (to be released December 9) and free to all developers. Modules are functional components that can be used as building blocks for templates and exhibits. Templates are stand-alone applications that can be populated with content and customized via external XML files.

Here’s a run down on the software that will be available November 15th on the Open Exhibits Website.

Open Exhibits core: Available free to any museum, educational organization, nonprofit, or student. It is essentially a free educational version of GestureWorks (it does require registration). It has built-in gesture support (eventually for over 200 gestures), a built-in multitouch simulator, and an open source gesture library.

Onscreen Keyboard (module)  - A multitouch-enabled, scaleable, rotatable, and “skinable” onscreen keyboard.

Image Viewer (module) – A “local” image viewer that can handle PNG, JPEG, SWFs and other Flash-friendly file formats. Multitouch gestures can be used to scale and rotate the images.It supports basic metadata (title, description, author, and publish) via an external XML file. It can be skinned and some parameters can be changed via XML.

The Image Viewer Module

Video Viewer (module) – A “local” video viewer that plays .FLV video files. Multitouch gestures can be used to scale and rotate the videos. It too supports basic metadata (title, description, author, and publish) via an external XML file. It also supports captions and it can be skinned and some parameters can be changed via XML.

Flickr Viewer (module) – A Flickr image and video viewer. This module can draw media from a Flickr account or group. Multitouch gestures can be used to scale and rotate the images and videos. It requires a Flickr API key. It supports basic metadata (title, description, author, and tags). It can be skinned and some parameters can be changed via XML.

YouTube Viewer (module) – A viewer that can play YouTube videos. It has basic video controls. Multitouch gestures can be used to scale and rotate the videos. It requires a YouTube API Key. It can be skinned and some parameters can be changed via XML.

Google Maps (module) – A module that allows a Google Map to be controlled via multitouch gestures. It requires a Google Maps API key. Some parameters can be changed via XML.

CollectionViewer (template) – A collection viewer that incorporates all of the viewer modules. It can connect to local or networked images and videos. It has support for multitouch gestures (and multiple users). The look and functional aspects of the application can be controlled via XML. The template can be populated with content and become a stand-alone exhibit or it the source code can be modified.

These initial modules and templates will be extended and modified as Open Exhibits continues. Additionally, the new site will allow other developers the ability to build off these components or submit their own components to share with the community. The fun begins on November 15th!


Oil Spill Exhibit Update

Back in June, we announced the release of a free multitouch application that combined NOAA oil spill data, visitor contributed Flickr photos, and Google Maps.  (You can see that blog post and video,  BP Oil Multitouch Map Mashup.) The application was developed exhibit using our own GestureWorks multitouch framework for Adobe Flash.

This application is still available to educational organizations such as science centers and aquariums.  Since then a few different museums and aquariums across the globe have expressed interest.

Just this week we got our first photographs back from Petrosains Discovery Centre in Mayalsia. The exhibit is in their Hot Science exhibit area.

Gulf Oil Spill Application at Petrosains Discovery Centre

McWane Science Center in Birmingham, Alabama will be including the exhibit as part of their Science of an Oil Spill exhibition opening October 2, 2010.  We hope to have pictures very soon.

Also, the application was mentioned today in Art | Participation an Italian language blog and community site that focuses on interactivity and multimedia, Ideum e la mostra digitale della catastrofe del Golfo.

If you are a museum, aquarium, or science center and you would like to use this free multitouch software, please email us for details.


Multitouch Tutorials Abound

. . . on the GestureWorks site. Today, we’ve posted a tutorial on how to make a multitouch twitter application in Flash. Not your cup of tea? Maybe you’d like to make a multitouch Google Maps/flickr mashup or just learn the basics on how to create multitouch applications in Flash & Flex.

Our tutorials have been some of the most visited pages on the GestureWorks support site and and we’re looking to expand the list even further. We’d love to hear suggestions on what kinds of tutorials you’d like to see on the site. Tweet us @gestureworks or comment on this post.


BP Oil Spill Multitouch Map Mashup

Just like everyone else over the last two months, we’ve watched the continuing oil spill in Gulf of Mexico with a sense of helplessness and despair. Not only did has this unnecessary accident taken the lives of 11 people, it continues to impact millions more.  From an environmental standpoint, it it is nothing short of a complete catastrophe.

To help educate the public about this unprecedented event, we’ve decided to release a free version of our multitouch-enabled Google Map and Flickr mashup application to educational organizations such as science centers and aquariums.  The Google Map and Flickr mashup combines oil spill and fishing restriction data from NOAA (National Oceanic and Atmospheric Association) with eyewitness photographic and video accounts from the Gulf of Mexico. You can check and and join the BP Oil Spill in the Gulf of Mexico Flickr group; it includes some amazing photographs of the crisis.

Here’s a video of the application in action.

The exhibit software requires a multitouch system. We are offering it for clients who have multitouch tables. We will be building a version that is optimized to work with the 3M 22″ multitouch monitor, a lower-cost system. If there is interest, we may offer a single touch version or Web version of the application later this summer. Please email us for details about this software.

The oil spill mashup application is similar to a Google Mapping and Flickr mashup exhibit that we developed for the California Science Center. In that application, we used a number of KML data overlays to show various features of the LA Basin. You can learn more about that exhibit, L.A. Zone Multitouch, Multiuser Table, on the ExhibitFiles website. That exhibit and the new Oil Spill mashup were both developed using GestureWorks multitouch software for Flash & Flex.

Finally, with reports that British Petroleum (BP) continues to try to block media coverage, we hope that this exhibit helps in some small way to better inform the public about this disaster.

Update: Some of you expressed interest in who was using the application.  A number of museums contact us with interest in it including: the McWane Science Center, Trelleborg Museum (Denmark), Vancouver Aquarium, and at Petrosains, The Discovery Center (Kuala Lumpur). I know that Petrosains and the McWane Science Center used the application on the floor.


Oakland Museum 2.0

This weekend was Oakland Museum’s non-stop, 31 hour opening celebration. I was lucky enough to get a preview on Thursday and it is an impressive redesign effort. The Gallery of California History and the Gallery of California Art have been thoroughly reinvented. The Gallery of California Natural Sciences opens in the Fall of 2012.

We played a small part in the opening, as both the Art and History galleries have exhibits that run on our multitouch tables. We worked with Oakland Museum to design and develop software for the California Land Grab exhibit found in the History gallery. This multitouch and multiuser exhibit allows visitors to view high-resolution historical maps of California and other documents. This application was developed with our own GestureWorks framework for Adobe Flash.  Below are some photographs of the California Land Grab exhibit. There are more photographs of this exhibit and our multitouch tables on the Ideum Flickr stream.

California Land Grab Multitouch Exhibit

California Land Grab Multitouch Exhibit

A lot of newspaper articles covered the weekend opening, you can find a complete listing on the Oakland Museum website. One article in particular that caught my attention was in the Contra Costa Times, who published a map of the new galleries with descriptions and photos showing exhibits of interest. Check out Oakland Museum, 2.0 The Layout to get an overview of the new galleries.


A Look Inside the New and Improved MT-50 Multitouch Table

When we have visitors to our studio, we always like to show off the inside our MT-50 multitouch table. We’re very proud of the care and workmanship that goes into each custom-built table and we pride ourselves on using the best quality components we can find. We’ve decided to extend our inside tour to the web. Below, is our one-minute video tour of the MT-50.

The MT-50 multitouch table continues to evolve and improve. In a recent test, we broke 60 points of simultaneous input, making it a true multiuser platform (six sets of hands!). Our powerful computer, screen resolution and size, and number of touch points all top Microsoft Surface. (Compare our specs to theirs and see for yourself.) Plus, you can develop multitouch software using Adobe Flash with our GestureWorks framework without Silverlight.

And don’t forget to check out our other videos and subscribe to multitouch exhibits on YouTube.

Page 1 of 3123