Cognitive Design

Imagine combining brain-computer interface technology, transcranial direct current stimulation, emotion sensors, eye tracking. other physiological sensors and augmented reality gear with interactive game play.  What a brew for cognitive designers! Well that’s what you will get at the NeuroGaming conference and expo, May 7 -8 in San Francisco at the Metreon.

Using brain signals to control game play opens up many possibilities beyond entertainment.  There are specific panels at the conference that will explore how neurogaming can accelerate wellness, learning and other cognitive functions.

Best of all you can go hands-on in the expo and experience:

* A brain controlled light and sound show
* Throwing trucks with your mind
* The latest brain wave reading headsets and devices
* Virtual reality and full immersion environments
* Haptic, motion and gesture control
* Neurocmodulators that electrically change brain states.

You will see both medical and consumer grade applications.

There is even a two day hackathon the weekend before the conference where you can design, build and show off your own neurogaming concepts.

I hope readers that attend will share their insights here on the Cognitive Design Blog.

A recent episode of Through the Wormhole explored that question.  They examined how experts can ask questions and read eye blinks to figure out which playing card was drawn  from a deck, use an fMRI machine to build up a dictionary that maps brain states to the things we see in the world and use a neurofeedback device to help us achieve peak performance and learn new skills 230% faster.

The work on neurofeedback for peak performance is being done by Advanced Brain Monitoring.  In the show they demonstrate how their neurofeeback device can be used to help amateurs mimic the brains states of an expert archer to accelerate skill development in using a bow and arrow.  The device is shown to the right. Note the clip at the neck line. For learners, it sends a haptic signal to let them  know when they have achieved the expert/flow brain state.

They are also using the device in number of other domains including improving teamwork in complex settings.

Definitely not an off the shelf solution but it is ready made for research oriented cognitive designers.

The NeuroGaming Conference and Expo runs May 1-3, 2013 in San Francisco.  In neurogames technology is used to more directly link game play to your brain, nervous system and body.   Examples include touch stimulation, augmented reality and gesture-based interfaces, brain-controlled games, emotional dynamics and even the direct electrical stimulation of the brain to improve performance. 

The conference will cover games, therapeutic games, investing and trends. In addition, eye tracking, brain monitoring and others tools that provide a robust but cost effective way of measuring mental states are covered.   For example, Advanced Brain Monitoring will be at the conference. They offer a wireless medical grade EEG monitoring unit (shown directly above) that should be useful for all sorts of cognitive design studies.  Check out a short video on how it is being used to help uncover the neuro-correlates of strong leadership. 

Best of all you can directly experience the games and tools on the expo floor.

I hope readers that attend the conference will share their impressions and photos. I am especially interested to hear if you believe neurogames offer a 10x improvement in the gaming experience.

Self control is a prized resource.  Most of my clients and many students are interested in what it is and how to get more of it.  To design programs, products and services that improve self control we must first understand what it feels like. More specifically, what are the unique mental states, frame of mind or thoughts and feelings associated with a state of self control.  Until we have insight into that, it will be tough to apply cognitive design to the challenge.

 So what does self control think and feel like? We can study the question with introspection, self reports, ethnographic observation and protocol analysis.  And hope for an articulation (narrative description) that is accurate and specific enough to provide design insights.  We can also study the question with neuroscience and brain scans.  While lacking an articulation of the psychological contents, a neuroscientific  study can never the less lead to interesting interventions.

As an example of the neuroscience approach, I recently found a TEDx talk, A Brain Computer Interface for Exercising the Self Control Muscle, by Jordan Silberman a PhD/MD student at the University of Rochester.   We might not always know what self control thinks or feels like but we do know what it looks like in the brain (image to the right). Knowing that it is possible to develop an neurofeedback training scheme (see below) that correlates the brain state of self control to an external training signal (a bar that moves up and down on the computer screen.

This simple device lets the user learn to put themselves in the mental state of self control through rapid trial and error. By inducing the state you exercise your self control muscle. The video shows one experiment where people that used the training are able to maintain self control far better than the control group.

This is an exciting result with a lot of implications for cognitive designers once the technology matures a bit. In the meantime  you have to wonder, do people that go through this training have a better insight into the think-and-feel of self control? Can it be articulated and used to drive design innovations now?

Interested to hear from readers – how do you characterize the frame of mind (intellectual, affective, motivational, volitional) for self control?

For a $100 you can buy a headset that takes your brainwaves (EEG) and uses them to control various iPhone apps.  Make a ball float, play tug-of-mind with someone, sync your brainwaves with a song or just learn to relax. More are in the works.

Xwave runs on Neurosky’s technology so they are introducing nothing new in terms of a brain-to-machine interface.  Bringing brain interface technology to the mobile platform could really help this technology take off. Especially since ambitious cognitive designers can participate in the Brainwave Developer Program and create some game changing apps!

Hope they make one for the Droid soon.

Entries for Scientific American’s World Changing Ideas Video Contest are due September 15th. They are looking for 2-5 minute videos that describe innovative ways to build a cleaner, healthier and safer world.  Winners get written up in Scientific American (great exposure).   Entries are judged on impact, scientific merit, originality, entertainment value and production quality. You can read about last year’s 20 winning entries to get an idea of what made the grade in 2009.

This contest is an excellent opportunity to exercise your talent in design thinking. For example, imagine how much cleaner, healthier and safer the world would be if we had low cost, high reliability, easy to use and noninvasive brain-machine interfaces?

Cognitive designers seek to go beyond usability and look-and-feel to create specific mental states or a “think-and-feel”. Designing for pleasure, emotion, meaning, pain relief and improved decision making, learning and behavioral self-control are only a few of the application areas.  The goal is to optimize our designs for how minds work. 

Taking a systematic approach to cognitive design requires that we can somehow get between the ears of the people we are designing for and understand inner mental life and how it is shaped by features, functions and forms.

So I am always on the look out for new tools and techniques for modeling mental states and processes. The holy grail is neural decoding or the ability to translate measurable data on brain activity into the meaning of thoughts, emotions and actions. In short, directly reading the mind. The state of the art in neural decoding was discussed at a recent Society for Neuroscience meeting in Chicago.  The New Scientist offers an excellence synopsis in Brain Scanner Can Tell What You are Thinking About.

Nothing yet for the designer’s toolkit but some very interesting developments:

“He (Jack Gallant) and colleague Shinji Nishimoto showed that they could create a crude reproduction of a movie clip that someone was watching just by viewing their brain activity. Others at the same meeting claimed that such neural decoding could be used to read memories and future plans – and even to diagnose eating disorders.” 

Being able to accurately and cost effectively translate biometric information from our nervous systems into the corresponding thoughts, feeling, motivations and intentions will be one of the major innovations of the 21st century.  Among other things, it will provide the foundation needed to take an exacting approach to optimizing our designs for how minds really work. Cognitive design unleashed.

Mindset by Neurosky is selling for $209 ($199 + $10 shipping). A simple and easy to use wireless brain-computer interface. The software development kit that was selling for $9000 a year ago is now free!  Although advanced research tools are $500 and premium support is $1500. They are creating a community for developers to sell apps.

It is engineered to reliably detect changes in mental focus/relaxation. This is a very limited readout of mental states but if it works as advertised it should provide a new platform for commercializing a wide variety of cognitive design innovations.  I have ordered mine.

Small, light, dry, and wireless neurofeedback system for $1300. Looks like a simple single-channel EEG system but might be enough to do some basic cognitive training with. The vendor, ICAP Technologies,  has a strong “new age” orientation that may turn some people off.  However, if the technology is reliable it could be a great way to give people a quick path to their first experience in using technology to externalize and modify mental states.

Here is a screen shot:

Would love to talk to someone that has made the purchase.  

For all fans of work in brain-computer interfaces (and I am one because they will revolutionize cognitive design), you must check out this photo journal from the director of the Brain Computer Interface project at IBM’s Deep Computing Institute.