Project Highlight: Mobile Therapy, Using MyExperience for Emotional Self-Awareness

Figure 1. Patients could self-report their mood

Dr. Margaret Morris from the Digital Health Group at Intel and colleagues from Oregon Health and Sciences and Columbia University recently published a paper entitled Mobile Therapy: Case Study Evaluations of a Cell Phone Application for Emotional Self-Awareness in the Journal of Medical Internet Research. They used the MyExperience tool for mood reporting and therapeutic exercises mediated by the mobile phone.

Here's a snippet from their abstract:

Background: Emotional awareness and self-regulation are important skills for improving mental health and reducing the risk of cardiovascular disease. Cognitive behavioral therapy can teach these skills but is not widely available.

Objective: This exploratory study examined the potential of mobile phone technologies to broaden access to cognitive behavioral therapy techniques and to provide in-the-moment support.

Methods: We developed a mobile phone application with touch screen scales for mood reporting and therapeutic exercises for cognitive reappraisal (ie, examination of maladaptive interpretations) and physical relaxation. The application was deployed in a one-month field study with eight individuals who had reported significant stress during an employee health assessment. Participants were prompted via their mobile phones to report their moods several times a day on a Mood Map—a translation of the circumplex model of emotion—and a series of single-dimension mood scales. Using the prototype, participants could also activate mobile therapies as needed. During weekly open-ended interviews, participants discussed their use of the device and responded to longitudinal views of their data. Analyses included a thematic review of interview narratives, assessment of mood changes over the course of the study and the diurnal cycle, and interrogation of this mobile data based on stressful incidents reported in interviews.

Figure 2. The MyExperience tool was also used for mobile cognitive therapy interventions. Here a glowing blue circle is animated to help a patient breathe in and out slowly.

Note that we featured Dr. Morris' work once before with the Mobile Heart Health post.

MyExperience at Measuring Behavior 2010

MyExperience will be discussed during a mini-workshop at Measuring Behavior 2010, the 7th International Conference on Methods and Techniques in Behavioral Research in Eindhoven, Netherlands. Details below:

Organizer

Jürgen Stumpp (Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany)

Aim of the meeting

In this user meeting we will show an overview of hand-held computer software solutions for psychological and behavioral monitoring. With this user meeting we want to provide a forum in which we can discuss the open-source MyExperience software as a solution for Psychological and Behavioral Monitoring. MyExperience is a context-aware data collection platform for capturing objective and subjective data as it is experienced. The MyExperience project was started by Intel Research, Seattle and the University of Washington in the spring of 2005. During the last year we ported the Platform to newer mobile phones and made several improvements and extensions which we would like to discuss with the participants. We would like to know if MyExperience fits the needs of the researchers in the field of behavioral monitoring and want to discuss further developments of MyExperience.

Program

We would like to do a small presentation on how MyExperience works and how it can help researchers in different fields to do studies with it. Also we would like to show how we improved MyExperience to be a complete solution for behavioral monitoring. We will bring a functional prototype that demonstrates the current development state and how MyExperience can be used in combination with mobile sensors that monitor physiological signals and triggers questionnaires on the hand-held computer.

(link)

MyExperience Version 0.9.1 Released

You can download MyExperience version 0.9.1 here. This was a bug fix release.

Release Notes:

- Fixed MyExperience.Analyzer error when trying to save results from query to csv file

- Added a check in Trigger.cs to see if the user did not specify a GetSensor or GetSensorStateSnapshot--if that's the case, we now print out a warning into the log file

- Fixed "The connection is already Open" in MyExperience.Analyzer

- Fixed problem when device does not use "Storage Card" for its external storage unit, MyExperienceLog.txt file getting saved in wrong directory. Now, the MyExperienceLog.txt is always stored in \Storage Card\MyExperience as designed

- Added a few more example XML files.

- Added build.bat and create_installers.bat to create the releases (should vastly reduce time it takes to create new releases and allow other developers to create releases as well)

MyExperience Version 0.9 Released

You can download MyExperience version 0.9 here.

The main window of the MyExperience application has been blank (pure white) since its inception. I have always planned on creating an interface here that could be customized through the MyExperience.xml file but have not yet gotten around to it. For example, this could be used by researchers to add a bar graph display that shows how many surveys have been completed and missed for the week. Although we're not to that point yet, I did add the ability to place an image in the background of the main window. See the screenshot below:

To change the background image, simply replace "front_screen.png" on your device (at \program files\myexperience). You can customize this with your own logo, instructions, or simply a calming image.

Another change of note is that the default MyExperience.xml file that is installed automatically has been greatly improved; it highlights many of the question/self-report features that make MyExperience unique.

Other changes in this release:

- Added a more comprehensive feature demo as the default MyExperience.xml file

- Fixed early timeout on PocketPC-based devices during surveys

- Fixed slight formatting issue in parameter text in questions

- Can now change the MyExperience frontpage screen by changing the image "front_screen.png"

- Updated MyExperience source base to Visual Studio 2008

- Updated MyExperience database backend from SQL Compact Edition 3.1 to SQL Compact Edition 3.5

- Updated MyExperience.Analyzer to work with SQL CE 3.0, 3.1 and 3.5 databases

- Minor update to TimeSensor. It no longer, by default, saves its state changes to the database (see the RecordStateChanges base property)

- Added two false-positive preventative measures to GsmMotionSensor: MinimumPhoneSignalStrengthThreshold and MinimumCellReadingCountThreshold. The ways this works is as follows: If you are sensed to be in motion and then either the phone signal strength drops below MinimumPhoneSignalStrengthThreshold or the number of visible cells drops below MinimumCellReadingCountThreshold, then the phone is pushed back into stationary mode and a state change is triggered. If you are already in the stationary state, nothing happens.

- Updated DatabasePopulator to throw an exception if an unknown tag is found

- Improved batch file installation routine

- Added better support for random question selection during surveys (e.g., GetRandomQuestionIds) See the Questions section of http://myexperience.wikispaces.com/Basic+Scripting+Functions

- Fixed an array handling bug in SimkinCS

- Added WiFi support to MyExperience. MyExperience.Sensors.Wifi contains : WifiDeviceListChangedSensor : triggers every time the list of good, very good or excellent wifi routers available change

- Added location inference support (e.g., user location tracking). MyExperience.Sensors.Wifi contains : SkyhookLocationSensor : localisation using skyhook. (see http://www.skyhookwireless.com/)

- Added CellInfoSensor.cs and NetworkConnectionSensor.cs

- Added auto-upgrade support to SQL CE database in Roam.SqlCe (see http://sourceforge.net/projects/roam -- Roam provides many of the foundational libraries for MyExperience)

Project Highlight: Studying Human Activities and the Environment

The College of Architecture and Urban Planning at the University of Washington used the MyExperience tool to study the relationship between the built environment and human activities. They conducted a study using MyExperience, GPS, and a wearable sensing platform (the Intel MSP) to record the activities and locations of 53 subjects over one week. In particular, they explored:

• what associations exist between different types of activities and different kinds of urban environments;
• what characterizes the land use patterns where people spend long periods of time;
• and do the properties of origins and destinations differ between short and long trips or trips made by different modes of transportation

In all, they gathered approximately 2,900 hours of data and ~2,900 surveys via the Experience Sampling Method (ESM) using MyExperience to provide self-report data about activities. MyExperience would prompt participants for a survey approximately every hour. These surveys asked questions about activity type, duration, purpose, and location.

GPS trace showing the path a person followed over several days annotated with self-report data from MyExperience. Stationary GPS points are shown in red, moving points are shown in green.

For more information on this research, see MobileSense - Sensing Modes of Transportation in Studies of the Built Environment by Jonathan Lester et al.

Project Highlight: Mobile Heart Health

Dr. Margaret Morris and Farzin Guilak at the Digital Health Group at Intel used the MyExperience tool to explore heart health, the physiological indicators of stress, and "active" mobile therapy (e.g., using the mobile phone to intervene during inferred times of patient stress).

From the article, "Mobile therapy—just-in-time coaching that is triggered by physiological indicators of stress—is the objective of Mobile Heart Health, an exploratory research project at Intel. The project aims to help people tune in to early signs of stress and modulate reactivity that could potentially damage their relationships and long-term health."

MyExperience Sensors
Morris used a variety of contextual MyExperience sensors to automatically assess stress level including location sensing, synchronization with calendaring applications, and a wearable ECG monitor (to track heart rate variability). The mobile therapy feedback "appear on the cell phone in response to cardiovascular signals (see Figure 1) or when the system detects contextual shifts associated with stress."

MyExperience User Interface

Morris also developed a variety of touchscreen feedback interfaces that allowed individuals to report their emotional states throughout the day. "The main mood scale was the 'Mood Map,', based on the circumplex model of emotion invites users to quickly indicate mood states according to valence and arousal (see Figure 3). These subjective reporting techniques were desirable as checks on sensors, and in their own right as brief windows for self-awareness.

To read more about this work, download the article directly here or browse the IEEE Pervasive Computing magazine that published it here.

Project Highlight: UbiGreen

The UbiGreen Mobile Transportation display was recently presented at CHI2009 in Boston, MA and is built off of the MyExperience tool. The UbiGreen Transportation Display is a prototype mobile phone application that semi-automatically senses and feeds back information about transportation behavior using the background (wallpaper) of the mobile phone (see screenshots below). In this way, the transit information is always available in a fairly casual manner so the user can see it as they use the phone for everyday purposes such as making a phone call or sending a text message. The full research paper is available here and the talk slides are available here.

The MyExperience.xml that powered it all is available for download here. Note that this file is rather large and complex as it illustrates an advanced use of MyExperience, but still could be educational for some (e.g., how the download and set wall paper actions were used).

The UbiGreen Mobile Transportation display used a variety of MyExperience Sensors, including:

• the GsmMotionSensor to sense user movement and automatically trigger a survey after this movement subsides (e.g., when the user transitions from being mobile to being stationary).
  
• UbiGreen also used MyExperience to interface with a wearable sensor (the MspSensor) that could automatically detect when the user was walking, running, or bicycling.
  
• the DesktopVisibleSensor to track how often the participant could see the wallpaper
• the DeviceIdleSensor to determine when the device was not being used by the user so we could run certain computations therby not affecting the interactivity of the device
• a host of sensors that tracked how the participants were using their phones
  

UbiGreen used multiple MyExperience Actions as well:

• the HttpPostAction was used to post sensor and ESM self-report data back to a server
• the DownloadAction was used to download appropriate wallpaper for the participant's phones
• the SetWallPaperAction was used to change the wallpaper of the user's phone (e.g., to a different tree or polar bear design)--remember, this was the main way that UbiGreen interfaced with its users, simply by changing the user's wallpaper
  
• the SendSmsAction was used to inform the research team when an emergency was occuring with a participant's device or wearable sensor (e.g., we would receive an SMS message if it appeared that our participant was not wearing their sensors that day)
• the RestartDeviceAction was used to force restart the device if a critical, unrecoverable error occurred
• the SurveyAction was used to contextually-prompt the user about trip information whenever vehicle travel was inferred but could not automatically be determined (e.g., we could not automatically disambiguate car from bus travel)
• the NotificationAction was used to alert the user that they had a self-report ESM survey waiting

Please email myexperience-users@lists.sourceforge.net or add a comment to this post if you would like more information.