Thursday, April 21, 2011

Alex Pang's Contemplative Computing articlefesto, take 1

Alex Pang, the researcher at MSR Cambridge whose work I've previously sweated just a bit, has recently (okay a month ago) published the first draft of a big article about what exactly he means by "Contemplative Computing." I want to call it a manifesto but he hasn't used the m-word so I won't either.

I quite like the article! I am glad that he is/was employed by Microsoft; I think he's got an approach towards creating technology that will help us improve our minds, and being at MSR means some of this could actually happen.

Some good quotes (and I'll attempt to weave them into a summary if you don't want to read the whole 60+-page article):

Intro, and what's wrong
"... we need to understand how tools initially built for the scientific laboratory or office may be ill-suited to the home or family; how the objectives of efficiency and optimization may not work in environments characterized by irreducible uncertainty and ambiguity."

"We need to develop personal tools to better control information technologies, and to see how technologies that often are described as irresistible and inevitable are really shaped by human decisions and choices (or the failure to make such decisions)."

"The Web is not frustrating because it does nothing but destroy your capacity to think; the problem is that it brings the promised land within view, but keeps it out of reach." It makes some kinds of knowledge work easy- like searching for the answer to a simple question- then makes you think "oh so all my work should be so easy." You reduce the time it takes to look up the year that Star Wars came out to literally 3 seconds (from at least 3 minutes in an encyclopedia) and feel like you should get a 60x speedup in everything else too. When writing your paper or app doesn't get 60x faster, you feel like you're losing out, and start surfing the web in hope of making up for lost time.

What is contemplation?
"Here, I define contemplation to be a form of detached, calm engagement." This is important. It's not just calm. It's that wonderful state of oneness with the machine (or with the musical instrument, or pen, or tennis racket).

And "detached" is important too: it's not (necessarily) video-game sucked-in-ness. You have the ability to monitor your senses and reactions, to be as aware of your own mind as you are of the guys around the corner with rocket launchers. "Detachment of one's emotions or ego doesn't prevent immersion in the moment; it's a precondition for it."

Anyway, is "contemplation" another word for Csikszentmihaly's "flow"? It's been a while since I read that book, but it feels the same.

Careful!
"While Csikszentmihaly wanted to understand how optimal experiences make for more complex and satisfying lives, readers of Flow in the Web and game worlds have tended not to ask questions like "How can we design computers in ways that let users experience flow throughout their lives" but rather "How can we use flow to make e-commerce sites stickier?""
Amen, sir! We must be careful if we seek to use these powers for any ends other than to better our lives! It's like martial arts: while you learn karate, you get a lot of powers. But as Mr. Miyagi knows, you don't learn karate to kick someone's ass.

Possible solutions
Pang then posits a few hypothetical pieces of software that could achieve this goal of contemplation.
- Proust is a writing software (or mode in MS Word) that removes distractions. I'd try it, but I don't just sit and write often enough to know if this would be useful. And when I do, it's often right here in Blogger and I have to find a bunch of links anyway.
- Quixote is a search tool that ... does a lot of things. "It's more like a space in which you search for and organize information..." Because, as I mentioned before, most questions are more complicated than "what year did star wars come out?" Imagine a mode to Google where you could search for N things and mind-map them out in a useful way.
- Sensei aims to bring together all the data from various lifelogging sources to help understand what causes contemplation.
- Xuanzang, though, was the most interesting: it's a tool to promote mindful wandering. You seed it ahead of time with high-quality things that you'd like to read/see/watch/hear while taking a break from concentrating on work. Cue it up, start going through these things (mindfully of course), and then after a few minutes, it asks you if you're refreshed enough to start working. This one had the highest product of awesomeness * doability.
To me, Sensei and Quixote sound wonderful, and at least a decade in the future. I'd be thrilled if someone were to start on either now.

Man, there's even more I could write, but this post is getting long and I'm super tired. Read the article if you're interested, it's only 60 pages.

Wednesday, April 13, 2011

Sleep ideas from Quantified Self Boston

are here. In general, it's all cool. In specific:

- Eric Smith tried wearing a Zeo while awake. It basically gave him nonsense data (said he was asleep a lot). This makes sense; it's trained on sleeping people. Still, too bad we can't just give people a Zeo and use that to test their wakefulness.

- Sanjiv Shah got a big result from wearing yellow glasses at night. Kind of like f.lux for the whole world. Interesting, especially given the fact that our circadian rhythms are generally 24.18 hours but electric lights stretch them to 25.

- "[Sleep doctor Matt Bianchi's] talk brought up a discussion around the relative value of exploring small effects. The thought is that we should look for simple changes that have big results, i.e., the low hanging fruit. A heuristic suggested was if, after 5-10 days, you’re not seeing a result, then move on to something else."
Fair enough. Especially when real-world things will give you big margins of error anyway. Say that there was a chemical in bananas that made me sleep 1% longer. There's no way I'll do an experiment that has enough power to find that effect in a way that I know it's not just chance; I'd have to collect data for years. Nor is it really that important.

(the weird part is: how did they pick 5-10 days? I've settled on 2 weeks as a pretty good test period for most life changes, but why 2 weeks then? It's like collecting data: getting "about 10 people" for simple experiments seems to be "pretty good." I guess you have to start somewhere.)

EDIT: while I'm on "cool things", here's an idea about dreams from Scientific American: dreams feel more profound to us than they do to other people because serotonin release is inhibited during dreams. (similarly, you get less serotonin while you're on LSD.)

I had another neat idea about dreams the other day (and now we've moved on to straight-up-guesses). So dreams might be caused by random firings in your brain. It's one of the hypotheses out there. If so, it might explain why dreams make so little sense: they're incompressible. If you generate a random string of characters, and pick a random compression scheme, odds are that you can't compress it much at all. So if you take a series of regular daily firings (say, a bunch of yellow lights in a circle) you might say "that's the sun" and compose a bunch of visual inputs into one quick phrase. But if you take random dream firings, it's really tough to compress them (and therefore remember them) at all.

Friday, April 8, 2011

A step back: is a wearable wakefulness system a good idea?

So I'm kicking around this idea for some kind of system that combines subjective sleepiness ratings, PVT, and some sort of biophysical signal to tell you how sleepy you are. Ideally, this should enable easy sleep research.

Is this worth it? Would you or anyone else use it? Would it actually enable sleep research, or would it just become another cool toy?

Personal sensing devices are apparently popular enough to sustain several companies today. In the sleep sensing space, there's at least Zeo and Wakemate. But that's just while you're asleep; Fitbit and Bodymedia/Bodybugg (a couple papers here and here) work while you're awake. Fitbit and Bodymedia are working on weight loss (how much activity you do) and how much you sleep. Bodymedia is interesting, because it has all these interesting sensors (GSR and heat flux) but they use them just to tell how active you are.

A couple other really neat papers are pushing the "wearable sense a bunch of stuff" system even farther:
1. the Optimi Project is using a multi-sensor system to detect mental health. ECG, EEG, Activity Sensing, Speech Analysis (?) and Sub-dermal cortisol (?!) combine to sense depression. It's been tested in the lab, and will be tested in the real world sometime this year. Paper reference (Majoe et al 2010).
2. the QUASAR system shows evidence for three things: first, that people are interested in personal sensors; second, that such things are feasible; and third, that every project everywhere is named "Quasar". ECG, EOG, EMG, etc. Matthews et al have described this as far back as 2007. I guess this is more of a "here are tools we can use" reference, but still a good reference to have.

(EDIT: incidentally, this Q-sensor is also interesting as another GSR tracker, and this just popped up in Forbes. I find myself feeling strongly about the whole Quantified Self movement, mostly positively, except for a few bits of nerdy rage because some people are "doing it wrong". Mostly positively, though.)

Thursday, April 7, 2011

Is skin conductivity/GSR a marker for sleepiness?

Galvanic skin response (GSR), AKA skin conductance/conductivity (and a lot of other names), might be a useful signal. It's cheap and easy to measure and widely known. It generally increases with arousal, and therefore is used in lie detectors. And E-Meters. Okay, maybe those are not great examples.

Davies and Krkovic (1964) found that EEG, skin conductance, and performance in a vigilance task all correlated for 10 college students. Skip forward 30 years, and Lim et al (1996) found out more details about the EEG-skin conductance (SCL) relationship. They found a correlation between each brain wave band/position pair and SCL, but then after searching for possible covariates (I might be botching this concept entirely) they came up with a model that only correlated Beta 3 (18-25Hz), Alpha 1 (8-10Hz), and SCL. I mean, the model was:
SCL = -0.685*EEG_Beta3 - 0.045*EEG_Alpha1 + 9.556
(both waves measured at Fz, which is the center of the frontal lobe). Generally, EEG and SCL both correlate with physiological arousal decline. Is that the same as sleepiness?

Bundele and Banerjee (2009) found that they can pretty well distinguish between pre-driving and post-driving skin conductance readings from drivers. They say that the difference is fatigue, so skin conductivity is a signal for fatigue. I say, could be fatigue, or could be stress or any number of other things that rise when you drive. Still, it's something.

Daniel Kramer (2007) studied performance on a video game and found that it correlated with skin conductivity. And Shimomura et al (2008) found that some analysis of skin conductivity correlated with task difficulty. These seem not really relevant here. But now I'm thinking, performance is linked to high arousal, and high arousal is linked to conductivity. If we just say "conductivity is about equal to arousal" and measure your arousal all day, maybe that would be higher on days where you're less sleepy.

So... what? Does GSR measure sleepiness? Probably not in any one case; if you measure your GSR right now, it's unlikely to say "GSR = 0.93, therefore you're sleepy." But maybe over time it'd be worth it.

EDIT: a couple other papers that I only have abstracts of:
Yamamoto and Isshiki (1992): well this looks promising. "We selected the GSR as a physiological index that indicates the awake level."
McDonald, Johnson, and Hord (1964): "Results showed that there were no differences between groups in GSR; however, the drowsy group showed consistently fewer spontaneous GSRs". (I guess GSRs happen in spikes, not just as a value; so maybe more awake people get more spikes in GSR)
Scholander (1961): response amplitudes of electrodermal activity was influenced by sleep deprivation.

Wednesday, April 6, 2011

Does Heart Rate Variability correlate with sleepiness?

Heart rate variability (HRV) is, well, how variable your heart rate is. If your heart beats slow, then fast, then slow, then fast, your HRV is high. If your heart rate is absolutely constant, your HRV is zero. Okay.

It seems to be a sort-of hand-wavey term; when you say "HRV", you might be talking about SDNN (standard deviation of intervals between heartbeats), rMSSD (square root of the mean squared difference of times between successive beats) or any number of other measurements. But these tend to correlate strongly, so it's not unreasonable to just talk about HRV.

In a study by Fang, Huang, Yang, and Tsai (2008), HRV didn't differ between normal patients and insomniacs. Peng, Lin, Sun, and Landis (2007) found that HRV helped their sleep-stage classifier, but they don't say anything about waking HRV. Back in 1973, Volow and Erwin found that HRV had a "marginally significant but unreliable" relationship to drowsiness onset. Furthermore, it's related to many things, but Wikipedia at least says nothing about sleep.

So why do I think that HRV might have something to do with sleepiness? Well, Kaida et al (2007) found that SDNN predicted worse performance on a vigilance task. However, they do note that this goes against some other results, like Hansen et al (2003) and Kohler et al (2006), which showed improved performance with increased HRV. Then there's Tsuchida, Bhuiyan, and Oguri (2009), who threw HRV and facial features into a classifier, finding a correlation with drowsiness of subjects in a driving simulator. ... which doesn't mean that HRV alone is a useful feature.

That's about all I've got for now. So far, HRV does not seem to correlate with sleepiness. It's a complicated measure, and there's just too much going on.

EDIT: it does help determine sleep stage, as Suzuki, Ouchi, Kameyama, and Takahashi (2009) found while combining it with an actigraph-ish watch. Also cool: the fact that they put HRV in a watch.