Monday, April 27, 2015

Thinking about metrics

Reading about effective altruism, the Open Philanthropy Project, GiveWell, etc, and thinking "good lord, how can they possibly hope to put a number on what's The Best Thing to do with your money?" It feels like they're taking a (to use the one design concept I sort of understand) wicked problem and trying to make it tame. Usually this doesn't go well; as the Vox article above hinted, you often end up only representing a couple of viewpoints, or making it worse by playing whack-a-mole by iteratively solving whatever problem you're thinking about at the moment.

But, I've got reason to assume, based on what GiveWell's done so far, that at least some of the Open Phil people are thinking about it in this broad sense.

And metrics aren't all bad! I think about WalkScore, which is limited and flawed, but is still a pretty solid and useful indicator of how nice it is to live in a place. And really, the thing is, we're often making decisions by metrics anyway, and often those metrics are suuuper flawed. Like GDP. So if I'm reading about a Social Progress Index on TED, sure, it might be a TED blowhard with another half-assed idea, but it doesn't have to be that good. I'd love to start talking about SPI instead of GDP, not because it's great, but because it's better.

Saturday, April 25, 2015

CHI 2015: some particularly interesting things

It was great. CHI, the biggest conference in Human-Computer Interaction, has revitalized me a bit; feeling grumpy and worn down, it helps to see a bunch of things that are actually exciting and realize that this "research" thing is actually ok and kind of worth doing sometimes. Spending a week in Seoul doesn't hurt. More on that later.

I ... want to write about so many things here. I'll try to just write about 10ish papers.

Talks I Enjoyed Because They Describe Interesting Phenomena Well

Your Money's No Good Here: The Elimination of Cash Payment on London Buses, Gary Pritchard, John Vines, Patrick Olivier. No cash means more work for customers, different work for drivers, drivers have to decide whether someone "might be vulnerable", loss of some social interaction, and the inability to gift. Among other ups and downs. But there is a curious side point: money is a technology too; why is that the assumed baseline?

A Muddle of Models of Motivation For Using Peer to Peer Economy Systems, Victoria Bellotti, Alexander Ambard, Daniel Turner, Christina Gassmann, Kamila Demkova, John M. Carroll. First of all, this was a top notch talk, flying through these theories but in a way I could still follow. Second, I like how they integrate a lot of these theories together. Or at least reference them all in the same place.

From Third to Surveilled Place: The Mobile in Irish Pubs, Norman Makoto Su, Lulu Wang. Phones change the pub dynamic in a few ways: you can look up quick facts instead of bantering about what the answer is (which is more fun anyway), you can use it for entertainment, but also everyone knows that if they do something silly (that is typically safe in a pub) you might be photographed and put on Facebook and that silly thing out of context will be embarrassing.

Talks I Enjoyed Because They Might Relate More Directly To My Work On Social Media, Urban Technology, Connecting People, and Maps

Modeling Ideology and Predicting Policy Change with Social Media: Case of Same-Sex Marriage, Amy X. Zhang, Scott Counts. They took tweets about same sex marriage in each state, and also compiled a list of same sex marriage bills that they were trying to pass in the same time. They could see correlations between words that indicate certain values (like "authority" or "fairness") and bills passing or failing. They could predict whether the laws would pass up to 87% accurately, where polls (which are slow etc) could only predict 70%.

"Everyone is Talking about it!": A Distributed Approach to Urban Voting Technology and Visualizations, Lisa Koeman, Vaiva Kalnikaite, Yvonne Rogers. They wanted to democratize data gathering and surface the data they gathered in multiple places. So they put a little thing with 3 buttons in shops along this road, and asked questions each day like "do you feel safe around here?", and then spray-chalked the % of people who said each thing on the chalk outside each store. Lots of super interesting results. Made me rethink the form of whatever I end up doing; not necessarily a thing-on-a-screen.

Making social matching context-aware - Design Concepts and Open Challenges, Julia M. Mayer, Starr Roxanne Hiltz, Quentin Jones. Sometimes you want to say "hey, you and you should meet." But there are a lot of factors that determine whether you should or not. This paper lays them all out pretty well. Some are individual, like maybe you are really busy right now. Some are temporal or spatial: in a train station, you're likely running for your train, not meeting randos. Some depend on how similar you are, relative to everyone else around.

An Evaluation of Interactive Map Comparison Techniques, Maria-Jesus Lobo. So if you have two layers on a map and you want to be able to see them both (to compare or whatever) what do you do? Answer: use a blended lens or a semi-transparent layer, not a slider or two maps next to each other.

Talks I Enjoyed Because Some Of My Friends Do Amazing Design Research

Making Multiple Uses of the Obscura 1C Digital Camera: Reflecting on the Design, Production, Packaging, and Distribution of a Counterfunctional Device, James Pierce, Eric Paulos. It's a camera encased in concrete except the button and the lens. So you can take thousands of photos, but if you ever want to get them out, you have to break the thing with a hammer. This is awesome: like Twitter or Snapchat, it puts a limit on a digital thing, and comes up with something really compelling in the process. Also, James packaged and sold it as a consumer device, on craigslist and in stores. This makes us think about it differently than a typical HCI study with an IRB etc., so it can generate new forms of knowledge. This paper and talk are pretty much the best.

Understanding Long-Term Interactions with a Slow Technology: An Investigation of Experiences with FutureMe, Will Odom. First, it's an interesting tool with a neat bunch of users: they send emails to their future selves. (or to someone else in the future.) Up to 60 years. And it's really deep; people send heavy stuff through this. So it's cool to hear about it, and it's cool to see "slow technology" actually deployed in action.

Talks I Wish I Saw But I Like The Papers

Can An Algorithm Know the "Real You": Understanding People's Reactions to Hyper-personal Analytics Systems. Jeffrey Warshaw, Tara Matthews, Steve Whittaker, Chris Kau, Mateo Bengualid, Barton A. Smith. They draw a line  between personalized (you like basketball) and hyper-personalized (your big 5 personality profile is ...) People find these hyper-personalized systems creepily accurate, more than humans, and they won't even correct them when they "make mistakes".

How Good is 85%? A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy, Matt Kay, Shwetak Patel, Julie Kientz. This paper addresses maybe my number one gripe with basically every ubicomp system, which is that it starts to recognize some thing with 85% accuracy, and then everyone says "ok, publish it" regardless of whether that accuracy (or precision or recall or whatever) is actually good enough to do anything in the real world. (if my self-driving car could see stop signs with 94% recall, I would not be in that car.)

Oh And Can We Talk About:

How good PSY was? Just the humblest dude, telling us about how he doesn't know social media, how one of his managers was like "we should upload Gangnam Style to Youtube" and he said "why?" Or how right now he's just trying not to be trying so hard, or about how when he performs everyone is just waiting for him to go "oppa Gangnam style" and that's ok? It was a simple talk, fun, and inspiring. Felt like the perfect end of the week.

How good Seoul was? Lots of fun sightseeing including a market of weird fish, temples all over, wonderful neighborhoods and cafes (my other blog will soon have a post about exactly that), super clean and safe, super easy, friendly people, lots of fun, and pretty cheap too, really. I... would hurry back to Korea, and encourage others to do the same, for work or fun.