The N questions you always need to answer for any research project, all the time

Especially when it's a new project, people will always ask you a lot of questions:

- What's your Research Question? (similarly, what's your Hypothesis?)

- What's your Contribution?

- Why is it Research?

- Who will it help? (or, who cares?)

- What are you doing?

- What problem are you addressing?

- Is that a real/important problem?

- Why will this solve that problem?

- How is your work different than (any one of 1000 tangentially-related things)?

- How is it done today, and what's wrong with that?

- How will you do it?

- How will you evaluate it?

It really helps if you can answer them all, all the time. You will get instant cred and people will let you do your thing. Unfortunately, it's kind of like air bubbles in a plastic sheet: when you squeeze some of them out, then some of them reappear. Like, if you nail down "how will you do it?" then people will ask "well if you know how to do it, then why is it research?" And if you nail down "Who cares?" down to a small subset, then people will ask "Is that an important problem?"

I'm going to order them from most to least important, in my mind: (note! I am not a grant funder.)

1. What are you doing? (Please, be concise. You get one sentence. Now try it in three sentences.)

2. What problem are you addressing? -- only if it's not obvious.

3. How is your work different than (100 closely-related things)? This is an important question if someone is actually bringing up something that they think is the same thing. This is not an important question if someone is just trying to sound clever.

4. Is this problem a real problem? -- downgraded because in HCI we solve lots of non-real-problems. And it's hard to tell what's a "real problem." If you mean "is it malaria?" then no, we're not solving malaria. You can always play problem-one-upmanship, and it's usually not a fun game to play.

5. Why will your work solve this problem? -- only worth asking if it's not obvious.

6. How is it done today, and what's wrong with that? -- downgraded because usually the answer is "it's not done today."

7. How will you evaluate it? -- again, sometimes it's hard to know until you do it.

These are sometimes not worth asking but people will anyway:

... 10. What's your Research Question or Hypothesis? -- This is valid for some kinds of research, like psychology. This is less valid in the more inventor-ish types of research. People will still ask it anyway.

11. How is your work different than (900 not-really-related things)? -- Sometimes people will ask this to try to sound smart.

12. How will you do it? -- if I knew, it wouldn't be research, would it? Still, people will ask this, and it helps to be able to wave your arms.

These are often not worth asking but people will anyway:

... 100. Why is it Research? -- Ugh. Academics love to ask this. Basically, why aren't you starting a company and doing this? And "because it's goddamn hard to start a company" or "this should be done but nobody wants to pay for it" don't count.

101. What's your Contribution? -- This is a thinly veiled version of "Why Is It Research?"

But yeah, I guess if you want to be good at research, answer all of them all the time.

Welcome to Domo

X

ADCHOICES IDENTIFIED

EDIT: the Domo server is now shut down. If you want access to any of this data, ask me to put you in touch with Shuguan Yang and Sean Qian who are running a server that has this data now. Or, ask me about the S3 bucket that it's all stored in.

I've told this to a lot of people so I've decided to store it all in one place. This guide will range from super-basic to kinda-complicated, so apologies if it's obvious in parts, and apologies if you get lost in parts. ALSO, if you're reading this and you're not new to our group and/or server, then you may have some advice for me, and I'd appreciate it!

Domo is our Amazon Web Services server. It's named after this guy:

On Domo, we have some coordinate geotagged tweets in some cities: (all stored in PostgreSQL database "tweet")
Pittsburgh: since Jan 22, 2014. (table tweet_pgh)
SF, NY: since June 13, 2014. (tweet_sf, tweet_ny)
Houston, Cleveland, Seattle, Miami, Detroit, Chicago, London: since November 7, 2014
Minneapolis: since March 18, 2015
San Antonio, Austin, and Dallas: since June 15, 2015
(everything after SF and NY is stored in tweet_(cityname)) where cityname is lowercase, all one word)
We also have tweets in Pittsburgh beyond just coordinate-geotagged tweets, in table tweet_pgh_all.
We also have Instagrams in Pittsburgh from fall 2014 to May 2016 (when Instagram shut off access to public geotagged Instagrams.) - table instagram_pgh
And some flickr photos and other misc data sets. (not in PostgreSQL; in /data/datasets/)

We really only interact with Domo via terminal windows, so if that's not your forte, you may have some difficulty. To log in, use "ssh (your username on Domo)@(Domo's hostname)"
If you want to make it easier, you can open ~/.ssh/config and add an entry:

Host domo
Hostname (Domo's hostname)
User (your username on Domo)

We store the tweets in PostgreSQL. If you've used other SQLs, it's pretty similar, but not the same. Things to know about Postgres and our DB in particular:

• psql tweet to connect to our database (which is called "tweet").
• \d to list all relations (aka tables, kinda)
• \d tablename to get more info about a certain table.
• The tweets go in basically direct from the Twitter 1% public feed (using this script). They're all stored as text and integers except for some things that are "hstores" - basically key-value sets - and the "coordinates", which are stored using PostGIS as Points.
• To access those Points, use some of the PostGIS functions. For example, SELECT ST_AsText to get it in a semi-readable format. ST_AsGeoJSON has been the most useful for me.
• To query all tweets within an area: SELECT * FROM tweet_pgh WHERE ST_MakeEnvelope(-79.9, 40.44, -79.899, 40.441, 4326) ~ coordinates; 
• (that "4326" is, for current purposes, a magic number. It means EPSG 4326/WGS-84 which is pretty much a standard for everything I do. So I always just leave it as 4326, and if you don't know better, I suggest you do too.)

Things to know about Domo:

• Change your password right away. Do this by typing "passwd" after you SSH in.
• Don't store things in your homedir! Our whole homedir partition only has about 8Gb. Obviously, that fills up fast. Store anything you can in /data - that has 1Tb. I might bug you sometimes to clean up your homedir if you end up using a lot of space.

When I add you to Domo, I'll tell you:

• your username on Domo
• your temporary password (change this as soon as possible)
• Domo's hostname (not shown here so we get attacked as little as possible)

You should tell me:

• if you want a username that's different than your email address, tell me ASAP and I'll create that and delete your old one.
• your github username so I can add you to our github organization.

Dan's note to himself:

• give the new person an account with sudo adduser username
• give them a postgresql account (CREATE USER username;) and give them permission to read all the tables (GRANT SELECT ON ALL TABLES IN SCHEMA public TO username)
• get their github username and give them access to the CMUChimpsLab organization too.

Emojis (and words) of Pittsburgh on the SUDS blog!

Hey, check it out. An article about work that Jennifer Chou and I did, on the Students for Urban Data Systems (CMU org) blog!

First paper(s) accepted, and thesis proposal proposal proposed!

It's an exciting time here for this mid-level PhD Student.

First! I've had a paper accepted to CHI, the biggest Human-Computer Interaction conference. Paper publishing is nice. It means that people can read about what you're doing, it means four people think your work is worthwhile and well done, and it's a tangible mark of success.

It's called "Getting Users' Attention in Web Apps in Likable, Minimally Annoying Ways", the title is pretty self explanatory, and it is but one more bit of sand in the large pile of research that's been done on notification systems. Still, web sites are still not very good at this, and our paper offers one potential piece of a solution, so for that reason I am pretty happy to have done it. Baby steps.

I have Anupriya Ankolekar and especially Josh Hailpern to thank for the opportunity to intern at HP Labs, the guidance in running the study and writing the paper, and the moral support throughout the tumultuous process. Thanks, you two, for helping out a research neophyte.

Second! I've had another paper, that I helped with, accepted to CHI, and this one with so many good friends and my fiancee Tatiana, so that's cool. "Mailing Archived Emails As Postcards: Probing the Value of Virtual Collections." This one's almost four years in the making, so thanks to Tati, Dave, and Jenny for gathering and analyzing data with me; Jason, Will, and John for advice along the way, and Beka, Will, and Dave for doing the writing heavy lifting. This was pretty epic, and I'm excited we can talk about it. Finally.

(I'll post both of the papers when stuff's more officially published.)

Third! Sorry if you are feeling down about your work and this feels like rubbing it in; I have been that student very many times (and will probably continue to be in the future) and I feel ya. I don't really mean to humblebrag, or even straight-up brag, so I've resisted facebooking or twittering any of this, but I'm really pretty relieved and excited to hit this milestone, and nobody reads this blog anyway. Plus, another data point for "hang in there, keep resubmitting your drafts, you'll win this game someday" I guess?

Fourth! I'm going to graduate someday. I've got my thesis proposal proposal done (meaning, I haven't done the proposal, but I've talked with most of my committee and figured out what my thesis proposal will be). Now all I have to do is run a study or two to plan the thing, do some background research, build a giant web app, do a lot more studies, etc etc etc, but you know, it's on the right track. I've got some great profs behind me, I'm stoked about the work, and I can do most of it while I'm with Tati. So.

Roads Greenery Buildings

What is your neighborhood made of?

We don't interact with zoning or construction in our everyday lives. We just know that some places are more pleasant than others. We don't really see the effects of dedicating half our space to parking lots and roads. We sort of know that New York is denser than suburban Ohio, but how dense is it?

More pragmatically, you may be looking for a place to live in a new city. You like your neighborhood now, so you wouldn't mind a place that "feels like" it. Obviously, midtown Manhattan won't feel like Squirrel Hill, Pittsburgh, but what neighborhood would?

Roads Greenery Buildings is an attempt to partially answer that question.

Give it an address, it will look up the place on Google Maps and Google Earth, and tell you the approximate amount of that place's nearby area that's taken up with roads, green space, and buildings. You can look up a few places to compare. Here we see that my neighborhood (the third one) has more roads and buildings than Carnegie Mellon (the first one) - which makes sense; CMU is a college campus with some big lawns. My neighborhood is also a little greener than nearby Oakland.

Here's a comparison of some neighborhoods in San Francisco, based on some coffee shops I like. Haus Coffee (the first) is in the greenest area (24th st. in the Mission is full of trees) but greenery is in short supply all around. This is to be expected; it's a big city. I was surprised to find the Ritual Hayes Valley branch (#4) to have so many roads nearby, but on reflection, there are a couple of big boulevards right there. Meanwhile, the area around Four Barrel (#2) and Saint Frank (#5) look the densest in terms of buildings.

This doesn't tell you everything, of course. The space calculations are imperfect, and there's no description of what the green space is (a highway median is less good than a nice park) or what the buildings are (a parking garage, a house, and an office skyscraper all get the same weight). But it's a start. I think of this (or, you know, the platonic ideal of this) as a peer to Walkscore: by no means the only tool that helps you understand a place, but one of many.

What's good? Depends on you, I guess, but I think this tool shows how places with more buildings tend to be more approachable and interesting, while green space often just makes things farther apart.

Try it out! (disclaimer: link worked as of May 2015; apologies if it's rotted since then.)

Hat tip to Andrew Alexander Price for the blog post that inspired this work. (More details.)

Thinking about metrics

Reading about effective altruism, the Open Philanthropy Project, GiveWell, etc, and thinking "good lord, how can they possibly hope to put a number on what's The Best Thing to do with your money?" It feels like they're taking a (to use the one design concept I sort of understand) wicked problem and trying to make it tame. Usually this doesn't go well; as the Vox article above hinted, you often end up only representing a couple of viewpoints, or making it worse by playing whack-a-mole by iteratively solving whatever problem you're thinking about at the moment.

But, I've got reason to assume, based on what GiveWell's done so far, that at least some of the Open Phil people are thinking about it in this broad sense.

And metrics aren't all bad! I think about WalkScore, which is limited and flawed, but is still a pretty solid and useful indicator of how nice it is to live in a place. And really, the thing is, we're often making decisions by metrics anyway, and often those metrics are suuuper flawed. Like GDP. So if I'm reading about a Social Progress Index on TED, sure, it might be a TED blowhard with another half-assed idea, but it doesn't have to be that good. I'd love to start talking about SPI instead of GDP, not because it's great, but because it's better.

CHI 2015: some particularly interesting things

It was great. CHI, the biggest conference in Human-Computer Interaction, has revitalized me a bit; feeling grumpy and worn down, it helps to see a bunch of things that are actually exciting and realize that this "research" thing is actually ok and kind of worth doing sometimes. Spending a week in Seoul doesn't hurt. More on that later.

I ... want to write about so many things here. I'll try to just write about 10ish papers.

Talks I Enjoyed Because They Describe Interesting Phenomena Well

Your Money's No Good Here: The Elimination of Cash Payment on London Buses, Gary Pritchard, John Vines, Patrick Olivier. No cash means more work for customers, different work for drivers, drivers have to decide whether someone "might be vulnerable", loss of some social interaction, and the inability to gift. Among other ups and downs. But there is a curious side point: money is a technology too; why is that the assumed baseline?

A Muddle of Models of Motivation For Using Peer to Peer Economy Systems, Victoria Bellotti, Alexander Ambard, Daniel Turner, Christina Gassmann, Kamila Demkova, John M. Carroll. First of all, this was a top notch talk, flying through these theories but in a way I could still follow. Second, I like how they integrate a lot of these theories together. Or at least reference them all in the same place.

From Third to Surveilled Place: The Mobile in Irish Pubs, Norman Makoto Su, Lulu Wang. Phones change the pub dynamic in a few ways: you can look up quick facts instead of bantering about what the answer is (which is more fun anyway), you can use it for entertainment, but also everyone knows that if they do something silly (that is typically safe in a pub) you might be photographed and put on Facebook and that silly thing out of context will be embarrassing.

Talks I Enjoyed Because They Might Relate More Directly To My Work On Social Media, Urban Technology, Connecting People, and Maps

Modeling Ideology and Predicting Policy Change with Social Media: Case of Same-Sex Marriage, Amy X. Zhang, Scott Counts. They took tweets about same sex marriage in each state, and also compiled a list of same sex marriage bills that they were trying to pass in the same time. They could see correlations between words that indicate certain values (like "authority" or "fairness") and bills passing or failing. They could predict whether the laws would pass up to 87% accurately, where polls (which are slow etc) could only predict 70%.

"Everyone is Talking about it!": A Distributed Approach to Urban Voting Technology and Visualizations, Lisa Koeman, Vaiva Kalnikaite, Yvonne Rogers. They wanted to democratize data gathering and surface the data they gathered in multiple places. So they put a little thing with 3 buttons in shops along this road, and asked questions each day like "do you feel safe around here?", and then spray-chalked the % of people who said each thing on the chalk outside each store. Lots of super interesting results. Made me rethink the form of whatever I end up doing; not necessarily a thing-on-a-screen.

Making social matching context-aware - Design Concepts and Open Challenges, Julia M. Mayer, Starr Roxanne Hiltz, Quentin Jones. Sometimes you want to say "hey, you and you should meet." But there are a lot of factors that determine whether you should or not. This paper lays them all out pretty well. Some are individual, like maybe you are really busy right now. Some are temporal or spatial: in a train station, you're likely running for your train, not meeting randos. Some depend on how similar you are, relative to everyone else around.

An Evaluation of Interactive Map Comparison Techniques, Maria-Jesus Lobo. So if you have two layers on a map and you want to be able to see them both (to compare or whatever) what do you do? Answer: use a blended lens or a semi-transparent layer, not a slider or two maps next to each other.

Talks I Enjoyed Because Some Of My Friends Do Amazing Design Research

Making Multiple Uses of the Obscura 1C Digital Camera: Reflecting on the Design, Production, Packaging, and Distribution of a Counterfunctional Device, James Pierce, Eric Paulos. It's a camera encased in concrete except the button and the lens. So you can take thousands of photos, but if you ever want to get them out, you have to break the thing with a hammer. This is awesome: like Twitter or Snapchat, it puts a limit on a digital thing, and comes up with something really compelling in the process. Also, James packaged and sold it as a consumer device, on craigslist and in stores. This makes us think about it differently than a typical HCI study with an IRB etc., so it can generate new forms of knowledge. This paper and talk are pretty much the best.

Understanding Long-Term Interactions with a Slow Technology: An Investigation of Experiences with FutureMe, Will Odom. First, it's an interesting tool with a neat bunch of users: they send emails to their future selves. (or to someone else in the future.) Up to 60 years. And it's really deep; people send heavy stuff through this. So it's cool to hear about it, and it's cool to see "slow technology" actually deployed in action.

Talks I Wish I Saw But I Like The Papers

Can An Algorithm Know the "Real You": Understanding People's Reactions to Hyper-personal Analytics Systems. Jeffrey Warshaw, Tara Matthews, Steve Whittaker, Chris Kau, Mateo Bengualid, Barton A. Smith. They draw a line  between personalized (you like basketball) and hyper-personalized (your big 5 personality profile is ...) People find these hyper-personalized systems creepily accurate, more than humans, and they won't even correct them when they "make mistakes".

How Good is 85%? A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy, Matt Kay, Shwetak Patel, Julie Kientz. This paper addresses maybe my number one gripe with basically every ubicomp system, which is that it starts to recognize some thing with 85% accuracy, and then everyone says "ok, publish it" regardless of whether that accuracy (or precision or recall or whatever) is actually good enough to do anything in the real world. (if my self-driving car could see stop signs with 94% recall, I would not be in that car.)

Oh And Can We Talk About:

How good PSY was? Just the humblest dude, telling us about how he doesn't know social media, how one of his managers was like "we should upload Gangnam Style to Youtube" and he said "why?" Or how right now he's just trying not to be trying so hard, or about how when he performs everyone is just waiting for him to go "oppa Gangnam style" and that's ok? It was a simple talk, fun, and inspiring. Felt like the perfect end of the week.

How good Seoul was? Lots of fun sightseeing including a market of weird fish, temples all over, wonderful neighborhoods and cafes (my other blog will soon have a post about exactly that), super clean and safe, super easy, friendly people, lots of fun, and pretty cheap too, really. I... would hurry back to Korea, and encourage others to do the same, for work or fun.

PhD Grind part 0.8 of N: picking a school

Man, we just had a bunch of students come through for open house, and I wish I could offer them some helpful advice, but we had a brief time to actually get to know them at all.

Sometimes people ask, why did you pick CMU? In order, it was 1. students, 2. profs, 3. outcomes, 4. everything else was fine. I think that decision order is pretty ok, but maybe profs should be tied for #1, not sure.

But! In more detail:
1. Students: I just felt pretty well at home with the students here. Hard to say why. Ok, not much more detail here, other than to say: this is important! Both the existing students, and the other incoming ones, will be working with you for many years. Maybe hanging out outside school too, depending on the place. (we're pretty social at CMU). You'll probably spend more time with them than the professors. You get a half a minute to get to know them at open houses, so that's frustrating, but if you find you really like (or dislike) the students at one place in particular, take that into account.

2. Professors: best case, you find an advisor. Second best case, you find a group of potential advisors where you'd be happy to be advised by any of them. (more info on this decision) If you don't have either of those, don't go to that place. Your advisor relationship is suuuuper important. More important than a typical boss. They will not only be your boss (so they can make your life easy or hard) but they will also be some inspiration, introduce you to other people, etc. So make sure you have an advisor idea before you sign the form.

One side note: if one place has your #1 top choice advisor, and another place has your #2 choice, go with #1, obviously. But if one place has your #1 top choice, and the other place has #2, #3, and #4, and you've never worked directly with any of them before, I'd say go with the 2-3-4 place. There's a lot of randomness that you don't figure out until you work with someone, and someone who seems perfect might just for whatever reason end up not a perfect match. So in that second case, go with the #2 advisor, and you've always got 3 and 4 if 2 doesn't work out.

3. Outcomes: notice what the graduating students are doing. If they're all getting the kind of sweet jobs you want to do, that's a good sign. If half are doing startups, and you want to do a startup, great! If they're all going to industry jobs, and you want to be a prof, not so great. If 2/3 of them are dropping out and sitting at home wallowing in depression, probably not so great either.

4. Everything else: these will probably not factor into your decision, but look out in case they do. City, stipend, classes, quals, teaching, office space, etc. Usually these will all be fine: the city is ok, the stipend is about the same, you have to take ~8-10 classes, you have to pass through some hazing ritual such as quals or comm talks or extra classes, you have to teach 2 classes, your office is a desk in some depressing basement, etc. These are all fine. But this is an area where there can be red flags: if you don't have a guaranteed stipend, if you have to teach every semester, if the city is super expensive and difficult to live in (lookin at you, Stanford) or in the middle of nowhere and depressing to you, the hazing/quals makes half the students quit, etc. If there are any red flags, they should factor into your decision as much as the above 3 points. Otherwise, don't worry about these things, they're fine.

So! If you've considered all these, go with your gut then, and you'll do fine. Also, if any current prospective students end up reading this and want to talk more, find my email (or twitter), drop me a line.

Not really research, but fun: Swot Perderder

I remember seeing stuff like this and this and thinking it was the funniest thing ever. I guess Know Your Meme categorizes it as "wurds", but to me, this will always be Swot Perderder, because this one in particular always made me crack up:

So I made a twitter bot that generates these:

Tweets by @swot_perderder

I got a list of ~500 foods (harder than it may seem), used the CMU pronouncing dictionary to translate word -> phonemes, then mapped each phoneme to a randomly chosen letter that either matched it closely (s -> s) or not so closely (s -> zh).

That's all for now. It'd be neat to make it interactive someday, or make it a reddit bot or something, because the world needs more of these misspelled foods.

Also, it'd be neat to see which of these are favorited/retweeted/etc more, because then we could refine the rules to make them even funnier. Yes.

Edit: code's here: https://github.com/dantasse/swot_perderder

Pittsburgh Tweets

Heyo. Here's where (the geotagged tweeters in) Pittsburgh tweeted in 2014.

CheeseHoods

CheeseHoods: a block of Swiss cheese as dense as your neighborhood.

CheeseHoods are blocks that look like Pittsburgh neighborhoods. They have holes in them like Swiss cheese. The denser the neighborhood is, in terms of dwellings per acre, the fewer holes it has.

Why neighborhood blocks? I thought it'd be fun to have a jigsaw puzzle of Pittsburgh, really. Plus, just as playing with world maps helps kids learn country names, playing with your city might help you learn more about it.

Why the holes? Density is important. Jane Jacobs wrote about it as one of the four most important characteristics in creating vibrant neighborhoods. Dwelling density, in particular, is quite important; human density can just indicate overcrowding, but dwelling density indicates vitality. I was interested to explore what density looks and feels like in 3D. Putting holes through a neighborhood seemed like an easy way to do so. Plus, it gives the least dense neighborhoods a rather icky pock-marked feel, while thriving denser ones are pleasantly solid, so this gives a really visceral feel to "density is good".

Pictures just Bloomfield, above. Below, all of Pittsburgh. Central Oakland is a shining example of density, but you can see right through Greenfield (below). Code's on github. Not embedding it here because there's a lot. I used some tools from another repo (get_dwelling_densities.py) plus some census data to calculate dwelling density for each neighborhood, then computed a json file of all the borders of each neighborhood and random hole locations (nghd_to_shape.py), then finally slurped those into a last script that created objects in Rhino (rhino_script.py). Play with 'em:

Bloomfield by dantasse on Sketchfab

Pittsburgh by dantasse on Sketchfab

PhD grind part X of N: Results from a two week time diary

I'm TAing with Jen Mankoff now, who's somewhat of an enthusiast for time management, so I'm trying to learn a few things from her. One thing she suggested was a "time diary" - just write down everything you do, so you can find out what's actually taking up your time. I did it for two weeks, and here's what I got per week (average of the two):

Research: 19 hrs
TAing: 12 hrs
Class: 4.8 hrs
Email/logistics: 7.6 hrs
Socializing: 6.7 hrs
Waste (checking the internet, etc): 4.8 hrs
Other (walking between buildings, lunches that didn't fit in other categories, fighting with the internet when it went down, WC breaks, etc): 2.5 hrs
Total: 57.2 hrs

Things I learned from this:

• I'm not doing so bad, part 1. I'm putting in a lot of time into research. (This was on the week before and after the CHI deadline, so "research" is a little higher than normal. So if you're reading this and you think "oh, I'm such a lazybones, I only work 50 hours a week", two things: 1. these are perhaps abnormally high weeks for me, and 2. count it out yourself, you may find you work more than you think.) But I'm at least putting a lot of time into "research", which is good.
• I'm not doing so bad, part 2. There's not a ton I could cut out. I guess I could cut the "waste" time down, but I don't think I'll ever hit 100% efficiency anyway, so 90% seems not so bad. Maybe I could cut down email/logistics, but there'll always be a need for some of it. I guess I could cut down socializing, but that... seems wrong. This "socializing" is the all-important "networking" if you want to be super utilitarian about it - this all may further the all-important career. Attending department lunches or lab group meetings, meeting visiting profs, hanging out with PhD friends and chatting, whatever. And no, I don't actually think about it as "networking" while I'm doing it.
• I thought class was a big time suck. Maybe not. (though, again, CHI weeks, I really pushed class out of the way. I spent a ton more time on class the week after I time-diaried.)
• Grad school is a great environment to do research on anything you want... after you finish your required stuff. And there's ~35 hours of required stuff a week. ("research" counts all the time I spent on research, including filling out IRBs etc, so even some of that wonderful 19hr chunk is not so wonderful.) Which means, if you're a 40 hour worker, you'll have 5 hours to do RESEARCH, and you'll be frustrated. If you're a 70 hour worker, you'll spend half your time doing RESEARCH, and it'll be great. ... Be warned.

Edit: related is this post where this guy Togelius comes to the exact same conclusion as me, but frames it as "increasing marginal utility." I guess it's an optimistic way to look at the same thing.

Or... I could also get a job writing dumb software for 35 hours a week and then do research in my spare time, and be equally effective. (and paid a lot more, and I get to crank out some dumb software in the meantime.) Well, except I wouldn't be equally effective; in those 35 hours, I get these 3 benefits:

1. I learn something from class, and something from teaching, and I guess something from filling out IRBs and stuff

2. being in the university gives me access to papers, conferences, research funds to run a study, etc.

3. I make actual friends while "networking".

1 is true, but I'd learn something from writing dumb software too. 2 is true but unfortunate; I mean, the system shouldn't exclude people who just don't happen to have a university affiliation. 3 is true, ok.

It still feels like a lot of waste and frustration for those three benefits.

PhD Grind Part 1 of N: advisor picking

Philip Guo posted a great guide called The PhD Grind, a memoir of his computer science grad school experience. Mostly just "here's what I did", not so much advice, but there's a bit of both. It was really helpful for me, as one more data point of what grad school can be like. I'd love it if I could publish a similar thing, and further help people who are going into this path.

(I'd have to preface it with a bunch of disclaimers, and the biggest one would be this: grad schools vary A LOT. Everything I say will be very relevant if you go to get a PhD at CMU in the HCII. If you're going to other schools in HCI or related fields, this will be about 90% relevant and accurate. If you're going to other schools in CS (non-HCI), this will be maybe 60% relevant. If you're going to grad school in another field, maybe 10%. Seriously, I have no idea what grad school in other fields are like, besides that a lot of them are broken and terrible and you should not go to them. Grad school in CS, particularly HCI, is one of the least broken types of grad school.)

Picking Advisors

Anyway, one thing I realized I've gained a lot of insight into is picking advisors. At CMU HCII, we got the first few weeks to meet with different advisors before we had to decide. (I think, if you're going to PhD school, especially in HCI/CS, you should have at least a pretty good idea of who your advisor will be before you accept an offer, considering how important it is. But anyway, CMU let us choose.)

When I was picking advisors, I didn't really know the questions to ask. I mostly asked, "I don't know, are they good? What are their pluses and minuses?" Most people would say yes, they're good, tell me a couple obvious pros/cons, and then say something about whether they're "hands on" or "hands off." This is mildly helpful, but imprecise; it lumps together a lot of different factors. It's also not very distinguishing, because most professors (at least here) are "mostly hands off". You'll also get some platitudes about how they're very supportive, and they care about their students, and etc. These are true too, but also not very distinguishing, because all our professors here are pretty great.

You want to ask questions that will distinguish between profs who are right for you and who are not! And you want to know more dimensions than good/bad, hands-on/hands-off. Here are things you should ask. Try to ask these in ways that don't imply a value judgment, because if there's a "good" and a "bad" option, people will almost always tell you the good one, because most advisor/student relationships are good. (if they're bad, they usually don't last very long.)

Ask the prof

• What grants do you have, or what project will I work on? (unless you have a fellowship.) Whether they tell you immediately or not, you will have to be officially working on one main project, and they will have to fund that somehow. If you can avoid it, don't go in with a vague area of focus (like "ubiquitous computing") and plan to figure out the project later.
• How big is your lab? By asking this, what you really want to figure out is: how much time do they have for you? And it can be fine if they have only a little time - depends on your style. Some people like to do their work and be left alone; if the advisor only meets with you once a week and rarely responds to emails, that can be enough. But some people like to work more collaboratively and meet/discuss/email more often. This is one segment of the hands-on/hands-off distinction. And I'd say having like 3-5 PhD students, plus a handful of Masters/undergrads/postdocs, so like 8-10 people total, seems like a medium sized lab.
• How much do they like/dislike collaboration with other students? I get the sense that most profs like collaboration, but maybe they'll give you a clue: some really like it, and others are kind of "meh, it's okay" about it.
• Do you anticipate any big life changes in the next 6ish years? They'll probably have a sabbatical year sometime in there. If they're pre-tenure, that review may come up halfway through your career; unlikely to be a problem, but if you're 3 years in and your prof doesn't get tenure (which means they get more or less fired), that might be tricky. Are they considering moving schools? (they probably won't tell you if they are, but worth a shot) Are they considering retiring?
• How intimidating are you? Okay, don't ask this, but get a sense of it. Some profs (usually older ones) are more intimidating than others. You should probably feel a healthy respect, but not fear; that will hamper your work and life. Take note of this feeling, because it's not likely to change a lot.
• Industry or academia? I mean, if you already know which path you want, ask the prof if they will be good at helping you get to that path. They will be pretty honest about this.

Ask the prof's current PhD students

• How much will the prof shield you from funding? Profs all try to do that, but sometimes it works out better than others. Good way to ask it: "Have you ever had to work on a project you weren't super into, because of funding? Tell me how that went." If none of the students have, that's pretty good/lucky; if they all have, take note of that.
• How often does the prof ask you for work-related things? (this is part of the hands-on/hands-off thing too.) Some profs bug you every day or two for something, big or small. Some are fine if you don't give them anything for a month. The micro-managey prof can be good if that motivates you to work better.
• How much will they ask you to do other stuff besides your research? Group meetings, mentoring students, maintaining servers, organizing stuff, meeting with funders, etc.
• How much do they take your feelings into account? (you might be able to tell this from interacting with the professor too) Some profs have a very academic, businesslike "let's not sugarcoat it, let's just argue to find the truth" kind of demeanor. Some profs are more, well, friendly. Again, not a good/bad; some students like to just talk shop and not be all touchy feely and can deal with blunt criticism. (it's okay if you don't like blunt criticism; make sure you find a more friendly professor then.) This goes along with, and is less important than, the "intimidating" thing above; the reason to ask the students too is just to see if they have mood shifts that makes them affable most days but terrifying on days when there's bad news or something.
• Related: what's the filter you have to put on this prof's speech? I've had profs that got waaaaay easier to work with after I applied the following filters:
• Prof. A sounds like they think everything that comes out of your mouth is really dumb. They don't think that. Prof. A is really on your side, it's just the way Prof. A talks.
• Prof. B will always sound a little disappointed in you. Prof. B is not actually disappointed in you, they just sound that way.
• Prof. C thinks everything is a great idea. You'll come out of meetings with Prof. C thinking you can do everything and solve the world! Furthermore, that you should do everything! This isn't true; Prof. C is just very optimistic and really doesn't want to crush your ideas.
• Prof. D is very friendly and easy to talk to, except Prof. D is always giving you advice, so you think that Prof. D thinks you are a fool or a small child. This isn't true; Prof. D doesn't actually think you are a fool, they're just trying to help.
• Prof. E is never satisfied. If Prof. E says "eh, that's ok", you basically cured cancer. If Prof. E says "that's a terrible idea and you've accomplished nothing", you're doing fine.
• This is just a sampling. Because your advisor is so critical to your work, it helps to figure out the little tiny nuances of how they communicate.
• How available is the prof, if you need help? Will they answer an email within a day? A couple days? Will they meet with you more than your once a week meeting if you need it? Can they help you to find other help if you need it?
• How much does the prof like/dislike collaboration with other students? Do they push collaboration? (this can be good or bad depending on how you like to work) Do they discourage collaboration? (it happens) Do they really try hard to build up a group dynamic in their lab, or is it all a bunch of people working more or less individually? (either can be good.)

Not sure who to ask, but it's good to figure it out

• How ambitious are they? You'll probably have to determine for yourself if you're on the "ambitious" side (want a professor job at a top tier university, want to be in the news, want to be in TR35, etc) or the "balanced life" side (want an industry or a prof-at-a-not-so-big-name-school job, have other interests you want to continue pursuing outside school, like the 9-5ish life, have other constraints like you have to finish in N years due to some visa thing, etc). Your profs are all going to be super successful, but if they're in the news all the time or getting big fancy awards or on track to do so, they might be on the "ambitious" side. Also, if they're pre-tenure, they're likely to be more ambitious. Also: it's okay to be "balanced life." I am.

Some things I learned from running a big Mechanical Turk study

I'm not a big crowd researcher, but Mechanical Turk can be a great platform. The key words are "can be". It sounds great: pay a thousand people a dollar to do your survey, and for $1k, you have a huge amount of data overnight! But it's not really that simple. Here are some things I've learned. (there are a lot.)

Human/study design:

• Read this. These folks have worked and researched on Turk a lot longer than I have. http://wiki.wearedynamo.org/index.php/Guidelines_for_Academic_Requesters
• Pay people enough. This is maybe the #1 thing I hear on Turk forums, and the #1 piece of advice I can give to make the whole thing a good experience. These are people doing work. It's not just screwing around for fun. HITs are hard. Turking is hard. See also: Jeff Bigham's experiences Turking for a day.
• $8/hour is a starting point. You're not getting by on $2/hour like you may have thought you could back in the early days. Or, if you're getting by on $2/hour, you are paying people sweatshop wages (and by the way, probably getting sweatshop level work). Why $8? People want to make about US minimum wage. It's a nice round benchmark, at least. It's still way cheaper than you could get it done any other way.
• Pay more if you can. The US minimum wage is (adjusted for inflation) historically low. $8/hour is in no way a living wage. (also, it assumes that Turkers spend no time searching for tasks or any other overhead. Seattle recently voted to raise their wage to $15/hour. Maybe you can too. Related: Dynamo's Fair Payment page.
• Turkers are good people. At least, the ones that you get for $8/hr are. They are not, for the most part, trying to scam you. Maybe 2% are. Accept that as a cost of doing business (that's what, 2 extra precious dollars?) and don't get too defensive about your task. 
• 98%/1000 is a good threshold. That is, require turkers to have 98% acceptance and 1000 HITs completed. When we tried 95%, we got a few more rejects (though not dramatically more). When we restricted it to 5000 HITs completed, there were only about 300 qualified people who would do our task.
• ACs (attention checks) are tough to get right. In our survey, we included three simple math questions: "what is two plus three?" etc. But then, even these are not perfect indicators of whether people are paying attention. We had 7-point likert scale answer options, so people would go try to click 5, and just miss and click 6 by mistake (or they'd be using a touchscreen or something). Also, dashing 15 minutes of work just because of one question seemed pretty cruel. I started accepting people if they got our ACs wrong, so long as they only missed one question and were just off by one. Relatedly:
• Spell out exactly what will make you reject people. Our HIT had a list: "You will be rejected if..." This makes it much easier to deal with somewhat-angry Turkers who write to you. Many times, Turkers will get really mad if you reject them for something you didn't warn them about. I think that's a feature of mturk, not a bug.
• Verifying they did surveys is mostly easy but not completely. You can't get their Turker number into your system. My standard approach is: you do our study, at the end we give you a number, then you enter the number into Mturk and we correlate your records with ours. About 1% of people don't understand this, or otherwise screw it up.
• Reputation matters. If you're a crummy requester, folks can rate you on Turkopticon (TO) and talk about you on Turker Nation (TN). But if you're good, they'll rate you up on TO/TN, post you on Reddit's HITs Worth Turking For (hwtf), follow you with TurkAlert, and occasionally really get into it.
• Engage. Get on Turker Nation and hwtf. (you can post your own HITs on hwtf, and in the requester forums on TN.) Respond to Turkers like you'd respond to people you hired to do a job, because they are people you hired to do a job.
• Conflict is tough. You have only blunt weapons, and so do they. Because most HITs are accepted, and because the difference between being a 95% turker and a 98% turker is so big, every rejection really hurts workers. Also, they can trash you on sites like Turkopticon- not sure how much that matters, but it doesn't help. One disgruntled worker can make things difficult for you. So if someone's getting all angry at you, you're kind of incentivized to just pay them and get them off your back, before they go reviewing you all over the place.
• Performance-based bonuses are good. We structured it as 30 cents base, plus 15 cents per Set you find. (our task was the game Set, where you try to find a bunch of sets of cards that fulfill certain criteria.) This meant that we ended up paying about $1.40 per person, but the best people would post in forums "I got $3.50 for this 15 minute HIT" or whatever. In general, most people ended up doing pretty well at the game, which I guess is a good sign that they were paying attention to it, which is good. We were worried that people would overlook our HIT because the base pay was low, but it was helpful that we could list it as a 30-cent HIT, and then say in the title "+ average $1.10 performance bonus!" (figure out that average value through pilot testing.)
• If you let them, workers will repeat your HIT. We started off posting 40 assignments of a HIT at a time, but noticed that about 3/4 of our users each time would be repeaters. They use stuff like TurkAlert. So if you want not to have repeaters, make sure you just post one big HIT. (or use other 

Technical:

• There's no official Python API, but boto is pretty good. Documentation can be sparse, but supplement it with the mturk API, and you can usually figure out what you need to do.
• The main Manage Hits page is mostly garbage. The one you want for most things, especially if you use the API too, is Manage Hits Individually. Looks like they wanted to replace the MHI page with the MH page, because it's got shiny new progress bars and stuff, but the shiny bars don't update very fast. At least MHI is up to date. Also, you can see how much you bonused each worker on MHI, and you can email workers without bonusing them.
• Except! Rejecting workers is best on the main MH page. It lets you easily republish those HITs to other people. (this is an option that you don't even get in the API. What a mess.)
• Also: The only way to approve an Assignment that you previously rejected is via the download/upload CSV, which you get to through the main MH page. Yes, this is pretty wonky.
• You can't change much about the HIT after you post it. But you can change the qualifications, and other minor details about the HIT. You can't change the price or the content. To change the qualifications: you have to use an API call that is sort of obscure: ChangeHitTypeOfHit. (first you have to register your new qualifications as a HitType by calling RegisterHitType.) Wah! If you need help with this, let me know, I've got a script I can send you.
• You're debugging while you're gathering data. When someone says "your site failed and that's why my data isn't complete"... are you going to reject them? Over forty five cents? Are you that sure your site is working perfectly? Are you a fool?

Public Relations

If corporations are people, what are their personalities? Can we have relationships with them? What is that like?

So I set off to explore the ways relationships between people and corporations could develop. As a result, I ended up studying the corporations themselves on social media. A corporation on social media is a strange new entity: it's sort of the corporation, but as it must be controlled in real time by a person, it's sort of one person too. It's a very public, very immediate spokesperson.

A person tweeting behind a corporate handle is neither that person (who would tweet about personal things) nor the company (which would tweet a boring party line, much like if you called a company on the phone and got a recording). It seems closer to the company identity, but I wanted to see if I could get to the human involved.

I started trying to find existing human conversations with companies, much like Chip Zdarsky and Applebee's. (or this, this, this, or this.) This proved fruitless, because most conversations with companies were someone complaining at a company, or else companies talking with other companies to try to appear fun; it was all business. So I went to create my own conversations.

I didn't want any of our conversations to be biased by anything particular about my identity, so I created @MarioLoweystro. Mario is intentionally as bland as possible. But he's obviously human; I didn't want anyone to think he's a bot. I then tweeted at some companies, trying to get to know the people behind them.

Here are my experiences.

And here's a guide showing how to make friends with corporations.

I started off with popular companies, including top-100 Twitter accounts, companies that are supposedly good at Twitter, and really big companies. This turned out not super fruitful - they tend to have millions of followers, and therefore, they can't talk with me. Their accounts are just broadcasts. But then I started tweeting at smaller companies, more local companies, less "sexy" companies. (from a list of Pittsburgh corporations.) They tended to be friendlier. As I went, I varied my styles and talked about different topics; to see what happened, check out my slides above.

Who were my best friends? Probably @WholeFoods, @Tesco, @Huntington_Bank, @Kennametal, and @Zappos. We had some good talks.

We had an exhibit at CMU too for all our final works:

I displayed the record of my conversations, the guide I created, and a bunch of name tags for the companies I met. Plus my computer, so you could try making friends with corporations too. (in fact, go log on to Twitter and try it yourself now!)

Thinking about predicting relationships and protests from Twitter

(There are some similarities.) I had a quick talk with Kenny Joseph the other day which got me to thinking about a couple things. Incidentally, we've got a project we'll be working on in Design Fiction that could use some thinking about the future consequences of these things.

On the protest front, there's this: Can Twitter predict major events such as mass protests?
What if a "they" could predict when you'll protest next, just based on your tweets? Or rather, if they could predict when someone would protest next?

On the relationships front, there's an idea in sociology that your conversation topics and your relationships co-evolve. I'm linking to this paper, even though I cannot say I understood it. But the idea is that you talk with your weak ties about pop culture things, and with your close friends about more niche things. They're not saying which way this evolves, which causes which, but it's a good marker at least of how someone's relationship status is now.

So the obvious dystopia is: The Government, The NSA, is watching all of us and they identify and Guantanamo all probable protesters. (or Turkey's government, or Egypt's, etc.)

But what about the corporate angle? One theme in Kenny's work is: how can we identify and change people's biases? In his case, it's to reduce violence against women and children, or potentially racial violence. What if they develop a powerful new technique to modify biases using Twitter, and Applebee's gets a hold of it? Do you get more people becoming friends of Applebees? More conversations like this?

Fulfillment Fitness

Related to the aforementioned "design fiction" project, but much more successful, has been Fulfillment Fitness.

http://fulfillmentfitness.com

Video link if the above is down

A reflection on the fulfillment centers that drive all your online purchases. Not really trying to be all exposing-cruelty about it, because it's hard to say how bad it is. News articles (one, two, three) would have you believe it's terrible, but people who work there (one two three four five six) seem more sanguine about it. Regardless, it is weird: people become basically robots stuck in a video game all day. (you could imagine SimFactory, where you manage a group of "workers", or a ripoff of Starcraft.) At the same time, sometimes people want to be robots stuck in a video game, like at the gym.

Anyway, it's been fun. Working with Angela and John has been great; they're both wizards, in their own ways. I'm learning how it's hard to get across a point, both to distill the point and to technically get it done. (I've learned to make many kinds of things, but they tend not to be the kind of things that tell stories or stir people's emotions.)

Tell Me Your Life Story, part 3

continued from part 2

It's up! https://tell-your-life-story.herokuapp.com/

You can now publish only part of your life story, if you want. (chunks you leave out will show up as "private".) That's kind of neat. Adds a little mystery to it.

A little clunkier, more words, for better usability.

I hope it makes sense. I have no idea if it will. Also I have no idea if people will think this is in any way cool or makes them think.

I don't know that a website is a good medium for this. It feels too cheap. It's like an *application*, something that you have to do, to be more efficient or store some data or something. I think if I had to do it again, I might print it out as a deck of (large) laminated cards, and give it out (or sell it, in the far future...) with dry erase pens. As a small-party-game, or a Coffee Table Thing, it's kind of fun and personal; as a website, it's not.

Preliminary results with a few friends and a few Turkers: friends find it interesting but don't want to share much. I guess that's fine, as with so few users there is little anonymity. Turkers mostly tell the school-college-work story. Also reasonable, as they're (I'm assuming) mostly trying to make a few bucks.
A couple things people posted are neat:
"1 - 8: Don't remember, had a trampoline."
"15 - 25: I still wanted to be a singer, but Imade sure I got high grades and got accepted into a "good" college. I ended up dropping out many times."

Enough talk, try it out!

Tell Me Your Life Story part 2

continued from part 1
... partial implementation and further design.

I've been thinking a website from the start, just because it's the easiest way to make an interactive thing that people can use. But if I'm going website, then horizontal (the way I've drawn out life stories on paper) is not so good:

There's no way you can space everything out sort of equally and still leave room to type in each box. So I went vertical.

and you can sort of type stuff in here and save it; functional, not yet pretty. So now a couple of questions:
1. how should it look and feel?
2. what should it do when you're done?

For look and feel, it's got to be expansive and welcoming. This should be a space for people to creatively re-imagine their lives. I'm not going to tell them their re-imagination is wrong, or they'll retreat back into the boring school-school-work story. So none of this:

And none of this:

And nothing computery and cold, like I usually dig:

But I don't want it to be new-agey woo anything-goes; no pastel blues and greens and handwriting:

How about cartography? You're mapping your life. Map-making is a good analogy here: you have to take an expanse of time or space that exists, but the way you draw your map (even the projection you use) incorporates your current bias about it.

Plus, I like this aesthetic. I think the old-fashioned look makes it appear valuable. Still, it's not imposing to draw on old-fashioned paper.

shout out to subtlepatterns.com ("rice paper 3") for the background.

For what it should do when you're done... well, I think if it just says "okay, thanks", that is not perhaps as provocative as it could be. There should be some way to display your story, I think, but it should also make you reflect on it.
"Is there anything you would change?"
"What did you learn from telling this story?"
"How do you feel about this story?"
"What happens in the next 5 years?"

(this is after you finish it; the 0-5, 5-8, etc would be filled in with whatever you entered)

Still to do: it would be neat to be able to compare this to some pre-existing milestones; maybe get your school/move/work dates in there too. Or maybe get common culturally-accepted milestones in there to compare your story to those.

A thing I've already learned from this: don't make websites for class projects. I spend more time debugging than I do thinking about the design or the overall experience of the thing.

Tell me your life story

A class project for Design Fiction.
The prompt: "Our formal critical selves" - do something looking at your past in a counterfactual kind of way. See how your life could have been if you had done something differently, or if something different had happened to you.
The timeframe: two weeks. (due the Tuesday after next)

Early thoughts:
1. All the time I've "wasted" throughout the years, and count up the hours I could have been doing something "productive", in this creepy totalitarian sense.

Then I could transform it into N masteries, just by dividing by 10,000! If I've slacked for 20,000 hours by now, I could be an expert in piano and chess. Easy as that. The humor comes from the fact that of course it is not as easy as that.

2. 23 and Me is weird. I have all this data about what risks for diseases I might have. I can't act on it at all. What does it mean that I have a 1.2% chance to get kidney disease? Should I eat more prunes?

I could make it a little more real by making a "wheel of Dan" where you spin a wheel and it tells you "okay, with this lifetime where you started with Dan's genes, you have Alzheimer's and gout." And then I could let you upload your 23 and Me data, and then spin your own wheel, and compare with me, and maybe ask you if you'd trade with me. Make it personal. And maybe compare these risks to other risks, like the odds of being struck by lightning. Whatever.

The downside is that very few people have done 23 and Me, so it's hard to make it actually personal.
Another downside: what am I trying to say? I think there are a lot of interesting things to say about 23 and me, but others say them better than I do, or else I don't really care about them.

3. Something about browser history or email relationships over time. Meh. Been done. I tried to mine all the neat personal data I have about myself, my sleep logs and fitbits and stuff, but I ended up with the same old personal-informatics gripe: what the hell is this data good for?

4. Life stories. I started looking for meaning in Flappy Bird, and then started looking at how people look for meaning in Flappy Bird, which is kind of funny because there is probably not that much meaning to find in Flappy Bird. But all the game developers want there to be some meaning in Flappy Bird, so they tell all kinds of stories, like "really polish a simple game mechanic and it'll shine" or "make sure you can restart quickly" or whatever.

We tell these stories about our lives.
"You may ask yourself, well, how did I get here?"
And most of the time, there's no real story, you just did.

I started looking at the story I tell myself. It was very broken up by where I lived and where I worked. New school, new job, new box in my life story. But

if you mix up those boxes, draw the lines really arbitrarily, you might hit some really more interesting stories. I drew a bunch of random boxes with arbitrary lines. The second box above is "what I liked at various ages", the fifth is "how I felt at various ages", and both are much more interesting than "where I worked."

I want to make a tool to help people retell their life story, but give them a little twist: let the system arbitrarily decide where the boundaries end.

Messing around with the flow here a little bit, or what happens after you tell your story.

More to come!