Large Data Collection and HCI

Why is it that economists have so much shared and common data to work with, while we in HCI do not? There is so much raw data out there for economists about the stock market, GNP, GDP, exchange rates, option prices, oil prices, car crashes, sumo wrestling, and so on. Imagine what HCI could be like if we could have that much rich data.

Just off the top of my head, some data sources that I'd love to be able to use:

  • Google search terms
  • Orkut and Friendster social network connections
  • Microsoft Windows crash data, (you know, those popups that appear after a program crashes, asking if you want to send it to MSFT. What programs crash most often? What trends are there over time?)
  • Yahoo IM, AIM, and MSN Messenger usage trends
  • ISP usage data (how much traffic is file sharing, web, IM, etc)
  • Yahoo web page usage trends (What happened when a change was made? What changes have been most popular? Least popular? Which parts of the navigation do people use most, ie nav bar, pictures, text links, etc? Is there a correlation between web page size and traffic?)
  • Ebay usage trends (what factors lead to the most popular sales? What indicators are there of fraud? How have sales changed over time? Is EBay now dominated by power sellers? What product trends are there?)


Unfortunately, these are all corporate secrets, and also have privacy issues involved. But think how much good we could do for everyone if this kind of data were available.

Comments

Post a Comment

Popular posts from this blog

How to Fix a Jammed Toyota Camry Trunk

This problem needs a higher pagerank, so I figured I would post the solution here. If your Toyota Camry trunk won't open, one possible reason is that it is set to valet mode. Valet mode means that you cannot open the trunk using the release lever inside the car. To set valet mode, you put the key into the trunk lock and turn it counterclockwise. You will know that your trunk is in valet mode if the lock is horizontal rather than vertical, and if you cannot open the trunk using the lever near the driver's seat. Of course, a problem is that sometimes the Camry can get stuck in valet mode, such that you can't use your key to get out of it. (You can see how I spent part of my Sunday morning ...) The solution turns out to be WD-40 . Spray some WD-40 on your key and on the lock. Put the key in, and jiggle it around, and happiness ensues. From an interaction design perspective, it sort of makes sense to have a valet mode. After all, the point of having a valet key is to limit the
Read more

Web 2.0 and Research

I've been chatting with many of my friends and colleagues about an issue that's been bugging me for a while, namely whether academic research has any role to play in the emerging Web 2.0 . I've been slowly coming to the conclusion that the answer is not much. I had a similar discussion with other researchers at HotMobile a few weeks ago. When the web first came out, pretty much every systems researcher ignored it because it was so ugly. The web was not very sophisticated in terms of distributed systems, HTTP lacked elegance, HTML conflated many different ideas, and so on. There were also not any really new ideas with the web, as evidenced by the fact that Tim Berners-Lee 's first paper on the Web was (probably rightfully) rejected from an ACM conference on hypertext. I'm sure one thing that really irked researchers about the nascent web was that it completely ignored the large body of work in hypertext and distributed systems that had preceded it. Even in 1997, as
Read more

[Research] Famous Rejected Papers

I've always thought it would be fun to compile a list of famous rejected research papers, ones that later turned out to be highly influential. For example, in My Life as a Quant , the author tells the story of how the famous Black-Scholes paper (which describes how options should be priced) was rejected several times. The work eventually led to a Nobel Prize in economics. (Yes, I know, there technically is no Nobel Prize in economics). Then there's George Akerlof 's work on asymmetric information . I don't recall exactly, but I think the reviewers thought it was too simplistic. It also led to a Nobel Prize in econ. And then there's Tim Berners-Lee 's original paper on the World Wide Web. He describes his experiences in his book Weaving the Web . I think it was submitted to a hypertext conference, but was accepted only as a demo. I'm guessing reviewers didn't see much novelty in the work, which probably was correct from a research perspective.
Read more