The Future of Work, or The World of Workcraft

A while back in 2010, I attended a CCC/CRA workshop in Ultra-Large-Scale Interaction. The goal of the workshop was to set out a broad research agenda for having thousands of people collaborate together on hard problems. A lot of the discussion focused on crowdsourcing, and how to apply crowdsourcing techniques to enable new kinds of work in the future.

Below is a scenario that Rick Wash, Michael Bernstein, and I put together for our vision of how crowdsourcing techniques for skilled workers might pan out in the future. We jokingly called it the World of Workcraft, since we drew some of our ideas from that popular game.

For example, in WoW, the game records your skill level as well as achievements demonstrating that you have some level of mastery. The game also has ways to facilitate assembling people based on their skills (e.g. tank, damage, healer), having these people operate in short-term groups to achieve specific objectives. We felt that some of these same ideas might be applicable for work in the future, where you might be able to quickly assemble groups of skilled people (rather than unskilled ones, as in MTurk), thus reducing the friction and transaction costs it takes to get things done as well as to find people with the right skill sets.

I've had our scenario lying around on my hard drive for a while, I figured it was about time to share it. I think that there are a lot of compelling ideas here from a broader perspective as well. We're starting to see tectonic shifts in how work is being done, in the form of more powerful technologies as well as a global workforce.

If we rewind back to the Industrial Revolution, we saw many major (and often painful) shifts as we transitioned from an agrarian society to an industrial one, including such things as urbanization, compulsory education, unions, social safety nets, feminism, and more. The changes we are now seeing as we continue accelerating towards an information society are likely just the tip of the iceberg, and will likely be just as significant (and painful) as the ones in the past.

At any rate, our scenario offers one glimpse at a possible future of what work might be like in the future. Incidentally, the two people running the workshop were Dan Olsen and Beth Mynatt, hence the names of our characters.

---------

Dan Mynatt has an idea for a technology start-up company, but he has training only in user interface design. He does not have the skills or manpower yet to staff an entire start-up: he wants to get his design ideas off the ground to show to some potential funders. This would typically take him weeks and considerable seed money, but Dan wants to get a refined mockup of his idea in one week and a limited budget. Dan's main skill is concept designing and storyboarding, so he begins by sketching lots of concept designs for his idea. He selects ten concepts that he likes. Normally he would spend weeks refining these interfaces and getting feedback, but with our system he can parallelize and recruit skilled workers for small design, implementation and evaluation tasks.

He logs on to our proposed task market to gather a flash team and workflow -- he wants to iterate on the designs, produce high-fidelity mockups and get user feedback. He uses a previous workflow template to specify the job. First, the designs are fed out to other designers. These designers give him feedback on the concepts -- professional crits and suggestions -- which he uses to revise the designs and cull down to seven to continue exploring. Within a few hours, he has now gotten the same amount of feedback he would have needed to wait hours or days to produce before.

Dan takes these seven designs and asks graphic designers to produce pixel-perfect mockups of them. He outsources a first usability study by recruiting a professional user testing engineer. The lead, Beth Olson, begins by developing metrics and tasks. She delegates each of these tasks to other user testing engineers, who in turn crowdsource the evaluation by asking unskilled users to run through the interface design alternatives.

For a moment, let's consider the perspective of one of the usability engineers, Judy Ackerman. She receives one of the high-fidelity mockups, as well as instructions to test it on at least ten users. She begins to play with the prototype himself, and discovers an obvious flaw in the design. Judy raises a flag to Dan, the manager, saying that the design needs a slight tweak before he can study it. Dan and Judy enter a conversation to refine the prototype, and Dan agrees to send it back to the mockup author for an edit. Finally, it comes back to Judy. She flips into the manager interface herself to author a user study workflow and get the requested feedback.

All this feedback makes its way back to Dan. Dan weeds out a few of the designs and produces some new alternatives: he is now down to three designs. Now he decides that it is time to build interactive prototypes and deploy them for feedback. First, Dan orchestrates a mini-development team to create interactive mockups for each of the designs. These teams include a graphic artist, at least one programmer, and a user testing engineer. This team builds and deploys a prototype, with the user testing engineer heading the evaluation.

Finally, given the feedback and prototypes, Dan selects a design recommendation. He tweaks the design, writes up documentation and finds a development team to help him execute the concept.

Why is life better?

• Can test many more alternatives (10 product ideas in the time I could have done 1 internally)
• Higher quality and lower risk (e.g. Threadless) because you get early feedback
• By incorporating lots of people at each stage, you get varied feedback and multiple perspectives at critical points.
• Supplement the shortcomings that come with a specialized skill set -- a designer is poor at coding and evaluation, but can ask for help in those situations.
• Empowering people who don't have any of these skills, e.g. a concept person who knows a pain point but doesn't know anything about programming or designing or (perhaps) project management