COBE 2016: Fourth Annual Workshop on Crowdsourcing and Online Behavioral Experiments


A workshop at the WWW 2016 Conference, Montreal, Canada

Workshop Date

Tuesday April 12th, 2016. 9:00AM - 12:30PM.


520E, Palais des congrès de Montréal, 159 St. Antoine St W, Montréal, QC H2Z 1H2. [Map]


6PM at Le Mal Nécessaire [Link] 1106B Saint Laurent Boulevard. [8 minute walk from WWW venue]


These are the accepted papers (in no particular order) Speakers have 22 minutes plus 8 for questions / transitions.

9:00 Eytan Bakshy, Drew Dimmery and John Myles White Design-based Adaptive experimentation
9:30 Andrew Mao TurkServer
10:00 Kevin Munger Tweetment Effects on the Tweeted: Social Norm Promotion on Online Harassers
10:30 Coffee Break
11:00 Greg Stoddard and Maria Glenski Guess the Karma: Predicting popularity on Reddit
11:30 Dan Goldstein, Jake Hofman, and Pablo Barrio A crowdsourced system to help people understand numbers in the news
12:00 Ming Yin, Mary Gray, Siddharth Suri, and Jennifer Wortman Vaughan The Communication Network Within the Crowd.
12:30 Workshop Ends


The World Wide Web has resulted in new and unanticipated avenues for conducting large-scale behavioral experiments. Crowdsourcing sites like Amazon Mechanical Turk, CrowdFlower, Upwork, TaskRabbit, among others, have given researchers access to a large participant pool that operates around the clock. As a result, behavioral researchers in academia have turned to crowdsourcing sites in large numbers. Moreover, websites like eBay, Yelp and Reddit have become places where researchers can conduct field experiments. Companies like Microsoft, Facebook, Google and Yahoo! conduct hundreds of randomized experiments on a daily basis. We may be rapidly reaching a point where most behavioral experiments will be done online.

The main purpose of this workshop is to bring together researchers conducting behavioral experiments online to share new results, methods and best practices.

Topics of Interest

Topics of interest for the workshop include but are not limited to:

Organizing Committee

Program Committee