In this study, Micky Chen (University of Amsterdam) presents an experimental study in form of a contest to analyze how design competitions might be used in crowdsourcing user interface design. From the abstract:
Crowdsourcing has gained great popularity over the past decade. Using the power of the crowd might be a useful source for next-generation software designing. However, little is known about whether the quality of one’s design can improve by using parts of other designs. To address this gap, I conducted a grounded theory study to analyze the role of crowdsourcing in creating user interface designs. An experimental study was conducted in the form of a contest to analyze how design competitions might be used in crowdsourcing user interface design. In the two-round contest, participants created a UI, which was subsequently distributed to the other participants, so each has access to all first round designs. The second round required a revised UI design, with participants being encouraged to borrow each other’s ideas in creating a revised design when they saw a fit. The results show that on average, participants’ designs improved and everyone used at least one idea from another person. This indicates that recombination and crowdsourcing leads to high(er) quality user interfaces. However, due to the limitations of the experiment setup, more research is needed into how to set up UX design contests so as to maximize the benefits of crowdsourcing.