Shortly before Christmas, I was engaged in discussion with a Swedish-based colleague about crowd-sourcing and the humanities. My colleague – an environmental archaeologist – posited that it could be demonstrated that crowd-sourcing was not an effective methodology for his area. Ask randomly selected members of the public to draw a Viking helmet. You would get a series of not dissimilar depictions – a sort of pointed or semi-conical helmet, with horns on either side. But Viking helmets did not have horns.
Having recently published a report for the AHRC on humanities crowd-sourcing, a research review which looked at around 100 publications, and about the same number of projects, activities, blogs etc, I would say the answer to this apparent fault is: don’t identify Viking helmets by asking the public to draw them. Obvious as this may sound, it is in fact just an obvious example of a complex calculation that needs to be carried out when assessing if crowd-sourcing is appropriate for any particular problem. Too often, we found in our review, crowd-sourcing was used simply because there was a data resource there, or some infrastructure which would enable it, and not because there was a really important or interesting question that could be posed by engaging the public – although we found honourable exceptions to this. Many such projects contributed to the workshop we held last May, which can be found here. To help identify which sorts of problems would be appropriate, we have developed – or rather, since this will undoubtedly involve in the future, I should say we are developing – a four facet typology of humanities crowd-sourcing scenarios. These facets are asset type (the content or data forming the subject of the activity), process type (what is done with that content) task type (how it is done), and the output type (the thing, resource or knowledge produced). What we are now working on is identifying – or trying to identify – examples of how these might fit together to form successful crowd-sourcing workflows.
To put it in the terms of my friend’s challenge: an accurate image of a Viking helmet is not an output which can be generated by setting creative tasks to underpin the process of recording and creating content, and the ephemeral and unanchored public conception of what a Viking helmet looks like is not an appropriate asset to draw from. Obvious as this may sound, it hints that a systematic framework for identifying where crowd-sourcing will, and won’t, work, is methodologically possible. And this could, potentially, be very valuable as the humanities faces increasing interest from well-organized and well-funded citizen science communities such as Zooniverse (which already supports and facilitates several of the early success stories in humanities crowd-sourcing such as Ancient Lives and OldWeather).
This of course raises a host of other issues. How on earth can peer-review structures cope with this, and should they try to? What motivates the public, and indeed academics, to engage with crowd-sourcing? We hint at some answers. Transparency and documentation is essential for the former area, and we found that in the latter, most projects swiftly develop a core community of very dedicated followerswho undertake reams of work, but – possibly like many more conventional collaborations – finding those people, or letting them find you, is not always easy.
The final AHRC report is available: Crowdsourcing-connected-communities.
Stuart Dunn 