the InMind blog

Is social psychology ready for the big science revolution?

• written by Frank Bosco

In this blog post, I will describe perhaps the greatest challenge facing social psychology (and other social sciences) in the coming decades: The curation and increased accessibility of research findings. I describe several big science efforts that lead to an eventual goal – a search engine of research findings capable of producing instant empirical summaries and translating findings using lay terms.

Big data, big science, big challenges. Social psychology is a vast, diverse field with no road map – no framework with which to organize and communicate its topics and findings. As we proceed into the 21^st century, some key big data and big science challenges will become difficult to ignore. Social psychology has amassed a corpus of research findings over approximately 100 years and that corpus has grown exponentially to contain potentially millions of findings, usually in the form of effect sizes such as the correlation coefficient. However, virtually none of these findings are curated and made easily accessible. There exists no search engine of social psychology’s findings.

Importantly, the millions of findings are, together, the building blocks of science. They are used as input to many advanced research methods including meta-analysis -- a sophisticated average of existing findings. Thus, the importance of effect sizes should not be understated. Indeed, as Cohen (1988) noted, that effect size is “what science is all about” (p. 532). Similarly, as noted by Schmidt (1992), “the empirical relations among variables [are]… the building blocks of theory” (p. 1177). Given the central role of such findings, one strategy through which sciences can “take stock” of existing evidence and begin to answer big sciencequestions comes in the form of research curation – archival and organizational efforts to catalog a field’s universe of findings.

Existing big science efforts. A variety of big science efforts have emerged over the last decade. As one example, the UCSD Map of Science provides a glimpse into the relationships within and between disciplines by analyzing the co-occurrence of publications’ keywords and citations. The Inter-Nomological Network(INN), part of the Human Behavior Project, makes available a large corpus of research instruments. Another impressive effort is located at the Archival Project, which summarizes the “key” finding from each article in social psychology. Such tools are highly valuable for researchers.

As a final example, the metaBUS project represents a big science effort to curate evidence from a multidisciplinary research domain. In this project, a hierarchical taxonomy of nearly 5,000 variables is being linked to more than 1,000,000 findings from several decades of published research. The taxonomy classifies variables according to IS-A links, branching from major classifications (e.g., attitudes, behaviors, person characteristics), and proceeding to finer levels of classification (e.g., people attitudes, psychological person characteristics). By applying advanced technologies and graphical user interfaces, metaBUS will allow consumers of science to get (at least) rudimentary answers to big science questions in near real-time (the preliminary database is freely available at www.frankbosco.com/data).

Can social psychology be curated? At least three key prerequisites must be present for efficient research curation to be realized. First, procedures must be developed and refined for the semi-automated extraction of research findings. In the field of applied psychology, this task is relatively easy because most findings are presented in correlation matrices, often resulting in the extraction of 50 or more findings per article (this is not the case for much of experimental social psychology). However, tools for the automated extraction of noncorrelational findings, when reported in the prescribed APA format, are available (e.g., http://mbnuijten.com/projects-2/statcheck/).

Second, a taxonomic classification of variables and constructs must be developed. We expect that this task will be relatively difficult to accomplish in social psychology compared to other areas. However, we’re optimistic that the taxonomy’s development is possible. Indeed, at the broadest level, many areas of social psychology seek to explain human behavior(one possible major “branch”) from psychological and demographic person characteristics (another possible branch). Additional attention would be needed to bring order to the vast number of experimental manipulations, each representing levels of a given construct. Other experimental areas in psychology, such as cognitive psychology, have begun to develop such maps (see: http://www.cognitiveatlas.org/).

Finally, research curation efforts require thousands of skilled labor hours. To construct a map of the field is a daunting task in its own right. Even more challenging is the coding of findings. Indeed, the vocabulary and jingle-jangle problems have led to identical terms having several meanings (e.g., performance) or many terms that refer to the same construct. These conditions lead us to believe that fully-automated, accurate research curation in social psychology is, at this point in time, not possible. However, with expertise and effort, such a map is possible.

What would efficient curation of scientific research provide? The curation of an entire field of research would have more benefits than space allows us to discuss. Indeed, it would be difficult for any scientist (or other consumer of science) to downplay a search engine of research findings in any scientific field. The argument would be tantamount to suggesting that internet search engines are useless. In addition, by adding annotations that facilitate layperson understanding of constructs, research curation could usher in a new age of the public understanding of science.

We ask a simple question: Will we, one day, search for and summarize scientific findings with the same level of sophistication afforded to CrockPots through internet retailers? We are optimistic that, as other fields realize the benefits of curation, the practice will become widespread. Importantly, research curation will have clear implications for scientists and lay consumers of science. Indeed, as a lay consumer (at best) of fields outside of my own familiarity, I am find myself wondering: What does the field of sociology look like? What do they actually study? Where are the “field maps” for sciences like these? In order to make advances in the public understanding of science, change is needed. The technology needed to bring about the curation, summarization, and communication of a cumulative body of scientific evidence is available… right now.

References:

Bosco, F. A., Aguinis, H. Singh, K., Field, J. G., & Pierce, C. A. (in press). Correlational effect size benchmarks. Journal of Applied Psychology.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Schmidt, F. L. (1992). What do data really mean? Research findings, meta-analysis, and cumulative knowledge in psychology. American Psychologist, 47, 1173-1181.