The Election Debate Visualisation (EDV) project is funded by the UK’s Engineering & Physical Sciences Research Council (EPSRC). This joint 3 year project kicked off Oct. 2013, conducted by the University of Leeds (School of Media and Communication) and the Open University (Knowledge Media Institute).

EDV is investigating how British citizens experienced the televised Prime Ministerial Debates in 2010, with a view to understanding how the 2015 debates might be experienced in new ways to give greater insight and engagement. These requirements will inform the use of novel computational approaches to representing what takes place in the debates, through novel replay interfaces. The project team brings a unique combination of Information Science, Political Communication and Design (University of Leeds) with Computer-Supported Argument Modelling & Visualisation (The Open University).

In more detail…

During the 2010 general election, the first ever televised leaders’ debates to be held in the United Kingdom took place. Research based on previous televised debates in other countries suggested that they could reach a wider audience than is usual for politically-related content and that, after watching them, normally apolitical debate-watchers might be better informed about election issues; more likely to discuss policies with their friends and families; and more likely to vote than those who are not exposed to the debates. In short, televised election debates perform a heuristic function, providing voters with resources that enable them to carry out their normative role as informed and reflective citizens of a representative democracy.

Research conducted by Coleman et al (2011) after the 2010 UK debates showed that there was a significant public appetite for this means of learning about the candidates and their policies, but that many viewers were left feeling uncertain about the meaning of and relationship between the competing arguments they had witnessed. One approach to this is inspired by media ‘uses and gratifications’ research (Katz et al,1974; Blumler, 1979) which has shown that people proactively seek out media outlets that will satisfy their self-perceived needs for information. Such needs can range across cognitive, surveillant, affective and parasocial dimensions. A key outcome of this research will be the provision of customised information paths for viewers with variant information needs.

In considering the best approach to presenting complex arguments to citizens with a view to generating better informed public debate about political issues, we – a group of scholars from different disciplines, incuding information science, political communication and design – have turned to the field of Computer-Supported Argumentation Visualisation (CSAV). CSAV has a track record of utilising innovative information techniques with a view to helping citizens make sense of  and reflect upon structures and flows of complex argumentation.

During the 2010 UK televised debates, the OU conducted an informal experiment using CSAV to publish interactive Web maps displaying graphically the argumentative moves of the prime-ministerial candidates during the televised debates.

The Dialogue Maps went live within an hour of each broadcast, and attracted modest interest from researchers and campaign teams, but there was no research programme in place to publicise the effort, investigate the potential or assess properly the impact. In the same election, the Prime Numerics project (Sosolimited) analysed the linguistic content of the debates, and used a range of visualisations to present these in an artistic/science fiction style to simulate the future of real-time analytics feedback for viewers.

These experiments, in combination with the wider literature reviewed below, motivate EDV. Researchers from different disciplines (Leeds: political communication and design; OU: human-computer interaction, learning technology, participatory planning and argument mapping) will co-design a novel set of visualisations that render televised debates in completely new ways, in order to enhance audience comprehension, engagement and confidence.

Computer-Supported Argument Visualisation

From a cognitive perspective – focusing on improving the quality of learning and rational discourse – there are two critical obstacles to raising the quality of public engagement with political policy. Argument complexity refers to the fact that contemporary issues and arguments are typically beyond the grasp of any individual, making it an intellectually challenging task to engage. Efforts are often made to simplify a debate in order to make a point, but this usually does not do justice to the true complexities of the issue. Representational form refers to how one encounters the debate as a citizen. Typically, debates are encountered in spoken form (e.g. speeches; interviews; TV debates) or in written prose (transcripts; newspaper articles; manifestos), neither of which provide good support for grasping how different parties are framing key issues, the claims they choose to substantiate, and the kinds of arguments being deployed.

Some of these problems are analogous to navigating unknown terrain without a map. Maps provide a way to see the landscape from an aerial view, separate from the limited perspective available from a given position ‘on the ground’. While maps are never neutral (there is always a cartographer making representational decisions), a good map provides a way to filter the complexity of all that might be rendered, in order to focus attention on those features that assist the task at hand. Following this analogy, the approach that this project takes is therefore grounded in the theory and practice of various forms of computer-supported collective ‘knowledge cartography’ developed at the OU and elsewhere (Okada et al., 2008), in particular, Computer-Supported Argumentation Visualisation (CSAV: Okada et al., 2008; De Liddo et al., 2012; Buckingham Shum, 2003; 2008).

CSAV research focuses on the use of information and interaction design techniques to present the structure of arguments. As documented by Buckingham Shum (2003) the earliest contributions to CSAV research came from pre-Web hypertext researchers inspired by the insights of scholars such as Horst Rittel in the field of Policy Sciences, with other relevant research strands from law, education, computer science and artificial intelligence. One of the earliest and most influential graphical hypertext systems to be based directly on the insights of Rittel was gIBIS, which was explicitly framed as a hypertext tool for ‘exploratory policy discussion’. Working with Conklin, the OU has since led the development of the direct descendent to gIBIS, called Compendium (funded by EPSRC, 2002-04).

CSAV research has developed a range of techniques for mapping discourse. Dialogue Mapping  was used in the 2010 Election Debate pilot referred to above, and supports very rapid capture of real-time discourse. Once edited and summarised these produce what are known as Argument Maps (Rider and Thomason, 2008), Argument Diagrams (Rowe and Reed, 2008) and Issue Maps (Conklin, 2008). These can range in complexity from relatively simple notations such as Rittel’s Issue-Based Information System (IBIS) ), to Reed et al.’s more formal rendering of argumentation schemes from the philosophy of argumentation. CSAV has been used to good effect in many sectors, with examples particularly relevant to this project including graphical argument maps that serve as a ‘policy memory’ (Renton and Macintosh, 2007), to summarise contributions to regional consultations (Ohl, 2008), or to provide transparency and accountability in community urban planning (Culmsee and Awati, 2011; De Liddo and Buckingham Shum, 2010). This research has also been influential in research relating to e-participation and online deliberation, with web applications such as Cope_It (Karacapilidis et al., 2009), Deliberatorium (Iandoli et al, 2009), DebateGraph (Baldwin and Price, 2008) and PolicyCommons developed by the Leeds team (Benn and Macintosh, 2012), building in part on the OU’s software.

Given the dominance of textual sources in public policy formulation, and the relative simplicity of document technology (compared to multimedia), it is not surprising that most argument mapping has been grounded in prose text. While there are technical solutions to the annotation of web-based video[3] annotations are typically simple textual notes, not embedded within a broader argument map. In contrast, the OU has demonstrated how videos can be embedded inside argument maps (Bailey et al., 2009; Buckingham Shum, 2009) – which enabled the 2010 Election Debate mapping – but this has yet to be demonstrated on web-hosted video. Related work in the Dialectical Argumentation Machines project focuses on CSAV within a semantic web infrastructure, including multi-user mapping of a topical radio debate, via a wallsize touchscreen (Reed, 2012).

However, multimedia capabilities alone are only part of the story for engaging the public. What has been missing from research into CSAV for political engagement is proper recognition that public opinion about some of the most important policy issues is formed via the mass media, which promote particular kinds of performances and audience experiences. It is in relation to these considerations that the interdisciplinarity of this research should give rise to productive and original outcomes. From the perspective of political communication, the research will focus not merely upon citizens’ cognitive requirements, but upon the ideological and affective processes whereby information is filtered, connected to experience and made usable in sometimes unexpected ways. (Coleman 2013) Unlike in most previous CSAV-based research, the research team will pay close attention to the ways in which the design and presentation of information has significant effects upon its reception, perceived credibility and political value. Rather than seeking to create a single aesthetic mode of representation, researchers will  seek to create a range of designs, appropriate for different types of user.

Research Hypothesis and Objectives

We hypothesise that gaps in citizens’ comprehension and knowledge during and after the televised election debates can be narrowed by offering them a web-based platform consisting of a suite of tools that visualise the debate arguments in meaningful and accessible ways; that such tools should be customised to the information needs of specific audience groups as well as audiences in general; that the success of such tools depends upon insights from design and political communication that are not typically incorporated into CSAV; and that effective evaluation of the use of such a suite of tools can be conducted with a view to assessing how tools were used by different audiences; how such use influenced understanding of the debates; and how such use influenced subsequent civic behaviour.

The objectives of this research are to identify the information needs of audiences and explore appropriate modes of cognitive and affective representation; develop a suite of visualisation tools that are specifically tailored to the needs of specific audience groups and of audiences in general; and explore the potential for visual communication to proactively create a suite of tools through which potential voters are engaged both at the level of design and development and as voters in the context of a general election.

We will develop an open-source web-based platform that incorporates a suite of visualisation tools, and develop a working model of how this platform can be embedded within a mixed-media ecology for covering and responding to issues of public political debate.  The platform will be designed with a view to i) responding to the information needs of audiences and specific types of audience member; ii) presenting the discursive content in ways that take account of the aesthetic and symbolic needs of information seekers; and iii) visualising not only the debaters’ arguments by adopting innovative CSAV methods, but also other features of the debates through the use of techniques such as word-cloud visualisations, and time-series analyses (where the emphasis is on visualising the chronology of the various speech acts during the debate, so that the context “in time” of key rhetorical events can be captured).

Programme and Methodology

EDV entails three stages. During the first stage, from October 2013 to May 2014, we are exploring citizens’ responses to recordings of the 2010 debates, identify cognitive and affective barriers preventing citizens from comprehending and decoding debaters’ claims, policy proposals, values and rhetorical performances; and share ideas with citizens about potential ways of representing future debates. This work is being conducted by Coleman and Moss, in a series of twelve focus groups. Focus groups are a form of group interview that capitalises on communication between research participants in order to generate data.  The method is particularly useful for exploring people’s knowledge and experiences and can be used to examine not only what people think but how they think and why they think that way. Our focus groups consist of 8-10 participants selected on the basis of their prior political experience, level of political engagement and interest, and political alignment, with a view to identifying a range of information needs that reflect those of specific audience groups as well as of audiences in general. Specifically, the groups are made up of:  2 x first-time voters,,2 x disengaged voters, 2 x undecided voters, 2 x party supporters, 2 x journalists and political mediators and 2 x active users of the Internet for political information.

Specific audience needs identified in the research may include a demand for information about how elections work; what the parties stand for and what their records are; how others, such as expert commentators, view particular issues; how to evaluate political claims, including possible fact-checking; and how to assess rhetorical performance and non-verbal signals of the debaters as political personalities. However, it is not possible to predict audience needs fully in advance: our final range of needs will be developed iteratively through the focus groups and so will not emerge until the end of the first stage of the research. To this extent, our approach to the research is inductive and constructivist, drawing closely upon the perceptions and evaluations of citizen/voters themselves, rather than applying a fixed theoretical model deductively. We expect this to allow us to move the approach to CSAV beyond its traditionally narrow rationalistic bias and to take account of how debate audiences respond to a range of rhetorical, affective, aesthetic, semantic and kinaesthetic cues. All focus group discussions will be transcribed fully and analysed by Coleman and Moss, who have a strong track record in the use of this method. The aim of this first stage will be to explore the range of cognitive tools and resources that would be of greatest value to potential voters and contribute to the design and development of appropriate visualisation software. The OU team and Wilson will contribute to the groups by either attending some of them, setting questions to be asked that relate to their own areas of expertise, or providing visualization mockups or demonstration movies which may be shown to participants to elicit reactions.

During the first stage of the project, OU and Wilson will be working on the development of conceptual models and designs for the proposed suite of tools. In the second stage, from June 2014 until the predicted election in May 2015, the research team will work together to build the suite of tools. Rather than develop a CSAV platform and usage model and invite scholars from other disciplines and potential users to participate in trials, our approach to constructing the platform and model will be participatory in two senses: i) Wilson, Coleman and Moss will be fully involved in the process from the outset, offering critiques of initial conceptual models and actively engaged in the work of developing a workable model that conforms to their conceptual specifications; and ii) we adopt a Participatory Design  approach, entailing working with users rather than merely for users (De Young, 1996). This approach means that we will involve all stakeholders at every stage of the design and development process. A series of eight user workshops will be run, some reflecting the groups used in stage 1, others comprising a random sample of potential users. The OU team will use these workshops to explore how different forms of Dialogue/Issue/Argument Map are used by different groups. Wilson will use them to explore design needs. Coleman and Moss will use them to refine their theoretical assumptions about the information needs of different groups.

This stage of the research is especially crucial for the development of the platform.  The results of the conceptual modelling of the first stage will feed into an ‘alpha’ prototype of the platform early in the second stage.  Development will then proceed in an iterative fashion as outlined above – i.e. involving stakeholder feedback at various stages – but also passing through clear milestones, namely a ‘beta’ prototype by December 2014 (half-way through the project) and deployment by March 2015 in time for the April/May 2015 election campaign.  See below for details of the Technical Deliverable.

The third stage will coincide with the broadcast of the televised debates in April/May 2015, and their aftermath. We shall recruit 30 people – 15 from the focus groups and workshops; others who have been hitherto unaware of the project – to test the tools during the debates. This will be followed up after the election by workshops in which participants will be shown excerpts from the debates and invited to use the tools to see whether they help them to make clearer sense of what is going on. Three methods will be used to evaluate user experiences: i) observation: we shall be present with people as they use the tools, paying sensitive attention to their patterns of use, difficulties in following certain points and discovery of heuristic short-cuts; ii) one-to-one interviews: using a semi-structured interview schedule, we shall ask a range of users about their experiences of watching the debates, using the tools and taking subsequent action, such as talking to others, fact-checking, supporting a campaign or voting; and iii) we shall conduct a short survey of all users, specifically geared to discovering the extent to which they felt able to follow and act upon arguments raised in the debates. (Should a mass-media organisation decide to adopt the tools as part of their election coverage, we would endeavour to extend the survey to their audiences.)

Technical Advances and Deliverables

The technical objective is to scale up the 2010 Debate mapping piloted by the OU, in three key respects:

1. Web User Interface. The user interface for replaying Election Debates with synchronised Argument Maps will be migrated from the current Compendium Java desktop client to a web-centric interface. Advances in designing rich, interactive interfaces, particularly for online video, will create a user experience that brings alive patterns both within and between the debates. The videos will be navigable in non-linear ways via the Argument Maps and other visualisations (e.g. Show all video clips about immigration; Show all clips which challenge this claim). Open annotation standards for the Web have been slow to stabilise, but we will review their status. We will also review web-visualisation technologies, to assess the most robust cross-platform, cross-browser options and open source libraries.
2. Debate Analytics. Novel analytics will appear in the Debate replay interface to help viewers assess the quality of the debate, informed by the participatory design process. The OU’s CSAV research will inform analytics designed to help argument appraisal and critical questioning, which will be trialled with stakeholders, e.g. Who disagrees most with whom? How many unsubstantiated claims are made? What kinds of arguments do the candidates deploy? Three ways to critique the assumptions behind this kind of argument…
3. Open Data. The data in the platform will be accessible via data export and dynamic querying via a RESTful API and SPARQL endpoint, and using the Argument Interchange Format to enable researchers and web services outside the project to benefit. This should greatly increase the project’s impact, since myriad user interfaces and analytical lenses can then be brought to bear on the dataset. Researchers and artists might render the data in novel ways, such as the Prime Numerics project introduced earlier. Ontology-based information extraction tools could automatically index the debate transcripts. Web news and data mining tools could assist in fact-checking, e.g. Did the candidate actually promise this last year? Which World Bank recommendation has been changed?

Summary

This research seeks to make advances in our understanding of citizens’ experience of televised political debates, when augmented by new digital tools that render significant patterns within and between debates that are normally invisible. Clearly, the project will not provide a panacea for political disenchantment or disengagement, which are rooted in multiple causes beyond the remit of our research. However, based on a clear understanding of voters’ information needs, and the principles of aesthetic and interaction design, this suite of tools should contribute to the nurturing of an electorate, from teenagers upwards, that is better informed, increasingly confident and more engaged.

References

Bailey H, Bachler M, Buckingham Shum S, et al. (2009) Dancing on the Grid: Using e-Science Tools to Extend Choreographic Research. Philosophical Transactions of the Royal Society A. 367: 2793-2806.

Benn, N., & Macintosh, A. (2012). PolicyCommons: Visualizing Arguments in Policy Consultation. In Proceedings of the Fourth International Conference on eParticipation (ePart 2012), 61–72.

Blumler, J. G. (1979). The role of theory in uses and gratifications studies. Communication Research, 6:9–36.

Buckingham Shum S. (2003) The Roots of Computer-Supported Argument Visualization. In: Kirschner PA, Buckingham Shum S and Carr C (eds) Visualizing Argumentation. London: Springer-Verlag, 3-24.

Buckingham Shum S. (2008) Cohere: Towards Web 2.0 Argumentation. Proceedings of the 2nd International Conference on Computational Models of Argument. 28-30 May 2008, Toulouse, France.

Buckingham Shum S. (2009) Choreographic Video Annotation (Blog Post Sep. 14, 2009), e-Dance Project, The Open University). Available at: http://bit.ly/VideoArgMaps

Coleman, S. (2013) How Voters Feel, NY: Cambridge University Press

Coleman, S. and Moss, G. (2012) Under construction: the field of online deliberation research. Journal of Information Technology & Politics 9 (Feb. 2012), 1-15.

Coleman, S., ed. (2011). Leaders in the Living Room—The Prime Ministerial Debates of 2010: Evidence, Evaluation and Some Recommendations. Oxford: Reuters Institute.

Conklin J. (2006) Dialogue Mapping: Building Shared Understanding of Wicked Problems, Chichester: Wiley.

Conklin J. (2008) Growing a Global Issue Base: An Issue-based Approach to Policy Deliberation. Directions and Implications of Advanced Computing; Conference on Online Deliberation. Berkeley, CA.

Culmsee P and Awati K. (2011) The Heretic’s Guide to Best Practices: The Reality of Managing Wicked Problems in Organisations, Bloomington IN: iUniverse Inc.

De Liddo A and Buckingham Shum S. (2010) Capturing and Representing Deliberation in Participatory Planning Practices. In: De Cindio F, Macintosh A and Peraboni C (eds) Fourth International Conference on Online Deliberation. Leeds, UK.

De Liddo A, Sándor Á and Buckingham Shum S. (2012) Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study. Computer Supported Cooperative Work 21: 417-448.

De Young, Laura. (1996). Organizational Support for Software Design, In Winograd, T. (ed), Bringing Design to Software, Addison-Wesley

Fishkin JS, Luskin RC. 2005. Experimenting with a democratic ideal: deliberative polling andpublic opinion. Acta Polit. 40:284–98

Gastil, J, and Dillard, J. (1999) Increasing Political Sophistication Through Public Deliberation. Political Communication 16 (January to March): 3–23.

Iandoli, L., M. Klein, and G. Zollo. 2009. “Enabling on-line deliberation and collective decision-making through large-scale argumentation: a new approach to the design of an Internet-based mass collaboration platform.” International Journal of Decision Support System Technology no. 1 (1):69-91.

Karacapilidis, N. I., et al. (2009). Tackling cognitively-complex collaboration with Cope_It!. International Journal of Web-Based Learning and Teaching Technologies, 4(3), 22–38.

Katz, E., Blumler, J. G., & Gurevitch, M. (1974). Utilization of mass communication by the individual. In J. G. Blumler & E. Katz (Eds.), The uses of mass communications: Current perspectives on gratifications research Beverly Hills, CA: Sage, pp.19-32

Lau, R. R., & Redlawsk, D. P. (2006). How voters decide: Information processing in election campaigns. New York: Cambridge University Press

Lupia, A, and McCubbins, M. (1998) The Democratic Dilemma: Can Citizens Learn What They Need to Know? Cambridge: Cambridge University Press.

McKinney, M. S., and D. B. Carlin, (2004)‘Political Campaign Debates’, in L. L. Kaid (ed.), Handbook of Political Communication Research (Mahwah, NJ, and London: Lawrence Erlbaum Associates,

Ohl R. (2008) Computer Supported Argument Visualisation: Modelling in Consultative Democracy around Wicked Problems. In: Okada A, Buckingham Shum S and Sherborne T (eds) Knowledge Cartography. London: Springer.

Okada A, Buckingham Shum S and Sherborne T. (2008) Knowledge Cartography. London, UK: Springer.

Reed C. (2012) Argument Analysis Wall: analysis in real time (Blog Post Jul. 19, 2012, Argumentation Research Group, Univ. Dundee). Available at: http://www.arg.dundee.ac.uk/?page_id=645

Renton, A., & Macintosh, A. (2007). Computer supported argument maps as a policy memory. The Information Society, 23(2), 125–133.

Rider Y and Thomason N. (2008) Cognitive and Pedagogical Benefits of Argument Mapping: L.A.M.P. Guides the Way to Better Thinking. In: Okada A, Buckingham Shum S and Sherborne T (eds) Knowledge Cartography. London: Springer.

Rittel HWJ. (1984) Second-generation design methods. In: Cross N (ed) Developments in Design Methodology. New York: Wiley, 317-327.

Rittel, H. W. J., & Webber, M. M. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169.

Rowe G and Reed C. (2008) Argument Diagramming: The Araucaria Project. In: Okada A, Buckingham Shum S and Sherborne T (eds) Knowledge Cartography. London: Springer.