A new tool to evaluate impacts of science communication on scientific literacy

A new tool to evaluate impacts of science communication on scientific literacy

Author: Isabelle Kingsley – University of NSW, Australia

Isabelle Kingsley – University of New South Wales
Carol Oliver – University of New South Wales
Martin Van Kranendonk – University of New South Wales

What evidence do we have that science communication improves scientific literacy? To date, the standard questionnaire has been the ‘go to’ instrument for measuring scientific literacy and impacts of science communication. However, researchers in the field are pointing to the need for more sensitive instruments (Cronje et al., 2011; Crall et al., 2013; Brossard et al., 2005).

We report on the development and testing of a new instrument—a digital game—designed to measure scientific literacy. Scientific literacy, as defined by Jon D. Miller (1983), involves knowledge of scientific constructs, understanding of the nature of science (NOS) and understanding of the societal impacts of science. This instrument focuses on the understanding of NOS dimension of Miller’s definition, which—we argue to some extent—is at the heart of scientific literacy.

This new tool is based on a Teachable Agent (TA), a learning technology, which uses the social metaphor of teaching a computer agent by creating a concept map that serves as the agent’s ‘brain’ (Schwartz and Arena, 2009). Concept maps are visual representations of knowledge used to measure changes in cognitive structure—that is, changes in or development of meanings of concepts (Novak and Gowin, 1984). Using concept maps and TAs, this instrument identifies the validity and complexity of ideas held by subjects about NOS and measures any changes in their cognitive structure, pre and post science communication activity, by comparing the choices they make in producing their concept maps to ‘teach’ their TA.

The instrument automatically scores concept maps via an algorithm, making it scalable and just as fast and easy to use as traditional questionnaires.

This study indicates that this instrument may be more sensitive and accurate than questionnaires at measuring the impacts of science communication on scientific literacy. Validation as well as further testing is required.

The author has not yet submitted a copy of the full paper.

Presentation type: Visual talk
Theme: Science
Area of interest: Investigating science communication practices