Let’s say you wanted to find out whether a new ice rink planned for your neighborhood would have a positive economic impact on the community and raise your property value. You’d have to consider a host of variables, such as the demographics of the surrounding area, utilities costs, the current use of the land and available parking.

A team that includes Colorado State University researchers has been awarded $11.5 million to develop a digital tool that helps people reach better decisions on complex matters, identifying factors like information gaps and biases. Their initial focus is the federal government’s intelligence analysis — not a local ice rink — but the tool could eventually be adapted to a variety of applications, from preparing for natural disasters to helping law enforcement solve complicated cases.

About TRACE

The multidisciplinary team of researchers from CSU, Syracuse University, University of Arizona and SRC Inc. will develop and test a web-based application called TRACE (Trackable Reasoning and Analysis for Collaboration and Evaluation). The application will use techniques such as debate and analogical reasoning along with crowdsourcing and collaboration tools to enhance users’ problem-solving abilities and foster creative thinking.

The CSU team is led by Associate Professor Rosa Mikeal Martey in the Department of Journalism and Media Communication and includes James Folkestad, a professor in the School of Education, and Benjamin Clegg, a professor in the Department of Psychology.

“This application will be a kind of digital workspace that guides people through tackling complicated, real-world questions,” Martey explained. “With real-time feedback about the analysis as they do it, TRACE will be open enough for people to do complicated work, but sophisticated enough to help them improve their thinking. Our background in digital communication, learning analytics and the psychology behind automated systems will help us design an innovative system that enhances the decisions analysts make without overloading them.”

The 4.5-year project is funded by a contract of up to $11.5 million from the CREATE (Crowdsourcing Evidence, Reasoning, Argumentation, Thinking and Evaluation) program of the Intelligence Advanced Research Projects Activity, a branch of the Office of the Director of National Intelligence, which heads the nation’s intelligence services. The project will engage faculty and graduate students through two campus research centers, the Communication and Technology Center headed by Martey and the multidisciplinary Center for the Analytics of Learning and Teaching (C-ALT) headed by Folkestad.

Rigorous testing

“What makes this project unique is that we are rigorously testing every aspect of our application using experimental research methods,” said the TRACE team’s principal investigator, Jennifer Stromer-Galley, a professor at Syracuse University. “When this project is done, we will have a proven, effective tool for people to use.”

“The data we collect will provide us with insight into how our tool is used and how that use relates to decision-making,” Folkestad said. “In addition, the fundamentals that we learn about designing for, detecting and analyzing behaviors will be applicable to other projects, such as course management technologies that we study at C-ALT.”

Improving team communication and creating an application that people actually want to use is a key concern of the TRACE team.

“Communication is key to effective and gratifying decision-making,” said Kate Kenski, an associate professor of communication and government and public policy at the University of Arizona. “Through TRACE, we hope to improve online communication platforms that help groups make more accurate and satisfying decisions when faced with important and sometimes urgent problem-solving tasks.”

Game-based principles

To foster richer user experiences, the TRACE application will apply game-based principles of human-computer interaction and engagement.

“Asking people to change the way they work and collaborate is challenging,” explained Martey. “We don’t just want people to end up with an accurate, effective analysis, we also want them to find new ways of engaging with the work and with each other while they do so. Understanding how and why people engage with TRACE can help us understand engagement with a wide range of digital communication tools.”

The system, which will be created by SRC Inc., a nonprofit research and development company in Syracuse, New York, will also include responsive mechanisms such as advanced natural language processing to help detect reasoning errors and provide guidance on a variety of tasks while promoting efficiency, accuracy and clarity in analysis.

Other applications

The potential benefits of a platform such as TRACE are expected to go beyond the intelligence community.

“We see the application we’re designing as one that can improve the quality of digital collaboration and decision-making of anyone who does complex analysis, whether they’re scientists, students or policy-makers,” Martey said.

The team brings together a large team of researchers, practitioners and graduate students with a range of expertise, including human-computer interaction, deliberation, crowdsourcing, game design, interface and software design, cognitive and decision sciences, as well as computational techniques.

In addition to Martey, Folkestad, Clegg, Stromer-Galley and Kenski, the team includes Syracuse faculty Nancy McCracken, Carsten Østerlund, Lu Xiao, Lael Schooler and David Kellen. It also includes two experts in intelligence analysis, Roc Myers and Sarah Taylor. CSU doctoral students Priya Harindranathan, Greg Russell and Rhema Zlaten are research assistants for the project.