OXFORD LIBRARY OF PSYCHOLOGY EDITED BY
JEROME R.
BUSEMEYER ZHENG
WANG JAMES T.
TOWNSEND & AMI
EIDELS
The Oxford Handbook of COMPUTATIONAL and MATHEMATICAL PSYCHOLOGY
The Oxford Handbook of Computational and Mathematical Psychology
OX F O R D L I B R A RY O F P S Y C H O L O G Y E D I T O R-I N-C H I E F
Peter E. Nathan AREA EDITORS
Clinical Psychology David H. Barlow
Cognitive Neuroscience Kevin N. Ochsner and Stephen M. Kosslyn
Cognitive Psychology Daniel Reisberg
Counseling Psychology Elizabeth M. Altmaier and Jo-Ida C. Hansen
Developmental Psychology Philip David Zelazo
Health Psychology Howard S. Friedman
History of Psychology David B. Baker
Methods and Measurement Todd D. Little
Neuropsychology Kenneth M. Adams
Organizational Psychology Steve W. J. Kozlowski
Personality and Social Psychology Kay Deaux and Mark Snyder
OXFORD LIBRARY OF PSYCHOLOGY
Editor-in-Chief
peter e. nathan
The Oxford Handbook of Computational and Mathematical Psychology Edited by
Jerome R. Busemeyer Zheng Wang James T. Townsend Ami Eidels
1
3 Oxford University Press is a department of the University of Oxford. It furthers the University’s objective of excellence in research, scholarship, and education by publishing worldwide. Oxford New York Auckland Cape Town Dar es Salaam Hong Kong Karachi Kuala Lumpur Madrid Melbourne Mexico City Nairobi New Delhi Shanghai Taipei Toronto With offices in Argentina Austria Brazil Chile Czech Republic France Greece Guatemala Hungary Italy Japan Poland Portugal Singapore South Korea Switzerland Thailand Turkey Ukraine Vietnam Oxford is a registered trademark of Oxford University Press in the UK and certain other countries. Published in the United States of America by Oxford University Press 198 Madison Avenue, New York, NY 10016 c Oxford University Press 2015 All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, without the prior permission in writing of Oxford University Press, or as expressly permitted by law, by license, or under terms agreed with the appropriate reproduction rights organization. Inquiries concerning reproduction outside the scope of the above should be sent to the Rights Department, Oxford University Press, at the address above. You must not circulate this work in any other form and you must impose this same condition on any acquirer. Library of Congress Cataloging-in-Publication Data Oxford handbook of computational and mathematical psychology / edited by Jerome R. Busemeyer, Zheng Wang, James T. Townsend, and Ami Eidels. pages cm. – (Oxford library of psychology) Includes bibliographical references and index. ISBN 978-0-19-995799-6 1. Cognition. 2. Cognitive science. 3. Psychology–Mathematical models. 4. Psychometrics. I. Busemeyer, Jerome R. BF311.O945 2015 150.1 51–dc23 2015002254
9 8 7 6 5 4 3 2 1 Printed in the United States of America on acid-free paper
Dedicated to the memory of Dr. William K. Estes (1919–2011) and Dr. R. Duncan Luce (1925–2012) Two of the founders of modern mathematical psychology
v
SHORT CONTENTS
Oxford Library of Psychology ix About the Editors xi Contributors
xiii
Table of Contents Chapters Index
xvii
1–390
391
vii
OX F O R D L I B R A R Y O F P SYC H O LO GY
The Oxford Library of Psychology, a landmark series of handbooks, is published by Oxford University Press, one of the world’s oldest and most highly respected publishers, with a tradition of publishing significant books in psychology. The ambitious goal of the Oxford Library of Psychology is nothing less than to span a vibrant, wide-ranging field and, in so doing, to fill a clear market need. Encompassing a comprehensive set of handbooks, organized hierarchically, the Library incorporates volumes at different levels, each designed to meet a distinct need. At one level are a set of handbooks designed broadly to survey the major subfields of psychology; at another are numerous handbooks that cover important current focal research and scholarly areas of psychology in depth and detail. Planned as a reflection of the dynamism of psychology, the Library will grow and expand as psychology itself develops, thereby highlighting significant new research that will impact on the field. Adding to its accessibility and ease of use, the Library will be published in print and, later on, electronically. The Library surveys psychology’s principal subfields with a set of handbooks that capture the current status and future prospects of those major subdisciplines. The initial set includes handbooks of social and personality psychology, clinical psychology, counseling psychology, school psychology, educational psychology, industrial and organizational psychology, cognitive psychology, cognitive neuroscience, methods and measurements, history, neuropsychology, personality assessment, developmental psychology, and more. Each handbook undertakes to review one of psychology’s major subdisciplines with breadth, comprehensiveness, and exemplary scholarship. In addition to these broadlyconceived volumes, the Library also includes a large number of handbooks designed to explore in depth more specialized areas of scholarship and research, such as stress, health and coping, anxiety and related disorders, cognitive development, or child and adolescent assessment. In contrast to the broad coverage of the subfield handbooks, each of these latter volumes focuses on an especially productive, more highly focused line of scholarship and research. Whether at the broadest or most specific level, however, all of the Library handbooks offer synthetic coverage that reviews and evaluates the relevant past and present research and anticipates research in the future. Each handbook in the Library includes introductory and concluding chapters written by its editor to provide a roadmap to the handbook’s table of contents and to offer informed anticipations of significant future developments in that field.
ix
An undertaking of this scope calls for handbook editors and chapter authors who are established scholars in the areas about which they write. Many of the nation’s and world’s most productive and best-respected psychologists have agreed to edit Library handbooks or write authoritative chapters in their areas of expertise. For whom has the Oxford Library of Psychology been written? Because of its breadth, depth, and accessibility, the Library serves a diverse audience, including graduate students in psychology and their faculty mentors, scholars, researchers, and practitioners in psychology and related fields. Each will find in the Library the information they seek on the subfield or focal area of psychology in which they work or are interested. Befitting its commitment to accessibility, each handbook includes a comprehensive index, as well as extensive references to help guide research. And because the Library was designed from its inception as an online as well as print resource, its structure and contents will be readily and rationally searchable online. Further, once the Library is released online, the handbooks will be regularly and thoroughly updated. In summary, the Oxford Library of Psychology will grow organically to provide a thoroughly informed perspective on the field of psychology, one that reflects both psychology’s dynamism and its increasing interdisciplinarity. Once published electronically, the Library is also destined to become a uniquely valuable interactive tool, with extended search and browsing capabilities, As you begin to consult this handbook, we sincerely hope you will share our enthusiasm for the more than 500-year tradition of Oxford University Press for excellence, innovation, and quality, as exemplified by the Oxford Library of Psychology. Peter E. Nathan Editor-in-Chief Oxford Library of Psychology
x
oxford library of psychology
ABOUT THE EDITORS
Jerome R. Busemeyer is Provost Professor of Psychology at Indiana University. He
was the president of Society for Mathematical Psychology and editor of the Journal of Mathematical Psychology. His theoretical contributions include decision field theory and, more recently, pioneering the new field of quantum cognition. Zheng Wang is Associate Professor at the Ohio State University and directs the Communication and Psychophysiology Lab. Much of her research tries to understand how our cognition, decision making, and communication are contextualized. James T. Townsend is Distinguished Rudy Professor of Psychology at Indiana
University. He was the president of Society for Mathematical Psychology and editor of the Journal of Mathematical Psychology. His theoretical contributions include systems factorial technology and general recognition theory. Ami Eidels is Senior Lecturer at the School of Psychology, University of Newcastle, Australia, and a principle investigator in the Newcastle Cognition Lab. His research focuses on human cognition, especially visual perception and attention, combined with computational and mathematical modeling.
xi
CONTRIBUTORS
Daniel Algom School of Psychological Sciences Tel-Aviv University Israel F. Gregory Ashby Department of Psychological and Brain Sciences University of California, Santa Barbara Santa Barbara, CA Joseph L. Austerweil Department of Cognitive Linguistic, and Psychological Sciences Brown University Providence, RI Scott D. Brown School of Psychology University of Newcastle Callaghan, NSW Australia Jerome R. Busemeyer Department of Psychological and Brain Sciences Cognitive Science Program Indiana University Bloomington, IN Amy H. Criss Department of Psychology Syracuse University Syracuse, NY Simon Dennis Department of Psychology The Ohio State University Columbus, OH Adele Diederich Psychology Jacobs University Bremen gGmbH Bremen 28759 Germany Chris Donkin School of Psychology University of New South Wales Kensington, NSW Australia
Ami Eidels School of Psychology University of Newcastle Callaghan, NSW Australia Samuel J. Gershman Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA Thomas L. Griffiths Department of Psychology University of California, Berkeley Berkeley, CA Todd M. Gureckis Department of Psychology New York University New York, NY Robert X. D. Hawkins Department of Psychological and Brain Sciences Indiana University Bloomington, IN Andrew Heathcote School of Psychology University of Newcastle Callaghan, NSW Australia Marc W. Howard Department of Psychological and Brain Sciences Center for Memory and Brain Boston University Boston, MA Brett Jefferson Department of Psychological and Brain Sciences Indiana University Bloomington, IN Michael N. Jones Department of Psychological and Brain Sciences Indiana University Bloomington, IN xiii
John K. Kruschke Department of Psychological and Brain Sciences Indiana University Bloomington, IN Yunfeng Li Department of Psychological Sciences Purdue University West Lafayette, IN Gordon D. Logan Department of Psychology Vanderbilt Vision Research Center Center for Integrative and Cognitive Neuroscience Vanderbilt University Nashville, TN Bradley C. Love University College London Experimental Psychology London, UK Dora Matzke Department of Psychology University of Amsterdam Amsterdam, the Netherlands Robert M. Nosofsky Department of Psychological and Brain Sciences Indiana University Bloomington, IN Richard W. J. Neufeld Departments of Psychology and Psychiatry, Neuroscience Program University of Western Ontario London, Ontario Canada Thomas J. Palmeri Department of Psychology Vanderbilt Vision Research Center Center for Integrative and Cognitive Neuroscience Vanderbilt University Nashville, TN Zygmunt Pizlo Department of Psychological Sciences Purdue University West Lafayette, IN Timothy J. Pleskac Center for Adaptive Rationality (ARC) Max Planck Institute for Human Development Berlin, Germany xiv
contributors
Emmanuel Pothos Department of Psychology City University London London, UK Babette Rae School of Psychology University of Newcastle Callaghan, NSW Australia Roger Ratcliff Department of Psychology The Ohio State University Columbus, OH Tadamasa Sawada Department of Psychology Higher School of Economics Moscow, Russia Jeffrey D. Schall Department of Psychology Vanderbilt Vision Research Center Center for Integrative and Cognitive Neuroscience Vanderbilt University Nashville, TN Philip Smith School of Psychological Sciences The University of Melbourne Parkville, VIC Australia Fabian A. Soto Department of Psychological and Brain Sciences University of California, Santa Barbara Santa Barbara, CA Joshua B. Tenenbaum Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Cambridge, MA James T. Townsend Department of Psychological and Brain Sciences Cognitive Science Program Indiana University Bloomington, IN Joachim Vandekerckhove Department of Cognitive Sciences University of California, Irvine Irvine, CA Wolf Vanpaemel Faculty of Psychology and Educational Sciences University of Leuven Leuven, Belgium
Eric-Jan Wagenmakers Department of Psychology University of Amsterdam Amsterdam, the Netherlands Thomas S. Wallsten Department of Psychology University of Maryland College Park, MD
Zheng Wang School of Communication Center for Cognitive and Brain Sciences The Ohio State University Columbus, OH Jon Willits Department of Psychological and Brain Sciences Indiana University Bloomington, IN
contributors
xv
CONTENTS
Preface
xix
1. Review of Basic Mathematical Concepts Used in Computational and Mathematical Psychology 1 Jerome R. Busemeyer, Zheng Wang, Ami Eidels, and James T. Townsend
Part I
•
Elementary Cognitive Mechanisms
2. Multidimensional Signal Detection Theory 13 F. Gregory Ashby and Fabian A. Soto 3. Modeling Simple Decisions and Applications Using a Diffusion Model 35 Roger Ratcliff and Philip Smith 4. Features of Response Times: Identification of Cognitive Mechanisms through Mathematical Modeling 63 Daniel Algom, Ami Eidels, Robert X. D. Hawkins, Brett Jefferson, and James T. Townsend 5. Computational Reinforcement Learning 99 Todd M. Gureckis and Bradley C. Love
Part II
•
Basic Cognitive Skills
6. Why Is Accurately Labeling Simple Magnitudes So Hard? A Past, Present, and Future Look at Simple Perceptual Judgment 121 Chris Donkin, Babette Rae, Andrew Heathcote, and Scott D. Brown 7. An Exemplar-Based Random-Walk Model of Categorization and Recognition 142 Robert M. Nosofsky and Thomas J. Palmeri 8. Models of Episodic Memory 165 Amy H. Criss and Marc W. Howard
Part III
•
Higher Level Cognition
9. Structure and Flexibility in Bayesian Models of Cognition 187 Joseph L. Austerweil, Samuel J. Gershman, Joshua B. Tenenbaum, and Thomas L. Griffiths xvii
10. Models of Decision Making under Risk and Uncertainty 209 Timothy J. Pleskac, Adele Diederich, and Thomas S. Wallsten 11. Models of Semantic Memory 232 Michael N. Jones, Jon Willits, and Simon Dennis 12. Shape Perception 255 Tadamasa Sawada, Yunfeng Li, and Zygmunt Pizlo
Part IV
•
New Directions
13. Bayesian Estimation in Hierarchical Models 279 John K. Kruschke and Wolf Vanpaemel 14. Model Comparison and the Principle of Parsimony 300 Joachim Vandekerckhove, Dora Matzke, and Eric-Jan Wagenmakers 15. Neurocognitive Modeling of Perceptual Decision Making 320 Thomas J. Palmeri, Jeffrey D. Schall, and Gordon D. Logan 16. Mathematical and Computational Modeling in Clinical Psychology 341 Richard W. J. Neufeld 17. Quantum Models of Cognition and Decision 369 Jerome R. Busemeyer, Zheng Wang, and Emmanuel Pothos Index
xviii
contents
391
P R E FA C E
Computational and mathematical psychology has enjoyed rapid growth over the past decade. Our vision for the Oxford Handbook of Computational and Mathematical Psychology is to invite and organize a set of chapters that review these most important developments, especially those that have impacted— and will continue to impact—other fields such as cognitive psychology, developmental psychology, clinical psychology, and neuroscience. Together with a group of dedicated authors, who are leading scientists in their areas, we believe we have realized our vision. Specifically, the chapters cover the key developments in elementary cognitive mechanisms (e.g., signal detection, information processing, reinforcement learning), basic cognitive skills (e.g., perceptual judgment, categorization, episodic memory), higher-level cognition (e.g., Bayesian cognition, decision making, semantic memory, shape perception), modeling tools (e.g., Bayesian estimation and other new model comparison methods), and emerging new directions (e.g., neurocognitive modeling, applications to clinical psychology, quantum cognition) in computation and mathematical psychology. An important feature of this handbook is that it aims to engage readers with various levels of modeling experience. Each chapter is self-contained and written by authoritative figures in the topic area. Each chapter is designed to be a relatively applied introduction with a great emphasis on empirical examples (see New Handbook of Mathematical Psychology (2014) by Batchelder, Colonius, Dzhafarov, and Myung for a more mathematically foundational and less applied presentation). Each chapter endeavors to immediately involve readers, inspire them to apply the introduced models to their own research interests, and refer them to more rigorous mathematical treatments when needed. First, each chapter provides an elementary overview of the basic concepts, techniques, and models in the topic area. Some chapters also offer a historical perspective of their area or approach. Second, each chapter emphasizes empirical applications of the models. Each chapter shows how the models are being used to understand human cognition and illustrates the use of the models in a tutorial manner. Third, each chapter strives to create engaging, precise, and lucid writing that inspires the use of the models. The chapters were written for a typical graduate student in virtually any area of psychology, cognitive science, and related social and behavioral sciences, such as consumer behavior and communication. We also expect it to be useful for readers ranging from advanced undergraduate students to experienced faculty members and researchers. Beyond being a handy reference book, it should be beneficial as
xix
a textbook for self-teaching, and for graduate level (or advanced undergraduate level) courses in computational and mathematical psychology. We would like to thank all the authors for their excellent contributions. Also we thank the following scholars who helped review the book chapters in addition to the editors (listed alphabetically): Woo-Young Ahn, Greg Ashby, Scott Brown, Cody Cooper, Amy Criss, Adele Diederich, Chris Donkin, Yehiam Eldad, Pegah Fakhari, Birte Forstmann, Tom Griffiths, Andrew Heathcote, Alex Hedstrom, Joseph Houpt, Marc Howard, Matt Irwin, Mike Jones, John Kruschke, Peter Kvam, Bradley Love, Dora Matzke, Jay Myung, Robert Nosofsky, Tim Pleskac, Emmanuel Pothos, Noah Silbert, Tyler Solloway, Fabian Soto, Jennifer Trueblood, Joachim Vandekerckhove, Wolf Vanpaemel, Eric-Jan Wagenmakers, and Paul Williams. The authors and reviewers’ effort ensure our confidence in the high quality of this handbook. Finally, we would like to express how much we appreciate the outstanding assistance and guidance provided by our editorial team and production team at Oxford University Press. The hard work provided by Joan Bossert, Louis Gulino, Anne Dellinger, A. Joseph Lurdu Antoine and the production team of Newgen Knowledge Works Pvt. Ltd., and others at the Oxford University Press are essential for the development of this handbook. It has been a true pleasure working with this team! Jerome R. Busemeyer Zheng Wang James T. Townsend Ami Eidels December 16, 2014
xx
preface
CHAPTER
1
Review of Basic Mathematical Concepts Used in Computational and Mathematical Psychology
Jerome R. Busemeyer, Zheng Wang, Ami Eidels, and James T. Townsend
Abstract
Computational and mathematical models of psychology all use some common mathematical functions and principles. This chapter provides a brief overview. Key Words: mathematical functions, derivatives and integrals, probability theory,
expectations, maximum likelihood estimation
We have three ways to build theories to explain and predict how variables interact and relate to each other in psychological phenomena: using natural verbal languages, using formal mathematics, and using computational methods. Human intuitive and verbal reasoning has a lot of limitations. For example, Hintzman (1991) summarized at least 10 critical limitations, including our incapability to imagine how a dynamic system works. Formal models, including both mathematical and computational models, can address these limitations of human reasoning. Mathematics is a “radically empirical” science (Suppes, 1984, p.78), with consistent and rigorous evidence (the proof ) that is “presented with a completeness not characteristic of any other area of science” (p.78). Mathematical models can help avoid logic and reasoning errors that are typically encountered in human verbal reasoning. The complexity of theorizing and data often requires the aid of computers and computational languages. Computational models and mathematical models can be thought of as a continuum of a theorizing process. Every computational model is based on a certain mathematical model, and almost every mathematical model can be implemented as a computational model.
Psychological theories may start as a verbal description, which then can be formalized using mathematical language and subsequently coded into computational language. By testing the models using empirical data, the model fitting outcomes can provide feedback to improve the models, as well as our initial understanding and verbal descriptions. For readers who are newcomers to this exciting field, this chapter provides a review of basic concepts of mathematics, probability, and statistics used in computational and mathematical modeling of psychological representation, mechanisms, and processes. See Busemeyer and Diederich (2010) and Lewandowsky and Ferrel (2010) for a more detailed presentations.
Mathematical Functions Mathematical functions are used to map a set of points called the domain of the function into a set of points called the range of the function such that only one point in range is assigned to each point in the domain.1 As a simple example, the linear function is defined as f (x) = a·x where the constant a is the slope of a straight line. In general, we use the notation f (x) to represent a function f that maps a domain point x into a range point y = f (x). If a
1
function f (x) has the property that each range point y can only be reached by a single unique domain point x, then we can define the inverse function f −1 (y) = x that maps each range point y = f (x) back to the corresponding domain point x. For example, the quadratic function is defined as the map f (x) = x 2 = x · x, and if we pick the number x = 3.5, then f (3.5) = 3.52 = 12.25. The quadratic function is defined on a domain of both positive and negative real numbers, and it does not have an inverse because, for example, ( − x)2 = x 2 and so there are two ways to get back from each range point y to the domain. However, if we restrict the domain to the non-negative real numbers, then the inverse defined of x 2 exists and it is the square root function √ √ on non-negative real numbers y = x 2 = x. There are, of course, a large number of functions used in mathematical psychology, but some of the most popular ones include the following. The power function is denoted x a where the variable x is a positive real number and the constant a is called the power. A quadratic function can be obtained by setting a = 2 but we could instead choose a = 0.50, which is the square root function √ x .50 = x, or we could choose a = −1, which produces the reciprocal x −1 = 1x , or we could choose any real number such as a = 1.37. Using a calculator, one finds that if x = 15.25 and a = 1.37, then 15.251.37 = 41.8658. One important property to remember about power functions is that x a · x b = x a+b and x b · y b = (x · y)b and (x a )b = x ab . Also note that x 0 = 1. Note that when working with the power function, the variable x appears in the base, and the constant a appears as the power. The exponential function is denoted e x where the exponent x is any real valued variable and the constant base e stands for a special number that is approximately e ∼ = 2.7183. Sometimes it is more convenient to use the notation e x = exp (x) instead. Using a calculator, we can calculate e 2.5 = 2.71832.5 = 12.1825. Note that the exponent can be negative −x < 0, in which case we can write e −x = e1x . If x = 0, then e0 = 1. The exponential function always returns a positive value, ex > 0, and it approaches zero as x approaches negative infinity. More complex forms of the exponential are often used. For example, you will later see the function
−
x−μ σ
2
, where x is a variable and μ and σ are e constants. In this case, it is more convenient to 2 2 − x−μ σ = exp − x−μ . This tells write this as e σ
2
you to first compute the squared deviation y = x−μ 2 and then compute the reciprocal exp1 y . σ () The exponential function obeys the property e x · ey = ex+y and (e x )a = e a·x . In contrast to the power function, the base of the exponential is a constant and the exponent is a variable. The (natural) log function is denoted ln (x) for positive values of x. For example, using a calculator, for x = 10, we obtain ln (10) = 2.3026. (We normally use the natural base e = 2.7183. If instead we used base 10, then log10 (10) = 1.) The log function obeys the rules ln (x ·y) = ln (x)+ln (y) and ln (x a ) = a · ln (x). The log function is the inverse of the exponential function: ln ( exp (x)) = x and the exponential function is the inverse of the log function exp ( log (x)) = x. The function ax where a is a constant and x is a variable can be rewritten in terms of the exponential function: define b = ln (a), then e bx = (eb )x = exp ( ln (a))x = ax . Figure 1.1 illustrates the power, exponential, and log functions using different coefficient values for the function. As can be seen, the coefficient changes the curve of the functions. Last but not least are the trigonometric functions based on a circle. Figure 1.2 shows a circle with its center located at coordinates (0, 0) in an (X , Y ) plane. Now imagine a line segment of radius r = 1 that extends from the center point to the circumference of the circle. This line segment intersects with the circumference at coordinates ( cos (t · π), sin (t · π) ) in the plane. The coordinate cos (t · π) represents the projection of the point on the circumference down onto the the X axis, and the point sin (t · π) is the projection of the point on the circumference to the Y axis. The variable t (which, for example can be time) moves this point around the circle, with positive values moving the point counter clockwise, and negative values moving it clockwise. The constant π = 3.1416 equals one-half cycle around the circle, and 2π is the period of time it takes to go all the way around once. The two functions are related by a translation (called the phase) in time: cos (t · π + (π/2)) = sin (t · π). Note that cos is an even function because cos (t · π ) = cos (−t · π), whereas sin is an odd function because − sin (t · π ) = sin ( − t · π ) . Also note that these functions are periodic in the sense that for example cos (t · π ) = cos (t · π + 2 · k · π ) for any integer k. We can generalize these two functions by changing the frequency and the phase. For example, cos (ω · tπ + θ ) is a cosine function with a frequency ω (changing the time it takes
review of basic mathematical concepts
Power function: y = xa
5 4
Exponential function: y = exp(x)
5
a=2
3
y
y
y
a = 1/2
y = log(x)
3
3
2
y = log(2x)
4
y = exp(x)
4
Log function: y = log(x)
5
y = log(0.5x) 2
2 y = exp(0)
1 0
a = –1 0
1
2
3
1
1 y = exp(–x) 4
5
0
0
1
2
3
4
5
0
0
10
x
x
20
30
40
50
x
1
1
0.8
0.8
0.6
0.6
0.4
0.4
0.2
0.2
Sin(Time)
Sin(x)
Fig. 1.1 Examples of three important functions, with various parameter values. From left to right: Power function, exponential function, and log function. See text for details.
0 –0.2
0 –0.2
–0.4
–0.4
–0.6
–0.6
–0.8
–0.8
–1 –1 –0.8 –0.6 –0.4 –0.2
0
0.2 0.4 0.6 0.8
–1
1
Cos(x)
0
0.2 0.4 0.6 0.8
0
1.2 1.4 1.6 1.8
2
Time
Fig. 1.2 Left panel illustrates a point on a unit circle with a radius equal to one. Vertical line shows sine, horizontal line shows cosine. Right panel shows sine as a function of time. The point on the Y axis of the right panel corresponds to the point on the Y axis of the left panel.
to complete a cycle) and a phase θ (advancing or delaying the initial value at time t = 0).
Derivatives and Integrals A derivative of a continuous function is the rate of change or the slope of a function at some point. Suppose f (x) is some continuous function. For a small increment , the change in this function is df (x) = f (x) − f (x − ) and the rate of change df (x) is the change divided by the increment = f (x)−f (x−) .
If the function is continuous, then as → 0, this ratio converges to what is called the derivative of the function at x, denoted as
d dx f
(x). The derivatives of many functions are derived in calculus (see Stewart (2012) or any calculus textbook for an introduction to calculus). d c·x For example, in calculus it is shown that dx e = c·x c · e , which says that the slope of the exponential function at any point x is proportional to the exponential function itself. As another example, it d a x = a · x a−1 , which is is shown in calculus that dx the derivative of the power function. For example, the derivative of a quadratic function a·x 2 is a linear function 2 · a · x, and the derivative of the linear function a · x is the constant a. The derivative of the cosine function is dtd cos (t) = − sin (t) and the derivative of sine is dtd sin (t) = cos (t).
review of basic mathematical concepts
3
Power function: y = xa; a = 2
4
4
3
3
Power function: y = x a; a = 0.5
y
5
y
5
2
2
1
1
0
0
1
2
3
4
5
0
0
1
x
2
3
4
5
x
Fig. 1.3 Illustration of the power function and its derivatives. The lines in both panels mark the power-function line. The slope of the dotted line (the tangent to the function) is given by the derivative of that function. (in this example, at x = 1).
Figure 1.3 illustrates the derivative of the power function at the value x = 1 for two different coefficients. The curved line shows the power function, and the straight line touches the curve at x = 1. The slope of this line is the derivative. The integral of a continuous function is the area under the curve within some interval (see Fig. 1.4). Suppose f (x) is a continuous function of x within the interval [a, b]. A simple way to approximate this area is to divide the interval into N very small steps, with a small increment being a step: [x0 = a, x1 = a + , x2 = a + 2, ..., xj = a + j · , ..., xN −1 = b − , xN = b] Then, compute the area of the rectangle within each step, · f xj , and finally sum all the areas of the rectangles to obtain an approximate area under the curve: N A ≈ · f (x1 ) + · f (x2 ) + · · · + · f xN = f xj · . j=1
As the number of intervals becomes arbitrarily large and the increments get arbitrarily small so that N → ∞ and → 0, this sum converges to the integral b A= f (x) · dx. a
If we allow the upper limit of the integral to be a variable, say z, then the integral becomes a function of z the upper limit, which can be written as F (z) = a f (x) · dx . What happens if we take the derivative of an integral? Let’s examine the change in the area divided by the increment 4
A (xN ) − A (xN −1 ) = f (xN ), A (xN ) − A (xN −1 ) = f (xN ) . This simple idea (proven more rigorously in a calculus textbook) leads to the first fundamental d theorem of integrals which states that dz F (z) = z f (z) , with F (z) = a f (x)dx. The fundamental theorem can then be used to find the integral of z a function. For example, the integral of 0 x a dx = d (a + 1)−1 z a+1 = z a . The (a + 1)−1 z a+1 because dz z α·x 1 α·z e dx = α e because dtd eα·z = α · integral of z α·z cos (t)dt = sin (z) because e . The integral of d dz sin (z) = cos (z). Computational and mathematical models are often described by difference or differential equations. These types of equations are used to describe how the state of a system changes with time. For example, suppose V (t) represents the strength of a neural connection between an input and an output at time t, and suppose x (t) is some reward signal that is guiding the learning process. A simple, discrete time linear model of learning can be V (t) = (1 − α) · V (t − 1) + α · x (t), where 0 ≤ α ≤ 1 is the learning rate parameter. We can rewrite this as a difference equation: dV (t) = V (t) − V (t − 1) = − α · V (t − 1) + α · x (t) = − α · (V (t − 1) − x (t)) . This model states that the change in strength at time t is proportional to the negative of the error signal, which is defined as the difference between the
review of basic mathematical concepts
300
70
250
60 6
50
200
x 10–3
8
40
4
150 30 100
20
50 0
2
10 0
500
1000
0
0 0
500
1000
0
20
40
60
80
100
Fig. 1.4 The integral of the function is the area under curve. It can be approximated as the sum of the areas of the rectangles (left panel). As the rectangles become narrower (middle), the sum of their areas converges to the true integral (right).
previous strength and the new reward. If we wish to describe learning as occurring more continuously in time we can introduce a small time increment t into the model so that it states dV (t) =V (t) − V (t − t) = − α · t · (V (t − t) − x (t)), which says that the change in strength is proportional to the negative of the error signal, with the constant of proportionality now modified by the time increment. Dividing both sides by the time increment t we obtain dV (t) = −α · (V (t − t) − x (t)), t and now if we allow the time increment to approach zero in the limit, t → 0, then the preceding equation converges to a limit that is the differential equation d V (t) = −α · (V (t) − x (t)), dt which states that the rate of change in strength is proportional to the negative of the error signal. Sometimes we can solve the differential equation for a simple solution. For example, the solution to the equation dtd V (t) = −α · V (t) + c is V (t) = ac − e−α·t because when we substitute this solution back into the differential equation, it satisfies the equality of the differential equation c d c − e −αt = −α · − e −α·t + c. dt α α A stochastic difference equation is frequently used in cognitive modeling to represent how a state changes across time when it is perturbed by noise. For example, if we assume that the strength of
a connection changes according to the preceding learning model, but with some noise (denoted as (t)) added, then we can use the following stochastic difference equation √ dV (t) = −α · t · (V (t − t) − x (t)) + (t) · t.
√ Note that the noise is multiplied by t instead of t in a stochastic difference equation. This is required so that the effect of the noise does not disappear as t → 0, and the variance of the noise remains proportional to t (which is the key characteristic of Brownian motion processes). See Battacharya and Waymire (2009) for an excellent book on stochastic processes.
Elementary Probability Theory Probability theory describes how to assign probabilities to events. See Feller (1968) for a review of probability theory. We start with a sample space that is a set denoted as , which contains all the unique outcomes that can be realized. For simplicity, we will assume (unless noted otherwise) that the sample space is finite. (There could be a very large number of outcomes, but the number is finite.) For example, if a person takes two medical tests, test A and test B, and each test can be positive or negative, then the sample space contains four mutually exclusive and exhaustive outcomes: all four combinations of positive and negative test results from tests A and B. Figure 1.5 illustrates the situation for this simple example. An event, such as the event A (e.g., test A is positive) is a subset of the sample space. Suppose for the moment that A, B are two events. The disjunctive event A or B (e.g., test A is positive or test B is positive) is represented as the union A∪B. The conjunctive event A and B (e.g., test A is
review of basic mathematical concepts
5
Test A Negative Test B
Negative Positive
Positive
A
A B
A∩B
B
A∩B
Fig. 1.5 Two ways to illustrate the probability space of events A and B. The contingency table (left) and the Venn diagram (right) correspond in the following way: Positive values on both tests in the table (the conjunctive event, A∩B) are represented by the overlap of the circles in the Venn diagram. Positive values on one test but not on the other in the table (the XOR event, A positive and B negative, or vice versa) are represented by the nonoverlapping areas of circles A and B. Finally, tests that are both negative (upper left entry in the table) correspond in the Venn diagram to the area within the rectangle (the so-called “sample space”) that is not occupied by any of the circles.
positive and test B is positive) is represented as the intersection A ∩ B. The impossible event (e.g., test A is neither positive nor negative), denoted , is an empty set. The certain event is the entire sample space . The complementary event “not A” is denoted A. A probability function p assigns a number between zero to one to each event. The impossible event is assigned zero, and the certain event is assigned one. The other events are assigned probabilities 0 ≤ p (A) ≤ 1 and p A¯ = 1 − p (A). However, these probabilities must obey the following additive rule: If A ∩ B = then p(A ∪ B) = p(A) + p(B). What if the events are not mutually exclusive so that A ∩ B = ? The answer is called the “or”, which follows from the previous assumptions: p(A ∪ B) = p (A ∩ B) + p A ∩ B¯ + p A¯ ∩ B = p (A ∩ B) + p A ∩ B¯ + p A¯ ∩ B + p (A ∩ B) − p (A ∩ B) = p (A) + p (B) − p (A ∩ B) . Suppose we learn that some event A has occurred, and now we wish to define the new probability for event B conditioned on this known event. The conditional probability p(B|A) stands for the probability of event B given that event A has p(A∩B) occurred, which is defined as p (B|A) = p(A) . p(A∩B)
Similarly, p(A|B) = p(B) is the probability of event A given that B has occurred. Using the definition of conditional probability, we can then define the “and” rule for joint probabilities as follows: the probability of A and B equals p(A∩B) = p (A) p (B|A) = p (B) p (A|B). An important theorem of probability is called Bayes rule. It describes how to revise one’s beliefs based on evidence. Suppose we have two mutually 6
exclusive and exhaustive hypotheses denoted H1 and H2 . For example H1 could be a certain disease is present and H2 is the disease is not present. Define the event D as some observed data that provide evidence for or against each hypothesis, such as a medical test result. Suppose p(D|H1 ) and p (D|H2 ) are known. These are called the likelihood’s of the data for each hypothesis. For example, medical testing would be used to determine the likelihood of a positive versus negative test result when the disease is known to be present, and the likelihood of a positive versus negative test would also be known when the disease is not present. We define p(H1 ) and p(H2 ) as the prior probabilities of each hypothesis. For example, these priors may be based on base rates for disease present or not. Then according to the conditional probability definition p (H1 ) p (D|H1 ) p (H1 |D) = p (D) =
p (H1 ) p (D|H1 ) . p (H1 ) p (D|H1 ) + p (H2 ) p (D|H2 )
The last line is Bayes’ rule. The probability p (H1 |D) is called the posterior probability of the hypothesis given the data. It reflects the revision from the prior produced by the evidence from the data. If there are M ≥ 2 hypotheses, then the rule is extended to be p (H1 ) p (D|H1 ) , p (H1 |D) = M k=1 p (Hk ) p (D|Hk ) where the denominator is the sum across k = 1 to k = M hypotheses. We often work with events that are assigned to numbers. A random variable is a function that assigns real numbers to events. For example, a person may look at an advertisement and then rate how effective it is on a nine-point scale. In this case, there are nine mutually exclusive and exhaustive categories to choose from on the rating
review of basic mathematical concepts
scale, and each choice is assigned a number (say, 1, 2,. . ., or 9). Then we can define a random variable X (R), which is a function that maps the category event R onto one of the nine numbers. For example, if the person chooses the middle rating option, so that R = middle, then we assign X (middle) = 5. For simplicity, we often omit the event and instead write the random variable simply as X . For example, we can ask what is the probability that the random variable is assigned the number 5, which is written as p(X = 5). Then we assign a probability to each value of a random variable by assigning it the probability of the event that produces the value. For example, p(X = 5) equals the probability of the event that the person picks the middle value. Suppose the random
variable has N values x1 , x2, . . . , xi , . . . , xN . In our previous example with the rating scale, the random variable had nine values. The function p(X = xi ) (interpreted as the probability that the person picks a choice corresponding to value xi ) is called the probability mass function for the random variable X . This function has the following properties: 0 ≤ p (X = xi ) ≤ 1; N
p (X = xi ) = p (X = x1 ) + p (X = x2 )
i=1
+ · · · + p (X = xN ) = 1. The cumulative probability is then defined as p(X ≤ xi ) = p (X = x1 ) + p (X = x2 ) + · · · + p (X = xi ) =
i p X = xj . j=1
Often we measure more than one random variable. For example, we could present an advertisement and ask how effective it is for the participant personally but also ask how effective the participant believes it is for others. Suppose X is the random variable for the nine-point rating scale for self, and let Y be the random variable for the nine-point rating scale for others. Then we can define a joint probability p(X = xi , Y = yj ), which equals the probability that xj is selected for self and that yj is selected for others. These joint probabilities form a two way 9 × 9 table with with p(X = xi , Y = yj ) in each cell. This joint probability function has the properties:
0 ≤ p X = xi , Y = yj ≤ 1 p X = xi , Y = yj = p(X = xi ) j
p X = xi , Y = yj = p(Y = yj ) i
p X = xi , Y = yj = 1. i
j
Finally, we can define the conditional probability of Y = yj given X = xi as p Y = yj |X = xi = p(X =xi ,Y =yj ) . p(X =xi ) Often, we work with random variables that have a continuous rather than a discrete and finite distribution, such as the normal distribution. Suppose X is a univariate continuous random variable. In this case, the probability assigned to each real number is zero (there are uncountably infinite many of them in any interval). Instead, we start by defining the cumulative distribution function F (x) = p (X ≤ x). Then we define the probability density at each value of x as the derivative of the cumulative probability d F (x). Using this definition for function, f (x) = dx the density, we compute the probability of X falling b in some interval [a, b] as p (X ∈ [a, b]) = a f (x)dx . The increment f (x) · dx for the continuous random variable is conceptually related to the mass function p (X = xi ). The probability density function for the normal
− √1 e 2π σ
x−μ σ
2
, where distribution equals f (x) = μ is the mean of the distribution and σ is the standard deviation of the distribution. The normal distribution is popular because of the central limit theorem, which states that the sample mean X¯ = Xi /N of N independent samples of a random variable X will approach normal as the number of samples becomes arbitrarily large, even if the original random variable X is not normal. We often work with sample means, which we expect to be approximately normal because of the central limit theorem.
Expectations When working with random variables, we are often interested in their moments (i.e., their means, variances, or correlations). See Hogg and Craig (1970) for a reference on mathematical statistics. These different moments are different concepts, but they are all defined as expected values of the random variables. The expectation of the random variable is defined as E [X ] = i p (X = xi ) · xi for the
review of basic mathematical concepts
7
discrete case and it is defined as E [X ] = f (x)·x ·dx for the continuous case. The mean of a random variable X , denoted μX is defined as the expectation of the random variable μX = E [X ]. The variance of a random variable X , denoted σX2 , is defined as the expectation of the squared deviation
around the mean: σX2 = Var (X ) = E (X − μ)2 . For example, in the discrete case, this equals σX2 = i p (X = xi ) · is the square (xi − μX )2 . The standard deviation
root of the variance, σX = σX2 . The covariance between two random variables (X , Y ), denoted σXY , is defined by the expectation of the product of deviations: σXY = cov (X , Y ) = E [(X − μX ) · (Y − μY )]. For case, this example, in the discrete equals σXY = i j p X = xi , Y = yj ((xi − μX ) yj − μY . The correlation is defined as ρXY = σXY σX ·σY . Often we need to combine two random variables by a linear combination Z = a · X + b · Y , where a, b are two constants. For example, we may sum two scores, a = 1 and b = 1, or take a difference between two scores, a = 1, b = −1. There are two important rules for determining the mean and the variance of a linear transformation. The expectation operation is linear: E [a · X + b · Y ] = a · E [X ] + b · E [Y ]. The variance operator, however, is not linear: var (a · X + b · Y ) = a2 var (X ) + b2 var (Y ) + 2ab · cov (X , Y ).
Maximum Likelihood Estimation Computational and mathematical models of psychology contain parameters that need to be estimated from the data. For example, suppose a person can choose to play or not play a slot machine at the beginning of each trial. The slot machine pays the amount x(t) on trial t, but this amount is not revealed until the trial is over. Consider a simple model that assumes that the probability of choosing to gamble on trial t, denoted p(t), is predicted by the following linear learning model V (t) = (1 − α) · V (t − 1) + α · x(t) p(t) =
1 1 + e −β·V (t)
This model has two parameters α, β that must be estimated from the data. This is analogous to the estimation problem one faces when using multiple linear regression, where the linear regression is the model and the regression coefficients are the model parameters. However, computational and mathematical models, such as the earlier learning model example, are nonlinear with respect to 8
the model parameters, which makes them more complicated, and one cannot use simple linear regression fitting routines. The model parameters are estimated from the empirical experimental data. These experiments usually consist of a sample of participants, and each participant provides a series of responses to several experimental conditions. For example, a study of learning to gamble could obtain 100 choice trials at each of 5 payoff conditions from each of 50 participants. One of the first issues for modeling is the level of analysis of the data. On the one hand, a group-level analysis would fit a model to all the data from all the participants ignoring individual differences. This is not a good idea if there are substantial individual differences. On the other hand, an individual level analysis would fit a model to each individual separately allowing arbitrary individual differences. This introduces a new set of parameters for each person, which is unparsimonious. A hierarchical model applies the model to all of the individuals, but it includes a model for the distribution of individual differences. This is a good compromise, but it requires a good model of the distribution of individual differences. Chapter 13 of this book describes the hierarchical approach. Here we describe the basic ideas of fitting models at the individual level using a method called maximum likelihood (see Myung, 2003 for a detailed tutorial on maximum likelihood estimation). Also see Hogg and Craig (1970) for the general properties of maximum likelihood estimates. Suppose we obtain 100 choice trials (gamble, not gamble) from 5 payoff conditions from each participant. The above learning model has two parameters (α, β) that we wish to estimate using the 100 × 5 = 500 binary valued responses. We can put the 500 answers in a vector D = [x1 , x2 , . . . , xt , . . . , x500 ] , where each xt is zero (not gamble) or one (gamble). If we pick values for the two parameters (α, β), then we can insert these into our learning model and compute the probability of gambling, p (t), for each trial from the model. Define p(xt , t) as the probability that the model predicts the value xt observed on trial t. For example, if xt =1, then p(x t , t) = p(t), but if xt = 0, then p(xt , t) = 1 − p(t) , where recall that p(t) is the predicted probability of choosing the gamble. Then we compute the likelihood of the observed sequence of data D given the model parameters (α, β) as follows: L (D|α, β) = p (x1 , 1) p (x2 , 2) · · · p (x500 , 500) .
review of basic mathematical concepts
300 250 200 150 100 50 0 100
200
300
400
500
600
700
800
Fig. 1.6 Example of maximum likelihood estimation. The histograms describe data sampled from a Gamma distribution with scale and shape parameters both equal to 20. Using maximum likelihood we can estimate the parameter values of a Gamma distribution that best fits the sample (they turn out to be 20.4 and 19.5, respectively), and plot its probability density function (solid line).
To make this computationally feasible, we use the log likelihood instead LnL (D|α, β) =
500 ln p(xt , t) t=1
This likelihood changes depending on our choice of α, β. Our goal is to pick (α, β) that maximizes LnL(D|αβ). Nonlinear search algorithms available in computational software, such as MATLAB, R, Gauss, and Mathematica can be used to find the maximum likelihood estimates. The log likelihood is a goodness-of-fit measure. Higher values indicate better fit. Actually the computer algorithms find the minimum of the badness of fit measure G 2 = −2 · LnL(D|α, β). Maximum likelihood is not restricted to learning models and it can be used to fit all kinds of models. For example, if we observe a response time on each trial, and our model predicts the response time for each trial, then the preceding equation can be applied with xt equal to the observed response time on a trial, and with p(xt , t) equal to the predicted probability for the observed value of response time on that trial. Figure 1.6 shows an example in which a sample of response time data (summarized in the figure by a histogram) was fit by a gamma distribution model for response time using two model parameters. Now suppose we have two different competing learning models. The model we just described has two parameters. Suppose the competing model is quite different and it is more complex with four parameters. Also suppose the models are not nested so that it is not possible to compute the same predictions for the simpler model using the more
complex model. Then we can compare models by using a Bayesian information criterion (BIC, see Wasserman, 2000, for review). This criterion is derived on the basis of choosing the model that is most probable given the data. (However, the derivation only holds asymptotically as the sample size increases indefinitely.) For each model we wish to compare, we can compute a BIC index: 2 + nmodel · ln (N ), where nmodel BICmodel = Gmodel equals the number of model parameters estimated from the data and N = number of data points. The BIC index is an index that balances model fit with model complexity as measured by number of parameters. (Note however, that model complexity is more than the number of parameters, see Chapter 13). It is a badness-of-fit index, and so we choose the model with the lowest BIC index. See Chapter 14 for a detailed review on model comparison.
Concluding comments This handbook categorizes models into three levels based on the questions asked and addressed by the models, and the levels of analysis. The first category includes models theorizing the elementary cognitive mechanisms, such as the signal detection process, the diffusion process, informaton processing, and reinforcement learning. They have been widely used in many formal models within and beyond psychology and cognitive science, ranging from basic visual perception to complex decision making. The second category covers models theorizing basic cognitive skills, such as perceptual identification, categorization, and episodic memory. The third category includes models theorizing higher level cognition, such as Bayesian cognition, decision making, semantic memory, and shape
review of basic mathematical concepts
9
perception. In addition, we provide two chapters on modeling tools, including Bayesian estimation in hierarchical models and model comparison methods. We conclude the handbook with three chapters on new directions in the field, including neurocognitive modeling, mathematical and computational modeling in clinical psychology, and cognitive and decision models based upon quantum probability theory. The models reviewed in the handbook make use of many of the mathematical ideas presented in this review chapter. Probabilistic models appear in chapters covering signal detection theory (Chapter 2), probabilistic models of cognition (Chapter 9), decision theory (Chapters 10 and 17), and clinical applications (Chapter 16). Stochastic models (i.e., models that are dynamic and probabilistic) appear in chapters covering information processing (Chapter 4), percpetual judgment (Chapter 6), and random walk/diffusion models of choice and response time in various cogntive tasks (Chapters 3, 7, 10, and 15). Learning and memory models are reviewed in Chapters 5, 7, 8, and 11. Models using vector spaces and geometry are introduced in Chapters 11, 12, and 17. The basic concepts reviewed in this chapter should be helpful for readers who are new to mathematical and computational models to jumpstart reading the rest of the book. In addition, each chapter is self-contained, presents a tutorial style introduction to the topic area exemplified by many
10
applications, and provides a specific glossary list of the basic concepts in the topic area. We believe you will have a rewarding reading experience.
Note 1. This chapter is restricted to real numbers.
References Battacharya, R. N. & Waymire, E. C. (2009). Stochastic processes with applications (Vol. 61). Philadelphia, PA: Siam. Busemeyer, J. R., & Diederich, A. (2009). Cognitive modeling. Thousand Oaks, CA: SAGE. Cox, D. R. & Miller, H. D. (1965). The theory of stochastic processes (Vol. 134). Boca Raton, FL: CRC Press. Feller, W. (1968). on An introduction to probability theory and its applications (3rd ed., Vol. 1). New York, NY: Wiley. Hintzman D.L. (1991). Why are formal models useful in psychology? In W. E. Hockley & S. Lewandowsky (Eds.), Relating theory and data: Essays on human memory in honor of Bennet B. Murdock (pp. 39–56). Hillsdale, NJ: Erlbaum. Hogg, R. V., & Craig, A. T. (1970). Introduction to mathematical statistics (3rd ed.). New York, NY: Macmillan. Lewandowsky, S. & Ferrel, S. (2010). Computational modeling in cognition: Principles and practice. Thousand Oaks, CA: SAGE. Myung, I. J. (2003). Tutorial on maximum likelihood estimation. Journal of Mathematical Psychology, 47, 90–100. Stewart, J. (2012). Calculus (7th ed.) Belmont, CA: Brooks/Cole. Suppes, P. (1984). Probabilistic metaphysics. Oxford: Basil Blackwell. Wasserman, L. (2000) Bayesian model selection and model averaging. Journal of Mathematical Psychology, 44(1), 92– 107.
review of basic mathematical concepts
PART
Elementary Cognitive Mechanisms
I
CHAPTER
2
Multidimensional Signal Detection Theory
F. Gregory Ashby and Fabian A. Soto
Abstract Multidimensional signal detection theory is a multivariate extension of signal detection theory that makes two fundamental assumptions, namely that every mental state is noisy and that every action requires a decision. The most widely studied version is known as general recognition theory (GRT). General recognition theory assumes that the percept on each trial can be modeled as a random sample from a multivariate probability distribution defined over the perceptual space. Decision bounds divide this space into regions that are each associated with a response alternative. General recognition theory rigorously defines and tests a number of important perceptual and cognitive conditions, including perceptual and decisional separability and perceptual independence. General recognition theory has been used to analyze data from identification experiments in two ways: (1) fitting and comparing models that make different assumptions about perceptual and decisional processing, and (2) testing assumptions by computing summary statistics and checking whether these satisfy certain conditions. Much has been learned recently about the neural networks that mediate the perceptual and decisional processing modeled by GRT, and this knowledge can be used to improve the design of experiments where a GRT analysis is anticipated. Key Words: signal detection theory, general recognition theory, perceptual separability,
perceptual independence, identification, categorization
Introduction Signal detection theory revolutionized psychophysics in two different ways. First, it introduced the idea that trial-by-trial variability in sensation can significantly affect a subject’s performance. And second, it introduced the field to the then-radical idea that every psychophysical response requires a decision from the subject, even when the task is as simple as detecting a signal in the presence of noise. Of course, signal detection theory proved to be wildly successful and both of these assumptions are now routinely accepted without question in virtually all areas of psychology. The mathematical basis of signal detection theory is rooted in statistical decision theory, which
itself has a history that dates back at least several centuries. The insight of signal detection theorists was that this model of statistical decisions was also a good model of sensory decisions. The first signal detection theory publication appeared in 1954 (Peterson, Birdsall, & Fox, 1954), but the theory did not really become widely known in psychology until the seminal article of Swets, Tanner, and Birdsall appeared in Psychological Review in 1961. From then until 1986, almost all applications of signal detection theory assumed only one sensory dimension (Tanner, 1956, is the principal exception). In almost all cases, this dimension was meant to represent sensory magnitude. For a detailed description of this standard univariate 13
theory, see the excellent texts of either Macmillan and Creelman (2005) or Wickens (2002). This chapter describes multivariate generalizations of signal detection theory. Multidimensional signal detection theory is a multivariate extension of signal detection to cases in which there is more than one perceptual dimension. It has all the advantages of univariate signal detection theory (i.e., it separates perceptual and decision processes) but it also offers the best existing method for examining interactions among perceptual dimensions (or components). The most widely studied version of multidimensional signal detection theory is known as general recognition theory (GRT; Ashby & Townsend, 1986). Since its inception, more than 350 articles have applied GRT to a wide variety of phenomena, including categorization (e.g., Ashby & Gott, 1988; Maddox & Ashby, 1993), similarity judgment (Ashby & Perrin, 1988), face perception (Blaha, Silbert, & Townsend, 2011; Thomas, 2001; Wenger & Ingvalson, 2002), recognition and source memory (Banks, 2000; Rotello, Macmillan, & Reeder, 2004), source monitoring (DeCarlo, 2003), attention (Maddox, Ashby, & Waldron, 2002), object recognition (Cohen, 1997; Demeyer, Zaenen, & Wagemans, 2007), perception/action interactions (Amazeen & DaSilva, 2005), auditory and speech perception (Silbert, 2012; Silbert, Townsend, & Lentz, 2009), haptic perception (Giordano et al., 2012; Louw, Kappers, & Koenderink, 2002), and the perception of sexual interest (Farris, Viken, & Treat, 2010). Extending signal detection theory to multiple dimensions might seem like a straightforward mathematical exercise, but, in fact, several new conceptual problems must be solved. First, with more than one dimension, it becomes necessary to model interactions (or the lack thereof ) among those dimensions. During the 1960s and 1970s, a great many terms were coined that attempted to describe perceptual interactions among separate stimulus components. None of these, however, were rigorously defined or had any underlying theoretical foundation. Included in this list were perceptual independence, separability, integrality, performance parity, and sampling independence. Thus, to be useful as a model of perception, any multivariate extension of signal detection theory needed to provide theoretical interpretations of these terms and show rigorously how they were related to one another.
14
Second, the problem of how to model decision processes when the perceptual space is multidimensional is far more difficult than when there is only one sensory dimension. A standard signaldetection-theory lecture is to show that almost any decision strategy is mathematically equivalent to setting a criterion on the single sensory dimension, then giving one response if the sensory value falls on one side of this criterion, and the other response if the sensory value falls on the other side. For example, in the normal, equal-variance model, this is true regardless of whether subjects base their decision on sensory magnitude or on likelihood ratio. A straightforward generalization of this model to two perceptual dimensions divides the perceptual plane into two response regions. One response is given if the percept falls in the first region and the other response is given if the percept falls in the second region. The obvious problem is that, unlike a line, there are an infinite number of ways to divide a plane into two regions. How do we know which of these has the most empirical validity? The solution to the first of these two problems— that is, the sensory problem—was proposed by Ashby and Townsend (1986) in the article that first developed GRT. The GRT model of sensory interactions has been embellished during the past 25 years, but the core concepts introduced by Ashby and Townsend (1986) remain unchanged (i.e., perceptual independence, perceptual separability). In contrast, the decision problem has been much more difficult. Ashby and Townsend (1986) proposed some candidate decision processes, but at that time they were largely without empirical support. In the ensuing 25 years, however, hundreds of studies have attacked this problem, and today much is known about human decision processes in perceptual and cognitive tasks that use multidimensional perceptual stimuli.
Box 1 Notation Ai Bj = stimulus constructed by setting component A to level i and component B to level j ai bj = response in an identification experiment signaling that component A is at level i and component B is at level j X1 = perceived value of component A X2 = perceived value of component B
elementary cognitive mechanisms
Box 1 Continued fij (x1 ,x2 ) = joint likelihood that the perceived value of component A is x1 and the perceived value of component B is x2 on a trial when the presented stimulus is Ai Bj gij (x1 ) = marginal pdf of component A on trials when stimulus Ai Bj is presented rij = frequency with which the subject responded Rj on trials when stimulus Si was presented P(Rj |Si ) = probability that response Rj is given on a trial when stimulus Si is presented
General Recognition Theory General recognition theory (see the Glossary for key concepts related to GRT) can be applied to virtually any task. The most common applications, however, are to tasks in which the stimuli vary on two stimulus components or dimensions. As an example, consider an experiment in which participants are asked to categorize or identify faces that vary across trials on gender and age. Suppose there are four stimuli (i.e., faces) that are created by factorially combining two levels of each dimension. In this case we could denote the two levels of the gender dimension by A1 (male) and A2 (female) and the two levels of the age dimension by B1 (teen) and B2 (adult). Then the four faces are denoted as A1 B1 (male teen), A1 B2 (male adult), A2 B1 (female teen), and A2 B2 (female adult). As with signal detection theory, a fundamental assumption of GRT is that all perceptual systems are inherently noisy. There is noise both in the stimulus (e.g., photon noise) and in the neural systems that determine its sensory representation (Ashby & Lee, 1993). Even so, the perceived value on each sensory dimension will tend to increase as the level of the relevant stimulus component increases. In other words, the distribution of percepts will change when the stimulus changes. So, for example, each time the A1 B1 face is presented, its perceived age and maleness will tend to be slightly different. General recognition theory models the sensory or perceptual effects of a stimulus Ai Bj via the joint probability density function (pdf ) fij (x1 , x2 ) (see Box 1 for a description of the notation used in this article). On any particular trial when stimulus Ai Bj is presented, GRT assumes that the subject’s percept can be modeled as a random sample from this
joint pdf. Any such sample defines an ordered pair (x 1 , x 2 ), the entries of which fix the perceived value of the stimulus on the two sensory dimensions. General recognition theory assumes that the subject uses these values to select a response. In GRT, the relationship of the joint pdf to the marginal pdfs plays a critical role in determining whether the stimulus dimensions are perceptually integral or separable. The marginal pdf gij (x 1 ) simply describes the likelihoods of all possible sensory values of X 1 . Note that the marginal pdfs are identical to the one-dimensional pdfs of classical signal detection theory. Component A is perceptually separable from component B if the subject’s perception of A does not change when the level of B is varied. For example, age is perceptually separable from gender if the perceived age of the adult in our face experiment is the same for the male adult as for the female adult, and if a similar invariance holds for the perceived age of the teen. More formally, in an experiment with the four stimuli, A1 B1 , A1 B2 , A2 B1 , and A2 B2 , component A is perceptually separable from B if and only if g11 (x1 ) = g12 (x1 )
and g21 (x1 ) = g22 (x1 )
for all values of x1 .
(1)
Similarly, component B is perceptually separable from A if and only if g11 (x2 ) = g21 (x2 ) and
g12 (x2 ) = g22 (x2 ), (2)
for all values of x 2 . If perceptual separability fails then A and B are said to be perceptually integral. Note that this definition is purely perceptual since it places no constraints on any decision processes. Another purely perceptual phenomenon is perceptual independence. According to GRT, components A and B are perceived independently in stimulus Ai Bj if and only if the perceptual value of component A is statistically independent of the perceptual value of component B on Ai Bj trials. More specifically, A and B are perceived independently in stimulus Ai Bj if and only if f ij (x1 , x2 ) = g ij (x1 ) gij (x2 )
(3)
for all values of x 1 and x 2 . If perceptual independence is violated, then components A and B are perceived dependently. Note that perceptual independence is a property of a single stimulus, whereas perceptual separability is a property of groups of stimuli. A third important construct from GRT is decisional separability. In our hypothetical experiment
multidimensional signal detection theory
15
with stimuli A1 B1 , A1 B2 , A2 B1 , and A2 B2 , and two perceptual dimensions X 1 and X 2 , decisional separability holds on dimension X 1 (for example), if the subject’s decision about whether stimulus component A is at level 1 or 2 depends only on the perceived value on dimension X 1 . A decision bound is a line or curve that separates regions of the perceptual space that elicit different responses. The only types of decision bounds that satisfy decisional separability are vertical and horizontal lines.
The Multivariate Normal Model So far we have made no assumptions about the form of the joint or marginal pdfs. Our only assumption has been that there exists some probability distribution associated with each stimulus and that these distributions are all embedded in some Euclidean space (e.g., with orthogonal dimensions). There have been some efforts to extend GRT to more general geometric spaces (i.e., Riemannian manifolds; Townsend, Aisbett, Assadi, & Busemeyer, 2006; Townsend & SpencerSmith, 2004), but much more common is to add more restrictions to the original version of GRT, not fewer. For example, some applications of GRT have been distribution free (e.g., Ashby & Maddox, 1994; Ashby & Townsend, 1986), but most have assumed that the percepts are multivariate normally distributed. The multivariate normal distribution includes two assumptions. First, the marginal distributions are all normal. Second, the only possible dependencies are pairwise linear relationships. Thus, in multivariate normal distributions, uncorrelated random variables are statistically independent. A hypothetical example of a GRT model that assumes multivariate normal distributions is shown in Figure 2.1. The ellipses shown there are contours of equal likelihood; that is, all points on the same ellipse are equally likely to be sampled from the underlying distribution. The contours of equal likelihood also describe the shape a scatterplot of points would take if they were random samples from the underlying distribution. Geometrically, the contours are created by taking a slice through the distribution parallel to the perceptual plane and looking down at the result from above. Contours of equal likelihood in multivariate normal distributions are always circles or ellipses. Bivariate normal distributions, like those depicted in Figure 2.1 are each characterized by five parameters: a mean on each dimension, a variance on each dimension, and a covariance or correlation between the values on 16
the two dimensions. These are typically catalogued in a mean vector and a variance-covariance matrix. For example, consider a bivariate normal distribution with joint density function f (x 1 ,x 2 ). Then the mean vector would equal μ1 (4) µ= μ2 and the variance-covariance matrix would equal σ12 cov12 = (5) cov21 σ22 where cov12 is the covariance between the values on the two dimensions (i.e., note that the correlation coefficient is the standardized covariance—that is, 12 the correlation ρ12 = cov σ1 σ2 ). The multivariate normal distribution has another important property. Consider an identification task with only two stimuli and suppose the perceptual effects associated with the presentation of each stimulus can be modeled as a multivariate normal distribution. Then it is straightforward to show that the decision boundary that maximizes accuracy is always linear or quadratic (e.g., Ashby, 1992). The optimal boundary is linear if the two perceptual distributions have equal variancecovariance matrices (and so the contours of equal likelihood have the same shape and are just translations of each other) and the optimal boundary is quadratic if the two variance-covariance matrices are unequal. Thus, in the Gaussian version of GRT, the only decision bounds that are typically considered are either linear or quadratic. In Figure 2.1, note that perceptual independence holds for all stimuli except A2 B2 . This can be seen in the contours of equal likelihood. Note that the major and minor axes of the ellipses that define the contours of equal likelihood for stimuli A1 B1 , A1 B2 , and A2 B1 are all parallel to the two perceptual dimensions. Thus, a scatterplot of samples from each of these distributions would be characterized by zero correlation and, therefore, statistical independence (i.e., in the special Gaussian case). However, the major and minor axes of the A2 B2 distribution are tilted, reflecting a positive correlation and hence a violation of perceptual independence. Next, note in Figure 2.1 that stimulus component A is perceptually separable from stimulus component B, but B is not perceptually separable from A. To see this, note that the marginal distributions for stimulus component A are the same, regardless of the level of component B [i.e., g 11 (x 1 ) =
elementary cognitive mechanisms
Respond A2B2 g22(x2)
Respond A1B2
f22(x1,x2)
f12(x1,x2)
Applying GRT to Data
X2
Xc2
g12(x2) g21(x2)
Respond A1B1
f21(x1,x2)
f11(x1,x2)
Respond A2B1
g11(x2) g11(x1) = g12(x1)
X1
larger perceived values of A). As a result, component B is not decisionally separable from component A.
g21(x1) = g22(x1)
Xc1
Fig. 2.1 Contours of equal likelihood, decision bounds, and marginal perceptual distributions from a hypothetical multivariate normal GRT model that describes the results of an identification experiment with four stimuli that were constructed by factorially combining two levels of two stimulus dimensions.
g 12 (x 1 ) and g 21 (x 1 ) = g 22 (x 1 ), for all values of x 1 ]. Thus, the subject’s perception of component A does not depend on the level of B and, therefore, stimulus component A is perceptually separable from B. On the other hand, note that the subject’s perception of component B does change when the level of component A changes [i.e., g 11 (x 1 ) = g 21 (x 1 ) and g 12 (x 1 ) = g 22 (x 1 ) for most values of x 1 ]. In particular, when A changes from level 1 to level 2 the subject’s mean perceived value of each level of component B increases. Thus, the perception of component B depends on the level of component A and therefore B is not perceptually separable from A. Finally, note that decisional separability holds on dimension 1 but not on dimension 2. On dimension 1 the decision bound is vertical. Thus, the subject has adopted the following decision rule: Component A is at level 2 if x 1 > X c1 ; otherwise component A is at level 1. Where X c1 is the criterion on dimension 1 (i.e., the x 1 intercept of the vertical decision bound). Thus, the subject’s decision about whether component A is at level 1 or 2 does not depend on the perceived value of component B. So component A is decisionally separable from component B. On the other hand, the decision bound on dimension x 2 is not horizontal, so the criterion used to judge whether component B is at level 1 or 2 changes with the perceived value of component A (at least for
The most common applications of GRT are to data collected in an identification experiment like the one modeled in Figure 2.1. The key data from such experiments are collected in a confusion matrix, which contains a row for every stimulus and a column for every response (Table 2.1 displays an example of a confusion matrix, which will be discussed and analyzed later). The entry in row i and column j lists the number of trials on which stimulus Si was presented and the subject gave response Rj . Thus, the entries on the main diagonal give the frequencies of all correct responses and the off-diagonal entries describe the various errors (or confusions). Note that each row sum equals the total number of stimulus presentations of that type. So if each stimulus is presented 100 times then the sum of all entries in each row will equal 100. This means that there is one constraint per row, so an n × n confusion matrix will have n × (n – 1) degrees of freedom. General recognition theory has been used to analyze data from confusion matrices in two different ways. One is to fit the model to the entire confusion matrix. In this method, a GRT model is constructed with specific numerical values of all of its parameters and a predicted confusion matrix is computed. Next, values of each parameter are found that make the predicted matrix as close as possible to the empirical confusion matrix. To test various assumptions about perceptual and decisional processing—for example, whether perceptual independence holds—a version of the model that assumes perceptual independence is fit to the data as well as a version that makes no assumptions about perceptual independence. This latter version contains the former version as a special case (i.e., in which all covariance parameters are set to zero), so it can never fit worse. After fitting these two models, we assume that perceptual independence is violated if the more general model fits significantly better than the more restricted model that assumes perceptual independence. The other method for using GRT to test assumptions about perceptual processing, which is arguably more popular, is to compute certain summary statistics from the empirical confusion matrix and then to check whether these satisfy certain conditions that are characteristic of perceptual separability or
multidimensional signal detection theory
17
independence. Because these two methods are so different, we will discuss each in turn. It is important to note however, that regardless of which method is used, there are certain nonidentifiabilities in the GRT model that could limit the conclusions that are possible to draw from any such analyses (e.g., Menneer, Wenger, & Blaha, 2010; Silbert & Thomas, 2013). The problems are most severe when GRT is applied to 2 × 2 identification data (i.e., when the stimuli are A1 B1 , A1 B2 , A2 B1 , and A2 B2 ). For example, Silbert and Thomas (2013) showed that in 2 × 2 applications where there are two linear decision bounds that do not satisfy decisional separability, there always exists an alternative model that makes the exact same empirical predictions and satisfies decisional separability (and these two models are related by an affine transformation). Thus, decisional separability is not testable with standard applications of GRT to 2 × 2 identification data (nor can the slopes of the decision bounds be uniquely estimated). For several reasons, however, these nonidentifiabilities are not catastrophic. First, the problems don’t generally exist with 3 × 3 or larger identification tasks. In the 3 × 3 case the GRT model with linear bounds requires at least 4 decision bounds to divide the perceptual space into 9 response regions (e.g., in a tic-tac-toe configuration). Typically, two will have a generally vertical orientation and two will have a generally horizontal orientation. In this case, there is no affine transformation that guarantees decisional separability except in the special case where the two vertical-tending bounds are parallel and the two horizontal-tending bounds are parallel (because parallel lines remain parallel after affine transformations). Thus, in 3 × 3 (or higher) designs, decisional separability is typically identifiable and testable. Second, there are simple experimental manipulations that can be added to the basic 2 × 2 identification experiment to test for decisional separability. In particular, switching the locations of the response keys is known to interfere with performance if decisional separability fails but not if decisional separability holds (Maddox, Glass, O’Brien, Filoteo, & Ashby, 2010; for more information on this, see the section later entitled “Neural Implementations of GRT”). Thus, one could add 100 extra trials to the end of a 2 × 2 identification experiment where the response key locations are randomly interchanged (and participants are informed of this change). If accuracy drops 18
significantly during this period, then decisional separability can be rejected, whereas if accuracy is unaffected then decisional separability is supported. Third, one could analyze the 2 × 2 data using the newly developed GRT model with individual differences (GRT-wIND; Soto, Vucovich, Musgrave, & Ashby, in press), which was patterned after the INDSCAL model of multidimensional scaling (Carroll & Chang, 1970). GRT-wIND is fit to the data from all individuals simultaneously. All participants are assumed to share the same group perceptual distributions, but different participants are allowed different linear bounds and they are assumed to allocate different amounts of attention to each perceptual dimension. The model does not suffer from the identifiability problems identified by Silbert and Thomas (2013), even in the 2 × 2 case, because with different linear bounds for each participant there is no affine transformation that simultaneously makes all these bounds satisfy decisional separability.
Fitting the GRT Model to Identification Data computing the likelihood function When the full GRT model is fit to identification data, the best-fitting values of all free parameters must be found. Ideally, this is done via the method of maximum likelihood—that is, numerical values of all parameters are found that maximize the likelihood of the data given the model. Let S1 , S2 , . . . , Sn denote the n stimuli in an identification experiment and let R1 , R2 , . . . , Rn denote the n responses. Let rij denote the frequency with which the subject responded Rj on trials when stimulus Si was presented. Thus, rij is the entry in row i and column j of the confusion matrix. Note that the rij are random variables. The entries in each row have a multinomial distribution. In particular, if P(Rj |Si ) is the true probability that response Rj is given on trials when stimulus Si is presented, then the probability of observing the response frequencies ri1 , ri2 , . . . , rin in row i equals ni ! P(R1 |Si )ri1 P(R2 |Si )ri2 · · · P(Rn |Si )rin ri1 !ri2 ! · · · rin ! (6) where ni is the total number of times that stimulus Si was presented during the course of the experiment. The probability or joint likelihood of observing the entire confusion matrix is the product
elementary cognitive mechanisms
of the probabilities of observing each row; that is, L=
n
n
ni P(Rj |Si )rij n r ij i=1 j=1 j=1
(7)
General recognition theory models predict that P(Rj |Si ) has a specific form. Specifically, they predict that P(Rj |Si ) is the volume in the Rj response region under the multivariate distribution of perceptual effects elicited when stimulus Si is presented. This requires computing a multiple integral. The maximum likelihood estimators of the GRT model parameters are those numerical values of each parameter that maximize L. Note that the first term in Eq. 7 does not depend on the values of any model parameters. Rather it only depends on the data. Thus, the parameter values that maximize the second term also maximize the whole expression. For this reason, the first term can be ignored during the maximization process. Another common practice is to take logs of both sides of Eq. 7. Parameter values that maximize L will also maximize any monotonic function of L (and log is a monotonic transformation). So, the standard approach is to find values of the free parameters that maximize n n
rij log P(Rj |Si )
(8)
i=1 j=1
estimating the parameters In the case of the multivariate normal model, the predicted probability P(Rj |Si ) in Eq. 8 equals the volume under the multivariate normal pdf that describes the subject’s perceptual experiences on trials when stimulus Si is presented over the response region associated with response Rj . To estimate the best-fitting parameter values using a standard minimization routine, such integrals must be evaluated many times. If decisional separability is assumed, then the problem simplifies considerably. For example, under these conditions, Wickens (1992) derived the first and second derivatives necessary to quickly estimate parameters of the model using the Newton-Raphson method. Other methods must be used for more general models that do not assume decisional separability. Ennis and Ashby (2003) proposed an efficient algorithm for evaluating the integrals that arise when fitting any GRT model. This algorithm allows the parameters of virtually any GRT model to be estimated via standard minimization software. The remainder of this section describes this method. The left side of Figure 2.2 shows a contour of equal likelihood from the bivariate normal
distribution that describes the perceptual effects of stimulus Si , and the solid lines denote two possible decision bounds in this hypothetical task. In Figure 2.2 the bounds are linear, but the method works for any number of bounds that have any parametric form. The shaded region is the Rj response region. Thus, according to GRT, computing P(Rj |Si ) is equivalent to computing the volume under the Si perceptual distribution in the Rj response region. This volume is indicated by the shaded region in the figure. First note that any linear bound can be written in discriminant function form as h(x1 , x2 ) = h(x) = b x + c = 0
(9)
where (in the bivariate case) x and b are the vectors
x1 x= and b = b1 b2 x2 and c is a constant. The discriminant function form of any decision bound has the property that positive values are obtained if any point on one side of the bound is inserted into the function, and negative values are obtained if any point on the opposite side is inserted. So, for example, in Figure 2.2, the constants b1 , b2 , and c can be selected so that h1 (x) > 0 for any point x above the h1 bound and h1 (x) < 0 for any point below the bound. Similarly, for the h2 bound, the constants can be selected so that h2 (x) > 0 for any point to the right of the bound and h2 (x) < 0 for any point to the left. Note that under these conditions, the Rj response region is defined as the set of all x such that h1 (x) > 0 and h2 (x) > 0. Therefore, if we denote the multivariate normal (mvn) pdf for stimulus Si as mvn(µi , i ), then P(Rj |Si ) = mvn(µi , i )dx1 dx2 (10) h1 (x) > 0; h2 (x) > 0 Ennis and Ashby (2003) showed how to quickly approximate integrals of this type. The basic idea is to transform the problem using a multivariate form of the well-known z transformation. Ennis and Ashby proposed using the Cholesky transformation. Any random vector x that has a multivariate normal distribution can always be rewritten as x = Pz + µ,
(11)
where µ is the mean vector of x, z is a random vector with a multivariate z distribution (i.e., a multivariate normal distribution with mean vector 0 and variance-covariance matrix equal to the identity
multidimensional signal detection theory
19
Cholesky
z2 h1*(z1, z2) = 0
h2(x1, x2) = 0
h1(x1, x2) = 0
Cholesky z1
x2 Respond Rj
h2*(z1, z2) = 0 Respond Rj
x1
Fig. 2.2 Schematic illustration of how numerical integration is performed in the multivariate normal GRT model via Cholesky factorization.
matrix I), and P is a lower triangular matrix such that PP = (i.e., the variance-covariance matrix of x). If x is bivariate normal then ⎤ ⎡ σ1 0 (12) P = ⎣ cov12 cov2 ⎦ σ22 − σ 212 σ1 1
The Cholesky transformation is linear (see Eq. 11), so linear bounds in x space are transformed to linear bounds in z space. In particular, hk (x) = b x + c = 0 becomes hk (Pz + µ) = b (Pz + µ) + c = 0 or equivalently hk ∗ (z) = (b P)z + (b µ + c) = 0
(13)
Thus, in this way we can transform the Eq. 10 integral to P(Rj |Si ) = mvn(µi , i )dx1 dx2 h1 (x) > 0; h2 (x) > 0 =
mvn(0, I)dz1 dz2
(14)
h1∗ (z) > 0; h2∗ (z) > 0 The right and left panels of Figure 2.2 illustrate these two integrals. The key to evaluating the second of these integrals quickly is to preload zvalues that are centered in equal area intervals. In Figure 2.2 each gray point in the right panel has a z 1 coordinate that is the center1 of an interval with area 0.10 under the z distribution (since there 20
are 10 points). Taking the Cartesian product of these 10 points produces a table of 100 ordered pairs (z 1 , z 2 ) that are each the center of a rectangle with volume 0.01 (i.e., 0.10 × 0.10) under the bivariate z distribution. Given such a table, the Eq. 14 integral is evaluated by stepping through all (z 1 , z 2 ) points in the table. Each point is substituted into Eq. 13 for k = 1 and 2 and the signs of h1 ∗ (z 1 , z 2 ) and h2 ∗ (z 1 , z 2 ) are determined. If h1 ∗ (z 1 , z 2 ) > 0 and h2 ∗ (z 1 , z 2 ) > 0 then the Eq. 14 integral is incremented by 0.01. If either or both of these signs are negative, then the value of the integral is unchanged. So the value of the integral is approximately equal to the number of (z 1 , z 2 ) points that are in the Rj response region divided by the total number of (z 1 , z 2 ) points in the table. Figure 2.2 shows a 10 × 10 grid of (z 1 , z 2 ) points, but better results can be expected from a grid with higher resolution. We have had success with a 100 × 100 grid, which should produce approximations to the integral that are accurate to within 0.0001 (Ashby, Waldron, Lee, & Berkman, 2001). evaluating goodness of fit As indicated before, one popular method for testing an assumption about perceptual or decisional processing is to fit two versions of a GRT model to the data using the procedures outlined in this section. In the first, restricted version of the model, a number of parameters are set to values that reflect the assumption being tested. For example, fixing all correlations to zero would test perceptual independence. In the second, unrestricted version of the model, the same parameters are free to vary. Once the restricted and unrestricted versions of the model have been fit, they can be compared through
elementary cognitive mechanisms
a likelihood ratio test: = −2(logLR − logLU ),
(15)
where LR and LU represent the likelihoods of the restricted and unrestricted models, respectively. Under the null hypothesis that the restricted model is correct, the statistic has a Chi-squared distribution with degrees of freedom equal to the difference in the number of parameters between the restricted and unrestricted models. If several non-nested models were fitted to the data, we would usually want to select the best candidate from this set. The likelihood ratio test cannot be used to select among such non-nested models. Instead, we can compute the Akaike information criterion (AIC, Akaike, 1974) or the Bayesian information criterion (BIC, Schwarz, 1978): AIC = −2logL + 2m
(16)
BIC = −2 logL + m log N
(17)
where m is the number of free parameters in the model and N is the number of data points being fit. When the sample size is small compared to the number of free parameters of the model, as in most applications of GRT, a correction factor equal to 2m(m + 1)/(n2 − m − 1) should be added to the AIC (see Burnham & Anderson, 2004). The best model is the one with the smallest AIC or BIC. Because an n × n confusion matrix has n(n − 1) degrees of freedom, the maximum number of free parameters that can be estimated from any confusion matrix is n(n − 1). The origin and unit of measurement on each perceptual dimension are arbitrary. Therefore, without loss of generality, the mean vector of one perceptual distribution can be set to 0, and all variances of that distribution can be set to 1.0. Therefore, if there are two perceptual dimensions and n stimuli, then the full GRT model has 5(n − 1) + 1 free distributional parameters (i.e., n – 1 stimuli have 5 free parameters—2 means, 2 variances, and a covariance—and the distribution with mean 0 and all variances set to 1 has 1 free parameter—a covariance). If linear bounds are assumed, then another 2 free parameters must be added for every bound (e.g., slope and intercept). With a factorial design (e.g., as when the stimulus set is A1 B1 , A1 B2 , A2 B1 , and A2 B2 ), there must be at least one bound on each dimension to separate each pair of consecutive component levels. So for stimuli A1 B1 , A1 B2 , A2 B1 , and A2 B2 , at least two bounds are required (e.g., see Figure 2.1). If, instead, there are 3 levels of each component,
then at least 4 bounds are required. The confusion matrix from a 2 × 2 factorial experiment has 12 degrees of freedom. The full model has more free parameters than this, so it cannot be fit to the data from this experiment. As a result, some restrictive assumptions are required. In a 3 × 3 factorial experiment, however, the confusion matrix has 72 degrees of freedom (9 × 8) and the full model has 49 free parameters (i.e., 41 distributional parameters and 8 decision bound parameters), so the full model can be fit to identification data when there are at least 3 levels of each stimulus dimension. For an alternative to the GRT identification model presented in this section, see Box 2.
Box 2 GRT Versus the Similarity-Choice Model The most widely known alternative identification model is the similarity-choice model (SCM; Luce, 1963; Shepard, 1957), which assumes that ηi j βj , P(Rj |Si ) = ηi k βk k
where ηij is the similarity between stimuli Si and Sj and βj is the bias toward response Rj . The SCM has had remarkable success. For many years, it was the standard against which competing models were compared. For example, in 1992 J. E. K. Smith summarized its performance by concluding that the SCM “has never had a serious competitor as a model of identification data. Even when it has provided a poor model of such data, other models have done even less well” (p. 199). Shortly thereafter, however, the GRT model ended this dominance, at least for identification data collected from experiments with stimuli that differ on only a couple of stimulus dimensions. In virtually every such comparison, the GRT model has provided a substantially better fit than the SCM, in many cases with fewer free parameters (Ashby et al., 2001). Even so, it is important to note that the SCM is still valuable, especially in the case of identification experiments in which the stimuli vary on many unknown stimulus dimensions.
multidimensional signal detection theory
21
X2
The Summary Statistics Approach The summary statistics approach (Ashby & Townsend, 1986; Kadlec and Townsend, 1992a, 1992b) draws inferences about perceptual independence, perceptual separability, and decisional separability by using summary statistics that are easily computed from a confusion matrix. Consider again the factorial identification experiment with 2 levels of 2 stimulus components. As before, we will denote the stimuli in this experiment as A1 B1 , A1 B2 , A2 B1 , and A2 B2 . In this case, it is convenient to denote the responses as a1 b1 , a1 b2 , a2 b1 , and a2 b2 . The summary statistics approach operates by computing certain summary statistics that are derived from the 4 × 4 confusion matrix that results from this experiment. The statistics are computed at either the macro- or micro-level of analysis. macro-analyses Macro-analyses draw conclusions about perceptual and decisional separability from changes in accuracy, sensitivity, and bias measures computed for one dimension across levels of a second dimension. One of the most widely used summary statistics in macro-analysis is marginal response invariance, which holds for a dimension when the probability of identifying the correct level of that dimension does not depend on the level of any irrelevant dimensions (Ashby & Townsend, 1986). For example, marginal response invariance requires that the probability of correctly identifying that component A is at level 1 is the same regardless of the level of component B, or in other words that P (a1 | A1 B1 ) = P (a1 | A1 B2 ) Now in an identification experiment, A1 can be correctly identified regardless of whether the level of B is correctly identified, and so P (a1 |A1 B1 ) = P (a1 b1 | A1 B1 ) + P (a1 b2 | A1 B1 ) For this reason, marginal response invariance holds on dimension X 1 if and only if P (ai b1 | Ai B1 ) + P (ai b2 | Ai B1 ) = P (ai b1 | Ai B2 ) + P (ai b2 | Ai B2 )
(18)
for both i = 1 and 2. Similarly, marginal response invariance holds on dimension X 2 if and only if 22
f12
f11
f22
f21
X1 g22(x1)
g12(x1)
g11(x1)
d 'AB2 |cAB2| g21(x1)
d 'AB1 |cAB1|
Hit False Alarm
Fig. 2.3 Diagram explaining the relation between macroanalytic summary statistics and the concepts of perceptual and decisional separability.
P (a1 bj | A1 Bj ) + P (a2 bj | A1 Bj ) = P (a1 bj | A2 Bj ) + P (a2 bj | A2 Bj )
(19)
for both j = 1 and 2. Marginal response invariance is closely related to perceptual and decisional separability. In fact, if dimension X 1 is perceptually and decisionally separable from dimension X 2 , then marginal response invariance must hold for X 1 (Ashby & Townsend, 1986). In the later section entitled “Extensions to Response Time,” we describe how an even stronger test is possible with a response time version of marginal response invariance. Figure 2.3 helps to understand intuitively why perceptual and decisional separability together imply marginal response invariance. The top of the figure shows the perceptual distributions of four stimuli that vary on two dimensions. Dimension X 1 is decisionally but not perceptually separable from dimension X 2 ; the distance between the means of the perceptual distributions along the X 1 axis is much greater for the top two stimuli than for the bottom two stimuli. The marginal distributions at the bottom of Figure 2.3 show that the proportion of correct responses, represented by the light-grey areas under the curves, is larger in the second level of X 2 than in the first level. The result would be similar if perceptual separability held and decisional separability failed, as would be the case for X 2
elementary cognitive mechanisms
if its decision bound was not perpendicular to its main axis. To test marginal response invariance in dimension X 1 , we estimate the various probabilities in Eq. 18 from the empirical confusion matrix that results from this identification experiment. Next, equality between the two sides of Eq. 18 is assessed via a standard statistical test. These computations are repeated for both levels of component A and if either of the two tests is significant, then we conclude that marginal response invariance fails, and, therefore, that either perceptual or decisional separability are violated. The left side of Eq. 18 equals P(ai |Ai B1 ) and the right side equals P(ai |Ai B2 ). These are the probabilities that component Ai is correctly identified and are analogous to “hit” rates in signal detection theory. To emphasize this relationship, we define the identification hit rate of component Ai on trials when stimulus Ai Bj is presented as Hai |Ai Bj = P ai Ai Bj = P ai b1 Ai Bj (20) + P ai b2 Ai Bj The analogous false alarm rates can be defined similarly. For example, Fa2 |A1 Bj = P a2 A1 Bj = P a2 b1 A1 Bj (21) + P a2 b2 A1 Bj In Figure 2.3, note that the dark grey areas in the marginal distributions equal Fa1 |A2 B2 (top) and Fa1 |A2 B1 (bottom). In signal detection theory, hit and false-alarm rates are used to measure stimulus discriminability (i.e., d ). We can use the identification analogues to compute marginal discriminabilities for each stimulus component (Thomas, 1999). For example, (22) dAB = −1 Ha2 |A2 Bj − −1 Fa2 |A1 Bj j where the function −1 is the inverse cumulative distribution function for the standard normal distribution. As shown in Figure 2.3, the value of dABj represents the standardized distance between the means of the perceptual distributions of stimuli A1 Bj and A2 Bj . If component A is perceptually separable from component B, then the marginal discriminabilities between the two levels of A must be the same for each level of B – that is, dAB1 = dAB2 (Kadlec & Townsend, 1992a, 1992b). Thus, if this equality fails, then perceptual separability is
violated. The equality between two d s can be tested using the following statistic (Marascuilo, 1970): d − d2 Z = 1 sd2 + sd2 1
(23)
2
where sd2 =
F (1 − F ) H (1 − H )
2 +
2 nn φ −1 (F ) ns φ −1 (H ) (24)
where φ is the standard normal probability density function, H and F are the hit and false-alarm rates associated with the relevant d , nn is the number of trials used to compute F, and ns is the number of trials used to compute H. Under the null hypothesis of equal d s, Z follows a standard normal distribution. Marginal hit and false-alarm rates can also be used to compute a marginal response criterion. Several measures of response criterion and bias have been proposed (see Chapter 2 of Macmillan & Creelman, 2005), but perhaps the most widely used criterion measure in recent years (due to Kadlec, 1999) is: (25) cAB = −1 Fa1 |A2 Bj j
As shown in Figure 2.3, this measure represents the placement of the decision-bound relative to the center of the A2 Bj distribution. If component A is perceptually separable from component B, but cAB1 = cAB2 , then decisional separability must have failed on dimension X 1 (Kadlec & Townsend, 1992a, 1992b). On the other hand, if perceptual separability is violated, then examining the marginal response criteria provides no information about decisional separability. To understand why this is the case, note that in Figure 2.3 the marginal c values are not equal, even though decisional separability holds. A failure of perceptual separability has affected measures of both discriminability and response criteria. To test the difference between two c values, the following test statistic can be used (Kadlec, 1999): c −c Z = 1 2 sc2 + sc2
(26)
F (1 − F )
2 . nn φ −1 (F )
(27)
1
2
where sc2 =
multidimensional signal detection theory
23
micro-analyses Macro-analyses focus on properties of the entire stimulus ensemble. In contrast, micro-analyses test assumptions about perceptual independence and decisional separability by examining summary statistics computed for only one or two stimuli. The most widely used test of perceptual independence is via sampling independence, which holds when the probability of reporting a combination of components P(ai bj ) equals the product of the probabilities of reporting each component alone, P(ai )P(bj ). For example, sampling independence holds for stimulus A1 B1 if and only if P(a1 b1 |A1 B1 ) = P(a1 |A1 B1 ) × P(b1 |A1 B1 ) = [P(a1 b1 |A1 B1 ) + P(a1 b2 |A1 B1 )] × [P(a1 b1 |A1 B1 ) + P(a2 b1 |A1 B1 )] (28) Sampling independence provides a strong test of perceptual independence if decisional separability holds. In fact, if decisional separability holds on both dimensions, then sampling independence holds if and only if perceptual independence holds (Ashby & Townsend, 1986). Figure 2.4A gives an intuitive illustration of this theoretical result. Two cases are presented in which decisional separability holds on both dimensions and the decision bounds cross at the mean of the perceptual distribution. In the distribution to the left, perceptual independence holds and it is easy to see that all four responses are equally likely. Thus, the volume of this bivariate normal distribution in response region R4 = a2 b2 is 0.25. It is also easy to see that half of each marginal distribution lies above its relevant decision criterion (i.e., the two shaded regions), so P(a2 ) = P(b2 ) = 0.5. As a result, sampling independence is satisfied since P(a2 b2 ) = P(a2 ) × P(b2 ). It turns out that this relation holds regardless of where the bounds are placed, as long as they remain perpendicular to the dimension that they divide. The distribution to the right of Figure 2.4A has the same variances as the previous distribution, and, therefore, the same marginal response proportions for a2 and b2 . However, in this case, the covariance is larger than zero and it is clear that P(a2 b2 ) > 0.25. Perceptual independence can also be assessed through discriminability and criterion measures computed for one dimension conditioned on the perceived value on the other dimension. Figure 2.4B shows the perceptual distributions of two stimuli that share the same level of component B (i.e., B1 ) and have the same perceptual mean on 24
dimension X 2 . The decision bound perpendicular to X 2 separates the perceptual plane into two regions: percepts falling in the upper region elicit an incorrect response on component B (i.e., a miss for B), whereas percepts falling in the lower region elicit a correct B response (i.e., a hit). The bottom of the figure shows the marginal distribution for each stimulus conditioned on whether B is a hit or a miss. When perceptual independence holds, as is the case for the stimulus to the left, these conditional distributions have the same mean. On the other hand, when perceptual independence does not hold, as is the case for the stimulus to the right, the conditional distributions have different means, which is reflected in different d and c values depending on whether there is a hit or a miss on B. If decisional separability holds, differences in the conditional d s and cs are evidence of violations of perceptual independence (Kadlec & Townsend, 1992a, 1992b). Conditional d and c values can be computed from hit and false alarm rates for two stimuli differing in one dimension, conditioned on the reported level of the second dimension. For example, for the pair A1 B1 and A2 B1 , conditioned on a hit on B, the hit rate for A is P(a1 b1 |A1 B1 ) and the false alarm rate is P(a1 b1 |A2 B1 ). Conditioned on a miss on B, the hit rate for A is P(a1 b2 |A1 B1 ) and the false alarm rate is P(a1 b2 |A2 B1 ). These values are used as input to Eqs. 22–27 to reach a statistical conclusion. Note that if perceptual independence and decisional separability both hold, then the tests based on sampling independence and equal conditional d and c should lead to the same conclusion. If only one of these two tests holds and the other fails, this indicates a violation of decisional separability (Kadlec & Townsend, 1992a, 1992b).
An Empirical Example In this section we show with a concrete example how to analyze the data from an identification experiment using GRT. We will first analyze the data by fitting GRT models to the identification confusion matrix, and then we will conduct summary statistics analyses on the same data. Finally, we will compare the results from the two separate analyses. Imagine that you are a researcher interested in how the age and gender of faces interact during face recognition. You run an experiment in which subjects must identify four stimuli, the combination of two levels of age (teen and adult) and two levels of gender (male and female). Each stimulus is presented 250 times, for a total of 1,000
elementary cognitive mechanisms
A R3
R4
R3
R4
R1
R2
R1
R2
B Hit Miss False Alarm
f21
f11
g11(X1 | B miss)
g21(X1 | B miss)
' miss d A|B |cA|B miss | g11(X1| B hit)
g21(X1| B hit)
d 'A|B hit |CA|B hit| Fig. 2.4 Diagram explaining the relation between micro-analytic summary statistics and the concepts of perceptual independence and decisional separability. Panel A focuses on sampling independence and Panel B on conditional signal detection measures.
trials in the whole experiment. The data to be analyzed are summarized in the confusion matrix displayed in Table 2.1. These data were generated by random sampling from the model shown in Figure 2.5A. The advantage of generating artificial data from this model is that we know in advance what conclusions should be reached by our analyses. For example, note that decisional separability holds in the Figure 2.5A model. Also, because the distance between the “male” and “female” distributions is larger for “adult” than for “teen,” gender is not perceptually separable from age. In contrast, the “adult” and “teen” marginal distributions are the same across levels of gender, so age is perceptually separable from gender. Finally, because all distributions show a positive correlation, perceptual independence is violated for all stimuli. A hierarchy of models were fit to the data in Table 2.1 using maximum likelihood estimation (as in Ashby et al., 2001; Thomas, 2001). Because there are only 12 degrees of freedom in the
data, some parameters were fixed for all models. Specifically, all variances were assumed to be equal to one and decisional separability was assumed for both dimensions. Figure 2.5C shows the hierarchy of models used for the analysis, together with the number of free parameters m for each of them. In this figure, PS stands for perceptual separability, PI for perceptual independence, DS for decisional separability and 1_RHO describes a model with a single correlation parameter for all distributions. Note that several other models could be tested, depending on specific research goals and hypotheses, or on the results from summary statistics analysis. The arrows in Figure 2.5C connect models that are nested within each other. The result of likelihood ratio tests comparing such nested models are displayed next to each arrow, with an asterisk representing significantly better fit for the more general model (lower in the hierarchy) and n.s. representing a nonsignificant difference in fit. Starting at the top of the hierarchy, it
multidimensional signal detection theory
25
Teen
Age Teen
Age
Adult
B
Adult
A
Male
Fermale
Male
Gender
Fermale Gender
C {PI, PS, DS} m=4 *
{PI, PS(Gender), DS} m=6
n.s.
n.s
n.s.
{1_RHO, DS} m=9
*
*
{1_RHO, PS(Age), DS} m=7
{1_RHO, PS(Gender), DS} m=7
*
{1_RHO, PS, DS} m=5
{PI, PS(Age), DS} m=6
*
{PI, DS} m=8
*
n.s.
n.s.
n.s.
*
{PS, DS} m=8
n.s.
{PS(Age), DS} m = 10
{PS(Gender), DS} m = 10
Fig. 2.5 Results of analyzing the data in Table 2.1 with GRT. Panel A shows the GRT model that was used to generate the data. Panel B shows the recovered model from the model fitting and selection process. Panel C shows the hierarchy of models used for the analysis and the number of free parameters (m) in each. PI stands for perceptual independence, PS for perceptual separability, DS for decisional separability and 1_RHO for a single correlation in all distributions.
Table 2.1. Data from a simulated identification experiment with four face stimuli, created by factorially combining two levels of gender (male and female) and two levels of age (teen and adult). Response Stimulus Male/Teen
26
Male/Teen
Female/Teen
Male/Adult
Female/Adult
140
36
34
40
Female/Teen
89
91
4
66
Male/Adult
85
5
90
70
Female/Adult
20
59
8
163
elementary cognitive mechanisms
Table 2.2. Results of the summary statistics analysis for the simulated Gender × Age identification experiment. Macroanalyses Marginal Response Invariance Test
Result
Conclusion
Equal P (Gender=Male) across all Ages Equal P (Gender=Female) across all Ages Equal P (Age=Teen) across all Genders Equal P (Age=Adult) across all Genders
z = −0.09, p>.1 z = −7.12, p<.001 z = −0.39, p>.1 z = −1.04, p>.1
Yes No Yes Yes
Test
d for level 1
d for level 2
Result
Equal d for Gender across all Ages Equal d for Age across all Genders
0.84 0.89
1.74 1.06
z = −5.09, p < .001 No z = −1.01, p >.1 Yes
Test
c for level 1
c for level 2
Result
Conclusion
Equal c for Gender across all Ages Equal c for Age across all Genders
−0.33 −0.36
−1.22 −0.48
z = 6.72, p < .001 z = 1.04, p >.1
No Yes
Marginal d Conclusion
Marginal c
Microanalyses Sampling Independence Stimulus
Response
Expected Observed Proportion Proportion
Result
Conclusion
Male/Teen Male/Teen Male/Teen Male/Teen Female/Teen Female/Teen Female/Teen Female/Teen Male/Adult Male/Adult Male/Adult Male/Adult Female/Adult Female/Adult Female/Adult Female/Adult
Male/Teen Female/Teen Male/Adult Female/Adult Male/Teen Female/Teen Male/Adult Female/Adult Male/Teen Female/Teen Male/Adult Female/Adult Male/Teen Female/Teen Male/Adult Female/Adult
0.49 0.21 0.21 0.09 0.27 0.45 0.23 0.39 0.25 0.11 0.21 0.09 0.04 0.28 0.10 0.79
0.56 0.14 0.14 0.16 0.36 0.36 0.02 0.26 0.34 0.02 0.36 0.28 0.08 0.24 0.03 0.65
z = 1.57, p > .1 z = −2.05, p < .05 z = −2.24, p < .05 z = 2.29, p < .05 z = 2.14, p < .05 z = −2.01, p < .05 z = −7.80, p < .001 z = −3.13, p < .01 z = 2.17, p < .05 z = −4.09, p < .001 z = 3.77, p < .001 z = 5.64, p < .001 z = 2.15, p < .05 z = −1.14, p > .1 z = −3.07, p < .01 z = −3.44, p < .001
Yes No No No No No No No No No No No No Yes No No
d |Hit
d |Miss
Result
Conclusion
0.84 1.83 0.89 0.83
1.48 2.26 1.44 1.15
z = −2.04, p < .05 z = −1.27, p > .1 z = −1.80, p > .05 z = −0.70, p > .1
No Yes Yes Yes
Conditional d Test Equal d Equal d Equal d Equal d
for Gender when Age=Teen for Gender when Age=Adult for Age when Gender=Male for Age when Gender=Female
multidimensional signal detection theory
27
Table 2.2. Continued Conditional c Test
c |Hit
c |Miss
Result
Conclusion
Equal c for Gender when Age=Teen Equal c for Gender when Age=Adult Equal c for Age when Gender=Male Equal c for Age when Gender=Female
−0.014 −1.68 −0.04 −0.63
−1.58 −0.66 −1.50 0.57
z = 6.03, p < .001 z = −4.50, p < .001 z = 6.05, p < .001 z = −4.46, p < .001
No No No No
is possible to find the best candidate models by following the arrows with an asterisk on them down the hierarchy. This leaves the following candidate models: {PS, DS}, {1_RHO, PS(Age), DS}, {1_RHO, DS}, and {PS(Gender), DS}. From this list, we eliminate {1_RHO, DS} because it does not fit significantly better than the more restricted model {1_RHO, PS(Age), DS}. We also eliminate {PS(Gender), DS} because it does not fit better than the more restricted model {PS, DS}. This leaves two candidate models that cannot be compared through a likelihood ratio test, because they are not nested: {PS, DS} and {1_RHO, PS(Age), DS}. To compare these two models, we can use the BIC or AIC goodness-of-fit measures introduced earlier. The smallest corrected AIC was found for the model {1_RHO, PS(Age), DS} (2,256.43, compared to 2,296.97 for its competitor). This leads to the conclusion that the model that fits these data best assumes perceptual separability of age from gender, violations of perceptual separability of gender from age, and violations of perceptual independence. This model is shown in Figure 2.5B, and it perfectly reproduces the most important features of the model that was used to generate the data. However, note that the quality of this fit depends strongly on the fact that the assumptions used for all the models in Figure 2.5C (decisional separability and all variances equal) are correct in the true model. This will not be the case in many applications, which is why it is always a good idea to complement the model-fitting results with an analysis of summary statistics. The results from the summary statistics analysis are shown in Table 2.2. The interested reader can directly compute all the values in this table from the data in the confusion matrix (Table 2.1). The macro-analytic tests indicate violations of marginal response invariance, and unequal marginal d and c values for the gender dimension, both of which suggest that gender is not perceptually separable from age. These results are uninformative about decisional separability. Marginal response 28
invariance, equal marginal d and c values all hold for the age dimension, providing some weak evidence for perceptual and decisional separability of age from gender. The micro-analytic tests show violations of sampling independence for all stimuli, and conditional c values that are significantly different for all stimulus pairs, suggesting possible violations of perceptual independence and decisional separability. Note that if we assumed decisional separability, as we did to fit models to the data, the results of the microanalytic tests would lead to the conclusion of failure of perceptual independence. Thus, the results of the model fitting and summary statistics analyses converge to similar conclusions, which is not uncommon for real applications of GRT. These conclusions turn out to be correct in our example, but note that several of them depend heavily on making correct assumptions about decisional separability and other features of the perceptual and decisional processes generating the observed data.
Extensions to Response Time There have been a number of extensions of GRT that allow the theory to account both for response accuracy and response time (RT). These have differed in the amount of extra theoretical structure that was added to the theory described earlier. One approach was to add the fewest and least controversial assumptions possible that would allow GRT to make RT predictions. The resulting model succeeds, but it offers no process interpretation of how a decision is reached on each trial. An alternative approach is to add enough theoretical structure to make RT predictions and to describe the perceptual and cognitive processes that generated that decision. We describe each of these approaches in turn.
The RT-Distance Hypothesis In standard univariate signal detection theory, the most common RT assumption is that RT
elementary cognitive mechanisms
decreases with the distance between the perceptual effect and the response criterion (Bindra, Donderi, & Nishisato, 1968; Bindra, Williams, & Wise, 1965; Emmerich, Gray, Watson, & Tanis, 1972; Smith, 1968). The obvious multivariate analog of this, which is known as the RT-distance hypothesis, assumes that RT decreases with the distance between the percept and the decision bound. Considerable experimental support for the RT-distance hypothesis has been reported in categorization experiments in which there is only one decision bound and where more observability is possible (Ashby, Boynton, & Lee, 1994; Maddox, Ashby, & Gottlob, 1998). Efforts to incorporate the RT-distance hypothesis into GRT have been limited to two-choice experimental paradigms, such as categorization or speeded classification, which can be modeled with a single decision bound. The most general form of the RT-distance hypothesis makes no assumptions about the parametric form of the function that relates RT and distance to bound. The only assumption is that this function is monotonically decreasing. Specific functional forms are sometimes assumed. Perhaps the most common choice is to assume that RT decreases exponentially with distance to bound (Maddox & Ashby, 1996; Murdock, 1985). An advantage of assuming a specific functional form is that it allows direct fitting to empirical RT distributions (Maddox & Ashby, 1996). Even without any parametric assumptions, however, monotonicity by itself is enough to derive some strong results. For example, consider a filtering task with stimuli A1 B1 , A1 B2 , A2 B1 , and A2 B2 , and two perceptual dimensions X 1 and X 2 , in which the subject’s task on each trial is to name the level of component A. Let P FA (RTi < t|Ai Bj ) denote the probability that the RT is less than or equal to some value t on trials of a filtering task when the subject correctly classified the level of component A. Given this, then the RT analog of marginal response invariance, referred to as marginal RT invariance, can be defined as (Ashby & Maddox, 1994) PFA (RT i ≤ t | Ai B1 ) = P FA (RT i ≤ t | Ai B2 ) (29) for i = 1 and 2 and for all t > 0. Now assume that the weak version of the RT-distance hypothesis holds (i.e., where no functional form for the RT-distance relationship is specified) and that decisional separability also holds. Then Ashby and Maddox (1994) showed that
perceptual separability holds if and only if marginal RT invariance holds for both correct and incorrect responses. Note that this is an if and only if result, which was not true for marginal response invariance. In particular, if decisional separability and marginal response invariance both hold, perceptual separability could still be violated. But if decisional separability, marginal RT invariance, and the RT-distance hypothesis all hold, then perceptual separability must be satisfied. The reason we get the stronger result with RTs is that marginal RT invariance requires that Eq. 29 holds for all values of t, whereas marginal response invariance only requires a single equality to hold. A similar strong result could be obtained with accuracy data if marginal response invariance were required to hold for all possible placements of the response criterion (i.e., the point where the vertical decision bound intersects the X 1 axis).
Process Models of RT At least three different process models have been proposed that account for both RT and accuracy within a GRT framework. Ashby (1989) proposed a stochastic interpretation of GRT that was instantiated in a discrete-time linear system. In effect, the model assumed that each stimulus component provides input into a set of parallel (and linear) mutually interacting perceptual channels. The channel outputs describe a point that moves through a multidimensional perceptual space during processing. With long exposure durations the percept settles into an equilibrium state, and under these conditions the model becomes equivalent to the static version of GRT. However, the model can also be used to make predictions in cases of short exposure durations and when the subject is operating under conditions of speed stress. In addition, this model makes it possible to relate properties like perceptual separability to network architecture. For example, a sufficient condition for perceptual separability to hold is that there is no crossing of the input lines and no crosstalk between channels. Townsend, Houpt, and Silbert (2012) considerably generalized the stochastic model proposed by Ashby (1989) by extending it to a broad class of parallel processing models. In particular, they considered (almost) any model in which processing on each stimulus dimension occurs in parallel and the stimulus is identified as soon as processing finishes on all dimensions. They began by extending definitions of key GRT concepts, such as perceptual
multidimensional signal detection theory
29
and decisional separability and perceptual independence, to this broad class of parallel models. Next, under the assumption that decisional separability holds, they developed many RT versions of the summary statistics tests considered earlier in this chapter. Ashby (2000) took a different approach. Rather than specify a processing architecture, he proposed that moment-by-moment fluctuations in the percept could be modeled via a continuous-time multivariate diffusion process. In two-choice tasks with one decision bound, a signed distance is computed to the decision bound at each point in time; that is, in one response region simple distance-to-bound is computed (which is always positive), but in the response region associated with the contrasting response the negative of distance to bound is computed. These values are then continuously integrated and this cumulative value drives a standard diffusion process with two absorbing barriers—one associated with each response. This stochastic version of GRT is more biologically plausible than the Ashby (1989) version (e.g., see Smith & Ratcliff, 2004) and it establishes links to the voluminous work on diffusion models of decision making.
Neural Implementations of GRT Of course, the perceptual and cognitive processes modeled by GRT are mediated by circuits in the brain. During the past decade or two, much has been learned about the architecture and functioning of these circuits. Perhaps most importantly, there is now overwhelming evidence that humans have multiple neuroanatomically and functionally distinct learning systems (Ashby & Maddox, 2005; Eichenbaum, & Cohen, 2004; Squire, 1992). And most relevant to GRT, the evidence is good that the default decision strategy of one of these systems is decisional separability. The most complete description of two of the most important learning systems is arguably provided by the COVIS theory of category learning (Ashby, Alfonso-Reese, Turken, & Waldron, 1998; Ashby, Paul, & Maddox, 2011). COVIS assumes separate rule-based and procedural-learning categorization systems that compete for access to response production. The rule-based system uses executive attention and working memory to select and test simple verbalizable hypotheses about category membership. The procedural system gradually associates categorization responses with regions of perceptual space via reinforcement learning. 30
COVIS assumes that rule-based categorization is mediated by a broad neural network that includes the prefrontal cortex, anterior cingulate, head of the caudate nucleus, and the hippocampus, whereas the key structures in the procedural-learning system are the striatum and the premotor cortex. Virtually all decision rules that satisfy decisional separability are easily verbalized. In fact, COVIS assumes that the rule-based system is constrained to use rules that satisfy decisional separability (at least piecewise). In contrast, the COVIS procedural system has no such constraints. Instead, it tends to learn decision strategies that approximate the optimal bound. As we have seen, decisional separability is optimal only under some special, restrictive conditions. Thus, as a good first approximation, one can assume that decisional separability holds if subjects use their rule-based system, and that decisional separability is likely to fail if subjects use their procedural system. A large literature establishes conditions that favor one system over the other. Critical features include the nature of the optimal decision bound, the instructions given to the subjects, and the nature and timing of the feedback, to name just a few (e.g., Ashby & Maddox, 2005, 2010). For example, Ashby et al. (2001) fit the full GRT identification model to data from two experiments. In both, 9 similar stimuli were constructed by factorially combining 3 levels of the same 2 stimulus components. Thus, in stimulus space, the nine stimuli had the same 3×3 grid configuration in both experiments. In the first experiment however, subjects were shown this configuration beforehand and the response keypad had the same 3 × 3 grid as the stimuli. In the second experiment, the subjects were not told that the stimuli fell into a grid. Instead, the 9 stimuli were randomly assigned responses from the first 9 letters of the alphabet. In the first experiment, where subjects knew about the grid structure, the bestfitting GRT model assumed decisional separability on both stimulus dimensions. In the second experiment, where subjects lacked this knowledge, the decision bounds of the best-fitting GRT model violated decisional separability. Thus, one interpretation of these results is that the instructions biased subjects to use their rule-based system in the first experiment and their procedural system in the second experiment. As we have consistently seen throughout this chapter, decisional separability greatly simplifies applications of GRT to behavioral data. Thus, researchers who want to increase the probability
elementary cognitive mechanisms
that their subjects use decision strategies that satisfy decisional separability should adopt experimental procedures that encourage subjects to use their rule-based learning system. For example, subjects should be told about the factorial nature of the stimuli, the response device should map onto this factorial structure in a natural way, working memory demands should be minimized (e.g., avoid dual tasking) to ensure that working memory capacity is available for explicit hypothesis testing (Waldron & Ashby, 2001), and the intertrial interval should be long enough so that subjects have sufficient time to process the meaning of the feedback (Maddox, Ashby, Ing, & Pickering, 2004).
Conclusions Multidimensional signal detection theory in general, and GRT in particular, make two fundamental assumptions, namely that every mental state is noisy and that every action requires a decision. When signal detection theory was first proposed, both of these assumptions were controversial. We now know, however, that every sensory, perceptual, or cognitive process must operate in the presence of inherent noise. There is inevitable noise in the stimulus (e.g., photon noise, variability in viewpoint) at the neural level and in secondary factors, such as attention and motivation. Furthermore, there is now overwhelming evidence that every volitional action requires a decision of some sort. In fact, these decisions are now being studied at the level of the single neuron (e.g., Shadlen & Newsome, 2001). Thus, multidimensional signal detection theory captures two fundamental features of almost all behaviors. Beyond these two assumptions, however, the theory is flexible enough to model a wide variety of decision processes and sensory and perceptual interactions. For these reasons, the popularity of multidimensional signal detection theory is likely to grow in the coming decades.
Acknowledgments Preparation of this chapter was supported in part by Award Number P01NS044393 from the National Institute of Neurological Disorders and Stroke, by grant FA9550-12-1-0355 from the Air Force Office of Scientific Research, and by support from the U.S. Army Research Office through the Institute for Collaborative Biotechnologies under grant W911NF-07-1-0072.
Note 1. This is true except for the endpoints. Since these endpoint intervals have infinite width, the endpoints are set at the z-value that has equal area to the right and left in that interval (0.05 in Figure 2.2).
Glossary Absorbing barriers: Barriers placed around a diffusion process that terminate the stochastic process upon first contact. In most cases there is one barrier for each response alternative. Affine transformation: A transformation from an n-dimensional space to an m-dimensional space of the form y = Ax + b, where A is an m × n matrix and b is a vector. Categorization experiment: An experiment in which the subject’s task is to assign the presented stimulus to the category to which it belongs. If there are n different stimuli then a categorization experiment must include fewer than n separate response alternatives. d : A measure of discriminability from signal detection theory, defined as the standardized distance between the means of the signal and noise perceptual distributions (i.e., the mean difference divided by the common standard deviation). Decision bound: The set of points separating regions of perceptual space associated with contrasting responses. Diffusion process: A stochastic process that models the trajectory of a microscopic particle suspended in a liquid and subject to random displacement because of collisions with other molecules. Euclidean space: The standard space taught in highschool geometry constructed from orthogonal axes of real numbers. Frequently, the n-dimensional Euclidean space is denoted by n . False Alarm: Incorrectly reporting the presence of a signal when no signal was presented. Hit: Correctly reporting the presence of a presented signal. Identification experiment: An experiment in which the subject’s task is to identify each stimulus uniquely. Thus, if there are n different stimuli, then there must be n separate response alternatives. Typically, on each trial, one stimulus is selected randomly and presented to the subject. The subject’s task is to choose the response alternative that is uniquely associated with the presented stimulus. Likelihood ratio: The ratio of the likelihoods associated with two possible outcomes. If the two trial types are equally likely, then accuracy is maximized when the subject gives one response if the likelihood ratio is greater than 1 and the other response if the likelihood ratio is less than 1. Multidimensional scaling: A statistical technique in which objects or stimuli are situated in a multidimensional space in such a way that objects that are judged or perceived as similar are placed close together. In most approaches, each object is represented as a single point and the space is constructed from some type of proximity data collected on the to-be-scaled objects. A common choice is to collect similarity ratings on all possible stimulus pairs.
multidimensional signal detection theory
31
Nested mathematical models: Two mathematical models are nested if one is a special case of the other in which the restricted model is obtained from the more general model by fixing one or more parameters to certain specific values. Nonidentifiable models: The case where two seemingly different models make identical predictions. Perceptual dimension: A range of perceived values of some psychologically primary component of a stimulus. Procedural learning: Learning that improves incrementally with practice and requires immediate feedback after each response. Prototypical examples include the learning of athletic skills and learning to play a musical instrument. Response bias: The tendency to favor one response alternative in the face of equivocal sensory information. When the frequencies of different trial types are equal, a response bias occurs in signal detection theory whenever the response criterion is set at any point for which the likelihood ratio is unequal to 1. Response criterion: In signal detection theory, this is the point on the sensory dimension that separates percepts that elicit one response (e.g., Yes) from percepts that elicit the contrasting response (e.g., No). Speeded classification: An experimental task in which the subject must quickly categorize the stimulus according to the level of a single stimulus dimension. A common example is the filtering task. Statistical decision theory: The statistical theory of optimal decision-making. Striatum: A major input structure within the basal ganglia that includes the caudate nucleus and the putamen.
References Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723. Amazeen, E. L., & DaSilva, F. (2005). Psychophysical test for the independence of perception and action. Journal of Experimental Psychology: Human Perception and Performance, 31, 170. Ashby, F. G. (1989). Stochastic general recognition theory. In D. Vickers & P. L. Smith (Eds.), Human information processing: Measures, mechanisms and models (pp. 435–457). Amsterdam, Netherlands: Elsevier Science Publishers. Ashby, F. G. (1992). Multidimensional models of categorization. In F. G. Ashby (Ed.), Multidimensional models of perception and cognition (pp. 449–483). Hillsdale, NJ: Erlbaum. Ashby, F. G. (2000). A stochastic version of general recognition theory. Journal of Mathematical Psychology, 44, 310–329. Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological Review, 105, 442–481. Ashby, F. G., Boynton, G., & Lee, W. W. (1994). Categorization response time with multidimensional stimuli. Perception & Psychophysics, 55, 11–27. Ashby, F. G., & Gott, R. E. (1988). Decision rules in the perception and categorization of multidimensional stimuli. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 33–53.
32
Ashby, F. G., & Lee, W. W. (1993). Perceptual variability as a fundamental axiom of perceptual science. In S.C. Masin (Ed.), Foundations of perceptual theory (pp. 369–399). Amsterdam, Netherlands: Elsevier Science. Ashby, F. G., & Maddox, W. T. (1994). A response time theory of separability and integrality in speeded classification. Journal of Mathematical Psychology, 38, 423–466. Ashby, F. G., & Maddox, W. T. (2005). Human category learning. Annual Review of Psychology, 56, 149–178. Ashby, F. G., & Maddox, W. T. (2010). Human category learning 2.0. Annals of the New York Academy of Sciences, 1224, 147–161. Ashby, F. G., Paul, E. J., & Maddox, W. T. (2011). COVIS. In E. M. Pothos & A. J. Wills (Eds.), Formal approaches in categorization (pp. 65–87). New York, NY: Cambridge University Press. Ashby, F. G., & Perrin, N. A. (1988). Toward a unified theory of similarity and recognition. Psychological Review, 95, 124– 150. Ashby, F. G., & Townsend, J. T. (1986). Varieties of perceptual independence. Psychological Review, 93, 154–179. Ashby, F. G., Waldron, E. M., Lee, W. W., & Berkman, A. (2001). Suboptimality in human categorization and identification. Journal of Experimental Psychology: General, 130, 77–96. Banks, W. P. (2000). Recognition and source memory as multivariate decision processes. Psychological Science, 11, 267–273. Bindra, D., Donderi, D. C., & Nishisato, S. (1968). Decision latencies of “same” and “different” judgments. Perception & Psychophysics, 3(2), 121–136. Bindra, D., Williams, J. A., & Wise, J. S. (1965). Judgments of sameness and difference: Experiments on decision time. Science, 150, 1625–1627. Blaha, L., Silbert, N., & Townsend, J. (2011). A general tecognition theory study of race adaptation. Journal of Vision, 11, 567–567. Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference understanding AIC and BIC in model selection. Sociological Methods & Research, 33, 261–304. Carroll, J. D., & Chang, J. J. (1970). Analysis of individual differences in multidimensional scaling via an N-way generalization of “Eckart-Young” decomposition. Psychometrika, 35, 283–319. Cohen, D. J. (1997). Visual detection and perceptual independence: Assessing color and form. Attention, Perception, & Psychophysics, 59, 623–635. DeCarlo, L. T. (2003). Source monitoring and multivariate signal detection theory, with a model for selection. Journal of Mathematical Psychology, 47, 292–303. Demeyer, M., Zaenen, P., & Wagemans, J. (2007). Lowlevel correlations between object properties and viewpoint can cause viewpoint-dependent object recognition. Spatial Vision, 20, 79–106. Eichenbaum, H., & Cohen, N. J. (2004). From conditioning to conscious recollection: Memory systems of the brain (No. 35). New York, NY: Oxford University Press. Emmerich, D. S., Gray, C. S., Watson, C. S., & Tanis, D. C. (1972). Response latency, confidence and ROCs in auditory signal detection. Perception & Psychophysics, 11, 65–72.
elementary cognitive mechanisms
Ennis, D. M., & Ashby, F. G. (2003). Fitting decision bound models to identification or categorization data. Unpublished manuscript. Available at http://www.psych.ucsb.edu/˜ashby/ cholesky.pdf Farris, C., Viken, R. J., & Treat, T. A. (2010). Perceived association between diagnostic and non-diagnostic cues of women’s sexual interest: General Recognition Theory predictors of risk for sexual coercion. Journal of mathematical psychology, 54, 137–149. Giordano, B. L., Visell, Y., Yao, H. Y., Hayward, V., Cooperstock, J. R., & McAdams, S. (2012). Identification of walked-upon materials in auditory, kinesthetic, haptic, and audio-haptic conditions. Journal of the Acoustical Society of America, 131, 4002–4012. Kadlec, H. (1999). MSDA_2: Updated version of software for multidimensional signal detection analyses. Behavior Research Methods, 31, 384–385. Kadlec, H., & Townsend, J. T. (1992a). Implications of marginal and conditional detection parameters for the separabilities and independence of perceptual dimensions. Journal of Mathematical Psychology, 36, 325–374. Kadlec, H., & Townsend, J. T. (1992b). Signal detection analyses of multidimensional interactions. In F. G. Ashby (Ed.), Multidimensional models of perception and cognition (pp. 181–231). Hillsdale, NJ: Erlbaum. Louw, S., Kappers, A. M., & Koenderink, J. J. (2002). Haptic discrimination of stimuli varying in amplitude and width. Experimental brain research, 146, 32–37. Luce, R. D. (1963). Detection and recognition. In R. D. Luce, R. R. Bush, & E. Galanter (Eds.), Handbook of mathematical psychology (Vol. 1, pp. 103–190). New York, NY: Wiley. Macmillan, N. A., & Creelman, D. (2005). Detection theory: A user’s guide (2nd ed.). Mahwah, NJ: Erlbaum. Maddox, W. T., & Ashby, F. G. (1993). Comparing decision bound and exemplar models of categorization. Perception & Psychophysics, 53, 49–70. Maddox, W. T., & Ashby, F. G. (1996). Perceptual separability, decisional separability, and the identification- speeded classification relationship. Journal of Experimental Psychology: Human Perception & Performance, 22, 795–817. Maddox, W. T., Ashby, F. G., & Gottlob, L. R. (1998). Response time distributions in multidimensional perceptual categorization. Perception & Psychophysics, 60, 620–637. Maddox, W. T., Ashby, F. G., Ing, A. D., & Pickering, A. D. (2004). Disrupting feedback processing interferes with rulebased but not information-integration category learning. Memory & Cognition, 32, 582–591. Maddox, W. T., Ashby, F. G., & Waldron, E. M. (2002). Multiple attention systems in perceptual categorization. Memory & Cognition, 30, 325–339. Maddox, W. T., Glass, B. D., O’Brien, J. B., Filoteo, J. V., & Ashby, F. G. (2010). Category label and response location shifts in category learning. Psychological Research, 74, 219–236. Marascuilo, L. (1970). Extensions of the significance test for oneparameter signal detection hypotheses. Psychometrika, 35, 237–243. Menneer, T., Wenger, M., & Blaha, L. (2010). Inferential challenges for General Recognition Theory: Mean-shift
integrality and perceptual configurality. Journal of Vision, 10, 1211–1211. Murdock, B. B. (1985). An analysis of the strength-latency relationship. Memory & Cognition, 13, 511–521. Peterson, W. W., Birdsall, T. G., & Fox, W. C. (1954). The theory of signal detectability. Transactions of the IRE Professional Group on Information Theory, PGIT-4, 171–212. Rotello, C. M., Macmillan, N. A., & Reeder, J. A. (2004). Sum-difference theory of remembering and knowing: a twodimensional signal-detection model. Psychological Review, 111, 588. Schwarz, G. E. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464. Shadlen, M. N., & Newsome, W. T. (2001). Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. Journal of Neurophysiology, 86, 1916–1936. Shepard, R. N. (1957). Stimulus and response generalization: A stochastic model relating generalization to distance in psychological space. Psychometrika, 22, 325–345. Silbert, N. H. (2012). Syllable structure and integration of voicing and manner of articulation information in labial consonant identification. Journal of the Acoustical Society of America, 131, 4076–4086. Silbert, N. H., & Thomas, R. D. (2013). Decisional separability, model identification, and statistical inference in the general recognition theory framework. Psychonomic Bulletin & Review, 20(1), 1–20. Silbert, N. H., Townsend, J. T., & Lentz, J. J. (2009). Independence and separability in the perception of complex nonspeech sounds. Attention, Perception, & Psychophysics, 71, 1900–1915. Smith, E. E. (1968). Choice reaction time: An analysis of the major theoretical positions. Psychological Bulletin, 69, 77– 110. Smith, J. E. K. (1992). Alternative biased choice models. Mathematical Social Sciences, 23, 199–219. Smith, P. L., & Ratcliff, R. (2004). Psychology and neurobiology of simple decisions. Trends in Neurosciences, 27, 161–168. Soto, F. A., Musgrave, R., Vucovich, L., & Ashby, F. G. (in press). General recognition theory with individual differences: A new method for examining perceptual and decisional interactions with an application to face perception. Psychonomic Bulletin & Review. Squire, L. R. (1992). Declarative and nondeclarative memory: Multiple brain systems supporting learning and memory. Journal of Cognitive Neuroscience, 4, 232–243. Swets, J. A., Tanner, W. P., Jr., & Birdsall, T. G. (1961). Decision processes in perception. Psychological Review, 68, 301–340. Tanner, W. P., Jr. (1956). Theory of recognition. Journal of the Acoustical Society of America, 30, 922–928. Thomas, R. D. (1999). Assessing sensitivity in a multidimensional space: Some problems and a definition of a general d . Psychonomic Bulletin & Review, 6, 224–238. Thomas, R. D. (2001). Perceptual interactions of facial dimensions in speeded classification and identification. Perception & Psychophysics, 63, 625–650. Townsend, J. T., Aisbett, J., Assadi, A., & Busemeyer, J. (2006). General recognition theory and methodology for dimensional independence on simple cognitive
multidimensional signal detection theory
33
manifolds. In H. Colonius & E. N. Dzhafarov (Eds.), Measurement and representation of sensations: Recent progress in psychophysical theory (pp. 203–242). Mahwah, NJ: Erlbaum. Townsend, J. T., Houpt, J. W., & Silbert, N. H. (2012). General recognition theory extended to include response times: Predictions for a class of parallel systems. Journal of Mathematical Psychology, 56, 476–494. Townsend, J. T., & Spencer-Smith, J. B. (2004). Two kinds of global perceptual separability and curvature. In C. Kaernbach, E. Schröger, & H. Müller (Eds.), Psychophysics beyond sensation: Laws and invariants of human cognition (pp. 89–109). Mahwah, NJ: Erlbaum.
34
Waldron, E. M., & Ashby, F. G. (2001). The effects of concurrent task interference on category learning: Evidence for multiple category learning systems. Psychonomic Bulletin & Review, 8, 168–176. Wenger, M. J., & Ingvalson, E. M. (2002). A decisional component of holistic encoding. Journal of Experimental Psychology: Learning, Memory, & Cognition, 28, 872–892. Wickens, T. D. (1992). Maximum-likelihood estimation of a multivariate Gaussian rating model with excluded data. Journal of Mathematical Psychology, 36, 213–234. Wickens, T. D. (2002). Elementary signal detection theory. New York, NY: Oxford University Press.
elementary cognitive mechanisms
CHAPTER
3
Modeling Simple Decisions and Applications Using a Diffusion Model
Roger Ratcliff and Philip Smith
Abstract
The diffusion model is one of the major sequential-sampling models for two-choice decision-making and choice response time in psychology. The model conceives of decision-making as a process in which noisy evidence is accumulated until one of two response criteria is reached and the associated response is made. The criteria represent the amount of evidence needed to make each decision and reflect the decision maker’s response biases and speed-accuracy trade-off settings. In this chapter we examine the application of the diffusion model in a variety of different settings. We discuss the optimality of the model and review its applications to a number of cognitive tasks, including perception,memory, and language tasks. We also consider its applications to normal and special populations, to the cognitive foundations of individual differences, to value-based decisions, and its role in understanding the neural basis of decision-making. Key Words: diffusion model, sequential-sampling, drift rate, choice, decision time,
accuracy, confidence, perceptual decision, memory decision, lexical decision
Diffusion Models for Rapid Decisions Over the last 30 or 40 years, there has been a steady development of models for simple decisionmaking that deal with both the accuracy of decisions and the time taken to make them. The models assume that decisions are made by accumulating noisy information to decision criteria, one criterion for each possible choice. The models successfully account for the probability that each choice is made and the response time (RT) distributions for correct responses and errors. The models are highly constrained by the behavior of these dependent variables. The most frequent applications of these models have been to tasks that require two-choice decisions that are made reasonably quickly, typically with mean RTs less than 1.0–2.0 s. This is fast enough that one can assume that the decisions come from a single decision process and not from multiple, sequential processes (anything much slower and the single-process assumption would be suspect).
The models have been applied successfully to many different tasks including perceptual, numerical, and memory tasks with a variety of subject populations, including older adults, children, dyslexics, and adults undergoing sleep deprivation, reduced blood sugar, or alcohol intoxication. An important feature of human decision-making is that the processing system is very flexible because humans can switch tasks, stimulus dimensions, and output modalities very quickly, from one trial to the next. There are many different kinds of decisions that can be made about any stimulus. If the stimulus is a letter string, decisions can be made about whether it is word or a nonword, whether it was studied earlier, whether the color is red or green, whether it is upper or lower case, and so on. Responses can be made in different modalities and in different ways in those modalities (for example, manually, vocally, or via eye movements). The same decision mechanism might operate for all these 35
tasks or the mechanism might be task and modality specific. For two-choice tasks, the assumption usually made is that all decision-related information, that is, all the information that comes from a stimulus or memory, is collapsed onto a single variable, called drift rate, that characterizes the discriminative or preference information in the stimulus. In some situations, subjects may be asked to make judgments based on more than one dimension that cannot be combined in this way. In such cases, the systems factorial methods of Townsend and colleagues (e.g., Townsend, 1972; see the review in Townsend & Wenger, 2004) may be able to be used to determine whether processing on the different dimensions is serial or parallel, or some hybrid of the two. In this chapter, we focus on one model of the class of sequential sampling models of evidence accumulation, the diffusion model (Ratcliff, 1978; Ratcliff & McKoon, 2008; Smith, 2000). A comparison of the diffusion model with other sequential-sampling models, such as the Poisson counter model (Townsend & Ashby, 1983), the Vickers accumulator model (Smith & Vickers, 1988; Vickers, 1970), and the leaky competing accumulator model (Usher & McClelland, 2001) can be found in Ratcliff and Smith (2004). In the diffusion model, for a two-choice task, noisy evidence accumulates from a starting point (Figure 3.1), toward one of two decision criteria or boundaries and the quality of the information that enters the decision process determines the rate of accumulation. Fitting the model to data provides estimates of drift rates, decision boundaries, and a parameter representing the duration of nondecision processes. The model’s ability to separate these components is one of its key contributions and places major constraints on its ability to explain data. Stimulus difficulty affects drift rate but not the criteria, and to a good approximation, speed-accuracy shifts are represented in the criteria, not drift rate. If difficulty varies, changes in drift rate alone must accommodate all the changes in performance, namely accuracy and the changes in the spreads and locations of the correct and error RT distributions. Likewise, changes in the criteria affect all the aspects of performance. In these ways, the model is tightly constrained by data. In a perceptual task, drift rate depends on the quality of the perceptual information from a stimulus; in a memory task, it depends on the quality of the match between a test item and memory. In 36
a brightness discrimination task, for example, if the accumulated evidence reaches the top boundary, a “bright” response is executed and a “dark” response would then correspond to the bottom boundary. Figure 3.1 shows an example, using a brightness discrimination task. Evidence accumulates from a stimulus to the “bright” boundary or to the “dark” boundary. The solid arrow shows the drift rate for a bright stimulus, the dashed arrow shows the drift rate for a less bright stimulus, and the dotted arrow shows the drift rate for a dark stimulus. The three paths in Figure 3.1 show three different outcomes, all with the same drift rate. Noise in the accumulation process produces errors when the accumulated evidence reaches the incorrect boundary and it produces variable RTs that form a distribution of RTs that has the shape of empirically obtained distributions. In the figure, one path leads to a fast correct decision, one to a slow correct decision, and one to an error. Most responses are reasonably fast, but there are slower ones that spread out the right-hand tails of the distributions (as in the distribution at the top of Figure 3.1). As drift rate changes from a large value to near zero, the mean of the RT distribution for both correct and error responses increases because the tail of the RT distribution spreads out. Figure 3.2 shows simulated individual RTs from the model as a function of drift rate, which is assumed to vary from trial to trial. The shortest RTs change little with drift rate, and so a fast response says nothing about the difficulty of the trial. The probability of obtaining a slow response from a high drift rate is very small (e.g., Figure 3.2) and so conditions with the slowest responses come from lower drift rates (see Ratcliff, Philiastides, & Sajda, 2009). Figure 3.1 shows the accumulation-of-evidence process. Besides this, there are processes that
Quality of Evidence from Perception or Memory
Bright
Dark
Fig. 3.1 The diffusion decision model with three simulated paths and three different drift rates.
elementary cognitive mechanisms
Individual Trial Response Time
correlation between RT and drift rate r = –0.336
1400 1200 1000 800 600 0.0
0.2
0.4
0.6
0.8
Drift Rate for the Trial Fig. 3.2 Plots of individual RTs as a function of drift rate for the trial. The parameters of the diffusion model were, boundary separation, a = 0.107, starting point z = 0.048, duration of processes other than the decision process, Ter = 0.48 s, SD in drift rate across trial η = 0.20, range in starting points sz = 0.02, range in nondecision time st = 0.18 s, drift rate v = 0.3.
encode stimuli, access memory, transform stimulus information into a decision-related variable that determines drift rate, and execute responses. These components of processing are combined into one “nondecision” component in the model, that has mean Ter . The total processing time for a decision is the sum of the time taken by the decision process and the time taken by the nondecision component. The boundaries of the decision process can be manipulated by instructions (“respond as quickly as possible” or “respond as accurately as possible”), differential rewards for the two choices, and the relative frequencies with which the two stimuli are presented in the experiment. Changes in instructions, rewards, or biases affect both RTs and accuracy but in the model, to a good approximation, the effects on RTs and accuracy are due to shifts in boundary settings alone, not drift rates or nondecision time. (However, if subjects are pushed very hard to go fast, then nondecision time and drift rates can be lower (e.g., Starns, Ratcliff, & McKoon, 2012.) Figure 3.3, left panel, shows boundaries moving in for speed relative to accuracy instructions and the right panel shows how subjects can be biased toward the top response versus the bottom response by moving decision criteria from the dashed line to the solid line settings. It is also possible (Figure 3.3 right panel) to adjust the zero point of drift rate (the drift rate criterion) to accommodate biases between the two responses (see Leite & Ratcliff, 2011; Ratcliff, 1985; Ratcliff & McKoon, 2008, Figure 3.3).
A problem with early random walk models, which were precursors to the diffusion model, was that they predicted equal correct and error RT distributions if the drift rates for two stimuli were equal in magnitude but opposite in sign (Laming, 1968; Stone, 1960; but see Link & Heath, 1975). This prediction is also made by the diffusion model in the absence of across-trial variability in model parameters. In fact, the patterns of the relative speed of correct versus error responses are as follows: with accuracy instructions and/or difficult tasks, errors are slower than correct responses, and with speed instructions and/or easy tasks, errors are faster than correct responses (Luce, 1986). In the diffusion model, the observed patterns of correct versus error RTs fall out naturally because there is trial-to-trial variability in drift rate and starting point (e.g., Ratcliff, 1981). Figure 3.4 illustrates how this mixing works with just two drift rates or two starting points instead of their full distributions. In Figure 3.4 left panel, the v1 drift rate produces high accuracy and fast responses, the v2 one lower accuracy and slow responses. The mixture of these produces errors slower than correct responses because 5% of the 400 ms process averaged with 20% of the 600 ms process gives a weighted mean of 560 ms, which is slower than the weighted mean for correct responses (491 ms). In Figure 3.4, right panel, the distributions to the left are for processes that start near the correct boundary (the dotted arrow shows the distance the process has to go to make an error—the larger the distance, the slower the response) and the distributions to the right are for processes that start further away from the correct boundary. Processes that start near to the correct boundary have few errors and those errors are slow, whereas processes that start further away have more errors and the errors are fast, leading to errors faster than correct responses. In practice, drift rate is assumed to be normally distributed from trial to trial and the starting point is uniformly distributed, but these specific functional forms are not critical (Ratcliff, 2013). Some researchers have argued that across-trial variability in the parameters is not needed (Palmer, Huk, & Shadlen, 2005; Usher & McClelland, 2001). However, it is unreasonable to assume that subjects can set their processing components to identical values on every equivalent trial of an experiment (i.e., ones with the same stimulus value). For drift rates, across-trial variability in drift rate is exactly analogous to variability in stimulus or memory strength in signal detection theory. Later
modeling simple decisions and applications
37
Speed/Accuracy tradeoff Boundary separation changes
Bias towards top boundary (dashed) changes to bias towards bottom boundary (solid)
Fig. 3.3 In the left panel, boundary separation alone changes between speed and accuracy instructions. In the right panel, the starting point varies with bias.
RT = 400 ms Pr = 0.95
Weighted Mean RT = 491ms
RT = 600 ms Pr = 0.80 a v1 z= a/2
v2
Pr = 0.95 RT = 350 ms a
Correct Responses
v
Pr = 0.80 RT = 450 ms
v
Weighted Mean RT = 396 ms
Correct Responses
z Error Responses
0 RT = 400 ms Pr = 0.05
RT = 600 ms Pr = 0.20
0 Pr = 0.20 RT = 350 ms
Weighted Mean RT = 560 ms
Error Responses Pr = 0.05 Weighted RT = 450 ms Mean RT = 370 ms
Fig. 3.4 Variability in drift rate and starting point and the effects on speed and accuracy. The left panel shows two process with drift rates v1 and v2 and the starting point halfway between the boundaries with correct and error RTs of 400 ms for v1 and of 600 ms for v2 . Averaging these two illustrates the effects of variability in drift rate across trials and in the illustration yields error responses slower than correct responses. The right panel shows processes with two starting points and drift rate v. Averaging processes with starting point 0.5a + 0.5 (high accuracy and short RTs) and starting point 0.5a − 0.5 (lower accuracy and short RTs) yields error responses faster than correct responses.
we describe an EEG study of perceptual decisionmaking that provides independent evidence for across-trial variability in drift rate and mention another that provides evidence for variability in starting point. It is important to understand that the diffusion model is highly falsifiable, not by mean RTs and accuracy values but by RT distributions. If empirical distributions are not right skewed, and do not shift and spread in exactly the right ways across experimental conditions, the model is falsified. Ratcliff (2002) generated sets of data with RT distributions that are plausible but never obtained in real experiments. For one set, the shapes and locations of the RT distributions were changed as a function of task difficulty, and for the other, the shapes and locations were changed as a function of speed versus accuracy instructions. For none of the resulting distributions was the model able to fit the data. In addition, the distributional predictions of the model are tested every time it is fit to empirical data. 38
expressions for accuracy and rt distributions For a two-boundary diffusion process with no across-trial variability in any of the parameters, the equation for accuracy, the proportion of responses terminating at the boundary at zero, is given by e−2va/s − e −2vz/s (1) P(v, a, z) = 2 e−2va/s − 1 (or 1−z/a if drift is zero), and the cumulative distribution of finishing times at the same boundary is given by 2
2
π s2 −vz e s2 a2 − 1 v2 + k2 π 2 s2 t ∞ a2 e 2 s2 2k sin kπz a 2 × v k2 π 2 s2 + k=1 s2 a2
G(t, v, a, z) = P(v, a, z) −
(2) where a is boundary separation (the top boundary is at a, the bottom boundary is at 0 and the
elementary cognitive mechanisms
distribution of finishing times is the distribution at the bottom boundary), z is the starting point, v is drift rate, and s is the SD in the normal distribution of within-trial variability (square root of the diffusion coefficient). These expressions can be derived as a solution of the partial differential equation for the first passage-time probability for the diffusion process (Feller, 1968). The results are described in detail in Ratcliff (1978) and Ratcliff and Smith (2004). Because Equation 2 contains an infinite sum, values of the RT density function need to be computed numerically. The series needs to be summed until it converges; this means that terms have to be added until subsequent terms become so small that they does not affect the total. This is complicated by the sine term, which can allow one value in the sum to be small, whereas the next one is not small. To deal with this practically, it is necessary to require that two or three successive terms are very small. The predictions from the model are obtained by integrating the results from Equations 1 and 2 over the distributions of the model’s across-trial variability parameters using numerical integration. In the standard model, drift rate is normally distributed across trials with SD η, the starting point is uniformly distributed with range sz , and nondecision time is uniformly distributed with range st . The predicted values are “exact” numerical predictions in the sense that they can be made as accurate as necessary (e.g., 0.1 ms or better) by using more and more steps in the infinite sum and more and more steps in the numerical integrations (packages that perform fitting are mentioned later). Alternative computational methods for obtaining predictions for diffusion models have been described by Smith (2000) and Diederich and Busemeyer (2003). The approach described by Smith uses integral equation methods derived from renewal theory. It was originally developed in mathematical biology to model the firing rates of integrate-and-fire neurons (Buonocore, Giorno, Nobile, & Ricciardi, 1990). The method is more computationally intensive than the infinite series approach of Equation 2, but has the advantage that it can be applied to processes in which the drift rates or decision criteria change over time or in which the accumulated information decays during the course of a trial. Smith (1995) and Smith and Ratcliff (2009) have proposed models in which drift rates depend on the outputs of visual and memory processes that change during a trial.
They obtained predictions for these models using the integral equation method. Diederich and Busemeyer (2003) proposed a matrix method for obtaining predictions for diffusion models. In their approach, a continuous-time, continuous-state diffusion process is approximated by a discrete-time, discrete-state birth-death process. The probability that the process takes a step up or down at each time point is characterized by a transition matrix whose entries express the rules by which the process evolves over time. By approximating the process in this way, the problem of obtaining RT distributions and response probabilities can be reduced to one of repeated matrix multiplication. This solution can be expensive computationally, but can be made more efficient by solving the associated algebraic eigenvalue problem, avoiding the need for repeated matrix multiplication. The method can also be applied to more complex problems that cannot be solved using the method of Equation 2 and has the advantage that it is very robust computationally. In some situations, it is important to generate predictions by simulation because simulated data can show the effects of all the sources of variability on a subject’s RTs and accuracy. The number of simulated observations can be increased sufficiently that the data approach the predictions that would be determined exactly from the numerical method. The expression for the update of evidence, x, on every time step t during the decision process, is determined by the drift rate, v, plus a noise term (Gaussian random variable, εi with SD σ ) to represent variability in processing: √ xi = vi t + σ ηi t
(3)
This equation provides the most straightforward method of simulating the diffusion process, but it is not the most efficient. Tuerlinckx, Maris, Ratcliff, & De Boeck (2001) examined four methods for simulating diffusion processes and found that a random walk approximation is better than using Equation 3. They also showed that a “rejection” method is even more efficient. However, if the process is nonstationary and complicated (e.g., with time varying drift rate, or boundaries that have some functional form) or there are several diffusion processes running to model multiple choice tasks, simulation is the simplest way to produce predictions, and the random walk approximation is likely the most efficient.
modeling simple decisions and applications
39
In fitting the diffusion model to data, accuracy and RT distributions for correct and error responses for all the conditions of the experiment must be simultaneously fit and the values of all of the components of processing estimated simultaneously. One commonly used fitting method uses quantiles of the RT distributions for correct and error responses for each condition (the 0.1, 0.3, 0.5, 0.7, and 0.9 quantile RTs). The model predicts the cumulative probability of a response at each RT quantile. Subtracting the cumulative probabilities for each successive quantile from the next higher quantile gives the proportion of responses between adjacent quantiles. For a chi-square computation, these are the expected values, to be compared to the observed proportions of responses between the quantiles (i.e., the proportions between 0.1, 0.3, 0.5, 0.7, and 0.9, are each 0.2, and the proportions below 0.1 and above 0.9 are both 0.1) multiplied by the number of observations. Summing over (Observed-Expected)2 /Expected for correct and error responses for each condition gives a single chi-square value that is minimized with a general SIMPLEX minimization routine. The parameter values for the model are adjusted by SIMPLEX until the minimum chi-square value is obtained. In any data set, there is the potential problem of outlier RTs, which could be fast (e.g., fast guesses) or slow (e.g., inattention). The quantile based method provides a good compromise that reduces the influence of outliers because the proportion of responses between the quantiles is used and extreme RTs within the bins have no influence on fitting. To additionally deal with outliers, a model of such processes is used in some model fitting approaches so that data is assumed to be a mixture of diffusion processes plus a small proportion of outliers. For details of the fitting methods for the standard diffusion model and modeling outliers, see Ratcliff and Tuerlinckx (2002). New methods for fitting the diffusion model have been developed recently and, over the last 6 or 7 years, fitting packages have been made available by Vandekerckhove and Tuerlinckx (2007) and Voss and Voss (2007). Also, Bayesian methods have been developed (Vandekerckhove, Tuerlinckx, & Lee, 2011) and a Bayesian package by Wiecki, Sofer & Frank (2013) has been made available. These Bayesian methods also implement hierarchical modeling schemes, in which model parameters for individual subjects are assumed to be random samples from population distributions that are specified within the model. The means and 40
variances of the population distributions, which are estimated in fitting, determine a range of probable values of drift rates and decision boundaries for individual subjects. Because all subjects are fit simultaneously using these methods, the parameters are constrained by the group parameters especially with low numbers of observations. The application of these hierarchical methods is in its infancy and some applications with large numbers of subjects, both simulated and real, that show their benefit over and above the more traditional methods are needed. To show how well the diffusion model fits data, we plot RT quantiles against the proportions for which the two responses are made. The top panel of Figure 3.5 shows a histogram for an RT distribution. The 0.1–0.9 quantile RTs and the 0.005 and 0.995 quantiles are shown on the x-axis. The rectangles represent equal areas of 0.2 probability mass between the 0.1–0.3, 0.3–0.5, etc. quantile RTs (and as can be seen, these represent the histogram reasonably well). These quantiles can be used to construct a quantile-probability plot by plotting the 0.1–0.9 quantile RTs vertically, as in the second panel of Figure 3.5, against the response proportion of that condition on the x-axis. Usually, correct responses are on the right of 0.5 and errors to the left (if there is no bias toward one or the other of the responses). Example RT distributions constructed from the equal area rectangles are also shown in grey. When there is a bias in starting point or when the two response categories are not symmetric (as in lexical decision and memory experiments), two quantile probability are needed, one for each response category. With quantile probability plots, changes in RT distribution locations and spread as a function of response proportion can be seen easily and compared with model fits. In the bottom panel of Figure 3.5, the 1–5 symbols are the data and the solid lines are the predictions from fits of the model to the data (with circles denoting the exact location of the predictions). As can be seen in this example, as response proportion changes from about 0.6 to near 1.0, the 0.1 quantile (leading edge) changes little, but the 0.9 quantile changes by as much as 400 ms. This is in line with the model predictions (e.g., Fig. 3.2). Also, as can be seen, error responses are slower than correct responses mainly in the spread, not in the leading edge location. Thus, quantile-probability plots allow all the important aspects of the data to be read from a single plot.
elementary cognitive mechanisms
Probability Density
Variants of the Standard Two-Choice Task 4
Up to this point, we have discussed how the diffusion model explains the results of experiments in which subjects respond with one of the two choices in their own time. The model has also been successfully applied to paradigms in which decision time is manipulated. Here we discuss three of these.
3 2 1
Quantiles
.9
error Pr = .3
1200 Reaction Time Quantiles (ms)
.005 .1 .3.5 .7
error Pr = .05
x
1000
.995
correct Pr = .7
Q
5 0.9
5
Q 0.9 correct Pr = .95
800
x
600
x x x
400
4 0.7
x
0.7
3 0.5
4 3
2 0.3
2
0.3
1 0.1
1
0.1
0.5
x x x x
200
Quantile Reaction Time (ms)
0.0
1200
0.2
0.4 0.6 0.8 Response Proportion
Dynamic pixel noise, brightness discrim. o o5 5o 5 5o o o5 5
1000 800 600 400
1.0
4o o4 33 o2 o o 2o 1 o 1o
o5
4o
o4
3o
3o
2o
o3 o2
2o
o3 o2
1o
o1
1o
o1
4o
o4
5o o4 4o o3 3o o2 2o o1 1o
Experiment 2, Ratcliff & Smith (2010) 0.0
0.2
0.4 0.6 0.8 Response Proportion
1.0
Fig. 3.5 The top panel shows a RT distribution overlaid with 0.1, 0.3, 0.5, 0.7, and 0.9 quantiles, where the area outside the .1 quantile ranges from 0.005 to 0.1 and the area outside the .9 quantile ranges from 0.9 to 0.995. The areas between each pair of middle quantiles are 0.2 and the areas below 0.1 and above 0.9 are 0.095. The quantile rectangles capture the main features of the RT distribution and therefore a reasonable summary of overall distribution shape. The middle panel shows quantile RTs for the 0.1, 0.3, 0.5 (median), 0.7, and 0.9 quantiles (stacked vertically) plotted against response proportion for each of the six conditions. Correct responses are plotted to the right, and error responses to the left. The bottom panel shows a quantile probability function from Ratcliff and Smith (2010, Experiment 2) with the numbers representing data and the lines representing predictions.
response signal and deadline tasks For response signal and deadline tasks, a signal is presented after the stimulus and a subject is required to respond as quickly as possible (in, say, 200–300 ms). For a deadline paradigm, the time between the stimulus and the signal is fixed across trials. For a response signal paradigm, the time varies from trial to trial (Reed, 1973; Schouten & Bekker, 1967; Wickelgren, 1977; Wickelgren, Corbett, & Dosher, 1980). With the deadline paradigm, subjects can adopt different strategies or criteria for different deadlines. This is not the case for the response signal paradigm in which processing can be assumed to be the same up to the signal. To apply the diffusion model to response signal data, Ratcliff (1988, 2006) assumed that there are response criteria just as for the standard twochoice task, and at some signal lag, responses come from a mixture of processes, those that have terminated at one or the other of the boundaries and those that have not. This is in accord with subjects’ intuitions that, at the long lags, the decision has already been made, the response has been chosen, and the subject is simply waiting for the signal. As the time between stimulus and signal decreases, a larger and larger proportion of processes will have failed to terminate. Differences among experimental conditions of different difficulties appear as differences in the proportions of accumulated information at the different lags. At the longest lags (2 or more seconds), all or almost all processes will have terminated. For nonterminated processes, there are two possibilities: that decisions are made on the basis of the partial information that has already been accumulated (Figure 3.6 top panel) or that they are simply guesses (Figure 3.6 middle panel). Ratcliff (2006) tested between these possibilities with a numerosity discrimination experiment (subjects decide whether the number of asterisks displayed on a PC monitor is greater than or less than 50). The same subjects participated in the response signal task and the standard task and examples of the response signal data and model fits are shown in Figure 3.7. When the model
modeling simple decisions and applications
41
Partial Information Model
Respond “Large”
Pr(“Large”) = sum of black areas at T1 Time Pr(“Small”) = sum of grey areas at T2
Respond “Small”
T2
T1
Guessing Model
Respond “Large”
Pr(“Large”) = sum of black areas at T1 & (1-P(guess))
Time Pr(“Small”) = sum of grey area at T2 & P(guess) T1
T2
Respond “Small”
Distribution of the density of non-terminated processes
Density of Processes
1.0 0.8 0.6 0.4 0.2 0.0 0.0
0.2
0.4
0.6
0.8
1.0
Time (s) Fig. 3.6 The top two panels shows two models for how the diffusion model accounts for response signal data. In the top panel, the proportion of “large” responses at time T1 is the sum of processes that have terminated at the “large” boundary (the black area above the boundary) and nonterminated processes (the black area still within the diffusion process), i.e., partial information. The middle panel shows the same assumption as the top panel except that if a process has not terminated, a guess is used instead of partial information. The bottom panel shows heat maps of simulated paths for the diffusion model. White corresponds to high path density and black to a low path density. For the diffusion model, the distribution to the right corresponds to the asymptotic distribution of path positions after about 0.2 seconds (i.e., the vertically oriented distributions in the top panel).
was fit to the two sets of data simultaneously, it fit well and it fit equally well for the two possibilities for nonterminated processes. In other words, “guessing” and partial information models could not be discriminated. 42
meyer, irwin, osman,& kounios, (1988) partial information paradigm This paradigm used a variant of the response signal task in which, on each trial, subjects responded either in the regular way unless a signal to respond
elementary cognitive mechanisms
1.0
0.4
1 2 3 1 4 123 24 35 5 45676 88 7 6
0.2
78
Response Proportion for “Large” Responses
0.8 0.6
0.0 1.0
1 2 1 3 2
0.8
1 23 4 45 3456 5 77 68 6 8 7 8
0.6 0.4 0.2
21 3
12 3
4
4
5
5
0
1 1 2 2 1 3
21 3
12 3
5
23 4 34
4
4
S1 6
6
6
7 8
7 8
78
21 3 4
213
213 4
5
5
4
6
45 5 56 6 87687 7 8
21
6
6 7 8
500 1000 1500 Response Signal Lag (ms)
7 8
21 3 4
S2 5
5
5
6 7 8
6 87
6 87
12 3
12 3
12 3
4
4
4
1 3
5
S3 7 8
0.0
123 4
132 4 324 765465 5 7 88 6 7 8
0
S4 5 6 87
5
5
6 87
876
500 1000 1500 Response Signal Lag (ms)
Fig. 3.7 Plots of response proportion as a function of response signal lag from a numerosity discrimination experiment (Ratcliff, 2006) for four subjects. The task required subjects to judged whether the number of dots in a 10x10 array was greater that 50 or less or equal to 50. The digits 1–8 (in reverse order) and the eight lines represent eight groupings of numbers of dots (e.g., 13–20, 21–30, 31–40, 41–50, 51–60, 61–70, 71–80, and 81–87 dots).
occurred, in which case they were to respond immediately. Thus, any trial could be a signal trial or a regular trial. Meyer et al. developed a method based on a race model that decomposes accuracy on the signal trials (at each signal lag) into a component from fast finishing regular trials and a component based on partial information. The predictions from the diffusion model matched those from Meyer et al. (1988). Results showed that partial information, in some tasks (see also Kounios, Osman, & Meyer, 1987) grew quickly and leveled off at about one-third the accuracy level of regular processes. Ratcliff (1988) examined the predictions of the diffusion model with the assumption that decisions on signal trials were a mixture of processes that terminated at a boundary and processes based on position in the decision process, that is, partial information. Therefore, if a process was above the starting point (i.e., the black area in the vertical distribution in the top panel of Figure 3.6), the decision corresponded to the choice at the upper boundary. Figure 3.6 bottom panel shows a heat map of the evolution of simulated diffusion processes. The map shows the density of processes as they begin at the starting point and spread out to the boundaries. The hotter the color (whiter), the more processes in that region. As time goes by, the color becomes
cooler because there are fewer and fewer processes that have not terminated. As in the top panel, the evolution of paths moves the mean position (the thick black line) from the starting point at 0.5 to a point a little above 0.6 by about 0.2 s. This produces an almost stationary distribution (the distribution to the right of the heat map), which gradually collapses over time (the two vertical distributions in the top panel of Figure 3.6). For the case in which partial information is used in the decision, the expression for the distribution of the positions x of decision processes at time t is given by: p(x, t) = e
v(x−z) s2
∞ 2 n=1
× sin
a
sin
nπ x a
e
nπz
− 12
a
v2 n2 π 2 s2 + 2 s2 a
t
(4)
where s2 is the diffusion coefficient, z is the starting point, a is the separation between the boundaries, and v is the drift rate. For model fitting, the expression in Equation 4 must be integrated over the normal distribution of drift rates and the uniform distribution of starting points to include variability in drift rate and starting point across trials. This can be accomplished with numerical integration using Gaussian quadrature. The series in
modeling simple decisions and applications
43
Equation 4 must be summed until it converges; this means that terms have to be added until subsequent terms become so small that they do not affect the total (i.e., the series has converged to within some criterion, e.g., 10−5 ). Then, to obtain the probability of choosing each response alternative, the proportion of processes between 0 and a/2 (for the negative alternative) and between a/2 and a (for the positive alternative) is calculated by integrating the expression for the density over position. time-varying processing Ratcliff (1980) examined two cases in which drift rate changes across the time course of processing. For one, drift rate changes discretely at one fixed time. Because there is an explicit expression for the distribution of evidence at that time, this distribution can be used as a starting distribution for a second diffusion process. If the time at which evidence changes is not a fixed time but has a distribution over time, this can be integrated over. This allows both response signal and regular RT tasks to be modeled. For another case, boundaries are removed completely and drift rate and the drift coefficient varied continuously over time. Only the first case has been used in modeling response signal data (as in Ratcliff, 1988, 2006). go/nogo task In the go/no go task, subjects are told to respond for one of the two choices but to make no response for the other choice. Withholding responses for one of the choices is similar to the response signal task in which responses must be held until the signal. Gomez, Ratcliff, and Perea (2007) proposed that there are two response boundaries for the go/no go task just as for the standard task, but subjects made a response only when accumulated evidence reaches the “go” boundary. Gomez et al. successfully fit the model simultaneously to data from the standard task and data from the go/no go task. They also tested a variant for which there was only one boundary, the “go” boundary, but this variant could not fit the data well. Application of the diffusion model simultaneously to the standard task and response signal task or to the standard task and go/no go tasks places powerful constraints on the model and, when it is successful, it offers new insights into the cognitive processes involved in these tasks. It also provides theoretical convergence between the three tasks, with two boundaries for all three tasks and withheld responses for the latter two. 44
The first conclusion is that applying models to multiple tasks simultaneously produces strong constraints on models that (if they successfully account for data) lead to new understanding of how the tasks are performed. In the context of the sequential sampling models discussed in this article, this approach yielded a new view of response signal performance: responses increase in accuracy over time mainly because the proportion of terminated processes increases and the increase in accuracy does not come entirely from the increasing availability of partial information. Moreover, versions of the models that provide quite good fits to the data from the standard RT and response signal tasks individually would not account for both sets of data simultaneously with parameters that were consistent across tasks.
Optimality In animal studies, performance has been described in terms of how close it comes to maximizing reward rate. This is part of a larger theme in neuroscience, which reprises the classical signal detection and sequential-sampling literatures, in which reward rate is used as a criterion for understanding whether neural computations approach optimality. For animals, how close performance is to optimal in terms of reward rate is a reasonable question to ask because animals are deprived of water or food and their overwhelming desire is to obtain them. Also they are trained for many sessions and so there is ample opportunity to optimize reward. However, when this kind of optimality is translated to human studies, the a priori reasonableness comes into question. This is because humans do not aim to get the most correct per unit time. Instead, they aim to get the most correct in the available time. If a student takes a 2-hour exam and obtains 60% correct in 1 hour, but another student gets 80% correct in 2 hours, the first has more correct per unit time, but the second would be more likely to pass the course. Bogacz, Brown, Moehlis, Holmes, & Cohen et al. (2006) performed extensive analyses of optimality and set the stage for analyses of data. They showed that optimality as defined by reward rate can be adjusted by changing boundary settings. If the boundaries are too far apart, subjects are accurate, but slow and so there are few correct per unit of time. If boundaries are too narrow, RT is short but accuracy is low and there are few correct responses per unit of time. Thus, there is a boundary setting that maximizes the number
elementary cognitive mechanisms
correct per unit of time and it is possible to test whether subjects set criteria near to this value. Starns and Ratcliff (2012) tested undergraduate subjects on a simple numerosity discrimination task in which different groups of subjects were tested at different levels of difficulty. They were tested in blocks of trials that had a fixed total duration for which they were instructed to get as many correct in the time allowed and in blocks of trials in which the number of trials was the same no matter how fast they went. Reward-rate optimality predicts that when difficulty increases, subjects should speed up and sacrifice accuracy.per unit time. Results showed subjects did the opposite, slowing down with increases in difficulty. This is the result we might expect from years of academic training to spend more time on difficult problems. Starns and Ratcliff (2010) analyzed several published data sets with young and older adults and found that young adults with accuracy feedback sometimes approached reward-rate optimality. But older adults rarely moved more than a few percent away from asymptotic accuracy. Young adults in the context of psychology experiments (or perhaps practice with video games, some of which promote speed) will sometimes be able to optimize performance in terms of number correct per unit of time. In general, however, concerns about accuracy that have been trained for years appear to dominate.
Domains of Application One criterion for how well a model performs is whether it simply reiterates what is already known from traditional analyses. Here we describe a number of applications, some of which provide new insights into processing, individual differences and differences among subject groups are obtained. But in other cases, even when the obvious results are obtained the model integrates the three dependent variables, namely, accuracy and correct and error RT distributions, into a common theoretical framework that provides explanations of data that many hypothesis-testing approaches do not. Hypothesistesting approaches usually select only accuracy or only mean RT as the dependent variable. In some cases, the two variables tell the same empirical story, but in other cases, they are inconsistent. The model based approach helps to resolve such inconsistencies.
Perceptual Tasks Recently diffusion models have been applied to psychophysical discrimination tasks in which
stimuli are presented very briefly, often at low levels of contrast, sometimes with backward masks to limit iconic persistence. The focus has been to understand the perceptual processes involved in the computation of drift rates. Psychophysical paradigms have historically been used mainly with threshold or accuracy measures but recent studies have collected accuracy and RT data. Ratcliff and Rouder (2000) and Smith, Ratcliff, and Wolfgang (2004) found that the diffusion model provided a good account of accuracy and distributions of RT from tasks with brief backwardmasked stimuli. They compared the model with a constant drift rate from starting point to boundaries to the model with varying drift rate. Drift rates might be thought to decrease over time if they either tracked stimulus information or were governed by a decaying perceptual trace. However, there was no evidence in either study of increased skewness in the RT distributions or very slow error RTs at short stimulus durations as would have been expected if the decision process had been driven by a decaying perceptual trace. Instead, it appears that the information that drives the decision is relatively durable. The standard application of the model assumes that, at some point in time after stimulus encoding, the decision process turns on, and evidence is accumulated toward a decision. This time is assumed to be the same across conditions and drift rate is assumed to be at a constant values from the point the process turns on. The assumption of a constant drift rate could be relaxed: Ratcliff (2002) generated predicted accuracy and RT quantiles for several conditions under the assumption that drift rate ramped up from zero to a constant level over 50 ms. He fit the standard model to these predicted values and found that the model fit well with nondecision time increased by 25 ms and with starting point, and nondecision time variability increased. Thus, a ramped onset of drift rate over a small time range will be indistinguishable from an abrupt onset. Smith and Ratcliff (2009) developed a model, the integrated system model, that is a continuousflow model comprised of perceptual, memory, and decision processes operating in cascade. The perceptual encoding processes are linear filters (Watson, 1986) and the transient outputs of the filters are encoded in a durable form in visual shortterm memory (VSTM), which is under the control of spatial attention. The strength of the VSTM trace determines the drift rate for the diffusion
modeling simple decisions and applications
45
process and the moment-to-moment variations in trace strength act as a source of noise in the decision process. Because the VSTM trace in the model increases over time (i.e., drift rate is time varying), predictions for the model are obtained using the integral equation methods described previously (Smith, 2000). The model has successfully accounted for accuracy and RT distributions in tasks with brief backward-masked stimuli. The main area of application of the integrated system model has been to tasks in which spatial attention is manipulated by spatial cues. In many cuing tasks, in which a single well-localized stimulus is presented in an otherwise empty display, attention shortens RT but increases accuracy only when stimuli are masked (Smith, Ratcliff, & Wolfgang, 2004; Smith, Ellis, Sewell, & Wolfgang, 2010). The model assumes that attention increases the efficiency with which perceptual information is transferred to VSTM and that masks interrupt the process of VSTM trace formation before it is complete. These two processes interact to produce a cuing effect in accuracy only when stimuli are masked but an unconditional effect in RT. The model has successfully accounted for the distributions of RT and accuracy in attention tasks in which the timing of stimulus localization is manipulated via onset transients and localizing markers (Sewell & Smith, 2012). These studies have helped illuminate the way in which performance is determined by perceptual, memory, attention, and decision processes acting in concert. Diederich and Busemeyer (2006) also considered the effects of attention on decision-making in a diffusion-process framework, studying decisions about multi-attribute stimuli for which it is plausible that people shift their attention sequentially from one attribute of a stimulus to the next. They assumed that some attributes would provide more information than others and modeled this successfully as a sequence of step changes in drift rate during the course of a trial.
Recognition Memory One of the early applications of the diffusion model was to recognition memory. In global memory models, a test item is matched against all memory in parallel, and the output is a single value of strength or familiarity (Gillund & Shiffrin, 1984; Hintzman, 1986; Murdock, 1982, and later, Dennis & Humphreys, 2001; McClelland & Chappell, 1998; Shiffrin & Steyvers, 1997). From this point of view, the diffusion model provides 46
a meeting point between the decision process and memory, specifically, the drift rate for a test item represents the degree of match between a test item and memory. In signal detection approaches to recognition memory, there has been considerable interest in the relative standard deviations (SDs) in strength between old and new test items, typically measured by confidence judgement paradigms. The common finding is that z-ROC functions (i.e., z-score transformed receiver operating characteristics) are approximately linear with a slope less than 1 (e.g., Ratcliff, Sheu, & Gronlund, 1992). There have been two interpretations of this finding. One is a single-process model that assumes the SD of memory strength is normally distributed, but the SD for old items is larger than that for new items. The other is a dual-process model in which the familiarity of old and new items comes from normal distributions with equal SDs but there is an additional recollection process (e.g., Yonelinas, 1997). In fits of the diffusion model to recognition memory data, it has been usually assumed that the SD in drift rate across trials is the same for studied and new items. Starns and Ratcliff (2014) performed an analysis of existing data sets that allowed the across-trial variability in drift rate to be different for studied and new items. They found that the across-trial variability in drift rate was larger (in about 66% of the cases for individual subjects) for studied items than for new items. It also turned out that the interpretations of the other model parameters did not change when variability was allowed to differ. The advantage of this analysis is that the relative variability of studied and new items was able to be determined from two-choice data and did not require confidence judgments.
Lexical Decision Much like recognition memory, a test item for lexical decision is matched against memory. The output is a value of how “wordlike” the item is. For sequential sampling models, proposals about how lexical items are accessed in memory must provide output values that, when mapped through a sequential sampling model, produce RTs and accuracy that fit data (Ratcliff, Gomez, & McKoon, 2004). (Note that there are other models that have integrated RT and accuracy with lexical processes, in particular, Norris, 2006). Often, lexical decision response time (RT) has been interpreted as a direct measure of the speed
elementary cognitive mechanisms
with which a word can be accessed in the lexicon. For example, some researchers have argued that the well-known effect of word frequency—shorter RTs for higher frequency words—demonstrates the greater accessibility of high frequency words (e.g., their order in a serial search, Forster, 1976; the resting levels of activation in units representing the words in a parallel processing system, Morton, 1969). However, other researchers have argued, as we do here, against a direct mapping from RT to accessibility. For example, Balota and Chumbley (1984) suggested that the effect of word frequency might be a by-product of the nature of the task itself, and not a manifestation of accessibility. In the research presented here, the diffusion model makes explicit how such a by-product might come about.
Semantic and Recognition Priming Effects For semantic priming, the task is usually a lexical decision. A target word is immediately preceded in a test list either by a word related to it (e.g., cat dog) or some other word (e.g. table dog). For recognition priming, the task is old/new recognition and a target word is immediately preceded by a word that was studied near to it in the list of items to be remembered or far from it. In the diffusion model, the simplest assumption about priming effects is that they result from higher drift rates for primed than unprimed items. It has been hypothesized that the difference in drift rates between primed and unprimed items arises from the familiarity of compound cues to memory (McKoon & Ratcliff, 1992; McNamara, 1992, 1994; Ratcliff & McKoon, 1988, 1994). The compound cue for an item is a multiplicative combination of the familiarity of the target word and the familiarity of the prime (see examples in Ratcliff & McKoon, 1988). If the prime and target words are related in memory, the combination produces a higher value of the joint familiarity than if they were not related. For primed items, the prime and target share associates in memory, the joint familiarity would be higher than if the prime and target do not share associates. This model was capable of explaining a number of phenomena in research on priming including the range of priming, the decay in priming, the onset of priming, and so on. McKoon and Ratcliff (2012) compared priming in word recognition to associative recognition. Subjects studied pairs of words and then performed either a single-word recognition task or
an associative recognition task (see also Ratcliff, Thapar, & McKoon, 2011). For the associative recognition task, subjects decided whether two words of a test pair had or had not appeared in the same pair at study. In the single-word task, some test words were immediately preceded in the test list by the other word of their studied pair (primed) and some by a word from a different pair (unprimed). Data from the two tasks were fit with the diffusion model and the results showed parallel behavior: the drift rates for associative recognition and those for priming were parallel across ages and IQ, indicating that they are based, at least to some degree, on the same information in memory.
Value-Based Judgments Busemeyer and Townsend (1993) developed a diffusion model called decision field theory to explain choices and decision times for decisions under uncertainty, and later Roe, Busemeyer, Townsend (2001) extended it to multi-alternative and multiattribute situations. According to the theory, at each moment in time, options are compared in terms of advantages/disadvantages with respect to an attribute, these evaluations are accumulated across time until a threshold is reached, and the first option to cross the threshold determines the choice that is made. The theory accounts for a number of findings that seem paradoxical from the perspective of rational choice theory. Usher and McClelland (2004) proposed another diffusion model to account for a similar range of findings. Milosavljevic, Malmaud, Huth, Koch, & Rangel (2010) examined several variants of diffusion models for value-based judgments. They found that the standard model with across-trial variability in model parameters provided a good account of data from their paradigm. More recently, Krajbich and Rangel (2011) have used a model similar in character to decision field theory. They examined value-based judgments for food items and had subjects choose which of two alternatives they preferred. They monitored eye fixations and in modeling, they assumed evidence was accumulated at a higher rate for the alternative fixated. Their model accounted for RTs and accuracy and for the influence of which of the two choices was fixated and for how long. Philiastides and Ratcliff (2013) examined valuebased judgments of consumer choices with brand names presented on some trials as well as the items for which the choices were made. When the quality of the brand name was in conflict with the perceived quality of the item, the probability of choosing the
modeling simple decisions and applications
47
item was lower then when they were consistent. Application of the diffusion model showed that the effect of the brand was to alter drift rate but none of the other parameters of the model. This means that the value of the stimulus and brand name were processed as a whole. Currently, there is a growing interest in the application of diffusion models to decision-making in marketing and economics, including neuroeconomics. Wide application of diffusion models in this domain are in their infancy, but the potential for theoretical advancement is great, as is demonstrated by these examples.
Aging The application of the diffusion model to studies of aging has been especially successful, producing a different view of the effects of aging on cognition than has been usual in aging research. The general finding in the literature has been that older adults are slower than young adults (but not necessarily less accurate) on most tasks, and this has been interpreted as a decline with age in all or almost all cognitive processes. However, application of the diffusion model showed that this is not correct (Ratcliff, Thapar, & McKoon, 2003, 2004, 2006, 2007; Ratcliff, Thapar, Gomez, & McKoon, 2004). For example, Ratcliff, Thapar, and McKoon (2010) tested old and young adults on numerosity discrimination, lexical decision, and recognition memory. What they found is that older adults had slower nondecision times and set their boundaries wider, but their drift rates were not lower than those of young adults. In contrast, in some tasks (associative recognition and letter discrimination), large declines in drift rate with age have been found (Ratcliff et al., 2011; Thapar et al., 2003).
Individual Differences The diffusion model has been used to examine individual differences. To do so requires that the SDs in model parameters from estimation variability are smaller than the SDs between subjects. In the aging studies described earlier, with about 45 minutes of data collection, individual differences in drift rates, boundary settings, and nondecision time were three to five times larger than the SDs of the model parameters. (See Ratcliff & Tuerlinckx, 2002, for tables of SDs in model parameters). Schmiedek, Oberauer, Wilhelm, Suβ, & Wittmann (2007) analyzed data from eight choice-RT tasks (including verbal, numerical, and 48
spatial tasks) from Oberauer, Suβ, Wilhelm, and Wittmann (2003). They found that drift rates in the diffusion model mapped onto working memory, speed of processing, and reasoning ability measures (each of these was measured by aggregated performance on several tasks). In aging studies by Ratcliff et al. (2010, 2011), IQs ranged from about 80 to about 140. Applying the model showed that drift rate varied with IQ (by as much as 2:1 for high versus lower IQ subjects) but boundary separation and nondecision time did not. This is the opposite of the pattern for aging. This dissociation provides strong support for the model because it extracts regularity from the three dependent variables (accuracy and correct and error RT distributions). Individual differences across tasks in model parameters provide strong evidence for common abilities across tasks. In the Ratcliff et al. (2010) study, in the lexical decision, item recognition, and associative recognition tasks, there are strong correlations across subjects in drift rate, and these correlated with IQ as measured by WAIS vocabulary and matrix reasoning. Also, boundary separation correlates across tasks as did nondecision time. These results show that the diffusion model extracts components of processing that show systematic individual differences across tasks. Consistent boundary setting across tasks is of special interest because boundary settings are optional, because they can be easily changed by instruction (e.g., go fast or be accurate). In most real-life situations, we rarely encounter more than single decisions on a particular stimulus class (except perhaps at Las Vegas or in psychology experiments). This means that there is little chance of adjusting decision criteria in real life because there is little extended experience with a task in which the decision maker can extract statistics from a long sequence of trials in which the structure of the trials does not change. The diffusion model assumes that a decision maker uses this decision mechanism across many tasks, and so we would expect to see correlations in boundary separation across tasks. This is a result that has been obtained whenever the comparison has been made.
Child Development A natural extension from the aging studies is to test children on similar tasks to those performed with older adults to trace the course of development within the model framework. Ratcliff, Love,
elementary cognitive mechanisms
Thompson, and Opfer (2012) tested several groups of children on a numerosity discrimination task and a lexical decision task. The results showed that relative to college age subjects, children’s drift rates were lower, boundary separation was larger, and nondecision time was longer. These differences were larger for younger relative to older children. In other laboratories, drift rates have been found to be lower for ADHD and dyslexic children relative to normal controls (ADHD, Mulder et al., 2010; dyslexia, Zeguers et al., 2011). These studies show that the diffusion model can be applied to data collected from children, a domain in which there has been relatively little research with decision models.
Clinical Applications In research on psychopathology and clinical populations, two-choice tasks are commonly used to investigate processing differences between patients and healthy controls. For highly anxious individuals, it is well-established that they show enhanced processing with threat-provoking materials, but this is found reliably only when there are two or more stimuli competing for processing resources, not one. However, when White, Ratcliff, Vasey, & McKoon (2010) applied the diffusion model to the RT and accuracy data from two-choice lexical decision task with single words that included threatening and control words, they found a consistent processing advantage for threatening words in high-anxious individuals, whereas traditional comparisons showed no significant differences Because the diffusion model makes use of both RT and accuracy data, it has more power to detect differences among subject populations than simply RT or accuracy alone. Studies of depression have had somewhat different patterns of results. Depressive symptoms are more closely linked with abnormal emotional processing with a negative emotional bias in clinical depression, even-handedness (i.e., no emotional bias) in dysphoria, and a positive emotional bias in nondepressed individuals. However, item recognition and lexical decision tasks often fail to produce significant results. White, Ratcliff, Vasey, & McKoon (2009) used the diffusion model to examine emotional processing in dysphoric (i.e., moderately high levels of depressive symptoms) and nondysphoric college students to examine differences in memory and lexical processing of positive and negative emotional words (which were presented among many neutral filler words). They found positive emotional bias in nondysphoric
subjects and even-handedness in dysphoric subjects in drift rates. As before, this pattern was not apparent with comparisons of reaction times or accuracy, consistent with previous null findings. One limitation of these studies and similar ones is that there may be relatively few materials with the right kinds of properties or structures (also in language processing experiments for example). The emotional word pools for the experiments only contained 30 words each. This left relatively few observations (especially for errors) to use in fitting the diffusion model, which would result in unreliable parameter estimates. To remedy this, the model was fit to all conditions simultaneously, including the neutral filler conditions which had hundreds of observations. The only parameter that was allowed to vary between the conditions was drift rate. Estimates for the other parameters were (e.g., nondecision time and boundary separation) largely determined by the filler conditions because the fitting method essentially weighted estimation of the parameters common to all conditions by the number of observations for each condition. Thus, the filler conditions largely determined all model parameters except the drift rates for the critical conditions, resulting in an increase in power. The results showed a bias for positive emotional words in the nondysphoric participants, but not in the dysphoric participants (White et al., 2009). This difference in emotional bias was not significant when the diffusion model was fit only to the emotional conditions with few observations, nor was it significant in comparisons of mean RT or accuracy. Another study examined the effects of aphasia in a lexical decision task. The impairments produce the exaggerated lexical decision reaction times typical of neurolinguistic patients. In diffusion model analyses, decision and nondecision processes were compromised, but the quality of the information upon which the decisions were based did not differ much from that of unimpaired subjects (Ratcliff, Perea, Colangelo, & Buchanan, 2004).
Manipulations of Homeostatic State Ratcliff and Van Dongen (2009) looked at effects of sleep deprivation with a numerosity discrimination task, van Ravenzwaaij, Dutilh, and Wagenmakers (2012) looked at the effects of alcohol consumption with a lexical decision task, and Geddes et al. (2010) looked at the effects of reduced blood sugar with a numerosity
modeling simple decisions and applications
49
discrimination task. Applying the model to all of these studies, the main effect was a reduced drift rate but with either small or no effect on boundary separation and nondecision time. These results show that the diffusion model is useful in providing interpretations of group differences among different subject populations. Furthermore, as noted earlier, the model can be used to examine individual differences (even with only 45 minutes of data collection for a task). This means that this modeling approach, when paired with the right tasks, may have a useful role to play in neuropsychological assessment.
Situations in Which the Standard Model Fails There are several cases in which the standard diffusion model fails to account for experimental data. These fall into two classes: one involves dynamic noise and categorical stimuli and the other involves conflict paradigms. For both, the main way the model fails is that there are cases for which the onset of the RT distribution (i.e., the leading edges) for one condition is delayed relative to the onset for other conditions. Ratcliff and Smith (2010) and Smith, Ratcliff, & Sewell (2014) tested letter discrimination, horizontal versus vertical bars discrimination, and Gabor patch orientation discrimination with stimuli degraded with either static noise or with dynamic noise. Noise was implemented by reversing the contrast polarity of some proportion of the pixels (randomly selected) for each of the letter, random bars, and Gabor patch stimuli. For dynamic noise, a different random sample of pixels was chosen on every frame of the display, whereas static noise used a single image with one random sample reversed. Dynamic noise and, to a lesser extent static noise, produced large shifts in the leading edges of the RT distribution. The shapes of the RT distributions were consistent with the model but increasing noise increased estimates of the nondecision time parameter Ter . This finding is inconsistent with the hypothesis that noise increases RTs simply by reducing the rate at which evidence accumulates in the decision process. Instead, it implies that noise delays the onset of the diffusion process. Smith, Ratcliff, and Sewell (2014) showed that shifts in onsets can be explained by Smith and Ratcliff ’s (2009) integrated system model. with the assumption that noise slows the process of forming a stable perceptual representation of the stimulus. In 50
the integrated system model, drift rate and diffusion noise grow in proportion to one another to an asymptote. Unlike the standard model, in which the onset of evidence accumulation is abrupt, the onset of evidence accumulation in the integrated system model is gradual, controlled by the growth of diffusion noise. Smith, Ratcliff & Sewell, 2014 showed that this model could explain the shifts in the onsets of RT distributions found by Ratcliff and Smith (2010). Smith, et al. (2014) also considered a second, release-from-inhibition model, which was motivated, in part, by physiological principles. They modeled release from inhibition using an Ornstein-Uhlenbeck (OU) diffusion process with a time-varying decay coefficient. In the OU process, information accumulation is opposed by a decay term that pulls the process back toward its starting point. The larger the decay, the harder it is for the process to accumulate enough information to reach a criterion and trigger a response. In the standard OU process, decay is proportional to the distance of the process from its starting point, but does not vary with time. Smith et al. (2014) assumed that decay was time-locked to the stimulus. At the start of the trial, before a perceptual representation of the stimulus is formed, the decay term is large and the process remains near its starting point with high probability. As stimulus information becomes available, the decay term progressively decreases, allowing information to accumulate in the same way as it does in the standard model. This model was also able to account for data like those reported by Ratcliff and Smith (2010). Because the inhibition process behaves somewhat like the standard model with variable starting point, the release-from-inhibition model was able to account for the fast errors found at high stimulus discriminability in dynamic noise tasks without the assumption of starting point variability. Ratcliff and Frank (2012) also found shifts in the leading edges of RT distributions in a reinforcement learning conflict experiment for which the stimuli were three pairs of letters (the same three throughout the experiment). On each trial, one of the pairs of letters were presented in random order and the subject had to choose and respond to one of the letters. One of the letters of the pair was reinforced more often than the other (in this case, reinforcement was simply a “correct” or “incorrect” message). After a training phase, on a small proportion of the trials, letters from different pairs were presented together. When the two letters
elementary cognitive mechanisms
were the highly reinforced members of the pairs, they were chosen nearly equally often and there was no slowing of the RT distribution. But when the letters that were reinforced with low probability were presented together, there was a delay in the leading edge of the RT distribution, an average delay of over 100 ms. This was explained in two ways, one in terms of the basal ganglia model of Frank (2006), and one in terms of the diffusion model. For the diffusion model to explain the data, a delay in the onset of the decision process could be used to produce good fits to the data. But this was, to some degree, a redescription of the empirical result. The basal ganglia model explained these conflict trials by an increase in threshold in the neural circuitry. This was linked to the diffusion model by showing that a transient increase in boundary separation was also capable of explaining the result (the delay in onset of the RT distribution). It turned out that an increase in boundary separation with an exponential decay mimics a delayed onset. White, Ratcliff and Starns (2011) also found leading edge shifts in a flanker task. In their experiment, a target angle bracket was presented that pointed in the direction of the correct response. On conflict trials, the target bracket was embedded in a string pointing the other way. Again, RT distributions could not be explained with only a difference in drift rates, but a model with drift rate changing over the time course of the decision, starting by being dominated by the flankers and then gradually focusing on the central symbol, was successful. All of these paradigms suggest that, in these conflict situations, drift rate is not stationary over time. It is necessary to go beyond the basic decision model and begin to integrate it with models of perceptual and cognitive processing.
Competing Two-Choice Models The diffusion model described to this point is one of a class of sequential sampling models that share many features. They all have given the same interpretations of effects of independent variables, which are the same across the models (e.g., Donkin, Brown, Heathcote, & Wagenmakers, 2011; Ratcliff, Thapar, Smith, & McKoon, 2005). This means, for example, that the effects of aging on model components are the same, whichever model is used. The leaky competing accumulator (LCA) model (Usher & McClelland, 2001) was developed as
c2
c1 Leak or Decay (k)
Criteria Leak or Decay (k)
Inhibition(β) Variable Start. Pt. Range sz
+ Gaussian Noise(σ)
1–v
Stimulus strength
+ Gaussian Noise(σ)
v
Fig. 3.8 An illustration of the leaky competing accumulator. The model includes an inhibition term (−kxj ) in which the increment to evidence in accumulator i is reduced as a function of activity in the other accumulator (xj ) and a decay term in which the increment to evidence is reduced as a function of activity in the accumulator (−βxi ). The decision criteria for the two accumulators are c1 and c2 , the accumulation rates are v1 and v2 (v1 +v2 =1), and there is variability in the starting points that is uniformly distributed across trials with range st . Variability in processing within a trial is normally distributed with standard deviation σ.
an alternative to the diffusion model. Part of the motivation was to implement neurobiological principles that the authors believed should be incorporated into RT models, especially mutual inhibition mechanisms and decay of information across time. In the LCA model, like the diffusion model, information is accumulated continuously over time. There are two accumulators, one for each response, as shown in Figure 3.8, and a response is made when the amount of information in one of the accumulators reaches its criterion amount. The rate of accumulation, the equivalent of drift rate in the diffusion model, is a combination of three components. The first is the input from the stimulus (v), with a different value for each experimental condition. If the input to one of the accumulators is v, the input to the other is 1−v so that the sum of the two rates is 1. The second component is decay in the amount of accumulated information, k, with size of decay growing as the amount of information in the accumulator grows, and the third is inhibition from the other accumulator, β, with the amount of inhibition growing as the amount of information in the other accumulator grows. If the amount of inhibition is large, the model exhibits features similar to the diffusion model because an increase in accumulated information for one of the response choices produces a decrease for the other choice.
modeling simple decisions and applications
51
Just as in the diffusion model, the accumulation of information is assumed to be variable over the course of a trial, with a normal distribution with standard deviation σ. Because of the decay and inhibition in the accumulation rates, the tails of RT distributions are longer than they would be if produced without these factors (cf., Smith & Vickers, 1988; Vickers, 1970, 1979; Vickers, Caudrey, Willson, 1971), which leads to good matches with the skewed shape of empirical distributions. The expression for the change in the amount of accumulated information at time t in accumulators i, is: ⎞ ⎛ βxj ⎠ t xi = ⎝vi − kxi − √ + σ ηi t
j =i
i = 1, 2
(5)
The amount of accumulated information is not allowed to take on values below zero, so if it is computed to be below zero, it is reset to zero. This is theoretically equivalent to constraining the diffusion process with a reflecting boundary at zero. The LCA model without across-trial variability in any of its components predicts errors slower than correct responses. To produce errors faster than correct responses and the crossover pattern such that errors are faster than correct responses for easy conditions and slower for difficult conditions, Usher and McClelland assumed variability in the accumulators’ starting points, just as is assumed in the diffusion model and by Laming (1968). In the diffusion model, moving a boundary position is equivalent to moving the starting point. Moving the starting point an amount y toward one boundary is the same as moving that boundary an amount y toward the starting point and the other boundary an amount y away from the starting point. In the LCA model, changing the starting point is not equivalent to changing a boundary position because decay is a function of the distance of the accumulated amount of evidence from zero. Increasing the starting point by an amount y increases decay by an amount proportional to y, but with the starting point at zero, reducing the boundary by y has no effect on decay. Usher and McClelland (2001) implemented variability in starting point by assuming rectangular distributions of the starting points with minimums at zero. No explicit solution is known for the pair of coupled equations in Eq. 5, when they are constrained by decision criteria and the requirement that the 52
accumulated information remain positive. Thus, as in Usher and McClelland (2001), predictions from the model are obtained by simulation. There have been several analyses of this model. Bogacz et al. (2006) showed that the model could be reduced to a single diffusion process if leak and inhibition were balanced and examined notions of optimality (but see van Ravenzwaaij, van der Maas, & Wagenmakers, 2012). The Linear Ballistic Accumulator (LBA, Brown and Heathcote, 2008) is similar to the LCA in that it uses two accumulators, but it has no within-trial variability, no decay, and no inhibition. The model assumes that the rate of evidence accumulation and the starting point for accumulation both vary randomly from trial to trial, but that the process of evidence accumulation itself is noise free. In essence, the model assumes that there is noise in the central nervous system on long, between-trial, time scales, but none on short, moment-to-moment, time scales that govern evidence accumulation within a trial. This assumption appears incompatible with the single-cell recording literature that has linked processes of evidence accumulation with neural firing rates in the oculomotor control system, because such neural spike trains are typically noisy. To reconcile these kinds of data with noiseless evidence accumulation requires an argument to the effect that individual neurons are noisy but the neural ensemble as a whole is effectively noise free. However, it is not clear that firing rates in weakly coupled networks of neurons exhibit the kinds of central limit theorem type properties that this argument requires (Zohary, Shadlen, & Newsome, 1994), and so the status central limit argument is unclear.
Multichoice Decision-Making and Confidence Judgments Recently, interest in the neuroscience domain in multichoice decision-making tasks has developed for visual search (Basso & Wurtz, 1998; Purcell et al., 2010) and motion discrimination (Niwa & Ditterich, 2008; Ditterich, 2010). In psychology, there have been investigations using generalizations of standard two-choice tasks (Leite & Ratcliff, 2010) and in absolute identification (Brown, Marley, Donkin, & Heathcote, 2008). In addition, confidence judgments in decision-making and memory tasks are multichoice decisions, and diffusion models are being applied in these domains (Pleskac & Busemeyer, 2010; Ratcliff & Starns, 2009, 2013; Van Zandt, 2002).
elementary cognitive mechanisms
It is clear that there is no simple way to extend the two-choice model to tasks with three or more choices. But models with racing accumulators can be extended. Some models with racing accumulators become standard diffusion models when the number of choices is reduced to two. Ratcliff and Starns (2013) proposed a model for confidence judgments in recognition memory tasks that uses a multiple-choice diffusion decision process with separate accumulators of evidence for each confidence choice. The accumulator that first reaches its decision boundary determines which choice is made. Ratcliff and Starns compared five algorithms for accumulating evidence and found that one of them produced choice proportions and full RT distributions for each choice that closely matched empirical data. With this algorithm, an increase in the evidence in one accumulator is accompanied by a decrease in the others with the total amount of evidence in the system being constant. Application of the model to the data from an earlier experiment (Ratcliff, McKoon, & Tindall, 1994) uncovered a relationship between the shapes of z-ROC functions and the behavior of RT distributions. For low-proportion choices, the RT distributions were shifted by as much as several hundred milliseconds relative to high proportion choices. This behavior and the shapes of z-ROC functions were both explained in the model by the behavior of the decision boundaries. For generality, Ratcliff and Starns (2013) also applied the decision model to a three-choice motion discrimination task in which one of the alternatives was the correct choice on only a low proportion of trials. As for the confidence judgment data, the RT distribution for the low probability alternative was shifted relative to the higher probability alternatives. The diffusion model with constant evidence accounted for the shift in the RT distribution better than a competing class of models. Research on multichoice decision making, including confidence judgments, is a growing industry but the constraints provided by RT distributions and response proportions for the different choices makes the modeling quite challenging.
One-Choice Decisions Relatively little work has been done recently on one-choice decisions. In these, there is only one key to press when a stimulus is detected. Ratcliff and Van Dongen (2011) tested a model that used a single diffusion process to represent the process of accumulating evidence. The main application was
to the psychomotor vigilance task (PVT) for which a millisecond timer is displayed on a computer screen and it starts counting up at intervals between 2 and 12 s after the subject’s last response. The subject’s task is to hit a key as quickly as possible to stop the timer. When the key is pressed, the counter is stopped, and the RT in milliseconds is displayed for 1 s. In single-choice decision-making tasks, the data are a distribution of RTs for hitting the response key. The one-choice diffusion model assumes the evidence begins accumulating on presentation of a stimulus until a decision criterion is hit, upon which, a response is initiated (Figure 3.9 illustrates the model). In the model, drift rate is assumed to vary from trial to trial. This relates it to the standard two-choice model, which makes this assumption to fit the relative speeds of correct and error responses. In application of the one-choice model to sleep deprivation data, across-trial variability in drift rate was needed to produce the long tails observed in the RT distributions. Ratcliff and Van Dongen (2011) fit the model to RT distributions and their hazard functions from experiments with the PVT with over 2000 observations per RT distribution per subject. With only changes in drift rate, they found that the model accounted for changes in the shape of RT distributions. In particular, changes in drift rate accounted for the change in hazard function shape moving from a high tail under no sleep deprivation to a low tail with sleep deprivation. They also fit data for which the PVT was tested every 2 hours for 36 hours of sleep deprivation and found that drift rate was closely related to an independent measure of alertness, which provides an external validation of the model.
Neuroscience One of the major advances in understanding decision making is in neuroscience applications using single cell recording in monkeys (and rats), human neuroscience including fMRI, EEG, and MEG. All these domains have had interactions between diffusion model theory and neuroscience measures. Hanes and Schall (1996) made the first connection between theory and single cell recording data, and this was taken up in work by Shadlen and colleagues (e.g., Gold and Shadlen, 2001). monkey neurophysiology In both psychology and neuroscience, theories of decision processes have been developed that
modeling simple decisions and applications
53
Across-trial drift rate distribution
Decision process starts
Stimulus onset
Decision process stops Response Td
Ta
0 v
Drift rate
Ter=Ta+Tb Time
Evidence criterion Drift rate (v) Starting point
a
0
Tb
Decision process:
Fig. 3.9 An illustration of the one-choice diffusion model. Evidence is accumulated at a drift rate v with SD across trials η, until a decision criterion at a is reached after time Td . Additional processing times include stimulus encoding time Ta and response output time Tb ; these sum to nondecision time Ter , which has uniform variability across trials with range st .
assume that evidence is gradually accumulated over time (Boucher, Palmeri, Logan, & Schall, 2007; Churchland, Kiani, & Shadlen, 2008; Ditterich, 2006; Gold & Shadlen, 2001, 2007; Grinband, Hirsch, & Ferrera, 2006; Hanes & Schall, 1996; Mazurek, Roitman, Ditterich, & Shadlen, 2003; Platt & Glimcher, 1999; Purcell et al., 2010; Ratcliff, Cherian, & Segraves, 2003; Ratcliff, Hasegawa, Hasegawa, Smith, & Segraves, 2007; Roitman & Shadlen, 2002; Shadlen & Newsome, 2001). In these studies, cells in the lateral intraparietal cortex (LIP), frontal eye field (FEF), and the superior colliculus (SC) exhibit behavior that corresponds to a gradual buildup in activity that matches the buildup in evidence in making simple perceptual decisions (see also Munoz & Wurtz, 1995; Basso & Wurtz, 1998). The neural populations that exhibit buildup behavior in LIP, FEF, and SC prior to a decision have been studied extensively. There is debate about where exactly the accumulation takes place, but it is clear that (at least) these three structures are part of a circuit that is involved in implementing the decision. These studies so far support the notion that there is a flow of information from LIP to FEF and then to SC prior to a decision. In modeling the neurobiology of the decision process, there are a number of models applied to a range of different tasks. They all have the common theme that they assume evidence is accumulated to a decision criterion, or boundary, and that accumulated evidence corresponds to activity in populations of neurons corresponding to the decision alternatives. The models considered here have been explicitly proposed as models of oculomotor decision making in monkeys or argued to describe the evidence accumulation process in humans or monkeys. The models fall into several
54
classes (Ratcliff & Smith, 2004; Smith & Ratcliff, 2004), including those that assume accumulation of a single evidence quantity taking on positive and negative values (Gold & Shadlen, 2000, 2001; Ratcliff, 1978; Ratcliff et al., 2003; Ratcliff, Van Zandt, & McKoon, 1999; Smith, 2000) and those that assume that evidence is accumulated in separate accumulators corresponding to separate decisions (Churchland, et al., 2008; Ditterich, 2006; Mazurek et al., 2003; Ratcliff et al., 2007; Usher & McClelland, 2001). In this latter class of models, accumulation can be independent in separate accumulators, or it can be interactive so that as evidence grows in one accumulator, it inhibits evidence accumulation in the other accumulator. The single accumulator model can be seen as implementing perfect inhibition because a positive increment toward one boundary is an increment away from the other boundary. The models with separate accumulators have the advantage because the two accumulators can be used to represent growth of activity in the populations of neurons corresponding to the two decisions. In the single diffusion process models, if the single process represented the aggregate activity in the two populations, then the growth of activity in the two populations would have to be perfectly negatively correlated. This is plausible if the resting activity level is relatively high in the neural populations (e.g., Roitman & Shadlen, 2002), but it is less plausible in populations in which the resting level is low (Hanes & Schall, 1996; Ratcliff et al., 2007). However, the two classes of models largely mimic each other at a behavioral level (Ratcliff, 2006; Ratcliff & Smith, 2004) and although the choice of models with racing diffusion processes seems to be superior in application in oculomotor
elementary cognitive mechanisms
responses in monkeys, this does not rule out the viability of the single accumulator model for human behavioral and neural data (Philiastides, Ratcliff, & Sajda, 2006; Ratcliff et al., 2009). Ratcliff et al. (2007; see also Ratcliff, Hasegawa, et al., 2011) applied a dual diffusion model to a brightness discrimination task. In the dual diffusion model, evidence for the two responses is accumulated by a pair of racing diffusion processes. In Ratcliff et al.’s model, there was competition at input (drift rates summed to a constant) but no inhibition (i.e., Figure 3.8 without the inhibition). Two rhesus monkeys were required to make a saccade to one of two peripheral choice targets based on the brightness of a central stimulus. Neurons in the deep layers of the SC exhibited a robust presaccadic activity when the stimulus specified a saccade toward a target within the neuron’s response field, and the magnitude of this activity was unaffected by level of difficulty. Activity following brightness stimuli specifying saccades to targets outside the response field was affected by task difficulty, increasing as the task became more difficult, and this modulation correlated with performance accuracy. The model fit the full complexity of the behavioral data, accuracy and RT distributions for correct and error responses, over a range of levels of difficulty. Using the parameters from the fits to behavioral data, simulated paths of the process were generated and these provided numerical predictions for the behavior of the firing rates in SC neurons that matched most but not all the effects in the data. Simulated paths from the model were compared to neuron activity. The assumption linking the paths to the neuron data is that firing rate is linearly related to position in the accumulation process; the nearer the boundary the decision process is, the higher the firing rate. The firing rate data show delayed availability of discriminative information for fast, intermediate, and slow decisions when activity is aligned on the stimulus and very small differences in discriminative information when activity is aligned on the saccade. The model produces exactly these patterns of results. The accumulation process is highly variable, allowing the process both to make errors, as is the case for the behavioral performance, and also to account for the firing rate results. Figure 3.10 shows sample results for the firing rate functions (black lines) and predicted firing rates (red lines). There have also been significant modeling efforts to relate models based on spiking neurons to
diffusion models (e.g., Deco, Rolls, Albantakis, & Ramo, 2013; Roxin & Ledberg, 2008; Wong and Wang, 2006). Smith (2010) made an explicit connection between diffusion processes at a macro behavioral level and shot noise processes at a slightly abstract neural level. Smith (2010) sought to show how diffusive information accumulation at a behavioral level could arise by aggregating neural firing rate processes. He modeled the representation of stimulus information at the neural level as the difference between excitatory and inhibitory Poisson shot noise processes. The shot noise process describes the cumulative effects of a number of time-varying disturbances or perturbations, each of which is initiated by a point event, which arrive according to a Poisson process. These discrete pulses are assumed to have exponential decay, and so, in time, some of these decaying traces add, and this is the shot noise process (e.g., Figure 3.1, Smith, 2010). In his model, the disturbances represent the flux in postsynaptic potentials in a cell population in response to a sequence of action potentials. Smith showed that the time integral of such Poisson shot-noise pairs follows an integrated OrnsteinUhlenbeck process, whose long-time scale statistics are very similar to those assumed in the standard diffusion model. His analysis showed how diffusive information at a behavioral level could arise from Poisson-like representations at the neural level. Subsequently Smith and McKenzie (2011) investigated a simple model of how long time scale information accumulation could be realized at a neural level. Wang (2002) previously argued that models of decision-making require information integration on a time scale that is an order of magnitude greater than any integration process found at a neural level. He argued that the most plausible substrate for such long-time scale integration is persistent activity in reverberation networks. Smith and McKenzie considered a very simple model of a recurrent loop in which spikes cycle around the loop with exponentially distributed cycle times and new spikes are added superposition. The activity in the loop could, therefore, be modeled as a superposition of Poisson processes. They showed that a model based on such recurrent loops could realize the kind of long-time scale integration process described by Wang and that it, too, exhibited a form of diffusive information accumulation that closely matches what is found behaviorally. In particular, the resulting model successfully predicted the RT distributions and
modeling simple decisions and applications
55
Align on stimulus
Monkey 11
Activity for correct response (spikes/s)
2% bright pixels EASY 150
150
100
100
50
50
0
0 0
Activity for error response (spikes/s)
45% bright pixels DIFFICULT
100
200
300
400
0
150
150
100
100
50
50
0
0 0
100 200 300 400 Time from stimulus onset (ms)
0
100
200
300
400
100 200 300 400 Time from stimulus onset (ms)
Fig. 3.10 Neural firing rates averaged over cells for firing rates aligned on the stimulus for the two monkeys from Ratcliff, Hasegawa et al. (2007). The firing rates are divided into thirds as a function the behavioral response (fastest third, middle third, and slowest third). The left hand column show easy conditions, bright responses to 98% white pixels and dark responses to 98% black pixels and the right hand column shows difficult conditions, bright responses to 55% white pixels and dark responses to 55% black pixels. The first row shows firing rates for cells in the receptive field of the target corresponding to the correct response and the correct response is made (target cell). The second row shows firing rates for cells in the receptive field of the target corresponding to the incorrect response for the stimulus when a correct response is made (competitor cell). The solid lines are the data and the dashed lines are model predictions.
choice probabilities from a signal detection experiment reported by Ratcliff and Smith (2004).
Human Neuroscience Diffusion models are currently being combined with fMRI and EEG techniques to look for stimulus-independent areas that implement decision-making (e.g., vmPFC, Heekeren, Marrett, Bandettini, & Ungerleider, 2004) and to map diffusion model parameters onto EEG signals (Philliastides et al., 2006). eeg support for across-trial variability in drift rate Philiastides, Ratcliff, and Sajda (2006) used a face/car discrimination task with briefly presented degraded pictures. They recorded EEGs from multiple electrodes during the task and then weighted and combined the electrical signals to obtain a single number or regressor that best discriminated between faces and cars. This was repeated over 60 56
ms windows from stimulus onset on up. The singletrial regressor was significant at two times, around 180 ms and around 380 ms. Ratcliff, Philiastides, and Sajda (2009) reasoned that, if the regressor was an index of difficulty, then in each condition of the experiment, responses could be sorted into those that the electrical signal said were more facelike and those that were more carlike. When responses were sorted and the diffusion model fit to the two halves of each condition, the drift rates for the two halves differed substantially but only for the later component at 380 ms. The diffusion model provides an estimate of nondecision time, which represents the duration of encoding and stimulus transformation processes prior to the decision time (as well as response output processes). This estimate shows that the decision process begins no earlier than 400 ms after stimulus onset, and so the late EEG signal component indexes difficulty on a trial-to-trial basis prior to the onset of the decision process. Therefore, these two features of the late component
elementary cognitive mechanisms
provide evidence that drift rate varies from trial to trial.
strength. These studies are the beginning of a new approach to brain structure and processing.
eeg support for across-trial variability in starting point Bode, Sewell, Lilburn, Forte, Smith and Stahl (2012) reported EEG evidence consistent with trial-to-trial biasing of the starting point of the diffusion process. They recorded EEG activity in a task requiring discrimination between briefly presented images of chairs or pianos that were presented in varying levels of noise and then backward masked. They applied a support vector machine pattern classifier to the EEG signals at successive time points and showed that decisions could be decoded (i.e., predicted) from the EEG several hundred milliseconds before the behavioral response. When the stimulus display contained only noise and no discriminative information, the decision outcome could still be predicted from the EEG, but only from the activity prior to stimulus presentation and not from any later time points. Bode et al. found that the RT distributions and accuracy in their task were well described by a diffusion model in which the starting point for evidence accumulation was biased toward the upper or lower boundary, depending on the participant’s previous choice history. They proposed that the information in the prestimulus EEG was a neural correlate of the process of setting the starting point, which occurs prior to the start of evidence accumulation. When the display contained no stimulus information and the drift of the diffusion process was zero, the primary determinant of the decision outcome would be the participant’s bias state: Processes starting near the upper boundary would be more likely to terminate at that boundary, and similarly for the lower boundary.
fmri A major problem with attempts to relate results from fMRI measurements to the growth of activity in decision-related brain areas is the sluggishness of the BOLD response. Despite this, there are many studies that use diffusion models in analyses of fMRI data. Mulder, Van Maanen, & Forstmann (2014) have reviewed a number of studies of perceptual decision making using fMRI methods and found evidence for regions associated with different components of diffusion models. Although there was some converegence, maps of the peakcoordinates of the activity for model components showed quite a large scatter across areas. This research would require a chapter by itself but the notion the some brain areas accumulate noisy evidence from other areas is certainly a mainstream belief in neuroscience and diffusion models are one theoretical framework that relates the neural to the behavioral level.
structural mri Studies that have examined structural connections between brain areas that are implicated in the control of decision making have found correlations between tract strength and decisionmaking variables. Forstmann et al. (2010) found a relationship between cortico-striatal connection strength and the ability of subjects to change their speed-accuracy tradeoff settings. Mulder, Boekel, Ratcliff, & Forstmann (2014) found correlations between subjects’ ability to bias their responses in response to reward and vmPFC-STN connection
Conclusions The use of diffusion models in representing simple decision-making in a variety of domains is an area of research that is seeing significant advances. The view that evidence is accumulated over time to decision criteria seems a settled view. The competing models seem to produce about the same conclusions about processing within experimental paradigms, and so broad interpretations do not depend on the specific model being used. In psychological applications, the basic theory and experimental applications are well established and somewhat mature. But application to individual differences (including neuropsychological testing) and different subject and patient populations are in their infancy. Also, neuroscience applications in both experimental and theoretical research are blossoming, with a variety of experimental methods being used as well as a variety of variants on the basic models developed in psychology.
Author Note Preparation of this chapter was supported by grants NIA R01-AG041176, AFOSR grant FA9550-11-1-0130, IES grant R305A120189, and by ARC Discovery Grant DP140102970.
modeling simple decisions and applications
57
Glossary Accumulator Model: A model in which positive increments are continuous random variable and the time at which the increments are made are discrete in time. The accumulators race to separate decision criteria. Confidence Judgments: Tasks in which responses are made on a discrete scale using different response keys. Decision Boundaries: These represent the amount of evidence needed to make a decision. Decision criteria: The amount of evidence for one or other alternative to make a decision. In diffusion models, the criteria are represented as boundaries on the evidence space. Diffusion Model: A model that assumes continuously available evidence in continuous time. Evidence accumulates in one signed sum and the process terminates when one of two decision criteria are reached. Diffusion Process: A process in which continuously variable noisy evidence is accumulated in continuous time. Drift rate: The average rate at which a diffusion process accumulates evidence. Go/Nogo Tasks: Tasks in which subjects respond to one stimulus type but hold their response until a time out for the other response. Leaky Competing Accumulator Model: A model in which evidence is continuously available in continuous time. Evidence is accumulated in separate accumulators (i.e., separate diffusion processes) and there is both decay in an accumulator and inhibition from other accumulators. Nondecision Time: Duration of processes other than the decision process. These include encoding time, response output time, memory access time in memory tasks, and the time to transform the stimulus representation to a decisionbased representation for perceptual tasks. Optimality: Often defined in terms of “reward rate” or the number correct per unit time in simple decision making experiments by analogy with animal experiments. Ornstein-Uhlenbeck diffusion process: This describes a noisy evidence accumulation process with leakage or decay; the standard (Wiener or Brownian motion) diffusion process describes a process in which there is no leakage. Poisson Counter Model: A model in which increments are discrete equal-sized units, but the time at which they arrive as the accumulators are Poisson distributed (exponential delays between counts). Poisson shot noise process: A process in which each point event in a Poisson process generates a continuous, time-varying disturbance or perturbation. The shot noise process is the cumulative sum of the perturbations. The shot noise process has been used as a model for a variety of phenomena, including the flow of electrons in vacuum tubes, the cumulative effects of earth tremors, and the flux in the postsynaptic potential in cell bodies in a neural population. PVT: The psychomotor vigilance test in which a counter starts counting up and the subject simply hits a key to stop it counting.
58
Random walk model: A discrete-time counterpart of the diffusion process. A diffusion process accumulates evidence in continuous time, whereas a random walk accumulates evidence at discrete time points. Response Signal and Deadline Tasks: Tasks in which the subject is required to respond at an experimenterdetermined time. The dependent variable is usually accuracy and the task measures how it grows over time in the decision process. Response Time Distributions: The distribution of times at which the decision process terminates (i.e., a histogram of times for data). Single Cell Recording in Animals: Recordings from single neurons often in awake behaving animals.
References Balota, D. A. & Chumbley, J. I. (1984). Are lexical decisions a good measure of lexical access? the role of word frequency in the neglected decision stage. Journal of Experimental Psychology: Human Perception and Performance, 10, 340–357. Basso, M. A. & Wurtz, R. H. (1998). Modulation of neuronal activity in superior colliculus by changes in target probability. Journal of Neuroscience, 18, 7519–7534. Bode, S., Sewell, D. K., Lilburn, S., Forte, J. D., Smith, P. L. & Stahl, J. (2012). Predicting perceptual decisions from early brain activity. Journal of Neuroscience, 32, 12488– 12498. Bogacz, R., Brown, E., Moehlis, J., Holmes, P. & Cohen, J. D. (2006). The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced choice tasks. Psychological Review, 113, 700–765. Boucher, L., Palmeri, T., Logan, G., & Schall, J. (2007). Inhibitory control in mind and brain: An interactive race model of countermanding saccades. Psychological Review, 114, 376–397. Brown, S. D., & Heathcote, A. J. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178. Brown, S. D., Marley, A. A. J., Donkin, C. & Heathcote, A. J. (2008). An integrated model of choices and response times in absolute identification. Psychological Review, 115, 396–425. Buonocore, A., Giorno, V., Nobile, A. G., & Ricciardi, L. (1990). On the two-boundary first-crossing- time problem for diffusion processes. Journal of Applied Probability, 27, 102–114. Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review, 100, 432–459. Churchland, A. K., Kiani, R., & Shadlen, M. N. (2008). Decision-making with multiple alternatives. Nature Neuroscience, 11, 693–702. Deco, G., Rolls, E. T., Albantakis, L., & Romo, R. (2013). Brain mechanisms for perceptual and reward-related decisionmaking. Progress in Neurobiology, 103, 194–213. Dennis, S. & Humphreys, M. S. (2001). A context noise model of episodic word recognition. Psychological Review, 108, 452–477.
elementary cognitive mechanisms
Diederich, A., & Busemeyer, J.R. (2003). Simple matrix methods for analyzing diffusion models of choice probability, choice response time, and simple response time. Journal of Mathematical Psychology, 47, 304–322. Diederich, A., & Busemeyer, J. (2006). Modeling the effects of payoff on response bias in a perceptual discrimination task: Bound-change, drift-rate-change, or twostage-processing hypothesis. Perception & Psychophysics, 68, 194–207. Ditterich, J. (2006). Computational approaches to visual decision making. In D. J. Chadwick, M. Diamond, & J. Goode (Eds.), Percept, decision, action: Bridging the gaps (p.114). Chichester, UK: Wiley. Ditterich, J. (2010). A comparison between mechanisms of multi-alternative perceptual decision making: Ability to explain human behavior, predictions for neurophysiology, and relationship with decision theory. Frontiers in Neuroscience, 4, 184. Donkin, C., Brown, S., Heathcote, A., & Wagenmakers, E. J. (2011) Diffusion versus linear ballistic accumulation: Different models for response time, same conclusions about psychological mechanisms? Psychonomic Bulletin & Review, 55, 140–151. Feller, W. (1968). An introduction to probability theory and its applications. New York, NY: Wiley. Forster, K. I. (1976). Accessing the mental lexicon. In R. J. Wales & E. Walker (Eds.), New approaches to language mechanisms (pp. 257–287). Amsterdam, Netherlands: NorthHolland. Forstmann, B. U., Anwander, A., Schafer, A., Neumann, J., Brown, S., Wagenmakers, E.-J., Bogacz, R., & Turner, R. (2010). Cortico-striatal connections predict control over speed and accuracy in perceptual decision making. Proceedings of the National Academy of Sciences, 107, 15916–15920. Frank, M.J. (2006). Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making. Neural Networks, 19, 1120–1136. Gillund, G., & Shiffrin, R.M. (1984). A retrieval model for both recognition and recall. Psychological Review, 91, 1–67. Geddes, J., Ratcliff, R., Allerhand, M., Childers, R., Wright, R. J., Frier, B. M., & Deary, I. J. (2010). Modeling the effects of hypoglycemia on a two-choice task in adult humans. Neuropsychology, 24, 652–660. Gold, J. I., & Shadlen, M. N. (2000). Representation of a perceptual decision in developing oculomotor commands. Nature, 404, 390–394. Gold, J. I., & Shadlen, M. N. (2001). Neural computations that underlie decisions about sensory stimuli. Trends in Cognitive Science, 5, 10–16. Gold, J. I., & Shadlen, M. N. (2007). The neural basis of decision making. Annual Review of Neuroscience, 30, 535–574. Gomez, P., Ratcliff, R., & Perea, M. (2007). A model of the go/no-go task. Journal of Experimental Psychology: General, 136, 347–369. Grinband, J., Hirsch, J., & Ferrera, V.P. (2006). A neural representation of categorization uncertainty in the human brain. Neuron, 49, 757–763.
Hanes, D. P., and Schall, J. D. (1996). Neural control of voluntary movement initiation. Science, 274, 427–430. Heekeren, H. R., Marrett, S., Bandettini, P. A., Ungerleider, L. G. (2004). A general mechanism for perceptual decisionmaking in the human brain. Nature, 431, 859–62. Hintzman, D. (1986). “Schema abstraction” in a multiple-trace memory model. Psychological Review, 93, 411–428. Krajbich, I., & Rangel, A. (2011). A multi-alternative drift diffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proceedings of the National Academy of Sciences, 108, 13852–13857. Kounios, J., Osman, A. M., & Meyer, D. E. (1987). Structure and process semantic memory: New evidence based on speed-accuracy decomposition. Journal of Experimental Psychology: General, 116, 3–25. Laming, D. R. J. (1968). Information theory of choice reaction time. New York: Wiley. Leite, F. P., & Ratcliff, R. (2010). Modeling reaction time and accuracy of multiple-choice decisions. Attention, Perception and Psychophysics, 72, 246–273. Leite, F. P., & Ratcliff, R. (2011). What cognitive processes drive response biases? A diffusion model analysis. Judgment and Decision Making, 6, 651–687. Link, S. W. & Heath, R. A. (1975). A sequential theory of psychological discrimination. Psychometrika, 40, 77–105. Luce, R. D. (1986). Response times. New York, NY: Oxford University Press. Mazurek, M. E., Roitman, J. D., Ditterich, J., & Shadlen, M. N. (2003). A role for neural integrators in perceptual decisionmaking. Cerebral Cortex, 13, 1257–1269. McClelland, J. L. & Chappell, M. (1998). Familiarity breeds differentiation: A Bayesian approach to the effects of experience in recognition memory. Psychological Review, 105, 724–760. McKoon, G., & Ratcliff, R. (1992). Spreading activation versus compound cue accounts of priming: Mediated priming revisited. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 1155–1172. McKoon, G., & Ratcliff, R. (2012). Aging and IQ effects on associative recognition and priming in item recognition. Journal of Memory and Language, 66, 416–437. McNamara, T. P. (1992 Priming and constraints it places on theories of memory and retrieval. Psychological Review, 99, 650–662. McNamara, T. P. (1994). Priming and theories of memory: A reply to Ratcliff and McKoon. Psychological Review, 101, 185–187. Meyer, D. E., Irwin, D. E., Osman, A. M., & Kounios, J. (1988). The dynamics of cognition: mental processes inferred from a speed-accuracy decomposition technique. Psychological Review, 95, 183–237. Milosavljevic, M., Malmaud, J., Huth, A., Koch, C., & Rangel, A. (2010). The Drift Diffusion Model can account for the accuracy and reaction times of value-based choice under high and low time pressure. Judgment and Decision Making, 5, 437–449. Morton, J. (1969). The interaction of information in word recognition. Psychological Review, 76, 165–178. Mulder, M. J., Boekel, W., Ratcliff, R., & Forstmann, B. U. (in press). Cortico-subthalamic connection predicts individual
modeling simple decisions and applications
59
differences in value-driven choice bias. Brain Structure & Function, 219, 1239–1249. Mulder, M. J., Bos, D., Weusten, J. M. H., van Belle, J., van Dijk, S. C., Simen, P., van Engeland, H., & Durson, S. (2010). Basic impairments in regulating the speed-accuracy tradeoff predict symptoms of attention-deficit/hyperactivity disorder. Biological Psychiatry, 68, 1114–1119. Mulder, M., van Maanen, L., & Forstmann, B. U. (2014). Perceptual decision neurosciences-A model-based review. Neuroscience, 277, 872–884. Munoz, D. P., & Wurtz, R. H. (1995). Saccade-related activity in monkey superior colliculus. I. Characteristics of burst and buildup cells. Journal of Neurophysiology, 73, 2313–2333. Murdock, B. B. (1982). A theory for the storage and retrieval of item and associative information. Psychological Review, 89, 609–626. Niwa, M., & Ditterich, J. (2008). Perceptual decisions between multiple directions of visual motion. Journal of Neuroscience, 28, 4435–4445. Norris, D. (2006). The Bayesian reader: explaining word recognition as an optimal Bayesian decision process. Psychological Review, 113, 327–357. Oberauer, K., Suβ, H-M., Wilhelm, O., Wittmann, W. W. (2003). The multiple faces of working memory: Storage, processing, supervision, and coordination. Intelligence, 31, 167–193. Palmer, J., Huk, A. C., & Shadlen, M. N. (2005). The effect of stimulus strength on the speed and accuracy of a perceptual decision. Journal of Vision, 5, 376–404. Philiastides, M., & Ratcliff, R. (2013). Influence of branding on preference-based decision making. Psychological Science, 24, 1208–1215. Philiastides, M. G., Ratcliff, R., & Sajda, P. (2006). Neural representation of task difficulty and decision making during perceptual categorization: A timing diagram. Journal of Neuroscience, 26, 8965–8975. Platt, M., & Glimcher, P. W. (1999). Neural correlates of decision variables in parietal cortex. Nature, 400, 233–238. Pleskac, T. J., & Busemeyer, J. R. (2010). Two-stage dynamic signal detection: A theory of choice, decision time, and confidence. Psychological Review, 117, 864–901. Purcell, B. A., Heitz, R. P., Cohen, J. Y., Schall, J. D., Logan, G. D., & Palmeri, T. J. (2010). Neurally-constrained modeling of perceptual decision making. Psychological Review, 117, 1113–1143. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108. Ratcliff, R. (1980). A note on modelling accumulation of information when the rate of accumulation changes over time. Journal of Mathematical Psychology, 21, 178–184. Ratcliff, R. (1985). Theoretical interpretations of speed and accuracy of positive and negative responses. Psychological Review, 92, 212–225. Ratcliff, R. (1988). A note on the mimicking of additive reaction time models. Journal of Mathematical Psychology, 32, 192– 204. Ratcliff, R. (2002). A diffusion model account of reaction time and accuracy in a two choice brightness discrimination task: Fitting real data and failing to fit fake but plausible data. Psychonomic Bulletin and Review, 9, 278–291.
60
Ratcliff, R. (2006). Modeling Response Signal and Response Time Data, Cognitive Psychology, 53, 195–237. Ratcliff, R. (2013). Parameter variability and distributional assumptions in the diffusion model. Psychological Review, 120, 281–292. Ratcliff, R., Cherian, A., & Segraves, M. (2003). A comparison of macaque behavior and superior colliculus neuronal activity to predictions from models of simple two-choice decisions. Journal of Neurophysiology, 90, 1392–1407. Ratcliff, R., & Frank, M. (2012). Reinforcement-based decision making in corticostriatal circuits: Mutual constraints by neurocomputational and diffusion models. Neural Computation, 24, 1186–1229. Ratcliff, R., Gomez, P., & McKoon, G. (2004). A diffusion model account of the lexical-decision task. Psychological Review, 111, 159–182. Ratcliff, R., Hasegawa, Y. T., Hasegawa, Y. P., Childers, R., Smith, P. L., & Segraves, M. A. (2011). Inhibition in superior colliculus neurons in a brightness discrimination task? Neural Computation, 23, 1790–1820. Ratcliff, R., Hasegawa, Y. T., Hasegawa, Y. P., Smith, P. L., & Segraves, M. A. (2007). Dual diffusion model for single-cell recording data from the superior colliculus in a brightnessdiscrimination task. Journal of Neurophysiology, 97, 1756– 1774. Ratcliff, R., Love, J., Thompson, C. A., & Opfer, J. (2012). Children are not like older adults: A diffusion model analysis of developmental changes in speeded responses, Child Development, 83, 367–381. Ratcliff, R., & McKoon, G. (1988). A retrieval theory of priming in memory. Psychological Review, 95, 385–408. Ratcliff, R., & McKoon, G. (1994). Retrieving information from memory: Spreading activation theories versus compound cue theories. Psychological Review, 101, 177–184. Ratcliff, R., & McKoon, G. (2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20, 873–922. Ratcliff, R., McKoon, G., & Tindall, M. H. (1994). Empirical generality of data from recognition memory receiveroperating characteristic functions and implications for the global memory models. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 763–785. Ratcliff, R., Perea, M., Colangelo, A., & Buchanan, L. (2004). A diffusion model account of normal and impaired readers. Brain & Cognition, 55, 374–382. Ratcliff, R., Philiastides, M. G., & Sajda, P. (2009). Quality of evidence for perceptual decision making is indexed by trialto-trial variability of the EEG. Proceedings of the National Academy of Sciences, 106, 6539–6544. Ratcliff, R., & Rouder, J.N. (2000). A diffusion model account of masking in letter identification. Journal of Experimental Psychology: Human Perception and Performance, 26, 127–140. Ratcliff, R., Sheu, C-F, & Gronlund, S.D. (1992). Testing global memory models using ROC curves. Psychological Review, 99, 518–535. Ratcliff, R. & Smith, P. L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111, 333–367. Ratcliff, R., & Smith, P. L. (2010). Perceptual discrimination in static and dynamic noise: the temporal relation
elementary cognitive mechanisms
between perceptual encoding and decision making. Journal of Experimental Psychology: General, 139, 70–94. Ratcliff, R., & Starns, J. J. (2009). Modeling confidence and response time in recognition memory. Psychological Review, 116, 59–83. Ratcliff, R., & Starns, J. J. (2013). Modeling confidence judgments, response times, and multiple choices in decision making: recognition memory and motion discrimination. Psychological Review, 120, 697–719. Ratcliff, R., Thapar, A., Gomez, P., & McKoon, G. (2004). A diffusion model analysis of the effects of aging in the lexicaldecision task. Psychology and Aging, 19, 278–289. Ratcliff, R., Thapar, A. & McKoon, G. (2003). A diffusion model analysis of the effects of aging on brightness discrimination. Perception and Psychophysics, 65, 523–535. Ratcliff, R., Thapar, A., & McKoon, G. (2004). A diffusion model analysis of the effects of aging on recognition memory. Journal of Memory and Language, 50, 408–424. 1 Ratcliff, R., Thapar, A., & McKoon, G. (2006). Aging, practice, and perceptual tasks: A diffusion model analysis. Psychology and Aging, 21, 353–371. Ratcliff, R., Thapar, A., & McKoon, G. (2007). Application of the diffusion model to two-choice tasks for adults 75–90 years old. Psychology and Aging, 22, 56–66. Ratcliff, R., Thapar, A., & McKoon, G. (2010). Individual differences, aging, and IQ in two-choice tasks. Cognitive Psychology, 60, 127–157. Ratcliff, R., Thapar, A., & McKoon, G. (2011). Effects of aging and IQ on item and associative memory. Journal of Experimental Psychology: General, 140, 46–487. Ratcliff, R., Thapar, A., Smith, P. L. & McKoon, G. (2005). Aging and response times: A comparison of sequential sampling models. In J. Duncan, P. McLeod, & L. Phillips (Eds.), Speed, Control, and Age, Oxford, England: Oxford University Press. Ratcliff, R., & Tuerlinckx, F. (2002). Estimating the parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability. Psychonomic Bulletin and Review, 9, 438–481. Ratcliff, R. & Van Dongen, H. P. A. (2009). Sleep deprivation affects multiple distinct cognitive processes. Psychonomic Bulletin and Review, 16, 742–751. Ratcliff, R. & Van Dongen, H.P.A. (2011). A diffusion model for one-choice reaction time tasks and the cognitive effects of sleep deprivation. Proceedings of the National Academy of Sciences, 108, 11285–11290. Ratcliff, R., Van Zandt, T., & McKoon, G. (1999). Connectionist and diffusion models of reaction time. Psychological Review, 106, 261–300. Reed, A.V. (1973). Speed-accuracy trade-off in recognition memory. Science, 181, 574–576. Roe, R. M., Busemeyer, J. R., & Townsend, J. T. (2001). Multialternative decision field theory: A dynamic connectionist model of decision-making. Psychological Review, 108, 370–392. Roitman, J. D. & Shadlen, M. N. (2002). Response of neurons in the lateral interparietal area during a combined visual discrimination reaction time task. Journal of Neuroscience, 22, 9475–9489.
Roxin, A., & Ledberg, A. (2008). Neurobiological models of two-choice decision making can be reduced to a onedimensional nonlinear diffusion equation. PLoS Computational Biology, 4, e1000046. Schmiedek, F., Oberauer, K., Wilhelm, O., Suβ, H-M., & Wittmann, W. (2007). Individual differences in components of reaction time distributions and their relations to working memory and intelligence. Journal of Experimental Psychology: General, 136, 414–429. Schouten, J. F., & Bekker, J. A. M. (1967). Reaction time and accuracy. Acta Psychologica, 27, 143–153. Sewell, D. K., & Smith, P. L. (2012). Attentional control in visual signal detection: Effects of abrupt-onset and noonset stimuli. Journal Of Experimental Psychology: Human Perception And Performance, 38, 1043–1068. Shadlen, M. N. & Newsome, W. T. (2001). Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. Journal of Neurophysiology, 86, 1916–1935. Shiffrin, R. M., & Steyvers, M. (1997). A model for recognition memory: REM: Retrieving effectively from memory. Psychonomic Bulletin and Review, 4, 145–166. Smith, P. L. (1995). Psychophysically principled models of visual simple reaction time. Psychological Review, 102, 567–593. Smith, P. L. (2000). Stochastic dynamic models of response time and accuracy: A foundational primer. Journal of Mathematical Psychology, 44, 408–463. Smith, P. L. (2010). From Poisson shot noise to the integrated Ornstein-Uhlenbeck process: Neurally-principled models of diffusive evidence accumulation in decision-making and response time. Journal of Mathematical Psychology, 54, 266– 283. Smith, P. L., Ellis, R., Sewell, D. K., & Wolfgang, B. J. (2010). Cued detection with compound integration-interruption masks reveals multiple attentional mechanisms. Journal of Vision, 10, 1–28. Smith, P. L., & McKenzie, C. (2011). Diffusive information accumulation by minimal recurrent neural models of decision making. Neural Computation, 23, 2000– 2031. Smith, P. L., & Ratcliff, R. (2004). The psychology and neurobiology of simple decisions, Trends in Neuroscience, 27, 161–168. Smith, P. L., & Ratcliff, R. (2009). An integrated theory of attention and decision making in visual signal detection. Psychological Review, 116, 283–317. Smith, P. L., Ratcliff, R., & Sewell, D. K. (2014). Modeling perceptual discrimination in dynamic noise: Time-changed diffusion and release from inhibition. Journal of Mathematical Psychology, 59, 95–113. Smith, P. L., Ratcliff, R., & Wolfgang, B. J. (2004). Attention orienting and the time course of perceptual decisions: response time distributions with masked and unmasked displays. Vision Research, 44, 1297–1320. Smith, P.L., & Vickers, D. (1988). The accumulator model of two-choice discrimination. Journal of Mathematical Psychology, 32, 135–168. Sperling, G. & Dosher, B. A. (1986). Strategy and optimization in human information processing. In K. Boff, L. Kaufman, and J. Thomas (Eds.), Handbook of perception and performance. (Vol. 1, pp. 1–65). New York, NY: Wiley.
modeling simple decisions and applications
61
Starns, J. J., & Ratcliff, R. (2010). The effects of aging on the speed-accuracy compromise: Boundary optimality in the diffusion model. Psychology and Aging, 25, 377–390. Starns, J. J., & Ratcliff, R. (2012). Age-related differences in diffusion model boundary optimality with both trial-limited and time-limited tasks. Psychonomic Bulletin and Review, 19, 139–145. Starns, J. J., & Ratcliff, R. (2014). Validating the unequalvariance assumption in recognition memory using response time distributions instead of ROC functions: A diffusion model analysis. Journal of Memory and Language, 70, 36–52. Starns, J. J., Ratcliff, R., & McKoon, G. (2012). Evaluating the unequal-variability and dual-process explanations of zROC slopes with response time data and the diffusion model. Cognitive Psychology, 64, 1–34. Stone, M. (1960). Models for choice reaction time. Psychometrika, 25, 251–260. Thapar, A., Ratcliff, R., & McKoon, G. (2003). A diffusion model analysis of the effects of aging on letter discrimination. Psychology and Aging, 18, 415–429. Townsend, J. T. (1972). Some results concerning the identifiability of parallel and serial processes. British Journal of Mathematical and Statistical Psychology, 25, 168–197. Townsend, J. T., & Ashby, F. G. (1983). Stochastic Modeling of Elementary Psychological Processes. Cambridge: Cambridge University Press. Townsend, J. T. & Wenger, M.J. (2004). A theory of interactive parallel processing: New capacity measures and predictions for a response time inequality series. Psychological Review, 111, 1003–1035. Tuerlinckx, F., Maris, E., Ratcliff, R., & De Boeck, P. (2001). A comparison of four methods for simulating the diffusion process. Behavior, Research, Instruments, and Computers, 33, 443–456. Usher, M. & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108, 550–592. Vandekerckhove, J. & Tuerlinckx, F. (2007) Fitting the Ratcliff diffusion model to experimental data. Psychonomic Bulletin & Review, 14, 1011–1026. Vandekerckhove, J., Tuerlinckx, F., & Lee, M. D. (2011). Hierarchical diffusion models for two-choice response times. Psychological Methods, 16, 44–62. van Ravenzwaaij, D., Dutilh, G., & Wagenmakers, E.-J. (2012). A diffusion model decomposition of the effects of alcohol on perceptual decision making. Psychopharmacology, 219, 1017– 1025. van Ravenzwaaij, D., van der Maas, H. L. J., & Wagenmakers, E.-J. (2012). Optimal decision making in neural inhibition models. Psychological Review, 119, 201–215. Van Zandt, T. (2002). Analysis of response time distributions. In J. T. Wixted (Vol. Ed.) & H. Pashler (Series Ed.), Stevens’
62
Handbook of Experimental Psychology (3rd ed.), Volume 4: Methodology in Experimental Psychology (pp. 461–516). New York, NY: Wiley. Vickers, D. (1970). Evidence for an accumulator model of psychophysical discrimination. Ergonomics, 13, 37–58. Vickers, D. (1979). Decision processes in visual perception. New York, NY: Academic . Vickers, D., Caudrey, D., & Willson, R. J. (1971). Discriminating between the frequency of occurrence of two alternative events. Acta Psychologica, 35, 151–172. Voss, A. & Voss, J. (2007) Fast-dm: A free program for efficient diffusion model analysis. Behavior Research Methods, 39, 767–775. Wang, X. J. (2002). Probabilistic decision making by slow reverberation in cortical circuits. Neuron, 36, 955–968. Watson, A. B. (1986). Temporal sensitivity. In K. R. Boff, L. Kaufman, & J. P. Thomas (Eds.), Handbook of perception and human performance (pp 6–1 to 6–43). New York, NY: Wiley. White, C. N., Ratcliff, R., & Starns, J. J. (2011). Diffusion models of the flanker task: Discrete versus gradual attentional selection. Cognitive Psychology, 63, 210–238. White, C., Ratcliff, R., Vasey, M. & McKoon, G. (2009). Dysphoria and memory for emotional material: A diffusion model analysis. Cognition and Emotion, 23, 181–205. White, C. N., Ratcliff, R., Vasey, M. W., & McKoon, G. (2010). Using diffusion models to understand clinical disorders. Journal of Mathematical Psychology, 54, 39–52. Wickelgren, W. A. (1977). Speed-accuracy tradeoff and information processing dynamics. Acta Psychologica, 41, 67–85. Wickelgren, W. A., Corbett, A. T., & Dosher, B. A. (1980). Priming and retrieval from short-term memory: A speed accuracy trade-off analysis. Journal of Verbal Learning and Verbal Behavior, 19, 387–404. Wiecki, T. V., Sofer, I. and Frank, M. J. (2013). HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python. Frontiers in Neuroinformatics, 7, 1–10. Wong, K.-F., & Wang, X.-J. (2006). A recurrent network mechanism for time integration in perceptual decisions. Journal of Neuroscience, 26, 1314–1328. Yonelinas, A. P. (1997). Recognition memory ROCs for item and associative information: The contribution of recollection and familiarity. Memory & Cognition, 25, 747–763. Zeguers, M. H. T., Snellings, P., Tijms, J., Weeda, W. D., Tamboer, P., Bexkens, A. & Huizenga, H.M. (2011). Specifying theories of developmental dyslexia: A diffusion model analysis of word recognition. Developmental Science, 14, 1340–1354. Zohary, E., Shadlen, M., & Newsome, W. (1994). Correlated neuronal discharge rate and its implications for psychophysical performance. Nature, 370, 140–143.
elementary cognitive mechanisms
CHAPTER
4
Features of Response Times: Identification of Cognitive Mechanisms through Mathematical Modeling
Daniel Algom, Ami Eidels, Robert X. D. Hawkins, Brett Jefferson, and James T. Townsend
Abstract
Psychology is one of the most recent sciences to issue from the mother-tree of philosophy. One of the greatest challenges is that of formulating theories and methodologies that move the field toward theoretical structures that are not only sufficient to explain and predict phenomena but, in some vital sense, necessary for those purposes. Mathematical modeling is perhaps the most promising general strategy, but even under that aegis, the physical sciences have labored toward that end. The present chapter begins by outlining the roots of our approach in 19th century physics, physiology, and psychology. Then, we witness the renaissance of goals in the 1960s, which were envisioned but not usually realizable in 19th century science and methodology. It could be contended that it is impossible to know the full story of what can be learned through scientific method in the absence of what cannot be known. This precept brings us into the slough of model mimicry, wherein even diametrically opposed physical or psychological concepts can be mathematically equivalent within specified observational theatres! Discussion of examples from close to half a century of research illustrate what we conceive of as unfortunate missteps from the psychological literature as well as what has been learned through careful application of the attendant principles. We conclude with a statement concerning ongoing expansion of our body of approaches and what we might expect in the future. Key Words: parallel processing, serial processing, mimicking, capacity, response times,
stochastic processes, visual search, redundant targets, history of response time measurement
From Past to Future: Main Currents in the Evolution of Reaction Time as a Tool in the Study of Human Information Processing If time has a history (Hawking, 1988), the timing of mental events certainly does. The idea that human sensations, feelings, or thoughts occur in real time seemed preposterous less than two centuries ago. When the idea has finally gained traction, its gradual acceptance in psychology has often been accompanied by much rancor that continued well beyond the development of the first attempts at measurement. After some early
progress that had been made in harnessing latency or reaction time (RT) to the study of psychological processes, Titchener (1905, p. 363) was still pondering whether “we have any right to speak of the ‘duration’ of mental processes.” Putting the term duration in inverted commas indicates the recent origin of usage of the term as well as Titchener’s own doubts about its validity or serviceability. Thirty years later, Robert Sessions Woodworth in his celebrated Experimental Psychology argued against acceptance of the first method to use reaction time. In a section poignantly titled,
63
“Discarding the subtraction method” (Woodworth 1938, p. 309), Woodworth expressed broader and deep seated reservations, observing that because “we cannot break up the reaction into successive acts and obtain the time for each act, of what use is reaction time?” Even more recent is Johnson’s (1955, p. 5) assertion that, “The reaction-time experiment suggests a method for the analysis of mental processes that turned out to be unworkable.” An onerous history granted, the use of RT is firmly established in modern cognitive psychology not least due to the general conceptual framework provided by the domain known as the informationprocessing approach. Within this framework, RT is used in a systematic, theoretically guided fashion in the quest to isolate the underlying processes and their interactions activated by a given experimental task (cf. Laming 1968; Luce 1986; Townsend and Ashby 1983; Welford 1980). Nevertheless, we would be remiss if we did not examine, if in passing, the essence of Woodworth’s reasoning. Woodworth’s concerns hark back to the forceful argument on the continuity of consciousness offered by William James in his seminal Principles of Psychology (see in particular, James 1890, Vol. 1, p. 244). In the chapter on the stream of thought, James contends that, due to its absolute continuity, thought or consciousness cannot be divided up for analysis. His attack is directed against the possibility of introspecting minute mental experiences, but the objection is equally cogent with respect to RT. When obtaining a value of RT, one measures the duration between two markers in time, usually that between some specified signal and the observer’s response. The RT is taken then to represent the time consumed by an internal process needed to perform a mental task. However, if mental processes are not amenable to partition, any pair of markers must be considered arbitrary. On a deeper level, the situation is a replica or subspecies of the relationship between nature and language as discussed by Friedrich Nietzsche (1873). Nature might well comprise a continuous whole, but human language (used to describe nature) is always discrete. How does one treat a continuous variable with discrete tools? Without dwelling on this issue in any depth, the upshot is clear. A fundamental, yet heretofore unarticulated assumption underlying all RT-based models, serial or parallel, is this: Natural mental functioning can be divided into separate, psychologically meaningful acts. Returning to history, why did the idea that mental acts occur in real, hence measurable, time 64
seem so incredible less than 200 years ago? The physiology of the human nervous system had made startling advances just around that time, but for many centuries the main thrust of attempts to understand the system along with the attendant sensations fell under the rubric of “vitalism.” Vitalism is the doctrine that there is a fundamental difference between living organisms and nonliving matter because the former entail something that is missing from the latter. Pinpointing just what this “something” was has proved elusive, yet the doctrine enjoyed widespread influence from antiquity (the Greek anatomist Galen held that vital spirits are necessary for life) to the 19th century (for all his great contributions to physiology, the towering figure of Johannes Müller subscribed to vitalism) to our own time (Freud’s “psychic energy,” “emerging property,” or even “mind” itself come to mind). Vitalism is best understood as opposition to the Cartesian extension of mechanistic explanations to biology (Bechtel and Richardson 1998; Rakover 2007). It is on this background of the strong influence of vitalism that researchers at the time believed that nerve conduction was instantaneous (in the order of the speed of light or faster) and that, in any rate, it was too fast to be measured.
Hermann von Helmholtz’s Measurement of the Speed of the Nerve Impulse Therefore, Hermann von Helmholtz (1821– 1894) along with his fellow students at Johannes Müller’s Berlin Institute of Physiology had to summon their best judgment and blood (signing their antivitalism oath) to rebuff their teacher, and espouse a strictly mechanistic position. Under the circumstances, it was a bold move on the part of Helmholtz and his peers to consider the moving nerve impulse as (merely) an event in space-time on a par with, say, that of a moving locomotive. Devising an ingenious method for measuring time, Helmholtz proceeded to measure the speed of the former. He stimulated a motor nerve in a frog’s leg and found that the latency of the muscular response depended on the distance of the stimulation from the muscle: the smaller the distance, the faster the response. Helmholtz’s calculations showed that the propagation of the impulse down the nerve was surprisingly slow, between 25 to 43 meters a second. Regardless of the value, it became evident that the speed of nerve conduction was finite and measurable! More boldly yet, Helmholtz turned to humans, asking participants to push a button
elementary cognitive mechanisms
when they felt stimulation in their leg. Predictably enough, people reacted to stimulation in the toe slower than to stimulation in the thigh. Helmholtz estimated the speed of nerve conduction in humans to be between 43 and 150 meters per second. The large range is notable, attesting to considerable variability. It was this variability, within-individuals as well as between-individuals, that discouraged Helmholtz from further pursuing RT research as a reliable means of psychological investigation. The last point is also notable because individual differences was the subject of a now-famous incident at the Greenwich observatory, which occurred half a century before Helmholtz’s measurements. Assistant astronomer David Kinnebrook was relieved of his job by his superior, Nevil Maskelyn, due to disagreement in reading the time that a star crossed the hairline in a telescope. The superior found that his assistant’s observations were a fraction of a second longer than his own. Twenty years later, this little-noticed incident (at the time) came to the attention of the German astronomer F. W. Bessel, who started to compare transit times by various astronomers. This first RT study revealed that all astronomers differed in their recordings. In order to cancel out individual variation from the astronomic calculations, Bessel set out to construct “personal equations” as a means to correct or equate differences among observers. Notice that the concept of “personal equation” assumes small (to nil) intra-individual variability in tandem with stable interindividual differences. Neither notion proved to be correct as Helmholtz witnessed with his observers. It turns out that variability, whether of intra- or inter-individual species, is a fixture of RT measurement. It is at this juncture that models developed within the generic framework of human information processing become truly valuable, attempting to disentangle the various sources of RT variability.
by Wilhem Wundt (1832–1920) and Franciscus Donders (1818–1889). Could RT measurement be refined to gauge duration of central processes, presumably reflecting mental activity in the brain itself? Wundt approached the question experimentally by probing the simultaneity of stimulus appearance in the conscious mind. Do stimuli presented at exactly the same (physical) time evoke similarly simultaneous sensations? In a simple experiment performed in his home in 1861, Wundt attached a calibrated scale to the end of the pendulum of his clock so that pendulum’s position at any time could be determined with precision. A needle fastened to the pendulum perpendicularly at its middle would strike a bell at the very instant that the pendulum reached a predefined position on the scale. Using this makeshift (yet accurate for the time) instrument (Figure 4.1), Wundt was observing his own mind: Hearing the sound of the bell, Wundt did not perceive the pendulum to be in the predetermined position but always away from there. Calculation based on the perceived distance of the pendulum from its original position showed the perceived time difference to be at around one-tenth of a second. Inevitably, Wundt concluded, people do not consciously experience the visual and auditory stimuli simultaneously, despite the fact that these stimuli occur at the same time. Encouraged by such data, Wundt subsequently attempted to measure specific central processes. A favorite topic was “apperception,” an early term for what is now known as attention. Wundt found that the RT to a given stimulus was shorter by one-tenth of a second if the observer concentrated on the response rather than on the stimulus. The
Studies of Reaction Time in Wundt’s Laboratory: Moving from the Periphery to the Center Note that for all his pioneering contribution, Helmholtz’s measurements were restricted to the periphery of the nervous system, to sensory and motor nerves transmitting impulses toward or from the brain (Fancher 1990). Even this result, as we recounted, was achieved after travelling a torturous road. Nevertheless, barely a decade after Helmholtz’s measurements in 1850, the following intriguing question was posed (separately)
Fig. 4.1 Schematic of Wundt’s thought meter.
features of response times
65
reason is that one has first to perceive the stimulus and then to apperceive it, that is, to decide whether it is the appropriate one for responding. When focusing on the response, the second of these processes is gratuitous. Consequently, Wundt proposed that apperception takes about one-tenth of a second. Regardless of the particular results, the significance of Wundt’s early foray into RT measurement lies in his bold thrust to probe the duration of mental processes of consequence to cognitive science and everyday life alike. Cognizant of its potential, Wundt’s home apparatus has been depicted as a “thought meter,” and the title of his own report (including subsequent data) aptly read, “Die Geschwindigkeit des Gedankens” (The speed of thoughts; Wundt 1892). Important work in Wundt’ laboratory was carried out on a related subject, the number of stimuli noticed simultaneously during a short glance. James McKeen Cattell (1860–1944), Wundt’s American student and assistant, first employed RT in the study of the visual span of attention or span of apprehension. However, a true pioneer in this domain was the Scottish philosopher, Sir William Hamilton, whose observations are reported in a posthumous book published in 1859. Hamilton spread out marbles on the ground and concluded that, on average, the span of visual attention is limited to 6–7 items. However, if the marbles are arranged in groups (of say two, three, or four marbles a group) the person can comprehend many more marbles because the mind considers each group as a unit. These results and conclusions anticipated those of George Miller a century later in his famous article on the “magical number seven” and on the effects of “chunking” (Miller, 1956). The power of grouping was expounded by Cattell himself who found that whole words could replace single unrelated letters, leaving invariant the number of units noticed within the span. Modern studies on the span of attention use short exposure times (at around 50 ms) in order to avoid eye movements and counting. As a result, observers actually report the contents of their short-term or “iconic” memory. George Sperling (1960), reviving interest in the subject in his groundbreaking studies on the information contained in brief visual presentations, concluded that the span was much larger than previously thought (in the order of 12–16 letters), but that it was also short lived. The very report by the observer can conceal the true size of the span; larger estimates are found when the deleterious effects of reporting are circumvented. 66
We surely have come a long way from Hamilton’s informal surmises. Nevertheless, his observations brought to the fore the idea of limited capacity (resources or attention) and even the idea of parallel processing. Murray (1988, p. 159), ever the keen reader, concluded that “Hamilton perceived consciousness as a kind of receptacle of limited capacity.” Needless to add, capacity and parallel processing are key concepts in the current approach known as human information processing.
Donders’ Complication Experiment and Method of Subtraction We already mentioned Franciscus Donders, the true pioneer of RT measurement in psychology. This Dutch physiologist (founder of modern ophthalmology among sundry achievements) developed the first influential, hence lasting procedure for measuring the duration of specific mental processes. Donders devised an experimental setup known as the complication experiment with an assorted method of RT measurement called the method of subtraction. The idea was to present tasks of increasing complexity and to subtract then the respective RTs in order to identify the duration of the added processes. The technique is best illustrated by the procedures used by Donders himself (Donders, 1868; we follow Murray’s 1988 depiction). In one variation, the a-method, a sound such as ki is presented by the experimenter and the observer reproduces it orally as quickly as possible (one should note that Donders was the first experimenter to use human [his own] voice in RT studies). The a-task is a simple reaction time experiment, recording the time it takes the observer to react to a predetermined stimulus by a predetermined response. In the b-method, one of several sounds is presented on a trial, and the observer repeats the sound as fast as possible. This variation is dubbed choice reaction time: Several different stimuli are presented and the observer responds to each of them differently. In the c-method, several sounds are given again, but the observer imitates only one of them and remains silent when the others are presented (this variation is now known as the go/no-go procedure). The differences between the respective RTs reflect the duration of the psychological processes involved. For example, the RT for the b-procedure entails both discrimination (or identification) of the stimulus presented and the selection of the appropriate response, whereas that for the c-procedure entails merely discrimination
elementary cognitive mechanisms
1.
A
R 2.
A
B
Ra
Rb
1.
A
2.
A
B
Output (RT1)
Output (RT2)
Mean RT = RT1 – RT2 Fig. 4.2 Illustration of the complication experiment and analysis by the method of subtraction. Top: A simple RT experiment (a single predetermined response made to a single predetermined stimulus) is complicated into a choice RT experiment (two different stimuli with a different response made to each). Bottom: The time it takes to perform the mental act of choice is estimated by subtracting the mean RT of the simple RT experiment from the mean RT of the choice RT experiment.
(or recognition, see Luce, 1986, p. 213). The mean difference (c−a) was taken by Donders to measure the duration of recognition, whereas that of (b−c) estimated the time consumed by the need to make a choice between responses (see Figure 4.2 for an outline of the Donders experiment and for the logic of the method of subtraction). In the scheme developed by Donders, there is a chain of discrete nonoverlapping processing systems. The duration of each process is measurable, assuming that each added experimental task uniquely taps one and only one of the processing systems. If the assumptions hold, the procedure succeeds in inferring the duration and eventually the attendant architecture of the psychological system under test. Consequently, the idea of subtraction has exerted a profound influence on RT theory and experimentation. Townsend and Ashby (1983) paid well-deserved homage to Donders by designating psychological processes carried out in a serial fashion (i.e., sequential and without overlap in processing time) as Dondersian systems. This much granted, closer scrutiny of the method (in particular, its underlying assumptions) uncovered several problems, so that the method has not been wholeheartedly accepted by students of RT. The
main criticisms are easily summarized because they are interconnected in final analysis. First, the experimental data collected by different investigators or by an individual investigator at different times proved extremely variable. For example, Donders (1868), Laming (1968), and Snodgrass, Luce, & Gealanter (1967) reported vastly different RTs for (c−a) and (b−c). Over and above the variability, the order of the differences is not preserved: Donders found (b−c) longer than (c−a), but those subsequent investigators found the opposite pattern. Wundt, an early champion of the method, was so discouraged by the large intra-individual variability that he abandoned his RT studies altogether. Second, the method requires that the added experimental task has no influence on any of the other tasks. The assumption of “pure insertion” (Sternberg, 1969a,b) asserts that the previous processes unfold in time precisely in the same fashion regardless of whether another process is inserted into the chain. If pure insertion is impossible in general or does not hold in particular cases, the assumptions of additivity and independence of the processes are also compromised. To compound the problem, the assumption of pure insertion is untestable with mean statistics although it might be with distributional statistics (Ashby & Townsend, 1980). The issue is not fully settled (cf. Luce, 1986, p. 215), and it is moot whether it can be fully settled with any mathematical or statistical test. The third criticism is even more fundamental. It concerns the relationship between the experimental task and the unobservable psychological process or subprocesses that the task is supposed to tap. It is not prima facie clear that by calling a task “response choice/selection” or “stimulus discrimination” the underlying psychological process is that of choice or discrimination. It is not even clear that the task taps a single process, excluding all sorts of subprocesses. The raison d’etre of the complication experiment is minimum complication, so that a single welldefined process is probed with each addition. This minimal-addition- or single-process principle is not readily testable (certainly not at the level of the mean) and it is even more difficult to satisfy in experimental practice. After all, how can one decide that the added task comprised the smallest complication possible (Külpe, 1895; Sternberg, 1969a,b). Symptoms of the problem have recurrently surfaced in the century following the Donders experiment. Where Donders called a given task “discrimination,” Wundt called the features of response times
67
Saul Sternberg’s Revival of the Donders Project: Inaugurating the Modern Study of Human Information Processing Reminiscent of the tale of Sleeping Beauty, Dondersian procedures were lying dormant for over a century. The prince-investigator reviving the technique was Saul Sternberg (1966, 1969a,b), and the magic kiss awakening renewed interest was his memory scan experiment. The participants are first shown a number of items. Then, they decide whether a test item was or was not present in the set just shown. Prototypical results are given in Figure 4.3. Two features of the data are noteworthy. First, RT is a linear function with a positive slope of the size of the memory set shown. Adding a single member to the memory set increases RT by the same constant amount. Second, targets and foils produce the same increment in RT, so that the slope of the function is the same for yes and for no responses (in Figure 4.3, the intercept, reflecting stimulus encoding, base and residual time, incidentally is also the same; however, the important feature is the parallelism of the targetpresent and target-absent functions). Sternberg interpreted the linear function with the positive slope to reflect serial processing such that the test 68
600
mean RT
same task, “cognition.” Donders’ c-task was conceived to tap stimulus recognition, but already in 1886 Cattell questioned its validity, arguing that the task entails processes beyond identification or recognition. More recently, Welford (1980), echoing Cattell’s concerns, concluded that the difference between the b-task (originally thought to tap response selection) and the c-task is one of degree and that both entail choice of the response. Wundt, acutely aware of the problem, conceived a new task, the d-procedure (meant to be a pure measure of recognition), to no avail. More than linguistic indeterminism is at stake. G. A. Smith (1977), for one, obtained data showing choice to be faster than recognition! How does one make choices among stimuli that one does not recognize? In the absence of a definite task-process association and theory, we cannot know with certainty the identity and order of the pertinent psychological processes. Given the problems, the method of subtraction was out of favor for many years with students of RT. The succeeding section will bring us into the modern era of cognitive research. Subsequent sections will revisit many of the concepts with more quantitative detail, but still with emphasis on a friendly style.
500 positive response negative response line of best fit
400
0
1
4 2 3 size of the positive set
5
6
Fig. 4.3 Prototypical results of Sternberg’s memory scan experiment.
item is compared with the memory representation of each of the items in the positive set – one item a time. He interpreted the parallelism of the slopes to mean that the search continues until the entire memory is exhausted even if an early item in the positive set matches the probe stimulus. Sternberg’s interpretation of his data is now known as the standard serial exhaustive search model. If search ceases as soon as a probe item is located, the process is said to self-terminate. Sternberg’s original (1966) analyses were stronger than many of the scores of studies that followed, due not only to invoking several control conditions but also in helping to rule out an important class of parallel models. Again, we will discuss this matter as well as other topics in this section in more quantitative detail subsequently. Sternberg’s conclusions seem compelling, but, as subsequent research has revealed, neither is forced by the data. The positive slope appears to have all the earmarks of serial processing, but a moment of reflection suffices to show that the same result follows in a natural fashion from parallel processing. Think of horse races (actual ones, not modeling metaphors) with a different number of horses in each race. The referee reports back to the organizer once each race is over (i.e., when the slowest horse crosses the finish line). Clearly, each race is parallel and exhaustive. It requires only a little intuition to conclude that the larger the number of horses, the longer the expected duration between the common start and the finishing time by the slowest horse (i.e., the RT-set size function has a positive slope). Now, if every horse runs just as fast and with the same random variation no matter how many other horses are present, then it can be shown that the increasing duration for all
elementary cognitive mechanisms
bimodal perception, see Bernstein, 1970), a result inconsistent with the prediction of the standard serial exhaustive model. Regardless of this particular result (the violation can be dealt with fairly easily by slight modification of the pertinent models), redundant target designs proved a powerful tool in revealing virtually all aspects of human information processing. Sternberg revived Donders’ method of subtraction in a further profound way. In his method of additive factors (Sternberg, 1966, 1969a,b), one does not eliminate or bypass a stage (as in the method of subtraction) but rather affects it selectively. Think of the standard memory scan experiment for an illustration. In the additive factors scheme, the operation of comparison comprises a single stage affected by the factor of size of the search set. Suppose that one adds another stage, stimulus encoding, affected by degrading the quality of the visual presentation. The logic of the method is as follows. Varying the number of stimuli in the search set affects comparison (and response) processes, whereas degrading the quality of the stimuli affects perceptual encoding. Additivity (of the mean RTs) holds if indeed the manipulations influence the respective processes selectively. If one further assumes independence, the incremental effects of added stages should be additive over accumulated RTs, too. The expected result in this two-stage serial model is shown in Figure 4.4. The influence of set size is revealed by the positive slopes of the RT curves and that of visual degradation by the longer RTs. Critically, the two factors do not interact as is evident in the parallelism of the slopes.
700 600 Poor quality 500 400 RT
the horses to finish bends over (i.e., increases by less and less an amount as the number of horses increases; see Townsend & Ashby 1983, p. 92 for a proof ) rather than being straight. Such a system, whether run by horses or by parallel perceptual or cognitive channels, is said to be unlimited capacity (e.g., Townsend, 1974; Townsend & Ashby 1978). Sternberg’s (1966) analyses did rule out this variety of parallel processing. Formal models of memory- or perceptualscanning have introduced the notion of limited capacity in performing the comparison process. In Townsend’s capacity reallocation model (Townsend, 1969, 1974; Townsend &Ashby, 1983; see also, Atkinson, Holmgren, & Juola, 1969), a finite amount of capacity is redistributed after completing the comparison of each item, the processing itself is always a parallel race between the remaining items. Such limited capacity, parallel exhaustive search models yield precisely the same predictions as Sternberg’s original model (e.g., positive parallel slopes for target-present and target-absent processing, absence of a serial position effect, and linear growth of variance with the numbers of items), some of which are not generally confirmed by experimental data. Following Townsend’s early development (1969, 1971), several classes of parallel models have been shown to predict Sternberg’s results (Corcoran, 1971; Murdock, 1971; Townsend, 1969, 1971a,b, 1972, 1974; Townsend & Ashby, 1983). Moreover, Sternberg’s data can be predicted by self-terminating rather than exhaustive search whether in parallel (e.g., Ratcliff, 1978) or even serial (e.g., Theios Smith, Haviland, Traupmann, & Moy 1973) models. The reader should consult Section 5 as well as Van Zandt and Townsend (1993) and Townsend and Colonius (1997) for more details on the topic of testing self-terminating versus exhaustive processing in parallel and serial models. The interrogation of Sternberg’s results entailed also (slight) experimental modifications. For example, the memory set can follow rather than precede the probe stimulus thus initiating what are usually termed visual search (or early target) experiments. Early examples of these designs are found in the studies by Estes and Taylor (1969), Atkinson et al. (1969), and van der Heijden (1975). A more consequential manipulation entails the inclusion of more than a single replica of the target stimulus in the search list. RT is found to decrease with the number of redundant targets (e.g., Baddeley & Ecob, 1973; Egeth, 1966; in
Good quality 300 200 100 0
0
1
2
3
4
6 5 Set Size
7
8
9
10
Fig. 4.4 Hypothetical results in an additive factors experiment in which additivity is seen to hold.
features of response times
69
The additive factors method, like the memory scan experiment, has engendered a very large amount of research, producing a wealth of valuable theorems (e.g., the independence of additivity and stochastic independence) and theoretical insights (e.g., success and failure in mimicry of serial systems by parallel systems). The last point will be particularly appreciated by those experiencing the frustration in convincing a graduate student (or a seasoned researcher!) that a positive slope does not, ipso facto, imply serial processing (Feature Integration Theory [Treisman & Gelade, 1980] is a poignant case in point). Criticisms and generalizations of the method unearthed further important information. For example, additivity does generally support separate processing stages, but interaction does not necessarily support a single stage. Statistical properties of analysis of variance (ANOVA) might compromise, to an extent, its value as the (sole) diagnostic tool (cf. Townsend, 1984). A really consequential feature of the method in virtually all modifications and generalizations (but see Schweickert, 1982) is that it tells us nothing about the order of occurrence of the various stages (or underlying processes). The ensuing problems were already noticed with respect to the original method by Donders, but they are equally serious with the method of additive factors. Sternberg’s landmark studies, along with the almost concomitant works by Sperling, Estes, Nickerson and Egeth and others, inaugurated the human information-processing approach in earnest. Where Donders, in his subtraction method, changed the nature of the tasks as well as the number of stimuli, Sternberg, in his memory experiment, did not change the task, only added items. It is easier to subtract numerical values of RT than entire psychological processes (cf. Marx & Cronan-Hillix, 1987). In his additive factors method, Sternberg showed that it was not even necessary to subtract processes, only to affect them experimentally in a selective way. Within a decade of Sternberg’s seminal contribution, virtually all students of RT and roughly half the community of cognitive psychologists (Lacmann, Lachmann, & Butterfield 1979) were conducting research employing or testing some aspect of Sternberg’s theory and methodology.
Basic Issues Expressed Quantitatively In the previous sections we surveyed some of the history of mental chronometry. Several key issues 70
were highlighted from historical and philosophical perspectives, all related to the notion of time and the role it plays in mental processes. First, mental events—our feelings, thoughts, and decisions—take time, and this time can be measured. Second, internal subprocesses can take place one at a time (and are hence called serial processes), or at the same time (parallel processes). Third, when several subprocesses take place, the system must await the completion of each and every one of these subprocesses before moving on to respond (exhaustive processing), or conversely, it can finish before that, say, upon the termination of any one of the subprocesses (minimum-time).1 Fourth, subprocesses may be independent from one another (or not), and so do the time durations taken to complete each subprocess. And finally, we introduced the idea that people may have a limited capacity—limited amount of resources (attention)—and hence can deal effectively with a limited amount of processing at any given time. In what follows we provide a formal treatment of each of these basic issues, along with illustrative examples. The first issue, regarding the temporal modeling of information processing, is ubiquitous in theoretical approaches to human cognition. We see this affirmed in several chapters of this book, such as Chapter 3 (Modeling Simple Decisions and Using a Diffusion Model) and Chapter 6 (A Past, Present, and Future Look at Simple Perceptual Judgment). Many models of perception and decision making are based on the premise that information, or evidence toward some target behavior is accumulated over time. Thus, to answer Titchener’s (1905) question, we have both the right and the obligation to speak about duration of mental processes. The remaining basic issues are discussed next in greater detail; the reader may find the following example helpful throughout this discussion. Suppose that you are a driver approaching an intersection. The sight of a red light or the sound of a policeman’s whistle signals you to stop and give way. One can think of the visual signal and the auditory signal as being processed in separate subsystems, which we call channels. We denote the time to process and detect a signal in each of the channels by tA (for the visual channel) and tB (for the auditory channel). We further make the assumption that both signals are presented at exactly the same time (we can relax this assumption subsequently). What can we learn about the time course of information processing? What can we learn about the
elementary cognitive mechanisms
+ tB , for any tA , tB > 0. This intuitive notion is true only as long as we assume that tA (and similarly tB ) is the same in the serial and the parallel cases.2 Is it realistic to expect our driver to bring her car to a stop only after she detects both sources of information? On intuitive grounds, one would prefer to act quickly on the basis of only one signal, whichever signal is detected first as a sign of danger. This issue is considered next.
relationship between the information-processing channels? The critical properties of architecture, stopping rule, and independence will now be introduced with only little mathematics. A rigorous mathematical statement regarding architecture (i.e., parallel and serial processes) appears in Section 4 of this Chapter. For more quantitative detail on these features, the reader should consult Townsend and Ashby (1983) or Townsend and Wenger (2004b, for a more recent statement).
Exhaustive versus Minimum-Time Stopping Rule
Architecture: Parallel Versus Serial Processing
Awaiting the completion of two subprocesses is referred to as exhaustive processing. The processing durations, tserial and tparallel for that strategy were given earlier. It is also possible to stop as soon as the first process is completed; in our example, as soon as the driver detects the red light or hears the policeman’s whistle. This strategy is referred to as minimum-time processing. The overall time it takes for a parallel system with a minimum-time rule is given by tparallel = min(tA , tB ). For a serial system, the total duration depends on the order of processing, tserial = tA if A is first, and tserial = tB if B is processed first. Needless to add, in a serial system that stops as soon as the first channel completes (as soon as the first signal is detected), the second channel will not have a chance to operate at all. Although other stopping rules are also possible, the exhaustive and minimum-time stopping rules are of particular interest. They are illustrated in Figure 4.5 (Panels a–d). Processing times for the different systems are summarized in Table 4.1.
As mentioned, two or more subprocesses can take place one at a time (serial), or at the same time (parallel). Figure 4.5 illustrates these modes of processing, where each arrow corresponds to a particular channel. It is convenient to consider the way the system operates—its architecture—through the prism of the time it takes to complete the processing of both signals. Suppose that the driver is unwilling to hit the brakes unless both signals are spotted, that is, she processes the two signals exhaustively. In the serial case (Panel a), the time to process both signals is the sum of the durations needed to process each channel, such that total tserial = tA + tB . In the parallel case, this time equals that needed to process the slower of the two processes, tparallel = max(tA , tB ). It is tempting to think that parallel processing will yield a faster braking response compared with serial processing (and more generally that parallel processing is more efficient than serial processing), given that max(tA , tB ) < tA (a)
(b) Process A
Process B
Decision
(c)
(e)
Process B
(d) Process A
Process B
Process A
Response
Decision Response
Decision
Response
Process A Process B
Decision Response
Process A Process B
∑
Decision Response
Fig. 4.5 Illustrations of serial (Panels a, c), parallel (b, d), and coactive (e) systems. Panels a and b demonstrate exhaustive processing, where both processes A and B must finish before a decision and response can be made. Panels c and d show minimum-time processing, where processing ceases once process A is completed (but B had not finished, as indicated by the broken line). Panel e illustrates a coactive mode of processing, where activation from two channels is summed before the decision stage.
features of response times
71
Table 4.1. Summary of overall completion times for the various models. tA and tB denote the time to process signals in channels A and B, respectively. Model and stopping rule
Overall completion time
Parallel exhaustive
max(tA , tB )
Serial exhaustive
tA + t B
Parallel minimum time
min(tA , tB )
Serial minimum time if channel A is processed first and B second if B is processed first and A second
tA tB
Stochastic Independence Two events are said to be statistically independent if the occurrence of one does not affect the probability of the other. For example, height and SAT score (standardized test score for college admissions in the United States) are independent if knowing the height of a person tells nothing about his SAT score. In the context of processing models, total completion-times of channels A and B are independent if knowing one does not tell us a thing about the value of the other. Our discussion of the architecture and stopping rule was simplified by the fact we assumed that processing is deterministic, rather than stochastic (probabilistic). A deterministic process always yields a fixed result, such that the effect or phenomenon we observe has no variability. For example, a deterministic process predicts that the time taken to drive from Sydney to Newcastle is always fixed, or that the time to choose between chocolate and vanilla flavors is the same every time we stop at the ice-cream parlor. Under this assumption, we were able to represent the time for processing in channels A and B by the fixed values, tA and tB . However, observations of human performance (and Sydney’s traffic) lead to the conclusion that behavior is quite variable and that it can probably be better described as a stochastic process. If so, processing time in any particular channel can no longer be characterized by a fixed value, but is represented by a random variable. A random variable does not have a single, fixed value but can rather take a set of possible values. These values can be characterized by probability distributions. The probability density function (pdf ) is defined by f(t) = p(T = t), and gives the likelihood that some 72
process, which takes random time T to complete, will actually be finished at time t. We can use f(t) to define stochastic independence. In probability theory, two random variables are independent if knowing the value of one tells nothing whatsoever about the values of the other (e.g., Luce, 1986, chapter 1). In processing models, total completion times of channels A and B are independent if knowing one, say tA , tells us nothing about the likelihood of various values of tB . Thus, we can express independence in terms of the joint pdfs, fAB (tA , tB ) = fA (tA ) · fB (tB ), which means that the joint density of processes A and B both finishing at time t is equal to the product of the probability of A finishing at time tA and the probability of B finishing at time tB .
Workload Capacity and the Capacity Coefficient We recounted earlier that the time to process multiple signals depends on the stopping rule and mode of processing (serial, parallel). Notably, processing also depends on the amount of resources available for processing, a notion that we call capacity. One may think of the cognitive system as performing some work, and the more subprocesses (channels) are engaged the greater the amount of work there is to perform. We define workload capacity as the fundamental ability of a system to deal with ever heavier task duties (Townsend & Eidels, 2011; see also Townsend & Ashby, 1978, 1983). A ready example is the increase in load from processing one signal to processing two or more signals. One may find it useful to think about work and capacity in terms of metaphors such as water pipes filling a pool, or tradesmen building a house. Suppose that the tradesmen operate in parallel (and, for illustration, deterministically) and that there is an infinite amount of resources (tools, building materials)—unlimited capacity. In that case, a twofold increase in the number of workers will cut to half the amount of time needed to build the house (assuming all tradesmen have the same workrate). Critically, adding more workers does not affect the labor rate of each individual worker. In a similar vein, increasing load on the cognitive system by increasing the number of to-be-processed items does not have an effect on the efficiency and time of processing each item alone. The time to process the visual signal (red light) when it is presented alone should be the same as the time to process the same signal when it is presented in tandem with the auditory signal (whistle by the policeman),
elementary cognitive mechanisms
tA|A = tA|AB . To clarify the notation, the subscript A|A indicates processing of signal A given that only signal A is present, whereas A|AB indicates processing of signal A when A and B are both present. If several channels are working toward the same goal and capacity is unlimited, then adding more channels should facilitate processing. It is possible however, that capacity is limited. In one special case, the overall amount of processing resources, X, can be a fixed value. With more and more channels coming into play, fewer resources can be allocated to each channel, and, consequently, the time to complete processing within each channel increases. So, for example, the time to process the visual signal is longer when the auditory signal is also present. Using the same notation as before, we can express this as tA|A < tA|AB and tB|B < tB|AB . Under limited capacity, performance with a given target is impaired as more targets are added to the task. Metaphorically, this is tantamount to tradesmen who are trying to work in parallel but share one set of tools. Worker A cannot work at the same rate that she did alone if she needs to await her partner handing over the hammer. Given that multiple workers or channels operate toward the same goal, a limited-capacity system can still complete processing faster than (or at least as fast as) any single channel alone (depending on the severity of the capacity limitation). However, a limitedcapacity system cannot be faster than an otherwise identical unlimited-capacity system. A third and at first curious case is that of super capacity. It is possible in principle that as more and more channels are called for action, the system recruits more resources (àla Kahneman, 1973) and is able to allocate to each of the channels more resources than what each channel originally had when it was working alone. In this case, tA|A > tA|AB and tB|B > tB|AB , and moreover, the more signals (and channels) there are, the faster the system completes processing. Under supercapacity, performance with a given target is improved as more targets are added to the task. We can model super capacity by way of a system in which channels A and B pool their activation into a single buffer, in which evidence is then compared against a single criterion. In that sense, processing channels can also join efforts to satisfy a common goal as could be the case in the tradesmen example. This mode of processing is often referred to as coactivation (e.g., Colonius & Townsend, 1997; Diederich & Colonius, 1991; Miller, 1978, 1982; Schwarz, 1994; Townsend & Nozawa, 1995;
Townsend & Eidels, 2011) and is illustrated in Figure 4.5e. Clearly, this type of model benefits from an increase in the number of relevant signals. With auditory and visual signals contributing to a single pool, evidence accumulates more quickly, and will surpass threshold faster. Thus, a coactive model is a natural candidate for supercapacity. However, it is not the only way supercapacity can be achieved in parallel systems as we shall see (Eidels, Houpt, Altieri, Pel, & Townsend 2011; Townsend & Wenger, 2004a). Townsend and Nozawa (1995) offered a measure of workload capacity known as the capacity coefficient: COR (t) =
log [SAB (t)] . log [SA (t) · SB (t)]
(1)
SA (t) and SB (t) are the survivor functions for completion times of processes A and B, and tell us the probability that channels A and B, respectively, did not finished processing by time t. SAB (t) is the survivor function for completion times of the system when channels A and B are both at work (e.g., when two targets are being processed simultaneously). We have already defined the pdf, f(t) = p(T = t), as the likelihood that a process that takes random time T to complete will actually be finished at time t. We can also define the probability that the process of interest is finished before or at time t, known as the cumulative distribution function (cdf ), F(t) = p(T ≤ t). The survivor function is the complement of the cdf, S(t) = 1 – F(t) = p(T>t), and tells us the probability that this process had not yet finished by time t. The capacity coefficient, COR (t), allows to assess performance in a system that processes multiple signals by comparing the amount of work done by the system when it processes two signals with the amount of work it does when each of the signals is presented alone. The subscript OR indicates that processing terminates as soon as subprocess A or subprocess B finishes (i.e., minimum-time termination). Townsend and Wenger (2004a) developed a complimentary capacity coefficient for the AND design, where the system can stop only after the two processes, A and B, are both finished: CAND (t) =
log [FA (t) ·FB (t)] . log [FAB (t)]
(2)
Equations 1 and 2 both apply to two channels, but the C(t) index can be easily generalized to account for more than two processes (Blaha & Townsend, 2006). The interpretation of COR (t) features of response times
73
and CAND (t) is the same, so that C(t) refers to both indices. Parallel-independent models are characterized by unlimited capacity, C(t) = 1. Capacity is C(t) < 1 in a limited capacity model, and it is C(t) > 1 with super capacity in force. Architecture (serial, parallel), stopping rule, and potential dependencies can also affect the capacity coefficient. For the effect of architecture, consider a serial model, which processes channel A first and then processes channel B. This model will take more time to complete, on average, than an otherwise identical parallel model in which processes A and B occur simultaneously. The former also results in C(t) < 1 – limited capacity. Breakdown of independence across channels also affects C(t) in a predictable manner. Townsend and Wenger (2004a) and Eidels et al. (2011) have shown that positive dependency (one channel “helps” the other) can lead to supercapacity, C(t) > 1, whereas negative dependency (one channel inhibits the other) can lead to limited capacity, C(t) < 1. The capacity coefficient is discussed further in the later section, Theoretical Distinctions, along with an illustrative example from the Stroop milieu. The interpretation of C(t) is particularly revealing when discussed with respect to the benchmark model that we describe next.
The standard parallel model can be considered as the “industry’s standard” in response-time modeling. This model is characterized by unlimited capacity and independent, parallel processing channels (attributes that yield the acronym UCIP, e.g., Townsend & Honey, 2007). If we further assume that the model can stop as soon as either one of the channels completes processing, we end up with an independent race model, illustrated earlier in Figure 4.5(d). Formally, the stochastic version of this model can be written as (3)
SA (t) and SB (t) are again the survivor functions for completion times of processes A and B and tell us the probability that channels A and B, respectively, did not finish by time t. Consider a model that stops processing as soon as either channel finishes (minimum-time processing), but will otherwise not stop as long as process A is still going on and process B is still going on (i.e., as long as both processes “survive,” hence the term survivor function). Because processing-channels A and B are 74
Smimimum−time (t) = S1 (t) · S2 (t) · ... · Sn (t) =
n
Si (t).
(4)
i=1
Given a parallel model, it is possible that the system stops only when all of its channels had completed processing (exhaustive processing). In the example, the system will stop only when both channel A and channel B stop. Assuming again that the channels are independent, the probability that the model completes processing by (at or before) time t is equal to the product of the probabilities of channels A and B finishing, FAB (t) = FA (t) · FB (t)
(5)
and in the more general form, with n channels,
The Benchmark Model: Parallel, Independent, Unlimited Capacity
SAB (t) = SA (t) · SB (t).
independent, we can multiply the probabilities so that the probability that the entire system does not stop by time t, SAB (t), is given by the product of the probabilities of A and B not finishing (see Eq. 3 again).3 We note that this equation describes a model with only two channels, but it can be generalized to any number of channels. The probability that an independent race model with n parallel channels does not complete by time t is given by the product of the probabilities of neither channel finishing,
Fexhaustive (t) = F1 (t)·F2 (t)·...·Fn (t) =
n
Fi (t) (6)
i=1
Two well-known RT inequalities also define the benchmark model. Miller (1978, 1982) proposed an upper bound for performance in the OR design (“respond as soon as you detect A or detect B”), the race model inequality: FAB (t) ≤ FA (t) + FB (t).
(7)
The inequality states that the cumulative distribution function for double-target displays, FAB (t), cannot exceed the sum of the single-target cumulative distribution functions if processing is an ordinary race between parallel and independent channels. Violations of the inequality imply supercapacity of a rather strong degree (Townsend and Eidels 2011; Townsend and Wenger 2004a). Grice, Canham, & Gwynee (1984) introduced a bound on limited capacity, often referred to as the Grice inequality:
elementary cognitive mechanisms
FAB (t) ≥ MAX [FA (t), FB (t)].
(8)
This inequality states that performance on doubletarget trials, FAB (t), should be faster than (or at least as fast as) that in the faster of the singletarget channels. If this inequality is violated, the simultaneous processing of two target signals is highly inefficient and the system is very limited capacity. An implication is that there is “no savings” or gains in moving from a single target to multiple targets (in OR designs). In Section 5 we shall demonstrate the use of the three assays of capacity in an OR design – C(t) and inequalities (7) and (8). Colonius and Vorberg (1994) proposed upper and lower bounds appropriate for AND tasks (“respond if you detect target A and target B”), which are analogous to OR tasks in the sense that their violations indicate supercapacity and limited capacity. Our benchmark model is, therefore, useful in serving as a gold standard against which performance can be compared and interpreted.
Conclusion Information-processing models can be characterized by the following four features referring to the relations among processing channels: architecture (serial, parallel), stopping rule (minimum-time, exhaustive), capacity (limited, unlimited, super), and stochastic (in)dependence. Most of these properties are latent and cannot be observed directly. Response times are useful tools in uncovering these properties, but in some cases the result is not unique. Model mimicry is thus the focus of the upcoming section. The caveats granted, recent advances in response-time modelling of cognitive processes proved useful in addressing some of the mimicking challenges (allowing researchers to identify critical features of human information-processing). The later section on Theoretical Distinctions outlines some of the advances, followed by applications of novel techniques from empirical literature. The reader might have noticed that some interesting topics such as the stochastic form of serial models were excluded from our discussion due to lack of space.4 However, the topics included in this chapter should give the reader a good understanding of elementary information-processing theory and a solid preparation for more specialized reading. Box 1 gives a practical illustration of the outstanding issues.
Model Mimicry Possessing the building blocks (architecture, stopping rule, capacity, and independence), we
Box 1 Is human capacity limited? We noted in this section that workload capacity—as measured by the capacity coefficient—could theoretically be limited, unlimited, or super, depending on whether the efficiency of processing decreases, is left unchanged, or increases with additional load (e.g., more signals to process). Cumulative evidence suggests that human capacity is limited (Kahneman, 1973), yet important and frequent situations of modern life, such as driving a car, require simultaneous processing of multiple signals. Therefore, a key question is whether human capacity is, in fact, limited, and what might be the consequences of such limitations in our everyday life. Strayer and Johnston (2001) studied the effects of mobile-phone conversations on performance in a concurrent (simulated) driving task. They found that conversations with either a hand-held or a hand-free mobile phone while driving resulted in a failure to detect traffic signals and in slower reactions to these signals when they were detected. The findings clearly suggest that human capacity is limited. However, in a more recent driving-simulator study Watson and Strayer (2010) have been able to identify a group of individuals—referred to as “supertaskers” who can perform multiple tasks without observed detriments. Although the majority of the participants showed significant performance decrements in the dual-task conditions (compared with a single-task condition of driving without distraction), a small minority of 2.5% showed no performance decrements. These supertaskers can be best characterized as having unlimited capacity (and possibly even supercapacity). The simulated-driving studies by Strayer and colleagues highlight some practical implications of uncovering latent mental constructs (capacity, in this example). now can expand purview to establishment of classes of models characterized by those properties. For example, exhaustive stopping rule in a serial model with independent identically distributed processing times, will have a mean response time equal to the sum of the mean response times for each channel E [RT ] = E [RTChannel 1 ] + E [RTChannel 2 ] + · · · + E [RTChannel n ] + nE [T0 ] , features of response times
75
where T0 is the base time to respond. Thus, for each channel added, we simply add its mean response time for the total average response time. But, is this the only model with such a prediction? In this section, we provide instances of overlap in predictions that arise from assuming various models. When one model can predict the results of another model, we face an instance of model mimicry. Though perhaps an obvious platitude, investigators rarely seem to concern themselves with the specter of mimicry. In this discussion, we emphasize total mimicry, that is, the existence of mathematical functions carrying the structure of one model to another in such a way as to render them completely equivalent. The upshot is that no data expressed at the same level as the mimicking equations can decide between competing models. Mimicry at other levels will be considered as well as some remedies to parallel-serial dilemma (in the following section).
Mean Response Time Predictions Recall that mean RT has been a useful tool in helping to determine (or eliminate) models best suited for data. Sternberg (1966), discussed in Section 2, supported a positive linear relationship between mean RT and set size. An early extension of this paradigm to conditions where the items were on display (instead of being stored in memory) was carried out by Atkinson et al. (1969) with largely similar results. The evidence for exhaustive processing was supported by the lack of an effect for the serial position of the target in the list. On the other hand, Nickerson (1966) argued that these data could be taken to favor self-terminating processing. In a seminal research with a different type of visual paradigm, same-different matching design with multiple targets, the data were interpreted as supporting a serial self-terminating process (Egeth, 1966). Even within the visual search paradigm, sometimes a self-terminating stopping is found and sometimes an exhaustive stopping is concluded. [See section, Theoretical Distinctions for further discussion of assessing the decisional stopping rule]. However, in none of these pioneering studies was the potential for confounding by other processing characteristics, especially capacity, taken into account. As we recounted, the early standard model was a serial, exhaustive model with equal mean processing times for every item. If one additionally assumes that each item or stage possesses the 76
same actual processing distribution (thus producing the equal mean processing times a fortiori) and that they are also independent, then one has the completestandard serial model as outlined earlier. For simplicity, assume that the mean processing time for each of the single items are all equal. Assume further that the target has equal probability of appearing in any of the n positions. On target-present trials, participants process n+1 2 items on average (yielding a positive linear relationship between mean RT and set-size). On target-absent trials, participants have to process the entire list, so that the average RT is n times the mean RT for a single item. Therefore, on both target-present and target-absent trials, there is a positive linear relationship between mean RT and set size. As we alluded in Section 2, it can be shown that unlimited capacity, independent parallel models do not generally make this prediction. These models, when using an exhaustive stopping rule, produce logarithmic-like functions that increase with set size, but not in a linear fashion (see Townsend and Ashby 1983, p.92). In the case of minimum time (i.e., race) stopping, they yield curvilinear decreasing mean RT functions. Interestingly, singletarget self-terminating processing reveals a flat, straight-line mean RT function for these models. Yet, the linear prediction of the standard Sternberg model is not unique to the serial class of models. Next, we introduce a particular parallel model, where the rate of processing depends on the number of items to be processed, that does yield the linear increase prediction. This model is just one of a multitude of models that can predict the linear relationship found in the data. Mean response times are a common measure used in determining the processing mechanisms in a task. Although illuminating with respect to the manipulated variables, the model conclusions made from such observations must consider the possibility of mimicry.
Supporting Mathematics: Serial Model Recall from the previous section, Basic Issues Expressed Quantitatively, that, in a serial model, items are processed one at a time. In minimumtime processing the target may appear in any of the available positions and processing stops when the target is found. As standard practice, E[Ii ] denotes the mean processing time of the ith item. Then, using mathematical induction, for target present trials, one has
elementary cognitive mechanisms
E(Response Time for n positions) = = = =
1 1 1 E[I1 ] + E[I1 + I2 ] + · · · + E[I1 + · · · + In ] n n n 1 1 E[I1 ] + · · · + [E[I1 ] + · · · + E[In ]] n n 1 2 n E[I1 ] + E[I1 ] + · · · + E[I1 ] n n n (n + 1) E [I1 ] 2
whereas on target-absent trials the result is simply E(Response Time for n items) = E[I1 + · · · + I n ] = nE[I1 ]. There is, thus, a linear relationship between number of items and mean response times for positive and negative responses. Of course, the minimum time serial prediction is simply E[I1 ], a flat straight line.
Supporting Mathematics: Parallel Model For clarity, a “stage” of processing is the time from one item finishing processing to the next item finishing processing. For example, in an exhaustive model with three items to be processed, any channel will have three stages: the time from start until the first item is processed, the time after the first item is processed to the time the second item is processed, and the time from the second item’s completed processing until the remaining item is finished processing. In a parallel model, the distribution of stage processing time takes the form of a difference between item processing times usually conditioned on channel information. Within-stage independence is defined as the statistical independence of stage processing times across two or more channels in the same stage, j. Across-stage independence assumes the independence of these times occurs within the same channel, but for different stages. Consider the within-stage independent parallel model with each item having a processing time following an exponential distribution with a rate inversely proportional to the number of items, n. In other words, the more items to be processed, the longer the actual processing time ofeach item will be. λ be the processing Thus, let gai j = exp − n−j+1 density for the i th item in stage j of processing. For example, stage-one processing on all items is ga1 1 = ga2 1 = · · · = gan 1 = exp − λn , whereas stage-two processing has density function λ . ga1 2 = ga2 1 = · · · = gan−1 1 = exp − n−1
We omit the reasoning due to space limitations, but the average processing time for each stage is λ1 . 1 So, the mean processing time for n items is n+1 2 λ 1 (positive response) and n λ (negative response). So, for λ = E[I11 ] this parallel model gives the same predictions as the aforementioned serial model for mean response times as functions of the number of items.
Intercompletion Time Equivalence We refer to the time required for a stage of processing as the intercompletion time. So in a serial model, the intercompletion times are just the processing times. We now examine the issue of model mimicry with respect to the distribution of the intercompletion times. We will show cases in which equivalence can occur between two common models, the across-stage independent serial model and a large class of parallel models that assume within-state independence. Across-stage independence is defined as the property that the probability density function of two or more stages of processing is the product of the component single stage density functions. Consider the case in which there are two channels, a and b, each dedicated to processing a particular item. To make the equivalence easy to follow, we write the serial model on the left side of the equations and the parallel model on the right. We use f for the pdf of the serial model, and g for the parallel model. p denotes the probability that a is processed first in the serial model. fa1 (ta1 )is the probability that it takes a the exact time of ta1 to finish in the first stage of processing. G is the cumulative distribution function of the respective subscript (for a parallel model). So Gb1 (ta1 ) denotes the probability that the first stage of processing for b will fail to finish before the time ta1 in a parallel model. Then, for the independent serial model to mimic the independent parallel model on all response time measurements it is necessary that: pfa1 (ta1 )fb2 (tb2 |ta1 ) ≡ ga1 (ta1 )Gb1 (ta1 )gb2 (tb2 |ta1 ),
(9)
(1 − p)fb1 (tb1 )fa2 (ta2 |tb1 ) ≡ gb1 (tb1 )Ga1 (tb1 )ga2 (ta2 |tb1 ),
(10)
For mimicry on the level of intercompletion times, we need equivalence for each stage of processing. For example, in the case in which where a is features of response times
77
processed first (preceding Eq. (9)) one needs to define f and p so that pfa1 (ta1 ) ≡ ga1 (ta1 )Gb1 (ta1 ). The three “equal” signs simply indicate that this equation must be true for all values of ta1 . This turns out to be readily done. Thus, there is a serial model that can completely mimic response time predictions from any given independent parallel model. This shows us that response time measurements are not enough to prove that there is a unique model for the processes involved in a task. Fortunately, there are distributions for the serial model that make parallel mimicry impossible. The upshot here is that this serial class of models is more general than that of the parallel models—the parallel class is mathematically contained within the serial class. This result provides one potential avenue for assessing architecture: Try to determine from the experimental data and appropriate statistics if processing satisfies serial but not parallel processing. If parallel models pass the tests, then these particular tests cannot discriminate (for that task) serial versus parallel architectures.
The Math Beneath the Mimicry Note that by integrating with respect to tb2 , (Eq. 9) reduces to pfa1 (ta1 ) ≡ ga1 (ta1 )Gb1 (ta1 ) [FirstStageProcessing] fb2 (tb2 | ta1 ) ≡ gb2 (tb2 | ta1 ) . [SecondStageProcessing] The same conclusions hold for Eq. (10) by integrating with respect to ta2 . This means that if there is intercompletion time equivalence, then there is total model equivalence.
where < a, b > denotes that a finishes before b. To show equivalence one needs to define fa1 , fb1 , fa2 , fb2 , and p for a serial model so that each stage of processing gives equivalent intercompletion time predictions. As above, for a second stage processing, simply set fb2 (tb2 |ta1 ) = gb2 (tb2 |ta1 ) and
fa2 (ta2 |tb1 ) = ga2 (ta2 |tb1 ).
Now we focus on fa1 and p. For equivalence, it is sufficient that pfa1 (t) = ga1 (t)Gb1 (t)
Integrating with respect to t, ∞ ga1 (t)Gb1 (t)dt. p= 0
By dividing by p in the equation 1 , fa1 (t) =
ga1 (t)Gb1 (t) ga1 (t)Gb1 (t) = ∞ . p 0 ga1 (t)Gb1 (t)dt
The remaining density, fb1 (t), can be solved in the same way as above, using the equation (1 − p)fb1 (t) = gb1 (t)Ga1 (t) and the fact that 1−p = 1−
∞ 0
ga1 (t)Gb1 (t)dt =
(2 )
∞ 0
gb1 (t)Ga1 (t)dt.
Thus, the serial model that mimics the parallel is given by: ∞ ga1 (t)Gb1 (t)dt p= 0
ga1 (t)Gb1 (t) fa1 (t) = ∞ 0 ga1 (t)Gb1 (t)dt
Proposition 1. Given any within-stage independent parallel model there is always a serial model that is completely equivalent to it. Proof. This proof generalizes to cases where there are more than two processing positions (Townsend 1976a). Consider the following within-stage independent parallel model:
(1 )
gb1 (t)Ga1 (t) fb1 (t) = ∞ 0 gb1 (t)Ga1 (t)dt fb2 (tb2 |ta1 ) = gb2 (tb2 |ta1 ) and fb2 (tb2 |ta1 ) = gb2 (tb2 |ta1 ).
ga1,b2 (ta1 , tb2 ,
) = ga1 (ta1 )Gb1 (ta1 )gb2 (tb2 |ta1 ) and gb1,a2 (tb1 , ta2 , ) = gb1 (tb1 )Ga1 (tb1 )ga2 (ta2 |tb1 ), 78
A Simple but Convincing Example Assume the exponential distribution for each position and channel in both a serial and a parallel model. Assume across-stage independence, too. We follow Townsend and Ashby’s (1983) convention and use different parameter notations for serial (u)
elementary cognitive mechanisms
and parallel (v) models. We will derive parameter mappings [p = f (va1 , va2 , vb1 , vb2 )] that leave the serial and parallel equations equivalent. Then the density function for the serial model is fa1,b2 (ta1 , tb2 ; ) = pua1 exp( − ua1 ta1 )ub2 exp( − ub2 tb2 ) and the distribution for the parallel model is ga1,b2 (ta1 , tb2 ; < a, b >) = va1 exp [− (va1 + vb1 ) ta1 ] vb2 exp (−vb2 tb2 ) . Step 1: Set ub2 = vb2 and ub2 = vb2 . Second stage equivalence achieved. Step 2: Suppose the order is < a, b > for serial processing. Then computing conditional means for the first stage processing, 1 , E s (T1 |< a, b >) = ua1 whereas for < b, a >, 1 E (T1 | < b, a > ) = . ub1 s
But, for the parallel process the mean for the first stage of processing is E p (T1 | < a, b > ) =
1 = E p (T1 | < b, a > ). va1 + vb1
So, for equivalence ua1 = ub1 . Step 3: Now turn to
p = Probability that a is first in a serial model = P s (< a, b >). Recall that
P p ( < a, b > ) = =
∞
ga1 (t)G b1 (t)dt
0 ∞ 0
va1 exp[ − (va1 + vb1 )t]dt
va1 = . va1 + vb1 So, set p=
va1 . va1 + vb1
In sum, we have guided ourselves to the following propositions. Proposition 2. Given an across-stage independent and exponential serial model such that ua1 = ub1 ,there is no exponential, within-stage independent,
and across-stage independent parallel model that is equivalent to it. Although a logically sound statement, it may carry too many assumptions on the processing densities to serve for practical application. Below is a more general theorem. Proposition 3. Given a serial model, then if there exists a within-stage independent parallel model that is equivalent to it, it can be found by setting t pfa1 t dt G a1 (t) = exp − ∫ 0 pF a1 (t ) + 1 − p F b1 (t ) t (1 − p)fb1 (t ) dt G b1 (t) = exp − 0 pF a1 (t ) + (1 − p)F b1 (t ) gb2 (tb2 |ta1 ) = fb2 (tb2 |ta1 ) and ga2 (ta2 |tb1 ) = fa2 (ta2 |tb1 ) .
Conclusions In the listed two examples, one can see how making assumptions about the model can yield overlapping predictions in response times and the relationship with number of channels (or items). These conclusions obviously sound a warning siren with regard to drawing hasty inferences from the traditional logic concerning behavior of architectures. It would appear that the best that one could achieve would be to posit several classes of models, for example both parallel and serial architectures, which may explain the data together. However, subsequent sections will reveal how our metatheoretical approach can lead to experimental designs that assay such characteristics at a more fundamental level. For even more generality in model mimicry, see Townsend and Ashby (1983, Chapter 14).
Theoretical Distinctions We now turn our attention toward theoretical conditions and measures under which models are not equivalent. Careful probing of these conditions will guide us to experimental designs that can overcome the parallel-serial dilemma and reliably distinguish between information-processing systems in a broad range of empirical settings. Because questions of processing characteristics have been motivated largely by the domain of visual and memory search, they provide the most natural examples. However, the field has recently moved features of response times
79
toward more general applicability, and this section appropriately culminates in a discussion of the Double Factorial Paradigm, which allows the experimenter to evaluate architecture, stopping rule, capacity, and stochastic independence in a single block of trials and which can be applied in a multitude of perceptual and cognitive tasks.
Architecture Distinctions based on Generality serial systems generality based on degenerate mimicking First, we will tie up some loose ends from the previous section by using the results of Proposition 3 to point out an example of a serial model that is unable to be mimicked by a parallel model. When considered alongside the fact that any parallel model has an equivalent serial model (Proposition 2), we arrive at our first fundamental distinction, which we will later refine: serial models can be more general than parallel models. Later, we will discover ways in which parallel models can be more general than serial models. Suppose we start with a serial model and want to check whether it can be mimicked by a parallel model. Recall that the four equations in Proposition 3 determine the form of that parallel model if it exists. Now, if either G a1 (t) or G b1 (t) fails to be a true survivor function (for example, if limt→∞ G(t) > 0), then the corresponding parallel model is deficient, and the serial model cannot be mimicked. A serial exponential model in which the first-stage rates are not equal (ua1 = ub1 ) is perhaps the simplest case in which the impossibility of mimicry can occur (Townsend, 1972). Following this line of reasoning, Townsend (1976a) derived a set of necessary and sufficient conditions for the existence of well-defined survivor functions. Given that every quantity in these conditions is in principle derivable from reaction time data, a testable mechanism is provided for rejecting the possibility of parallel mimicry and confirming a true serial architecture. Ross and Anderson (1981) were among the first to apply these conditions to empirical data, testing an assumption of Anderson’s Adaptive Character of Thought (ACT) model that the spread of activation in memory search is parallel and independent. In the process of applying Townsend’s conditions, they encountered and addressed several obstacles. The authors had to consider extending the theoretical results to account for the possibility of having a mixture or convolution of different reaction time densities (e.g., in 80
hybrid models) and also find techniques to analyze the tail behavior of empirical distributions. They overcame these obstacles and their data indicated that it could have been produced by a parallel system of the type envisioned by the ACT model. As we have noted, however, the data could not rule out serial processing, it just failed to reject parallel processing. parallel systems generality based on partial processing So far, we have considered processing models, even dynamic models, as functions of mapping stimuli onto RT and response probabilities. At this point in our development, we must enrich these models with the concept of a state space. Simply put, the state space of a dynamic system is the set of values that the system may obtain. A variety of models, including sequential sampling and random-walk models, traditional counting models, and multi-stage activation models (Ratcliff and Smith 2004), suppose that as some cognitive process unfolds dynamically in time, evidence is gathered from each element ai in the stimulus space, and a threshold ci must be reached before the element ai has been completely processed. The state space, therefore, constrains the possible values that the amount of evidence can take at any given time. It turns out that probing the continuum of accumulated evidence reflected in the state space can be very informative in distinguishing between architectures. In fact, it reverses the mimicry-based serial systems generality proven earlier, such that we can use it to reject serial systems and legitimately confirm parallelism. To formalize this reasoning, we denote the amount of information that has been sampled from the element ai at some point in time by γi (t). This function can be either discrete or continuous, as required by the phenomena being modeled. In a traditional discrete-stage model, γi (t) could represent, say, the number of “features” from the feature set that have been sampled from an alphanumeric character (see Figure 4.6 for an example from visual search). Here, the state space is finite, since each character has a finite number of features that can be processed. A Poisson process, on the other hand, operates over a countably infinite state space, such that the possible amount of evidence gathered from each item can be put in correspondence with the integers. Finally, a continuous (uncountably infinite) scale for γi (t) is familiar from connectionist models of cognition,
elementary cognitive mechanisms
Serial
Parallel
Fig. 4.6 Illustration of the underlying state space in visual search. The two panels represent stimulus displays for which a participant is instructed to “find the S.” Individual features of each letter are grey and dotted if they have not yet been processed. If processing is cut off at some point—for instance, if the display terminates—the participant may be left with some letters in a partial state of processing. In the left panel, the letter “E” is in a partial state. Notice that the serial processor must treat the letters one at a time, so there is at most one stimulus in a partial state. In the right panel, on the other hand, all four of the letters are in partial states of processing because the parallel processor has no such restriction.
such as McClelland and Rumelhart’s (1981; see also McClelland, Ramelhart, & the PDP Research Group, 1986) interactive activation model, where it is commonly referred to as the activation value of a node. This is also generally the case in diffusion processes. If the state space contains more than one value, then we can, in principle, consider partial processing of elements (in the discrete, featurebased case, this corresponds to some but not all of the features in an object been processed at the end of the trial; see Figure 4.6). This helps us distinguish between architectures in the following way. A serial system can only sample features from a single element ai at once, and only moves on to begin sampling from another element after completing the first. Thus, there can be at most one element in a partial state of completion (i.e., 0 < γi (t) < Ci ) in a serial system, whereas a parallel system can have arbitrarily many elements in such a state. One way to distinguish a parallel processor from a serial processor in an empirical setting, therefore, is to observe the underlying state space while more than one item could be in a partial state of completion. Townsend and Evans (1983) developed a fullreport experiment based on this premise. They collected second guesses from participants and examined the pattern of accuracy. Each underlying state (e.g., “item 1 totally processed; item 2 partially processed”) maps onto some accuracy pattern (e.g., “item 1 correct and item 2 incorrect on first guess; item 2 correct on second guess”). However, serial models cannot produce underlying states with more
than one item partially processed, so the two models predict observably different distributions of accuracies. The authors tested the hypothesis that processing was parallel against a null hypothesis where it was serial. In the statistical analysis, the predictions were passed through two progressively stricter “sieves” and the serial null hypothesis was unable to be rejected. This work was later expanded by Van Zandt (1988) to demonstrate patterns of individual differences in parallel and serial processing. One individual may perform a task in a serial mode, whereas a different individual experiencing identical experimental conditions may perform in a parallel mode. serial systems generality based on order The next fundamental distinction is the way in which order of completion is selected (Townsend and Ashby 1983, Chapters 3 and 15). As a concrete context for discussion, consider a typical visual search task in which a participant is instructed to decide whether a particular target is present in a display with distractors (e.g., the letter H in an array of other letters). Suppose that this search were carried out serially. It is plausible, then, that the participants could causally direct their attention, choosing at each stage which item will be examined next. They could even decide on some search strategy a priori: “Start with the left-hand stimulus on the top of the display and scan from left to right, top to bottom.” In any case, it is apparent that the order of processing is not affected at all by the rates of completion of the various items. If this task were carried out in parallel, on the other hand, order of completion is entirely determined by the relative rates of different items. If item a can be processed faster than item b, then the two items will be completed in the order more frequently than in the order simply by the stochastic nature of processing, and no a priori decision is able to affect this ordering. parallel systems generality based on the identity of items or channels To summarize our conclusions thus far, we have seen that the rate of processing at each stage in both serial and parallel models can depend upon the identity and order of previously processed items, but only in serial models can the rates potentially depend upon a predetermined order of future items. Parallel models are capable of a different kind of flexibility, however—dependency on the identity of items that have started, but not yet finished features of response times
81
completion (Townsend 1976b). Consider a visual search with three items, a, b, and c. All items are being processed simultaneously, so if item c is particularly inscrutable and takes more effort to process, its presence in the display might slow down the completion time of items a and b, even if a or b end up finishing first. This is behavior that cannot be mimicked by a serial model. Note also that not all parallel models necessarily have this property; it is an additional degree of freedom we can draw upon when constructing a parallel model to fit observed data. If a is the target in a self-terminating UCIP model, for example, processing times would be independent of the identity of c by definition. However, a limited capacity parallel model predicts the desired behavior. parallel-serial testing paradigm These simple observations about generality based on order and identity are the foundation of the early Parallel-Serial Tester (PST) paradigm (Snodgrass and Townsend 1980; Townsend 1976b; Townsend and Ashby 1983, chapter 13; Townsend and Snodgrass 1974). The PST paradigm is built on three separate conditions of a simple matching experiment, in which a participant must search through a list of two items for targets. In Condition 1, the participant gives Response 1 (R1) if the target item appears on the left of the list and Response 2 (R2) if the target item appears on the right. Condition 2 is a simple AND task, where the target must appear in both positions for response R1, and Condition 3 is a simple OR task, where the target may appear in either position for response R1. Condition 1 is used to get a baseline measure of order effects, which is compared with the cases of Conditions 2 and 3. The processing time of an item under serial processing cannot depend upon the identity of other uncompleted items, so each intercompletion time that we measure empirically must be the processing time of a single item. Intercompletion times under parallel processing face no such constraint. Although the mathematical details and precise predictions for both models are fleshed out in Townsend and Ashby (1983, Chapter 13; see also Townsend 1976b), the basic result for serial systems forces the sum of mean reaction times in the two possible orders of Condition 1 to equal the sum of mean reaction times in the redundant conditions. If the two sums violate this equality, we have empirical evidence for parallel processing. This paradigm was used successfully by Neufeld and McCarty (1994) to investigate the effect of 82
stress (e.g., periodic high intensity sound) on performance in the three conditions described above, with letters Q, R, T, and Z as stimuli. Contrary to expectations, they found that the presence of a stressor made the system more likely to operate in parallel. In his dissertation, Vollick (1994) applied PST to the clinical setting. It was suggested that the cognitive impairments found in paranoid schizophrenics do not stem from architectural issues, as they process stimuli in parallel the same as healthy individuals, but rather from inefficient deployment of their processing capacity.
Stopping Rule Distinctions based on Set-Size Functions We take a short break from the serial-parallel dilemma to consider ways to distinguish between exhaustive and self-terminating stopping rules. Since Sternberg’s classic work (see Section 2), it has been common to test the stopping rule by examining slope difference in response times to different set-sizes. Consider again the class of standard serial models, where the processing random variable is the same across all experimental variables such as processing position, location in a display, identity and so on. As we recounted, if processing is exhaustive in such models, one predicts that the slope of the lines would be equal, regardless of whether the target is present, every item must be checked. In addition, it is predicted that no display position effects on RT will be found in the data. If the participant is able to terminate the process as soon as the target is found, on the other hand, one predicts the mean RT on positive trials to be lower on average than on negative trials. Townsend and Ashby (1983, Chapter 7) pointed out that if processing times can vary with display position, processing position, or identity (e.g., regardless of whether an item matches the probe), then exhaustive models could actually violate the above predictions. However, subsequent theoretical effort discovered that exhaustive systems are nonetheless extremely limited in how far they can deviate from those standard serial model predictions. A series of “impossibility” theorems (Townsend & Colonius,1997; Townsend & Van Zandt, 1990) have shown that large classes of exhaustive models of any architecture are incapable of producing significant slope differences. Further, they also showed that the ability of such models to evince strong display position effects is severely
elementary cognitive mechanisms
delimited. Commensurate with the other results of this section, these results offer us mechanisms to reject the possibility of exhaustive processing and confirm self-termination. Van Zandt and Townsend (1993) give a broad review of empirical applications of this test, ultimately concluding that participants employ a self-terminating stopping rule whenever they can properly do so in an overwhelming majority of experiments.
Distinctions based on Reaction Time Distributions The search for an empirically tractable way to distinguish between underlying cognitive processing systems reached a milestone with the development of an experimental protocol called the Double Factorial Paradigm (DFP; Eidels, Townsend, & Algom 2010; Townsend, & Nozawa, 1995; Wenger & Townsend, 2001), which yields a singularly powerful test not only of the serial-parallel distinction but also of stopping rule and capacity (as well as stochastic independence, though less directly). This paradigm rests within the theoretical framework of system factorial technology, an extensive generalization of Sternberg’s additive factors methodology (Ashby & Townsend, 1980; Townsend, 1984; Townsend & Ashby, 1983; see also Dzhafarov 2003; Kujala & Dzhafarov, 2008; Schweickert, 1978; Schweickert, Fisher, & Sung 2012; Schweickert & Townsend, 1989). double factorial paradigm The DFP entails two concurrent manipulations, each creating a factorial design (hence the double factorial paradigm). The first manipulation varies the number of presented targets (workload) in the visual search task posed. This present-absent manipulation is ideal for probing the capacity of the system. The second manipulation (note that the designation “first” and “second” does not imply logical or temporal order) pertains to the salience of the stimulus features. The salience manipulation is ideal for probing the serial-parallel distinction along with further aspects of processing at both the mean and distribution levels. Consider a Stroop display in which the words RED and GREEN are each presented in red or green ink, and a trial consists of a single word displayed in a single color (Eidels et al., 2010; Melara & Algom, 2003; Stroop 1935). Suppose further that (any kind of ) “redness” is defined as the target, so that RED in red ink comprises a redundant target display, RED in green ink and
GREEN in red ink comprise one-target displays, and GREEN in green ink is a no-target display. Note that the display can contain two, one, or zero targets, so that the effect of redundant targets is tested, too. The presence-absence factorial design thus created (WORD: target-present [RED], target-absent [GREEN] crossed with ink color: target-present [red], target-absent [green]), depicted at the bottom of Figure 4.7, enables the use of the capacity coefficient (Townsend & Nozawa, 1995) as well as important RT inequalities (Grice, Canham, & Gwyane, 1984; Miller, 1982; Colonius & Vorberg, 1994; see, Luce, 1986; Townsend & Eidels, 2011). To carry out the salience manipulation in our example, the target word RED can appear in a highly legible or in a poorly legible font and, similarly, the target red ink color can appear in a focal or in an off-focal wavelength. The goal is to selectively speed up or slow down the processing of the specific feature, that is, to manipulate one channel without affecting the other. This second factorial design (word-target salience [high, low] X color-target salience [high, low]), depicted at the top of Figure 4.7, turns out to be highly diagnostic with respect to serial-parallel distinctions when paired with the mathematical machinery of systems factorial technology framework described next. mean interaction contrast Consider the subset of trials in which both targets are presented. Hypothetical reaction time results for the four salience combinations are presented in Figure 4.8. Panel B depicts an additive outcome, implying no interaction across different levels of salience for the two channels. Panels A and C depict the two different species of interactions that might arise: A is overadditive, implying that processing is slow only when both channels are slow, and B is the under additive species, implying that processing is fast only when both channels are fast. Each factorial plot can be summarized by a simple statistic: Double difference (the difference of the pair of differences between the two values defining each line), or Mean Interaction Contrast (MIC), (RT LL − RT HL ) − (RT LH − RH HH ), where RT is the mean RT and L and H denote low and high salience conditions, respectively. Mean interactive contrast is zero for an additive outcome, negative for underadditive interaction, features of response times
83
Color Salience High
Low
High Word Salience Low
Target Word Distractor Target
Distractor Color
Fig. 4.7 Schematics of the double factorial paradigm (DFP) experiment.
and positive for overadditive interaction. These factorial plots at the level of the mean are already diagnostic with respect to the question of serial versus parallel processing. The diagnosis is qualified by the stopping rule in force, indicating whether processing is minimum time self-terminating or exhaustive (Townsend, 1974). We will now derive MIC predictions for four different models summarized in Figure 4.9: parallel self-terminating, serial self-terminating, parallel exhaustive, and serial exhaustive. First, assume a parallel architecture with a selfterminating stopping rule. If either channel is fed a strong signal, then that channel completes processing quickly, implying that the overall response will be fast. If both channels are fed weak signals though (like a race of two weak and old horses against each other), even the faster one will take a long time. This gives rise to an overadditive interaction: MIC(t) < 0. If we assume a serial architecture with the same stopping rule, the total processing time is just the average time taken at each stage, assuming different processing orders are equally likely. Thus, when both channels are fed strong signals, the response is fast, and when both channels are low intensity, the response is slow. On mixed intensity trials, the faster channel is processed first half the time and the 84
slower channel is processed first in the remaining half; hence, the overall RT is the mean of these two completion times. This is consistent with an additive outcome: MIC(t) = 0. In an exhaustive architecture, the system must await processing of both signals (which practically means awaiting completion of processing of the low-intensity signal) before a decision is made. This means that, in a parallel race, the response is fast only when both channels are high intensity, meaning an underadditive interaction: MIC(t) > 0. Applying the same logic to serial processing reveals that the architecture is still additive (although completion times differ from those attained with a self-terminating stopping rule). Put succinctly, MIC is always zero in serial processing, positive in parallel processing with a minimum time stopping rule, and negative in parallel processing within an exhaustive architecture. survivor interaction contrast These distinctions at the level of the mean are useful, but their extension to the distribution level provides further constraints and information. The four RT distributions (i.e., different combinations of target salience) can be sliced into small time bins and a factorial plot, similar to those in Figure 4.8, can be derived at each time bin. The resulting MICs
elementary cognitive mechanisms
A. Over-additive
B. Additive
Low Color salience High Color salience
Mean RT (ms)
Mean RT (ms)
550 500 450 400
C. Under-additive
600
600
550
550 Mean RT (ms)
600
500 450 400
350
450 400
350 Low High Word salience
500
350 High Low Word salience
Low High Word salience
Fig. 4.8 Three different outcomes of the Mean Interaction Contrast in a Stroop experiment.
can then be plotted as a function of time, t, for the entire distribution. The mathematical predictions become simpler when applying the momentary values of the respective survivor functions, S(t), (rather than the MICs or CDFs), and the resulting curve is dubbed the Survivor Interaction Contrast (SIC; Townsend and Nozawa 1995). The SIC permits a distinction between different species of parallel models (e.g., race versus co-activation) and Townsend and Nozawa (1995) derived fully diagnostic functions for various combinations of architecture and stopping rule (summarized in Figure 4.9). selective influence Analysis at the distribution level can also provide supporting evidence on selective influence, a critical stipulation for derivations based on the factorial manipulations. Selective influence means that a given factor or manipulation affects only the intended process or channel. A distinct ordering of the survivor functions (for the four RT distributions) is predicted with the following condition in force: S(LOW, LOW) > S(LOW, HIGH), S(HIGH, LOW) > S(HIGH, HIGH) at all time t. Recall that the first factor is the salience of the word and the second factor is the salience of the color. Violation of this ordering compromises interpretations based on the interaction contrasts. One should note though that the presence of the predicted ordering does not prove the assumption of selective influence. It is a necessary but not sufficient condition (i.e., the same ordering could
have been obtained even if selective influence were violated). workload capacity Finally, consider briefly the other leg of the DFP, the factorial manipulation on the number of targets presented. This design is well suited to probe the sensitivity of the system to changes in workload. Recall the definition of the capacity coefficient that we gave in the section, Basic Issues Expressed Quantitatively, keeping in mind that the log survivor function is identical to the integrated hazard function, H(t). The numerator in our Stroop task example comprises the redundant-target trials in which the word RED is written in red ink. In the denominator, we have the same functions estimated from trials in which each channel appears in isolation. So: COR (t) =
H (RED in red )(t) H (RED in green)(t) + H (GREEN in red)(t)
Conceptually, this can be thought of as measuring the processing relationship between the “whole” in the numerator and the “sum of its parts” in the denominator. We can think of the capacity coefficient as measuring the workload capacity relative to the benchmark UCIP model. Reiterating the predictions from the section Basic Issues Expressed Quantitively, when channels are independent and parallel (as in the standard UCIP model), then the ratio is 1 and the system is unlimited capacity. When presenting two (multiple) targets concurrently slows the rate of processing, C(t) < 1, the system is limited capacity, which is less efficient than UCIP. Furthermore, if the system is limited to the extent features of response times
85
Parallel Minimum Time
Mean RT (ms)
MIC
SIC
LL LH MIC < 0 L
HL HH Time H Time
Mean RT (ms)
LL Parallel Exhaustive
HL LH HH MIC > 0
Serial Minimum Time
Mean RT (ms)
L LL LH
Mean RT (ms)
HL Time
HH MIC = 0 L
Serial Exhaustive
H
H
LL LH
HL
Time
HH MIC > 0 L
H
Fig. 4.9 Each combination of architecture and stopping rule has an MIC and SIC signature, as described in the text. See Townsend and Nozawa (1995) for derivations.
that performance with double targets is worse than with the faster of the two individual targets, or in terms of the capacity coefficient, then COR (t) ≤
log{MIN [SA (t), SB (t)]} log [SA (t) · SB (t)]
and the Grice inequality is violated (Townsend & Eidels, 2011). In this case, the system is of severely limited capacity. Conversely, if the presence of two targets speeds up the ability of the system to process each target, C(t) > 1, the system is supercapacity, which is more efficient than UCIP. Note that, when (target) signals are presented simultaneously to two channels, detection is usually faster than when a single signal is presented in one channel. This redundant target effect can derive from mere statistical facilitation (the minimum of two random variables has a smaller mean even than that of the faster of the individual random variables alone). However, if the 86
channels interact, a larger redundant target effect can be expected. If the former is the case, the Milleror race-model inequality (see the section Basic Issues Expressed Quantitatively) must hold: F (RED in red)(t) ≤ F (RED in green)(t) + F (GREEN in red)(t), where F denotes the RT CDFs for the respective redundant target and the two single target conditions. When this race-model inequality is violated, traditional thinking has been that all parallel race models are thereby falsified (also observe that satisfying it does not mean that the model is necessarily parallel). However, it is straightforward to construct parallel race models exhibiting super workload capacity that readily violate Miller’s race model bound (for examples, see Townsend & Wenger, 2004a). Such models can be created through mutual channel facilitation. It has been suggested that configural perception (e.g., words,
elementary cognitive mechanisms
faces, other Gestalts) may be explained by such parallel channel facilitation (e.g., Eidels, Townsend, & Pomerantz, 2008; Wenger & Townsend, 2001; 2006; but see Eidels et al., 2010 and Eidels, 2012). In their original development, Townsend and Nozawa (1995) showed that the Miller and Grice inequalities can be recast as statements about the capacity of the system. When both hold, the system capacity falls between those extremes and, thus, could be of moderately limited capacity, unlimited capacity, or moderately supercapacity. If Grice inequality is violated, the system is of severely limited capacity. If Miller’s inequality is violated, the system is of supercapacity (at that t). In a related development, Colonius (1990) has shown earlier that if the marginal probabilities F (Red in red)(t) and F (Green in red)(t) are invariant from the single target to the redundant target conditions, then the Miller and Grice inequalities correspond to maximum negative and positive dependence between parallel channels (see also Van Zandt, 2002). Subsequently, Townsend and Wenger (2004) showed that, interestingly, actual dynamic parallel systems whose channels interact by assisting one another (e.g., an increase in information in one channel leads to an increase in other channels) typically don’t produce invariant probabilities and do produce supercapacity and maybe violate Miller’s inequality (in apparent, but not real, contradiction of Colonius’ mathematically impeccable theorems). Conversely, typical dynamic parallel systems with mutually inhibiting channels evidence negative correlations, again cause failure of marginal invariance, and effect strong slow-downs of processing and possible violations of the Grice inequality.
Cognitive-Psychological Complementarity How well or poorly have these developments been incorporated into mainstream cognitive psychology? Without attempting an exhaustive analysis of the vast volume of research in current cognitive science, it is fair to say that the influence on substantive theorizing and experimentation of the mathematical developments has been uneven at best. Although early work on parallel and serial processing by Estes, Murdock, Schweickert, Dzhafarov, Egeth, Bernstein, Biederman, and Townsend (among others) has been well accepted in mainstream cognitive psychology of the time (early 1970s) and had a sobering influence on the field, this work has been subsequently ignored and superseded by the following groundless logic:
If, in a search task, the mean RT is linearly related to the number of stimuli and this function has a positive slope, then serial processing is implied. Theories based on this untenable statement engulfed the field to the extent that mathematical proofs and violations have been completely ignored. It is only during the past decade that cognitive psychology has finally overcome this detour from logic and mathematical rigor.
Ignoring Parallel-Serial Mimicry: The Case of a Linear RT-Set Size Function with a Positive Slope Treisman’s celebrated work on feature integration theory (e.g., Treisman & Gelade, 1980; Treisman & Gormican, 1988; Treisman & Schmidt, 1982) can serve as a convenient point of departure. This work suggests that when searching for a target that differs from nontargets in terms of a single conspicuous feature (e.g., color, orientation, or shape), the number of elements in the display matters little (feature search). However, when the target is defined in terms of a conjunction of features (such as a red vertical line among red tilted lines and green vertical lines), search time increases linearly with the number of elements in the display (conjunction search). In the theory, the main diagnostic tool to tell the two types of search apart is the slope of the respective RT-set size functions. The steep slopes obtained with conjunction targets are interpreted to implicate serial search, whereas the much shallower slopes with the one-dimension feature targets are interpreted to implicate parallel search. Feature integration theory has had a tremendous impact on attention research – as of the printing of this chapter, those three articles alone combine for close to 10,000 citations in the literature. And, this theory is mentioned as a major accomplishment in Treisman’s achievement of the highest scientific honor the United States can offer, the coveted National Medal of Science. The associated burgeoning literature helped to uncover valuable aspects of the cognitive processes engaged when people search for a target (whether in a cluttered computer screen or a crowded airport terminal). A less salutary outcome of this trend has been neglect of the possibility of mimicry. Many investigators have ignored proof that a putatively serial (mean)RT function can be mimicked by a parallel one and vice versa. For a trivial yet telling example, Townsend (1971a) was referenced by Treisman and Gelade (1980) but the citation concerned the makeup of the stimuli, features of response times
87
not the assay of parallel versus serial processing! Generations of cognitive psychologists appear to have been rendered oblivious to the developments in mathematical psychology on the importance and (im)possibility of distinguishing between parallel and serial processing based on straight line mean RT functions (cf. Townsend, 1971a; 1990a; Townsend & Wenger, 2004b). The difference between feature search and conjunction search impacted ensuing research to the extent that quite severe violations of the original pattern of results and conclusions, undermining basic tenets of the theory, were largely overlooked. Consider the study by Pashler (1987) for an instructive example. In relatively small displays of up to eight items, Pashler found the same slope for target-present and target-absent trials. This result is consistent with a serial exhaustive search rather than with the serial self-terminating search suggested by the theory. In a further experiment, Pashler added a second target on some portion of the trials and found a redundancy gain, the mean RT was faster when the display included two targets than when there was a single target. Redundancy gain is incompatible with both exhaustive and selfterminating varieties of serial models (cf. Egeth & Mordkoff, 1991; see also, Egeth, Virzi, & Garbart, 1984). These findings disconfirm the theory themselves, but the fact remains that they (along with a fair number of similar results) generated little traction at the time. The bright side to the story though is the increased use of the redundant target heuristics. From a modest start in the late 1960s (e.g., Bernstein, Blaken, Randolph, & Hughes, 1968; Egeth, 1966), this type of experimental design evolved into a major tool not only for investigations into visual search but for uncovering aspects of elementary cognitive processes in general. Nevertheless, outside of the work of a few investigators such as H. Egeth and colleagues and our own research, the visual search literature and that focusing on redundancy have unfortunately been largely nonintersecting. Egeth and Mordkoff (1991) used the redundant target design in tandem with Miller’s race model inequality as a further means of theoretical resolution. They concluded that the large violations of the inequality found were incompatible with any species of serial processing (and with certain varieties of parallel processing). In another interesting interrogation, Pashler and Badgio (1985) included trials in which all 88
items were visually degraded and found the effects of set size and degradation to be additive. The additive pattern clearly refutes models of serial identification. A conceptually similar study (indeed, complementary to that by Pashler & Badgio, 1985; cf. Pashler, 1998) was conducted by Egeth and Dagenbach (1991). In their study, the observers searched two-element displays in which each item could be visually degraded independently of the other element. The authors found a subadditive pattern, confirming again a parallel process of letter identification. Townsend and Nozawa (1995) investigated the redundant target paradigm along with a range of RT inequalities in a more general context, developing measures for the identification of different cognitive architectures. Survivor function interaction contrasts and processing capacity play key roles in this effort. In particular, Townsend and Nozawa showed that Miller’s inequality (among other RT inequalities) is actually a statement about the capacity of the process under test. What these developments demonstrate is the futility of drawing strong conclusions based on any simple RT (detection) function, if for no other reason than the brutal reality that many if not all such functions can be mathematically mimicked (e.g., Dzhafarov, 1993, 1997). A broader angle of attack is needed, one guided by a system of theorems and associated tools (the SFT proved serviceable in that role) within the framework of which absolute or mean RTs or density functions serve as points of departure. In this respect, Townsend and Wenger (2004a) generalized the earlier results to include conjunctive, rather than only disjunctive, decisions and illustrated their findings within the large class of interacting channels, parallel, linear stochastic systems. A very selective review of the major findings in the field of speeded visual search during the past three decades reveals that a wealth of stimulus properties (spatial distribution of the items, targetdistractor similarity, stimulus discriminability or task difficulty, practice, common shape and/or semantic category, and even particular attributes such as form and color) have increasingly replaced the number of stimuli (set size) as the variable of interest. The dichotomy, efficient-inefficient search, has been gradually superseding the dichotomy, parallel-serial processing. This course comprises a rather mixed bag. On the one hand, it reflects the growing recognition by cognitive scientists of the pertinent mathematical developments. In this respect, it took some 20 years
elementary cognitive mechanisms
for psychologists to finally conclude that models are needed that move “beyond Treisman’s original proposal that conjunction search always operate serially” (Pashler, 1998, p. 143). On the other hand, the same course also reflects a tendency of moving away from the issue of parallel versus serial processing altogether. This is unfortunate because, for all the difficulties involved, the issue is tractable and it is consequential for a wide range of cognitive processes. What we need is a naturally emerging integration of a given RT model and a certain cognitive theory. RT data, especially those embedded within a larger system, provide rich information about cognitive processing. Nevertheless, given the prevalence of mathematical and statistical equivalence, RTs, even when sustained by explicit models, will not always be diagnostic (cf. Van Zandt, 2002). It is at this juncture that the need for substantive theory becomes pellucid. Van Zandt (2002, p. 506) concludes that it is, therefore, “very important that RT analyses be conducted in the context of . . . mechanistic . . . explanations of the process under study.” Speeded visual search continues to fascinate investigators because it is such a ubiquitous human activity (from locating your baggage on the conveyor belt in the terminal to picking up your favorite Cheerios in the crowded aisle of the grocery store to finding your article in the list of those appearing in the journal). The theory proposed and periodically revised by Wolfe (1994; Cave & Wolfe, 1990), Guided Search, suggests that all kinds of searches (whether feature or conjunction) involve two consecutive stages. The first stage entails the simultaneous activation of all potential target features. Activity in the second stage is guided by the outcome of the first (i.e., the distribution of activations of the various features), testing serially combinations of activated features until one matches the target. The theory entails the notions of parallel and serial processing, but envisages situations in which either one can become gratuitous. Incorporating further flexible features, the theory is able to account reasonably well for a broad range of data. Another influential approach implicates similarity as the major determinant of search (Duncan & Humphreys, 1989). A little noticed aspect of the original Treisman experiments is that each nontarget shares a feature with the conjunction target (hence is similar to the target), whereas in feature search each nontarget is different from the
target. Duncan and Humphreys showed that search is easy for a distinctive target on the background of relatively uniform distractors but it is difficult on the background of highly diverse distractors. More recently, Ben-David and Algom (2009; see also Fific, Townsend, & Eidels, 2008) applied the machinery of SFT to uncover the influence of species of target and distractor similarity and sameness (physical, nominal, semantic) on various aspects of the architecture of the underlying process. The additive factors method itself has been incorporated into mainstream cognitive research to the extent that, more often than not, Sternberg is no longer even referenced. Of the multitude of studies, the sustained program of research by Besner and his associates (see Risko et al. 2010, for a recent contribution) stands out for the methodic application of the additive factors method to probe reading processes. For example, in the study by Borowsky and Besner (1993), context or meaning was found to interact with word frequency, on the one hand, and with stimulus quality, on the other hand, yet the latter two factors were additive. The pattern of joint effects was accommodated by a multistage activation model. Nonetheless, it might be well worth to employ the kinds of strategies outlined herein to falsify or accommodate the various types of models in a nonparametric fashion. When discussing models, we should address (but space does not allow us to truly address) the issue of the degree to which processing across different stages is discrete or in cascade. That is, we conceive of processing on different items or subsystems as occurring in a sequential manner but which may overlap in time. These are often referred to as continuous flow systems. Taylor (1976) was one of the first to proceed to a quantitative analysis of such models but others soon followed (e.g., McClelland, 1979; Miller, 1988). Let us just note that McClelland (1979) sanctioned the use of additive factor methodology to identify separable stages of processing, and that, separately, Ashby and Townsend (1980), Ashby (1982) and Roberts and Sternberg (1993), too, have demonstrated that purely cascaded models can produce additive effects on the mean RTs (provided certain boundary conditions are respected; see O’Malley & Besner, 2008). Schweickert and Mounts (1998) studied and made predictions from a quite general class of continuous flow systems. The issues are quite complicated and the interested reader should consult Logan (2002) on the broad distinction between discrete and continuous processing. More general and robust features of response times
89
metatheoretical effort is required to experimentally and effectively segregate such systems from ordinary parallel and serial systems.
Extending the Metatheory to Encompass Accuracy The great bulk of the theory enveloped in this chapter has featured response times. However, certain (included) theory-driven experimental designs were based on accuracy, such as the paradigms utilizing second-guesses. Recall that the secondguess strategy exploited the fundamental ability of parallel systems to represent many objects (items, features, channels) in partial states of completion as opposed to strict serial systems being confined to a single object in a partial state of completion. Compared with factorial strategies, these latter techniques have been woefully underused but, as noted, have seen some renewed activity recently. There are some other directions that should be broached.
Extending Capacity Theory to Include Accuracy: Moving Beyond Simple Speed-Accuracy Tradeoff The motivation to extend our capacity theory to include accuracy is twofold. Foremost, the more observable variables there are to constrain models and theories, the better. Additionally, measures that can simultaneously gauge and combine speed and accuracy can address important questions that neither alone is able to do. A specific manifestation of the relation of speed to accuracy arose in the 1960s and was called the speed-accuracy tradeoff (e.g., Pew, 1969; Pachella 1974; Swanson & Briggs, 1969; Yellott, 1971). The idea here is that one must be wary when observing say, a speed-up in one’s data and drawing perceptual or cognitive conclusions. The reason is that the error rate may have increased, perhaps reflecting an alteration of a decision criterion rather than an improvement in cognitive efficiency. It has been obligatory in cognitive psychology ever since, when either response times or accuracy changes, to check out how the other is varying. For instance, if the experimenter increases workload in a task and errors increase, she/he makes sure that response times stay the same or increase. It is then concluded that there is no speed-accuracy trade-off. This inference is unwarranted. Consider the following possibility: The workload is harder and errors increase, but the participants have 90
nonetheless also increased their decision criterion a modest amount, but not enough to offset the increase in errors. So, there has indeed been a speed-accuracy trade-off in the sense that even higher inaccuracy would have occurred had not the participants altered their criterion. This kind of subtlety requires a quantitative approach to be adjudicated. Our tactic has been to extend the response-time based workload-capacity function developed earlier (see Basic Issues Expressed Quantitatively and Theoretical Distinctions sections) to include accuracy (Townsend & Altieri, 2012). Detail is ruled out in this chapter, but the basic trick is to work out the predictions for the standard parallel class of models that are themselves enlarged to generate errors. In addition, for most of the speed-accuracy combinations, the value-loaded term capacity is inappropriate. For instance, is it higher capacity and, therefore, “better” to be fast and inaccurate or slow and accurate? For such reasons, it was necessary to introduce value-free terminology, in this case, the term assessment function called A(t). Then the assessment functions are assembled, as was the traditional statistical function, in a distribution-free and nonparametric manner. A simplified, symbolic formula is Pobs speed is fast and error occurs A (t) = Ppar (speed is fast and error occurs) where obs=observed from data and par=theoretical prediction on the basis of the standard parallel model. Furthermore, either the numerator or denominator can be decomposed into separate accuracy and conditional response time elements. Thus, Pobs(speed is fast and error occurs) = Pobs(speed is fast and error occurs) Pobs(error occurs). Then comparison of the observed and predicted quantities can aid in a number of useful theoretical inferences. For instance, initial analysis of an AND condition carried out by Ami Eidels (personal communication; see Townsend & Altieri, 2012) shows that the above A(t) > 1 indicating that the observed joint event of error-plus-speed in terms of response times and inaccuracy greatly exceeded that expected from the standard unlimited capacity independent parallel model. Furthermore, errorplus-slow tended to move in the other direction; that is, toward the A(t) = 1 line. Next, the correctplus-speed A(t) < 1 by a massive degree, whereas that for correct-plus-slow was a bit higher but still
elementary cognitive mechanisms
quite low. All this is predicted by parallel models, which are limited capacity. We next decomposed the statistics as just described and discovered that inaccuracy was considerably greater from that predicted by the standard model. This aspect then contributed to this specific A(t) > 1. However, it also transpired that, given that an error occurred, the likelihood of a fast response was also greater than that expected from the standard prediction. In fact, it appeared that overall, speed was greater than expected whether conditioned on correct or incorrect performance although most impressive when incorrect. Some tentative interpretations of these results, as well as for an OR condition, were offered in Townsend and Altieri (2012). However, it is simply the case that we know very little about how even parameterized models will reflect limited capacity effects when both accuracy as well as response times are analyzed. For instance, suppose, as is usually assumed, that changes in difficulty, when accomplished within trial blocks, indicate only changes in correct and incorrect processing speeds, not any change in decision criteria. The truth is, we don’t know to what extent an increase in errors, as in the AND data above, exhibit lower or higher (as earlier) speeds than would be predicted by the standard parallel model, all on the bases of only changes in processing rates. We anticipate that analysis of traditional models of response times and accuracy, such as parallel diffusions and races as well as serial models plus examination of many data sets, will show the way to a considerably enhanced understanding of speed and accuracy. As a single example of this type of progress, we mention a recent expansion of the accuracy-oriented general recognition theory (Ashby & Townsend, 1986) to stochastic parallel systems, which thereby includes response times as well and the patterns of confusion (Townsend, Houpt, & Silbert, 2012).
Model Mimicking in Psychological Science Psychological, cognitive, and brain sciences are, outside the most ludicrous parody of behaviorism, black box enterprises in which hypotheses about inner workings must be made and tested. The brain sciences are included here due to the brain’s amazing complexity. Even if we were without society’s ethical and moral strictures (and a good thing we are not), the brain’s machinery is so mysterious that even with behavioral and neuroscientific
approaches together, we are a little like a bunch of educated squirrels watching people drive, then poking about under the automobile’s hood at night and drawing profound squirrel-science inferences about auto design, functioning, and maintenance. Nonetheless, the resources of mathematical modeling, neuroscientific knowledge and techniques, and excellent behavioral and neuropsychological experimental designs offer the best we can hope for. Before moving on and as stated in our introduction, all the warnings about pitfalls associated with various aspects of mathematical modeling pale in comparison with verbal theorizing. Electrical engineering and computer science have long possessed rigorous quantitative bodies of knowledge; we could call them metatheories, of how to infer the internal mechanisms and dynamics from observable behavior. One of the most elegant of these is that associated with deterministic finite-state automata. Of course, when the number of states becomes infinite or random aspects intrude, things get more complicated. Yet even these accommodate mathematically rigorous and applicable methodologies. Naturally, the obverse side of the coin of “degree of uniqueness” of a prospective description of an observed system is the “class of mimicking models” (using our terminology). Even within the class of finite-state automata, however much data is collected, there will always be an equivalence class of machines able to predict said data. If the average graduate student in psychology were a little better prepared in mathematics, at least one course in such a topic would provide a beneficial and sobering message regarding the challenges facing them in their careers. The fact that even so diametrically opposed concepts as are embodied in parallel versus serial processing systems can readily be mathematically equivalent within common and popular paradigms should, along with the implicit forewarnings from other sciences, lead an incipient science like psychology to emphasize the study of such challenges in training their students and planning and carrying out their own research programs. Alas, that prescription does not seem likely to eventuate in the foreseeable future. However, mathematical psychology can strive to better train their own people in these matters, and conduct their own research accordingly. It should be evident that the use of metatheory to experimentally segregate large classes of models (e.g., all parallel models) within a certain domain comes up short with regard to specifying a highly features of response times
91
precise and detailed computational account. We have, therefore, proposed that researchers adopt a kind of successive sieve approach, where finergrain models are probed at each step. Thus, after determining, say, serial processing in a memory task, then one might begin to assess certain particular process distributions. This approach is complementary to Platt’s strong inference tactic, but not the same. Next, it is important to ponder the different echelons at which model and theory mimicking can take place, as we see in the following subsection.
Species of Mimicry First, there is mathematical equivalence such as we have discovered over many decades (e.g., Townsend 1972, 1976a; Townsend & Ashby, 1983; Townsend & Wenger, 2004b; Williams, Eidels, & Townsend, 2014). Little had been accomplished in terms of model identification in psychology outside of seminal work in mathematical learning theory by Greeno and Steiner (1968). Their rigorous efforts explored identifiability issues in learning and memory theories based on Markov chains. However, within the realm of closed-form proofs of mathematical equivalence, the work of Dzhafarov should be mentioned. Consider the quite general class of all models based on a race of two or more parallel channels. The Grice models (e.g., Grice et al., 1984) are members of this class. They place all the variance in the decision bounds. Other models such as the Smith and Vickers (1988; see also Vickers, 1979) accumulator model or the race models of Townsend and Ashby (1983) or Smith and Van Zandt (2000) place the variance in the state space of the channel activations. Dzhafarov (1993) proved that in the absence of assumption of specific distributions, these classes are mathematically equivalent within the usual experimental designs. As we have seen, with a little luck and lots of hard work, one may aspire to employ the very metatheory used to demonstrate model mimicking to aid in the design of experiments that test the model classes at deeper levels. Second, there is incomplete but exact mathematical mimicking to consider. For instance, a class of models might mimic a nonequivalent type of model at, say, the level of the first and/or second moments (i.e., mean and variance) but not be completely equivalent. A case in point is the prediction of mean response time additivity by standard serial models. As one 92
could expect, this prediction can be made by a huge class of alternative models, as proven by Townsend (1990b). The constraints put on the mimicking model class are extraordinarily weak. Just including variances helps a lot but, of course, does not totally remediate the problem (e.g., Schneider & Shiffrin, 1977; Townsend & Ashby 1983, chapters 6, 7). Third, there is mimicking by approximation. Though perhaps not so intriguing as mathematical equivalence, it is more widespread than the latter and at least as underappreciated by the average researcher. Examples of this type of mimicking in the present venue are the abililty of sequential, but not strictly serial, continuous flow dynamic models to predict approximate additivity of mean response times (e.g., McClelland 1979; see also Schweickert et al., 2012, chapter 6). It is likely that the third type of mimicking is that which threatens the bulk of model testing in the literature. Ordinarily, two, sometimes more, parameterized models are compared in their ability to fit or predict numerical data. Now, if psychology possessed a level of precision of measurement even remotely close to that in, say, physics or many areas of chemistry, then this policy might be quite optimal. Why test entire classes of models, when you can move directly to the precise model, with all its exact formulas and estimate parameters and thus be done? Unfortunately, the tolerance of psychological data is much too coarse for such a hope to be realized. Some help is afforded by way of comparison of models against one another, rather than simply fitting one’s favorite model to the data. However, even here, the possibility exists that one or the other model will simply fail to fit the data, even as well as another model, due only to the specific quantitative formulation of the psychological precepts, rather than the fundamental characteristics of the latter. For instance, consider an investigator who has correctly pinpointed the proper architecture, stopping rule, and so on for a task but failed to employ the valid associated stochastic process. Thus, perhaps the architecture is standard parallel, with each channel being described by a race between the correct and incorrect alternatives, the race distributions being gamma. Our hapless investigator inappropriately has selected Weibull distributions to describe each channel’s race. Meanwhile, a theoretical competitor might produce an incorrect specification of the architecture (e.g., serial), but employed a set of stochastic processes that adventitiously provide a superior fit.
elementary cognitive mechanisms
Another vital aspect of model testing in cognitive science, has always been the issue of whether one model is merely more complex than another and thus can reach sectors of the data space that are unavailable to its competitors. Attempts to ameliorate this challenge have long depended on the assumption that if two models possess equal numbers of parameters, they must be of equal complexity. This is rarely the case, as has been recognized for some time. Indeed, we have shown that large classes of models of classification (e.g., identification, categorization) can be much more falsifiable than other classes, though they possess a huge number of parameters (Townsend & Landon, 1983). A special case of some interest is the wellknown similarity choice model (Luce, 1963), which is much more flexible in its model fitting ability (often referred to as the champion model of human pattern recognition) than competitors, such as the overlap model (Townsend, 1971b) though the number of parameters is identical. A deep and vital antidote, when it can be brought to bear, is the quantification of model flexibility-to-fit data, through cutting edge theories of complexity (Myung & Pitt, 2009).This approach is able to quantify a model’s complexity and compensate for it in model comparisons. The main obstacle here is that so far, only models with certain types of pellucid specification are subject to this analysis, at least in realistic computational terms. Nonetheless, as computers’ powers continue to augment, this strain of technology offer hope for the future of complex psychological science. We have had little to say, beyond mere platitudes, concerning aspirations of merging principled quantitative modeling with cognitive and sensory neuroscience and especially neuroimaging. Some extremely credible senior psychological commentators are frankly skeptical of the contributive power of neuroscience and in particular, neuroimaging, to the cognitive sciences. We urge serious reflection on the observations of William Uttal (e.g., 2001) in this vein. Though we do not fully agree with his final inferences, we are convinced that his astute scrutiny can only serve to improve the science and its associated methodologies. The past several decades have seen a vibrant growth of modeling extending from basic psychophysics to higher mental processes to complex social phenomena. It seems inevitable that such multifaceted models, even when found to predict or fit data in an accurate manner, will be vulnerable to a broad spectrum of competitive, mimicking
alternative conceptions. We believe that fields endeavoring to explain or predict human behavior, including those based on neurophysiology, will progress faster by developing appropriate metatheories of mimicking, and how to best circumvent these experimentally.
Conclusions and the Future • The resurgence in the 1950s and 1960s of research attempting to identify mental mechanisms and its continuation today prove that cognitive psychology can be cumulative as well as scientific. • Mathematical models have made substantive ingress to cognitive psychology since 1950 and made many contributions to rigorous theory building and testing. • Nonetheless, even in the realm of mathematical modeling, mimicry of one model or
Box 2 Model Mimicking in Psychology From one vantage point, it might seem that model and theory mimicking reside in a fairly small and technical dominion of scientific psychology. Yet, in a broad sense mimicking is ubiquitous. Every time two theoretical explanations are in contention, it is because the data at hand are not decisively in favor of one over the other. That is, mimicking at one or more levels is occurring and it is up to the champions of either approach, or “innocent bystanders”, to invent new observations to resolve the issue. Within the realm of verbalized theory, what often happens is that the two theoretical structures evolve, due to ministrations from their advocates in order to conform to the latest data. The end result is that the theories may end up still handling the larger corpus of data, each being significantly more complex (and thus more difficult to falsify), and yet remaining empirically indistinguishable. A famous case in point is the decades-long clash of the more behavioristic theory of Clark Hull versus the more cognitive theory of Edwin Tolman. Though fundamentally distinct in their theoretical foundations, their long-lasting struggle must be said to have eventually faded away inconclusively. Mathematical modeling and the explicit study of model mimicking seems a promising remedy for such ailments.
features of response times
93
class of models by other models, poses a formidable challenge to science building in this complex arena. • Metatheory is, in our approach, a theoretical and quantitative enterprise that attempts to formulate highly general mathematical characterizations of psychological notions in such a way as to point toward development of robust experimental methodologies for systems identification. • The reviewed body of research on metatheory has led to redoubtable methodologies for assessing various strategic aspects of elementary cognition in a manner that is resistant to model mimicking. Most of the new theory and technology is founded on distribution and parameter-free theorems and methods. • As of now, the metatheory and associated methodologies are largely segregated into those resting on RTs as observable variables and those relying on patterns of accuracy as observable variables. • A major goal for the immediate future is to create a unified theory that merges both RTs as well as accuracy. First steps have been taken in that direction. • Very little is known about the perils from model mimicking to incremental science in more complex spheres of cognitive science. It may be that such theoretical research will be indispensable to future methodologies, if cognitive science is to avoid devolvement into a maundering, largely inconclusive field.
Notes 1. Of course, in-between cases are often used, for instance, that of the highly popular single-target-among-distractors. In such a design, the processor may cease as soon as the target is located. 2. If t A , t B are independent random variables then the previous statement will hold true. However, if we assign distinct random variables to the actual processing times in parallel and serial systems, then very broad questions can be asked and answered with regard to vital parallel-serial mimicking issues (Townsend & Ashby 1983, chapter 1). 3. From Eq. 1 we can also derive the pdf, f(t), of an independent race model with two parallel channels (i.e. the probability that channel A or channel B finish at time t), which is fAB (t) = fA (t) · SB (t) + SA (t) · fB (t). 4. Stochastic serial models are a bit more complex, since one needs to take into account the order by which process occur (e.g., channel A before B or vice versa). Full treatment is given in Townsend and Ashby (1983).
94
Glossary Across-stage independence: Assumes the independence of intercompletion times in serial models. It is defined as the property that the probability density function of two or more stages of processing is the product of the component single-stage density functions. log [S
(t)]
AB , is a Capacity coefficient: COR (t) = log [S (t)·S A B (t)] measure for processing efficiency as workload (number of signals to process) increases. C(t)=1 indicates unlimited capacity – performance is identical to that of a benchmark UCIP model (see later). C(t) < 1 and C(t) > 1 indicate limited and super-capacity, respectively. COR (t) is appropriate for OR tasks, while a comparable index, CAND (t), exists for the AND case, with a different formula but similar interpretation.
Cumulative distribution function (cdf ): F(t) = p(T≤t), gives the probability that the process of interest is finished before or at time t. Deterministic process: Always yields a fixed result, such that the effect or phenomenon we observe has no variability. Exponential distribution: A probability distribution that describes the time between events in a Poisson process. It is very useful in response time modelling, and has the form f(t)=ve−vt , where v is the rate parameter. It also has the “memory-less” property, meaning that the likelihood of an event to occur in the next instance of time is independent of how much time had already passed. Grice inequality: FAB (t) ≥ MAX [FA (t), FB (t)]. This inequality states that performance on double-target trials, FAB (t), should be faster than (or at least as fast as) that in the faster of the single-target channels. If this inequality is violated, the simultaneous processing of two target-signals is highly inefficient and the system is very limited capacity. For instance, if FA (t) = FB (t) then COR (t) < 12 . The special case when COR (t) = 1/2 is referred to as fixed capacity. Intercompletion time: The time required for a stage of processing to be completed. In a serial model, the intercompletion times are just the processing times. Mean Interaction Contrast (MIC): A test statistic for the interaction between two factors with two levels each, which allows assessment of architecture and stopping rule from mean RTs. Calculated as the difference between differences of mean RTs in a factorial experiment, MIC = (RTLL −RT HL ) − (RT LH −RT HH ), where RT is the mean RT and L and H denote low and high salience conditions, respectively. Because all stopping rules for serial models predict that MIC = 0, they cannot be distinguished for serial models by MIC. Probability density function (pdf ): f(t) = p(T = t), gives the likelihood that some process that takes random time T to complete will actually be finished at time t. Race model (Miller’s) inequality: FAB (t) ≤ FA (t) + FB (t). This inequality states that the cumulative distribution function for double-target displays, FAB (t), cannot exceed the sum of the single-target cumulative distribution functions if processing is a race between parallel channels, with the added constraint that the marginal distributions for A and B do not change from when one of the two channels is
elementary cognitive mechanisms
Glossary presented to when both are presented. This stipulation is known as context invariance. When the upper bound implied in the inequality is violated, capacity must be super; that is, COR (t) > 1. “Stage” of processing: The time from one item finishing processing to the next item finishing processing. Stochastic independence: Two events are independent if the occurrence of one does not affect the probability of the other. This concept can be expressed in terms of the joint pdfs, fAB (tA , tB ) = fA (tA ) · fB (tB ), which means that the joint density of processes A and B both finishing at time t is equal to the product of the probability of A finishing at time tA and the probability of B finishing at time tB . Stochastic process: The events cannot be characterized by fixed values and should be represented by a random variable. A random variable does not have a single, fixed value but rather takes a set of possible values, with their likelihood characterized by a probability distribution. Survivor function: S(t) = 1 – F(t) = p(T > t), This function is the complement of the cdf, and tells us the probability that the process of interest had not yet finished by time t. Survivor Interaction Contrast [SIC(t)]: Same as MIC but calculated for survivor functions, S(t), rather than mean RT at each time bin of t. SIC(t) = [SLL (t)- SHL (t)] – [SLH (t)SHH (t)]. The SIC(t) functions predict distinctive curves for serial and parallel models for various stopping rules. UCIP model: A processing model characterized by Unlimited Capacity and Independent Parallel processing channels. Within-stage independence: The statistical independence of intercompletion times across two or more parallel channels in the same stage.
References Ashby, F. G. (1982). Deriving exact predictions from the cascade model. Psychological Review, 89, 599–607. Ashby, F. G., & Townsend, J. T. (1980). Decomposing the reaction time distribution: Pure insertion and selective influence revisited. Journal of Mathematical Psychology, 21, 93–123. Ashby, F. G., & Townsend, J. T. (1986). Varieties of perceptual independence. Psychological Review, 93, 154–179. Atkinson, R. C., Holmgren, J. E., & Juola, J. F. (1969). Processing time as influenced by the number of elements in a visual display. Perception&Psychophysics, 6, 321–326. Baddeley, A. D., & Ecob, J. R. (1973). Reaction time and short term memory: Implications of repetition effects for the high-speed exhaustive scan hypothesis. Quarterly Journal of Experimental Psychology, 25, 229–240. Bechtel, W., & Richardson, R. C. (1998). Vitalism. In E. Craig (Ed.), Encyclopedia of philosophy. London, England: Routledge. Ben-David, B. M., & Algom, D. (2009). Species of redundancy in visual target detection. Journal of Experimental Psychology: Human Perception & Performance, 35, 958–976.
Bernstein, I. H. (1970). Can we see and hear at the same time? Some recent studies of the intersensory facilitation of reaction time. Acta Psychologica, 33, 21–35. Bernstein, I. H., Blake, R., Randolph, R., & Hughes, M. H. (1968). Effects of time and event uncertainty upon sequential information processing. Perception & Psychophysics, 3, 177–184. Blaha, L. M., & Townsend, J. T. (2006, May). Parts to wholes: Configural learning fundamentally changes the visual information processing system. Vision Sciences Society Annual Meeting, Sarasota, Florida. Borowsky, R., & Besner, D. (1993). Visual word recognition: A multistage activation model. Journal of Experimental Psychology: Learning, Memory, & Cognition, 19, 813–840. Cave, K. R., & Wolfe, J. M. (1990). Modeling the role of parallel processing in visual search. Cognitive Psychology, 22, 225– 271. Colonius, H. (1990). Possibly dependent probability summation of reaction time. Journal of Mathematical Psychology, 34, 253– 275. Colonius, H., & Townsend, J. T. (1997). Activation-state representation of models for the redundant signals effect. In A. A. J. Marley (ed.), Choice, decision, and measurement: Essays in honor of R. Duncan Luce, Mahwah, NJ: Erlbaum. Colonius, H., & Vorberg, D. (1994). Distribution inequalities for parallel models with unlimited capacity. Journal of Mathematical Psychology, 38, 35–58. Corcoran, D. W. J. (1971). Pattern recognition. Middlesex, PA: Penguin. Diederich, A., & Colonius, H. (1991). A further test of the superposition model for the redundant-signals effect in bimodal detection. Perception & Psychophysics, 50, 83–86. Donders, F. C. (1868). Die Schnelligkeit Psychischer Processe. Archiv fur Anatomie und Physiologie und Wissenschaflitche Medizin, 657–681. Duncan, J., & Humphreys, G. W. (1989). Visual search and stimulus similarity. Psychological Review, 96, 433–458. Dzhafarov, E. N. (1993). Grice representability of response time distribution families. Psychometrika, 58, 281–314. Dzhafarov, E. N. (1997). Process representations and decompositions of response times. In A. A. J. Marley (Ed.), Choice, decision, and measurement: Essays in honor of R. Duncan Luce (pp. 255–278). Mahwah, NJ: Erlbaum. Dzhafarov, E. N. (2003). Selective influence through conditional independence. Psychometrika, 68, 7–25. Egeth, H. E. (1966). Parallel versus serial processes in multidimensional stimulus discrimination. Perception & Psychophysics, 1, 245–252. Egeth, H. E., & Dagenbach, D. (1991). Parallel versus serial processing in visual search: Further evidence from subadditive effects of visual quality. Journal of Experimental Psychology: Human Perception & Performance, 17, 551–560. Egeth, H. E., Virzi, R. A., & Garbart, H. (1984). Searching for conjunctively defined targets. Journal of Experimental Psychology: Human Perception & Performance, 10, 32–39. Egeth, H. E., & Mordkoff, J. T. (1991). Redundancy gain revisited: Evidence for parallel processing of separable dimensions. In J. Pomerantz and G. Lockhead (Eds.), The perception of structure (pp. 131–143). Washington, DC: APA.
features of response times
95
Eidels, A. (2012). Independent race of colour and word can predict the Stroop effect. Australian Journal of Psychology, 64, 189–198. Eidels, A., Townsend, J. T., & Algom, D. (2010). Comparing perception of Stroop stimuli in focused versus divided attention paradigms: Evidence for dramatic processing differences. Cognition, 114, 129–150. Eidels, A., Houpt, J. W., Altieri, N., Pei, L., & Townsend, J. T. (2011). Nice guys finish fast and bad guys finish last: Facilitatory vs. inhibitory interaction in parallel systems. Journal of Mathematical Psychology, 55, 176–190. Eidels, A., Townsend, J. T., & Pomerantz, J. R. (2008). Where similarity beats redundancy: The importance of context, higher order similarity, and response assignment. Journal of Experimental Psychology: Human Perception & Performance, 34, 1441–1463. Estes, W. K., & Taylor, H. A. (1966). Visual detection in relation to display size and redundancy of critical elements. Perception&Psychophysics, 1, 9–16. Fancher, R. E. (1990). Pioneers of psychology. New York, NY: Norton. Fific, M., Townsend, J. T., & Eidels, A. (2008). Studying visual search using systems factorial methodology with target– distractor similarity as the factor. Perception & Psychophysics, 70, 583–603. Greeno, J. G., & Steiner, T. E. (1968). Comments on “Markovian processes with identifiable states´lGeneral considerations and applications to all-or-none learning. Psychometrika, 33(2), 169–172. Grice, G. R., Canham, L., & Gwynne, J. W. (1984). Absence of a redundant-signals effect in a reaction time task with divided attention. Attention, Perception, & Psychophysics, 36 (6), 565– 570. Hawking, S. W. (1988). A brief history of time. London, England: Bantam. James, W. (1890). Principles of psychology, 2 vols. New York, NY: Dove. Johnson, D. M. (1955). The psychology of thought and judgment. New York, NY: Harper. Kahneman, D. (1973). Attention and effort. Englewood Cliffs, NJ: Prentice-Hall. Kujala, J. V., & Dzhafarov, E. N. (2008). Testing for selectivity in the dependence of random variables on external factors. Journal of Mathematical Psychology, 52(2), 128–144. Külpe, O. (1895). Grundriss der psychologie [Outline of psychology], (translated by E. B. Titchener). New York, NY: Macmilan. Lacmann, R., Lachman, J. L., & Butterfield, E. C. (1979). Cognitive psychology and information processing: An introduction. Hillsdale: Erlbaum. Laming, D. (1968). Information theory of choice-reaction times. London, England: Academic. Logan, G. D. (2002). Parallel and serial processing. In J. Wixted (Ed.), Stevens’ handbook of experimental psychology (Vol 4 pp. 271–300). New York, NY: Wiley. Luce, R. D. (1963). A threshold theory for simple detection experiments. Psychological Review, 70, 61–79. Luce, R. D. (1986). Response times. New York, NY: Oxford University Press.
96
Marx, M. H., & Cronan-Hillix, W. A. (1987). Systems and theories in psychology. New York, NY: McGraw-Hill. McClelleand, J. L. (1979). On the time relations of mental processes: An examination of systems of processes in cascade. Psychological Review, 86, 287–330. McClelland, J. L., & Rumelhart, D. E. (1981). An interactive activation model of context effects in letter perception: I. An account of basic findings. Psychological Review, 88 (5), 375. McClelland, J. L., Rumelhart, D. E., & the PDP research group. (1986). Parallel distributed processing: Explorations in the microstructure of cognition (Vol. II). Cambridge, MA: MIT Press. Melara, R. D., & Algom, D. (2003). Driven by information: A tectonic theory of Stroop effects. Psychological Review, 110(3), 422–471. Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review, 63, 81–97. Miller, J. O. (1978). Multidimensional same-different judgments: Evidence against independent comparisons of dimensions. Journal of Experimental Psychology: Human Perception & Performance, 4, 411–422. Miller, J. O. (1982). Divided attention: Evidence for coactivation with redundant signals. Cognitive Psychology, 14, 247–279. Miller, J. O. (1988). Discrete and continuous models of information processing: Theoretical distinctions and empirical results. Acta Psychologica, 67, 191–257. Murdock, B. B. (1971). Four channel effects in short-term memory. Psychonomic Science, 24, 197–198. Murray, D. J. (1988). A history of western psychology. Enaglewood Cliffs, NJ: Prentice Hall. Myung, J. I., & Pitt, M. A. (2009). Optimal experimental design for model discrimination. Psychological Review, 116, 499–518. Nickerson, R. S. (1966). Response times with a memorydependent decision task. Journal of Experimental Psychology, 72(5), 761–769. doi:10.1037/h0023788 Neufeld, R. W., & McCarty, T. S. (1994). A formal analysis of stressor and stress−proneness effects on simple information processing. British Journal of Mathematical and Statistical Psychology, 47 (2), 193–226. Nietzsche, F. (1873). Über Wahrheit und Lüge im auSSermoralischen Sinn (On Truth and Lies in an Extra-Moral Sense). In Friedrich Nietzsche (Ed.), The birth of tragedy and other writings. Cambridge, England: Cambridge University Press, 1999. O’Malley, S., & Besner, D. (2008). Reading aloud: Qualitative differences in the relation between stimulus quality and word frequency as a function of context. Journal of Experimental Psychology: Learning, Memory, & Cognition, 34, 1400–1411. Qualitative Pachella, R. (1974). The interpretation of reaction time in information processing research. In B. H. Kantowitz (Ed.), Human Information Processing: Tutorials in performance and cognition. Hillsdale, NJ: Erlbaum. Pashler, H. (1987). Detecting conjunctions of color and form: Reassessing the serial search hypothesis. Perception & Psychophysics, 41, 191–201.
elementary cognitive mechanisms
Pashler, H. (1998). The psychology of attention. Cambridge, MA: MIT Press. Pashler, H., & Badgio, P. C. (1985). Visual attention and stimulus identification. Journal of Experimental Psychology: Human Perception & Performance, 11, 105–121. Pew, R. W. (1969). The speed-accuracy operating characteristic. Attention and Performance II. Amsterdam, Netherlands: North-Holland, 1969. Rakover, S. (2007). To Understand a Cat: Methodology and Philosophy. Amsterdam, Netherlands: John Benjamins. Ratcliff, R. A. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108. Ratcliff, R., & Smith, P. L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111(2), 333. Risko, E. F., Stolz, J. A., & Besner, D. (2010). Spatial attention modulates feature cross talk in visual word processing Attention Perception & Psychophysics, 72, 989–998. Roberts, S., & Sternberg, S. (1993). The meaning of additive reaction-time effects: Tests of three alternatives. In D. E. Meyer and S. Kornblum (Eds.) Attention and performance XIV: Synergies in experimental psychology, artificial intelligence, and cognitive neuroscience (pp. 611–653). Cambridge, MA: MIT Press. Ross, B. H., & Anderson, J. R. (1981). A test of parallel versus serial processing applied to memory retrieval. Journal of Mathematical Psychology, 24(3), 183–223. Schneider, W., & R. M. Shiffrin. (1977). Controlled and automatic human information processing: 1. Detection, search, and attention. Psychological Review, 84, 1–66 Schwarz, W. (1994). Diffusion, superposition, and the redundant-targets effect. Journal of Mathematical Psychology, 38, 504–520. Schweickert, R. (1978). A critical path generalization of the additive factor method: Analysis of a Stroop task. Journal of Mathematical Psychology, 18(2), 105–139. Schweickert, R. (1982). Scheduling decisions in critical path networks of mental processes. Paper presented in the meeting of the Society for Mathematical Society. Schweickert, R., Fisher, D.L., & Sung, K. (2012). Discovering cognitive architecture by selectively influencing mental processes. Advanced Series on Mathematical Psychology. Singapore: World Scientific. Schweickert, R., & Mounts, J. R. W. (1998). Additive effects of factors on reaction time and evoked potentials in continuous flow models. In C. Dowling, F. Roberts, and P. Theunes (Eds.) Recent progress in mathematical psychology (pp. 311– 327). Mahwah, NJ: Erlbaum. Schweickert, R., & Townsend, J. T. (1989). A trichotomy method: Interactions of factors prolonging sequential and concurrent mental processes in stochastic PERT networks. Journal of Mathematical Psychology, 33, 328–347. Smith, G. A. (1977). Studies of compatibility and models of choice reaction time. In S. Dornic (Ed.), Attention and performance VI, Hillsdale, NJ: Erlbaum. Smith, P. L., & Van Zandt, T. (2000). Time-dependent Poisson counter models of response latency in simple judgment. British Journal of Mathematical and Statistical Psychology, 53, 293–315.
Smith, P. L., & Vickers, D. (1988). The accumulator model of two-choice discrimination. Journal of Mathematical Psychology, 32, 135–168. Snodgrass, J. G., Luce, R. D., & Galanter, E. (1967). Some experiments on simple and choice reaction time. Journal of Experimental Psychology, 75, 1–17. Snodgrass, J. G., & Townsend, J. T. (1980). Comparing serial and parallel models: Theory and implementation. Journal of Experimental Psychology: Human Perception & Performance,6, 330–354. Sperling, G. (1960). The information available in brief visual presentations. Psychological Monographs, 74, 1–29. Sternberg, S. (1966). High speed scanning in human memory. Science, 153, 652–654. Sternberg, S. (1969a). The discovery of processing stages: Extensions of Donders’ method. In W. D. Koster (Ed.), Attention and performance II. Acta Psychologica, 30, 276–315. Sternberg, S. (1969b). Memory scanning: Mental processes revealed by reaction-time experiments. American Scientist, 57, 421–457. (Reprinted in J. S. Antrobus (Ed.), Cognition and Affect (pp. 13–58). Boston: Little, Brown). Strayer, D. L., & Johnston, W. A. (2001). Driven to distraction: Dual-task studies of simulated driving and conversing on a cellular telephone. Psychological science, 12, 462–466. Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of experimental psychology, 18(6), 643. Swanson, J. M. & Briggs, J. E. (1969). Information processing as a function of speed vs. accuracy. Journal of Experimental Psychology, 81, 223–289. Taylor, D. A. (1976). Stage analysis of reaction time. Psychological Bulletin, 83, 161–191. Theios, J., Smith, P. G., Haviland, S. E., Traupmann, J., & Moy, M. C. (1973). Memory scanning as a serial self-terminating process. Journal of Experimental Psychology, 97, 323–336. Titchener, E. B.(1905). Experimental psychology. New York, NY: Macmillan. Townsend, J. T. (1969). Mock parallel and serial models and experimental detection of these. Purdue centennial symposium on information processing. Purdue, IN: Purdue University Press. Townsend, J. T. (1971a). A note on the identifiability of parallel and serial processes. Perception & Psychophysics, 10, 161–163. Townsend, J. T. (1971b). Theoretical analysis of an alphabetic confusion matrix. Perception & Psychophysics, 9, 40–50. Townsend, J. T. (1972). Some results concerning the identifiability of parallel and serial processes. British Journal of Mathematical and Statistical Psychology, 25, 168–199. Townsend, J. T. (1974). Issues and models concerning the processing of a finite number of inputs. In B. H. Kantowitz (Ed.), Human information processing: Tutorials in performance and cognition (pp. 133–168). Hillsdale, NJ: Erlbaum. Townsend, J. T. (1976a). Serial and within-stage independent parallel model equivalence on the minimum completion time. Journal of Mathematical Psychology, 14, 219–238. Townsend, J. T. (1976b). A stochastic theory of matching processes. Journal of Mathematical Psychology, 14, 1–52. Townsend, J. T. (1984). Uncovering mental processes with factorial experiments. Journal of Mathematical Psychology, 28, 363–400.
features of response times
97
Townsend, J. T. (1990a). Serial vs. parallel processing: Sometimes they look like tweedledum and tweedledee but they can (and should) be distinguished. Psychological Science, 1, 46–54. Townsend, J. T. (1990b). A potpourri of ingredients for Horse (Race) Soup. (Technical Report #32). Cognitive Science Program. Bloomington, Bloomington, IN. Townsend, J. T., & Altieri, N. (2012). An accuracy–response time capacity assessment function that measures performance against standard parallel predictions. Psychological Review, 119(3), 500–516. Townsend, J. T., & Ashby, F. G. (1978). Methods of modeling capacity in simple processing systems. In J. Castellan and F. Restle (Eds.), Cognitive Theory Vol. III (pp. 200–239). Hillsdale, NJ: Erlbaum. Townsend, J. T., & Ashby, F. G. (1983). The stochastic modeling of elementary psychological processes. Cambridge, England: Cambridge University. Townsend, J. T., & Colonius, H. (1997). Parallel processing response times and experimental determination of the stopping rule. Journal of Mathematical Psychology, 41, 392– 397. Townsend, J.T., & Eidels, A., (2011). Workload capacity spaces: A unified methodology for response time measures of efficiency as workload is varied. Psychonomic Bulletin & Review, 18, 659–681. Townsend, J. T., & Evans, R. (1983). A systems approach to parallel-serial testability and visual feature processing. In H. G. Geissler (Ed.), Modern Issues in Perception (pp. 166– 189). Berlin: VEB Deutscher Verlag der Wissenschaften. Townsend, J. T., & Honey, C. J. (2007). Consequences of base time for redundant signals experiments. Journal of Mathematical Psychology, 51, 242–265. Townsend, J. T., Houpt, J. & Silbert, N. H. (2012). General recognition theory extended to include response times: Predictions for a class of parallel systems. Journal of Mathematical Psychology. Available at: http://dx.doi.org/10.1016/j.jmp. 2012.09.001 Townsend, J. T., & Landon, D. E. (1983). Mathematical models of recognition and confusion in psychology. International Journal of Mathematical Social Sciences, 4, 25–71. Townsend, J. T., & Nozawa, G. (1995). Spatio-temporal properties of elementary perception: An investigation of parallel, serial and coactive theories. Journal of Mathematical Psychology, 39, 321–360. Townsend, J. T., & Snodgrass, J. G. (1974). A serial vs. parallel testing paradigm when “same” and “different” comparison rates differ. Paper presented to Psychonomic Society, Boston, MA. Townsend, J. T., & Van Zandt, T. (1990). New theoretical results on testing self-terminating vs. exhaustive processing in rapid search experiments. In H. G. Geissler, M. H. Müller, and W. Prinz (Eds), Psychophysical explorations of mental structures. Stuttgart: Hogrefe and Huber Publishers. Townsend, J. T., & Wenger, M. J. (2004a). A theory of interactive parallel processing: New capacity measures and predictions for a response time inequality series. Psychological Review, 111, 1003–1035.
98
Townsend, J. T., & Wenger, M. J. (2004b). The serial parallel dilemma: A case study in a linkage of theory and method. Psychological Bulletin& Review, 11, 391–418. Treisman, A. M., & Gelade, G. (1980). A Feature-integration theory of attention. Cognitive Psychology, 12, 97–136. Treisman, A. M., & Gormican, S. (1988). Feature analysis in early vision: Evidence from search asymmetries. Psychological Review, 95, 15–48. Treisman, A. M., & Schmidt, H. (1982). Illusory conjunctions in the perception of objects. Cognitive Psychology, 14, 107– 141. Uttal, W. R. (2001). The new phrenology. Cambridge, MA: MIT Press. van der Heiden, A. H. C. (1975). Some evidence for limited capacity parallel self-terminating process in simple visual search tasks. Acta Psychologica, 39, 21–41. Van Zandt, T. (1988). Testing Serial and Parallel Processing Hypotheses in Visual Whole Report Experiments. (Master’s thesis). Indiana University Van Zandt, T. (2002). Analysis of response time distributions. In J. Wixted (Ed.), Stevens’ handbook of experimental psychology, (Vol 4, pp. 461–516). New York, NY: Wiley. Van Zandt, T.,& Townsend, J. T. (1993). Self-terminating vs. exhaustive processes in rapid visual, and memory search: An evaluative review. Perception & Psychophysics, 53(5) 563–580. Vickers, D. (1979). Decision processes in visual perception. New York, NY: Academic. Vollick, D. N. (1994). Stochastic models of encoding-latency means and variances in paranoid schizophrenia. (Doctoral dissertation). Retrieved from ProQuest Dissertations and Theses. (Accession Order No. NN93247). University of Western Ontario, Canada. Watson, J. M., & Strayer, D. L. (2010). Supertaskers: Profiles in extraordinary multitasking ability. Psychonomic Bulletin & Review, 17, 479–485. Welford, A. T. (1980). (Ed.), Reaction Times. London: Academic. Wenger, M. J.,& Townsend, J. T. (2001). Computational, Geometric, and Process Issues in Facial Cognition: Progress and Challenges. Mahwah, NJ: Erlbaum. Wenger, M. J.,& Townsend, J. T. (2006). On the costs and benefits of faces and words: Process characteristics of feature search in highly meaningful stimuli. Journal of Experimental Psychology: Human Perception & Performance, 32, 755–779. Williams, P., Eidels, A., & Townsend, J. T. (2014). The resurrection of Tweedledum and Tweedledee: Bimodality cannot distinguish serial and parallel processes. Psychonomic Bulletin & Review. In press. Wolfe, J. M. (1994). Guided search 2.0: A revised model of visual search. Psychonomic Bulletin & Review, 1, 202–238. Woodworth, R. S. (1938). Experimental Psychology. New York: Holt. Wundt, W. (1892). Die geschwindigkeit des gedankens (The velocity of thought).Die Gartenlaube, 26, 263–265. Yellott, J. I. (1971). Correction for fast guessing and the speed-accuracy trade-off in choice reaction time. Journal of Mathematical Psychology, 8, 159–199.
elementary cognitive mechanisms
CHAPTER
5
Computational Reinforcement Learning
Todd M. Gureckis and Bradley C. Love
Abstract
Reinforcement learning (RL) refers to the scientific study of how animals and machines adapt their behavior in order to maximize reward. The history of RL research can be traced to early work in psychology on instrumental learning behavior. However, the modern field of RL is a highly interdisciplinary area that lies that the intersection of ideas in computer science, machine learning, psychology, and neuroscience. This chapter summarizes the key mathematical ideas underlying this field including the exploration/exploitation dilemma, temporal-difference (TD) learning, Q-learning, and model-based versus model-free learning. In addition, a broad survey of open questions in psychology and neuroscience are reviewed. Key Words: reinforcement learning, explore/exploit dilemma dynamic, decision-making
Introduction There are few general laws of behavior, but one may be that humans and other animals tend to repeat behaviors that have led to positive outcomes in the past and avoid those associated with punishment or pain. Such tendencies are on display in the behavior of young children who learn to avoid touching hot stoves following a painful burn, but behave in school when rewarded with toys. This basic principle exerts such a powerful influence on behavior, it manifests throughout our culture and laws. Behaviors that society wants to discourage are tied to punishment (e.g., prison time, fines, consumption taxes), whereas behaviors society condones are tied to positive outcomes (e.g., tax credits for fuel-efficient cars). The scientific study of how animals use experience to adapt their behavior in order maximize rewards is known as reinforcement learning (RL). Reinforcement learning differs from other types of learning behavior of interest to psychologists (e.g., unsupervised learning, supervised learning) since it deals with learning from feedback that is largely evaluative rather than corrective. A restaurant diner doesn’t necessarily learn that eating at a particular
business is “wrong,” simply that the experience was less than exquisite. This particular aspect of RL – learning from evaluate rather than corrective feedback – makes it a particularly rich domain for studying how people adapt their behavior based on experience. The history of RL can be traced to early work in behavioral psychology (Thorndike, 1911; Skinner, 1938). However, the modern field of RL is a highly interdisciplinary area at the crossroads of computer science, machine learning, psychology, and neuroscience. In particular, contemporary research on RL is characterized by detailed behavioral models that make predictions across a wide range of circumstances, as well as neuroscience findings that have linked aspects of these models to particular neural substrates. In many ways, RL today stands as one of the major triumphs of cognitive science in that it offers an integrated theory of behavior at the computational, algorithmic, and implementational (i.e., neural) levels (Marr, 1982). The purpose of this chapter is to provide a general overview of RL and to illustrate how this approach is used to understand decisionmaking and learning behavior in humans and
99
other animals. We begin in the following section with a historical perspective, tracing the roots of contemporary reinforcement-learning research to early work on learning behavior in animals. In the section A Computational Perspective on Reinforcement Learning we introduce a general computational formalism for understanding RL, based largely on the work of Sutton & Barto (1998). Along the way we discuss some of the critical aspects of RL including the trade-off between exploration and exploitation, credit assignment, and errordriven learning. The section on Neural Correlates of RL focuses on the neural basis of RL. The section Contemporary Issues in RL Research describes a variety of contemporary research questions as well as open areas for future investigation.
A Historical Perspective
Time to escape (seconds)
The early roots of RL research can be traced to the work of Thorndike (1911). Thorndike studied learning behavior in animals, most notably cats. In perhaps his most well-known experiment, he placed a cat into a specially designed “puzzle box,” which could be opened from the inside via various latches, strings, or other mechanisms (see Figure 5.1). The cat was encouraged to escape the box by the presentation of food on the outside of the box. The key issue for Thorndike was how long it would take the cat to escape. Once placed in this situation, the cat usually began experimenting with different ways to escape (pressing levers, pulling cords, pawing at the door). After some time and effort, they would stumble on the particular mechanism that opened the cage. This process was repeated across a number of trials, each time recording the total time until the cat
escaped. Over time, the cats would learn that particular actions (e.g., pressing a lever, or pulling a cord) would lead to the desired outcome (food) and engage in this behavior more rapidly, avoiding actions that had previously been unsuccessful at opening the box. In some sense, the correct behavior to escape was selected out of the full behavioral repertoire of the cat, whereas others were eliminated. There are several key features of Thorndike’s experiments that are central to RL, which we discuss throughout the chapter. The first is that successful escape from the puzzle box requires sufficient exploration of alternative actions. If the cat repeatedly tries the same unsuccessful action, escape is hopeless. On the other hand, after the cat escapes, it becomes better at engaging in less exploration and exploiting previously successful actions so that escape is faster. Of course, dropping off exploration too quickly could cause the cat to miss alternative means of escape that are less effortful or time consuming. Thus, effective behavior depends on a delicate balance between exploration and exploitation of options (see section Balacing Exploration and Exploitation—A Computational View and Varieties of Exploration in Humans). A second feature of Thorndike’s experiments is that the goal (escaping the box to get the food) is a complex, multi-action sequence. When the cat first succeeds in escaping, this raises the question of credit assignment: which of the multiple actions the animal might have tried were responsible for the escape? This problem can be especially difficult in a sequential decision-making setting because an action taken at one point in time may only have the desired effect some time later. For example, perhaps the cat pulls a string at one point in time,
Trials Fig. 5.1 Left: An illustration of Thorndike’s puzzle box experiments. Right: The time recorded to escape the box is reduced over repeated trials as the cat becomes more efficient at selecting the actions which lead to escape.
100
elementary cognitive mechanisms
presses a lever, and then pulls the string a bit harder, which opens the box. On the next trial, should the lever be pressed or should the string be pulled? Contemporary research in computational RL provides a computational solution to how agents might solve this credit assignment problem (see the section on A Computational Perspective on Reinforcement Learning). Reinforcement learning is also closely related to the idea of instrumental conditioning and the operant conditioning paradigm pioneered by psychologists in the behaviorist tradition (e.g., Skinner, 1938). Operant conditioning represents a refinement of the basic experiment design utilized by Thorndike. In a typical experiment, a rat might be placed in an isolated chamber with a lever that can be depressed. The experimenter records the frequency by which the rat presses the lever and at various points in time provides reward (e.g., food pellets) or punishment (e.g., electric shocks). The main question of interest is how the voluntary behavior of the animal is modified by various reward or punishment schedules. There continue to be a variety of theories of operant conditioning, but all share a basic view that, through learning, associations are strengthened (or weakened in the case of punishment) between elements that follow one another in time. Another major historical influence on modern RL was the work of Tolman (1948). Prior to Tolman’s work, psychologists largely viewed animal behavior as a product of associative stimulusresponse (S-R) learning and chaining of basic S-R behaviors. Tolman (1948) argued that many aspects of rodent behavior seemed to contradict this basic model. For example, Tolman showed that in maze tasks, rats could quickly re-route a path in a maze around a trained route that was experimentally blocked with an obstacle. This, he argued, could not be accomplished on the basis of pure stimulus-response learning since the relevant stimulus-response pairings for the new routes were never directly experienced by the rat. Instead, it appeared that rats use a “cognitive map” of the maze that allowed them to flexibly plan goal-directed sequences of behavior. Although Tolman’s work is often seen as antithetical to basic principles of conditioning, modern RL approaches have directly explored the distinction between more reactive, associative forms of reinforcement learning and more cognitive, planning-based approaches (a distinction referred to as model-based versus model-free RL, see section Model-Based versus
Model-Free Learning). The Box 1 describes common experimental approaches to studying RL.
A Computational Perspective on Reinforcement Learning The ability to adaptively make decisions that avoid punishment and maximize rewards is a core feature of intelligent behavior. Computational RL (CRL) is a theoretical framework for understanding how agents (both natural or artificial) might learn to make such decisions. CRL is not simply a description of how agents decide and learn, but offers insight into the overall function or objective of adaptive decision making. This is sometimes known as a “rational analysis” of behavior since it seeks to understand the purpose of some behavior independent of the exact mechanism by which it is accomplished (Marr, 1982; Anderson, 1990). Once the objective of learning has been made clear, CRL goes on to posit a family of related learning algorithms or mechanisms all designed to allow an embodied, autonomous agent to control their environment in effective ways. The core methods in the field were developed jointly by researchers in both computer science and psychology (Sutton, 1988; Sutton & Barto, 1981, 1998)1 as well as operations research (Bertsekas & Tsitsiklis, 1996).
The Goal All else being equal, it seems likely that animals seek to avoid punishment and maximize reward through their choices and actions (e.g., Thorndike’s “Law of Effect”). This basic idea is reflected in the first key assumption in CRL, namely that agents strive to maximize the long-term reward they experience from the environment. Formally, if the reward experienced at time t is rt , then the goal of learning and decision making is to maximize the expectation of reward over the long run future, that is, ∞ γ k rt+k+1 . (1) E k=0
The term γ in Equation 1 represents a discount factor that gives greater weight to rewards that are experienced sooner rather than later. This property is desirable for mathematical reasons (making the sum finite), but also because humans and other animals seem to have similar preferences for immediate over delayed rewards (Myerson & Green, 1995; Frederick, Loewenstein, & O’Donoghue, 2002).
computational reinforcement learning
101
Defining the Decision Environment: Basic Definitions Most applications of CRL, particularly applied to human and animal behavior, assume that the world can be viewed as a particular type of dynamic process know as a Markov decision process or MDP. The MDP assumption simply asserts that the world is composed of a set of finite states (S), actions (A), a set of transition probabilities (T ), and rewards (R). The states of a MDP refer to different distinct situations the agent might be in. The notion of a “state” in RL is sufficiently general to cover many different types of situations. For example, being in a particular location in a maze facing a particular direction might be one state. States might also involve elements internal to the agent like “being hungry” or, for a robot, “having a low battery” (see the section on The Influence of State Representations on Learning for a discussion of the notion of states in the context of human learning). Actions are the decision options that are available to the agent across all different states of the world. In some MDP problems, all actions are available in all states, whereas in others, different subsets of actions might be available, depending on the state. For example, if facing a wall at the end of a dead-end hallway, an agent’s available actions are practically limited to those that allow it to turn around. Like the definition of states, actions can be internal to the agent (e.g., encode the currently attended object in memory). Transition probabilities determine the dynamics of the environment. A transition probability can be defined as the probability of a new state s on the next time step (t + 1) given the current state is s and the agent selects action a: P(st+1 = s |st = s, at = a)
(2)
Note that the next state depends only on the current state and action and not on the full history of actions up to that point. This is known as the Markov assumption. Although this assumption may be violated in the real world, for many situations, this assumption appears reasonable, and it greatly simplifies the mathematics involved. The R determines how rewards or punishments are distributed in the environment. In a psychology experiment, the experimenter might manipulate how reward is provided to learners to alter their behavior. Similarly, roboticists who use RL to train autonomous systems often must adapt the reward function provided to the robot so the system 102
meets certain engineering objectives (see section on Varieties of Reward for a discussion of rewards in the context of human learning). In a MDP, rewards can be probabilistic (like the reward associated with buying a lottery ticket), thus it often makes sense to talk in terms of averages or expected values of rewards. In particular, we could summarize the expected value of rewards on trial t + 1 as E[rt+1 |st = s, at = a, st+1 = s ].
(3)
The function R assigns the value of reward received for each possible state transition. Together, these four elements (S, A, T , and R) completely define the decision problem (or MDP) facing the agent. Any particular task or environment can be defined by providing particular parameters or numbers for these four quantities (usually each can be expressed as a function or a matrix). Critically, the description of a particular task as an MDP does not assume any particular agent has complete knowledge about these four quantities (in fact in most realistic settings it would be impossible for an agent to completely know these aspects of the world). What is important in an MDP is simply defining the generic nature of the decision-making problem facing the agent. We later will discuss how CRL algorithms provide guidance on how agents learn to make effective decisions in such an environment given various amounts of knowledge about the world and experience.
Assigning Value to States and Actions Given the description of the environment as an MDP, it is clear that states and actions in the world that lead to greater expected reward have greater value to the agent. However, it is not only the states and actions immediately resulting in reward that may have value to an agent. For example, an action that takes the agent closer to a rewarding state can also be valuable (in the long run) by virtue of what it means for the agent’s future prospects. One clear example of this is second-order conditioning (Rescorla, 1980). In second-order conditioning, a stimulus such as a light is first paired with a negative outcome like a shock. Next, a second stimulus such as a tone is paired with the light (but without the shock). At test, animals show anticipation of the shock following the presentation of the tone even though this item was never directly paired with the negative outcome. In other words, the tone appears to be a proxy for the negative
elementary cognitive mechanisms
outcome because tones tend to beget lights, and lights tend to beget shocks. One way to quantify this “proxy” value is as the expected sum of future rewards available from each state, denoted V (st ): ∞ k V (st = s) = E γ rt+k+1 |st = s (4) k=t
In the second-order conditioning example just described, the long-term expected value at the onset of the light is largely negative, but so is the longterm expected value at the onset of the tone since they both ultimately will lead to shock. However, the state tied to the tone would have a slightly higher value since the punishment is delayed further in the future. This notion of estimating the “proxy” value of state or actions that later lead to reward is what allows RL to offer a solution to the credit assignment problem described earlier. The value of actions that in turn lead to others actions that provide reward is captured by the evaluations on long-term rather than immediate prospects. Of course, the valuation of particular states is very sensitive to the agent’s discounting parameter, γ . If γ is very small, the agent cares only about immediate reward (i.e., is myopic), and, thus, actions or states that result in direct reward are highly valued. If γ is larger, then future prospects are taken into account. One of the most important and interesting aspects of CRL is that the value of each state in Equation 4 can be defined recursively in terms of the value of other states: k V (st = s) = E[ ∞ k=t γ rt+k+1 |st = s] k = E[rt+1 + γ ∞ k=t γ rt+k+2 |st = s] = E[rt+1 + γ V (st+1 )|st = s]. (5) In other words, the value of a state st is the expectation of the reward experienced leaving that state (rt+1 ) plus a discounted estimate of the future reward available from the possible successor states, st+1 . Making the expectation in Eq. 5 more explicit: V (st = s) = π(s, a) Pssa [Rssa + γ V (s )] (6) a
s
where π(s, a) is the agent’s current probability of choosing action a in state s, Pssa is the probability of transitioning from state s to s given action a, and Rssa is the reward expected from that same action (using the notation developed by Sutton & Barto, 1998). Finally, V (s ) is the estimated value of future reward for the next state (i.e., Eq. 4). This approach
to averaging over possible outcomes weighted by their probability of occurrence is central to almost all work on judgement and decision-making (e.g., Bernoulli, 1954). One helpful way to think about this relationship is as a tree (see Figure 5.2). At the root of the tree is the agent’s current state (s). The value of that state will depend first on which action the agent takes (a), which is represented by the first set of branches in the figure. Which action is selected in turn will determine which of a variety of different rewards might be experienced (r) while transitioning to the new state s . The value of the current state V (s) is thus a weighted average of the value of all possible successor states based on both the agent’s decisionmaking, as well as the dynamics of the environment. This is similar to analyses of games like tic-tactoe where different actions lead to different game states and thus the value of any particular game state will depend on the available actions and states going forward. The recursive relationship between successive states is called the Bellman equation and is an important feature of MDPs as we will see next.
Making Good Decisions One way an agent could maximize reward would be to learn a set rules about how to behave in each state. Often this set of rules is known as a policy, which tells the agent which action to select in each state in order to maximize the expectation of longterm reward (Eq. 1). We first hinted at the idea of a policy in Eq. 6 when we needed to consider the agent’s probability of selecting each possible action (π(s, a)). The complete policy for how to respond in each possible state of the environment is denoted simply as π.
r
V(s’) s’
a V(s) s
Fig. 5.2 The value of the current state s can be estimated by “looking forward” from the current state to the expected reward r and the value of the possible successor states s , averaged over all possible actions and state transitions.
computational reinforcement learning
103
Ideally, the agent would like to learn the optimal policy, which is the one that returns the most reward over the long term across all possible states of the environment. There are many different methods for learning optimal policies for MDPs including dynamic programming and Monte-Carlo methods (Bertsekas & Tsitsiklis, 1996; Sutton & Barto, 1998). However some of these methods are less computationally practical for biological agents with finite resources or incomplete of knowledge about the environment. However, two particular methods of learning optimal (or near-optimal) policies have had an extremely important influence on research on learning and decision-making in humans and other animals: temporal difference (TD) learning and Q-learning. In the following sections, we describe the basic idea behind each of these algorithms. Later we will discuss how the features of these algorithms are related to the learning behavior of animals. However, first it is worth pointing out how estimating the value of each state can help the agent make decisions. Since the values for each state represent the expected sum of future rewards available from that state (Eq. 4), it means the agent is maximizing reward (given the current policy) if it chooses actions that have the highest probability of transitioning to states with high values. For example, we can talk about the long-term value of making a decision in a particular situation (more specifically a state-action pair) as: Pssa [Rssa + γ V (s )] (7) Q(st = s, at = a) = s
which is just a simplification of Eq. 6, which doesn’t average over different possible actions. A good strategy for the agent would be to choose the action in the current state s which maximizes the value of Q(s, a). This is trivially true since Q(s, a) measures the expected long-run reward available from taking that action. To maximize reward over the long term, the agent just needs to choose actions that maximize this quantity in each state. However, agents must first learn good estimates of V (s) or Q(s, a), which is what TD and Q-learning methods accomplish.
Temporal Difference (TD) Learning As the previous section described, an agent who has learned the values for each state in the environment can behave optimally simply by choosing actions in each state that maximize the estimated long-term reward. However, the main 104
issue is how to efficiently learn these values. One possibly naïve way to do this would be to follow a policy, π, keeping track of any rewards experienced along the way and at some point updating the value of V (st ) based on that experience. The downside is that it will take a lot of experience before the agent knows anything about the value of any particular state (conditioned on the policy). However, a more efficient, online, incremental learning rule that estimates V π (st ) can be derived by observing the relationship described in Eq. 5. The important insight of this equation is that the value of V π (st ) depends on the average value of the next state V π (st+1 ). This fact can be used to “bootstrap” estimates of V π (st ) more quickly. We will illustrate the basic idea with an example, then discuss the mathematics. In Figure 5.2, we showed how the value of a given state in an MDP is determined by the “tree” of successor outcomes available from that state. In Figure 5.3, we imagine that the agent begins in state s and has already estimated its long-term value to be 0.5, then selects action a. As a result of that selection, the agent receives a reward equal to r = 1.0 and transitions to a new state, s , which it has previously estimated to have value of 1.2. Assuming the discount parameter, γ is set to 0.9, this would appear to be a better outcome than the agent expected. In particular, the agent expects to receive on average 0.5 reward units going forward from state s, but on this choice the agent estimates and receives r +γ V (s ) = 1.0+0.9· 1.2 = 2.08. This discrepancy suggests that the agent might be wise to revise its estimate of V (s) to be higher (this is a better than expected state). One approach could be to simply replace the agent’s estimate of V (s) so that it equals 2.08. However, remember that V (s) represents the long-term reward available from a state averaged over all possible actions, rewards, and successor states. Thus, it would be to drastic to replace the old value completely with a single experienced outcome. Instead, the agent could simply move its estimate of V (s) a step in the direction of r + γ V (s ). As long as the step size isn’t too large, it can be shown that this will allow the value of V (s) to eventually converge on the true long-term estimate. On some trials, it might move a little too high, on some trials, a little to low, but in the limit it should converge toward the mean of the different experiences. Once the value of V (s) is updated, the current state is set to s and the process continues. The important point is that the value of V (s) can be
elementary cognitive mechanisms
V(s’) = 1.2 r=1.0 1. Select an action
s’
a
2. Experience reward and new state
V(s)=V(s) + α[r +γV(s’)-V(s)]
3. Update V(s) based on outcome s
Fig. 5.3 The key steps in the temporal-different (TD) update. First, an action is selected from the current state based on the agent’s current policy. Next, the reward is experienced along with information about the successor state. The agent can lookup its estimate of the value of this state from memory or assign a default value if the successor state had never been visited. Next the value of the original state is updated in light of the experienced outcome. In this way estimates of the value of each state can be “bootstrapped.” When the agent’s estimate of V (s) is accurate the error in expected reward and experienced reward will drop to zero on average. Learning is based on a temporal difference in what was expected and what was actually experienced.
bootstrapped based on the outcome of individual choices. After the agent makes a choice, it compares the value of the reward it experienced to the long-term estimate going forward and adjusts that estimate, V (s), up or down accordingly. Formally, the error in the agent’s current estimate of V (s) can be denoted using an intermediate variable called the prediction error, commonly denoted δ = rt+1 + γ V (st+1 ) − V (st ),
(8)
which measures the difference between the experienced and estimated value of current state. Prediction errors can be positive (when events worked out better than expected) or negative (when events work out worse then expected). When this value drops to zero it means that the current value estimates are all consistent with one another and thus represent the true values. In the section on the Neural Correlates of RL we will describe how neurons encoding prediction errors have been identified in the brains of animals. An incremental rule for adjusting V (st ) can be written using this prediction error: V (st ) = V (st ) + α[rt+1 + γ V (st+1 ) − V (st )] = V (st ) + αδ. (9) in this equation α is known as the learning rate and represents the “step size” of the update to V (s). In other words, V (s) is moved slightly in the direction of the prediction error (assuming α < 1.0)2 . This simple, incremental method of learning the values of states is known as temporal difference learning
(or TD). The name is largely descriptive: estimates of the long-term value of a state are based on differences between what was expected and what was experienced in time.
Q-learning Q -learning is a modification of the basic temporal-difference3 algorithm, which learns the value of state-action pairs, Q(s, a) directly (rather than estimating the value of individual states, V (s)). This is often a more useful quantity to estimate since we are often interested in the value of particular choices. A mentioned earlier, given direct Q(s, a) estimates computing a policy is simple: Choose the action in the current state associated with the largest value of Q(s, a). As a result, Q-learning obviates the need for a separate representation of the policy. From the perspective of a biological agent, learning and choice are made more compatible with few demands of extra processing (e.g., to compute V (s) estimates into Q(s, a) estimates via Eq. 7). In effect, the learned Q-values (i.e., Q(s, a)) tell the agent directly what choice to make. Following Eq. 9, an incremental update rule for Q(s, a) can be defined: Q(st , at ) = Q(st , at ) + αδ.
(10)
where δ takes a slightly different form than in standard TD learning: δ = rt+1 + γ max Q(st+1 , a) − Q(st , at ) a
(11)
Q-learning is considered an “off-policy” learning algorithm because the prediction error for the current choice assumes that the agent will
computational reinforcement learning
105
choose the best action on the next state (i.e., the maxa Q(st+1 , a) in Eq. 11) rather than follow their current policy (which might include the possibility of choosing to explore a different action). Q-learning is particularly important in computer science applications of RL because it can be proven to converge on the optimal policy for the particular MDP given that all state-action pairs are visited infinitely often (Watkins, 1989)4 . In addition, it has shown considerable success as a model of how humans learn in dynamic decision-making tasks in which past actions influence future rewards (e.g., Gureckis & Love, 2009a,b; Otto, Gureckis, Love, & Markman, 2009).
highest value of Q(s, a), but choosing randomly from the available alternatives with probability ε. This policy also ensures that each state action pair will be visited infinitely (assuming infinite time). The downside of ε-greedy is that it continues to explore with probability ε even after the agent has lots of experience with the environment (and the Qvalues may have converged to their accurate values). Another approach, known as the “softmax” strategy, explores probabilistically, but allows the current estimates of Q(s, a) to influence the probability of exploration (Sutton & Barto, 1998). According to the softmax rule, the probability of choosing action ai in any state st is given by e Q(ai ,st )·τ P(ai , st ) = N Q(a ,s )·τ j t j=1 e
Balancing Exploration and Exploitation—A Computational View In Thorndike’s puzzle-box experiments (see the section A Historical Perspective), we noted that the cat needs to explore various actions to learn which allow escape from the box. However, after a bit of learning, it becomes better to “exploit” action sequences that are known to be effective. An analogous situation arises in TD and Qlearning algorithms, just described. Choosing the action with the highest value of Q(s, a) in each state will lead to optimal long-term decision-making. However, this assumes that the current estimates of Q(s, a) are already accurate. At the start of learning in a novel task or environment this is unlikely to be the case. Instead it will take many updates of Eq. 10 in each state to ensure that the estimated values have converged. Thus, agents learning via TD and Q-learning must also balance the need to explore (in order to learn) and exploit (in order to actually maximize reward). Early in a task, the agent shouldn’t necessarily trust the current estimates of Q(s, a) too much and should sometimes choose actions that actually have lower estimated values. This is because those estimates may be incorrect and, with experience, would be adjusted upward or downward. One way to balance these competing concerns would be to choose the action associated with the highest value of Q(s, a) most of the time (exploit), but some fraction of the time to choose an action randomly (explore). As long as the probability of exploring is nonzero but not too large, this strategy can help the agent learn the true Q-values for any particular environment. This explore/exploit strategy is often known as the ε-greedy algorithm and entails choosing the option associated with the 106
(12)
where τ is a parameter that determines how closely the choice probabilities are biased in favor of the value of Q(ai , st ) and N is the number of available actions in state st . In general, the probability of choosing option ai is an increasing function of the estimated value of that action, Q(ai ), relative to the other action (see also Luce, 1959). However, the τ parameter controls how deterministic responding is. When τ → 0, each option is chosen randomly (the impact of learned values is effectively eliminated). Alternatively, as τ → ∞, the model will always select the highest valued option (also known as “greedy” action selection). Interestingly, the probability of exploration in the softmax model is sensitive to the degree of “competition” between choices. Assuming τ > 0, when all possible actions have similar values of Q(s, a), exploration becomes more likely. However, when one action is greatly superior, it will tend to be selected more often. In addition, in the softmax rule, the value of τ might be adjusted as the experience in the task accumulates (e.g., Busemeyer and Stout, 2002) to favor exploitation over exploration (similar to simulated annealing). The later section on The Varieties of Exploration in Humans discusses in more detail open issues surrounding how people balance exploration and exploitation.
Neural Correlates of RL Interest in computational RL stems not only from its utility in computer science applications (e.g., Tesauro, 1994; Bagnell and Schneider, 2001), but the fact that human and animal brains appear to use similar types of learning
elementary cognitive mechanisms
algorithms. Indeed, modern RL represents a powerful theoretical framework for thinking about systems-level neuroscience. On one hand this outcome is not surprising since the pioneering work in computer science on RL was directly inspired by psychological research on basic learning processes (Sutton, 1988; Sutton & Barto, 1998). However, the discovery of extremely close correspondences between the predictions of RL algorithms and the operation of particular neural systems in mammal brains represents a major scientific advance (see Niv, 2009 for an excellent review and history of these developments).
The Reward Prediction Error Hypothesis Perhaps the most famous discovery related to RL and the brain is the contribution RL has made to understanding the computational role of dopamine in learning. As Niv (2009) describes, early theories of dopamine suggested it encoded the reward value of particular stimuli in the environment. For example, dopamine neurons recorded within the midbrains of monkeys while they performed simple conditioning experiments showed increased firing rates immediately following the delivery of rewarding stimuli such as food (e.g., Schultz, Apicella, & Ljungberg, 1993). However, if this rewarding stimulus was consistently preceded by a conditioned stimulus (CS, e.g., a light or a tone) then the reward-related firing pattern would extinguish across trials. Instead, the neurons would begin firing upon the presentation of the CS. For example, Figure 5.4, Panel A shows an example trial prior to any learning in the experiment. Shortly after the delivery of the unexpected reward (R) there is a large phasic spike in neural firing. However, panel B shows the same neurons later in a trial after a CS is repeatedly paired (following a fixed delay) to the reward. After many trials, the neurons no longer respond vigorously to the onset of the reward (R) but instead show a phasic burst of activity shortly after the presentation of the CS. Finally, panel C shows the firing pattern of the neurons late in training when the reward is unexpectedly withheld after the presentation of the CS (i.e., “No R”). Here, there is a strong phasic burst of activity following the CS, but a noticeable drop in firing when the reward is withheld. This puzzling pattern of neural firing was ultimately deciphered using to the concept of a temporal-difference based prediction error, which we introduced in Eq. 8 (Montague, Dayan, Person, & Sejnowski, 1995; Montague, Dayan, &
Sejnowski, 1996; Schultz et al., 1997). A critical assumption in the neurocomputational model is that different points in time represent different states in the MDP (see Daw, Courville, & Touretzky, 2006a; Ludvig, Sutton, & Kehoe, 2012, for a contemporary discussions of the representation of time in RL models). Note that, in a given state (i.e., time point), s, prediction errors (δ) will be positive when experienced outcomes are better than expected (i.e., γ V (s ) + r>V (s)) and are negative when experienced outcomes are worse than expected. Early in training, the unexpected delivery of a rewarding stimulus leads to a positive prediction error (see Figure 5.4, panel D). As training trials in a condition experiment are repeated, the learned values for such states increase until the prediction error drops to zero. At the same time, TD algorithms “pass back” the proxy value of one state to preceding states (assuming γ > 0). Eventually, the prediction error associated with the unexpected presentation of positively valued CS (i.e., the start of the trial) prompts a new positive prediction error (the start of the trial is unexpected, but generally positive by proxy since it will always lead to later reward). Likewise, withholding an expected delivery of reward following the CS results in a negative prediction error (see Figure 5.4, panel F, closely matching the drop in firing observed in panel C). The importance of this finding is hard to overstate. Research in computer science and behavioral psychology had identified prediction error as a key signal for enabling incremental learning from experience (i.e., section on A Computational Perspective on Reinforcement Learning). The discovery that particular neurons in the brains of monkey reliably code a similar signal suggested an understanding not only of the detailed empirical phenomena, but of the overall function that dopamine plays in learning from experience.
Model-Based Analysis of fMRI Another, very general, way in which RL has informed our understanding of the brain is in the analysis of data collected from humans using fMRI (functional magnetic resonance imaging). Traditional studies using fMRI often contrasted the patterns of brain activity recorded during particular task-relevant states with a suitably defined baseline (e.g., passively looking at a fixation cross). However, RL researchers have pioneered the use of computational models to help structure fMRI data in a trial-by-trial fashion (Daw, 2011; Ashby, 2011). The idea
computational reinforcement learning
107
A
D
Early in training Trial 1 Trial 5 Trial 10
1.0
0
(No CS)
R –1.0
B
E
1.0
Late in training, reward delivered
0
CS
R
C
F
–1.0 Late in training, reward withheld
1.0
0
–1
0 CS
1
2s (No R)
–1.0 CS
R
Fig. 5.4 Figure adapted from Niv (2009). Panels A–C are taken from Schultz, Dayana, and Montague (1997) and shows raster plots of recorded dopaminergic neurons at various stages of a classical conditioning experiment. The rows along the bottom of each panel represent individual neurons. Time within the current trial flows from left to right across the page. Black dots in a row reflect a recorded neural spike at that point in time. Along the top is a histogram of the total firing rate summed across all recorded cells (essentially the marginal distribution of the points below). Panels D–F plot the predictions of the temporal difference algorithm (section Temporal Difference (TD) Learning), in particular, the prediction error term from Eq. 8 at various point within a trial and as training progresses. Refer to the text in the section The Reward Prediction Error Hypothesis for a full description.
is that computational models inspired by RL algorithms posit the existence of certain latent variables. The trial-by-trial changes in prediction error described earlier are one example, but so are other latent variables assumed by RL algorithms such as the values of particular state-action pairs (i.e., Q(s, a)). During learning, these variables fluctuate dynamically based on the experience and decisions of the learning agent. When first fit to human behavior (i.e., patterns of choices), these models provide excellent targets for structuring analyses of fMRI data. For example, regressors can be constructed representing the trial-by-trial fluctuations in prediction error (δ), which are then correlated with the fluctuations in measured blood oxygen level dependent (BOLD) signal (Platt & Glimcher, 1999; Sugrue, Corrado, & Newsome, 2004; Daw, O’Doherty, Seymour, Dayan, & Dolan, 2006b; Ahn, Krawitz, Kim, Busemeyer, & Brown, 2011). 108
Analyses of this type have revealed regions in the human brain that appear reliably responsive to prediction errors (O’Doherty, Dayan, Friston, Critchley, & Dolan, 2003; McClure, Berns, & Montague, 2003), and has given insight into how people trade off between exploration and exploitation (Daw, O’Doherty, Seymour, Dayan, & Dolan, 2006b). Even outside the field of RL, such model-based analysis approaches are changing the way fMRI data are analyzed (e.g., see Davis et al., 2012a,b, in the domain of category learning or Anderson et al. (2008) in problem solving and reasoning).
Contemporary Issues in RL Research Model-Based versus Model-Free Learning Temporal difference learning methods (such as TD and Q-learning described earlier) depend on direct experience to estimate the value of particular actions. For example, an agent learning via Qlearning will not understand that hot stoves should
elementary cognitive mechanisms
not be touched until it tries grasping a hot stove and experiences the large negative reward associated with that state-action pair. In this sense, these algorithms are much like early associative theories of conditioning described earlier: They learn stimulus-response associations between states, actions, and rewards based on direct experience. However, as the work by Tolman (1948) and others showed, animals exhibit a much richer and more flexible class of learning behaviors. For example, the ability of a rat to re-route it’s path around a novel obstacle would seem to depend on a richer knowledge about the structure of the maze than is assumed by temporal-difference learning methods. In fact, TD and Q-learning are are classified by computer scientists as “model-free” learning algorithms because they do not require the agent to know about the underlying structure of the environment (e.g., the set of transition probabilities, T , and rewards, R defined earlier). In model-free RL, estimates of V (s) or Q(s, a) suffice to enable adaptive behavior. In this case, the model refers to something akin to a relatively rich mental model of the task environment. In contrast, other types of RL algorithms (called model-based RL) emphasize the learning of the transition probabilities and rewards. Once the agent has a representation of these quantities it becomes possible to compute “on the fly” the values of V (s) or Q(s, a) at any point in time (using the Bellman insight in Figure 5.2). In addition, representing the transition probabilities and rewards explicitly allows the agent to plan into the future, forecasting the outcome of possible action sequences. This is exactly the type of behavior exhibited by Tolman’s clever maze-running rats and it likely an important part of human decision making as well (e.g., Sloman, 1996). The distinction between model-based and model-free RL is not purely theoretical. In fact, a growing body of work has explored the idea that multiple learning systems in the brain specialize in these respective types of learning and decision making (Daw, Niv, & Dayan, 2005; Daw, Gershman, Seymour, Dayan, & Dolan, 2011; Otto, Gershman, Markman, & Daw, 2013). In particular, the idea is that habitual behaviors (i.e., those that have been repeated many times and are executed without much thought) may be akin, computationally, to model-free learning. Such behaviors are acquired slowly and depend heavily on direct experience. In contrast, modelbased RL is more akin to more cognitively effortful
forms of planning and reasoning. Research in animal conditioning has pointed to a similar distinction between goal-directed and habitual behaviors (Dickinson, 1985; Dickinson & Balleine, 2004). The precise computational definition given to these two forms of learning allow model-based and model-free RL to make dissociable behavioral predictions in a variety of tasks. For example, Simon and Daw (2011a) present fMRI evidence for dissociable neural systems that separately represent model-based and model-free RL in a realistic spatial navigation task. Similarly, Simon & Daw (2011b) found evidence for the contribution for differentiable model-based and model-free learning systems in a simple sequential decision making task, which become more or less volatile over time. Consistent with the idea that model-based system utilizes more cognitive resources such as working memory and executive control, Otto, Gershman, Markman, & Daw (2013) found that participants’ learning behavior was better fit by model-free algorithms when they performed a sequential decision making task under working memory load. Computational RL may also provide insight into why these two, seemingly redundant, systems exist. For example, Daw, Niv, & Dayan (2005) argue the model-free system takes longer to learn (since getting good estimates for Q(s, a) can take considerable time and experience). On the other hand, model-based systems are able to guide behavior more quickly. However, the model-based system is more error prone since it requires an accurate representation of the transition probabilities in the environment. Thus, there are times where the accuracy of the model-free system will exceed the performance of the model-based system. In effect, the two systems are specialized for learning at different time scales. The full details of various model-based RL algorithms is beyond the scope of the present chapter (see Daw, 2012, for a summary).
The Influence of State Representations on Learning A critical component of almost all existing RL algorithms is the notion of a state (owing the to close relationship between these algorithms and the mathematics of Markov decision processes). In these models, a state is essentially a context or situation in which a certain set of choices are
computational reinforcement learning
109
available. For example, the idea of a “temporal difference” error signal implicitly assumes that the deviation between the expected and experienced outcomes is conditioned on the current state or context (Schultz et al., 1997). However, a rigorous account of what exactly constitutes a state is often left aside in neurobiological models. To the degree that such models can eventually be extended to explain behavior in more complex learning situations, it is critical that the field adopt a better understanding of how the state representation that the learner adopts influences learning and how such representations are acquired. The importance of state representation are particularly apparent in sequential decision making contexts where agents make a series of decisions in order to maximize reward. For example, a robot navigating a building could be in a particular state (e.g., a certain position and orientation in a particular hallway) and could make a decision (e.g., turn left), causing a change in state. According to the RL framework, the overall goal of the agent to to make the decision in each state that would maximize its long-term reward. However the way that the agent represents the structure of the environment can strongly influence its ability to achieve this goal. For example, an agent with very limited sensors might have difficulty differentiating between various hallways and intersections and thus would have trouble deciding which action to take in any given case. A similar problem faces human learners. For example, Gureckis and Love (2009b) studied a sequential decision-making task where human participants had to learn to avoid an immediate, short-term gain in order to maximize long-term reward (see also Herrnstein, Lowenstein, Prelec, & Vaughan, 1993) (see Box 1 for an overview of these types of tasks). Critically, the task was structured such that the reward received on any trial depended on the history of the participant’s response on previous trials. Finding the optimal long-term strategy in such environments is difficult because the relationship between distinct task states (in this case, different patterns of prior responses) and the reward on any trial is unclear. Gureckis and Love suggested one reason people show difficulty in such tasks may not stem entirely from an impulsive bias toward immediate gains, but instead because they adopt the wrong state representation of the task. A series of experiments found that providing simple perceptual cues that help to disambiguate successive tasks states greatly improved participants’ 110
ability to find the optimal solution (Gureckis & Love, 2009a,b; Otto, Gureckis, Love, & Markman, 2009). Corresponding simulations with an artificial RL agent based on Q-learning (see the section on Q-Learning) showed that, like humans in the experiment, enriching the discriminability of distinct task states greatly improves learning in the task. In Gureckis and Love’s (2009b) experiment, state cues were simple binary lights that unambiguously mapped to distinct task states. However, in the real world, such state cues are unlikely to be so direct. Consider a case in which you must decide which area of a lake is best for fishing. Your options might be the shallows by the shore, or the deep sections in the center. However, a number of factors such as season, time of day, presence of long grass for breeding, or the turbidity of the water may also be relevant. Successful decision-making thus requires the integration of a variety of cues in order to identify the current state and enable effective behavior. Note that the question of how people combine multiple cues in order to make predictions about the environment has been extensively studied in the categorization literature. However, the close relationship between work in categorization and RL has only recently been acknowledged (Shohamy, Myers, Kalanithi, & Gluck, 2008; McDonnell & Gureckis, 2009; Gureckis & Love, 2009b; Redish, Jensen, Johnson, & Kurth-Nelson, 2007; Gershman, Blei, & Niv, 2010).
Varieties of Exploration in Humans Effective RL often requires a delicate balance of exploratory and exploitative behavior (Sutton & Barto, 1998; Steyvers, Lee, & Wagenmakers, 2009). For example, consider the problem of choosing where to dine out from a set of competing options. The quality of restaurants often changes over time such that one cannot be certain which restaurant is currently best. In this type of “nonstationary” environment, one either chooses the bestexperienced restaurant so far (i.e., exploit) or visits a restaurant that was inferior in the past but now may have improved (i.e., explore). Even in stationary environments, knowing when and how much to explore is a difficult and important problem. For example, when outcomes are stationary in time but noisy, uncertainty about which option is best can complicate decision-making. Closely related problems occur in the foraging literature (Kamil,
elementary cognitive mechanisms
Box 1 Experimental Approaches Given the description in the section on A Computational Perspective on Reinforcement Learning of common RL algorithms, it is useful to consider the range of behavioral phenomena these theories have been used to explain. Overall, the diversity of tasks that have been modeling using RL is a testament to the very general framework it provides for thinking about human and animal behavior. Classical and Instrumental Conditioning As mentioned earlier, the contemporary field of RL was anticipated by early work in classical and instrumental conditioning. Such early studies identified notions of contingency, reward, and prediction error as possible determinants of learning (e.g., Rescorla and Wagner, 1972; Wagner and Rescorla, 1972). It is not surprising then that RL algorithms have continued to be influential as theories of basic conditioning phenomena. For example, the temporal difference methods described earlier were largely developed by theorists to explain continuous-time effects on conditioning as well as second-order conditioning, two classical (i.e., Pavlovian) phenomena that are difficult to account for using standard models such as the Rescorla-Wagner model (Niv and Schoenbaum, 2008). When modeling classical conditioning, the standard TD learning algorithm described in the section on Temporal Difference (TD) learning is often used. This is natural since, in classical conditioning paradigms, the participant does not have any control over the task (e.g., there are no choices to be made). For example, in an eye-blink conditioning experiment, a tone might sound moments before a puff of air is delivered to the participant’s eye. When applied to classical conditioning phenomena, particular assumptions are made about the representations of “states” in the system. For example, TD models of classical conditioning (see the section on The Reward Prediction Hypothesis) assume that continuous time is divided into arbitrarily small discrete units, each of which constitutes a different state in an MDP (Sutton, 1995; Daw, Courville, & Touretzky, 2006a; Ludvig, Sutton, & Kehoe, 2012). The value of being in the state of hearing
the tone might be lower since it means a puff of air is coming soon. RL algorithms have also been applied to instrumental conditioning paradigms explaining issues like the effects of reward rate on responding and response vigor (e.g., Niv, Daw, Joel, & Dayan, 2007). Here it is natural to consider learning algorithms like Q-learning or actor-critic architectures (not reviewed here but see Konda and Tsitsiklis, 1999) where agents explicitly are estimating the value of particular actions in different states and adjusting decision policies based on these experiences. Bandit Tasks A particular kind of instrumental conditioning paradigm that has strongly influenced research on RL in humans is the multi-armed bandit task. The basic experiment procedure is named after the slot machines in casinos which deliver payouts probabilistically when the “arm” of the machine is pulled (in fact, these machines are sometimes known as “one-armed bandits”). In the multi-armed bandit task, a number of choice options are presented to the subject. On each trial the subject can select one of the bandits. Selected bandits pay out a random reward from a distribution specified by the experimenter. The goal of the subject is to maximize the total reward they earn across a finite number of trials (see Steyvers, Lee, & Wagenmakers, 2009; Lee, Zhang, Munro, & Steyvers, 2011, for discussion of human experimentation and modeling). Bandit tasks are simple experiments that expose many of the core aspects of RL described earlier. For example, the learner has to balance exploration and exploitation between different bandits in order to ensure they are consistently choosing the best one. In addition, bandit tasks involve incremental learning of the valuation of different options across multiple trials. In some variants of a bandit task, the payout probability of the different options drifts randomly over time (sometimes called “restless bandits”) further encouraging continuous learning and exploration (e.g., Daw, O’Doherty, Seymour, Dayan, & Dolan, 2006b; Yi, Steyvers, & Lee, 2009).
computational reinforcement learning
111
Box 1 Continued Sequential Decision Making Tasks A key feature of RL models is the ability to learn to make sequences of decisions to achieve some goal. Experiments assessing this behavior tend to use tasks with greater structure than a traditional bandit task. For example, researchers have considered variants on bandit tasks where the reward rates of different options are linked together in particular ways. For example, Knox, Otto, Stone, & Love (2011) explored a “leap-frog” type tasks where the overall reward rates of two bandits take turns increasing in value at random points during the experiment. As discussed later, this additional structure favors different types of exploration strategies that are more structured in time (see the section on The Varieties of Exploration in Humans). Relatedly, some researchers have explore dynamic decision making tasks where the payoffs available from different actions depends on the past history of choices made by the agent (Neth, Sims, & Gray, 2006; Gureckis & Love, 2009a,b; Otto, Gureckis, Love, & Markman, 2009; Otto, Markman, Gureckis, & Love 2010; Otto & Love, 2010). In these cases, optimal, reward-maximizing strategies may require more complex sequential decisions similar to playing a game like tic-tac-toe or chess (see the section The Influence of State Representations on Learning). Researchers have even used complex video games that involve spatial navigation to interrogate basic learning and decision processes (e.g., Simon and Daw, 2011).
Krebs, & Pulliams, 1987; Stephens & Krebs, 1986). Animals must decide when to abandon a resource patch (e.g., a lake) and seek out a new resource. Optimal foraging requires keeping an estimate of the current reward rate and the expected reward rate for moving to a new resource patch (Kamil, Krebs, & Pulliams, 1987). Exploring when one should exploit and, conversely, exploiting when one should explore both incur costs. For example, an actor who excessively exploits will fail to notice when another action becomes superior. Conversely, an actor who excessively explores incurs an opportunity cost by frequently forgoing the high-payoff option. How often one should explore should vary as a function 112
of the environment. For environments that are volatile and undergo frequent change, one should explore more often (Daw, Gershman, Seymour, Dayan, & Dolan, 2011; Knox, Otto, Stone, & Love, 2011). In contrast, there is little reason to explore a well-understood environment that never changes. Other factors that affect how often one should explore include the task horizon (i.e., how many more opportunities there are to make a choice) and how rewarding the environment is in general (Rich & Gureckis, 2014; Steyvers, Lee, & Wagenmakers, 2009). To return to the restaurant scenario, there is little reason on the last day of vacation to try a new restaurant when all the restaurants besides one’s favorite have proven to be horrible. On the other hand, if it’s early in the vacation and most restaurants are generally good, then it makes sense to explore. The exploration methods considered earlier, such as softmax (Eq. 12) and ε-greedy (see section on Balancing Exploration and Exploitation—A Computational View) do not explicitly consider these issues concerning exploration. However, other methods for regulating uncertainty do. Next, a number of methods of exploring in dynamic-decision environments are considered. The methods are arranged from least to most sophisticated. Each method has its place in both engineering applications and for modeling human behavior. trial independent “random” exploration This form of exploration is the simplest and most commonly used in psychology, neuroscience, and computer science (e.g., Sutton & Barto, 1998). On each trial, a value estimate is gathered for each possible action. Exploiting corresponds to choosing the highest value option, whereas exploring corresponds to choosing some other action. This form of exploration is referred to as trial independent because the probability of choosing an action is not influenced by what was chosen on past trials or what will be chosen on future trials. The only factor determining the probability of selecting an action on the current trial is its current value. Examples of choice procedures using this scheme are softmax (Eq. 12) and ε-greedy. The strengths of these forms of exploration include simplicity. For example, these methods do not require a complex analysis of the environment. Because every action has some chance of being sampled on every trial, a well-designed learning rule (e.g., Q-learning) will
elementary cognitive mechanisms
eventually discover the optimal policy. Weakness includes that there is no guarantee that one is exploring when one should. systematic exploration Trial independent random exploration does not conform to popular intuition about what exploration is. Consider a historic explorer, such as Christopher Columbus, sailing toward the new world. His ships did not move east one day, west the next, and then south on the third day as they might according to a softmax exploration procedure where a choice is made on every trial. Instead, a series of linked and similar (i.e., repeated) actions were taken to truly move into unchartered territory. In certain tasks, exploration that reaches novel states requires consistent action across trials, which is not readily achievable with trial independent random exploration. Several consistent choices might move an agent to a novel state that several thousand “random” choices would never reach. There are certain tasks in which people likely explore in a similar fashion. Otto, Markman, Gureckis, & Love (2010) conducted a dynamic decision task that had a local optimum that people could only escape by repeatedly taking actions that resulted in lower immediate reward, but higher subsequent reward (cf. Bogacz, McClure, Li, Cohen, & Montague, 2007). These authors found that, when people were in a motivational state that fosters cognitive flexibility (cf. Higgins, 2000), subjects tended to be streaky in their choices, repeating the same option for several trials in a row. In particular, these subjects were best modeled by a choice procedure that repeated the previous choice with probability p and with probability 1−p, the softmax procedure determined the choice. This generally leads to systematic, streaky patterns of exploration. optimal planning The preceding methods for balancing exploration and exploitation are heuristic in nature. More ideally, one would calculate an overall plan in which exploration was integral to maximizing total reward. For example, for an ideal actor that uses its beliefs and plans ahead, there is no distinction between exploration and exploitation as all actions are following a policy that maximizes expected value (Gittins, 1979). This may seem like a philosophical distinction but it is conceptually and practically important. The preceding methods acknowledge that there is
information value in taking actions that currently have lower-than-expected reward. These methods are blind to what the true best policy is, so there is a need to explore in hopes of discovering it. In contrast, an ideal actor that performs optimal planning chooses to “explore” because it is the next move in the plan that has the highest expected value. In this sense, it is incorrect to say exploration ever occurs in an ideal actor model because every move is exploitative of long-term reward. One issue is that computing the ideal actor’s policy is computationally challenging and often can only be done for certain problems (cf., Kaelbling, Littman, & Cassandra, 1998). Optimal solutions can be calculated for relatively simple problems, such a finite horizon n-armed bandit task (Gittins, 1979; Steyvers, Lee, & Wagenmakers, 2009). Interestingly, recent work in psychology suggests that people actually can adopt such optimal decision policies in certain situations (e.g., Rich & Gureckis, 2014; Knox, Otto, Stone, & Love, 2011). Another example includes modeling navigation in mazes where people begin from a random starting position (Stankiewicz, Legge, Mansfield, & Schlicht, 2006).
Varieties of Reward In RL, rewards are fundamental in that they define the task for the agent. Rewards would seem to be straightforward, transparent, and immutable, the reward is simply the signal the environment provides and the agent receives. In reality, the concept of reward in RL is much richer, complex, and subtle. Although rewards can follow in a straightforward manner from the environment, in many RL domains and applications, rewards are another aspect of the overall system that is manipulated by the agent designer or human experimenter. For example, in Gureckis and Love (2009a) human and artificial agents both performed better in a difficult RL problem when low levels of noise were added to the reward signal. In other words, corrupting the reward signal with noise actually increased agent performance. The reason for this surprising outcome was that agents tended to underexplore initially and the added noise in the reward signal had the effect of increasing agent exploration, which benefitted the agent in the longrun. Although not presented this way in the original paper, this is a case in which a limitation in an agent can be addressed by the design of a reward signal.
computational reinforcement learning
113
This notion has been explored formally in machine learning (Singh, Lewis, Barto, & Sorg, 2010). When an agent is computationally bounded in some sense, there can be an alternative reward signal that will lead to the agent performing better than the external reward signal. Likewise, agents are often endowed with additional internal reward signals to create basic drives or motivations, such as curiosity (Schmidhuber, 1990; Oudeyer & Kaplan, 2007) or the desire for information (Gureckis & Markant, 2012). From a psychological perspective, models of human behavior have been formulated in which actions (and corresponding rewards) include internal actions, such as storing information into a working memory store (Dayan, 2012; Gray, Sims, Fu, & Schoelles, 2006; Todd, Niv, & Cohen, 2008). The costs associated with these internal operations can be optimized by learning algorithms, such as Q-learning, to capture human performance given task constraints. Finally, alternative ways of training agents sidestep many of the demands of traditional RL schemes that require extensive exploration of undesirable states to converge on a policy. One approach is learning by demonstration (Abbeel, Coates, & Ng, 2013). In this line of work, a difficult task is demonstrated to an agent, which dramatically speeds learning. For example, an agent can learn to control a helicopter much faster by having certain maneuvers demonstrated (i.e., observe the actions and corresponding outcomes) than by self-exploring the huge space of action sequences and subsequent rewards. Another approach is interactive shaping in which an outside teacher provides a reward signal to the agent rather than the environment providing the reward signal. Agents can master difficult tasks, such as playing Tetris, much more quickly by receiving evaluative feedback from an observing human teacher than by learning to play by exploring and experiencing subsequent rewards (Knox, Glass, Love, Maddox, & Stone, 2012; Knox & Stone, 2009). The humansupplied feedback effectively transforms an RL task into a supervised learning task in which only the immediate action is rewarded. Many of the inherent difficulties of RL problems, such as delayed rewards and learning the proper sequence of actions, are sidestepped by this approach.
Concluding remarks Reinforcement learning represents a dynamic confluence of ideas from psychology, neuroscience, computer science, and machine learning. The 114
power and generality of the framework is made clear by the diverse range of theoretical and practical ideas that it has help to articulate. That said, there are many important open issues in the field that have been hinted in this chapter.
Acknowledgments The authors wish to thank Julie Hollifield for help with the figures and Nathaniel Daw, David Halpern, John McDonnell, Yael Niv, Alex Rich, and A. Ross Otto for helpful discussions in the preparation of the manuscript. TMG was supposed by grant number BCS-1255538 from the National Science Foundation and contract D10PC20023 from Intelligence Advanced Research Projects Activity (IARPA) via Department of the Interior (DOI).
Notes 1. Sutton and Barto (1998) in particular provides a very clear technical introduction to the area. 2. This discussion implicitly assumes the learning rate, α, is constant, however, the learning rate might be adjusted based on experience in the task (e.g., dropping toward zero over time) or based on the overall volatility of the environment (see Sutton & Barto, 1998, chapter 2 for a further discussion). This issue is also of interest in the psychology and neuroscience literatures (Behrens, Woolrich, Walton, & Rushworth, 2007; Krugel, Biele, Mohr, Li, & Heekeren, 2009; Nassar & Gold, 2013). 3. In fact, both the TD algorithm described in the section Temporal Difference (TD) Learning and Q-learning are considered “temporal-difference methods” although the TD algorithm shares the name with this more general class. 4. This might sound like an extreme demand to make on optimal learning behavior for a biological agent, but it is a general requirement of all optimal convergence algorithms. In practice, algorithms such as Q-learning can converge on effective policies even in complex tasks given reasonable amounts of experience with the environment.
Glossary Credit Assignment: The problem of assigning blame to actions when rewards are delayed in time. For example, if you leave your cup on the edge of a table, then three days later nudge the table with your hip and break the cup, which action is most responsible for the negative outcome? In one way it is the earlier action of leaving the cup in a dangerous place. However, traditional theories of conditioning might assume that more immediate actions are to blame. RL deals with this problem by having the value of action depend not only on their immediate outcomes but on the sum of future rewards available following that action. Explore/Exploit Dillemma: In environments in which the reward associated with different actions are unknown, agents face a decision dilemma to either return to options
elementary cognitive mechanisms
Glossary previously known to be positive (exploit), or to explore relatively unknown options. The optimal balance between exploration and exploitation is computable in some environments but generally can be computationally intractable. Temporal Difference (TD) learning: A method of computational RL that learns the value of actions through experience. In particular, in TD, learning is based on a deviation between what is expected at one point in time and what is actually experienced. TD is explained in detail in the section on Temporal Difference (TD) Learning. Policy: A set of rules that determine which action an agent should selected in each possible state in order to maximize the expectation of long-term reward. A everyday example of a policy would be a list of directions from getting from location A to location B. At each intersection the directions (policy) tell the agent which decision to make. State: Distinct situations an agent may be in. For example, being stopped at a red light on the corner of Houston St. and Broadway in NYC might be a state. States derive their importance in RL from the dependence of most RL methods on the underlying mathematics of Markov Decision Processes (MDPs). Reward: Typically a scalar value that determines the quality or desirability of an outcome or situation. Typically rewards are designed by society, experimenters, or roboticists to guide the behavior of agents in particular ways. The goal of an RL agent is to maximize the reward received over the long term. Myopic behavior: The behavior of RL agents often depends on the degree to which they value long-term versus short-term outcome. A parameters (γ ) in most RL models controls this trade-off. When γ is zero, RL agents make choices that only value immediate rewards. This is often described as “myopic” behavior because it disregards future consequences of actions.
References Abbeel, P., Coates, A., Ng, & A. Y. (2013). Autonomous helicopter aerobatics through apprenticeship learning. International Journal of Robotics Research, 32, 458–482. Ahn, W.-Y., Krawitz, A., Kim, W., Busemeyer, J., & Brown, J. (2011). A model-based MRI analysis with hierarchical bayesian parameter estimation. Journal of Neuroscience, Psychology, and Economics, 4(2), 95–110. Anderson, J. (1990). The adaptive character of thought. Mahwad, NJ: Erlbaum. Anderson, J., Carter, C., Fincham, J., Qin, Y., Ravizza, S., & Rosenberg-Lee, M. (2008). Using fmri to test models of complex cognition. Cognitive Science, 32, 1323–1348. Ashby, F. (2011). Statistical analysis of fMRI data. Cambridge, MA: MIT Press. Bagnell, J., & Schneider, J. (2001). Autonomous helicopter control using reinforcement learning policy search methods (pp. 1615–1620). In International Conference on Robotics and Automation. IEEE.
Behrens, T., Woolrich, M., Walton, M., & Rushworth, M. (2007). Learning the value of information in an uncertain world. Nature Neuroscience, 10(9), 1214–1221. Bernoulli, D. (1954). Exposition of a new thoeyr on the measurement of risk. Econometrica, 22, 23–36. Bertsekas, D., & Tsitsiklis, J. (1996). Neuro-dynamic programming. Belmont, MA: Athena Scientific. Bogacz, R., McClure, S., Li, J., Cohen, J., & Montague, P. (2007). Short-term memory traces for action bias in human reinforcement learning. Brain Research, 1153, 111–121. Busemeyer, J., & Stout, J. (2002). A contribution of cognitive decision models to clinical assessment: Decomposing performance on the bechara gambling task. Psychological Assessment, 14(3), 253–262. Davis, T., Love, B., & Preston, A. (2012a). Learning the exception to the rule: Model-based fmri reveals specialized representations for surprising category members. Cerebral Cortex, 22, 260–273. Davis, T., Love, B., & Preston, A. (2012b). Striatal and hippocampal entropy and recognition signals in category learning: Simultaneous processes revealed by model-based fmri. Journal of Experimental Psychology: Learning, Memory, & Cognition, 38, 821–839. Daw, N. (2011). Trial-by-trial data analysis using computational models. In E. A. Phelps, T. W. Robbins, and M. Delgado (Eds.), Affect, learning and decision making, attention and performance XXIII. Oxford University Press. Daw, N. (2012). Model-based reinforcement learning as cognitive search: Neurocomputational theories. In P. Todd, T. Hills, and T. Robbins (Eds.) (2012). Cognitive search: Evolution, algorithms, and the brain. MIT Press, Cambridge, MA. Daw, N., Courville, A., & Touretzky, D. (2006a). Representation and timing in theories of the dopamine system. Neural Computation, 18, 1637–1677. Daw, N., Gershman, S., Seymour, B., Dayan, P., & Dolan, R. (2011). Model-based influences on humans’ choices and striatal prediction errors. Neuron, 69, 1204–1215. Daw, N., Niv, Y., & Dayan, P. (2005). Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control. Nature Neuroscience, 8(12), 1704–1711. Daw, N., O’Doherty, J., Seymour, B., Dayan, P., & Dolan, R. (2006b). Cortical substrates for exploratory decision in humans. Nature, 441, 876–879. Dayan, P. (2012). How to set the switches on this thing. Curr Opin Neurobiol, 22(6), 1068–1074. doi: 10.1016/j.conb.2012.05.011. Dickinson, A. (1985). Actions and habits: The development of behavioural autonomy. Philosophical Transactions of the Royal Society B: Biological Sciences, 308, 67–78. Dickinson, A., & Balleine, B. (2004). The role of learning in the operation of motivational systems. In R. Gallistel (Ed.), Stevens’ handbook of experimental psychology: Vol. 3. Learning, motivation, and emotion, (3rd ed.). Hoboken, NJ: Wiley. Frederick, S., Loewenstein, G., & O’Donoghue, T. (2002). Time discounting and time preference: A critical review. Journal of Economic Literature, XL, 351–401.
computational reinforcement learning
115
Gershman, S., Blei, D., & Niv, Y. (2010). Context, learning, and extinction. Psychological Review, 117, 197–209. Gittins, J. C. (1979). Bandit processes and dynamic allocation indices. Journal of the Royal Statistical Society, Series B, 41, 148–177. Gray, W. D., Sims, C. R., Fu, W. T., & Schoelles, M. J. (2006). The soft constraints hypothesis: A rational analysis approach to resource allocation for interactive behavior. Psychological Review, 113(3), 461–482. Gureckis, T., & Love, B. C. (2009a). Learning in noise: Dynamic decision-making in a variable environment. Journal of Mathematical Psychology, 53, 180–193. Gureckis, T., & Love, B. C. (2009b). Short term gains, long term pains: How cues about state aid learning in dynamic environments. Cognition, 113(3), 293–313. Gureckis, T., & Markant, D. (2012). A cognitive and computational perspective on self-directed learning. Perspectives in Psychological Science, 7, 464–481. Herrnstein, R., Lowenstein, G., Prelec, D., & Vaughan, W. (1993). Utility maximization and melioration: Internalities in individual choice. Journal of Behavioral Decision Making, 6, 149–185. Higgins, E. T. (2000). Making a good decision: Value from fit. American Psychologist, 55, 217–1230. Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101, 99–134. Kamil, A. C., Krebs, J. R., & Pulliams, H. R. (1987). Foraging Behavior. London, England: Plenum. Knox, W., Glass, B., Love, B., Maddox, W., & Stone, P. (2012). How humans teach agents. International Journal of Social Robotics, 4(4), 409–421. Knox, W. B., Otto, A. R., Stone, P., & Love, B. C. (2011). The nature of belief-directed exploratory choice in human decision-making. Frontiers in psychology, 2, 398. doi: 10.3389/fpsyg.2011.00398 Knox, W. B., & Stone, P. (2009). Interactively shaping agents via human reinforcement: The tamer framework. In Proceedings of The Fifth International Conference on Knowledge Capture (K-CAP 2009). pp. 9–16. Konda, V., & Tsitsiklis, J. (1999). Actor-critic algorithms. In Neural Information Processing Systems. Cambridge, MA: MIT Press. Krugel, L., Biele, G., Mohr, P., Li, S.-C., & Heekeren, H., (2009). Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions. Proceedings of the National Academy of Sciences, 106, 17951– 17956. Lee, M., Zhang, S., Munro, M., & Steyvers, M., (2011). Psychological models of human and optimal performance in bandit problems. Cognitive Systems Research, 12, 164–174. Luce, R. D. (1959). Individual choice behavior: A theoretical analysis. Westport, CT: Greenwood Press. Ludvig, E., Sutton, R., & Kehoe, E. (2012). Evaluating the td model of classical conditioning. Learning and Behavior, 40(3), 305–319. Marr, D. (1982). Vision: A computational investigation into the human representation and processing of visual information. New York, NY: Freeman.
116
McClure, S., Berns, G., & Montague, P. (2003). Temporal prediction errors in a passive learning task activate human striatum. Neuron, 38(2), 339–346. McDonnell, J., & Gureckis, T. (2009). How perceptual categories influence trial and error learning in humans. In Multidisciplinary Symposium on Reinforcement Learning. Montreal, Canada. http://www.gureckislab.org/papers/McDonnell GureckisMSRL.pdf Montague, P., Dayan, P., Person, C., & Sejnowski, T. (1995). Bee foraging in uncertain environments using predictive hebbian learning. Nature, 377 (6551), 725–728. Montague, P., Dayan, P., & Sejnowski, T. (1996). A framework for mesencephalic dopmaine system based on predictive hebbian learning. Journal of Neuroscience, 16 (5), 1936– 1947. Myerson, J., & Green, L. (1995). Discounting of delayed rewards: Models of individual choice. Journal of the Experimental Analysis of Behavior, 64, 263–276. Nassar, M., & Gold, J. (2013). A healthy fear of the unknown: Perspectives on the interpretation of parameter fits from computational models in neuroscience. PLOS Computational Biology, 9(4), e1003015. Neth, H., Sims, C., & Gray, W. (2006). Melioration dominates maximization: Stable suboptimal performance despite global feedback. In: R. Sun, and N. Miyake (Eds.), Proceedings of the 28th Annual Meeting of the Cognitive Science Society. Hillsdale, NJ: Erlbaum. Niv, Y. (2009). Reinforcement learning in the brain. Journal of Mathematical Psychology, 53(3), 139–154. Niv, Y., Daw, N., Joel, D., & Dayan, P. (2007). Tonic dopamine: Opportunity costs and the control of response vigor. Psychopharamcology, 191(3), 507–520. Niv, Y., & Schoenbaum, G. (2008). Dialogues on prediction errors. Trends in Cognitive Sciences, 12(7), 265–272. O’Doherty, J., Dayan, P., Friston, K., Critchley, H., & Dolan, R. (2003). Temporal difference learning model accounts for responses in human ventral striatum and orbitofrontal cortex during pavlovian appetitive learning. Neuron, 38, 329–337. Otto, A., Gershman, S., Markman, A., & Daw, N. (2013). The curse of planning: Dissecting multiple reinforcement learning systems by taxing the central executive. Psychological Science, 24(5), 751–761. Otto, A., Gureckis, T., Love, B., & Markman, A. (2009). Navigating through abstract decision spaces: Evaluating the role of state knowledge in a dynamic decision making task. Psychonomic Bulletin and Review, 16 (5), 957–963. Otto, A., & Love, B. (2010). You don’t want to know what you’re missing: When information about forgone rewards impedes dynamic decision making. Judgment and Decision Making, 5, 1–10. Otto, A., Markman, A., Gureckis, T., & Love, B. (2010). Regulatory fit in a dynamic decision-making environment. Journal of Experimental Psychology: Learning, Memory, and Cognition. 36 (3), 797–804. Oudeyer, P. Y., & Kaplan, F. (2007). What is intrinsic motivation? A typology of computational approaches. Frontiers in Neurorobotics, 1(6).
elementary cognitive mechanisms
Platt, M., & Glimcher, P. (1999). Neural correlates of decision variables in parietal cortex. Nature, 400(6741), 233–238. Redish, A., Jensen, S., Johnson, A., & Kurth-Nelson, Z. (2007). Reconciling reinforcement learning models with behavioral extinction and renewal: Implications for addition, relapse, and problem gamling. Psychological Review, 114(3), 784–805. Rescorla, R. (1980). Pavlovian second-order conditioning: Studies in associative learning. Hillsdale, NJ: Erlbaum. Rescorla, R., & Wagner, A. (1972). A theory of pavolvian conditioning: Variations in the effectiveness of reinforcement and non-reinforcement. In A. Black, and W. Prokasy (Eds.), Classical conditioning II: Current research and theory. (pp. 64–99). New York, NY: Appleton-Century-Crofts. Rich, A. S., & Gureckis, T. M. (2014). The value of approaching bad things. in P. Bello, M. Guarini, M. McShane, & B. Scassellati (Eds.) Proceedings of the 36th Annual Conference of the Cognitive Science Society. Austin, TX: Cognitive Science Society. Schmidhuber, J. (1990). A possibility for implementing curiosity and boredom in model-building neural controllers. Proc. of the International Conference on Simulation of Adaptive Behavior: From Animals to Animais, pp.222–227 published by MIT Press. Schultz, W., Apicella, P., & Ljungberg, T. (1993). Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task. Journal of Neuroscience, 13(3), 900–913. Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of predicion and reward. Science, 275, 1593–1598. Shohamy, D., Myers, C., Kalanithi, J., & Gluck, M. (2008). Basal ganglia and dopamine contributions to probabilistic category learning. Neuroscience and Biobehavioral Reviews, 32(2), 219–236. Simon, D., & Daw, N. (2011a). Neural correlates of forward planning in a spatial decision task in humans. Journal of Neuroscience, 31, 5526–5539. Simon, D. A., & Daw, N. D. (2011b). Environmental statistics and the trade-off between model-based and TD learning in humans. In: J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, & K. Weinberger (eds.), Advances in neural information processing systems, 24, 127–135. Singh, S., Lewis, R. L., Barto, A. G., & Sorg, J. (2010). Intrinsically motivated reinforcement learning: An evolutionary perspective. IEEE Transanctions on Autonomous Mental Development, 2(2), 70–82.
Skinner, B. (1938). The behavior of organisms: An experimental analysis. Oxford, England: Appleton-Century. Sloman, S. (1996). The empirical case for two systems of reasoning. Psychological Bulletin, 119(1), 3–22. Stankiewicz, B., Legge, G., Mansfield, J., & Schlicht, E. (2006). Lost in virtual space: Studies in human and ideal spatial navigation. Journal of Experimental Psychology: Human Perception and Performance, 32, 688–704. Stephens, D., & Krebs, J. (1986). Foraging theory. Princeton, NJ: Princeton University Press. Steyvers, M., Lee, M., & Wagenmakers, E. (2009). A bayesian analysis of human decision-making on bandit problems. Journal of Mathematical Psychology, 53, 168–179. Sugrue, L., Corrado, G., & Newsome, W. (2004). Matching behavior and the representation of value in the parietal cortex. Science, 304, 1782–1787. Sutton, R. (1988). Learning to predict by the method of temporal difference. Machine Learning, 3, 9–44. Sutton, R. (1995). TD models: Modeling the world at a mixture of time scales. In Proceedings of the 12th International Conference on Machine Learning. Morgan-Kaufmann, (pp. 531–539). Sutton, R., & Barto, A. (1981). Toward a modern theory of adaptive networks: Expectation and prediction. Psychological Review, 88, 135–170. Sutton, R., & Barto, A. (1998). Reinforcement learning: An introduction. Cambridge, MA: MIT Press. Tesauro, G. (1994). Td-gammon, a self-teaching backgammon program, achieves master-level play. Neural Computation, 6 (2), 215–219. Thorndike, E. (1911). Animal intelligence: Experimental studies. New York, NY: Macmillan. Todd, M. T., Niv, Y., & Cohen, J. D. (2009). Learning to use working memory in partially observable environments through dopaminergic reinforcement. In: D. Koller, D. Schuurmans, Y. Bengio, & L. Bottou (eds.) Advances in Neural Information Processing Systems 21, 1689–1696. Tolman, E. (1948). Cognitive maps in rats and men. Psychological Review, 55(4), 189–208. Wagner, A., & Rescorla, R. (1972). Inhibition in pavlovian conditioning: Application of a theory. In R. Boake, and M. Halliday, (Eds.), Inhibition and learning (pp. 301–336). London, England: Academic. Watkins, C. (1989). Learning from delayed rewards. Ph.D. thesis, Cambridge University, Cambridge, England. Yi, S., Steyvers, M., & Lee, M. (2009). Modeling human performance on restless bandit problems using particle filters. Journal of Problem Solving, 2(2), 33–53.
computational reinforcement learning
117
PART
Basic Cognitive Skills
II
CHAPTER
6
Why Is Accurately Labeling Simple Magnitudes So Hard? A Past, Present, and Future Look at Simple Perceptual Judgment
Chris Donkin, Babette Rae, Andrew Heathcote, and Scott D. Brown
Abstract
Absolute identification is a deceptively simple task that has been the focus of empirical investigation and theoretical speculation for more than half a century. Since Miller’s (1956) seminal paper the puzzle of why people are severely limited in their capacity to accurately perform absolute identification has endured. Despite the apparent simplicity of absolute identification, many complicated and robust effects are observed in both response latency and accuracy, including capacity limitations, strong sequential effects and effects of the position of a stimulus within the set. Constructing a comprehensive theoretical account of these benchmark effects has proven difficult, and existing accounts all have shortcomings. We review classical empirical findings, as well as some newer findings that challenge existing theories. We then discuss a variety of theories, with a focus on the most recent proposals, make some broad conclusions about general classes of models, and discuss the challenges ahead for each class.
Absolute or perfect pitch, the ability to identify the notes played on a musical instrument, is a very rare ability, and has been a subject of scientific study since the late 19th century (Ellis, 1876). It is surprising that identifying musical notes is so difficult, since humans routinely identify a huge number of things in day-to-day life: such as faces, voices, and places. Throughout the first half of the 20th century, however, psychology researchers uncovered that the difficulty most people experience in naming notes is representative of a general deficit in identifying simple stimuli that vary on just one dimension. This early research culminated in Miller’s (1956) seminal “7 ± 2” paper, in which he argued that humans were capable of accurately identifying only 5 to 9 stimuli that varied on a single dimension, regardless of the modality of that stimulus. Miller’s work also made prominent the field of study that is the focus of this chapter, absolute identification.
In an absolute identification task, a set of N stimuli that vary on a single physical dimension are assigned a set of labels (usually the numbers 1 through N ). On any given trial, participants are presented with one stimulus and asked to produce the corresponding label. For example, absolute pitch is a version of absolute identification in which the stimuli are tones varying in frequency and the labels are the musical note names A#, C, and so on. Other common versions of absolute identification use lines varying in length, or pure tones varying in loudness. Since Miller’s (1956) work, studies of absolute identification using a variety of stimulus dimensions and perceptual modalities have revealed an intricate pattern of phenomena behind this puzzling limitation in human ability. The complexity of the behavior elicited by such a seemingly simple task has ensured the enduring interest of the area, for example, as summarized by Shiffrin 121
Benchmark Phenomena Capacity Limitations Early in the history of absolute identification researchers were particularly intrigued by the difficulty of the task. A common method of assessing performance limits was to increase the size of the stimulus set, beginning with just two stimuli, and observe classification accuracy. Perfectly accurate performance usually failed after as few as five items. Initially, this result was demonstrated in experiments where the range of the stimulus set is held constant. Figure 6.1 demonstrates the severe nature of the capacity limitation using data from Pollack (1952) and Garner (1953). One might think that increasing the stimulus range would facilitate performance but, surprisingly, the severe limit of 7 ± 2 persists even when the range of 122
basic cognitive skills
4 Information Transmitted (bits)
and Nosofsky (1994). Absolute identification has also been of interest because of its links to other key areas of psychology, such as categorization— absolute identification is just categorization with one item per category (Nosofsky, 1986, 1997), and magnitude production—the reverse of absolute identification: given a label, participants try to produce the stimulus (De Carlo & Cross, 1990; Zotov, Jones, & Mewhort, 2011). More tantalizing is the suggested link between absolute identification and basic short term memory research. As Miller pointed out, both paradigms focus on memory, and share the same severe performance limit (7 ± 2), suggesting a common, and deep-seated, cognitive mechanism [this is also suggested by recently identified links in the sequential effects between short-term memory and absolute identification (Malmberg & Annis, 2012)]. This extensive study of absolute identification has yielded a wide range of robust benchmark phenomena not only in terms of the accuracy of responses but also in dynamic aspects, such as the time to make responses and the effect of previous responses on subsequent responses. Such a richness of data makes absolute identification a difficult challenge for cognitive modeling. We begin this chapter by giving an overview of the benchmark phenomena. These classical benchmarks focus on response accuracy. We then summarize key theoretical approaches that have been applied to absolute identification and describe how the latest models within these approaches have expanded their explanatory reach to response time (RT) as well as accuracy. We finish by discussing some of the recent, and ongoing issues in the field.
3
2
1
Pollack 1952 Garner 1953 Perfect Performance
2
4
6
8 10 12 14 16 Number of Stimuli (N)
18
20
Fig. 6.1 Performance, in terms of information transmitted, as a function of the number of stimuli used in the experiment. The data are taken from two experiments in which severe limits on performance were observed, reported by Pollack (1952) and Garner (1953). Note that k bits transmitted corresponds to the perfect identification of 2k stimuli. Reproduced from Figure 1, Stewart, Brown and Chater (2005).
stimuli is increased dramatically. In fact, once adjacent stimuli are perceptually discriminable (i.e. the spacing between stimuli is well above the just-noticeable difference), further increases to the range effectively have no influence on performance (Braida & Durlach, 1972). A stimulus set can be characterized by the number of bits of information necessary to classify its members. Perfect classification of two stimuli requires 1 bit of information, four stimuli require 2 bits of information, and in general N stimuli require log2 N bits of information. In a review of 25 studies using a wide variety of stimuli including frequency and intensity of tones, taste, hue of colours, and magnitude of lines and areas, Stewart, Brown, and Chater (2005) found the mean information limit was 2.48 bits. Figure 6.1 shows how increasing the number of stimuli increases the amount of information required for perfect classification. As illustrated, classification performance falls below the information required for perfect classification as the number of stimuli increases beyond four, and asymptotes not much above two bits. Hence accuracy becomes progressively worse as set size increases above 4. Historically, it has been found that the capacity limitation is remarkably resistant to practice. For example, Garner’s (1953) participants performed
Bow Effects The bow effect is one of the most robust results in absolute identification tasks (Kent & Lamberts, 2005; Pollack, 1953; Weber et al., 1977). As shown in Figure 6.2, response accuracy follows a Ushape when it is plotted as a function of stimulus position. That is, in any set of to-be-identified stimuli, the smaller and larger stimuli are better identified than items in the middle of the stimulus range. This effect is particularly interesting because it is independent of the absolute size of the stimulus. That is, the ability to identify any particular stimulus depends on its relative position within the set of stimuli being identified. For example, a stimulus can go from being very accurately identified, when it is the smallest stimulus in some set, but later be identified at chance-level accuracy, when it is one of the central stimuli in a new set (Lacouture & Marley, 1995; Lacouture, 1997). The addition of new stimuli to a set causes existing stimuli in the set to be identified less accurately, but this effect varies greatly across the stimulus range. Stimuli near the edge of of the range suffer much less from the introduction of additional stimuli than stimuli near the center of the range.
Sequential Effects There has been growing recent interest in sequential effects, that is, the influence of recent stimuli and responses on the current decision (Gilden,
1.0
1.4 Mean RT (sec.)
0.9 Proportion Correct
around 12,000 identification trials with tones of different loudnesses,1 yet at the end of the experiment they were still limited to identifying the equivalent of just four stimuli correctly. Weber, Green, and Luce (1977) also had participants complete 12,000 trials identifying just six tones varying in loudness and found an improvement in response accuracy of just 6%. Final performance for these participants was well below ceiling, despite there being extensive practice, monetary incentives, and only six tones. Hartman’s (1954) participants practiced for 8 weeks, and although they demonstrated substantial improvement, they still could only perfectly identify five stimuli, which is well within Miller’s limit. Such results have established what has subsequently been treated as a truism about absolute identification: there is a severe limitation in human ability to identify unidimensional stimuli, and this limit is unaffected by practice. However, recent results suggest that practice is not always ineffectual, at least for some stimulus dimensions. We will return to this issue later.
0.8 0.7
1.2
1.0
0.8
0.6 0.5 1
2
3
4
5
6
7
8
9 10
1
2
3
4
5
6
7
8
9
Stimulus Rank
Fig. 6.2 Performance, in terms of proportion of correct responses (left) and mean response time (right) for each stimulus. The different lines correspond to different stimulus set size conditions. The data are taken from Lacouture and Marley (1995), and demonstrate the bow effect, and the consistent performance for stimuli towards the edge of the stimulus range despite the number of stimuli in the set. Reproduced from Figures 5 and 7 of Lacouture and Marley (2005). Lacouture, Y., & Marley, A. A. J. (2004). Choice and response time processes in the identification and categorization of unidimensional stimuli. Perception & Psychophysics, 66, 1206–1226. With kind permission from Springer Science and Business Media.
Thornton, & Mallon, 1995: Van Orden, Holden, & Turvey, 2003; Wageumakers, Farrell & Ratcliff, 2004, 2005). Absolute identification researchers were early pioneers in such analyses (Holland & Lockhcad, 1968; Ward & Lockhead, 1970, 1971), showing that the occurrence of identification errors depends in a complicated way on previous stimuli and responses. These sequential effects come in two broad varieties, and turn out to be one of the most challenging aspects of absolute identification for theoretical accounts. Some researchers even believe these sequential effects are the defining feature of absolute identification (e.g., Laming, 1984; Stewart et al., 2005). In what follows, we refer to the current decision trial as “trial N ”, which allows us to refer to the preceding trials as N − 1, N − 2 and so on. assimilation and contrast The response made on trial N is “assimilated” toward (i.e., tends to be similar to) the stimulus presented on trial N−1. For example, suppose that stimulus number 5 is presented on trial N . If the stimulus on trial N−1 had been number 1 (which is smaller), then an error on trial N is more likely to be an underestimate (e.g., number 4) than an overestimate (number 6) (Holland & Lockhead, 1968; Luce, Nosofsky, Green, & Smith, 1982; Stewart et al., 2005; Ward & Lockhead, 1970). Stimuli further back in the sequence of trials have an opposite influence on responses. The response on trial N tends to “contrast” with (i.e., be dissimilar to) the stimuli on trials N−2, N−3, and further back in the sequence. For example, incorrect
why is accurately labeling simple magnitudes so hard?
123
Theories of Absolute Identification We now give a brief overview of the different theoretical accounts of the benchmark phenomena just described. Fascination with the counterintuitive and intricate nature of absolute identification has spawned many theories and our coverage does not aim to be encyclopeadic (we recommend Stewart et al., 2005, who give an excellent and comprehensive coverage). Rather, we aim to cover prominent examples of several different conceptual classes of theories, beginning with Thurstonian models and then covering three types of theories that have yielded models still under active investigation.
Thurstonian Models The basic Thurstonian model assumes that any given stimulus evokes a noisy representation of 124
basic cognitive skills
0.4
Mean Error at Trial N + X
responses tend to be overestimates of current stimuli when the stimuli were small on trial N−2, and vice versa. The contrast effect is usually smaller than the assimilation effect, and though it tends to decrease as trials further back are considered, contrast can persist until about trial N−5 (Holland & Lockhead, 1968; Lacouture, 1997; Ward & Lockhead, 1971). One apparent difficulty in thinking about assimilation and contrast is that stimulus labels and responses are correlated, simply because participants typically perform above chance level accuracy. This can make it difficult to establish whether responses on the current trial are assimilated toward the previous stimulus or the previous response. Nevertheless, progress has been made on this question by using autoregressive measurement models (e.g.: De Carlo, 1992; De Carlo & Cross, 1990). These analyses have generally supported the idea that assimilation and contrast occur mostly as described earlier: as biases caused by previous stimuli, rather than responses. Intriguingly, recent work by Malmberg and Annis (2012) has shown that assimilation and contrast effects have close analogues in short-term memory decisions. The combination of assimilation and contrast, shown in Figure 6.3, poses a particularly challenging set of results for theories of absolute identification. A successful theoretical account of absolute identification must predict that responses are biased toward the previous stimulus (assimilation) but that the bias switches direction (contrast) as that stimulus recedes in memory. Several possibilities for how this occurs have been proposed, and we now discuss them.
0.2
0.0
–0.2
–0.4 1
2
3 4 5 6 7 Trials (X) Since Stimulus N
8
Fig. 6.3 Assimilation (at lag X = 1) and contrast effects (at X > 1) in data from Holland & Lockhead (1968). When X = 1, the average error is negative for smaller stimuli (filled symbols) and positive for larger stimuli (unfilled symbols). That is, when the stimuli on the previous trial were small, stimulus magnitudes on the current trial were underestimates, whereas stimulus magnitudes were overestimated when the previous stimuli were large. When X > 1, the pattern reverses, and a contrast effect is observed. Reproduced from Figure 4 of Brown, Marley, Donkin, and Heathcote (2008).
absolute magnitude on an internal scale. The scale is divided by decision criteria into a set of response categories. The criteria that divide the response categories are allowed to vary between trials because of imperfect memory or shifting response bias. The noisy response categories can account for information limits when the number of stimuli increase for a fixed range, as the bounds become closer and so the noisy stimulus representation and the noisy category bounds lead to more errors in responding. Durlach and Braida (1969) extended the basic Thurstonian model to account for the invariance of performance with changes in the range of stimuli. They proposed that, because of limits on memory, only recently presented stimuli are used to define the context within which the current stimulus is presented. This additional source of variability is modeled as being proportional to the range of stimuli, and as such can produce the required invariance. Basic Thurstonian models can also account for bow effects, but their account is not entirely satisfactory. The models predict greater accuracy for edge stimuli because Thurstonian models that predict erroneous responses are mostly adjacent to the correct response, and edge stimuli have only one adjacent response, whereas internal stimuli have two. This account is sometimes called a “response
restriction” account, because improved accuracy is predicted for edge stimuli simply because they have fewer neighbouring stimuli to be confused with. Although this restricted response explanation certainly does play some role in bow effects, it fails to predict the observed gradual decrease in accuracy that is observed as stimuli become more internal to the set. Many researchers have used a d -based measure of performance (Luce et al., 1982), in place of raw accuracy. The d measure is, under some assumptions, insensitive to response effects such as bias and response restriction. The d measure still reliably shows deep bow effects, and this is something that response restriction explanations of the bow effect cannot explain. Braida et al., (1984) elaborated the basic Thurstonian model’s account of bow effects to allow for greater variability for stimuli near the center of the range than the ends. They explained this by proposing that stimuli are judged against two “anchors” at the extreme ends of the stimulus range, and the distances between these anchors and the presented stimulus are counted using a noisy measurement unit. The further a stimulus is from those anchors, the noisier the distance measurement becomes. This produces a gradual bow effect with increasingly frequent identification errors toward the middle of the measurement range. Luce, Green, and Weber (1976) proposed a related model, suggesting that bow effects may be due to more attention being paid to the edges of the stimulus range, thus reducing the variability of stimulus representations in those regions. Triesman (1985) modified the basic Thurstonian model in a different way, in order to accommodate sequential effects. Triesman and Williams’s (1984) criterion-setting theory proposed that the criteria that set the bounds for response categories change on a trial-by-trial basis due to two factors: a stabilizing mechanism and a tracking mechanism. The tracking mechanism moves the criteria away from the previously observed stimulus. This is based on the assumption that, in the real world stimuli do not occur randomly, but that, given something is perceived, it is likely to reappear again sometime soon. The tracking mechanism moves the criterion away from the previous stimulus (thereby expanding the stimulus range for the response just given) so that when it is presented again, it is more often correctly identified. The stabilizing mechanism moves the criteria such that they are centered on the mean of the previous stimuli. The stabilizing mechanism acts to counteract the
tracking mechanism, moving criteria back in a way that maintains balanced responding in the long run. Treisman assumed that the tracking mechanism is stronger than the stabilizing mechanism, but decays more quickly. In this way, Treisman’s model accounts for assimilation to trial N −1, where the tracking mechanism dominates, and contrast away from previous trials, where the stabilizing mechanism dominates.
Exemplar Models Exemplar models have proven very successful in accounting for categorization behavior, and this makes them promising candidates for theories of absolute identification because of the close similarity between absolute identification and categorization. Exemplar models assume absolute identification is accomplished by determining the similarity between a to-be-identified stimulus, and the memory representations for previous stimuli. Each stimulus is assumed to be represented in memory along with its associated label. The probability of response i is proportional to the similarity between the current stimulus j and all exemplars for response i. In general, exemplar models face the problem of not being able to account for the fundamental information limit of absolute identification: when the range of the stimulus set is increased but the number of stimuli remains fixed, these models predict that the memory representations should be more easily discriminable, and so performance should improve. As Braida and Durlach (1972) showed, this does not happen. Nosofsky (1997) extended exemplar models to also make predictions about the time taken to make decisions, rather than just the decision that is made. His exemplar-based random walk model (EBRW; Nosofsky & Palmeri, 1997) assumes that the representations of stimuli in memory are normally distributed across the stimulus dimension. Upon presentation of a stimulus, the exemplars race to be retrieved from memory with a speed that is an increasing function of the similarity between the current stimulus and each exemplar. Each time an exemplar is retrieved, a counter for the associated response is incremented, whereas the counters associated with other responses are decremented. The race continues until one counter reaches a threshold and the corresponding response is given. Nosofsky’s model accounts for bow effects by assuming that stimuli toward the edge of the range have smaller variance in their perceptual
why is accurately labeling simple magnitudes so hard?
125
representation. The EBRW model makes no attempt to account for sequential effects or the information limit phenomena. It is worth noting, however, that the EBRW model is one of the few absolute identification models to consider response time. Indeed, the EBRW makes detailed and precise predictions for response times, including full response time distributions for both correct and incorrect responses. Petrov and Anderson (2005) proposed an exemplar model of absolute identification called ANCHOR based on the ACT-R architecture (Anderson, 1990, Anderson & Lebiere, 1998). They assume that unidimensional stimuli are encoded by a perceptual subsystem into an absolute magnitude. The magnitude is then processed by the central subsystem, comparing it with some exemplars or “anchors” stored in long-term memory. The central subsystem is thought to be dynamic and evolves from trial to trial throughout the absolute identification task. The perceptual processing that creates the magnitude and the selection of the anchor exemplar are both stochastic, with selection of the exemplar based on similarity between the current stimulus and memory representations, as well as the previously presented stimulus. The ANCHOR model accounted for bow effects in accuracy (Petrov & Anderson, 2005) via the restricted response set explanation of bow effects, and, therefore, it suffers the same difficulty as Thurstonian models in accounting for bow effects in d . Though not explicitly tested, Stewart et al. (2005) suggest that the ANCHOR model would probably be able to account for information limit when range and set size were varied due to noisy parts of the model unrelated to spacing of stimuli. The ANCHOR model accounts for assimilation by making exemplars that have been recently used more likely to be retrieved, but does not attempt to account for contrast effects. Kent and Lamberts (2005) proposed an exemplar model of absolute identification based on an adaptation of Lamberts’s (2000) extended generalized context model for response times (EGCM-RT). EGCM-RT, which in turn is based on Nosofsky’s (1986) general context model, assumes that information about the current stimulus is sampled repeatedly until sufficient evidence is accumulated to choose a response. Kent and Lamberts’ model successfully accounts not only for the basic bow effect that is observed in accuracy but also for the inverted bow effect that is observed in response time (i.e., responses to stimuli from the 126
basic cognitive skills
center of the range are slower). It does so because stimuli toward the end of the range are relatively more isolated. This adds a different mechanism to the restricted response set explanation of bow effects, because not only do edge stimuli have fewer competing response alternatives, they also have smaller summed similarity than stimuli in the center of the stimulus range. A gradual decrease in accuracy and increase in response time is predicted, because summed similarity increases gradually for stimuli that are closer to the center of the range. Without modification, the EGCM-RT was not able to account for systematic differences in the shape of bow effects as a function of the number of stimuli to be identified (set size; see Figure 6.2). However, Kent and Lamberts (2005) were able to capture these differences by assuming that the amount of information sampled from a stimulus was a decreasing function of the number of pieces of information already sampled— a plausible assumption of “diminishing returns.” By assuming that additional samples from a stimulus were decreasingly useful in discriminating between members of the stimulus set, an information limit was imposed on the basic EGCM-RT model. Kent and Lamberts did not attempt to account for sequential effects.
Relative Judgement Models The previous Thurstonian and exemplar models tended to focus more heavily on explaining fundamental capacity limitations and bow effects, and paid less attention to explaining sequential effects in absolute identification data. The defining feature of relative judgment models, on the other hand, is that decisions are based on the difference between current and previous stimuli (or responses), rather than being based directly on representations of the absolute magnitudes of stimuli. This leads to a natural focus on sequential effects. Relative versus absolute judgments is not just a theoretical distinction in absolute identification modeling but is also important in some applied areas. For example, musicians are frequently trained in the skill of relative judgment (judging musical “intervals”) but almost never in absolute judgment, because relative judgment is useful in musical performance, whereas absolute judgment is not (and may even be maladaptive). This difference may help explain the rarity of “perfect pitch” in musicians, which we discuss later.
Holland and Lockhead (1968) proposed that responses are made by combining feedback from the previous trial and the perceived distance between the current and previous stimuli. This basic relative judgment mechanism accounts for assimilation and contrast by simply assuming that the judged difference between the current and previous stimuli is biased toward the previous stimulus and away from earlier stimuli. For example, consider the case in which a small stimulus was presented on the previous trial (N−1). This means that, on average, the stimuli presented on earlier trials (N−2, N−3, . . .) were probably larger stimuli. The memories for these larger earlier stimuli interfere with the judgment of the distance between the previous and current stimulus in a way that causes the distance to be underestimated. Hence, a response based on this distance will assimilate to the stimulus presented on trial N−1. As Stewart et al. (2005) point out, this approach can explain assimilation and contrast on average, but fails to provide a general account. To recount their example, imagine that a small stimulus on trial N−1, say 3 is followed by a smaller stimulus, 2. Now, the interference from more previous stimuli will cause an overestimate of the distance between the stimuli, and produce contrast, whereas assimilation is usually still observed in such cases. Laming (1984) proposed a strict version of a relative judgment model, assuming that no absolute information is used, and that only the difference between the previous and current stimulus is considered. In particular, decisions are assumed to be made in a relatively coarse manner, such that the current stimulus is judged in terms of only five categories: “much less than,” “less than,” “equal to,” “more than,” or “much more than.” Such limited categorical information provides a natural account of the fundamental capacity limits, within a relative judgment framework. Stewart et al. (2005) proposed the most current and successful relative judgment model of absolute identification. In their model, only the series of differences between each stimulus and the next is represented internally. These differences, along with feedback from the previous trial, are used to produce a response. The formula used to generate a response, RN on trial DC
N is: RN = FN −1 + N λ,N −1 +ρZ , where FN −1 C is the feedback on trial N − 1, DN ,N −1 is the difference between the representations of differences on trial N and N −1, λ is a scaling parameter,
C ρ is a parameter that increases with DN ,N −1 , and Z is a normally distributed variable with mean zero and standard deviation σ . Stewart et al. (2005) demonstrate that their instantiation of a relative judgment model is capable of producing all the classical response-choice related benchmark phenomena. Stewart et al., (2005) model accommodates the information limit and bow effect in accuracy through related mechanisms. The information limit is accounted for via the inclusion of a parameter (Z ), which is assumed to represent noise in the mapping of the physical stimulus to its internal representation. Bow effects are accounted for by assuming that Z is scaled by parameter ρ, which depends on the position of the current stimulus within the range of stimuli. For example, if there are eight stimuli to be identified, the previous feedback was 7 and the current stimulus is judged to be larger, then the influence of the noise, σ , is smaller than when the stimulus is closer to the center of the stimulus range. The relative judgment model accounts for assimilation and contrast by assuming that the judgment of the distance between the previous and current stimulus is influenced by previous distance judgments, and that their impact decreases with their position in the trial sequence (i.e., the distance judgment on trial N−1 has the most influence, followed by that on trial N−2, etc.).
Restricted Capacity Marley and colleagues have proposed a number of models that attribute poor performance in absolute identification to limited capacity in memory or attention. Marley and Cook (1984, 1986) assume that the full range of stimuli in an experiment are mapped onto an experimental context, which might be thought of as a fixed-capacity attention or memory store. When a stimulus is presented, its relative position within the set of stimuli is located within a context by reference to “anchors” located near the ends of the stimulus range. The relative position of the stimulus is then used to judge its magnitude and subsequently assign a response label. Capacity limitations in the model are built into the way the context is maintained. Attention to the stimulus range (context) of the experiment is assumed to be maintained by constant “rehearsal” of that particular segment of the stimulus dimension. Rehearsal is assumed to operate as a Poisson pulse process that directs activity to the relevant context, but this activation is assumed to passively decay.
why is accurately labeling simple magnitudes so hard?
127
The power of the rehearsal process is assumed to be fixed within an individual, thus yielding a limit on performance when either more stimuli are added, or the range of stimuli is increased. Marley and Cook’s (1984, 1986) model naturally predicts the bow effect in response accuracy and d because the relative magnitude of a stimulus is judged according to the amount of rehearsal activity between the current stimulus location, and the closest anchor. The magnitude of stimuli in the center of the range will, therefore, be estimated with greater variance than edge stimuli, since the noise in the Poisson rehearsal process will have more influence on estimation (i.e., similar to Durlach & Braida, 1969). Lacouture and Marley (1995, 2004) proposed an alternative means of explaining the information capacity limit and bow effect through their mapping model. The model assumes a basic and very simple absolute magnitude estimate as its input, and transforms it into a set of response-output strengths for each of the possible responses via an internal (“hidden”) unit, in a similar way that a set of tuning curves might transform a magnitude estimate into a set of response tendencies. Since the hidden unit normalizes the input into a unit (0–1) range, a small and fixed amount of noise in the input magnitude estimate has greater influence when the number of response alternatives increases, as more stimuli are packed into the same psychological space. In Lacouture and Marley (2004), the output strengths coming from the mapping model were used to drive a leaky, competing accumulator (LCA) model (Usher & McClelland, 2001). This expanded the explanatory scope of their model to include RT, not only in terms of mean RT but also variability in RT (i.e., the full distribution of RT). The restricted capacity models just reviewed make no attempt to account for any of the benchmark sequential effects in absolute identification. The SAMBA model (Brown et al., 2008) extends these earlier models, and incorporates elements of both the Poisson rehearsal and mapping processes from the previous two models, but uses a deterministic version of the LCA, Brown and Heathcote’s (2005) ballistic accumulator model. The authors claimed SAMBA to be the most comprehensive theory of absolute identification yet proposed because they showed it was capable of accounting for all of the aforementioned benchmark phenomena in absolute identification not just in terms of response choices, but also in terms of the full distribution of response times. 128
basic cognitive skills
SAMBA is an acronym for the model’s three stages. The Selective Attention aspect of Marley and Cook (1984) produces a magnitude estimate for a stimulus, which is then used as an input to the Mapping model of Lacouture and Marley (1995). The mapping process transforms the single magnitude estimate into N response strengths, which then drive a corresponding number of Ballistic Accumulators. The ballistic accumulators are a set of evidence accumulators that accrue activation at a rate determined by the output of the mapping stage of the model. Evidence in each accumulator increases ballistically (that is, deterministically, without moment-to-moment noise), suffers from passive decay, and is inhibited by the evidence in other accumulators. The information limit and bow effects arise out of the first two stages of the SAMBA model. SAMBA provides an advance over earlier restricted capacity models by also accounting for assimilation and contrast. Assimilation is accounted for by assuming that the evidence in accumulators passively decays between trials, beginning from their level at the time at which a response was made. For example, this means that the accumulator that won the race on the previous trial will begin the current trial with the highest level of activation. This benefit for responses close to previous stimuli leads to assimilation toward responses made on the previous trial. Contrast is incorporated into the selective attention stage of the model. Brown et al. (2008) assumed that between trials, rehearsal activity is preferentially directed to the elements of the context corresponding to the stimulus presented on the previous trial. On subsequent trials, this additional activation around previous stimuli leads to contrast away from those stimuli. That is, since magnitude is estimated by summing the amount of activation between an anchor and the stimulus, increased activity will “push” the magnitude estimation away from the previous stimulus. Table 6.1 summarizes the ability of each model class to account for existing benchmark phenomena. All the classes of models discussed so far are basically capable of accounting for what we are calling historical benchmarks: capacity limitations, bow effects, and sequential effects. Interested readers should see Stewart et al. (2005) for a more detailed table outlining the various models’ ability to account for historical benchmark data. For the remainder of the chapter we will focus on recent issues in absolute identification. As seen in Table 6.1, these issues pose a challenge for many theories,
Table 6.1. A simplified summary of the ability of models within each class to account for historical and more recent phenomena. Empirical Phenomena Historical Benchmarks Model Type Thurstonian Exemplar Relative Capacity
Current Issues
Capacity Limitations
Bow Effects
Sequential Effects
RT
Uneven Spacing
Inter-trial Interval
Yes Yes Yes Yes
Yes Yes Yes Yes
Yes No Yes Yes
No Yes No Yes
No No Yesa Yes
No No No Nob
Learning No No No No
a Depending b But
on the definition one allows for a “purely relative” model–see Brown et al., (2009). see Donkin, Chan, and Tran (2014) for a revised version of SAMBA capable of accounting for this phenomenon
and there are a number of results that yet remain unresolved. We will not attempt to resolve these issues here, but instead hope to lay out a guide as to what should constrain model development moving forward.
Current Issues in Absolute Identification Absolute and Relative Judgment An unresolved tension in the modeling of absolute identification concerns the merits of “absolute” versus “relative” accounts. Recently, the line between absolute and relative models has become blurred, leading to some important tasks for future research. Most immediately, exactly what defines a purely relative versus purely absolute model needs clarification? Subsequently, the limitations of each class need to be more clearly established, especially if the current “integrated” models, which include both absolute and relative aspects, are to be justified. Absolute and relative judgment models are defined in a similar spirit to the way the two terms are used in the judgment of musical notes, earlier. An “absolute” account proposes that each new stimulus is assigned a label by comparison with some stable referents stored in memory, such as the extreme ends of the stimulus range. A “relative” account proposes that each new stimulus is judged instead against the memory of only very recently observed stimuli (perhaps just the most recent single stimulus). The earliest Thurstonian and exemplar models were purely absolute, in that each stimulus was judged only against long-term referents. Similarly, the earliest relative judgment models (Holland & Lockhead, 1968; Laming, 1984) were purely relative in that stimuli were compared only to the most recently-observed prior stimulus. These “pure” models had difficulty accounting for fundamental phenomena. The purely relative models could not
easily account for bow effects in d , or capacity limitations, and the purely absolute models could not easily account for any of the sequential effects. Purely relative and purely absolute models can be envisaged as the endpoints on a continuum. For example, a purely relative model can be made a little less relative by allowing comparisons against the last two or three observed stimuli. The end point of such a process is an absolute model: an exemplar model, where all previous stimuli are used for comparison. Similarly, a purely absolute model can be made a little more relative by allowing a stimulus to be judged by comparison to recent stimuli, as well as to long-term referents. To accommodate the growing list of complicated benchmark phenomena, recent models have moved away from purely relative or purely absolute accounts. For example, the SAMBA model (Brown et al., 2008) was able to accommodate almost all benchmark phenomena with purely absolute assumptions. However, one phenomenon was only able to be modeled within SAMBA’s framework by including a relative process, judgment against the most recently observed magnitude. The phenomenon in question was the improved accuracy and response times observed for repeated stimuli, and for stimuli that are similar, but not identical, to the previous stimulus. It seems ad hoc to incorporate an otherwise unnecessary model component just to accommodate this one phenomenon, and so future work might investigate whether this phenomenon can be explained more parsimoniously. If it is possible to explain this phenomenon without recourse to a relative judgment process, this could simplify theoretical arguments, by once again having a purely absolute model that fits the data. On the other hand, the purely relative models of Holland and Lockhead (1968) and Laming
why is accurately labeling simple magnitudes so hard?
129
(1984) were augmented by Stewart et al. (2005). Stewart et al.’s relative judgment model includes some elements which might or might not be classed as “absolute”, depending on taste. For example, in Stewart et al.’s model, the amount of variability in the internal representation of a magnitude depends on the distance between that magnitude and the end of the range. This comparison with the end of the range sounds just like an absolute judgment procedure, but it could avoid that label by instead supposing that magnitudes being represented are response magnitudes, rather than stimulus magnitudes. How this argument would hold up in general is unclear: for example, if non-numeric response labels were used, as is customary in categorization tasks. Brown et al. (2009) tried to identify the shortcomings of relative judgment accounts without recourse to particular theories or detailed and questionable assumptions about precisely which mechanism are absolute vs. relative. They noted that relative judgment theories proceed by allowing exactly one piece of absolute knowledge about stimulus magnitude: the difference in magnitude between adjacent stimuli in the decision set. This knowledge is necessary to translate observed differences in magnitude between successive stimuli into predicted differences in their response labels. For example, suppose a set of 10 lines are used, with each line differing in length by 2 cm from its neighbors. If the previous trial used line 4 (which is known, after the feedback is given) and the current line is 6 cm shorter, then knowledge of the 2 cm spacing in the set can be used to deduce that the current line must be three responses smaller (line 1). Brown et al. reasoned that this kind of account must break down in the more general case, when the spacing between adjacent stimuli in the set is uneven. In that case, a relative difference of 6 cm between the current and previous lines implies a different number of response units depending on which line was shown. Brown et al. (2009) re-ran the classic experiment of Lockhead and Hinson (1986), in which absolute identification was performed on three loudness levels. In one condition, the loudness values were evenly spaced (2 dB apart) but in the other conditions, the gap between one pair of stimuli was three times as large (6 dB). As shown in Figure 6.4, a relative judgment account failed to accommodate data from the unevenly-spaced conditions. Brown et al. interpreted this as a failure of purely relative models. However, Stewart and 130
basic cognitive skills
Matthews (2009) explored an augmented version of the relative judgment model, endowed with knowledge of all the various stimulus spacings between all pairs of stimuli. Although such a model can easily accommodate the data, what is less clear is the status of that model on the absolute versus relative continuum. Proponents of relative models might construe such knowledge as purely relative, as it involves only knowledge of differences between stimuli. On the other hand, proponents of absolute models might construe such knowledge as purely absolute, as it assumes a long-term memory for the structure of the entire stimulus set. The most attractive way forward for the debate between absolute and relative accounts is probably by resolution of the exact definition of the basic terms. However, since that is likely a more difficult task than it appears, an alternative way forward is by simply fitting the existing models to data, and comparing their performance on quantitative measures of goodness of fit. This approach has worked well in other fields, and avoids many of the problems inherent in the search for qualitative differences between classes of models.
Learning The limit on people’s ability to learn to identify more than around seven unidimensional stimuli was firmly established early in the history of absolute identification research. However, the remarkable levels of achievement that are displayed by experts after extended practice in a broad range of domains—e.g., musicians with absolute pitch— makes this limit puzzling. The following quote, taken from Shiffrin and Nosofsky (1994) sums up this conundrum with capacity limits nicely. As an anecdotal example, Robert M. Nosofsky started his research career around 1980 in the acoustical laboratory of David Green and R. Duncan Luce, two researchers who happened at the time to be studying absolute identification of loudness . . . As a cocky young graduate student, Nosofsky “knew” that with a bit of practice, he could surely learn to perfectly identify a set of 12 loudnesses. After locking himself in one of Green’s sound-proof booths for several weeks, and hearing tone after tone after tone, his absolute identification performance remained unchanged. He did succeed, however, in increasing substantially his need for psychotherapy.
Research in the last 10 years, however, suggests that Professor Nosofsky was just unlucky to choose
Low Spread Even Spread High Spread
Response Accuracy
73 Spread: 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3
Low
76
79 82 85 Intensity (dB)
88
Even
91
High
Data
SAMBA
RJM
Stimulus
1
2
3
1
2 3 Previous Stimulus
1
2
3
1 2 3
Fig. 6.4 Top panel: Schematic illustration of the stimuli used in Brown et al.’s (2009) experiment. Bottom panel: Response accuracy (y-axis) for each stimulus (symbols) conditioned on the previous stimulus (x-axis). The upper row of panesl show data, the middle row show predictions from the SAMBA model, and the lower row of panels show predictions from the RJM. The three columns correspond to the low-spread, even-spread and high-spread conditions. Reproduced from Figures 1 and 4, Brown et al. (2009).
the particular stimulus continuum (loudness), and that other stimulus dimensions are more amenable to learning, at least for some people. Rouder, Morey, Cowan, and Pfaltz (2004) showed that at least one of their participants, the redoubtable RM, was capable of far exceeding a limit of seven items in absolute identification of line length. Indeed, all three participants in their experiments were shown to display some learning. However, after extended practice, RM (one of the authors) displayed a remarkable ability to accurately identify
up to almost 20 stimuli. The other two participants, despite not performing at quite the same level, both achieved performance equivalent to the perfect identification of around 10–13 items, both above Miller’s limit of 9. Dodds, Donkin, Brown, and Heathcote (2011) replicated the key finding from Rouder et al.’s (2004) study. Dodds et al. then extended the findings in a series of seven experiments. Many more participants were observed to show substantial learning, though none quite reached the high bar
why is accurately labeling simple magnitudes so hard?
131
4.0 3.5 3.0 2.5
3 3
3 5
461 2
5
53
5 1 6 42
3
1 42 6
1 2 64
3 5
5
1 64 2
1 6 4 2
3 5
3 5
1 4 6
1 4 6
2
2
3 5
35
1 64
1 4 6
16
8
2
2.0 5132 64
1.5
51 6 432
526 3 1
2 635
4
1 4
3
4
23 6 51
2 3 516 4
2 36 51
6
7
4
4
2 361 5 4
32 15 6 4
32
8
9
10
51 6
1.0
4
Equivalent Number of Correctly−Identified Stimuli
Information Transmitted (bits)
4.5
2
1
2
3
4
5
6
7
8
9
10
1
2
5
Practise Session
Fig. 6.5 Information transmission as a function of sessions of practice plotted for two experiments from Dodds et al. (2011). The left panel of the figure is a replication of the unusually large amount of learning observed for lines of varying length. In the right panel, however, we see the more standard limited capacity for tones of varying loudness, that was more commonly reported in absolute identification experiments. Reproduced from Figures 1 and 5, Dodds et al. (2011).
for absolute identification of line lengths set by RM. Dodds et al. (2011) showed that the learning could be observed for stimulus modalities other than line lengths, including the separation between dots (which removed the confound of brightness with line length), and the degree of angular rotation of a line. As shown in Figure 6.5, auditory stimulus sets, particularly the tone loudnesses that repelled Nosofsky’s determined assault, were shown to be much more difficult to learn. Tones of varying frequency presented an interesting case, given the popular belief in the existence of absolute or perfect pitch. Four of the six participants who practiced with tones of varying frequencies showed no substantial improvement. However, one participant showed learning on the order of that displayed by RM with line lengths, and another showed smaller, but still substantial, improvement. By the end of just 10 experimental sessions, the best performer could identify about 16 items and was showing of no signs of a slowing in their learning rate, suggesting they could have achieved even better performance with further practice. Dodds et al.’s (2011) results might be crudely described as showing that absolute identification performance can be improved for all kinds of stimulus sets with extended practice; except for stimulus sets based on tones of varying loudness. This result resolves the apparent contradiction between new and old findings on practice effects. These new results, however, raise a host of new issues. For one, there is the unresolved question of why tone loudness cannot be learned, whereas other stimulus types can: and further, why the ease of learning
132
basic cognitive skills
might differ between other stimulus sets. In further work, Dodds, Rae, and Brown (2012) found that those continua supporting learning may have a psychological representation of magnitude that is more complex than the simple, one-dimensional physical structure of the stimulus set (see also Dodds, Donkin, Brown, & Heathcote, 2010). For example, although lines vary on a single physical dimension—length—they may, after extended practice, develop a higher-dimensional psychological representation. This greater dimensionality provides additional information for the participants to learn, and may allow for continued improvement in performance (Rouder, 2001). For lines of various length, it is not clear what these extra dimensions of information might be. However, for some other stimulus sets, more is known. For example, tones that vary in pitch form a physically onedimensional set, but this dimension is represented psychologically as a two-dimensional helix: the wellknown separation of musical notes into chroma and octaves. Dodds et al.’s (2011) study also revealed considerable individual differences in the ability to learn. The best predictor of such differences seems to be initial performance. That is, participants who performed well in the first few hundred trials also were also those who showed the greatest improvement with subsequent practice. The direction of this correlation can be considered surprising. One might naively expect a negative correlation, since poorer initial performance leaves greater room for improvement. An open question about the nature of learning in absolute identification regards the
reason for the observed correlation. One possibility lies in a link between the psychological representation of stimulus sets and performance. Dodds et al. (2012) observed that extended practice was related to a move away from simple (one-dimensional) psychological representations toward more complex representations. Although there is a chicken-andegg problem that requires attention here, it suggests that improved absolute identification performance occurs through reorganization of the psychological representation of the stimulus set. An important unanswered question is how theories of absolute identification should incorporate learning. Accounting for learning is no trivial task, especially since most models were developed to account for exactly the opposite—severe and resistant capacity limitations. Dodds et al. (2011) made inroads to understanding how to account for the effects of practice by looking at what practice does to the previously described benchmark phenomena. Bow effects remain across practice with approximately the same magnitude. Similarly, the stimulus presented on trial N −1 maintains an assimilative effect at the beginning and end of practice. However, the contrast effect from stimuli further back in the trial sequence (N −2, N −3, etc.) is greatly reduced by practice. This reduction in contrast effects with practice can be seen as an adaptive behavior. Contrast effects represent incorrect responses, but such errors can be adaptive in situations where the to-be-identified stimuli drift slowly with time. Such drift did not occur in the experiment, which makes learning to reduce contrast effects adaptive. Incorporating learning into absolute identification models by modulating contrast mechanisms is, therefore, a good start to accounting for practice effects. However, changes in contrast are unlikely to be the sole locus of learning; for example, in the SAMBA model we have found that completely removing the contrast mechanism leads to an improvement in performance that was only about one-third of the size observed in some of Dodds et al.’s participants. Clearly, practice effects provide a challenge to existing models claiming to aspire to a complete account of absolute identification. The set of empirical results we have summarized provide clues about the locus of the effect of practice, as well as providing concrete targets for model fitting. Further, it seems likely that the increase in performance for some modalities may be, at least partially, driven by the development of higher-order representations of stimuli. Finally, the magnitudes
of individual differences in practice effects should be in some way related to initial performance.
Absolute Identification vs. Perfect Pitch “Perfect pitch,” also known as absolute pitch, is the ability to perform almost perfectly on an absolute identification task where the stimuli are a set of musical notes that must be labeled with their chroma and octave (e.g. “C4”, also known as “middle C”). Absolute pitch is considered to be rare with only 1 in 10,000 of the general population reported as having the ability (Bachem, 1955: Takeuchi & Hulse, 1993). Rates are said to rise to 1 in 1,500 among amateur musicians (Profita & Bidder, 1988), and up to 1 in 7 in highly accomplished musicians (Baharloo, Johnston, Service, Gitschier, & Freimer, 1998). However, those rare people who possess absolute pitch are able to accurately identify dozens of different stimuli, well beyond the limits observed in other stimulus sets. This raises questions about the difference between absolute pitch and absolute identification using other stimulus sets. Miller (1956) noted this apparent dichotomy, but was unable to explain it. An obvious candidate explanation for the difference in performance between absolute identification tasks in general and absolute pitch tasks is the difference in the physical dimensionality of the stimuli. Live or recorded musical notes are sometimes used in absolute pitch tasks, but these stimuli are physically multidimensional, consisting of a fundamental frequency and a series of harmonic overtones. There are also marked variations in timbre, volume, resonance and decay characteristics across the registers of the musical range (Lockhead & Byrd, 1981; Terhardt & Seewann, 1983). Any of these attributes might be used by observers to out-perform the usual limitations of absolute identification. However, even when the stimuli are truly one-dimensional—computer-generated sine tones—some people can still exhibit absolute pitch performance (Athos et al., 2007). This may represent an endpoint of the kind of learning observed by Dodds et al. (2011) and Dodds et al. (2012), where extended practice altered the psychological representation of pure tones, supporting improved identification. This explanation is further supported by the prevalence of “octave errors” in identification of pitch (where a note is mistaken for a note with the same chromatic name in a different octave). Another possible explanation for good performance in absolute pitch tasks is that people,
why is accurately labeling simple magnitudes so hard?
133
especially musicians, have more exposure to the stimuli generally used in absolute pitch tasks, namely musical notes. Previous thinking considered absolute pitch to be an innate quality possessed by very few people, and a skill unable to be learned (e.g., Revesz, 1953). However, many recent researchers agree there is a critical period, under the age of approximately eight, when children can learn absolute pitch if exposed to enough musical training, with several researchers noting a correlation between absolute pitch ability and musical training during the critical period (e.g., Levitin & Rogers, 2005). However, exposure to musical training during this critical period is not sufficient to develop absolute pitch, as most people who are musically trained during this period do not develop the ability (Baharloo et al., 1998). There are conflicting views about whether learning is possible beyond the critical period. Some researchers maintain absolute pitch cannot be learned to any level of fluency after the critical period (e.g., Zatorre, 2003), whereas others argue that with enough practice, learning is possible (e.g., Lundin, 1963). Dodds et al.’s (2011) results support the latter view. Evidence that absolute pitch is a learned ability, rather than innate, comes from the nonuniform performance across the range of notes. People with absolute pitch most accurately identify those notes to which they have had more exposure. Middle C and other white-key notes on the piano are often most accurately identified (Lundin, 1963; Takeuchi & Hulse, 1993), perhaps because these are the first notes learned in standard piano training (Athos et al., 2007; Miyazaki, 2004). Similarly, violin players are able to more accurately identify the open A string, as they are very familiar with this tone (Brammer, 1951), and in general musicians more accurately identify pitch on their own instrument (Takeuchi & Hulse, 1993). Prior work on absolute pitch has often been confounded by the presence of feedback, and by the skill known as “relative pitch”. Relative pitch is the ability to identify differences between notes, rather than individual notes in isolation (Miyazaki, 1995). The difference between two notes is known as a “musical interval,” and musicians are trained to identify these intervals accurately. Relative pitch plays an important role in musicianship, whereas absolute pitch does not. Confounds arise when absolute pitch is tested in the usual way, by a succession of notes with feedback on the correct answer provided after each. Participants with good relative pitch skills can use the judged intervals, combined 134
basic cognitive skills
with knowledge of the previous note from feedback, to perform accurately, even without possessing absolute pitch skills. Some experimenters have attempted to separate the contribution of relative and absolute pitch by using interference tasks between trials (e.g. Zatorre & Beckett, 1989), or separating successive tones by long time intervals (Baharloo et al., 1998). A simpler technique to control for relative pitch is to remove feedback information, which makes knowledge of the intervals unhelpful. However, feedback must be given at some points, both for motivation and to stop poorly performing participants from becoming wildly inaccurate. A compromise involves alternate blocks with feedback and without feedback (Ward & Lockhead, 1970). Speakers of tonal languages (e.g., Mandarin, Cantonese, Thai) have been reported to have an advantage in pitch perception due to their use of pitch in conveying the meaning of words (e.g., Deutsch, Henthorn, Marvin, & Xu, 2005). This hypothesis incorporates the ideas that extended practice with identifying pitch (when speaking a tonal language) and exposure to this task during childhood (when learning language) might both support the development of absolute pitch. Various studies have provided indirect support for the assumed link, between tonal language and absolute pitch, such as Pfordresher and Brown (2009), who found that native tonal language speakers were better able to imitate and perceptually discriminate musical pitch. Deutsch et al. (2009) investigated the pitch ranges of female speech in two relatively isolated villages in China, and found that the pitch range of speech is heavily influenced by an absolute pitch template that is developed through long-term exposure. However, other studies have reported that tonal language speakers have no advantage in absolute pitch (e.g., Bent, Bradlow, & Wright, 2006; Schellenberg & Trehub, 2007). In a recent experiment from our lab (not yet published), 60 native Chinese language speakers identified pure sine tones in a standard absolute identification paradigm. All 60 participants were university students at Sichuan University, Chengdu, China, 30 of whom were enrolled in the Music Faculty, and 30 of whom were enrolled in the Faculty of Administration. Although the music students outperformed the administration students when feedback was provided (p ≤ 0.01), there was no difference between the two groups when no feedback was given (p = 0.57). Further, both groups’ mean performances for feedback and nonfeedback blocks were well below Miller’s
(1956) upper limit of 9 stimuli, suggesting that native Chinese [tonal] language speakers do not have an advantage in identifying unidimensional tones.
Response Times Response times used to be rarely collected in absolute identification experiments, and they have not been nearly as often subjected to the detailed analysis given to response choices. Previous research has identified several effects in mean response times that are analogous to effects in choice. These include the effects already outlined: bow effects, in which responses to extreme stimuli are faster than those to middle stimuli Lacouture, 1997; Lacouture & Marley, 1995, 2004), and capacity limitations, where RTs slow down as the number of stimuli increases (Kent & Lamberts, 2005) (see Figure 6.2 for an example of these). Sequential effects on mean response times have also been observed due to response repetition (Petrov & Anderson, 2005) and assimilation (Lacouture, 1997). It could be argued that response times can be safely ignored because they fail to provide additional constraint on theories of absolute identification beyond the constraints provided by response choice data. This argument rests on the assumption that effects on response time are always just the inverse of effects on response choices (e.g., increased accuracy is always associated with decreased response time). Given this assumption, any theory that can successfully account for the effect of some manipulation on accuracy will need only invert those predictions to also account for response times. Such inversion can be accomplished by feeding the output of the model into evidence accumulators (like those used in the SAMBA model of Brown et al., 2008). Large outputs drive the accumulators more quickly, resulting in the required faster and more accurate responses. However, Donkin, Brown, Heathcote, and Marley (2009) showed that an inverse relationship between accuracy and response time is not always observed. In a re-analysis of data collected by Lacouture (1997), manipulations of the distance between stimuli sometimes had large effects on choice probability but no effects on response time. For example, in one condition, participants were presented with a stimulus set composed of five smaller lines (1–5) and five longer lines (6–10), with a much larger gap between stimuli 5 and 6 than between other adjacent stimuli. As might be expected, participants were more accurate with
stimuli that lay on the bounds of the gap between the two subsets of stimuli; they never confused stimuli 5 and 6. Despite this benefit in accuracy, however, responses to the stimuli on the edge of the center gap were just as slow as when standard equally spaced stimuli are used. This, and similar, results are important because they cannot be accounted for by any model that simply relies on a strict inverse relationship between accuracy and response time. More generally, such a dissociation between accuracy and response time runs counter to the claim that response time is not worthy of investigation in absolute identification. The result also provides a strong challenge to models of absolute identification. Donkin et al. (2009) showed that the SAMBA model provides a natural account of the dissociation between accuracy and speed due to stimulus spacing (see Figure 6.6). The credit, however, must go to the mapping model from Lacouture and Marley (1995), because the dissociation arises out of the mapping stage of SAMBA. Recall that the mapping model takes a single magnitude estimate, z, and transforms it into a set of K response strengths, Ri , i = 1.. K . This transformation is accomplished via the formula Ri = (2Yi − 1)z − Yi2 + 1, where Yi measures the average magnitude estimate for stimuli classified with response i. The transformation is parameter-free, and depends only on the relative position of each of the K stimuli in the stimulus space (as recorded by Yn ). When these Yn s represent stimuli with a large, central gap, the mapping produces response strengths that yield high accuracy for stimuli adjacent to the gap, but response times that remain slow. What made the fits of the SAMBA model to these data particularly compelling was that they were essentially parameter free. The spacing experiments from Lacouture (1997) were part of a larger experiment with other manipulations, including a control condition. In this baseline condition, a standard set of 10 equally spaced stimuli were used, and Brown et al. (2008) had already shown that SAMBA could fit these control data. Donkin et al. (2009) fixed SAMBA’s parameters to those estimated in the controlled condition, and then allowed only the representation of the stimulus spacings (the Yi s) to change across conditions, with the further constraint that those Yi s veridically represented the stimulus spacings used in the experiment. With this setup, SAMBA gave accurate quantitative predictions for the data from all the spacing conditions.
why is accurately labeling simple magnitudes so hard?
135
Spacing Condition
Accuracy
RT (sec.)
C−L
C−S
E−L
E−S
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 1.8 1.6 1.4 1.2 1.0
1 2 3 4 5 6 7 89 10 1 2 3 4 5 6 7 8 9 10 12 3 4 5 6 7 8 9 10 12 34 5 6 7 8 910 Response
Fig. 6.6 Dashes above the top panels provide a schematic representation of the stimuli used in Lacouture’s (1997) experiment. The second row shows response accuracy, and the third row shows mean RT for correct responses, both as functions of response. Data are shown as points with standard error bars that are joined by solid lines, and SAMBA fits are shown with dotted lines, and the arrows between panels show where large gaps separate adjacent stimuli. These gaps lead to different effects on response times than accuracy. Reproduced from Figure 2, Donkin, Brown, Heathcote, and Marley (2009). Donkin, C., Brown, S. D., Heathcote, A., & Marley, A. A. J. (2009). Dissociating speed and accuracy in absolute identification: The effect of unequal stimulus spacing. Psychological Research, 73, 308–316. With kind permission from Springer Science and Business Media.
Intertrial Interval and Sequential Effects Different models of absolute identification make different predictions about the influence of manipulating the time between decisions. As noted by Matthews and Stewart (2009), there have been three basic accounts for sequential effects: criterion shifts, memory confusions, and selective attention. These three explanations correspond (approximately) to the Thurstonian, relative judgment, and restricted capacity model classes outlined earlier. The Thurstonian models assume that sequential effects arise out of shifts in criteria; a fast-moving tracking process that produces assimilation, and a relatively slow-moving stabilizing mechanism that produces contrast. Both of these shifting processes decay in strength over time. If the time between trials is increased, then the criteria will have more time to return to their original positions, resulting in a reduction in both assimilation and contrast. The relative judgment models also predict a reduction in sequential effects with increased intertrial interval due to decay, but the decay is located in a different process. Sequential effects in relative-judgment models are caused by the interference from memories of the previous stimuli. Assuming that memory decays with time, then more time between stimulus presentations would reduce the confusion coming from memory. In agreement with the other classes of models, the selective-attention-based SAMBA model
136
basic cognitive skills
also predicts that assimilation will decrease with intertrial interval, but it disagrees in predicting that contrast will actually increase. The reduction in assimilation again comes about because of decay, in this case in the activity of response accumulators in the decision stage of the model. More time will allow the starting activity in accumulators to return to baseline levels, thus reducing assimilation to the previous response. The increase in contrast is a by-product of SAMBA’s existing explanation for how contrast occurs in general, the re-allocation of attention to the representation of the previous stimulus. Recall that the stimulus context is maintained via a rehearsal process in which attention is randomly allocated across the range of stimuli used in the current experiment. Contrast occurs because the location in which the previous stimulus was presented receives increased attention. So, although rehearsal activity decays across the rest of the stimulus space, the previous stimulus location is given preferential rehearsal. If this selective attention is allowed to continue for a longer time, due to an increased intertrial interval, the contrast effect will increase in magnitude. Matthews and Stewart (2009) manipulated the time between trials to be either 0 or 5 seconds in an otherwise standard absolute identification experiment. They found that relative to the small intertrial interval condition, assimilation was reduced, and contrast increased when the time
Box 1 Surprising limitations
Box 2 Sequential effects are everywhere
Perhaps the most surprising aspect of absolute identification is how difficult it is. The task itself sounds almost trivially simple: remember the labels that correspond to, say, 10 stimuli. Despite weeks of training, perfect performance remains out of reach; remarkable, when the same person might be able to understand quantum mechanics, identify thousands of species of birds, and almost certainly have a lexicon of many tens of thousands of words. The key difference between absolute identification and the many tasks that paradoxically are much easier is that the stimuli being identified vary on only a single dimension. Ten lines that vary only in their length are very difficult to discriminate, even when the spacing between the lines is large enough such that any pair of them can be told apart trivially easily. Of course, if these lines join together to produce shapes that differ on multiple dimensions, accurate identification becomes much easier. As such, the ability to store and retrieve information in long-term memory, as well as our ability to categorize our environment into such a rich structure must depend heavily on the multidimensional nature of our world. Absolute identification is an important paradigm because it represents one of the simplest and clearest cases in which the capacity limits on human information processing can be studied.
Previously encountered stimuli, and the responses they elicit, have a strong influence on the decision made on any given trial in an absolute identification task. Responses on trial N appear to be more like responses on the immediately preceding trial, and less like the stimuli presented on trials further back in the sequence. Although perhaps most comprehensively studied within the field of absolute identification, sequential effects also appear in many other decision-making tasks. For example, perceptual contrast has been observed in categorization tasks (Jones, Love, & Maddox, 2006; Zotov et al., 2011), assimilation has been observed in recognition memory tasks (although it is argued to be of a different nature than that in absolute identification, Malmberg & Annis, 2012), and the sequence of trials has a systematic influence on performance in simple two-alternative choice tasks (e.g., Gilden, 2001; Jones, Curran, Mozer, & Wilder, 2013). Absolute identification provides an important window into understanding sequential effects, as the task itself is so simple. There exist a number of potential models for how previous stimuli and responses influence behavior within the realm of absolute identification. In contrast, most models of categorization, recognition memory, and even simple decision-making fail to account for sequential effects, despite their well-documented existence.
between trials was longer. This pattern of results appears to align with the predictions from SAMBA and to contradict those from Thurstonian and relative judgment models. However, a more sophisticated analysis of the data that used regression equations to separate out the influence of previous stimuli and previous responses revealed a more challenging pattern of results for theories of absolute identification. Responses from participants were fit with a regression equation in which both stimuli and responses were included as predictors: RN = r0 + α0 SN + α1 SN −1 + β1 RN −1 + α2 SN −2 + β2 RN −2 + eN , where RN is the response made on the N th trial, SN is the stimulus presented on the N th trial, α and β are regression coefficients, and eN is a normally distributed error term. Based
on this equation, Matthews and Stewart (2009) found that assimilation to responses does not decrease as intertrial interval increases (i.e., that β1 remains positive and of approximately the same magnitude in both short and long intertrial interval conditions), but rather that contrast to the previous stimuli increases as intertrial interval increases (i.e., both α1 and α2 become more negative). The observed increase in contrast with intertrial interval is predicted by the selective attention explanation for contrast, but is inconsistent with the Thurstonian and relative judgement accounts for sequential effects. Recall, however, that SAMBA predicted a decrease in assimilation to the previous response with inter-trial interval. Though the raw data appear to agree with SAMBA, the regression coefficients reported by Matthews and Stewart (2009) suggest that the observed lack of an assimilation effect in the long intertrial interval condition
why is accurately labeling simple magnitudes so hard?
137
is due to an increase in contrast to the previous stimulus, rather than to a reduction in assimilation to the previous response. This suggests that either the mechanism by which SAMBA produces assimilation is inappropriate and needs adjustment or that the decay process is extremely slow.
Conclusion Absolute identification is an apparently simple task: take a set of lines that vary only in length, or tones that vary only in loudness; label them in a straightforward way; and ask an observer to recall the correct labels, when shown stimuli one at a time. Despite this apparent simplicity, there is a wealth of complicated and robust patterns that have been identified in absolute identification data over the past 60 years. Early theoretical accounts of absolute identification were inspired by some of the most fundamental data patterns; for example, the relative judgment models were inspired by the ubiquitous sequential effects, and the limited capacity models by the observed performance limits. Since they were inspired by just one of the many fundamental phenomena, these theories all had problems in accounting for the full range of data. More recent theories are more complex, and share structure with more than one type of earlier theory. These new theories aim to provide a comprehensive account of the full range of absolute identification phenomena (Petrov & Anderson, 2005; Stewart et al., 2005; Brown et al., 2008). This leaves some clear challenges for future empirical and theoretical research in absolute identification: • What exactly constitutes a “purely relative” and “purely absolute” theoretical account? Can the current models be modified to perfectly fit either category? If so, can a pure model explain the full range of data? • What mechanism underpins the newly observed learning effects in absolute identification? Are tones of varying loudness the only stimulus set that is impossible to learn? • To what extent are the sequential effects in absolute identification malleable? They are apparently altered substantially by practice, and by the timing of the experimental procedure. These findings may provide new constraints on existing theories.
Further into the future, a challenge for absolute identification research will be to build links with 138
basic cognitive skills
other paradigms. Some paradigms clearly are closely related to absolute identification, such as magnitude estimation, magnitude production, and categorization. It should be relatively simple for empirical work to explore the links between these fields, but a more difficult challenge will be to integrate and unify theoretical accounts in the different fields, where possible.
Note 1. Although the terms loudness and intensity, and the terms pitch and frequency, have different meanings, we use them almost interchangeably for reasons of clarity.
Glossary Absolute Models: These theories assume that identification decisions rely entirely on long-lasting, internal representations. Simple forms of exemplar models, Thurstonian models, and restricted capacity models fall into this category of model. ACT-R: A production-system based cognitive architecture explanation of human cognitive and perceptual systems that has been mapped to brain areas. Assimilation: Responses are more likely to be closer to the response (and stimulus) made on the previous trial. Bow Effect: Improved performance for stimuli toward the edge of the stimulus range, so-called because performance plotted as a function of stimulus position has a bow-shaped curve. Capacity: Processing resources used to identify stimuli. Performance in absolute identification tasks suggest that humans are severely limited in their capacity when identifying unidimensional stimuli. Contrast: Responses are more likely to be further away from the stimulus (and response) from 2–5 trials previous. d : A signal-detection-theory based measure of how well two stimuli can be discriminated. Stimulus Dimensionality: The number of physical continua on which a set of stimuli vary. Absolute identification research focuses on the classification of stimuli varying on just one physical dimension that is assumed to be mapped onto a single psychological dimension. Exemplar Model: Theory assuming that stimuli are represented and stored as individual memory traces (i.e., exemplars). The similarity between the stimulus presented on a given trial and those stored in memory is critical for decision-making. Evidence Accumulators: Hypothetical processing units used to account for choice and response time in decisionmaking tasks. Each potential response is given its own accumulator, wherein evidence is collected and accrues. The first accumulator to reach a threshold level of evidence gives the chosen response, and the time taken to reach threshold gives decision time.
Glossary Modality: The perceptual system to which a stimulus is presented. For example, tones varying in frequency are from the auditory modality. Relative Models: These theories assume that the identification of unidimensional stimuli is based on the judgement of the difference between the current stimulus and temporary representations of recently presented stimuli (and/or the corresponding responses). Sequential Effects: The influence of previous stimuli and responses on behavior on the current trial.
References Anderson, J. R. (1990). The adaptive character of thought. Hillsdale, NJ: Erlbaum. Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Mahwah, NJ: Erlbaum. Athos, E., Levinson, B., Kistler, A., Zemansky, J., Bostrom, A., Frimer, N., & Gitschier, J. (2007). Dichotomy and perceptual distortions in absolute pitch ability. Proceedings of the National Academy of Sciences, 104(37), 14795–14800. Bachem, A. (1955). Absolute pitch. Journal of the Acoustical Society of America, 27 , 1180-1185. Baharloo, S., Johnston, P., Service, S., Gitschier, J., & Freimer, N. (1998). Absolute pitch: An approach for identification of genetic and nongentic components. American Journal of Human Genetics, 62, 224–231. Bent, T., Bradlow, A., & Wright, B. (2006). The influence of linguistic experience on the cognitive processing of pitch in speech and non-speech sounds. Journal of Experimental Psychology: Human Perception and Performance, 32, 97–103. Braida, L. D., & Durlach, N. I. (1972). Intensity perception: Ii. resolution in one-interval paradigms. Journal of Acoustical Society of America, 51, 483–502. Braida, L. D., Lim, J. S., Berliner, J. E., Durlach, N. I., Rabinowitz, W. M., & Purks, S. R. (1984). Intensity perception: Xiii. perceptual anchor model of context-coding. Journal of the Acoustical Society of America, 76 , 722–731. Brammer, L. (1951). Sensory cues in pitch judgment. Journal of Experimental Psychology, 41(5), 336–340. Brown, S. D., & Heathcote, A. (2005). A ballistic model of choice response time. Psychological Review, 112, 117–128. Brown, S. D., Marley, A., Dodds, P., & Heathcote, A. (2009). Purely relative models cannot provide a general account of absolute identification. Psychonomic Bulletin & Review, 16 , 583–593. Brown, S. D., Marley, A., Donkin, C., & Heathcote, A. J. (2008). An integrated model of choices and response times in absolute identification. Psychological Review, 115(2), 396–425. De Carlo, L. (1992). Intertrial interval and sequential effects in magnitude scaling. Journal of Experimental Psychology: Human Perception and Performance, 18, 1080–1088. De Carlo, L., & Cross, D. (1990). Sequential effects in magnitude scaling: Models and theory. Journal of Experimental Psychology: General, 119, 375–396.
Deutsch, D., Henthorn, T., Marvin, E., & Xu, H. (2005). Absolute pitch among american and chinese conservatory students: Prevalence differences, and evidence for a speechrelated critical period. Journal of the Acoustical Society of America, 119(2), 719–722. Deutsch, D., Le, J., Dooley, K., Henthorn, T., Shen, J., & Head, B. (2009). Absolute pitch and tone language: Two new studies. Proceedings of the 7th Triennial Conference of the European Society for the Cognitive Sciences of Music. Finland, 2009, 69–73. Dodds, P., Donkin, C., Brown, S. D., & Heathcote, A. (2010). Multidimensional scaling methods for absolute identification data. In S. Ohlsson & R. Catrambone (Eds.), Proceedings of the 32nd annual conference of the cognitive science society. Portland, OR: Cognitive Science Society. Dodds, P., Donkin, C., Brown, S. D., & Heathcote, A. (2011). Increasing capacity: Practice effects in absolute identification. Journal of Experimental Psychology: Learning, Memory & Cognition, 37 , 477–492. Dodds, P., Rae, B., & Brown, S. D. (2012). Perhaps unidimensional is not unidimensional. Cognitive Science, 36(8), 1542–1555. Donkin, C., Brown, S. D., Heathcote, A., & Marley, A. A. J. (2009). Dissociating speed and accuracy in absolute identification: The effect of unequal stimulus spacing. Psychological Research, 73, 308–316. Donkin, C., Chan, V., & Tran, S. (2014). The effect of blocking inter-trial interval on sequential effects in absolute identification. Quarterly Journal of Experimental Psychology. Durlach, N. I., & Braida, L. D. (1969). Intensity perception. i. preliminary theory of intensity resolution. Journal of the Acoustical Society of America, 46 , 372–383. Ellis, A. J. (1876). On the sensitiveness of the ear to pitch and change of pitch in music. Journal of the Royal Musical Association, 3, 1–32. Garner, W. R. (1953). An informational analysis of absolute judgments of loudness. Journal of Experimental Psychology, 46 , 373–380. Gilden, D. L. (2001). Cognitive emissions of 1/f noise. Psychological Review, 108, 33–56. Gilden, D. L., Thornton, T., & Mallon, M. W. (1995). 1/f noise in human cognition. Science, 267 , 1837–1839. Hartman, E. (1954). The influence of practice and pitchdistance between tones on the absolute identification of pitch. American Journal of Psychology, 67 , 1–14. Holland, M. K., & Lockhead, G. R. (1968). Sequential effects in absolute judgments of loudness. Perception & Psychophysics, 3, 409–414. Jones, M., Curran, T., Mozer, M. C., & Wilder, M. H. (2013). Sequential effects in response time reveal learning mechanisms and event representations. Psychological Review, 120, 628–666. Jones, M., Love, B. C., & Maddox, W. T. (2006). Recency effects as a window to generalization: Separating decisional and perceptual sequential effects in category learning. Journal of Experimental Psychology: Learning, Memory & Cognition, 32, 316–332. Kent, C., & Lamberts, L. (2005). An exemplar account of the bow and set size effects in absolute identification. Journal of
why is accurately labeling simple magnitudes so hard?
139
Experimental Psychology: Learning, Memory, and Cognition, 31, 289–305. Lacouture, Y. (1997). Bow, range, and sequential effects in absolute identification: A response-time analysis. Psychological Research, 60, 121–133. Lacouture, Y., & Marley, A. (1995). A mapping model of bow effects in absolute identification. Journal of Mathematical Psychology, 39, 383–395. Lacouture, Y., & Marley, A. A. J. (2004). Choice and response time processes in the identification and categorization of unidimensional stimuli. Perception & Psychophysics, 66 , 1206–1226. Lamberts, K. (2000). Information-accumulation theory of speeded classification. Psychological Review, 107 , 226–260. Laming, D. (1984). The relativity of "absolute" judgements. British Journal of Mathematical and Statistical Psychology, 37 , 152–183. Levitin, D., & Rogers, S. (2005). Absolute pitch: Perception, coding, and controversies. TRENDS in Cognitive Sciences, 9, 26–33. Lockhead, G. R., & Byrd, R. (1981). Practically perfect pitch. Journal of the Acoustical Society of America, 70(2), 387–389. Lockhead, G. R., & Hinson, J. M. (1986). Range and sequence effects in judgment. Perception & Psychophysics, 40, 53–61. Luce, R. D., Green, D. M., & Weber, D. L. (1976). Attention bands in absolute identification. Perception & Psychophysics, 20, 49–54. Luce, R. D., Nosofsky, R. M., Green, D. M., & Smith, A. F. (1982). The bow and sequential effects in absolute identification. Perception & Psychophysics, 32, 397–408. Lundin, R. (1963). Can perfect pitch be learned? Music Educators’ Journal, 49(5), 49–51. Malmberg, K., & Annis, J. (2012). On the relationship between memory and perception: sequential dependencies in recognition memory testing. Journal of Experimental Psychology: General, 141(2), 233–259. Marley, A. A. J., & Cook, V. T. (1984). A fixed rehearsal capacity interpretation of limits on absolute identification performance. British Journal of Mathematical and Statistical Psychology, 37 , 136–151. Marley, A. A. J., & Cook, V. T. (1986). A limited capacity rehearsal model for psychological judgments applied to magnitude estimation. Journal of Mathematical Psychology, 30, 339–390. Matthews, W., & Stewart, N. (2009). The effect of inter-stimulus intervla on sequential effects in absolute identification. Quarterly Journal of Experimental Psychology, 62, 2014–2029. Miller, G. (1956). The magical number seven, plus or minus two: Some limits on our capacity for information processing. Psychological Review, 63, 81–97. Miyazaki, K. (1995). Perception of relative pitch with different references: Some absolute-pitch listeners can’t tell musical interval names. Perception & Psychophysics, 57 (7), 962–970. Miyazaki, K. (2004). How well do we understand absolute pitch? Acoustical Science and Technology, 25(6), 426–432. Nosofsky, R. M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39–57.
140
basic cognitive skills
Nosofsky, R. M. (1997). An exemplar-based random-walk model of speeded categorization and absolute judgment. In A. A. J. Marley (Ed.), Choice, decision and measurement (pp. 347–365). Hillsdale, NJ: Erlbaum. Nosofsky, R. M., & Palmeri, T. J. (1997). An exemplar–based random walk model of speeded classification. Psychological Review, 104, 266–300. Petrov, A., & Anderson, J. R. (2005). The dynamics of scaling: A memory-based anchor model of category rating and absolute identification. Psychological Review, 112(383–416). Pfordresher, P., & Brown, S. (2009). Enhanced production and perception of musical pitch in tone language speakers. Attention, Perception, & Psychophysics, 71, 1385-1398. Pollack, I. (1952). The information of elementary auditory displays. Journal of the Acoustical Society of America, 24, 745–749. Pollack, I. (1953). The information of elementary auditory displays: Ii. Journal of the Acoustical Society of America, 25, 765–769. Profita, J., & Bidder, T. (1988). Perfect pitch. American Journal of Medical Genetics, 29, 763-771. Revesz, G. (1953). Introduction to the psychology of music. London: Longmans Green. Rouder, J. N. (2001). Absolute identification with simple and complex stimuli. Psychological Science, 12, 318–322. Rouder, J. N., Morey, R. D., Cowan, N., & Pfaltz, M. (2004). Learning in a unidimensional absolute identification task. Psychonomic Bulletin & Review, 11, 938–944. Schellenberg, E., & Trehub, S. (2007). s there an asian advantage for pitch memory? Music Perception, 25, 241–252. Shiffrin, R. M., & Nosofsky, R. M. (1994). Seven plus or minus two: A commentary on capacity limitations. Psychological Review, 101, 357–361. Stewart, N., Brown, G. D. A., & Chater, N. (2005). Absolute identification by relative judgment. Psychological Review, 112, 881–911. Stewart, N., & Matthews, W. (2009). Relative judgment and knowledge of the category structure. Psychonomic Bulletin & Review, 16 , 594–599. Takeuchi, A., & Hulse, S. (1993). Absoute pitch. Psychological Bulletin, 113(2), 345–361. Terhardt, E., & Seewann, M. (1983). Aural key identification and its relationship to absolute pitch. Music Perception, 1, 63–83. Treisman, M. (1985). The magical number seven and some other features of category scaling: Properties for a model of absolute judgment. Journal of Mathematical Psychology, 29, 175–230. Treisman, M., & Williams, T. C. (1984). A theory of criterion setting with an application to sequential dependencies. Psychological Review, 91, 68–111. Usher, M., & McClelland, J. L. (2001). On the time course of perceptual choice: The leaky competing accumulator model. Psychological Review, 108, 550–592. Van Orden, G. C., Holden, J. G., & Turvey, M. T. (2003). Self–organization of cognitive performance. Journal of Experimental Psychology: General, 132, 331–350. Wagenmakers, E.-J., Farrell, S., & Ratcliff, R. (2004). Estimation and interpretation of 1/f α noise in human cognition. Psychonomic Bulletin & Review, 11, 579–615.
Wagenmakers, E.-J., Farrell, S., & Ratcliff, R. (2005). Human Cognition and a Pile of Sand: A Discussion on Serial Correlations and Self–organized Criticality Human cognition and a pile of sand: A discussion on serial correlations and self–organized criticality. Journal of Experimental Psychology: General, 134, 108–116. Ward, L. M., & Lockhead, G. R. (1970). Sequential effect and memory in category judgment. Journal of Experimental Psychology, 84, 27–34. Ward, L. M., & Lockhead, G. R. (1971). Response system processes in absolute judgment. Perception & Psychophysics, 9, 73–78. Weber, D. L., Green, D. M., & Luce, R. D. (1977). Effect of practice and distribution of auditory signals on
absolute identification. Perception & Psychophysics, 22, 223–231. Zatorre, R. (2003). Absolute pitch: A model for understanding the influence of genes and development on neural and cognitive function. Nature Neuroscience, 6 (7), 692–695. Zatorre, R., & Beckett, C. (1989). Multiple coding strategies in the retention of musical tones by possessors of absolute pitch. Memory & Cognition, 17 (5), 582–589. Zotov, V., Jones, M., & Mewhort, D. J. K. (2011). Contrast and assimilation in categorization and exemplar production. Attention, Perception, & Psychophysics, 73, 621–639.
why is accurately labeling simple magnitudes so hard?
141
CHAPTER
7
An Exemplar-Based Random-Walk Model of Categorization and Recognition
Robert M. Nosofsky and Thomas J. Palmeri
Abstract
In this chapter, we provide a review of a process-oriented mathematical model of categorization known as the exemplar-based random-walk (EBRW) model (Nosofsky & Palmeri, 1997a). The EBRW model is a member of the class of exemplar models. According to such models, people represent categories by storing individual exemplars of the categories in memory, and classify objects on the basis of their similarity to the stored exemplars. The EBRW model combines ideas ranging from the fields of choice and similarity, to the development of automaticity, to response-time models of evidence accumulation and decision-making. This integrated model explains relations between categorization and other fundamental cognitive processes, including individual-object identification, the development of expertise in tasks of skilled performance, and old-new recognition memory. Furthermore, it provides an account of how categorization and recognition decision-making unfold through time. We also provide comparisons with some other process models of categorization. Key Words: categorization, recognition, exemplar model, response times, automaticity,
random walk, memory search, expertise, similarity, practice effects
Introduction A fundamental issue in cognitive psychology and cognitive science concerns the manner in which people represent categories and make classification decisions (Estes, 1994; Smith & Medin, 1981). There is a wide variety of process-oriented mathematical models of categorization that have been proposed in the field. For example, according to prototype models (e.g., Posner & Keele, 1968; Smith & Minda, 1998), people represent categories by storing a summary representation, usually presumed to be the central tendency of the category distribution. Classification decisions are based on the similarity of a test item to the prototypes of alternative categories. According to decision-boundary models (e.g., Ashby & Maddox, 1993; McKinley & Nosofsky, 1995), people construct boundaries, usually assumed to be linear or quadratic in form, to divide a stimulus space 142
into category response regions. If an object is perceived to lie in Region A of the space, then the observer emits a Category-A response. According to rule-plus-exception models (e.g., Davis, Love, & Preston, 2012; Erickson & Kruschke, 1998; Nosofsky, Palmeri, & McKinley, 1994), people construct low-dimensional logical rules for summarizing categories, and they remember occasional exceptions that may be needed to patch those rules. In this chapter, however, our central focus is on exemplar models of classification. According to exemplar models, people represent categories by storing individual exemplars in memory, and classify objects on the basis of their similarity to the stored exemplars (Hintzman, 1986; Medin & Schaffer, 1978; Nosofsky, 1986). For instance, such models would assume that people represent the category of “birds” by storing in memory the vast collection of different robins, sparrows, eagles (and
so forth) that they have experienced during their lifetimes. If a novel object were similar to some of these bird exemplars, then a person would tend to classify it as a bird. Although alternative classification strategies are likely to operate across different experimental contexts, there are several reasons why we chose to focus on exemplar models in this chapter. One reason is that models from that class have provided a successful route to explaining relations between categorization and a wide variety of other fundamental cognitive processes, including individual-object identification (Nosofsky, 1986, 1987), the development of automaticity in tasks of skilled performance (Logan, 1988; Palmeri, 1997), and old-new recognition memory (Estes, 1994; Hintzman, 1988; Nosofsky, 1988, 1991). A second reason is that, in our view, for most “natural” category structures (Rosch, 1978) that cannot be described in terms of simple logical rules, exemplar models seem to provide the best-developed account for explaining how categorization decision-making unfolds over time. Thus, beyond predicting classification choice probabilities, exemplar models provide detailed quantitative accounts of classification response times (Nosofsky & Palmeri, 1997a). We now briefly expand these themes before turning to the main body of our chapter. One of the central goals of exemplar models has been to explain relations between categorization and other fundamental cognitive processes, including old-new recognition memory (Estes, 1994; Hintzman, 1988; Nosofsky, 1988, 1991; Nosofsky & Zaki, 1998). Whereas in categorization people organize distinct objects into groups, in recognition the goal is to determine if some individual object is “old” (previously studied) or “new.” Presumably, when people make recognition judgments, they evaluate the similarity of test objects to the individual previously studied items (i.e., exemplars). If categorization decisions are also based on similarity comparisons to previously stored exemplars, then there should be close relations between the processes of recognition and categorization. A well-known model that formalizes these ideas is the generalized context model (GCM; Nosofsky, 1984, 1986, 1991). In the GCM, individual exemplars are represented as points in a multidimensional psychological space, and similarity between exemplars is a decreasing function of the distance between objects in the space (Shepard, 1987). The model presumes that both classification and recognition decisions are based on
the “summed similarity” of a test object to the exemplars in the space. By conducting similarityscaling studies, one can derive multidimensional scaling (MDS) solutions in which the locations of the exemplars in the similarity space are precisely located (Nosofsky, 1992). By using the GCM in combination with these MDS solutions, one can then achieve successful fine-grained predictions of classification and recognition choice probabilities for individual items (Nosofsky, 1986, 1987, 1991; Shin & Nosofsky, 1992). A significant development in the application of the GCM has involved extensions of the model to explaining how the categorization process unfolds through time. So, for example, the exemplar model not only predicts choice probabilities, but also predicts categorization and recognition response times (RTs). This development is important because RT data often provide insights into cognitive processes that would not be evident based on examination of choice-probability data alone. Nosofsky and Palmeri’s (1997a,b) exemplar-based randomwalk (EBRW) model adopts the same fundamental representational assumptions as does the GCM. However, it extends that earlier model by assuming that retrieved exemplars drive a random walk process (e.g., Busemeyer, 1985; Link, 1992; Ratcliff, 1978). This exemplar-based random-walk model allows one to predict the time course of categorization and recognition decision making. In this chapter, we provide a review of this EBRW model and illustrate its applications to varieties of categorization and recognition choiceprobability and RT data. In the section on The Formal EBRW Model of Categorization RTs we provide a statement of the formal model as applied to categorization. As will be seen, the EBRW model combines ideas ranging from the fields of choice and similarity, to the development of automaticity, to RT models of evidence accumulation and decision making. In the section Effects of Similarity and Practice on Speeded Classification, we describe applications of the model to speeded perceptual classification, and illustrate how it captures fundamental phenomena including effects of similarity and practice. In the section Automaticity and Perceptual Expertise we describe how the EBRW accounts for the development of automatic categorization and perceptual expertise. In the section Using Probabilistic Feedback Manipulations to Contrast the Predictions From the EBRW Model and Alternative Models we describe experimental manipulations that have been used to try to
an exemplar-based random-walk model
143
“Category A” +A Category B perceptual processing
stimulus i
i
Category A dij j
–B
psychological space
random walk
(a)
(b)
time Fig. 7.1 Schematic illustration of the ideas behind the exemplar-based random-walk model. Panel a: Exemplars are represented as points in a multidimensional psychological space. Similarity is a decreasing function of the distance between objects in the space. Presentation of a test probe causes the exemplars to be “activated” in accord with how similar they are to that probe. The activated exemplars “race” to be retrieved. Panel b: The retrieved exemplars drive a random-walk process for making categorization decisions. Each time that an exemplar from Category A is retrieved, the random walk takes unit step towards the Category-A response threshold, and likewise for Category B. The retrieval process continues until one of the response thresholds is reached.
distinguish between the predictions of the EBRW model and some other major models of speeded categorization, including decision-boundary and prototype models. Finally, in the section Extending the EBRW Model to Predict Old-New Recognition RTs, we describe recent developments in which the EBRW model has been used to account for oldnew recognition RTs. We then provide conclusions and questions for future research in the section Conclusions and New Research Goals.
The Formal EBRW Model of Categorization RTs The EBRW model and the GCM build upon classic theories in the areas of choice and similarity (Shepard, 1957, 1987). As described in the introduction, in the model, exemplars are represented as points in a multidimensional psychological space (for an illustration, see Figure 7.1a). The distance between exemplars i and j (dij ) is given by the Minkowski power model, d ij =
K
ρ wk x ik − x jk
ρ1 ,
(1)
k=1
where xik is the value of exemplar i on psychological dimension k; K is the number of dimensions that define the space; ρ defines the distance metric of the space; and wk (0 < wk , wk = 1) is the weight given to dimension k in computing distance. In situations involving the classification of holistic 144
basic cognitive skills
or integral-dimension stimuli (Garner, 1974), which will be the main focus of the present chapter, ρ is set equal to 2, which yields the familiar Euclidean distance metric. The dimension weights wk are free parameters that reflect the degree of “attention” that subjects give to each dimension in making their classification judgments (Carroll & Wish, 1974). In situations in which some dimensions are far more relevant than others in allowing subjects to discriminate between members of contrasting categories, these attention-weight parameters may play a significant role (e.g., Nosofsky, 1986, 1987). In the experimental situations considered in this chapter, however, all dimensions tend to be relevant and the attention weights will turn out to play a relatively minor role. The similarity of test item i to exemplar j is an exponentially decreasing function of their psychological distance (Shepard, 1987), sij = exp( − c · d ij ),
(2)
where c is an overall sensitivity parameter that measures overall discriminability in the psychological space. The sensitivity governs the rate at which similarity declines with distance. When sensitivity is high, the similarity gradient is steep, so even objects that are close together in the space may be highly discriminable. By contrast, when sensitivity is low, the similarity gradient is shallow, and objects are hard to discriminate.
Each exemplar j is stored in memory (along with its associated category feedback) with memorystrength mj . As will be seen, the precise assumptions involving the memory strengths vary with the specific experimental paradigm that is tested. For example, in some paradigms, we attempt to model performance on a trial-by-trial basis, and assume that the memory strengths of individual exemplars decrease systematically with their lag of presentation. In other paradigms, we attempt to model performance in a transfer phase that follows a period of extended training. In that situation, we typically assume that the memory strengths are proportional to the frequency with which each individual exemplar was presented in common with given category feedback during the initial training phase. When a test item is presented, it causes all exemplars to be “activated” (Figure 7.1a). The activation for exemplar j, given presentation of item i, is given by (3) aij = mj sij . Thus, the exemplars that are most highly activated are those that have the greatest memory strengths and are highly similar to the test item. In modeling early-learning behavior, we also presume that “background” elements exist in memory at the start of training that are not associated with any of the categories. These background elements are presumed to have fixed activation b, independent of the test item that is presented. Borrowing from Logan’s (1988) highly influential instance theory of automaticity, the EBRW assumes that presentation of a test item causes the activated stored exemplars and background elements to “race” to be retrieved. For mathematical simplicity, the race times are presumed to be independent exponentially distributed random variables with rates proportional to the degree to which exemplar j is activated by item i (Bundesen, 1990; Logan, 1997; Marley, 1992). Thus, the probability density that exemplar j completes its race at time t, given presentation of item i, is given by f (t) = aij · exp(− aij · t).
(4)
This assumption formalizes the idea that although the retrieval process is stochastic, the exemplars that tend to race most quickly are those that are most highly activated by the test item. The exemplar (or background element) that “wins” the race is retrieved. Whereas in Logan’s (1988) model the response is based on only the first retrieved exemplar, in the EBRW model exemplars from multiple retrievals
drive a random-walk evidence accumulation process (e.g., Busemeyer, 1982; Ratcliff, 1978). In a twocategory situation, the process operates as follows (for an illustration, see Figure 7.1b): First, there is a random-walk counter with initial setting zero. The observer establishes response thresholds representing the amount of evidence needed to make either a Category-A response (+A) or a Category-B response (–B). Suppose that exemplar x wins the race on a given step. If x received Category-A feedback during training, then the random-walk counter is increased by unit value in the direction of +A, whereas if x received Category-B feedback, the counter is decreased by unit value in the direction of –B. (If a background element wins the race, the counter’s direction of change is chosen randomly.) If the counter reaches either threshold +A or –B, the corresponding categorization response is made. Otherwise, the races continue, another exemplar is retrieved (possibly the same one as on the previous step), and the random walk takes its next step. Given the processing assumptions outlined earlier, Nosofsky and Palmeri (1997a) showed that, on each step of the random walk, the probability (pi ) that the counter is increased in the direction of threshold +A is given by pi = (S iA + b)/(S iA + S iB + 2b),
(5)
where SiA denotes the summed activation of all currently stored Category-A exemplars given presentation of item i (and likewise for SiB ). (The probability that the counter is decreased in the direction of Category B is given by qi = 1-pi .) Thus, as the summed activation of Category-A exemplars increases, the probability of retrieving Category-A exemplars, and thereby the probability of moving the counter in the direction of +A, increases. The categorization decision time is determined jointly by the total number of steps required to complete the random walk and by the speed with which those individual steps are made. Given these random-walk processing assumptions, it is straightforward to derive analytic predictions of classification choice probabilities and mean RTs for each stimulus at any given stage of the learning process (Busemeyer, 1982). The relevant equations are summarized by Nosofsky and Palmeri (1997a, pp. 269–270, 291–292). Here, we focus on some key conceptual predictions that follow from the model. A first prediction is that rapid classification decisions should be made for items that are highly similar to exemplars from their own target
an exemplar-based random-walk model
145
P(A|i) = (S iA +b)γ /[(S iA +b)γ +[(S iB +b)γ ]. (6) 146
basic cognitive skills
This equation is the descriptive equation of choice probability that is provided by the GCM, one of the most successful formal models of perceptual classification (for discussion, see, e.g., Nosofsky, 1986; Wills & Pothos, 2012).1 Thus, besides providing a formal account of classification RTs, the EBRW model provides a processing interpretation for the emergence of the GCM response rule.
Effects of Similarity and Practice on Speeded Classification In their initial tests of the EBRW model, Nosofsky and Palmeri (1997a) conducted a speeded classification task using the category structure shown in Figure 7.2. The stimuli were a set of 12 Munsell colors of a constant red hue varying in their brightness and saturation2 . As illustrated in the figure, half the exemplars were assigned by the experimenter to Category A (squares) and half to Category B (circles). On each trial, a single color was presented, the observer classified it into one of the two categories, and corrective feedback was then provided. Testing was organized into 150 blocks of 12 trials (30 blocks per day), with each color presented once in a random order in each block. (See Nosofsky and Palmeri, 1997a, pp. 273–274 for further methodological details.) The mean RTs observed for one of the participants are shown in Figure 7.3. The top panel illustrates the mean RT for each individual color, averaged across the final 120 blocks. The diameter of the circle enclosing each stimulus is linearly related to the stimulus’s mean RT. To help interpret these data, the figure also displays a dotted boundary of equal summed-similarity to the exemplars
7/
1
5/
5
4
6 8
4/ 3/
2 3
6/ Brightness
category and that are dissimilar to exemplars from the contrast category. Under such conditions, all retrieved exemplars will tend to come from the target category, so the random walk will march consistently toward that category’s response threshold. For example, if an item is highly similar to the exemplars from Category A, and dissimilar to the exemplars from Category B, then the value pi in Eq. 5 will be large, so almost all steps in the random walk will move in the direction of +A. By contrast, if an item is similar to the exemplars of both Categories A and B, then exemplars from both categories will be retrieved; in that case, the random walk will meander back and forth, leading to a slow RT. A second prediction is that practice in the classification task will lead to more accurate responding and faster RTs. Early in training, before any exemplars have been stored in memory, the background-element activations are large relative to the summed-activations of the stored exemplars (see Eq. 5). Thus, the random-walk step probabilities (pi ) hover around .5, so responding is slow and prone to error. As the observer accumulates more category exemplars in memory, the summed activations SiA and SiB grow in magnitude, so responding is governed more by the stored exemplars. A second reason that responding speeds up is that the greater the number of exemplars stored in memory, the faster the “winning” retrieval times tend to be (cf. Logan, 1988). The intuition is that the greater the number of exemplars that participate in the race, the higher is the probability that some particular exemplar will finish quickly. Therefore, as more exemplars are stored, the speed of the individual steps in the random walk increases. As discussed later in this chapter, many more fine-grained predictions follow from the model, some of which are highly diagnostic for distinguishing between the predictions of the EBRW model and alternative models. We describe these predictions in the context of the specific experimental paradigms in which they are tested. One other important formal aspect of the model involves its predictions of classification choice probabilities. In particular, in the special case in which the response thresholds +A and −B are set an equal magnitude γ from the starting point of the random walk, the model predicts that the probability that item i is classified in Category A is given by
10 /4
9 11
/6
7
12
/10 /8 Saturation
/12
Fig. 7.2 Schematic illustration of the category structure tested by Nosofsky and Palmeri (1997a, Experiment 1).
(A) 2.0
1
1.5
2 1.0
4
3
7
Brightness
0.5
6 0.0
5 –0.5
9
8 12
–1.0
11
–1.5
10 –2.0 –2.0 –1.5 –1.0 –0.5 0.0 0.5 Saturation
0.1
1.5
2.0
(B) 1900 observed predicted
1800 1700
Mean Response Time
1600 1500 1400 1300 1200 1100 1000 900 800 0
15
30
45
60
75 90 Block (N)
105
120
135
150
Fig. 7.3 Data from Experiment 1 of Nosofsky and Palmeri (1997a). (Top) Mean RTs for individual colors (RT proportional to the size of the circle). (Bottom) Mean RTs averaged across all colors as a function of grouped blocks of practice.
of Category A and Category B. Points falling to the upper right have greater summed similarity to Category A, and points falling to the lower left have greater summed similarity to Category B. As can be seen, the mean RTs tend to get faster as one moves farther away from the boundary of equal summed similarity. The bottom panel of Figure 7.3 provides another perspective on the data. This panel plots the overall mean RT for all 12 colors for each “grouped-block” of practice in the task, where a grouped-block corresponds to five consecutive individual blocks. It is evident that there is a speed-up with practice, with the lion’s share of the speed-up occurring during the early blocks.
To apply the EBRW model to these data, we first derived a multidimensional scaling (MDS) solution for the colors by having the participant provide extensive similarity ratings between all pairs of the colors. A two-dimensional scaling solution yielded a good account of the similarity ratings. The solution is depicted in the top panel of Figure 7.3, where the center of each circle gives the MDS coordinates of the color. We then used the EBRW model in combination with the MDS solution to simultaneously fit the mean RTs associated with the individual colors (Figure 7.3, top panel) as well as the aggregated mean RTs observed as a function of grouped-blocks of practice (Figure 7.3, bottom panel). Specifically, the MDS solution provides the coordinate values xik that enter into the EBRW model’s distance function (Equation 1). For simplicity, in fitting the model, we assumed that on each individual block of practice, an additional token of each individual exemplar was stored in memory with strength equal to one. (To reduce the number of free parameters, we set the background-element activation b equal to zero.) The first step in the analysis was to use the model to predict the mean RT for each individual color in each individual block. Then, for each individual color, we averaged the predictions across Blocks 31–150 to predict the overall individual-color mean RTs. Likewise, we averaged the predicted mean RTs over all colors in each grouped-block of practice to predict the speed-up curves. We fitted the model by searching for the free parameters that minimized the total sum of squared deviations between predicted and observed mean RTs across both data sets. These free parameters included the overall sensitivity parameter c; an attention-weight parameter w1 (with w2 = 1 – w1 ); and a response-threshold parameter +A (with +A = |–B|). In addition, we estimated a mean residualtime parameter μR ; a scaling constant v for transforming the number of steps in the random walk into ms; and an individual step-time constant α (see Nosofsky & Palmeri, 1997a, pp. 268–270 for details). The model-fitting results for the individualcolor RTs are displayed in Figure 7.4, which plots observed against predicted mean RTs for each individual color. The model provides a reasonably good fit (r = 0.89), especially considering that it is constrained to simultaneously fit the speed-up curves. The model predicts these results because colors far from the exemplar-based boundary (e.g., 2, 4, 7, and 10—see top panel of Figure 7.3) tend to be similar only to exemplars from their own category.
an exemplar-based random-walk model
147
1400 9
Observed Mean Response Time
1300
12
1200
3
1100
6
1000 900
11
800 700
2
5 8 1
104 7
600 600 700 800 900 1000 1100 1200 1300 1400 Predicted Mean Response Time Fig. 7.4 Scatterplot of observed individual-color mean RTs against the predictions from the EBRW model.
Thus, only exemplars from a single category are retrieved, and the random walk marches in rapid fashion to the appropriate response threshold. By contrast, colors close to the boundary (e.g., 3, 9, and 12) are similar both to exemplars from their own category and to the contrast category. Thus, exemplars from both categories are retrieved and the random walk meanders back and forth, leading to slow mean RTs. The fits to the speed-up curves are shown along with the observed data in the bottom panel of Figure 7.3. Again, the model captures these data reasonably well (r = 0.931). It predicts the speed-up for essentially the same reason as in Logan’s (1988) model: As practice continues, an increased number of exemplars are stored in memory. The greater the number of exemplars that race to be retrieved, the faster are the winning retrieval times, so the individual steps in the random walk take place more quickly.3
Automaticity and Perceptual Expertise The EBRW provides a general theory of categorization, automaticity, and the development of perceptual expertise (Nosofsky & Palmeri, 1997a; Palmeri, 1997; Palmeri, Wong, & Gauthier, 2004). In some real-world domains, novices learn to categorize objects by first learning to use a set of explicit verbal rules. This novice categorization can be a relatively slow, deliberate, attentiondemanding process. With experience, as people develop perceptual expertise, categorization often 148
basic cognitive skills
becomes fast and automatic. Logan (1988) generally attributed the development of automaticity in cognitive skills to a shift from strategic algorithmic processes to exemplar-based memory retrieval. Palmeri (1997) specifically examined the development of automaticity in categorization as a shift from a rule-based process to an exemplar-based process assumed by EBRW (see also Johansen & Palmeri, 2002). In Palmeri (1997, 1999), subjects were told to use an explicit rule in a dot-counting categorization task.They were asked to categorize random patterns containing between six and eleven dots according to numerosity and did so over multiple sessions. RTs observed in the first session increased linearly with the number of dots, as shown in Figure 7.5A. The dot patternswere repeated across multiple sessions, giving subjects an opportunity to develop automaticity in the task. Indeed, RTs observed in the 13th session were flat with numerosity, a signature of automaticity. In a subsequent transfer test, shown in Figure 7.5B, new unrelated test patterns had categorization RTs that were a linear function of numerosity, just like the first session, and old training patterns continued to be categorized with the same RTs irrespective of numerosity, just like the last session. Dot patterns of low or moderate similarity to the training patterns were categorized with RTs intermediate to the new unrelated and old training patterns. Categorization RTs over sessions and numerosity were successfully modeled as a horse race between an explicit dot-counting process, whose stochastic finishing time increased linearly with the number of dots in the pattern; and an exemplar-based categorization process determined by the similarity of a dot pattern to stored exemplars of patterns previously categorized—an elaboration of EBRW.4 When few patterns have been experienced, categorization is based entirely on the finishing times of the explicit counting process, predicting increased RTs with increased numerosity (Figure 7.5C). With experience, the memory strength mj (Eq. 3) of exemplars in EBRW increases, causing a faster accumulation of perceptual evidence to a response threshold. Faster categorizations based on exemplar retrieval imply increased likelihood that those categorizations finish more quickly than explicit counting. With sufficient experience, EBRW finishes before counting for nearly all categorizations, predicting flat RTs with numerosity. The similaritybased retrieval in EBRW also predicts the transfer results, as shown in Figure 7.5D.
(C) EBRW Predictions
(A) Observed Data 1
3000
2500 2
2000
3 4
1500
Response Time
2500 Response Time (ms)
1
3000
2
2000
3
1500
4 5 6 10 13
5 1000
1000
6 10 13
500
500 6
7
8 9 Numerosity
10
11
6
(B) Observed Data
Unrelated Low
2000 Moderate 1500
1000
Old Session 13 8
9
10
11
Numerosity
Predicted Response Time
Response Time (ms)
2500
7
10
11
Unrelated
3000
Session 1
6
8 9 Numerosity
(D) EBRW Predictions
3000
500
7
2500 Low 2000 Moderate
1500
1000 Old 500
6
7
8 9 Numerosity
10
11
Fig. 7.5 Data from Experiment 1 of Palmeri (1997). Subjects categorized dot patterns varying in numerosity over thirteen training sessions and these were followed by a transfer test. (A) Response time (ms) as a function of numerosity in each training session (1-13). (B) Response time (ms) as a function of numerosity during transfer for old patterns, new unrelated patterns, and patterns of low and moderate similarity to old patterns. (C and D) EBRW predictions.
A shift from rules to exemplars is just one common characteristic of the development of perceptual expertise that EBRW can help explain (see Palmeri et al., 2004; Palmeri & Cottrell, 2009). As one other brief example, consider the well-known basic-level advantage. The seminal work of Rosch, Morris, Gray, Johnson, & BoyesBraem (1976) showed that people are faster to verify the category of an object at an intermediate basic level of abstraction (“bird”) than at more superordinate (“animal”) or subordinate (“Northern Cardinal”) levels. With expertise, subordinate-level categorizations are verified as quickly as basic-level categorizations (Tanaka & Taylor, 1991). One explanation for the basic-level advantage and related findings in novices is that basic-level categorizations
reflect an initial stage of visual processing, perhaps as early as object segmentation (Grill-Spector & Kanwisher, 2005; see also Jolicoeur, Gluck, & Kosslyn, 1984). For novices, subordinate categorizations are slow because basic-level categorizations must be made first. For experts, the stage of basic-level categorization is somehow bypassed. EBRW provides an alternative account (see Mack & Palmeri, 2011; Mack, Wong, Gauthier, Tanoka, & Palmeri, 2007). Faster or slower categorization at different levels of abstraction need not reflect stages of processing but may instead reflect differences in the speed of evidence accumulation in the random walk. Mack et al. (2007) simulated basic-level and subordinate-level categorization by novices and experts. For these
an exemplar-based random-walk model
149
simulations, subordinate categories were simply assumed to be individual identities of objects within clusters that were the basic-level categories. For moderate levels of the sensitivity parameter, c, which reflects the overall discriminability in the psychological space (Eq. 2), a basic-level advantage was predicted. But for high levels of sensitivity, reflecting the greater discriminability that may come with perceptual expertise, equivalent RTs for the subordinate and basic levels were predicted. Further empirical evidence has supported the hypothesisthat differences in RTs at different levels of abstraction reflect how quickly perceptual evidence accumulates rather than differences in discrete visual processing stages (e.g., Mack, Gauthier, Sadr, & Palmeri, 2008; Mack & Palmeri, 2010).
Using Probabilistic Feedback Manipulations to Contrast the Predictions from the EBRW Model and Alternative Models One the major alternative modeling approaches to predicting multidimensional classification RTs arises from a general framework known as “decisionboundary theory” (e.g., Ashby & Townsend, 1986). According to this framework, people construct decision boundaries for dividing a perceptual space into category response regions. Test items are assumed to give rise to noisy representations in the multidimensional perceptual space. If a perceived representation falls in Region A of the space, then the observer classifies the item into Category A. Most past approaches to generating RT predictions from decision-boundary theory have involved applications of the RT-distance hypothesis (Ashby, Boynton, & Lee, 1994). According to this hypothesis, RT is a decreasing function of the distance of a stimulus from the decision boundary. More recent models that formalize the operation of decision boundaries posit perceptual-sampling mechanisms that drive random-walk or diffusion processes, similar to those of the EBRW model (e.g., Ashby, 2000; Fific, Little, & Nosofsky, 2010; Nosofsky & Stanton, 2005). For example, in Nosofsky and Stanton’s (2005, p. 625) approach, on each step of a random walk, a percept is sampled randomly from the perceptual distribution associated with a presented stimulus. If the percept falls in Region A (as defined by the decision boundary), then the random walk steps toward threshold A, otherwise it steps toward threshold – B. The perceptual-sampling process continues until either threshold is reached. Such models provide 150
basic cognitive skills
process interpretations for why RT should get faster (and accuracy should increase) as distance of a stimulus from a decision boundary increases. According to the EBRW model and decisionboundary models, the nature of the memory representations that are presumed to underlie categorization are dramatically different (i.e., stored exemplars versus boundary lines). Despite this dramatic difference, it is often difficult to distinguish between the predictions from the models. The reason is that distance-from-boundary and relative summed-similarity tend to be highly correlated in most experimental designs. For example, as we already described with respect to the Figure 7.3 (top panel) structure, items far from the boundary tend to be highly similar to exemplars from their own category and dissimilar to exemplars from the contrast category. Thus, both the distance-fromboundary model and the EBRW model tend to predict faster RTs and more accurate responding as distance from the boundary increases. In one attempt to discriminate between the RT predictions of the models, Nosofsky and Stanton (2005) conducted a design that aimed to decouple distance-from-boundary and relative summed similarity (for a related approach, see Rouder & Ratcliff, 2004). The key idea in the design was to make use of probabilistic feedback manipulations associated with individual stimuli in the space. The design is illustrated in Figure 7.6. The stimuli were again a set of 12 Munsell colors of a constant hue, varying in their brightness and saturation. As illustrated in the figure, Colors 1–6 belonged to Category A, whereas Colors 7–12 belonged to Category B. To help motivate the predictions, we have drawn a diagonal linear decision boundary for separating the two categories of colors into response regions. Given reasonable assumptions (for details, see Nosofsky & Stanton, 2005), this boundary is the optimal (ideal-observer) boundary according to decision-boundary theory. That is, it is the boundary that would maximize an observer’s percentage of correct categorization decisions. In generating predictions, decision-bound theorists often assume that observers will use a boundary with an optimal form. However, we will consider more general possibilities later in this section. The key experimental manipulation was that, across conditions, either Stimulus Pair 4/8 or Stimulus Pair 5/9 received probabilistic feedback, whereas all other stimuli received deterministic feedback. Specifically, in Condition 4/8, Stimulus 4 received Category-A feedback with probability 0.75
but received Category- B feedback with probability 0.25; whereas Stimulus 8 received Category- B feedback with probability 0.75 and Category- A feedback with probability 0.25. Analogous probabilistic feedback assignments were given to Stimuli 5 and 9 in Condition 5/9. In each condition, we refer to the pair that received probabilistic feedback as the probabilistic pair and to the pair that received deterministic feedback as the deterministic pair. The key conceptual point is that, because of the symmetric probabilistic assignment of stimuli to categories, the optimal boundary for partitioning the space into response regions is the same diagonal linear decision boundary that we have already illustrated in Figure 7.6. There is no way to adjust the boundary to achieve more accurate responding in the face of the probabilistic feedback assignments. Furthermore, because the probabilistic and deterministic pairs are an equal distance from the decision boundary, the most natural prediction from that modeling approach is that RTs for the probabilistic and deterministic pairs should be the same. By contrast, the EBRW model predicts that the probabilistic pair should be classified more slowly (and with lower accuracy) than the deterministic pair. For example, in Condition 4/8, in cases in which stimulus 4 is presented and tokens of exemplar 4 are retrieved from memory, 0.75 of the steps in the random walk will move in the direction of threshold +A, but 0.25 of the steps will
2
7
Brightness
6
1
5
4
6
5
4
3
9
8
7
3 4
10
6
12
11 8 Saturation
10
12
Fig. 7.6 Schematic illustration of the category structure used in Nosofsky and Stanton’s (2005) probabilistic-feedback experiment.
move in the direction of threshold –B. By contrast, presentations of the deterministic pair will result in more consistent steps of the random walk, leading to faster RTs and more accurate responding. Across two experiments (in which the instructions varied the relative emphasis on speed versus accuracy in responding), the qualitative pattern of results strongly favored the predictions from the EBRW model compared to the distance-fromboundary model. In particular, in both experiments, observers responded more slowly and with lower accuracy to the probabilistic pair than to the deterministic pair. These results were observed at both early and late stages of testing and were consistent across the majority of the individual participants. As a converging source of evidence, the EBRW model also provided better overall quantitative fits to the individual-subject choiceprobability and RT data than did the distancefrom-boundary model, including versions of the latter model in which the slope and y-intercept of the linear boundary were allowed to vary as free parameters. As suggested by Nosofsky and Stanton (2005, p. 623), the results were particularly intriguing because they pointed toward a stubborn form of suboptimality in human performance: In the Figure 7.6 design, subjects would perform optimally by simply ignoring the probabilistic feedback and classifying objects based on the linear decision boundary. Nevertheless, despite being provided with monetary payoffs for correct responses, subjects’ behavior departed from that optimal strategy in a manner that was well predicted by the EBRW model. Similar forms of evidence that favor the predictions from the EBRW model have been reported in related studies that manipulated the overall familiarity of individual study exemplars rather than probabilistic feedback assignments (e.g., Nosofsky & Palmeri, 1997a; Verguts, Storms, and Tuerlinckx, 2003). Although the focus of Nosofsky and Stanton’s (2005) study was to contrast predictions from the EBRW model and decision-bound models, the designs also yielded sharp contrasts between the EBRW model and prototype models.5 Specifically, Nosofsky and Stanton (2005, p. 610) formulated a prototype-based random-walk (PBRW) model, analogous in all respects to the EBRW model, except that the category representation was presumed to correspond to the central tendency of each category distribution rather than to the individual exemplars. It turns out that for the stimulus spacings and
an exemplar-based random-walk model
151
probabilistic stimulus-category assignments used in the Figure 7.6 design, the central tendency of each category is equidistant to the probabilistic and deterministic stimulus pairs. Thus, the PBRW model predicted incorrectly that those pairs would show identical choice probabilities and RTs. Not surprisingly, therefore, the PBRW yielded far worse quantitative fits to the full sets of choice-probability and RT data than did the EBRW model.
Extending the EBRW Model to Predict Old-New Recognition RTs Overview As noted in the introduction, a central goal of exemplar models is to explain not only categorization, but other fundamental cognitive processes such as old-new recognition. The GCM has provided successful accounts of old-new recognition choice probabilities in wide varieties of experimental situations. When applied to recognition, the GCM assumes that each item from a study list is stored as a distinct exemplar in memory. The observer is presumed to sum the similarity of a test item to these stored exemplars. The greater the summed similarity, the more familiar is the test item, so the greater is the probability with which the observer responds “old.” Indeed, the GCM can be considered a member of the general class of “global matching” models of old-new recognition (e.g., Clark & Gronlund, 1996; Gillund & Shiffrin, 1984; Hintzman, 1988; Murdock, 1982). Within this broad class, an important achievement of the model is that it predicts fine-grained differences in recognition probabilities for individual items based on their fine-grained similarities to individual exemplars in the study set (e.g., Nosofsky, 1988, 1991; Nosofsky & Zaki, 2003). Just as has been the case for categorization, a major development in recent years has involved extensions of the GCM in terms of the EBRW model to allow it to account for recognition RTs (Donkin & Nosofsky, 2012a,b; Nosofsky, Little, Donkin, & Fific, 2011; Nosofsky & Stanton, 2006). In this section we describe these formal developments and illustrate applications to variants and extensions of the classic Sternberg (1966) shortterm probe-recognition paradigm. In this paradigm, subjects are presented on each trial with a short list of items (the memory set) and are then presented with a test probe. The subjects judge, as rapidly as possible, while minimizing errors, whether the probe occurred in the memory set. Fundamental variables that are manipulated in the paradigm 152
basic cognitive skills
include the size of the memory set; whether the test probe is old (a “positive” probe) or new (a “negative” probe); and, if old, the serial position with which the positive probe occurred in the memory set. Whereas in the standard version of the Sternberg paradigm the to-be-recognized items are generally highly distinct entities, such as alphanumeric characters, recent extensions have examined recognition performance in cases involving confusable stimuli embedded in a continuous-dimension similarity space (e.g., Kahana & Sekuler, 2002). We focus on this type of extended similarity-based paradigm in the initial part of this section; however, we will illustrate applications of the EBRW model to the more standard paradigm as well.
The Formal Model The EBRW-recognition model makes the same representational assumptions as does the categorization model: (a) Exemplars are conceptualized as occupying points in a multidimensional similarity space; (b) similarity is a decreasing function of distance in the space (Eqs. 1 and 2); (c) activation of the exemplars is a joint function of their memory strength and their similarity to the test probe (Equation 3); and (d) the stored exemplars race to be retrieved with rates proportional to their activations (Eq. 4). In our previous applications to categorization, a single global level of sensitivity (c in Eq. 2) was assumed that applied to all exemplar traces stored in long-term memory. In application to short-term recognition paradigms involving highsimilarity lures, however, allowance is made for a form of exemplar-specific sensitivity. In particular, an observer’s ability to discriminate between test item i and a nonmatching exemplar-trace j will almost certainly depend on the recency with which exemplar j was presented: Discrimination is presumably much easier if an exemplar was just presented than if it was presented in the distant past. In the version of the model applied by Nosofsky et al. (2011), a separate sensitivity parameter cj was estimated for each individual lag j on the study list, where lag is counted backward from the presentation of the test probe to the memoryset exemplar. For example, for the case in which memory set-size is 4, the exemplar in the fourth serial position has Lag 1, the exemplar in the third serial position has Lag 2, and so forth. Likewise, the memory strengths of the individual exemplars (mj ) were also assumed to depend on lag j: Presumably, the more recently an exemplar
was presented on the study list, the greater its memory strength (e.g., McElree & Dosher, 1989; Monsell, 1978). (Although the effects are smaller, in modeling short-term recognition with the EBRW, allowance is also typically made for a primacy effect on memory strength. The memory strength of the item in the first serial position of the memory set is given by PM · mj , where mj is the memory strength for an item with lag j, and PM is a primacy-multiplier parameter.) To adapt the EBRW model to the domain of old-new recognition, it is assumed that the observer establishes what are termed “criterion elements” in the memory system. These elements are similar to the “background elements” used for modeling the early stages of category learning. Just as is the case for the stored exemplars, upon presentation of a test probe, the criterion elements race to be retrieved. However, whereas the retrieval rates of the stored exemplars vary with their lag-dependent memory strengths and their similarity to the test probe, the retrieval rates of the criterion elements are independent of these factors. Instead, the criterion elements race with some fixed rate β, independent of the test probe that is presented. The setting of β is presumed to be, at least in part, under the control of the observer. Finally, the retrieved exemplars and criterion elements drive a random-walk process that governs old-new recognition decisions. The observer sets response thresholds +OLD and –NEW that establish the amount of evidence needed for making an “old” or a “new” response. On each step of the random walk, if an old exemplar wins the retrieval race, then the random-walk counter takes a step in the direction of the +OLD response threshold; whereas if a criterion element wins the race, then the counter takes a step in the direction of the –NEW threshold. The retrieval process continues until one of the thresholds is reached. Given the processing assumptions outlined earlier, then on each step of the random walk, the probability that the counter steps in the direction of the +OLD threshold is given by pi = F i /(F i + β),
(7)
where Fi gives the summed activation (“familiarity”) of the test probe to all old exemplars on the study list (and β is the fixed setting of criterion-element activation). Note that test probes that match recently presented exemplars (with high memory strengths) will cause high summed familiarity (Fi ), leading the random walk to march quickly to the
+OLD threshold and resulting in fast old RTs. By contrast, test probes that are highly dissimilar to the memory-set items will not activate the stored exemplars, so only criterion elements will be retrieved. In this case, the random walk will march quickly to the –NEW threshold, resulting in fast new RTs. Through experience in the task, the observer is presumed to learn an appropriate setting of the criterion-element activation β, such that summed activation (Fi ) tends to exceed β when the test probe is old, but tends to be less than β when the test probe is new. In this way, the random walk will tend to step toward the appropriate response threshold on trials in which old versus new probes are presented.
Experimental Tests In Nosofsky et al.’s (2011) initial experiment for testing the model, the stimuli were a set of 27 Munsell colors that varied along the dimensions of hue, brightness, and saturation. Similarity-scaling procedures were used to derive a precise MDS solution for the colors. The design of the probe-recognition experiment involved a broad sampling of different list structures to provide a comprehensive test of the model. There were 360 lists in total. The size of the memory set on each trial was either 1, 2, 3 or 4 items, with an equal number of lists at each set size. For each set size, half the test probes were old and half were new. In the case of old probes, the matching item from the memory set occupied each serial position equally often. To create the lists, items were randomly sampled from the full set of stimuli, subject to the constraints described earlier. Thus, a highly diverse set of lists was constructed, varying not only in set size, old/new status of the probe, and serial position of old probes, but also in the similarity structure of the lists. Because the goal was to predict performance at the individual-subject level, three subjects were each tested for approximately 20 one-hour sessions, with each of the 360 lists presented once per session. As it turned out, each subject showed extremely similar patterns of performance, and the fits of the EBRW model yielded similar parameter estimates for the three subjects. Therefore, for simplicity, and to reduce noise in the data, we report the results from the analysis of the averaged subject data. In the top panels of Figure 7.7 we report summary results from the experiment. The topright panel reports the observed mean RTs plotted as a function of: (a) set size, (b) whether the probe
an exemplar-based random-walk model
153
Observed
Observed 1200
0.8
Mean RT Observed
p(Error) Observed
1
0.6 0.4 0.2 0
1
2
3
4
1000 800 600 400
Lure
1
2
Lag
Mean RT Predicted
p(Error) Predicted
0.6 0.4 0.2 2
3
Lure
Set Size 1 Set Size 2 Set Size 3 Set Size 4
1200
0.8
1
4
Core Model
Core Model 1
0
3 Lag
4
Lure
1000 800 600 400
Lag
1
2
3
4
Lure
Lag
Fig. 7.7 Summary data from the short-term memory experiment of Nosofsky, Little, Donkin, and Fific (2011). (Top) Observed error rates and mean RTs. (Bottom) Predictions from the EBRW model.
was old or new (i.e., a lure), and (c) the lag with which old probes appeared in the memory set. For old probes, there was a big effect of lag: In general, the more recently a probe appeared on the study list, the shorter was the mean RT. Indeed, once one takes lag into account, there is little remaining effect of set size on the RTs for the old probes. That is, as can be seen, the different set size functions are nearly overlapping. The main exception is a persistent primacy effect, in which the mean RT for the item at the longest lag for each set size is “pulled down.” (The item at the longest lag occupies the first serial position of the list.) By contrast, for the lures, there is a big effect of set size, with longer mean RTs as set size increases. The mean proportions of errors for the different types of lists, shown in the top-left panel of Figure 7.7, mirror the mean RT data just described. The goal of the EBRW modeling, however, was not simply to account for these summary trends. Instead, the goal was to predict the choice probabilities and mean RTs observed for each of the individual lists. Because there were 360 unique lists in the experiment, this goal entailed simultaneously predicting 360 choice probabilities and 360 mean RTs. The results of that model-fitting goal are shown in the top and bottom panels of 154
basic cognitive skills
Figure 7.8. The top panel plots, for each individual list, the observed probability that the subjects judged the probe to be “old” against the predicted probability from the model. The bottom panel does the same for the mean RTs. Although there are a few outliers in the plots, overall the model achieves a good fit to both data sets, accounting for 96.5% of the variance in the choice probabilities and for 83.4% of the variance in the mean RTs. The summary-trend predictions that result from these global fits are shown in the bottom panels of Figure 7.7. It is evident from inspection that the EBRW does a good job of capturing these summary results. For the old probes, it predicts the big effect of lag on the mean RTs, the nearly overlapping set-size functions, and the facilitation in RT with primacy. Likewise, it predicts with good quantitative accuracy the big effect of set size on the lure RTs. The error-proportion data (left panels of Figure 7.7) are also well predicted, with the main exception that a primacy effect was predicted but not observed for the size-2 lists. The explanation of these results in terms of the EBRW model is straightforward. According to the best-fitting parameters from the model (see Nosofsky et al., 2011, Table 2), more recently
1.0
P(old) Observed
0.8
0.6 Size 1 Lag 1 Size 2 Lag 1 Size 2 Lag 2 Size 3 Lag 1 Size 3 Lag 2 Size 3 Lag 3 Size 4 Lag 1 Size 4 Lag 2 Size 4 Lag 3 Size 4 Lag 4
0.4
0.2
Lure Size 1 Lure Size 2 Lure Size 3 Lure Size 4
0.0
0.2
0.0
0.4 0.6 P(old) Predicted
1.0
0.8
1400
1200
Mean RT Observed
1000
800
Size 1 Lag 1 Size 2 Lag 1 Size 2 Lag 2 Size 3 Lag 1 Size 3 Lag 2 Size 3 Lag 3 Size 4 Lag 1 Size 4 Lag 2 Size 4 Lag 3 Size 4 Lag 4
600
400
200 200
Lure Size 1 Lure Size 2 Lure Size 3 Lure Size 4 400
600
800 1000 Mean RT Predicted
1200
1400
Fig. 7.8 Scatterplots of observed and EBRW-predicted old recognition probabilities and mean RTs associated with individual lists from the short-term memory experiment of Nosofsky, Little, Donkin, and Fific (2011).
an exemplar-based random-walk model
155
presented exemplars had greater memory strengths and sensitivities than did less recently presented exemplars. From a psychological perspective, this pattern seems highly plausible. For example, presumably, the more recently an exemplar was presented, the greater should be its strength in memory. Thus, if an old test probe matches the recently presented exemplar, it will give rise to greater overall activation, leading to faster mean old RTs. In the case of a lure, as set size increases, the overall summed activation yielded by the lure will also tend to increase. This pattern arises both because a greater number of exemplars will contribute to the sum, and because the greater the set size, the higher is the probability that a least one exemplar from the memory set will be highly similar to the lure. As summed activation yielded by the lures increases, the probability that the random walk takes correct steps toward the –NEW threshold decreases, so mean RTs for the lures get longer. Beyond accounting well for these summary trends, inspection of the detailed scatterplots in Figure 7.8 reveals that the model accounts for fine-grained changes in choice probabilities and mean RTs depending on the fine-grained similarity structure of the lists. For example, consider the choice-probability plot (Figure 7.8, top panel) and the Lure-Size-4 items (open diamonds). Whereas performance for those items is summarized by a single point on the summary-trend figure (Figure 7.7), the full scatterplot reveals extreme variability in results across different tokens of the Lure-Size4 lists. In some cases the false-alarm rates associated with these lists are very low, in other cases moderate, and in still other cases the false-alarm rates exceed the hit rates associated with old lists. The EBRW captures well this variability in false-alarm rates. In some cases, the lure might not be similar to any of the memory-set items, resulting in a low false-alarm rate; whereas in other cases the lure might be highly similar to some of the memory-set items, resulting in a high false-alarm rate. The application reviewed earlier involved a version of a short-term probe-recognition paradigm that used confusable stimuli embedded in a continuous-dimension space. However, the EBRW model has also been applied successfully to more standard versions of such paradigms that involve easy-to-discriminate stimuli such as alphanumeric characters. In those applications, instead of adopting MDS approaches, a highly simplified model of similarity is used: The similarity of a probe to 156
basic cognitive skills
itself is set equal to one, whereas the similarity between a probe and any nonmatching item is set equal to a free parameter s. Not only has this simple version of the EBRW model captured many of the classic patterns of results involving mean RTs in such paradigms, it also accounts successfully for the detailed RT-distribution data that have been observed (Nosofsky et al., 2011; Donkin & Nosofsky, 2012a,b). Furthermore, applications of the EBRW model to both continuous-similarity and discrete versions of the probe-recognition paradigm have led to the discovery of an interesting regularity involving memory strength. As noted earlier, in the initial tests of the model, separate-memory strength parameters were estimated corresponding to each individual lag on the study list. It turns out, however, that the estimated memory strengths follow almost a perfect power function of this lag. For example, in an experiment reported by Donkin and Nosofsky (2012a), participants studied 12-item lists consisting of either letters or words, followed by a test probe. Separate RT-distribution data for hits and misses for positive probes were collected at each study lag. (RT-distribution data for false alarms and correct rejections for negative probes were collected as well.) The EBRW model provided an excellent quantitative account of this complete set of detailed RT-distribution and choice-probability data.6 The discovery that resulted from the application of the model is illustrated graphically in Figure 7.9. The figure plots, for each of four individual participants who were tested, the estimated memory-strength parameters against lag. As shown in the figure, the magnitudes of the memory strengths are extremely well captured by a simple power function. Interestingly, other researchers have previously reported that a variety of empirical forgetting curves are well described as power functions (e.g., Anderson & Schooler, 1991; Wickelgren, 1974; Wixted & Ebbesen, 1991). For example, Wixted and Ebbesen (1991) reported that diverse measures of forgetting, including proportion correct of free recall of word lists, recognition judgments of faces, and savings in relearning lists of nonsense syllables, were well described as power functions of the retention interval. Wixted (2004) considered a variety of possible reasons for the emergence of these empirical powerfunction relations and concluded that the best explanation was that the strength of the memory traces themselves may exhibit power-function decay. The model-based results from Donkin and Nosofsky (2012a,b) lend support to Wixted’s
1.0
1.0
0.8
0.8
0.6
0.6 Participant 3
Memory Strength
Participant 1 0.4
0.4
0.2
0.2
0.0
0.0 1
3
5
7
1
9
1.0
1.0
0.8
0.8
0.6
3
5
7
9
0.6 Participant 4
Participant 2 0.4
0.4
0.2
0.2
0.0
0.0 1
3
5 Lag
7
9
1
3
5 Lag
7
9
Fig. 7.9 Model-based results from the probe-recognition experiment of Donkin and Nosofsky (2012a). Estimated memory strengths (open circles) are plotted as a function of lag, along with the best-fitting power functions. Donkin, C., & Nosofsky, R.M. (2012). A power-law model of psychological memory strength in short-term and long-term recognition. Psychological Science, 23, 625–634. Adapted with permission of SAGE Publications.
suggestion and now motivate the new research goal of unpacking the detailed psychological and neurological mechanisms that give rise to this discovered power law of memory strength.
Conclusions and New Research Goals In sum, the EBRW is an important candidate model for explaining how the process of categorization unfolds over time. The model combines assumptions involving exemplar-based category representation and processes of evidence accumulation within a unified framework to account for categorization and recognition choice-probability and RT data. As reviewed in this chapter, it accounts successfully for a wide variety of fundamental effects in these domains including effects of similarity, distance-from-boundary, familiarity, probabilistic feedback, practice, expertise, set size, and lag. Although we were able to sample only a limited number of example applications in this single chapter, we should clarify that the exemplar model has been applied in a wide variety of stimulus domains and to varied category and study-list structures. The stimulus domains include colors, dot patterns, multidimensional cartoon drawings, geometric forms, schematic faces, photographs of
faces, alphanumeric characters, words, and pictures of real-world objects. The category structures include small collections of continuous-dimension stimuli separated by boundaries of varying degrees of complexity, normally distributed category structures, high-dimensional stimuli composed of discrete dimensions, and categories generated from statistical distortions of prototypes (e.g., see Richler & Palmeri, 2014). Furthermore, growing neural evidence ranging from single-unit records to functional brain imaging supports a number of the processing assumptions embodied in models like EBRW (see Box 1). Finally, as illustrated in our chapter, a key theme of the theoretical approach is that, despite the dramatically different task goals, the processes of categorization and old-new recognition may be closely related (but see Box 2 for discussion of a major theoretical debate regarding this issue in the cognitive-neuroscience literature). A likely reason for the model’s success is that it builds on the strengths of classic previous approaches for understanding processes of choice and similarity, the development of automaticity, and evidence accumulation in decision-making and memory. Beyond accounting for categorization and recognition, we believe that the EBRW model can serve
an exemplar-based random-walk model
157
Box 1 Neural Evidence Supports Mechanistic Assumptions in EBRW. EBRW proposes many mechanistic assumptions, such as exemplar representations, attention weights along relevant dimensions, and accumulation of perceptual evidence. In most modeling, these are supported by evaluating predictions of behavioral data like accuracy and RTs. But we can now evaluate particular mechanistic assumptions using relevant neural data from brain regions hypothesized to instantiate those mechanisms. For example, Mack, Preston and Love, (2013; see also Palmeri, 2014) turned to patterns of brain activity measured with fMRI to evaluate whether the brain represents categories using exemplars or prototypes. Subjects learned to classify objects into one of two categories and then in the scanner were tested on training and transfer objects without feedback (Medin & Schaffer, 1978). Typical fMRI analyses would correlate brain activity with stimuli or responses, for example highlighting regions that modulate with categorization difficulty. Instead, Mack and colleagues first fitted exemplar and prototype models to individual subjects’ categorization responses; despite the fact that these models made fairly similar behavioral predictions, they differed in patterns of summed similarity to their respective exemplar or prototype representations. Mack and colleagues showed that patterns of individual subjects’ brain activity were more consistent with patterns of summed similarity predicted by an exemplar model than those predicted by a prototype model. According to exemplar models, learning to categorize objects can cause selective attention to relevant psychological dimensions, stretching psychological space to better allow subjects to discriminate between members of contrasting categories. Neurophysiology and fMRI have suggested that category-relevant dimensions can be emphasized in visual cortex. After monkeys learned to categorize multidimensional objects, neurons in inferotemporal cortex were more sensitive to variations along a relevant dimension than an irrelevant dimension (De Baene, Ons, Wagemans, & Vogels, 2008; Sigala and Logothetis 2002; see also Gauthier & Palmeri, 2002). Similarly, after people learned object categories, psychological
158
basic cognitive skills
stretching of relevant dimensions was accompanied by neural stretching of relevant dimensions measured by fMRI (Folstein, Palmeri, & Gauthier, 2013; see also Folstein, Gauthier, & Palmeri, 2012). Finally, mathematical psychology and systems neuroscience have converged on accumulation of perceptual evidence as a general theoretical framework to explain the time course of decision making (see Palmeri, Schall, & Logan, this volume). Some neurons show dynamics predicted by accumulator models, other neurons show activity consistent with encoded perceptual evidence to be accumulated over time, and an ensemble of neurons predicts the time course of decisions made by awake behaving monkeys (e.g., Purcell, Schall, Logan, & Palmeri, 2012; Zandbelt, Purcell, Palmeri, Logan, Schall, 2014).
Box 2 The Exemplar Model Accounts for Dissociations Between Categorization and Recognition Demonstrated in the Cognitive-Neuroscience Literature. Interestingly, in contrast to the theme emphasized in this chapter, the prevailing view in the cognitive neuroscience literature is that separate cognitive/neural systems mediate categorization and recognition (Smith, 2008). The main source of evidence involves the demonstration of intriguing dissociations between categorization and recognition. For example, studies have demonstrated that amnesics with poor recognition memory perform at normal levels in categorization tasks involving the same types of stimuli (e.g., Knowlton & Squire, 1993). Nevertheless, formal modeling analyses have indicated that even these dissociations are consistent with the predictions from the exemplar model (e.g., Nosofsky, Denton, Zaki, Murphy-Knudson, & Unverzagt 2012; Nosofsky & Zaki, 1998; Palmeri & Flanery, 1999, 2002; Zaki, Nosofsky, Jessup, & Unverzagt, 2003). The general approach in the modeling was to assume that amnesics have reduced ability to discriminate among distinct exemplars in memory. This reduced discriminability is particularly detrimental to old-new recognition, which may
Box 2 Continued require the observer to make fine-grained distinctions between old versus new items. However, the reduced discriminability is not very detrimental in typical tasks of categorization, which may require only gross-level assessments of similarity to be made. A more direct challenge to the exemplar-model hypothesis comes from brain-imaging studies that show that distinct brain regions are activated when observers engage in recognition vs. categorization tasks (Reber et al., 1998a,b). Exemplar theorists have responded, however, by providing evidence that these brain-imaging dissociations may not reflect the operation of separate neural systems devoted to categorization versus recognition per se. Instead, the brain-imaging dissociations may reflect changes in stimulus-encoding strategies across task situations (Gureckis, James, & Nosofsky, 2011), differences in the precise stimuli that are tested (Nosofsky, Little, & James, 2012; Reber et al., 2003), as well as adaptive changes in parameter settings that allow observers to meet the competing task goals of categorization versus recognition (Nosofsky et al., 2012). as a useful analytic device for assessing human performance. For example, note that Ratcliff ’s (1978) diffusion model has been applied to analyze choice behavior in various special populations, including elderly adults, sleep-deprived subjects, and so forth (see Chapter 2 of this volume). The model-based analyses provide a deeper understanding of the locus of the cognitive/perceptual deficits in such populations by tracing them to changes in diffusion-model drift rates, responsethreshold settings, or residual times. The EBRW model has potential to reveal even more fine-grained information along these lines. For example, in that model, the random-walk step probabilities (i.e., drift rates) emerge from cognitive/perceptual factors such as overall sensitivity, attention-weight settings, and memory strengths of stored exemplars, each of which can be measured by fitting the model to data obtained in suitable categorization and recognition paradigms. Although exemplar models have been applied to help interpret the behavior of amnesic subjects and patients with mild memory-based cognitive impairment (see Box 2), we have only scratched the surface of such potential applications to many more clinical groups.
Finally, an important theme in the categorization literature is that there may be multiple systems of categorization (e.g., Ashby, Alfonso-Reese, Turken, & Waldron, 1998; Erickson & Kruschke, 1998; Johansen & Palmeri, 2002; Nosofsky, Palmeri, & McKinley, 1994). A classic idea, for example, is that many categories may be represented and processed by forming and evaluating logical rules. In some modern work that pursues this avenue, researchers have considered the RT predictions from logicalrule models of classification. Furthermore, such approaches have been used to develop sharp contrasts between the predictions of the ERBW model and rule-based forms of category representation and processing (Fific, Little, & Nosofsky, 2010; Lafond, Lacouture, & Cohen, 2009; Little, Nosofsky, & Denton, 2011). In domains involving highly separable-dimension stimuli in which the category structures can be described in terms of exceedingly simple logical rules, evidence has been mounting that the logical-rule models provide better accounts of the detailed patterns of RT data than does the EBRW model. An important target for future research is to develop a deeper understanding of these multiple forms of categorization, to learn about the experimental factors that promote the use of each strategy, and to explain the manner in which exemplar-based and rule-based systems may interact.
Acknowledgments This work was partially supported by NSF grants SMA-1041755 and SBE-1257098 and by AFOSR grant FA9550-14-1-0307.
Notes
1. In the context of the GCM, the parameter γ is referred to as a response-scaling parameter. When γ =1, observers respond by “probability-matching” to the relative summed similarities. As γ grows greater than 1, observers respond more deterministically with the category that yields the greater summed similarity (Ashby & Maddox, 1993; McKinley & Nosofsky, 1995). 2. The version of the EBRW model described in this chapter is applicable to “integral-dimension” stimuli, which are encoded and perceived holistically. A common example of such integral-dimension stimuli are colors varying in hue, brightness, and saturation. Because there has been extensive previous scaling work indicating that similarity relations among these stimuli are extremely well described in terms of these dimensions, we often use these stimuli in our tests of the EBRW model. An extended version of the EBRW model has also been developed that is applicable to separable-dimension stimuli (Cohen & Nosofsky, 2003). In this version, rather than encoding stimuli in holistic fashion, the encoding of individual stimulus dimensions is a stochastic process, and similarity
an exemplar-based random-walk model
159
relations between a test item and the stored exemplars change dynamically during the time course of processing (see also Lamberts, 2000). 3. Because accuracies were near ceiling in the present experiment, we focused our analysis primarily on the patterns of RT data. However, in a follow-up study, Nosofsky and Alfonso-Reese (1999) tested conditions that allowed examination of how both speed and accuracy changed during the early stages of learning. By including the background-activation parameter b in its arsenal, the EBRW model provided good quantitative fits to not only the speed-up in mean RTs, but to the improvements in choice-accuracy data as well. (As noted by Nosofsky and Palmeri, 1997a, p. 291, with b=0, the EBRW model does not predict changes in response accuracy.) 4. The original EBRW model (Nosofsky & Palmeri, 1997a) applies to two-alternative forced- choice responses. There is a single accumulator whose value increases or decreases as evidence accumulates in the random walk until an upper or lower response threshold is reached. Numerosity judgments in Palmeri (1997) permitted six possible responses, so EBRW was extended to allow multiple alternatives. Each response alternative was associated with its own counter, so with six numerosity responses there were six counters. Whenever an exemplar was retrieved with the label associated with a particular counter, the value of that counter was incremented. A response was made whenever the value of one of the counters exceeded all the rest by some relative amount. With only two alternatives, this multiple counter model with a relative threshold response rule generally mimics a standard random-walk model with one accumulator with a positive and negative threshold. 5. There is a long history of debate, too extensive to be reviewed in this chapter, between the proponents of exemplar and prototype models. For examples of recent research that has argued in favor of the prototype view, see Minda and Smith (2001), Smith and Minda (1998, 2000) and Smith (2002). For examples from the exemplar perspective, see Nosofsky (2000), Nosofsky and Zaki (2002), Palmeri and Flanery (2002), and Zaki and Nosofsky (2007). 6. The version of the exemplar-recognition model reported by Donkin and Nosofsky (2012a) assumed a linear-ballistic accumulation process (Brown & Heathcote, 2008) instead of a random-walk accumulation process. However, the same evidence for a power-law relation between memory strength and lag was obtained regardless of the specific accumulation process that was assumed. We should note as well that, in fitting complete RT distributions for correct and error responses, such as occurred in the Donkin and Nosofsky (2012a) experiment, the exemplar model makes provision for drift-rate variability and response-threshold variability across trials (e.g, Donkin & Nosofsky, 2012a; Nosofsky & Stanton, 2006), in a manner analogous to the approach used in Ratcliff ’s diffusion model (e.g, Ratcliff, Van Zandt, & McKoon, 1999).
Glossary Attention-weight parameters: a set of parameters in the GCM and EBRW models that describe the extent to which each dimension is weighted when computing distances among objects.
160
basic cognitive skills
Automaticity: ability to perform some task at a satisfactory level without requiring conscious attention or effort and without limits in capacity. Background element: a hypothetical construct in the GCM and EBRW models that describes initial background noise in people’s memories for members of alternative categories. Basic-level of categorization: an intermediate level of a category hierarchy that is hypothesized to lead to privileged forms of cognitive processing. Categorization: process in which observers classify distinct objects into groups. Criterion element: a hypothetical entity in the EBRW recognition model. Retrieval of criterion elements leads the random walk to step in the direction of the NEW response threshold. Biases in the random-walk step probabilities are determined by the strength of the criterion elements. Decision-boundary model: model of categorization that assumes that people form boundaries to divide a stimulus space into category-response regions. Exemplar model: model of categorization that assumes that observers store individual examples of categories in memory. Exemplar-based random-walk (EBRW) model: an extension of the generalized context model that explains how the processes of categorization and recognition unfold over time. Exemplars stored in memory “race” to be retrieved, and the retrieved exemplars drive a random-walk decision-making process. Exponential distribution: a probability distribution that describes the time between events, in which the events occur continuously and independently at a constant average rate. Generalized context model (GCM): a member of the class of exemplar models. In the GCM, exemplars are represented as points in a multidimensional psychological space, and similarity is a decreasing function of distance in the space. Integral-dimension stimuli: stimuli composed of individual dimensions that combine into unitary, integral wholes. Memory strength parameters: parameters in the GCM and EBRW models that describe the strength with which the exemplars are stored in memory. Minkowski power model: a model for computing distances between points in a space. Multidimensional scaling: a modeling technique for representing similarity relations among objects. The objects are represented as points in a multidimensional psychological space and similarity is a decreasing function of the distance between the points in the space. Prototype model: model of categorization that assumes that observers represent categories by forming a summary representation, usually assumed to be the central-tendency of the category distribution.
Glossary Prototype-based random-walk (PBRW) model: a model that is analogous in all respects to the EBRW model, except that the category representation corresponds to the central tendency of each category distribution rather than to the individual exemplars. Random walk: a mathematical model that describes a path of outcomes consisting of a sequence of random steps. Response thresholds: parameters in evidenceaccumulation models that determine how much evidence is required before a response is initiated. Recognition memory: process in which observers decide whether objects are “old” (previously experienced) or “new.” Response-scaling parameter: a parameter in the GCM that describes the extent to which observers respond using probabilistic versus deterministic response rules. RT-distance hypothesis: hypothesis that categorization RT is a decreasing function of the distance of a stimulus from the decision boundary. Rule-plus-exception model: model of categorization that assumes that people classify objects by forming simple logical rules and remembering occasional exceptions to those rules. Sensitivity parameter: a parameter in the GCM and EBRW models that describes overall discriminability among distinct items in the multidimensional psychological space. Short-term probe-recognition task: task in which observers are presented with a short list of to-beremembered items followed by a test probe. The observers judge as rapidly as possible, while trying to minimize errors, whether the probe is old or new.
References Anderson, J. R., & Schooler, L. J. (1991). Reflections of the environment in memory. Psychological Science, 2, 396–408. Ashby, F. G. (2000). A stochastic version of general recognition theory. Journal of Mathematical Psychology, 44, 310–329. Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological Review, 105, 442–481. Ashby, F. G., Boynton, G., & Lee, W. W. (1994). Categorization response time with multidimensional stimuli. Perception & Psychophysics, 55, 11–27. Ashby, F. G., & Maddox, W. T. (1993). Relations between prototype, exemplar, and decision bound models of categorization. Journal of Mathematical Psychology, 37, 372–400. Ashby, F. G., & Townsend, J. T. (1986). Varieties of perceptual independence. Psychological Review, 93, 154–179. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice reaction time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178. Bundesen, C. (1990). A theory of visual attention. Psychological Review, 97, 523–547.
Busemeyer, J. R. (1982). Choice behavior in a sequential decision-making task. Organizational Behavior and Human Performance, 29, 175–207. Busemeyer, J. R. (1985). Decision making under uncertainty: A comparison of simple scalability, fixed-sample, and sequential-sampling models. Journal of Experimental Psychology: Learning, Memory, and Cognition, 11, 538–564. Carroll, J. D., & Wish, M. (1974). Models and methods for three-way multidimensional scaling. In D. H. Krantz, R. C. Atkinson, R. D. Luce, and P. Suppes (Eds.), Contemporary developments in mathematical psychology (Vol. 2). San Francisco: W. H. Freeman. Clark, S. E., & Gronlund, S. D. (1996). Global matching models of recognition memory: How the models match the data. Psychonomic Bulletin & Review, 3, 37–60. Cohen, A. L., & Nosofsky, R. M. (2003). An extension of the exemplar-based random-walk model to separable-dimension stimuli. Journal of Mathematical Psychology, 47, 150–165. Davis, T., Love, B. C., & Preston, A. R. (2012). Learning the exception to the rule: Model-based fMRI reveals specialized representations for surprising category members. Cerebral Cortex, 22, 260–273. De Baene, W., Ons, B., Wagemans, J., & Vogels, R. (2008). Effects of category learning on the stimulus selectivity of macaque inferior temporal neurons. Learning & Memory, 15(9), 717–727. Donkin, C., & Nosofsky, R. M. (2012a). A power-law model of psychological memory strength in short-term and long-term recognition. Psychological Science, 23, 625–634. Donkin, C., & Nosofsky, R. M. (2012b). The structure of shortterm memory scanning: An investigation using responsetime distribution models. Psychonomic Bulletin & Review, 19, 363–394. Erickson, M. A., & Kruschke, J. K. (1998). Rules and exemplars in category learning. Journal of Experimental Psychology: General, 127, 107–140. Estes, W. K. (1994). Classification and cognition. New York: Oxford University Press. Fific, M., Little, D. R., & Nosofsky, R. M. (2010). Logical-rule models of classification response times: A synthesis of mentalarchitecture, random-walk, and decision-bound approaches. Psychological Review, 117, 309–348. Folstein, J., Gauthier, I., & Palmeri, T. J. (2012). Not all morph spaces stretch alike: How category learning affects object perception. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38(4), 807–820. Folstein, J., Palmeri, T. J., & Gauthier, I. (2013). Category learning increases discriminability of relevant object dimensions in visual cortex. Cerebral Cortex, 23(4), 814–823. Garner, W. R. (1974). The processing of information and structure. Potomac, Md.: Earlbaum. Gauthier, I., & Palmeri, T. J. (2002). Visual neurons: Categorization-based selectivity. Current Biology, 12, R282– 284. Gillund, G., & Shiffrin, R. M. (1984). A retrieval model for both recognition and recall. Psychological Review, 91, 1–65. Grill-Spector, K., & Kanwisher, N. (2005). Visual recognition: As soon as you know it is there, you know what it is. Psychological Science, 16, 152–160. Gureckis, T. M., James, T. W., & Nosofsky, R. M. (2011). Re-evaluating dissociations between implicit and explicit
an exemplar-based random-walk model
161
category learning: An event-related fMRI study. Journal of Cognitive Neuroscience, 23, 1697–1709. Hintzman, D. L. (1986). “Schema abstraction” in a multipletrace memory model. Psychological Review, 93, 411–428. Hintzman, D. L. (1988). Judgments of frequency and recognition memory in a multiple-trace memory model. Psychological Review, 95, 528–551. Jolicoeur, P., Gluck, M. A., & Kosslyn, S. M. (1984). Pictures and names: Making the connection. Cognitive Psychology, 16, 243–275. Johansen, M. K., & Palmeri, T. J. (2002). Are there representational shifts during category learning? Cognitive Psychology, 45, 482–553. Kahana, M. J., & Sekuler, R. (2002). Recognizing spatial patterns: A noisy exemplar approach. Vision Research, 42, 2177–2192. Knowlton, B., & Squire, L. (1993). The learning of categories: Parallel brain systems for item memory and category knowledge. Science, 262 (5140), 1747–1749. Lafond, D., Lacouture, Y., & Cohen, A. L. (2009). Decision tree models of categorization response times, choice proportions, and typicality judgments. Psychological Review, 116, 833– 855. Lamberts, K. (2000). Information accumulation theory of categorization. Psychological Review, 107, 227–260. Link, S. W. (1992). The wave theory of difference and similarity. Hillsdale, NJ: Earlbaum. Little, D. R., Nosofsky, R. M., & Denton, S. E. (2011). Response -time tests of logical-rule models of categorization. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37, 1–27. Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95, 492–527. Logan, G. D. (1997). The CODE theory of visual attention: An integration of space-based and object-based attention. Psychological Review, 103, 603–649. Mack, M., Gauthier, I., Sadr, J., & Palmeri, T. J. (2008). Object detection and basic-level categorization: Sometimes you know it is there before you know what it is. Psychonomic Bulletin & Review, 15(1), 28–35. Mack, M. L., Preston, A. R., & Love, B. C. (2013). Decoding the brain’s algorithm for categorization from its neural implementation. Current Biology, 23(20), 2023–2027. Mack, M. L., & Palmeri, T. J. (2010). Decoupling object detection and categorization. Journal of Experimental Psychology: Human Perception and Performance, 36 (6), 1067-1-79. Mack, M. L., & Palmeri, T. J. (2011). The timing of visual object categorization. Frontier in Psychology, 2:165. Mack, M. L., Wong, A. C.-N., Gauthier, I., Tanaka, J. W., & Palmeri, T. J. (2009). Time-course of visual object categorization: Fastest does not necessarily mean first. Vision Research, 49, 1961–1968. Mack, M. L., Wong, A. C.-N., Gauthier, I., Tanaka, J. W., & Palmeri, T. J. (2007). Unraveling the time-course of perceptual categorization: Does fastest mean first? In the Proceedings of the Twenty-Ninth Annual Meeting of the Cognitive Science Society. Marley, A. A. J. (1992). Developing and characterizing multidimensional Thurstone and Luce models for identification and preference. In F. G. Ashby (Ed.), Multidimensional models
162
basic cognitive skills
of perception and cognition (pp. 299–333). Hillsdale, NJ: Earlbaum. McElree, B., & Dosher, B. A. (1989). Serial position and set size in short-term memory: The time course of recognition. Journal of Experimental Psychology: General, 118, 346–373. McKinley, S. C., & Nosofsky, R. M. (1995). Investigations of exemplar and decision bound models in large, ill-defined category structures. Journal of Experimental Psychology: Human Perception and Performance, 21(1), 128. Medin, D. L., & Schaffer, M. M. (1978). Context theory of classification learning. Psychological Review, 85, 207–238. Minda, J. P., & Smith, J. D. (2001). Prototypes in category learning: The effects of category size, category structure, and stimulus complexity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 775–799. Monsell, S. (1978). Recency, immediate recognition memory, and reaction time. Cognitive Psychology, 10, 465–501. Murdock, B. B., Jr. (1982). A theory for the storage and retrieval of item and associative information. Psychological Review, 89, 609-626. Nosofsky, R. M. (1984). Choice, similarity, and the context theory of classification. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 104–114. Nosofsky, R. M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39–57. Nosofsky, R. M. (1987). Attention and learning processes in the identification and categorization of integral stimuli. Journal of Experimental Psychology: Learning, Memory, & Cognition, 13, 87–109. Nosofsky, R. M. (1988). Exemplar-based accounts of relations between classification, recognition, and typicality. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 700–708. Nosofsky, R. M. (1991). Tests of an exemplar model for relating perceptual classification and recognition memory. Journal of Experimental Psychology: Human Perception and Performance, 17, 3–27. Nosofsky, R. M. (1992). Similarity scaling and cognitive process models. Annual Review of Psychology, 43, 25–53. Nosofsky, R. M. (2000). Exemplar representation without generalization? Comment on Smith and Minda’s (2000) “Thirty categorization results in search of a model”. Journal of Experimental Psychology: Learning, Memory and Cognition, 26, 1735–1743. Nosofsky, R. M., & Alfonso-Reese, L. A. (1999). Effects of similarity and practice on speeded classification response times and accuracies: Further tests of an exemplar-retrieval model. Memory & Cognition, 27(1), 78–93. Nosofsky, R. M., Denton, S. E., Zaki, S. R., MurphyKnudsen, A. F., & Unverzagt, F. W. (2012). Studies of implicit prototype extraction in patients with mild cognitive impairment and early Alzheimer’s disease. Journal of Experimental Psychology: Learning, Memory, and Cognition, 38, 860–880. Nosofsky, R. M., Little, D. R., Donkin, C., & Fific, M. (2011). Short-term memory scanning viewed as exemplar-based categorization. Psychological Review, 118, 280–315.
Nosofsky, R. M., Little, D. R., & James, T. W. (2012). Activation in the neural network responsible for categorization and recognition reflects parameter changes. Proceedings of the National Academy of Sciences, 109, 333–338. Nosofsky, R. M., & Palmeri, T. J. (1997a). An exemplar-based random walk model of speeded classification. Psychological Review, 104, 266–300. Nosofsky, R. M., & Palmeri, T. J. (1997b). Comparing exemplar-retrieval and decision-bound models of speeded perceptual classification. Perception & Psychophysics, 59, 1027–1048. Nosofsky, R. M., Palmeri, T. J., & McKinley, S. C. (1994). Ruleplus-exception model of classification learning. Psychological Review, 101, 53–79. Nosofsky, R. M., & Stanton, R. D. (2005). Speeded classification in a probabilistic category structure: Contrasting exemplar-retrieval, decision-boundary, and prototype models. Journal of Experimental Psychology: Human Perception and Performance, 31, 608–629. Nosofsky, R. M., & Stanton, R. D. (2006). Speeded oldnew recognition of multidimensional perceptual stimuli: Modeling performance at the individual-participant and individual-item levels. Journal of Experimental Psychology: Human Perception and Performance, 32, 314–334. Nosofsky, R. M., & Zaki, S. R. (1998). Dissociations between categorization and recognition in amnesic and normal individuals: An exemplar-based interpretation. Psychological Science, 9(4), 247–255. Nosofsky, R. M., & Zaki, S. R. (2002). Exemplar and prototype models revisited: Response strategies, selective attention, and stimulus generalization. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28, 924–940. Nosofsky, R. M., & Zaki, S. R. (2003). A hybrid-similarity exemplar model for predicting distinctiveness effects in perceptual old-new recognition memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 1194– 1209. Palmeri, T. J. (1997). Exemplar similarity and the development of automaticity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 324–354. Palmeri, T. J. (1999). Theories of automaticity and the power law of practice. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 543–551. Palmeri, T. J. (2014). An exemplar of model-based cognitive neuroscience. Trends in Cognitive Sciences, 18(2), 67–69. Palmeri, T. J., & Cottrell, G. (2009). Modeling perceptual expertise. In D. Bub, M. Tarr, & I Gauthier (Eds.), Perceptual expertise: bridging brain and behavior. Oxford University Press. Palmeri, T. J., & Flanery, M. A. (1999). Learning about categories in the absence of training. Psychological Science, 10(6), 526–530. Palmeri, T. J., & Flanery, M. A. (2002). Memory systems and perceptual categorization. In B. H. Ross (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 41, pp. 141–189). San Diego: Academic. Palmeri, T. J., Schall, J. D. & Logan, G. D. (2014). Neurocognitive modeling of perceptual decision making. In J. R. Busemeyer, Z. Wang, J. T. Townsend, & A. Eidels
(Eds.), Oxford handbook of computational and mathematical Psychology. Oxford University Press. Palmeri, T. J., Wong, A. C.-N., & Gauthier, I. (2004). Computational approaches to the development of perceptual expertise. Trends in Cognitive Science, 8, 378–386. Posner, M. I., & Keele, S. W. (1968). On the genesis of abstract ideas. Journal of Experimental Psychology, 77, 353–363. Purcell, B. A., Schall, J. D., Logan, G. D., & Palmeri, T. J. (2012). From salience to saccades: multiple-alternative gated stochastic accumulator model of visual search. Journal of Neuroscience, 32(10), 3433–3446. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108. Ratcliff, R., Van Zandt, T., & McKoon, G. (1999). Connectionist and diffusion models of reaction time. Psychological Review, 106, 261–300. Reber, P. J., Gitelman, D. R., Parrish, T. B., & Mesulam, M. M. (2003). Dissociating explicit and implicit category knowledge with fMRI. Journal of Cognitive Neuroscience, 15(4), 574–583. Reber, P. J., Stark, C. E. L., & Squire, L. R. (1998a). Cortical areas supporting category learning identified using fMRI. Proceedings of the National Academy of Sciences, 95, 747–750. Reber, P. J., Stark, C. E. L., & Squire, L. R. (1998b). Contrasting cortical activity associated with category memory and recognition memory. Learning and Memory, 5, 420–428. Richler, J. J., & Palmeri, T. J. (2014). Visual category learning. Wiley Interdisciplinary Reviews in Cognitive Science, 5, 75–94. Rosch, E. (1978). Principles of categorization. In E. Rosch & B. B. Lloyd (Eds.), Cognition and categorization. Hillsdale, NJ: Erlbaum. Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., & Boyes-Braem, P. (1976). Basic objects in natural categories. Cognitive Psychology, 8, 382–439. Rouder, J. N., & Ratcliff, R. (2004). Comparing categorization models. Journal of Experimental Psychology: General, 133, 63–82. Shepard, R. N. (1957). Stimulus and response generalization: A stochastic model relating generalization to distance in psychological space. Psychometrika, 22, 325–345. Shepard, R. N. (1987). Toward a universal law of generalization for psychological science. Science, 237, 1317–1323. Shin, H. J., & Nosofsky, R. M. (1992). Similarity scaling studies of dot pattern classification and recognition. Journal of Experimental Psychology: General, 121, 278–304. Sigala, N., & Logothetis, N. K. (2002). Visual categorization shapes feature selectivity in the primate temporal cortex. Nature, 415(6869), 318–320. Smith, E. E. (2008). The case for implicit category learning. Cognitive, Affective, and Behavioral Neuroscience, 8, 3–16. Smith, E. E., & Medin, D. L. (1981). Categories and concepts. Cambridge, MA: Harvard University Press. Smith, J. D. (2002). Exemplar theory’s predicted typicality gradient can be tested and disconfirmed. Psychological Science, 13, 437–442. Smith, J. D., & Minda, J. P. (1998). Prototypes in the mist: The early epochs of category learning. Journal of
an exemplar-based random-walk model
163
Experimental Psychology: Learning, Memory and Cognition, 24, 1411–1436. Smith, J. D., & Minda, J. P. (2000). Thirty categorization results in search of a model. Journal of Experimental Psychology: Learning, Memory and Cognition, 26, 3–27. Sternberg, S. (1966). High speed scanning in human memory. Science, 153, 652–654. Tanaka, J. W., & Taylor, M. (1991). Object categories and expertise: is the basic level in the eye of the beholder? Cognitive Psychology, 23, 457–482 Verguts, T., Storms, G., & Tuerlinckx, F. (2003). Decisionbound theory and the influence of familiarity. Psychonomic Bulletin & Review, 10, 141–148. Wickelgren, W. A. (1974). Single-trace fragility theory of memory dynamics. Memory & Cognition, 2, 775–780. Wills, A. J., & Pothos, E. M. (2012). On the adequacy of current empirical evaluations of formal models of categorization. Psychological Bulletin, 138, 102–125.
164
basic cognitive skills
Wixted, J. T. (2004). On common ground: Jost’s (1897) law of forgetting and Ribot’s (1881) law of retrograde amnesia. Psychological Review, 111, 864–879. Wixted, J. T., & Ebbesen, E. B. (1991). On the form of forgetting. Psychological Science, 2, 409–415. Zaki, S. R., & Nosofsky, R. M. (2007). A high-distortion enhancement effect in the prototype-learning paradigm: Dramatic effects of category learning during test. Memory & Cognition, 35, 2088–2096. Zaki, S. R., Nosofsky, R. M., Jessup, N. M., & Unverzagt, F. W. (2003). Categorization and recognition performance of a memory-impaired group: Evidence for single-system models. Journal of the International Neuropsychological Society, 9(3), 394–406. Zandbelt, B. B., Purcell, B. A., Palmeri, T. J., Logan, G. D., & Schall, J. D. (2014). Response times from ensembles of accumulators. Proceedings of the National Academy of Sciences, 111(7), 2848–2853.
CHAPTER
8
Models of Episodic Memory
Amy H. Criss and Marc W. Howard
Abstract Episodic memory refers to memory for specific episodes from one’s life, such as working in the garden yesterday afternoon while enjoying the warm sun and chirping birds. In the laboratory, the study of episodic memory has been dominated by two tasks: single item recognition and recall. In single item recognition, participants are simply presented a cue and asked if they remember it appearing during the event in question (e.g., a specific flower from the garden) and in free recall they are asked to generate all aspects of the event. Models of episodic memory have focused on describing detailed patterns of performance in these and other laboratory tasks believed to be sensitive to episodic memory. This chapter reviews models with a focus on models of recognition with a specific emphasis on REM (Shiffrin & Steyvers, 1997) and models of recall with a focus on TCM (Howard & Kahana, 2002). We conclude that the current state of affairs, with no unified model of multiple memory tasks, is unsatisfactory and offer suggestions for addressing this gap. Key Words: episodic memory, computational modeling, free recall, recognition
Tulving (1972, 1983, 2002) coined the term episodic memory to refer to the ability to vividly remember specific episodes from one’s life. Episodic memory is often framed in contrast to other forms of memory which are not accompanied by the same experience. For instance, asking a subject “what did you have for breakfast?” usually elicits an episodic memory. In the process of answering the question, subjects will sometimes report reexperiencing the event as if they were present. They might remember being in their kitchen, with the morning sun shining in and the sound of the radio, the smell of the coffee in the process of retrieving the information that they had a bagel for breakfast. Other times, subjects may not have memory for all associated details, but instead have a fuzzy general memory for the event. Both vivid and fuzzy episodic memories are situated in a particular spatio-temporal context. The nature of these two types of episodic experiences is under
debate (see Box 1). In contrast, subjects can frequently answer factual questions, such as “what is the capital of France?” without any knowledge about the specific moment when they learned that piece of information. The association of a memory with a specific spatial and temporal context is considered the hallmark of episodic memory. This chapter is about mathematical and computational models of episodic memory. This is something of an unusual topic. One could argue that there are no mathematical models of episodic memory as defined here. To date, there are no quantitative models that have attempted to describe how or why the distinctive internal experience associated with episodic memory sometimes takes place and sometimes does not. In contrast, models of episodic memory have focused on describing detailed patterns of performance in a set of laboratory memory tasks believed to be sensitive 165
Box 1 Dual process models of recognition You go to the grocery store and pass many other shoppers. You pass one shopper who seems familiar. You consider saying hello, but you can’t quite figure out how you know this person. If you were asked to perform an item recognition test (“have you seen this person before?”), you would have been much more likely to say yes for this shopper than for one of the other shoppers. Perhaps you would even express high confidence that you had seen the familiar shopper before. After thinking about it for a while, you might later remember that you met this person at a meeting last semester. You might remember his or her name, position and even details of your interaction and be able to report these pieces of information. The ability to distinguish this one familiar face from all the other unfamiliar faces you passed in the grocery store certainly requires some form of memory. Similarly, the ability to remember the details about your experience with that person also requires some form of memory. The question is whether those two abilities are best understood as points along a continuum or as distinct forms of memory, typically referred to as familiarity—a general sense of knowing that a probe is old—and recollection—the vivid recall of specific details about the probe. This question has been a major source of disagreement. It has been actively pursued in mathematical modeling (e.g., Klauer & Kellen, 2010; DeCarlo, 2003), behavioral studies (e.g., Hintzman & Curran, 1994; Rotella, Macmillan, Reeder, & Wong, 2005), and a wide variety of cognitive neuroscience techniques (Fortin, Agster, & Eichenbaum, 2002; Rugg & Curran, 2007; Staresina, Fell Dunn. Axmacher, & Henson, 2013; Wilding, Doyle, & Rugg, 1995; Wixted & Squire, 2011). A major problem leading to this debate has been difficulty in extracting satisfactory measures of these two putative latent processes using observable data. Although signal detection approaches have been popular (Wixted, 2007; Yonelinas & Parks, 2007) in solving this problem, signal detection is by no means definitive (Malmberg, 2002; Province & Rouder, 2012).
166
basic cognitive skills
to episodic memory. This is extremely important because the experimenter must have control over the stimuli the participant has experienced in order to evaluate the success or failure of memory retrieval. In the laboratory, the study of episodic memory has been dominated by two tasks: single item recognition and recall. In both recognition and recall tasks, subjects are typically presented with a series of stimuli—a list—and then tested on their memory for that experience later. In recognition, participants discriminate between studied targets and unstudied foils. In recall, participants are asked to generate some member of the stimulus set. There exist successful process models for both individual tasks but no single model that captures a wide range of theoretically and empirically important data in both tasks. In part, this is because a number of variables differentially affect performance in these two tasks, and in part because the methodological details in the two domains often vary in ways that preclude direct comparison. The division of this chapter into recognition and recall sections reflects the fact that efforts to provide a common modeling framework have not been successful thus far. This is a major gap in our understanding—a unified model of episodic memory that provides a quantitative description of the data from the various paradigms would be much preferable. In this chapter, we first present an overview of the important data and models in each area.
Models of Recognition Memory Recognition memory tests are among the most widely used experimental paradigms for the study of episodic memory. Here, a to-be-remembered event is created, typically in the form of a list of individually presented items (words, pictures, etc). After a delay ranging from a few seconds to 7 or more days, memory is tested by intermixing the studied items (from the to-be-remembered list) along with foils that did not occur during the study episode. In forced choice recognition, a target item is presented alongside a foil or foils and participants are instructed to select the target. In single item recognition, test items are presented one by one and the participant is asked to endorse studied items and reject foil items. Multiple measures describe performance: a hit is correctly endorsing a studied item, a correct rejection is correctly identifying a foil, a false alarm is incorrectly endorsing a foil as having been
studied, and a miss is incorrectly rejecting a studied item. Although measuring accuracy in recognition memory is itself an active topic of research, in general, the larger the difference between the hit rate (proportion of hits to old probes) and the false alarm rate (the proportion of false alarms to new probes), the more accurate is episodic memory. In forced choice, the measurement of performance is simply percent correct. Based on these and other measures, such as the time for providing a response or the confidence associated with the response, a number of detailed mathematical models have been developed to explain episodic memory.
Global matching models The global matching models were successful for decades (Gillund & Shiffrin, 1984; Hintzman, 1984; Humphreys, Bain, & Pike, 1989; Murdock, 1982). The premise of these models was that the search of episodic memory included a comparison to a relevant set of items and the memory decision was based on the overall or global match between the probe and the set to which it was compared. the matched filter model The basic idea of global matching models is concisely illustrated by Anderson’s matched filter model (Anderson, 1973). Let the list be composed of N unique items represented as vectors fi , where the subscript denotes which of the N vectors is referred to. Let the vectors be randomly chosen Gaussian deviates such that ! (1) E fi T fj = δij (where δij = 1 if i = j and δij = 0 otherwise) and the similarity of the vectors to have variance given by ! Var fi T fj = σ 2 . We will treat σ as a free parameter but, in general, it will be specfied by the distribution of the features of fi . Now, in the matched filter model, the list is represented simply as the sum of the list items. At each time step, the sum mi is updated according to mi = mi−1 + fi ,
(2)
such that at the end of the list mn =
N
fi .
(3)
i=1
To model recognition, we take the vector corresponding to a probe and match it against the list.
Denoting the vector corresponding to the probe stimulus as fp we have fp T fi . (4) Dp = fp T mn = i
Notice that this decision variable has different distributions depending on whether the probe was on the list. If the probe is old, then fp should match one of the list items, resulting in " 1 old E Dp = . (5) 0 new Moreover, the variance of Dp is a function of the length of the list Var Dp = N σ 2 . (6) The matched filter model is an example of global matching model. It is a matching model because the similarity of the probe stimulus to the contents of memory is calculated and drives the decision process. It is a global matching model because the match is calculated not only to information stored about the probe stimulus during study; rather the match from other study items also contributes to the decision. This concept is perhaps best understood in contrast to direct access models (Dennis & Humphreys, 2001; Glanzer & Adams, 1990) that posit a direct comparison of the test item to its corresponding memory trace. In global matching models, errors in memory result from similarity between the test item and memory traces with similar features. More formally, the probability that Dp is greater than some criterion is higher for old probes than for new probes; for a fixed criterion c, P Dp > c|old > P Dp > c|new . However, in order to correspond to performance in the task (which is typically far from perfect) the criterion (and the variances) must be chosen such that the match from some new probes exceeds the criterion. According to the matched filter model, and global matching models more generally, this happens because a particular new probe happens to match well with the study list. We can generate several other predictions from global matching models from these expressions as well. First, Eq. 6 tells us that the the discriminability of the decision goes down with σ and with the length of the list N . If the vectors are chosen such that the similarity is a normal deviate, then the discriminability between the old and the new distributions is d = √ 1 . This makes a straightforward Nσ experimental prediction—that accuracy should go models of episodic memory
167
down as the length of the study list increases. This finding is typically observed in recognition memory experiments (Criss & Shiffrin, 2004; Ohrt & Gronlund, 1999; Shiffrin, Huber, & Marinelli, 1995 but see (21)). the demise of global matching models Two observations contributed to the demise of the global matching models for recognition memory: the generality of the mirror effect and the null list strength effect. The mirror effect refers to the finding that when the hit rate is higher for a particular experimental variable, the false alarm rate is lower. This is a challenge for global matching models because the strength of target items appears to leapfrog the foil items. Suppose we perform an experiment in which we observe some hit rate and false alarm rate. Now, we change the experiment such that each studied item is presented five times rather than just once. Now, the mean of Dp for the old probes will be 5 rather than 1 but the mean of Dp for the new probes will still be zero. The variance of the distributions should also increase, to 5N σ 2 . If the criterion c is fixed across experiments, we would expect the hit rate, P(Dp > c|old) to be higher for the list with the repeated stimuli. However, the false alarm rate should also increase, in contrast to the experimental results. In order to account for the mirror effect in the context of the matched filter model, we would have to assume that the criterion, C, changes across experiments, which is akin to unprincipled curve-fitting. The mirror effect is quite general and is observed for a wide variety of experimental variables including repetition, changes in presentation time, and word frequency (e.g., Glanzer & Adams, 1985, 1990). Most global matching models did not attempt to account for the mirror effect at all and those that did relied on a changing criterion (Gillund & Shiffrin, 1984), which is largely inconsistent with the empirical data (e.g., Glanzer, Adams, Iverson, & Kim, 1993). The null list strength effect posed a more fundamental challenge to global matching models. The null list strength effect is the finding that the strength of other study items does not affect recognition memory accuracy (Shiffrin, Ratcliff, Clark, 1990; Ratcliff, Clark & Shiffrin, 1990). To make this more concrete, consider two experiments. In one experiment, as before, we present all the items five times. We would expect accuracy to be much higher in this pure strong condition relative to the pure weak condition, where all the items are 168
basic cognitive skills
presented only once. But now consider a mixed list experiment in which half the list items are presented five times and half are presented only once. We refer to probes presented five times during study in the mixed list as mixed-strong probes and the probes presented only once as mixed-weak probes. Note that the mean of Dp for the pure-strong and mixed-strong probes should be identical. However, the variance should not be the same. The mixedstrong probes should be subject to less noise from the weak items in the mixed list and should thus have higher accuracy than the pure-strong probes. Following similar logic, we would expect accuracy to be lower for the mixed-weak probes than for the pure-weak probes due to additional interference from the strong items on the study list. In contrast to this very strong prediction, this pattern of results does not hold (Ratcliff et al., 1990; Shiffrin et al., 1990), reflecting a fundamental problem for global matching models.
The Retrieving Effectively from Memory (REM) Model The pervasiveness of the mirror effect and the discovery of the null list strength effect created a paradigm shift wherein a new set of models incorporating Bayesian principles were developed to account for recognition memory. The REM model (Shiffrin & Steyvers, 1997) is the most thoroughly explored of these approaches and we will focus on it extensively here. As in the global matching models, a probe is compared to each of the traces in memory. There are two key insights that allow REM to overcome the weaknesses of the global matching models. First, the comparison between the probe and the contents of memory incorporates both positive evidence of a match but also negative evidence for nonmatch. That is, rather than simply the absence of evidence, REM can incorporate positive evidence for absence. This is a powerful assumption. As a stimulus is studied more extensively, it means that this stimulus can provide both more positive evidence that the trace matches an old probe, but also more negative evidence that it doesn’t match a new probe. This provides a mechanism for altering the new item distribution rather than assuming that encoding is restricted to altering the target distribution. Second, the decision rule takes into account the nature of the environment and expected memory evidence based on that prior knowledge. REM is a Bayesian model with the core assumption that processes underlying memory are optimal
given noisy information on which to base a decision. There are two types of memory traces in REM: lexical-semantic and episodic. Lexical-semantic traces are accumulated across the lifespan and are thus complete, accurate, and de-contextualized relative to episodic traces. Episodic traces are formed during a given episode and are updated with item, context, and sometimes associative features during each presentation in a given context. REM is a simulation model wherein a set of episodic and lexical-semantic traces are generated for each simulated subject as described next. representation A memory trace consists of multiple types of information. Item features represent a broad range of information about the stimulus including the meaning of the stimulus and orthographicphonological units. Context represents the internal and external environment at the time of encoding. Associative features are sometimes generated and represent information relating multiple items (e.g., a stimulus-specific association formed during or prior to the experiment). All features are drawn from a geometric distribution with parameter g. The probability that a feature takes the value ν is ν−1 . (7) P(ν) = g 1 − g A geometric distribution assures that some features will be relatively common and others will be relatively rare. Evidence provided by a matching feature is a function of the base rate of that feature: matching a common feature provides less evidence than matching a rare feature. storage During each experience with a stimulus, the lexical-semantic trace for a stimulus is retrieved from memory and updated with the current context features. In a typical recognition memory experiment the lexical-semantic traces are used solely for the purposes of generating and testing episodic traces. Therefore, the theoretical principle of updating lexical-semantic traces with current context is typically not implemented in a simulation (c.f., Schooler, Shiffrin, & Raaijmakers, 2001). An episodic memory trace is formed by storing each lexical-semantic feature and the context feature with ∗ some probability (u) per unit of time (t). Given that a feature is stored, the correct value is stored with some probability (c). Otherwise, a random value from the geometric distribution is stored. Features that are not stored during encoding are denoted
by a zero indicating a lack of information. Thus, episodic memory is incomplete (i.e., some features are not stored), prone to error (i.e., an incorrect feature value may be stored), and context-bound (i.e., contains a set of features representing the context). Study of a pair results in the concatenation of the two sets of item features and shared context features. Depending on the goals at encoding, associative features that capture relationships between the two stimuli may also be encoded in the vector (e.g., Criss & Shiffrin, 2004, 2005). Processes at retrieval are necessarily different for different tasks as they depend on the information provided as a cue and the required output. Next we only consider recognition memory, but note that REM has been applied to multiple memory tasks including judgments of frequency (Malmberg, Holden, & Shiffrin, 2004), free recall (Malmberg & Shiffrin, 2005), cued recall (Diller, Nobel, & Shiffrin, 2001), and associative recognition (Criss & Shiffrin, 2004, 2005).
retrieval According to REM, there are an immense number of traces that have been laid down over an extremely long time. In order to restrict the comparison to the relevant event, reinstated context features identifying the context to be searched are used to define the activated set. In a typical experiment with a single study list, this step simply limits the comparisons of the test cue to the study list and is often implemented by assumption for simplicity, which we assume here. The basis for a memory decision in REM is the likelihood computation. The likelihood reflects evidence in favor of the test cue as the ratio of the probability that the cue matches a trace in memory given the data compared to the probability that the cue does not match an item in memory given the data. Here, data refer to the match between the cue and the contents of the activated subset of episodic memory. The item features from test cue j are retrieved from its lexical-semantic trace and compared to each item in the activated set, indexed by i. A likelihood ratio, indicating how well the test cue j matches memory trace i is computed using λij = (1 − c)nq
ν−1 nm ∞ c + (1 − c) gsys 1 − gsys . (8) ν−1 gsys 1 − gsys ν=1
models of episodic memory
169
The gsys parameter is the long-run base rate. This is a fixed value, estimated by the system based on experience. This base rate value may differ from the g values in Eq. 7, which gives the value of the g parameter for the stimulus itself. The number of nonzero features that mismatch is nq and the number of features that match and have the ν is nm . Missing features (value of zero) are ignored. Note that the amount of evidence provided by a matching feature depends on the feature value. This is one way in which prior knowledge contributes to the decision. Specifically, in a geometric distribution, low values are common and, therefore are likely to match by chance. These values provide little evidence when they match. In contrast, large values are uncommon and, therefore unlikely to match by chance, providing greater evidence when they match. Thus, knowledge about the statistics of the environment (i.e., rarity of features, estimated by gsys ) learned over the course of life contributes to the evidence of match between a test cue and the contents of each memory trace. For single item recognition the decision that test cue j was present during the relevant context is based on the average of the likelihood ratios. If the average exceeds a criterion, the item is endorsed as from the list, otherwise it is rejected.
Word Frequency and Null List Strength Effects in REM REM can provide an account of the mirror effect based on properties of the words themselves, such as word frequency. The probability of a given feature value (g) and the expectation of feature values (gsys ) are both specified in REM. REM has used the g parameter to model the effects of normative word frequency. Specifically, low frequency words (LF) are assumed to have more uncommon features (i.e., a lower value of g is used to generate the stimuli). In contrast, high frequency (HF) words have relatively common features. The common features of HF words tend to match other features of memory traces by chance increasing the false alarm rate. Additionally, the likelihood ratio includes prior information by taking into account the base rate of features (gsys ) such that matching unexpected features contribute more evidence in favor of endorsing the test, increasing the hit rate for LF words. Together, the stimulus representation (g) and prior expectations (gsys ) generate a word frequency mirror effect, consistent with empirical data. 170
basic cognitive skills
The null list strength account in REM is based on differentiation. The Subjective Likelihood Model (SLiM) of McClelland and Chappell (1998) shares the mechanism of differentiation and for that reason also predicts a null list strength effect for recognition. Differentiation refers to the idea that the more that is known about an item, the less confusable that item is with any other randomly chosen item. Obvious applications are a bird expert or a radiologist, who have such knowledge in their area of expertise that they can quickly and accurately identify a Rusty Blackbird or a cancerous tumor, whereas a novice simply sees a bird or a blurry grayscale image. In episodic memory, an item becomes differentiated by being well practiced within a specific contextual episode, for example by repetition in an experiment. Within the differentiation models, an episodic memory trace is updated during repetition, which causes the memory trace to be more accurate and more complete. Note that updating was a departure from the standard assumption of storing additional exemplars or additional memory traces with repetition (e.g., Hintzman, 1986). In REM, differentiation is implemented by assuming that if an item is recognized as having been previously experienced in a given context, then the best matching trace is updated such that any missing (zero valued) features have the potential of being replaced in accordance with the encoding mechanism described earlier. If an item is not recognized as a repetition, then a new memory trace is stored. In the original REM model, updating only occurred during study for simplicity. However, more recent applications incorporate updating and encoding at test (Criss, Malmberg, & Shiffrin, 2010). When a memory trace is stored with higher quality, that is when more features are stored, not only is it a better match to a later comparison with the lexical-semantic trace from which it was generated, but it is also a poorer match to other test items. In Eq. 8, note that the matching and mismatching features contribute to the overall evidence in favor of the test item. As the total number of stored features in a given memory trace increases, due to additional encoding, so too does the total number of features matching the target trace. Critically, the total number of mismatching features for any item other than the corresponding target trace also increases. The net result is that although a strengthened target item matches better and will be better remembered, it is not at the cost of the other studied items. Differentiation models
correctly predict that recognition memory is not harmed by increasing the strength of the other items on the study list. In fact, in some cases, the models predict a small negative list strength effect such that, for a given item, memory may slightly improve as the strength of other studied items increases (see Shiffrin et al., 1990). In summary the differentiation models were developed to address shortcomings of the global matching models, in particular their failure to capture the robust empirical findings of the word frequency (WF) mirror effect and the null-list strength effect. In REM, the WF effect is due to the assumed distribution of features along with a Bayesian decision rule that gives more weight to unexpected matches or alternatively downplays expected matches. The null-list strength effect is due to differentiation of well-learned items caused by updating memory traces.
The Empirical Consequences of Updating The updating mechanism in REM was necessary to produce differentiation and to account for empirical data. Auspiciously, this same mechanism makes critical a priori predictions that appeared in the literature after the model was conceived. First, updating memory traces during the encoding of test items results in output interference. Output interference (OI) is the finding that memory accuracy decreases over the course of testing (Murdock & Anderson, 1975; Roediger & Schmidt, 1980; Tulving & Arbuckle, 1963, 1966; Wickens, Born, & Allen, 1963). Output interference is not a new finding, but a detailed understanding of the manifestation of OI in recognition memory is (Criss, Malmberg, & Shiffrin, 2011; Malmberg, Criss, Gangwani, & Shiffrin, 2012). Figure 8.1 shows a typical pattern of OI in recognition testing (left panel) along with predictions from REM. The middle panel shows predictions for REM where remembered items cause the best matching episodic trace to be updated, as described earlier. The right panel shows predictions for a version of REM where updating does not occur; instead, a new trace is added to memory for each test item. Both the predictions of REM with updating and the data show a dramatic decrease in the rate of endorsing target items as old as a function of test position and the nearly flat function for foils. In contrast, the multitrace version of REM in which additional traces are stored with each test item predicts a shallow decrease in the hit rate along with an increase in the false-alarm
rate. Both implementations of REM predict output interference in the sense that overall accuracy (e.g., dprime) decreases across test position. However, only the updating model predicts the precise pattern of observed data. A second prediction that follows directly from the differentiation mechanism is a higher hit rate and lower false alarm rate following a strongly encoded list compared to a weakly encoded list. This finding is called the strength-based mirror effect (SBME) and has been widely replicated (Cary & Reder, 2003; Criss, 2006, 2009, 2010; Glanzer & Adams, 1985; Starns, White, & Ratcliff, 2010; Stretch & Wixted, 1998). The WF mirror effect is related to the nature of the stimuli, whereas the SBME is related to the encoding conditions. Both findings co-occur (Criss, 2010; Stretch & Wixted, 1998) and despite the shared label of mirror effect, they result from entirely different mechanisms in REM. Increasing the strength with which a study list is encoded via levels of processing, repetition, or study time increases the number of stored features and produces a distribution of λ that is higher for strongly than weakly encoded targets. The same fact—that strongly encoded memory traces contain more information—produces a distribution of lower λ for foil items. Foils match a strongly encoded memory trace less well than a weakly encoded memory trace. Thus, a list containing all strongly encoded targets will match any given foil poorly, reducing the false alarm rate. Not only are the HR and FAR patterns that make up the SBME well predicted by REM, but REM makes additional specific predictions that have been confirmed with behavioral experiments (see Criss & Koop, in press for a review). For one, the actual distribution of estimated memory strength follows the pattern predicted by REM (Criss, 2009). Further, the interaction between target-foil similarity and encoding strength presents just as predicted by REM (Criss, 2006; Criss, Aue, & Kilic, 2014). If one conceives of the λ values as the driving force behind a random walk or diffusion model, then the rate at which the walk reaches a boundary is consistent with REM, that is, targets and foils following a strongly encoded list have a steeper approach (e.g., larger drift rate) to the decision bound (Criss, 2010; Criss, Wheeler, & McClelland, 2013). In summary, the global matching models were found to be inadequate on the basis of several findings, critically, the WF mirror effect and null-list strength effect. REM was developed to models of episodic memory
171
1.0 A
B
C
P(OLD)
0.8 0.6 0.4 0.2 foils targets 0.0 Test Block Fig. 8.1 Panel A shows data (Koop, Criss, & Malmberg, in press) that is representative of output interference. The panel gives the probability of endorsing old probes (targets) and new probes (foils) for several test blocks. The hit rate is P(OLD) for targets; the false-alarm rate is P(OLD) for foils. The data reveal a steep decline in the hit rate and flat false alarms across test position. Panel B shows the standard REM model where remembered items cause updating of an episodic memory trace. Updating produces patterns of data consistent with empirical findings. Panel C shows a version of REM where each test item causes the storage of a new memory trace (i.e., no updating). Such a model predicts a shallow decrease in the hit rate and increase in the false alarm rate, inconsistent with observed data.
account for these data and others. Two key features of REM are updating a single context-bound episodic trace and a Bayesian decision rule that takes into account positive and negative evidence from the stimulus as well as expectations based on the environment. These properties not only accounted for the problematic data but also lead to specific and fortuitous predictions. Differentiation, the same mechanism that was required to predict the data that lead to the demise of a whole class of models, predicted the observed pattern of output interference and strength-based mirror effects.
An Alternative Idea: Context-Noise Models Context-noise models (Dennis & Humphreys, 2001) were also developed to account for data problematic for the global matching models. However, they took a very different approach. Context-noise models assume that memory evidence is based on the similarity between the test context and the previous contexts in which the test item was encountered. There is no comparison between the test item and any other studied item, in fact no other items from episodic memory enter the decision process. Briefly, the model works as follows. Each time a word is encoded, the current context is bound to the word. During test, the item causes retrieval of 172
basic cognitive skills
its prior contexts. Those contexts are compared to the context in question, the test context in a typical recognition experiment. The recognition decision is made based on how well the retrieved contexts of the test item match the test context. It should be clear that context information is the only factor that contributes to memory evidence and, thus, such models predict no effects of list composition per se. Neither the number nor strength of the other studied items affect the decision because those items are never compared to the test item, thus the model easily predicts a null list strength effect. The WF mirror effect is predicted on the assumption that common words have more prior contexts that interfere with the ability to isolate the tested context. Although the context noise models are limited—for example, they only apply to recognition and must generate post-hoc explanations for many findings including output interference and the SBME— they certainly advanced the field by making context a touchstone for models of recognition. Unlike models of recall that emphasize context, models of recognition have largely neglected context. The item-context-noise debate sparked by the introduction of the Dennis & Humphreys model of has brought to the forefront the fact that context must be taken seriously in recognition models but also raised questions about the nature of context.
Models of Episodic Recall In recall tasks, subjects must report their memory for an event by producing a stimulus. Recall tasks most commonly use words as stimuli. In cued recall, subjects are given a probe stimulus as a cue to retrieve a particular word from the list. Most commonly, pairs of words are studied, with one member of the pair serving as a cue for recall of the other (but see Phillips, Shiffrin, & Atkinson, 1967; Nelson, McEvoy, & Schreiber, 1990; Tehan & Humphreys, 1996; for other possibiities). In free recall, subjects are presented with a list of words, typically one at a time and then asked to recall the words in the order they come to mind. In serial recall, subjects are asked to produce the stimuli in order, typically starting at the beginning of the list. Although there is a well-developed literature modeling serial recall, the serial recall task is most commonly described as a function of working memory rather than as a function of episodic memory,1 and we will not discuss it here. The fundamental question in episodic recall is to determine what constitutes the cue. In cued recall, this might seem like an obvious question. Given the pair DOG-QUEEN the subject is presented DOG as a cue at test and correctly recalls QUEEN. Is it not sufficient to understand this as a simple, almost Pavlovian, association between some distributed representation of the word DOG and the word QUEEN? This is not sufficient. If the subject is asked instead to recall the first word that comes to mind when hearing the word DOG it is likely the subject would recall CAT. If asked to recall a word that rhymes with dog the subject would likely recall LOG. If asked to remember a specific event from their life that involved a dog (or the word DOG ) it is unlikely that the subject would recall QUEEN. All these tasks take the same nominal cue but result in very different responses. From this we conclude that the cue stimulus itself is not enough to account for the subjects’ responses. In free recall the problem is even more acute; in free recall there is no external cue whatsoever. Free recall must proceed solely on the basis of some set of internal cues. Several concepts—fixed-list context, variable context, short-term memory, and temporally varying context—have been introduced to detailed models of recall tasks to attempt to solve these problems.
Cued recall In cued recall, subjects are presented with pairs of stimuli, such as ABSENCE -HOLLOW , PUPIL -RIVER , and so forth. At test, the subject
is given one of the stimuli, such as ABSENCE and asked to produce the corresponding member of the pair, i.e., say (or write) HOLLOW. One way to approach cued recall is to form an association between the stimuli composing a pair. We will see that this assumption is ultimately limited for recall more broadly, but illustrates several important properties of models of recall. For that reason, we will spend some time examining an extremely simple model of association in memory. simple linear association In the matched filter model we constructed a memory vector that was the sum of the vectors corresponding to the list items. In a linear associator, we again form a sum, but now of a set of outer product matrices. Each matrix provides the outer product of the first members of a pair with the second members of a pair. These associations can be understood as changing the synaptic weights between two vector spaces according to a Hebbian rule. Such associations between distinct items are referred to as heteroassociative. Here we follow the heteroassociative model of J. A. Anderson, Silverstein, Ritz, and Jones (1977). Let us refer to vector corresponding to the first member of the ith studied pair as fi and the second member as gi . Now, the matrix storing the associations between each stimulus and each response can be described by Mi = Mi−1 + fi gi T .
(9)
To model the association, we can probe the matrix with a probe stimulus, Mgp . Here we find this to be fi gi T gp . Mi gp = (10) i
That is, the output is a combination of the vector for each first member fi weighted by the degree to which the paired stimulus gi stimulus matches the probe vector. Equations 9 and 10 can be understood as describing a simple neural network connecting f and g, with M understood as a simple Hebbian matrix describing the connections between them (see Figure 8.2). In recall tasks, subjects report one word at a time, rather than a mixture of words. This can be reconciled with a deblurring mechanism in which one takes an ambiguous stimulus and perceives one of several possibilities. This is somewhat analogous to the problem of perception in which one identifies a particular stimulus from a blurry input. One can imagine a number of physical processes that could be used to accomplish this models of episodic memory
173
recall with a particular word and the probability of transitions between stimuli after recall has been initiated.
f M g Fig. 8.2 Schematic of a neural network interpretation of Eq. 9. Two sets of “neurons,” f and g are connected by a weight matrix M. As each pair is presented, the values of the elements in f and g are set to the values corresponding to the stimulus in that pair. The connections between an individual element in g and an individual element in f are strengthened according to the product of those two elements, that is, the corresponding element of the outer product of the two patterns.
deblurring (e.g., J. A. Anderson et al., 1977; Sederberg, Howard, & Kahana, 2008), but in many applications, researchers simply assume that the probability of recalling a particular word is some phenomenological function of its activation relative to the activation of competing words (e.g., Howard & Kahana, 2002a) or a sampling and recovery process (e.g., Raaijmakers & Shiffrin, 1980). A series of models can be developed that can be thought of as variations on the basic theme illustrated by Eq. 10. These models differ in the associative mechanism but are similar in that they store the associations between each of the members of the pairs and provide a noisy output when given a probe. TODAM utilizes a convolution/correlation associative mechanism rather than the outer product (e.g., Murdock 1982). The matrix model (Humphreys et al., 1989) utilizes a triple association between the two members of the pair and a fixedlist context vector to form a three tensor that can be probed with the conjunction of a probe item and a test context.
Free recall Free recall raises two computational problems that are probably central to the question of episodic memory. First, how do subjects initiate recall in the absence of a particular cue? Second, how do those cues change across the unfolding process of retrieval? Models of free recall have not focused on differences in associative mechanisms, nor detailed assumptions about how items are represented, but on representations that are used to initiate recall and how those representations change across retrieval attempts. These two problems can be concisely summarized by two classes of empirically observable phenomena: the probability that subjects initiate 174
basic cognitive skills
empirical properties of recall initiation in free recall The finding that subjects can direct their recall to a specific region of time has also had a large effect on hypotheses about the cue used to initiate free recall. Shiffrin (1970) gave subjects a series of lists of varying lengths for free recall. However, rather than having subjects recall the most recent list, he had subjects direct their recall to the list before the most recent list. Remarkably, the probability of recall depended on the length of the target list rather than the length of the intervening list. This suggested a representation of the list per se that was used to focus the subjects’ retrieval attempts.2 But the most dramatic effect in the initiation of free recall is the recency effect manifest in the probability of first recall. The probability of first recall gives the probability that the first word the subject recalls came from each of the positions within the list. When the test is immediate, this measure shows a dramatic advantage for the last items in the list (Figure 8.3a). This, combined with the fact that the recency effect is reduced when a delay intervenes between presentation of the last word and the test (Glanzer & Cunitz, 1966; Postman & Phillips, 1965) led many researchers to attribute the recency effect in immediate free recall to the presence of a short-term memory buffer (e.g., Atkinson & Shiffrin, 1968). However, the recency effect measured in the probability of first recall is present even when the time between the last item in the list and the test is much longer. Similarly, when subjects make errors in free recall by recalling a word from a previous list, the intrusions are more likely for recent lists (Zaromb et al., 2006). The debate among researchers modeling free recall in the last several years has focused on whether these longer-term recency effects depend on a different memory store than the short-term effects (Davelaar, Goshen-Gottstein, Ashkenazi, Haarmann, & Usher, 2005; Lehman & Malmberg, 2012) or if recency effects across time scales reflect a common retrieval mechanism (Sedcrberg et al., 2008; Shankar & Howard, 2012). context as a cue for recall To solve the problem of initiating recall researchers have appealed to a representation of “context,” some information that is not identical to
(b)
(a) Immediate Delayed Continuous Distractor
0.2 Probability of First Recall
Probability of First Recall
0.5 0.4 0.3 0.2
0.15
0.1
0.05
0.1 0
0 1 2 3 4 5 6 7 8 9 10 11 12
Serial Position
10
20 30 List number
40
Fig. 8.3 The recency effect in recall initiation across time scales. The probability of first recall gives the probability that the first word the subject free recalls came from each position within the list. a. In immediate free recall, the test comes immediately after presentation of the last list item. A dramatic recency effect results. In delayed free recall, a distractor task intervenes after presentation of the last word in the list. The recency effect is sharply attenuated. In continuous distractor free recall, a distractor is presented after the last item, but also between presentation of each item in the list. The recency effect in the probability of first recall recovers. Here the distractor interval was approximately 16 s. After Howard & Kahana (1999). b. Subjects studied and recalled 48 lists of words. At the end of the experiment, they recalled all the words they could remember from all lists. Probability of first recall is shown as a function of the list the word came from. Here the recency effect extends over a few hundred seconds. After Howard, Youker, & Venkatadass (2008). c a. After Howard & Kahana (1999), 2008, Elsevier; b. After Howard, Youker, & Venkatadass (2008),with kind permission from Springer Science and Business Media.
the stimuli composing the list but that nonetheless enables the subject to focus their memory search on a subset of the stimuli that could potentially be generated. Context can function as a cue for retrieval of items from the appropriate list if it is associated to the list stimuli during learning. We can think of a straightforward extension of Eq. 9: Mi = Mi−1 + fi ci T ,
(11)
where ci the state of the context vector at time step i. The “context” available at the time of test can be used as a probe for recall from memory (Figure 8.4). In much the same way that words in memory were activated to the extent that they were paired with the probe word in Eq. 10, the words in memory will be activated to the extent that the cue context resembles the state of context available when they were encoded. fi c i T c , (12) Mi c = i
where c is the context available at the time of test. We can see the power of proposing a context representation from Eq. 12: each studied item fi is activated to the extent that its encoding context ci overlaps with the probe context. Obviously, the choice of how context varies has a tremendous effect on the behavioral model that
results. One choice is to have the state of context be constant within a list but completely different from the state of context that obtains when one studies the next list. Models that exploit a fixed list context have been successful in describing many detailed aspects of free recall performance (Raaijmakers & Shiffrin, 1980, 1981). If there is a binary difference between the context of each list, then it is not possible to describe recency effects across lists (Glenberg, Bradley, Stevenson, Kraus, Tkachuk, & Gretz, 1980; Howard et al., 2008; Zaromb et al., 2006). Similarly, if the context is constant within a list, then fixed list context cannot be utilized to account for recency effects within the list. Another choice is to have context change gradually across lists (J. R. Anderson & Bower, 1972), or across time per se. Following Estes (1955), Mensink and Raaijmakers (1988) introduced a model of interference effects in paired associate learning where the state of context gradaully changed over intervals of time. The state of context at test is the cue for retrieval of words from the list. If the state changes gradually during presentation of the list items, then the state at the time of test will be a better cue for words from the end of the list than for words presented earlier. As a consequence, this produces a recency effect. This approach has been applied to describing the recency effect in free
models of episodic memory
175
(b)
(a) f
f
M
M
c
c
noise (c) f M
C IN
c
Fig. 8.4 Context as an explanatory concept. a. Stimuli f are associated to states of context c. Compare to Figure 8.2, Eqs. 9, 11. b. In contextual variability models, context changes gradually from moment to moment by integrating a source of external noise. See Eq. 13. c. In retrieved context models, the changes in context from moment to moment are caused by the input stimuli themselves. Repeating an item can also cause the recovery of a previous state of temporal context. See Eqs. 15, 16.
recall both within and across lists (Davelaar et al., 2005; Howard & Kahana, 1999; Sirotin, Kimball, & Kahana, 2005). If one can arrange for the state of context to change gradually over long periods of time, recency effects can be observed over similarly long periods of time. Murdock (1997) used a particularly tractable model of variable context that illustrates this idea. At time step i, the context representation is updated as # (13) ci = ρci−1 + 1 − ρ 2 ηi . where ηi is a vector of random features chosen at time step i (Figure 8.4b). The noise vectors are chosen such that the expectation value of the inner product of any two vectors is zero and the expectation value of the inner product of a noise vector with itself is 1. Now, it is easy to verify that the expectation of the inner product of states of context falls off exponentially: ! (14) E ci T cj = ρ |i−j| . Here we can see that the parameter ρ controls the rate at which context drifts in this formulation. As a consequence, Eq. 13, coupled with Eq. 12 results in an exponentially decaying activation for list items. empirical properties of recall transitions in free recall After the first recall is generated, transitions from one word to the next also show lawful properties. 176
basic cognitive skills
Broadly speaking, recall transitions show sensitivity both to the study context of the presented words as well as similarities between the words themselves. For instance, given that the subject has just recalled some word from the list, the next word recalled is more likely to be from a nearby position within the list than from a distant position within the list, showing a sensitivity to relationships induced by the study context. Similarly, the subject is more likely to recall a word from the list that is semantically related to the just-recalled word than to recall an unrelated word from the list, showing a sensitivity to the properties of the words themselves. The tendency to recall words from nearby serial positions in sequence is referred to as the contiguity effect (Kahana, 1996; Sederberg, Miller, Howard, & Kahana, 2010). The contiguity effect is manifest not only in free recall, but in a wide variety of other episodic memory tasks as well (see Kahana, Howard, & Polyn, 2008 for a review). Like the recency effect, the contiguity effect is also manifest across time scales as well (Howard & Kahana, 1999; Howard et al., 2008; Kiliç, Criss, & Howard, 2013). In addition to the sensitivity to the temporal context in which words are studied, subjects’ recall transitions also reflect the spatial context in which words were studied. Miller, Lazarus, Polyn, and Kahana (2013) had subjects study a list of objects while traveling on a controlled path within a virtual reality environment
to navigate to different locations where stimuli were experienced. After exploration, free recall of the stimuli was tested. Because the sequence of locations was chosen randomly, the sequential contiguity of the stimuli and the spatial contiguity of the stimuli were decorrelated. At test, the recall transitions that subjects exhibited reflected not only the temporal proximity along the path, but also the spatial proximity within the environment. Moreover, Polyn, Norman, and Kahana (2009a, 2009b) had subjects study concrete nouns using one of two orienting tasks. For some words, subjects would make a rating of its size (“Would this object fit in a shoebox?”); for other words subjects would rate its animacy. Polyn et al. (2009a, 2009b) found that recall transitions between words studied using the same orienting task were more common than transitions between words studied using different orienting tasks. Because the words in the list are randomly assigned, the preceding effects must reflect new learning during the study episode. In addition, participants’ memory search also reflects properties of the words themselves acquired from learning prior to the experimental session. It has long been known that, given that a list including pairs of associated words (TABLE, CHAIR ) randomly assigned to serial positions, the pairs are more likely to be recalled together (Bousfield, 1953). This effect generalizes to lists chosen from several categories— when the words are presented randomly, subjects nonetheless organize them into categories during recall (e.g., Cofer, Bruce, & Reicher, 1966). Interestingly, the time taken to retrieve words within a category is faster than the retrieval time necessary to transition from one category to another (Pollio, Kasschau, & DeNise, 1968; Pollio, Richards, & Lucas, 1969). Semantic relatedness is also a major factor affecting recall errors (e.g., Deese, 1959; Roediger & McDermott, 1995). The effect of semantic relatedness on memory retrieval can even be seen when the words do not come from well-defined semantic categories. Semantic similarity can be estimated between arbitrary pairs of words using automatic computational methods such as latent semantic analysis (Landauer & Dumais, 1997) or the word association space (Steyvers, Shiffrin, & Nelson, 2004). There are elevated transition probabilities even between words with relatively low values of semantic similarity (Howard & Kahana, 2002b; Sederberg et al., 2010).
retrieved context models In the previous subsection we saw that many researchers have appealed to a representation of “context” that is distinct from the representations of the words in the list to explain free-recall initiation. We also saw that properties of context could be used to account for recency effects within and across lists if the states of context changed gradually. But a context that is independent of the list items seems like a poor choice to account for transitions between subsequent recalls, which seem to reflect the properties of the items themselves, both learned and pre-experimental. One approach is to have multiple cues contribute to retrieval. That is, analogous to the Humphreys et al. (1989) model discussed earlier, one could have both direct item-to-item associations, as in Eq. 9, and context-to-item associations, as in Eq. 11. When an item is available, either because it is provided as an experimental cue or because it has been successfully retrieved in free recall, one can use both the item and the context to focus retrieval. These two sources of information can then be combined, perhaps multiplicatively, to select candidate words for recall (Raaijmakers & Shiffrin, 1980, 1981). This approach could readily account for semantic associations if the similarity of the vectors corresponding to different words reflects the semantic similarity between the meaning of those two words (Kimball, Smith, & Kahana, 2007; Sirotin et al., 2005). One could account for the contiguity effect over shorter time scales via direct item-to-item associations if associations are formed in a shortterm memory during study of the list. That is, rather than having the two simultaneously presented members of a pair be associated to one another in Eq. 9, one could form associations between all the words simultaneously active in short-term memory. At any one moment during study of the list, the last several items are likely to remain active in short-term memory. As a consequence, a particular word in the list is likely to have strengthened associations to words from nearby positions within the list. Recall of that item would provide a boost in accessibility of other words from nearby in the list. However, in much the same way that short-term memory has difficulty accounting for the recency effect across long time scales, this still leaves the question of how to account for contiguity effects across longer time scales.
models of episodic memory
177
(b) (a)
4
0.3
Across-list Association (z score)
Conditional Resp. Prob.
0.4 Delayed IPI~2s IPI~8s IPI~16s
0.2
0.1
0
3 2 1 0 –1 –2 –3
–4 –3 –2 –1 0 1 2 3 4
–30
Lag
–20
–10 0 10 Across-list Lag
20
30
Fig. 8.5 Transitions between words in free recall are affected by the order in which the words were presented. Here the probability of a recall transition from one word to the next is estimated as a function of the distance between the two words. Suppose that the 10th word in the study list has just been recalled. The lag-CRP at position +2 estimates the probability that the next word recalled will be from position 12; the lag-CRP at position −3 estimates the probability that instead the next word recalled will be from position 7. a. The lag-CRP is shown for four conditions of a delayed free recall. There was always approximately 16 s between the presentation of the last word and the time of test. The duration of the distractor interval between words (the IPI) was manipulated across conditions. After Howard & Kahana (1999). b. Subjects studied and recalled 48 lists of words. At the end of the experiment, they recalled all the words they could remember from all lists. This plot estimates the excess probability of a transition between lists, expressed as a z-score, as a function of the distance between the lists. That is, given that the just-recalled word came from list 10, a list lag of −3 corresponds to a transition to a word from list 7. Here the contiguity effect extends over a few hundred seconds. After Howard et al. (2008). c a. After Howard & Kahana (1999), 2008, Elsevier; b. After Howard, Youker, & Venkatadass (2008); With kind permission from Springer Science and Business Media.
Retrieved context models address this problem by postulating a coupling between words and a gradually changing state of context. Rather than contextual drift resulting from random fluctuations, context is driven by the presented items. Each word i provides some input cIN i : # (15) ci = ρci−1 + 1 − ρ 2 cIN i . For a random list of once-presented words, these inputs are uncorrelated, resulting in contextual drift analgous to Eq. 14. However, because the inputs are caused by the stimuli, the model is able to account for contiguity effects (Figure 8.4c). The key idea enabling the contiguity effect is the assumption that retrieving a word also results in recovery of the state of context in which that word was encoded. That is, if the word presented at time step i is repeated at some later time step r, then IN cIN r = γ ci−1 + (1 − γ )ci .
(16)
In addition to the input that stimulus caused when it was initially presented, it also enables recovery of the state of context present when it was presented, ci−1 . Because this state resembles the context when neighboring items were presented, a contiguity effect naturally results. If context changes gradually over long periods of time, the contiguity 178
basic cognitive skills
effect naturally persists over those same periods of time (Howard & Kahana, 2002a; Sederberg et al., 2008). Computational models describing details of free recall dynamics, including semantic transition effects have been developed (Sederberg et al., 2008; Polyn et al., 2009a). autonomous search models In retrieved context models, retrieval of an item causes recovery of a previous state of temporal context, resulting in the contiguity effect. This is not the only possibility, however. Consider a brief thought experiment. Try to recall as many of the 50 United States as possible (if the reader is unfamiliar with U.S. geography, the experiment should work just as well with any well-learned geographical region with more than a dozen or so entities). Most subjects recall geographically contiguous states (MAINE, VERMONT, NEW HAMPSHIRE . . . ).3 Examining the recall protocols, we would observe a spatial analog of a contiguity effect—if the subject has just recalled a word from a particular spatial location, the next word the subject retrieves would also tend to be from a nearby spatial location. However, it is not necessarily the case that remembering one state (Michigan) caused recovery of a nearby state (Wisconsin). Rather, both words might have been recalled because the search
happened to encounter a part of the map containing both of those states (the Great Lakes region). Autonomous search models provide an account of the temporal contiguity effect that is similar in spirit to that described above. In the Davelaar et al. (2005) account of the contiguity effect, retrieving a word has no effect on the state of memory used as a cue for the next retrieval. Rather, a state of context evolves according to some dynamics during study. It is reset to the state at the beginning of the list during recall and evolves according to the same dynamics during recall. Because it tends to revisit states in a similar order, the sequence of retrievals is correlated with the study order. Similarly, Farrell (2012) described temporal contiguity effects as resulting from the retrieval dynamics of hierarchical groups of chunked contexts (see also Ezzyat & Davachi, 2011). Autonomous search models have difficulty in accounting for genuinely associative contiguity effects. For instance, in cued recall, the experimenter chooses the cue—the fact that the correct pair is retrieved cannot be attributed to autonomous retrieval dynamics. Rather, the cue is utilized to recover the correct trace. The contiguity effect is observed under circumstances where the cue is randomized, eliminating the possibility that correlations between study and retrieval cause the contiguity effect (Kiliç et al., 2013; Howard, Venkatadass, Norman, & Kahana, 2007; Schwartz, Howard, Jing, & Kahana, 2005). The strong form of autonomous models—that memory search is independent of the products of previous memory search— must be false. Nonetheless, the geographical search thought experiment is compelling. A challenge going forward is to mechanistically describe the representations that could support a temporally defined search through memory analogous to the spatial search in the geography thought experiment.
Summary and Conclusions • A variety of detailed process models of performance have been developed in a variety of episodic memory tasks. • In recognition, differentiation and Bayesian decision rules have been important steps in advancing our understanding of recognition. • The major driver of models in recall has been an attempt to understand the nature of the context representation—more broadly what constitutes the cue in free-recall tasks.
Open Problems and Future Directions • If nothing else, the diversity of models demonstrates that behavioral data alone is not sufficient to result in a consensus model of any of the tasks we have considered, let alone a general theory of episodic memory. Going forward, early steps to use neurobiological constraints on process models of memory (e.g., Criss, Wheeler, & McClelland, 2013; Howard, Viskontas, Shankar, & Fried, 2012; Manning, Polyn, Litt, Baltuch, & Kahana, 2011) must be expanded. The results of these experiments must also affect the hypothesis space of models going forward. • The textbook definition of episodic memory is the experience of a “jump back in time” such that the subject vividly reexperiences a particular moment from his life. One of the major limitations in constructing models of episodic memory is that we do not have a coherent idea about how to represent time—context in the models we have discussed here may change gradually over time but is nonetheless ahistorical. Richer representations of temporal history (Shankar & Howard, 2012) may be able to provide a more unified approach to episodic memory (Howard, Shankar, Aue, & Criss, in press). • It is frustrating that there are so many differences between the item recognition models we have discussed and the recall models. The contiguity effect may provide a point of contact that could lead to the unification of these classes of models. Successful recognition of a list item during test seems to leave neighboring items in an elevated state of availability. Schwartz et al. (2005) presented travel scenes for item recognition testing. They systematically manipulated the lag between successively tested old probes. That is, after presenting old item i, they tested old item i + lag. They found that when |lag| was small, memory for the second old probe was enhanced, but only when subjects endorsed the first probe with high confidence. The recovery of previous states given an old probe has been the focus of connectionist models of recall and recognition (Norman & O’Reilly, 2003; Hasselmo & Wyble, 1997) suggesting a point of contact between mathematical models of memory and connectionist modeling, perhaps via dual process assumptions (see Box 1).
models of episodic memory
179
Acknowledgments The work was supported by the U.S. National Science Foundation grants to AHC (0951612) and MH (1058937).
Notes 1. For instance amnesia patients with essentially complete loss of episodic memory are typically unimpaired at serial recall of a short list of digits. 2. Recent studies have significantly elaborated this empirical story (Jang & Huber, 2008; Unsworth, Spillers, & Brewer, 2012; Ward & Tan, 2004). 3. Occassionally, subjects will try to recall in alphabetical order (ALABAMA, ALASKA, ARIZONA . . .) or according to some other idiosyncratic retrieval strategy, but that is not central to the point.
References Anderson, J. A. (1973). A theory for the recognition of items from short memorized lists. Psychological Review, 80(6), 417– 438. Anderson, J. A., Silverstein, J. W., Ritz, S. A. & Jones, R. S. (1977). Distinctive features, categorical perception, and probability learning: Some applications of a neural model. Psychological Review, 84, 413–451. Anderson, J. R., & Bower, G. H. (1972). Recognition and retrieval processes in free recall. Psychological Review, 79(2), 97–123. Atkinson, R. C., Shiffrin, R. M. (1968). Human memory: A proposed system and its control processes. K. W. Spence J. T. Spence (Eds.), The psychology of learning and motivation (Vol. 2, p. 89–105). New York, NY: Academic. Bousfield, W. A. (1953). The occurrence of clustering in the recall of randomly arranged associates. Journal of General Psychology, 49, 229–240. Cary, M., Reder, L. M. (2003). A dual-process account of the list-length and strength-based mirror effects in recognition. Journal of Memory and Language, 49(2), 231–248. Cofer, C. N., Bruce, D. R. & Reicher, G M. (1966). Clustering in Free Recall as a Function of Certain Methodological Variations. Journal of Experimental Psychology, 71, 858–866. Criss, A. H. (2006). The consequences of differentiation in episodic memory: Similarity and the strength based mirror effect. Journal of Memory and Language, 55)(4), 461–478. Criss, A. H. (2009). The distribution of subjective memory strength: list strength and response bias. Cognitive Psychology, 59(4), 297–319. doi: 10.1016/j.cogpsych.2009.07.003 Criss, A. H. (2010). Differentiation and response bias in episodic memory: evidence from reaction time distributions., Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(2), 484–499. Criss, A. H., Aue, W. R., & Kılıç, A. (2014). Age and response bias: Evidence from the strength based mirror effect. Quarterly Journal of Experimental Psychology, 67(10), 1910–1924 Criss. A. H. & Koop, G. J. (in press). Differentiation in Episodic Memory. In Raaijmakers, J., Criss, A. H., Goldstone, R.,
180
basic cognitive skills
Nosofsky, R., & Steyvers, M. (Eds.). Cognitive Modeling in Perception and Memory: A Festschrift for Richard M. Shiffrin. Psychology Press. Criss, A. H., Malmberg, K. J. & Shiffrin, R. M. (2011). Output interference in recognition memory. Journal of Memory and Language, 64, 316–326. Criss, A. H., & Shiffrin, R. M. (2004). Context noise and item noise jointly determine recognition memory: a comment on Dennis and Humphreys (2001). Psychological Review, 111(3), 800–807. Criss, A. H., & Shiffrin, R M. (2005). List discrimination in associative recognition and implications for representation, Journal Experimental Psychology: Learning, Memory and Cogntion, 31(6), 1199–1212. doi: 10.1037/02787393.31.6.1199. Criss, A. H., Wheeler, M. E., McClelland, J. L. (2013). A differentiation account of recognition memory: evidence from fmri. Journal of Cognitive Neuroscience, 25(3), 421–35. doi: 10.1162/jocn_a_00292 Davelaar, E. J., Goshen-Gottstein, Y., Ashkenazi, A., Haarmann, H. J., & Usher, M. (2005). The demise of short-term memory revisited: empirical and computational investigations of recency effects. Psychological Review, 112(1), 3–42. DeCarlo, L. T. (2003). Source monitoring and multivariate signal detection theory, with a model for selection. Journal of Mathematical Psychology, 47, 292–303. Deese, J. (1959). On the prediction of occurrence of particular verbal intrusions in immediate recall. Journal of Experimental Psychology, 58, 17–22. Dennis, S., & Humphreys, M. S. (2001). A context noise model of episodic word recognition. Psychological Review, 108(2), 452–478. Dennis, S., Lee, M. D., & Kinnell, A. (2008). Bayesian analysis of recognition memory: The case of the list-length effect. Journal of Memory and Language, 59(3), 361–376. Diller, D. E., Nobel, P. A., & Shiffrin, R. M. (2001). An ARC-REM model for accuracy and response time in recognition and recall. Journal Experimental Psychology: Learning, Memory and Cogntion, 27(2), 414–35. Estes, W. K. (1955). Statistical theory of spontaneous recovery and regression. Psychological Review, 62, 145–154. Ezzyat, Y., & Davachi, L. (2011). What constitutes an episode in episodic memory?. Psychological Science, 22(2), 243–52. doi: 10.1177/0956797610393742 Farrell, S. (2012). Temporal clustering and sequencing in shortterm memory and episodic memory. Psychological Review, 119(2), 223–71. doi: 10.1037/a0027371 Fortin, N. J., Agster, K. L. & Eichenbaum, H. B. (2002). Critical role of the hippocampus in memory for sequences of events. Nature Neuroscience, 5(5), 458–62. Gillund, G. & Shiffrin, R. M. (1984), A retrieval model for both recognition and recall. Psychological Review, 91, 1–67. Glanzer, M., & Adams, J. K. (1985). The mirror effect in recognition memory. Memory & Cognition, 13(1), 8–20. Glanzer, M., & Adams, J K. (1990). The mirror effect in recognition memory: data and theory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(1), 5–16. Glanzer, M., & Cunitz, A. R. (1966). Two storage mechanisms in free recall. Journal of Verbal Learning and Verbal Behavior, 5, 351–360.
Glanzer, M., Adams, J. K., Iverson, G. J., & Kim, K. (1993). The regularities of recognition memory. Psychological Review, 100, 546–567. Glenberg, A. M., Bradley, M. M., Stevenson, J. A., Kraus, T. A., Tkachuk, M. J., & Gretz, A. L. (1980). A twoprocess account of long-term serial position effects. Journal of Experimental Psychology: Human Learning and Memory, 6, 355–369. Hasselmo, M. E., & Wyble, B. P. (1997). Free recall and recognition in a network model of the hippocampus: simulating effects of scopolamine on human memory function. Behavioural Brain Research, 89(1–20 1–34. Hintzman, D. L. (1984). MINERVA 2: A simulation model of human memory. Behavior Research Methods, Instruments & Computers, 16(2), 96–101. Hintzman, D. L. (1986). “Schema abstraction” in a multipletrace memory model. Psychological Review, 93, 411–428. Hintzman, D. L., & Curran, T. (1994). Retrieval dynamics of recognition and frequency judgements: Evidence for separate processes of familiarity and recall. Journal of Memory and Language, 33, 1–18. Howard, M. W., & Kahana, M. J. (1999). Contextual variability and serial position effects in free recall. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 923–941. Howard, M. W., & Kahana, M. J. (2002a). A Distributed Representation of Temporal Context. Journal of Mathematical Psychology, 46(3), 269–299. Howard, M. W., & Kahana, M. J. (2002b). When does semantic similarity help episodic retrieval?. Journal of Memory and Language, 46(1), 85–98. Howard, M. W., Shankar, K. H., Aue, W., & Criss, A H. (in press). A distributed representation of internal time. Psychological Review. doi: 10.1037/a0037840 Howard, M. W., Venkatadass, V., Norman, K. A., & Kahana, M. J. (2007). Associative processes in immediate recency. Memory & Cognition, 35, 1700–1711. Howard, M. W., Viskontas, I. V., Shankar, K. H., & Fried, I. (2012). A neural signature of mental time travel in the human MTL. Hippocampus, 22(PMC3407826), 1833– 1847. Howard, M. W., Youker, T. E., & Venkatadass, V. (2008). The persistence of memory: Contiguity effects across several minutes. Psychonomic Bulletin & Review, 15, 58–63. Humphreys, M. S., Bain, J. D., & Pike, R. (1989). Different ways to cue a coherent memory system: A theory for episodic, semantic, and procedural tasks. Psychological Review, 96, 208–233. Jang, Y., & Huber, D. E. (2008). Context retrieval and context change in free recall: recalling from long-term memory drives list isolation. Journal Experimental Psychology: Learning, Memory and Cogntion, 34(1), 112–127. doi: 10.1037/02787393.34.1.112 Kahana, M. J. (1996). Associative retrieval processes in free recall. Memory & Cognition, 24, 103–109. Kahana, M. J., Howard, M., & Polyn, S. (2008). Associative Processes in Episodic Memory. In H. L. Roediger III (Ed.), Cognitive psychology of memory, Vol. 2 of Learning and Memory – A Comprehensive Reference (J. Byrne, Editor) (p. 476–490). Oxford Elsevier.
Kili c, A., Criss, A. H., & Howard, M. W. (2013). A causal contiguity effect that persists across time scales. Journal Experimental Psychology: Learning, Memory and Cogntion, 39(1), 297–303. doi: 10.1037/a0028463 Kimball, D. R., Smith, T. A. & Kahana, M. J. (2007). The fSAM model of false recall. Psychological Review, 114(4), 954–993. Klauer, K. C., & Kellen, D. (2010). Toward a complete decision model of item and source recognition: A discrete-state approach. Psychonomic Bulletin & Review, 17 465–478. Koop, G. J., Criss, A. H., & Malmberg, K. J. (in press). The role of mnemonic processes in pure-target and pure-foil recognition memory. Psychonomic Bulletin & Review. Landauer, T. K., Dumais, S. T. (1997). Solution to Plato’s problem : The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240. Lehman, M., & Malmberg, K. J. (2012). A Buffer Model of Memory Encoding and Temporal Correlations in Retrieval. Psychological Review. doi: 10.1037/a0030851 Malmberg, K. J. (2002). On the form of ROCs constructed from confidence ratings. Journal of Experimental Psychology: Learning, Memory, and Cognition, 28(2), 380–387. Malmberg, K. J., Criss, A. H., Gangwani, T. H., & Shiffrin, R. M. (2012). Overcoming the negative consequences of interference from recognition memory testing. Psychological Science, 23(2), 115–119. doi: 10.1177/0956797611430692 Malmberg, K. J., Holden, J. E., & Shiffrin, R. M. (2004). Modeling the effects of repetitions, similarity, and normative word frequency on old-new recognition and judgments of frequency. Journal Experimental Psychology: Learning, Memory and Cogntion, 30(2), 319–331. Malmberg, K. J., & Shiffrin, R. M. (2005). The “oneshot” hypothesis for context storage. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31(2), 322–336. Manning, J. R., Polyn, S. M., Litt, B., Baltuch, G., & Kahana, M. J. (2011). Oscillatory patterns in temporal lobe reveal context reinstatement during memory search. Proceedings of the National Academy of Science, USA, 108(31), 12893–12897. McClelland, J. L., & Chappell, M. (1998). Familiarity breeds differentiation: a subjective-likelihood approach to the effects of experience in recognition memory. Psychological Review, 105(4), 724–760. Mensink, G. J. M., Raaijmakers, J. G. W. (1988). A model for interference and forgetting. Psychological Review, 95, 434–55. Miller, J. F., Lazarus, E. M., Polyn, S. M., & Kahana, M. J. (2013). Spatial clustering during memory search. Journal Experimental Psychology: Learning, Memory and Cogntion, 39(3), 773–781. doi: 10.1037/a0029684 Murdock, B. B. (1982). A theory for the storage and retrieval of item and associative information. Psychological Review, 89, 609–626. Murdock, B. B. (1997). Context and mediators in a theory of distributed associative memory (TODAM2). Psychological Review, 104(2), 839–862.
models of episodic memory
181
Murdock, B. B., & Anderson, R. E. (1975). Encoding, storage and retrieval of item information. R L. Solso (Ed.), Information Processing and Cognition: The Loyola Symposium (p. 145–194). Hillsdale, New Jersey: Erlbaum. Nelson, D. L., McEvoy, C. L., & Schreiber, T. A. (1990). Encoding context and retrieval conditions as determinants of the effects of natural category size. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(1), 31–41. Norman, K. A., & O’Reilly, R. C. (2003). Modeling hippocampal and neocortical contributions to recognition memory: a complementary-learning-systems approach. Psychological Review, 110(4), 611–646. Ohrt, D. D., & Gronlund, S. D. (1999). List-length Effects and Continuous memory: Confounds and Solutions. C. Izawa (Ed.), On Human Memory: Evolution, Progress, and Reflections on the 30th Anniversary of the Atkinson-Shiffrin Model (p. 105–125). Mahwah, NJ: Erlbaum. Phillips, J. L., Shiffrin, R. J., & Atkinson, R. C. (1967). The effects of list length on short-term memory. Journal of Verbal Learning and Verbal Behavior, 6, 303–311. Pollio, H. R., Kasschau, R. A., & DeNise, H. E. (1968). Associative structure and the temporal characteristics of free recall. Journal of Verbal Learning and Verbal Behavior, 10, 190–197. Pollio, H. R., Richards, S., & Lucas, R. (1969). Temporal properties of category recall. Journal of Verbal Learning and Verbal Behavior, 8, 529–536. Polyn, S. M., Norman, K. A., & Kahana, M. J. (2009a). A context maintenance and retrieval model of organizational processes in free recall. Psychological Review, 116, 129–156. Polyn, S. M., Norman, K. A., & Kahana, M. J. (2009b). Task context and organization in free recall. Neuropsychologia, 47(11), 2158–2163. Postman, L., & Phillips, L. W. (1965). Short-term temporal changes in free recall. Quarterly Journal of Experimental Psychology, 17, 132–138. Province, J. M., & Rouder, J. N. (2012). Evidence for discretestate processing in recognition memory. Proceedings of the National Academy of Sciences USA, 109(36), 14357–14362. doi: 10.1073/pnas.1103880109 Raaijmakers, J. G. W., & Shiffrin, R. M. (1980). SAM: A theory of probabilistic search of associative memory. G H. Bower (Ed), The psychology of learning and motivation: Advances in research and theory (Vol. 14, p. 207–262). New York: Academic. Raaijmakers, J. G. W., & Shiffrin, R. M. (1981). Search of associative memory. Psychological Review, 88, 93–134. Ratcliff, R., Clark, S. E., & Shiffrin, R. M. (1990). List-strength effect: I. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16, 163–178. Roediger, H. L., & McDermott, K. B. (1995). Creating false memories: remembering words not presented in lists. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21, 803–814. Roediger, H. L., & Schmidt, S. R. (1980). Output interference in the recall of categorized and paired-associate lists. Journal of Experimental Psychology: Human Learning and Memory, 6(1), 91–102.
182
basic cognitive skills
Rotello, C. M., Macmillan, N. A., Reeder, J. A., & Wong, M. (2005). The remember response: subject to bias, graded, and not a process-pure indicator of recollection. Psychonomic Bulletin & Review, 12(5), 865–873. Rugg, M. D., & Curran, T. (2007). Event-related potentials and recognition memory. Trends in Cognitive Science, 11(6), 251– 257. Schooler, L. J., Shiffrin, R. M., & Raaijmakers, J. G. (2001). A Bayesian model for implicit effects in perceptual identification. Psychological Review, 108(1), 257–272. Schwartz, G., Howard, M. W., Jing, B., & Kahana, M. J. (2005). Shadows of the past: Temporal retrieval effects in recognition memory. Psychological Science, 16(11), 898–904. Sederberg, P. B., Howard, M. W., & Kahana, M. J. (2008). A context-based theory of recency and contiguity in free recall. Psychological Review, 115, 893–912. Sederberg, P. B., Miller, J. F., Howard, M. W., & Kahana, M. J. (2010). The temporal contiguity effect predicts episodic memory performance. Memory & Cognition, 38, 689–699. Shankar, K. H., & Howard, M. W. (2012). A scaleinvariant representation of time. Neural Computation, 24, 134–193. Shiffrin, R. M. (1970). Forgetting: Trace erosion or retrieval failure?. Science, 168, 1601–1603. Shiffrin, R. M., Huber, D. E., & Marinelli, K. (1995). Effects of category length and strength on familiarity in recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 21(2), 267–287. Shiffrin, R. M., Ratcliff, R., & Clark, S. E. (1990). List-strength effect: II. Theoretical mechanisms. Journal of Experimental Psychology: Learning, Memory, and Cognition, 16(2), 179–195. Shiffrin, R. M. & Steyvers, M. (1997). A Model for Recognition Memory: REM — Retrieving Effectively From Memory. Psychonomic Bulletin and Review, 4, 145–166. Sirotin, Y. B., Kimball, D. R., Kahana, M. J. (2005). Going beyond a single list: Modeling the effects of prior experience on episodic free recall. Psychonomic Bulletin & Review, 12, 787–805. Staresina, B. P., Fell, J., Dunn, J. C., Axmacher, N., & Henson, R. N. (2013). Using state-trace analysis to dissociate the functions of the human hippocampus and perirhinal cortex in recognition memory. Proceedings of the National Academy of Sciences, 110(80), 3119–24. doi: 10.1073/pnas.1215710110 Starns, J. J., White, C. N., & Ratcliff, R. (2010). A direct test of the differentiation mechanism: REM, BCDMEM, and the strength-based mirror effect in recognition memory. Journal of Memory and Language, 63(1), 18–34. Steyvers, M., Shiffrin, R. M., Nelson, D. L. (2004). Word association spaces for predicting semantic similarity effects in episodic memory. A. Healy (Ed.), Experimental Cognitive Psychology and its Applications: Festschrift in Honor of Lyle Bourne, Walter Kintsch and Thomas Landauer (pp. 237–249). Washington, D.C.: American Psychological Association. Stretch, V., & Wixted, J T. (1998). On the difference between strength-based and frequency-based mirror
effects in recognition memory. Journal Experimental Psychology: Learning, Memory and Cogntion, 24(6), 1379–1396. Tehan, G., & Humphreys, M. S. (1996). Cuing effects in shortterm recall. Memory & Cognition, 24(6), 719–732. Tulving, E. (1972). Episodic and semantic memory. E. Tulving & W. Donaldson (Ed.), Organization of Memory. (pp. 381– 403). New York: Adademic. Tulving, E. (1983). Elements of episodic memory. New York, NY: Oxford University Press. Tulving, E. (2002). Episodic memory: from mind to brain. Annual Review of Psychology, 53, 1–25. Tulving, E., & Arbuckle, T. (1963). Sources of intratrial interference in immediate recall of paired associates. Journal of Verbal Learning and Verbal Behavior, 1(5), 321–334. Tulving, E., & Arbuckle, T Y. (1966). Input and output interference in short-term associative memory. Journal of Experimental Psychology, 72, 145–150. Unsworth, N., Spillers, G. J., & Brewer, G. A. (2012). Evidence for noisy contextual search: examining the dynamics of list-before-last recall. Memory, 20(1), 1–13. doi: 10.1080/09658211.2011.626430 Ward, G., & Tan, L. (2004). The effect of the length of tobe-remembered lists and intervening lists on free recall: a reexamination using overt rehearsal. Journal of Experimental
Psychology: Learning, Memory, and Cognition, 30(6), 1196– 1210. Wickens, D. D., Born, D. G. & Allen, C. K. (1963). Proactive inhibition and item similarity in short-term memory. Journal of Verbal Learning and Verbal Behavior, 2(5), 440–445. Wilding, E. L., Doyle, M. C., & Rugg, M. D. (1995). Recognition memory with and without retrieval of context: an event-related potential study. Neuropsychologia, 33(6), 743–767. Wixted, J. T. (2007). Spotlighting the Probative Findings: Reply to Parks and Yonelinas (2007). Psychological Review, 114, 203–209. Wixted, J. T., & Squire, L. R. (2011). The familiarity/recollection distinction does not illuminate medial temporal lobe function: response to Montaldi and Mayes. Trends in Cognitive Sciences, 15(8), 340–1. doi: 10.1016/j.tics.2011.06.006 Yonelinas, A. P., & Parks, C. M. (2007). Receiver operating characteristics (ROCs) in recognition memory: a review. Psychological Bulletin, 133(5), 800–832. Zaromb, F. M., Howard, M. W., Dolan, E. D., Sirotin, Y. B., Tully, M., Wingfield, A., & Kahana, M. J. (2006). Temporal associations and prior-list intrusions in free recall. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 792–804.
models of episodic memory
183
PART
III
Higher Level Cognition
CHAPTER
9
Structure and Flexibility in Bayesian Models of Cognition
Joseph L. Austerweil, Samuel J. Gershman, Joshua B. Tenenbaum, and Thomas L. Griffiths
Abstract
Probability theory forms a natural framework for explaining the impressive success of people at solving many difficult inductive problems, such as learning words and categories, inferring the relevant features of objects, and identifying functional relationships. Probabilistic models of cognition use Bayes’s rule to identify probable structures or representations that could have generated a set of observations, whether the observations are sensory input or the output of other psychological processes. In this chapter we address an important question that arises within this framework: How do people infer representations that are complex enough to faithfully encode the world but not so complex that they “overfit” noise in the data? We discuss nonparametric Bayesian models as a potential answer to this question. To do so, first we present the mathematical background necessary to understand nonparametric Bayesian models. We then delve into nonparametric Bayesian models for three types of hidden structure: clusters, features, and functions. Finally, we conclude with a summary and discussion of open questions for future research. Key Words: inductive inference, Bayesian modeling, Nonparametrics, Bias-variance
tradeoff, Categorization, Feature representations, Function learning, Clustering
Introduction Probabilistic models of cognition explore the mathematical principles behind human learning and reasoning. Many of the most impressive tasks that people perform—learning words and categories, identifying causal relationships, and inferring the relevant features of objects—can be framed as problems of inductive inference. Probability theory provides a natural mathematical framework for inductive inference, generalizing logic to incorporate uncertainty in a way that can be derived from various assumptions about rational behavior (e.g., Jaynes, 2003). Recent work has used probabilistic models to explain many aspects of human cognition, from memory to language acquisition (for a representative sample, see Chater and Oaksford, 2008). There
are existing tutorials on some of the key mathematical ideas behind this approach (Griffiths and Yuille, 2006; Griffiths, Kemp, & Tenenbaum, 2008a) and its central theoretical commitments (Tenenbaum, Griffiths, & Kemp, 2006; Tenenbaum, Kemp, Griffiths, & Goodman, 2010a; Griffiths, Chater, Kemp, Perfors, & Tenenbaum, 2010a). In this chapter, we focus on a recent development in the probabilistic approach that has received less attention—the capacity to support both structure and flexibility. One of the striking properties of human cognition is the ability to form structured representations in a flexible way: we organize our environment into meaningful clusters of objects, identify discrete features that those objects possess, and learn relationships between those features, without 187
apparent hard constraints on the complexity of these representations. This kind of learning poses a challenge for models of cognition: how can we define models that exhibit the same capacity for structured, flexible learning? And how do we identify the right level of flexibility, so that we only postulate the appropriate level of complexity? A number of recent models have explored answers to these questions based on ideas from nonparametric Bayesian statistics (Sanborn, Griffiths, & Navarro, 2010; Austerweil & Griffiths 2013; see Gershman and Blei 2012 for a review), and we review the key ideas behind this approach in detail. Probabilistic models of cognition tend to focus on a level of analysis that is more abstract than that of many of the other modeling approaches discussed in the chapters of this handbook. Rather than trying to identify the cognitive or neural mechanisms that underlie behavior, the goal is to identify the abstract principles that characterize how people solve inductive problems. This kind of approach has its roots in what Marr (1982) termed the computational level, focusing on the goals of an information processing system and the logic by which those goals are best achieved, and was implemented in Shepard’s (1987) search for universal laws of cognition and Anderson’s (1990) method of rational analysis. But it is just the starting point for gaining a more complete understanding of human cognition—one that tells us not just why people do the things they do, but how they do them. Although we will not discuss it in this chapter, research on probabilistic models of cognition is beginning to consider how we can take this step, bridging different levels of analysis. We refer the interested reader to one of the more prominent strategies for building such a bridge, namely rational process models (Sanborn et al., 2010). The plan of this chapter is as follows. First, we introduce the core mathematical ideas that are used in probabilistic models of cognition—the basics of Bayesian inference—and a formal framework for characterizing the challenges posed by flexibility. We then turn to a detailed presentation of the ideas behind nonparametric Bayesian inference, looking at how this approach can be used for learning three different kinds of representations—clusters, features, and functions.
describe the basic logic behind using Bayes’ rule for inductive inference. Then, we explore two of the main types of hypothesis spaces for possible structures used in statistical models: parametric and nonparametric models.1 Finally, we discuss what it means for a nonparametric model to be “Bayesian” and propose nonparametric Bayesian models as methods combining the benefits of both parametric and nonparametric models. This sets up the remainder of the article, where we compare the solution given by nonparametric Bayesian methods to how people (implicitly) solve this dilemma when learning associations, categories, features, and functions.
Basic Bayes After observing some evidence from the environment, how should an agent update her beliefs in the various structures that could have produced the evidence? Given a set of candidate structures and the ability to describe the degree of belief in each structure, Bayes’s rule prescribes how an agent should update her beliefs across many normative standards (Oaksford and Chater, 2007; Robert, 1994). Bayes’s rule simply states that an agent’s belief in a structure or hypothesis h after observing data d from the environment, the posterior P(h|d ), should be proportional to the product of two terms: her prior belief in the structure, the prior P(h), and how likely the observed data d would be had it been produced by the candidate structure, called the likelihood P(d |h). This is given by P(d |h)P(h) , P(h|d ) = h ∈H P(d |h )P(h ) where H is the space of possible hypotheses or latent structures. Note that the summation in the denominator is the normalization constant, which ensures that the posterior probability is still a valid probability distribution (sums to one). In addition to specifying how to calculate the posterior probability of each hypothesis, a Bayesian model prescribes how an agent should update her belief in observing new data dnew from the environment given the previous observations d P(dnew |d ) = P(dnew |h)P(h|d ) h
=
h
Mathematical Background In this section, we present the necessary mathematical background for understanding nonparametric Bayesian models of cognition. First, we 188
higher level cognition
P(dnew |h)
P(d |h)P(h) . h ∈H P(d |h )P(h )
The fundamental assumptions of Bayesian models (i.e., what makes them “Bayesian”) are (a) agents express their expectations over structures as probabilities and (b) they update their expectations according
to the laws of probability. These are not uncontroversial assumptions in psychology (e.g., Bowers and Davis 2012; Jones and Love 2011; Kahneman et al. 1982; McClelland et al. 2010, but also look at the replies Chater et al. 2011; Griffiths, Chater, Norris, & Pouget 2012; Griffiths, Chater, Kemp, Perfors, & Tenenbaum 2010b). However, they are extremely useful because they provide methodological tools for exploring the consequences of adopting different assumptions about the kind of structure that appears in the environment.
Parametric and Nonparametric One of the first steps in formulating a computational model is a specification of the possible structures that could have generated the observations from the environment, the hypothesis space H. As discussed in the Basic Bayes subsection, each hypothesis h in a hypothesis space H is defined as a probability distribution over the possible observations. So, specifying the hypothesis space amounts to defining the set of possible distributions over events that the agent could observe. To make the model Bayesian, a prior distribution over those hypotheses also needs to be specified. From a statistical point of view, a Bayesian model with a particular hypothesis space (and prior over those hypotheses) is a solution to the problem of density estimation, which is the problem of estimating the probability distribution over possible observations from the environment.2 In fact, it is the optimal solution given that the hypothesis space faithfully represents how the environment produces observations, and the environment randomly selects a hypothesis to produce observations with probability proportional to the prior distribution. In general, a probability distribution is a function over the space of observations, which can be continuous, and thus is specified by an infinite number of parameters. So, density estimation involves identifying a function specified by an infinite number of parameters, as theoretically, it must specify the probability of each point in a continuous space. From this perspective, a function is analogous to a hypothesis and the space of possible functions constructed by varying the values of the parameters defines a hypothesis space. Different types of statistical models make different assumptions about the possible functions that define a density, and statistical inference amounts to estimating the parameters that define each function.
The statistical literature offers a useful classification of different types of probability density functions, based on the distinction between parametric and nonparametric models (Bickel and Doksum, 2007). Parametric models take the set of possible densities to be those that can be identified with a fixed number of parameters. An example of a parametric model is one that assumes the density follows a Gaussian distribution with a known variance, but unknown mean. This model estimates the mean of the Gaussian distribution based on observations and its estimate of the probability of new observations is their probability under a Gaussian distribution with the estimated mean. One property of parametric models is that they assume there exists a fixed set of possible structures (i.e., parameterizations) that does not change regardless of the amount of data observed. For the earlier example, no matter how much data the model is given that is inconsistent with a Gaussian distribution (e.g., a bimodal distribution), its density estimate would still be a Gaussian distribution because it is the only function available to the model. In contrast, nonparametric models make much weaker assumptions about the family of possible structures. For this to be possible, the number of parameters of a nonparametric model increases with the number of data points observed. An example of a nonparametric statistical model is a Gaussian kernel model, which places a Gaussian distribution at each observation and its density estimate is the average over the Gaussian distributions associated with each observation. In essence, the parameters of this statistical model are the observations, and so the parameters of the model grow with the number of data points. Although nonparametric suggests that nonparametric models do not have any parameters, this is not the case. Rather, the number of parameters in a nonparametric model is not fixed with respect to the amount of data. One domain within cognitive science where the distinction between parametric and nonparametric models has been useful is category learning (Ashby and Alfonso-Reese, 1995). The computational problem underlying category learning is identifying a probability distribution associated with each category label. Prototype models (Posner and Keele, 1968; Reed, 1972), approach this problem parametrically, by estimating the mean of a Gaussian distribution for each category. Alternatively, exemplar models (Medin and Schaffer, 1978; Nosofsky, 1986) are nonparametric, using each observation as a parameter; each category’s density estimate for
structure and flexibility in bayesian models of cognition
189
(a)
Parametric
(b)
Nonparametric
(c)
Nonparametric Bayesian
Fig. 9.1 Density estimators from the same observations (displayed in blue) for three types of statistical models: (a) parametric, (b) nonparametric, and (c) nonparametric Bayesian models. The parametric model estimates the mean of a Gaussian distribution from the observations, which results in the Gaussian density (displayed in black). The nonparametric model averages over the Gaussian distributions centered at each data point (in red) to yield its density estimate (displayed in black). The nonparametric Bayesian model puts the observations into three groups, each of which gets its own Gaussian distribution with mean centered at the center of the observations (displayed in red). The density estimate is formed by averaging over the Gaussian distributions associated with each displayed in black.
an new observation is a function of the sum of the distances between the new observation to previous observations. See Figure 9.1(a) and 9.1(b) for examples of parametric and nonparametric density estimation, respectively. One limitation of parametric models is that the structure inferred by these models will not be the true structure producing the observations if the true structure is not in the model’s hypothesis space. On the other hand, many nonparametric models are guaranteed to infer the true structure given enough (i.e., infinite) observations, which is a property known as consistency in the statistics literature (Bickel and Doksum, 2007). Thus, as the number of observations increase, nonparametric models have lower error than parametric models when the true structure is not in the hypothesis space of the parametric model. However, nonparametric models typically need more observations to arrive at the true hypothesis when the true hypothesis is in the parametric model’s hypothesis space. Box 1 describes the bias-variance tradeoff, which expounds this intuition and provides a formal framework for understanding the benefits and problems with each approach.
Putting Them Together: Nonparametric Bayesian Models Although people are clearly biased toward certain structures by their prior expectations (like parametric models), the bias is a soft constraint, meaning that people seem to be able to infer seemingly arbitrarily complex models given enough evidence (like nonparametric models). In the remainder of 190
higher level cognition
the article, we propose nonparametric Bayesian models, which are nonparametric models with prior biases toward certain types of structures, as a computational explanation for how people infer structure but maintain flexibility.
Box 1 The bias-variance trade-off Given that nonparametric models are guaranteed to infer the true structure, why would anyone use a parametric model? Although it is true that nonparametric models will converge to the true structure, they are only guaranteed to do so in the limiting case of an infinite number of observations. However, people do not get an infinite number of observations. Thus, the more appropriate question for cognitive scientists is how do people infer structures from a small number of observations, and which type of model is more appropriate for understanding human performance? There are many structures consistent with the limited and noisy evidence typically observed by people. Furthermore, when observations are noisy, it becomes difficult for an agent to distinguish between noise and systematic variation due to the underlying structure, a problem known as “overfitting.” When there are many structures available to the agent, as is the case for nonparametric models, this becomes a serious issue. So, although nonparametric models have the upside of guaranteed convergence to the appropriate structure, they have the downside of being
Box 1 continued prone to overfitting. In this box, we discuss the bias-variance trade-off, which provides a useful framework for understanding the tradeoff between ultimate convergence to the true structure and overfitting. The bias-variance trade-off is a mathematical result, which demonstrates that the amount of error an agent is expected to make when learning from observations can be decomposed into the sum of two components (German, Bienenstock, & Doursat, 1992; Griffiths et al., 2010): bias, which measures how close the expected estimated structure is to the true structure, and variance, which measures how sensitive the expected estimated structure is to noise (how much it is expected to vary across different possible observations). Intuitively, increasing the number of possible structures by using a larger parametric or a fully nonparametric model reduces the bias of the model because a larger hypothesis space increases the likelihood that the true structure is available to the model. However, it also increases the variance of the model because it will be harder to choose among them given noisy observations from the environment. On the other hand, decreasing the possible number of structures available by using a small parametric model increases the bias of the model, because unless the true structure is one of the structures available to the parametric model, it will not be able to infer the true structure. Furthermore, using a small parametric model reduces the variance, because there are fewer structures available to the model that are likely to be consistent with the noisy observations. The bias-variance tradeoff presents a trade-off: reducing the bias of a model by using a nonparametric model with fewer prior constraints comes at the cost of less efficient inference and increased susceptibility to overfitting, resulting in larger variance. How do people resolve the bias-variance trade-off? In some domains, they are clearly biased because some structures are much easier to learn than others (e.g., linear functions in function learning; Brehmer 1971, 1974). So in some respects people act like parametric models, in that they use strong constraints to infer structures. However, given enough training, experience, and the right kind of information, people can infer extremely complex structures
(e.g., McKinley and Nosofsky 1995). Thus, in other respects, people act like nonparametric models. How to reconcile these two views remains an outstanding question for theories of human learning. Hierarchical Bayesian models offer one possible answer, where agents maintain multiple hypothesis spaces and infer the appropriate hypothesis space to perform Bayesian inference over, using the distribution of stimuli in the domain (Kemp, Perfors, & Tenenbaum, 2007) and the concepts agents learn over the stimuli (Austerweil and Griffiths, 2010a). In principle, a hierarchical Bayesian model could be formulated that includes both parametric and nonparametric hypothesis spaces, thereby inferring which is appropriate for a given domain. Formulating such a model is an interesting challenge for future research.
Nonparametric Bayesian models are Bayesian because they put prior probabilities over the set of possible structures, which typically include arbitrarily complex structures. They posit probability distributions over structures that can, in principle, be infinitely complex, but they are biased towards “simpler” structures (those representable using a smaller number of units), which reduces the variance that plagues classical nonparametric models. The probability of data under a structure, which can be very large for complex structures that encode each observation explicitly (e.g., each observation in its own category), is traded off against a prior bias toward simpler structures, which allow observations to share parameters. This bias toward simpler structures is a soft constraint, allowing models to adopt more complex structures as new data arrive (this is what makes these models “nonparametric”). Thus, nonparametric Bayesian models combine the benefits of parametric and nonparametric models: a small variance (by using Bayesian priors) and a small bias (by adapting their structure nonparametrically). See Figure 9.1(c) for an example of nonparametric Bayesian density estimation. Nonparametric Bayesian models can be classified according to the type of hidden structure they posit. For the previously discussed category learning example, the hidden structure is a probability distribution over the space of observations. Thus, the prior is a probability distribution over probability distributions. A common choice for this prior is the
structure and flexibility in bayesian models of cognition
191
Dirichlet process (Ferguson, 1973), which induces a set of discrete clusters, where each observation belongs to a single cluster and each cluster is assigned to a randomly drawn value. Combining the Dirichlet process with a model of how observed features are generated by clusters, we obtain a Dirichlet-process mixture model (Antoniak, 1974). As we discuss in the following section, elaborations of the Dirichlet process mixture model have been applied to many psychological domains as varied as category learning (Anderson, 1991; Sanborn, Canini, & Navarro, 2008b), word segmentation (Griffiths, & Johnson, 2009), and associative learning (Gershman, Blei, & Niv, 2010; Gershman, and Niv, 2012). Although many of the applications of nonparametric Bayesian models in cognitive science have focused on the Dirichlet process mixture model, other nonparametric Bayesian models, such as the Beta process (Hjort, 1990; Thibaux and Jordan, 2007) and Gaussian process (Rasmussen and Williams, 2006), are more appropriate when people infer probability distributions over observations that are encoded using multiple discrete units or continuous units. For example, feature learning is best described by a hidden structure with multiple discrete units. The Beta process (Griffiths and Ghahramani, 2005, 2011; Hjort, 1990; Thibaux and Jordan, 2007) is one appropriate nonparametric Bayesian model for this example, and as we discuss in the section Inferring Features: What Is a Perceptual Unit?, elaborations of the Beta process have been applied to model feature learning (Austerweil and Griffiths, 2011, 2013), multimodal learning (Yildirm and Jacobs, 2012), and choice preferences (Görür, Jäkel; Miller, Griffiths). Finally, Gaussian processes are appropriate when each observation is encoded using one or more continuous units; we discuss their application to function learning in the section Learning Functions: How Are Continuous Quantities Related? (Griffiths, Lucas, Williams, & Kalish, 2009).
Inferring Clusters: How Are Observations Organized into Groups? One of the basic inductive problems faced by people is organizing observations into groups, sometimes referred to as clustering. This problem arises in many domains, including category learning (Clapper and Bower, 1994; Kaplan and Murphy, 1999; Pothos and Chater, 2002), motion perception (Braddick, 1993), causal inference (Kemp, Tenenbaum, Niyogi, & Griffiths, 2010), word 192
higher level cognition
segmentation (Werker and Yeung, 2005), semantic representation (Griffiths, Steyvers, & Tenenbaum, 2007), and associative learning (Gershman et al., 2010). Clustering is challenging because in real world situations the number of clusters is often unknown. For example, a child learning language does not know a priori how many words there are in the language. How should a learner discover new clusters? In this section, we show how clustering can be formalized as Bayesian inference, focusing in particular on how the nonparametric concepts introduced in the previous section can be brought to bear on the problem of discovering new clusters. We then describe an application of the same ideas to associative learning.
A Rational Model of Categorization Categorization can be formalized as an inductive problem: given the features of a stimulus (denoted by x), infer the category label c. Using Bayes’s rule, the posterior over category labels is given by: P(x, c) P(x|c)P(c) = . c P(x|c )P(c ) c P(x, c )
P(c|x) =
From this point of view, category learning is fundamentally a problem of density estimation (Ashby and Alfonso-Reese, 1995) because people are estimating a probability distribution over the possible observations from each category. Probabilistic models differ in the assumptions they make about the joint distribution P(x, c). Anderson (1991) proposed that people model this joint distribution as a mixture model: P(x, c|z)P(z), P(x, c) = z
where z ∈ {1, . . . , K } denotes the cluster assigned to x (z is the traditional notation for a cluster, and is analogous to the hypothesis h in the previous section). From a generative perspective, observations are generated by a mixture model from the environment according to the following process: to sample observation n, first sample its cluster zn from P(z), and then the observation xn and its category cn from the joint distribution specified by the cluster, P(x, c|zn ). Each distribution specified by a cluster might be simple (e.g., a Gaussian), but their mixture can approximate arbitrarily complicated distributions. Because each observation only belongs to one cluster, the assignments zn = {z1 , . . . , zn } encode a partition of the items into K distinct clusters, where a partition is a grouping of items into mutually
exclusive clusters. When the value of K is specified, this generative process defines a simple probabilistic model of categorization, but what should be the value of K ? To address the question of how to select the value of K , Anderson assumed that K is not known a priori, but rather learned from experience, such that K can be increased as new data are observed. As the number of clusters grows with observations and each cluster has associated parameters defining its probability distribution over observations, this rational model of categorization is nonparametric. Anderson proposed a prior on partitions that sequentially assign observations to clusters according to: P(zn = k|zn−1 ) =
mk n−1+α α n−1+α
if mk > 0 (i.e., k is old) if mk = 0 (i.e., k is new)
where mk is the number of items in zn−1 assigned to cluster k, n is the total number of items observed so far, and α ≥ 0 is a parameter that governs the total number of clusters.3 As pointed out by Neal (2000), the process proposed by Anderson is equivalent to a distribution on partitions known as the Chinese restaurant process (CRP; Aldous, 1985; Blackwell and MacQueen, 1973). Its name comes from the following metaphor (illustrated in Figure 9.2): Imagine a Chinese restaurant with an unbounded number of tables (clusters), where each table can seat an unbounded number of customers (observations). The first customer enters and sits at the first table. Subsequent customers sit at an occupied table with a probability proportional to how many people are already sitting there (mk ), and at a new table with probability proportional to α. Once all the customers are seated, the assignment of customers to tables defines a partition of observations into clusters. The CRP arises in a natural way from a nonparametric mixture modeling framework (see Gershman and Blei, 2012, for more details). To see this, consider a finite mixture model where the
cluster assignments are drawn from: θ ∼ Dirichlet(α/K , . . . , α/K ) zi ∼ Multinomial(θ ),
where Dirichlet( · ) denotes the K -dimensional Dirichlet distribution, where θ roughly corresponds to the relative weight of each block in the partition. As the number of clusters increases to infinity (K → ∞), this distribution on zn is equivalent to the CRP. Another view of this model is given by a seemingly unrelated process, the Dirichlet process (Ferguson, 1973), which is a probability distribution over discrete probability distributions. It directly generates the partition and the parameters associated with each block in the partition. Marginalizing over all the possible ways of getting the same partition from a Dirichlet process defines a related distribution, the Pólya urn (Blackwell and MacQueen, 1973), which is equivalent to the CRP when the parameter associated with each block is ignored. For this reason, a mixture model with a CRP prior on partitions is known as a Dirichlet process mixture model (Antoniak, 1974). One of the original motivations for developing the rational model of categorization was to reconcile two important observations about human category learning. First, in some cases, the confidence with which people assign a new stimulus to a category is inversely proportional to its distance from the average of the previous stimuli in that category (Reed, 1972). This, in conjunction with other data on central tendency effects (e.g., Posner and Keele, 1968), has been interpreted as people abstracting a “prototype” from the observed stimuli. On the other hand, in some cases, people are sensitive to specific stimuli (Medin and Schaffer, 1978), a finding that has inspired exemplar models that memorize the entire stimulus set (e.g., Nosofsky, 1986). Anderson (1991) pointed out that his rational model of categorization can capture both of these findings, depending on the inferred partition structure: when all items are assigned to the same cluster, the model is equivalent to forming a single
(a)
(b)
Chinese Restaurant Process Table 1 1
Tables 1
Table 2
Table 3 3
...
2
3
Customers
2
for i = 1, . . . , n,
Fig. 9.2 (a) The culinary representation of the Chinese restaurant process and (b) the cluster assignments implied by it.
structure and flexibility in bayesian models of cognition
193
prototype, whereas when all items are assigned to unique clusters, the model is equivalent to an exemplar model. However, in most circumstances, the rational model of categorization will partition the items in a manner somewhere between these two extremes. This is a desirable property that other recent models of categorization have adopted (Love, Medin, & Gureckis, 2004). Finally the CRP serves as a useful component in many other cognitive models (see Figure 7 and Box 2).
Associative Learning As its name suggests, associative learning has traditionally been viewed as the process by which associations are learned between two or more stimuli. The paradigmatic example is Pavlovian conditioning, in which a cue and an outcome (e.g., a tone and a shock) are paired together repeatedly. When done with rats, this procedure leads to the rat freezing in anticipation of the shock whenever it hears the tone. Association learning can be interpreted in terms of a probabilistic causal model, which calculates the probability that one variable, called the cue (e.g., a tone), causes the other variable, called the outcome (e.g., a shock). Here y encodes the presence or absence of the outcome, and x similarly encodes the presence or absence of the cue. The model assumes that y is a noisy linear function of x: y∼N (wx, σ 2 ). The parameter w encodes the associative strength between x and y and σ 2 parameterizes the variability of their relationship. This model can be generalized to the case in which multiple cues are paired with an outcome by assuming that the associations are additive: y ∼ N ( i wi xi , σ 2 ), where i ranges over the cues. The linear-Gaussian model also has an interesting connection to classical learning theories such as the Rescorla-Wagner model (Rescorla and Wagner, 1972), which can be interpreted as assuming a Gaussian prior on w and carrying out Bayesian inference on w (Dayan, Kakade, Montague 2000; Kruschke, 2008). Despite the successes of the Rescorla-Wagner model and its probabilistic variants, they incorrectly predict that there should only be learning when the prediction error is nonzero (i.e., when y− i wi xi = 0), but people and animals can still learn in some cases. For example, in sensory preconditioning (Brogden, 1939), two cues (A and B) are presented together without an outcome; when A is subsequently paired with an outcome, cue B acquires associative strength despite never being paired with the outcome. Because A and B presumably start 194
higher level cognition
out with zero associative strength and there are no prediction errors during the preconditioning phase, the Rescorla-Wagner model predicts that there should be no learning. The model fails to explain how preconditioning enables B to subsequently acquire associative strength from A-outcome training.
Box 2 Composing Richer Nonparametric Models Although this chapter has focused on several of the most basic nonparametric Bayesian models and their applications to basic psychological processes, these components can also be composed to build richer accounts of more sophisticated forms of human learning and reasoning (bottom half of Figure 7). These composites greatly extend the scope of phenomena that can be captured in probabilistic models of cognition. In this box, we discuss a number of these composite models. In many domains, categories are not simply a flat partition of entities into mutually exclusive classes. Often they have a natural hierarchical organization, as in a taxonomy that divides life forms into animals and plants; animals into mammals, birds, fish, and other forms; mammals into canidae, felines, primates, rodents, . . . ; canidae into dogs, wolves, foxes, coyotes, . . . ; and dogs into many different breeds. Category hierarchies can be learned via nonparametric models based on nested versions of the CRP, in which each category at one level gives rise to a CRP of subcategories at the level below (Griffiths et al., 2008b; Blei, Griffiths, & Jordan, 2010). Category learning and feature learning are typically studied as distinct problems (as discussed in the sections Inferring Clusters: How Are Observations Organized into Groups? and Inferring Features: What Is a Perceptual Unit?), but in many real-world situations people can jointly discover how best to organize objects into classes and which features best support these categories. Nonparametric models that combine the CRP and IBP can capture these joint inferences (Austerweil and Griffiths, 2013). The model of Salakhutdinov et al. (2012) extends this idea to hierarchies, using the nested CRP combined with a hierarchical Dirichlet process topic model to jointly learn hierarchically structured object categories and
Box 2 continued hierarchies of part-like object features that support these categories. The CrossCat model (Shafto et al., 2011) allows us to move beyond another limitation of typical models of categorization (parametric or nonparametric): the assumption that there is only a single best way to categorize a set of entities. Many natural domains can be represented in multiple ways: animals may be thought of in terms of their taxonomic groupings or their ecological niches, foods may be thought of in terms of their nutritional content or social role; products may be thought of in terms of function or brand. CrossCat discovers multiple systems of categories of entities, each of which accounts for a distinct subset of the entities’ observed attributes, by nesting CRPs over entities inside CRPs over attributes. Traditional approaches to categorization treat each entity individually, but richer semantic structure can be found by learning categories in terms of how groups of entites relate to each other. The Infinite Relational Model (IRM; Kemp et al. 2006, 2010) is a nonparametric model that discovers categories of objects that not only share similar attributes, but also participate in similar relations. For instance, a data set for consumer choice could be characterized in terms of which consumers bought which products, which features are present in which products, which demographic properties characterize which users, and so on. IRM could then discover how to categorize products and consumers (and perhaps also features and demographic properties), and simultaneously uncover regularities in how these categories relate (for) example, that consumers in class X tend to buy products in class Y). Nonparametric models defined over graph structures, such as the graph-based GP models of Kemp and Tenenbaum (2008, 2009), can capture how people reason about a wider range of dependencies between the properties of entities and the relations between entities, allowing that objects in different domains can be related in qualitatively different ways. For instance, the properties of cities might be best explained by their relative positions in a two-dimensional map, the voting patterns of politicians by their orientation along a one-
dimensional liberal-conservative axis, and the properties of animals by their relation in a taxonomic tree. We could also distinguish animals’ anatomical and physiological properties, which are best explained by the taxonomy, from behavioral and ecological properties that might be better explained by their relation in a directed graph such as a food web. Perhaps most intriguingly, nonparametric Bayesian models can be combined with symbolic grammars to account for how learners could explore the broad landscape of different model structures that might describe a given domain and arrive at the best model (Kemp and Tenenbaum, 2008; Grosse et al., 2012). A grammar is used to generate a space of qualitatively different model families, ranging from simple to complex, each of which defines a predictive model for the observed data based on a GP, CRP, IBP or other nonparametric process. These frameworks have been used to build more human-like machine learning and discovery systems, but they remain to be tested as psychological accounts of how humans learn domain structures.
Sensory preconditioning and other related findings have prompted consideration of alternative probabilistic models for associative learning. Courville, Daw, & Touretzky, (2006) proposed that people and animals posit latent causes to explain their observations in Pavlovian conditioning experiments. According to this idea, a single latent cause generates both the cues and outcomes. Latent cause models are powerful because they can explain why learning occurs in the absence of prediction errors. For example, during sensory preconditioning, the latent cause captures the covariation between the two cues; subsequent conditioning of A increases the probability that B will also be accompanied by the outcome. An analogous question about how to pick the number of clusters in a mixture model arises in associative learning: How many latent causes should there be in a model of associative learning? To address this question, Gershman and colleagues (Gershman et al., 2010; Gershman and Niv, 2012) used the CRP as a prior on latent causes. This allows the model to infer new latent causes when the sensory statistics change, but otherwise it prefers a small number of latent causes. Unlike previous models that define the number of latent causes
structure and flexibility in bayesian models of cognition
195
a priori, Gershman et al. (2010) showed that this model could explain why extinction does not tend to erase the original association: Extinction training provides evidence that a new latent cause is active. For example, when conditioning and extinction occur in different contexts, the model infers a different latent cause for each context; upon returning to the conditioning context, the model predicts a renewal of the conditioned response, consistent with empirical findings (see Bouton, 2004). By addressing the question of how agents infer the number of latent causes, the model offered new insight into a perplexing phenomenon.
Inferring Features: What Is a Perceptual Unit? The types of latent structures that people use to represent a set of stimuli can be far richer than clustering the stimuli into groups. For example, consider the following set of animals: domestic cats, dogs, goldfish, sharks, lions, and wolves. Although they can be represented as clusters (e.g., PETS and WILD ANIMALS or FELINES, CANINES, and SEA ANIMALS), another way to represent the animals is using features, or multiple discrete units per animal (e.g., a cat might be represented with the following features: HAS TAIL, HAS WHISKERS, HAS FUR, IS CUTE, etc.). Feature representations can be used to solve problems arising in many domains, including choice behavior (Tversky, 1972), similarity (Nosofsky, 1986; Tversky, 1977), and object recognition (Palmer, 1999; Selfridge and Neisser, 1960). Analogous to clustering, the appropriate feature representation or even the number of features for a domain is not known a priori. In fact, a common criticism of feature-based similarity is that there is an infinite number of potential features that can be used to represent any stimulus and that human judgments are mostly determined by the features selected to be used in a given context (Goldmeier, 1972; Goodman, 1972; Murphy and Medin, 1985). How do people infer the appropriate features to represent a stimulus in a given context? In this section, we describe how the problem of inferring feature representations can be cast as a problem of Bayesian inference, where the hypothesis space is the space of possible feature representations. Because there is an infinite number of feature representations, the model used to solve this problem will be a nonparametric Bayesian model. Then, we illustrate two psychological applications. 196
higher level cognition
A Rational Model of Feature Inference Analogous to other Bayesian models of cognition, defining a rational model of feature inference amounts to applying Bayes’s rule to a specification of the hypothesis space, how hypotheses produce observations (the likelihood), and the prior probability of each hypothesis (the prior). Following previous work by Austerweil and Griffiths (2011), we first define these three components for a Bayesian model with a finite feature repository and then define a nonparametric Bayesian model by allowing an infinite repository of features. This allows the model to infer a feature representation without presupposing the number of features ahead of time. The computational problem of feature representation inference is as follows: Given the Ddimensional raw primitives for a set of N stimuli X (each object is a D-dimensional row vector xn ), infer a feature representation that encodes the stimuli and adheres to some prior expectations. We decompose a feature representation into two components: an N × K feature ownership matrix Z, which is a binary matrix encoding which of the K features each stimulus has (i.e., znk = 1 if stimulus n has feature k, and znk = 0 otherwise), and a K × D feature image matrix Y, which encodes the consequence of a stimulus having each feature. In this case, the hypothesis space is the Cartesian product of possible feature ownership matrices and possible feature image matrices. As we discuss later in further detail, the precise format of feature image matrix Y depends on the format of the observed raw primitives. For example, if the stimuli are the images of objects and the primitives are D binary pixels that encode whether light was detected in each part of the retina, then a stimulus and a feature image, x and y respectively, are both D-dimensional binary vectors. So if x is the image of a mug, y could be the image of its handle. Applying Bayes’s rule and assuming that the feature ownership and image matrices are independent a priori, inferring a feature representation amounts to optimizing the product of three terms P(Z, Y|X) ∝ P(X|Y, Z)P(Y)P(Z), where P(X|Y, Z), the likelihood, encodes how well each object xn is reconstructed by combining together the feature images Y of the features the object has, which is given by zn , P(Y) encodes prior expectations about feature images (e.g., Gestalt laws), and P(Z) encodes prior expectations about feature ownership matrices.4 As the likelihood and feature image prior are more straightforward to
define and specific to the format of the observed primitives, we first derive a sensible prior distribution over all possible feature ownership matrices before returning our attention to them. Before delving into the case of an infinite number of potential features, we derive a prior distribution on feature ownership matrices that has a finite and known number of features K . As the elements of a feature ownership matrix are binary, we can define a probability distribution over the matrix by flipping a weighted coin with bias πk for each element znk . We do not observe πk and so, we assume a Beta distribution as its prior. This corresponds to the following generative process iid
πk ∼ Beta(α/K , 1), iid
for k = 1, . . . , K
znk |πk ∼ Bernoulli(πk ),
for n = 1, . . . , N .
Due to the conjugacy of Bernoulli likelihoods and Beta priors, it is relatively simple to integrate out π1 , . . . , πK to arrive at the following probability distribution, P(Z|α), over finite feature ownership representations. See Bernardo and Smith (1994) and Griffiths and Ghahramani (2011) for details. Analogous to the method discussed earlier for constructing the CRP as the infinite limit of a finite model, taking the limit of P(Z|α) as K → ∞ yields a valid probability distribution over feature ownership matrices with an infinite number of potential features.5 Note that as K → ∞, the prior on feature weights gets concentrated at zero (because α/K → 0). This results in an infinite number of columns that simply contain zeroes, and thus, these features will have no consequence for the set of stimuli we observed (as they are not assigned to any stimuli). Because both the number of columns K → ∞ and the probability of an object taking a feature (probability that znk = 1) πk → 0 at corresponding rates, there is a finite, but random, number of columns have at least one nonzero element (the features that have been taken by at least one stimulus). This limiting distribution is called the Indian buffet process (IBP; Griffiths and Ghahramani 2005, 2011), and it is given by the following equation % & N α K+ −1 n P(Z |α) = $2n −1 exp −α h=1 Kh n=1 ×
K+ (N − mk )(mk − 1) k=1
N
,
where K+ is the number of columns with at least one nonzero entry (the number of features taken by
at least one object), and Kh is the number of features with history h, where a history can be thought of as the column of the feature interpreted as a binary number. The term containing the history penalizes features that have equivalent patterns of ownership and it is a method for indexing features with equivalent ownership patterns. Analogous to the CRP, the probability distribution given by this limiting process is equivalent to the probability distribution on binary matrices implied by a simple sequential culinary metaphor. In this culinary metaphor, “customers,” corresponding to the stimuli or rows of the matrix, enter an Indian buffet and take dishes, corresponding to the features or columns of the matrix, according to a series of probabilistic decisions based on how the previous customers took dishes. When the first customer enters the restaurant, she takes a number of new dishes sampled from a Poisson distribution with parameter α. As customers sequentially enter the restaurant, each customer n takes a previously sampled dish k with probability mk /n and then samples a number of new dishes sampled from a Poisson distribution with parameter α/n. Figure 9.3(a) illustrates an example of a potential state of the IBP after three customers have entered the restaurant. The first customer entered the restaurant and sampled two new dishes from the Poisson probability distribution with parameter α. Next, the second customer entered the restaurant and took each of the old dishes with probability 1/2 and sampled one new dish from the Poisson probability distribution with parameter α/2. Then, the third customer entered the restaurant and took the first dish with probability 2/3, did not take the second dish with probability 1/3, and did not take the third dish with probability 2/3. The equivalent feature ownership matrix represented by this culinary metaphor is shown in Figure 9.3(b). As previously encountered features are sampled with probability proportional to the number of times they were previously taken and the probability of new features decays as more customers enter the restaurant (it is Poisson distributed with parameter given by α/N where N is the number of customers), the IBP favors feature representations that are sparse and have a small number of features. Thus, it encodes a natural prior expectation toward feature representations with a few features, and can be interpreted as a simplicity bias. Now that we have derived the feature ownership prior P(Z), we turn to defining the feature image
structure and flexibility in bayesian models of cognition
197
(a)
(b)
Indian Buffet Process
1
Dish 1
3
1
Dish 2 2
1
...
Dish 3 2
2
3
Customers
2
Dishes
Fig. 9.3 (a) The culinary representation of the Indian buffet process and (b) the feature ownership matrix implied by it.
prior P(Y) and the likelihood P(X|Y, Z), which will finish a specification of the nonparametric Bayesian model. Remember that the feature image matrix contains the consequences of a stimulus having a feature in terms of the raw primitives. Thus, in the nonparametric case, the feature image matrix can be thought of containing the D-dimensional consequence of each of the K+ used features. The infinite unused features can be ignored because a feature only affects the representation of a stimulus if it is used by that stimulus. In most applications to date, the feature image prior is mostly “knowledge-less.” For example, the standard prior used for stimuli that are binary images is an independent Bernoulli prior, where each pixel is on with probability φ independent of the other pixels in the image. One exception is one of the simulations by Austerweil and Griffiths (2011), who demonstrated that using a simple proximity bias (an Ising model that favors adjacent pixels to share values, Geman and Geman, 1984) as the feature image prior results in more psychologically plausible features: The feature images without using the proximity bias were not contiguous and were speckled, whereas the feature images using the proximity bias were contiguous. For grayscale and color images (Austerweil and Griffiths, 2011; Hu, Zhai, Williamson, & Boyd-Graber, 2012), the standard “knowledge-less” prior generates each pixel from a Gaussian distribution independent of the other pixels in the image. Analogous to the feature image priors, the choice of the likelihood depends on the format of the raw dimensional primitives. In typical applications, the likelihood assumes that the reconstructed stimuli is given by the product of the feature ownership and image matrices, ZY and penalizes the deviation between the reconstructed and observed stimuli (Austerweil and Griffiths, 2011). For binary images, the noisy-OR likelihood (Pearl, 1988; Wood, Griffiths, & Ghahramani, 2006) is used, which amounts to thinking of each feature as a “hidden cause” and has support as a psychological 198
higher level cognition
explanation for how people reason about observed effects being produced by multiple hidden causes (Cheng, 1997; Griffiths and Ghahramani, 2005). For grayscale images, the linear-Gaussian likelihood is typically used (Griffiths and Ghahramani, 2005; Austerweil and Griffiths, 2011), which is optimal under the assumption that the reconstructed stimuli is given by ZY and that the metric of success is the sum squared error between the reconstructed and observed stimuli. Recent work in machine learning has started to explore more complex likelihoods, such as explicitly accounting for depth and occlusion (Hu, Zhai, Williamson, & BoydGraber 2012). Formalizing more psychologically valid feature image priors and likelihoods is a mostly unexplored area of research that demands attention. After specifying the feature ownership and image prior and the likelihood, a feature representation can be inferred for a given set of observations using standard machine learning inference techniques, such as Gibbs sampling (Geman and Geman, 1984) or particle filtering (Gordon, Salmond, & Smith, 1993). We refer the reader to Austerweil and Griffiths (2013), who discuss Gibbs sampling and particle filtering for feature inference models and analyze their psychological plausibility. What features should people use to represent the image in Figure 9.4(a)? When the image is in the context of the images in Figure 9.4(b), Austerweil and Griffiths (2011) found that people and the IBP model infer a single feature to represent it, namely the object itself, which is shown in Figure 9.4(d). Alternatively, when the image is in the context of the images in Figure 9.4(c), people and the IBP model infer a set of features to represent it, which are three of the six parts used to create the images, which are shown in Figure 9.4(e).6 Importantly, Austerweil and Griffiths (2011) demonstrated that two of the most popular machine-learning techniques for inferring features from a set of images, principal component analysis (Hyvarinen,
(a)
(b)
(c)
(d)
(e)
Fig. 9.4 Effects of context on the features inferred by people to represent an image (from Experiment 1 of Austerweil & Griffiths, 2011). What features should be used to represent the image shown in (a)? When (a) is in the context of the images shown in (b), participants and the nonparametric Bayesian model represent it with a single feature, the image itself, which is shown in (d). Conversely, when (a) is in the context of the images shown in (c), it is represented as a conjunction of three features, shown in (e). Participants and the nonparametric Bayesian model generalized differently due to using different representations.
Karhunen, & Oja 2001) and independent component analysis (Hyvarinen et al., 2001), did not explain their experimental results. This suggests that the simplicity bias given by the IBP is similar to the prior expectations people use to infer feature representations, although future work is necessary to more precisely test and formally characterize the biases people use to infer feature representations. Furthermore, this contextual effect was replicated in grayscale 3-D rendered images and in a conceptual domain, suggesting that it may be a domain-general capability. Unlike the IBP and most other computational models of feature learning, people tend to use features that are invariant over transformations (e.g., translations or dilations) because properties of features observed from the environment do not occur identically across appearances (e.g., the retinal image after eye or head movements). By modifying the culinary metaphor such that each time a customer takes a dish she draws a random spice from a spice rack, the transformed IBP (Austerweil and Griffiths, 2010b) can infer features invariant over a set of transformations. Austerweil and Griffiths (2013) explain a number of previous experimental results and outline novel contextual effects predicted by the extended model that they subsequently confirm through behavioral experiments. Other extensions to this same framework have been used to explain multimodal representation learning (Yildirm and Jacobs, 2012), and in one extension, the IBP and CRP are used together to infer features diagnostic for categorization (Austerweil and Griffiths, 2013).
Choice Behavior According to feature-based choice models, people choose between different options depending on their preference for the features (called an aspect) that each option has. One influential feature-based choice model is the elimination by aspects model (Tversky, 1972). Under this model, the preference of a feature is assumed to be independent of other features and defined by a single, positive number, called a weight. Choices are made by repeatedly selecting a random feature weighted by their preference and removing all options that do not contain the feature until there is only one option remaining. For example, when a person is choosing which television to purchase, she is confronted with a large array of possibilities that vary on many features, such as IS LCD, IS PLASMA, IS ENERGY EFFICIENT, IS HIGH DEFINITION, and so on, each with their own associated weight. Because the same person may make different choices even when they are confronted with the same set of options, the probability of choosing option i over option j, pij , is proportional to the weights of the features that option i has, but option j does not have, relative to the features that option j has, but option i does not have. Formally, this is given by pij =
k wk zik (1 − zjk )
k wk zik (1 − zjk ) +
k wk (1 − zik )zjk
,
where zik = 1 denotes that option i has feature k and wk is the preference weight given to feature k.
structure and flexibility in bayesian models of cognition
199
Although modeling human choice behavior using the elimination by aspects model is straightforward when the features of each option and their preference weights are known; it is not straightforward how to infer what features a person uses to represent a set of options and their weights given only the person’s choice behavior. To address this issue, Görür et al. (2006) proposed the IBP as a prior distribution over possible feature representations and a Gamma distribution as a prior over the feature weights, which is a flexible distribution that only assigns probability to positive numbers. They demonstrated that human choice for which celebrities participants from the early 1970s one would prefer to chat with (Rumelhart and Greeno, 1971) is just as well described by the elimination by aspects model given a set of features defined by a modeler or when the features are inferred with the IBP as the prior over possible feature representations. As participants are familiar with the celebrities and the celebrities are related to each other according to a hierarchy (i.e., politicians, actors, and athletes), Miller et al. (2008) extended the IBP such that it infers features in a manner that respects the given hierarchy (i.e., options are more likely to have the same features to the degree that they are close to each other in the hierarchy). They demonstrated that the extended IBP explains human choice judgments better and uses fewer features to represent the options. Although the IBP-based extensions helps the elimination by aspects model overcome some of its issues, the extended models are unable to account for the full complexity of human choice behavior (e.g., attraction effects; Huber, Payne, & Puto, 1982). Regardless, exploring choice models that include a feature-inference process is a promising direction for future research, because such models can potentially be incorporated with more psychologically valid choice models (e.g., sequential sampling models of preferential choice; Roe, Busemeyer, & Townsend 2001).
Learning Functions: How Are Continuous Quantities Related? So far, we have focused on cases in which the latent structure to be inferred is discrete—either a category or a set of features. However, latent structures can also be continuous. One of the most prominent examples is function learning, in which a relationship is learned between two (or more) continuous variables. This is a problem that people often solve without even thinking about it, as when 200
higher level cognition
learning how hard to press the pedal to yield a certain amount of acceleration when driving a rental car. Nonparametric Bayesian methods also provide a solution to this problem that can learn complex functions in a manner that favors simple solutions. Viewed abstractly, the computational problem behind function learning is to learn a function y = f (x) from a set of real-valued observations xN = (x1 , . . . , xN ) and tN = (t1 , . . . , tN ), where tn is assumed to be the true value obscured by some kind of additive noise (i.e., tn = yn +n , where n is some type of noise). In machine learning and statistics, this is referred to as a regression problem. In this section, we discuss how this problem can be solved using Bayesian statistics, and how the result of this approach is related to a class of nonparametric Bayesian models known as Gaussian processes. Our presentation follows that in Williams (1998).
Bayesian linear regression Ideally, we would seek to solve our regression problem by combining some prior beliefs about the probability of encountering different kinds of functions in the world with the information provided by x and y. We can do this by applying Bayes’s rule, with p(tN |f , xN )p(f ) , F p(tN |f , xN )p(f ) df
p(f |xN , tN ) =
(1)
where p(f ) is the prior distribution over functions in the hypothesis space F , p(tN |f , xN ) is the likelihood of observing the values of tN if f were the true function, and p(f |xN , tN ) is the posterior distribution over functions given the observations xN and yN . In many cases, the likelihood is defined by assuming that the values of tn are independent given f and xn , and each follows a Gaussian distribution with mean yn = f (xn ) and variance σ 2 . Predictions about the value of the function f for a new input xN +1 can be made by integrating over this posterior distribution. Performing the calculations outlined in the previous paragraph for a general hypothesis space F is challenging, but it becomes straightforward if we limit the hypothesis space to certain specific classes of functions. If we take F to be all linear functions of the form y = b0 + xb1 , then we need to define a prior p(f ) over all linear functions. As these functions can be expressed in terms of the parameters b0 and b1 , it is sufficient to define a prior over the vector b = (b0 , b1 ), which we can do by assuming that b follows a multivariate Gaussian distribution with mean
zero and covariance matrix b . Applying Eq. 1, then, results in a multivariate Gaussian posterior distribution on b (see Rasmussen and Williams, 2006, for details) with −1 T XTN tN E[b|xN , tN ] = σt2 −1 b + XN XN −1 1 T −1 cov[b|xN , yN ] = b + 2 XN XN σt where XN = [1N xN ] (i.e., a matrix with a vector of ones horizontally concatenated with xN +1 ). Because yN +1 is simply a linear function of b, the predictive distribution is Gaussian, with yN +1 having mean [1 xN +1 ]E[b|xN , tN ] and variance [1 xN +1 ]cov[b|xN , tN ][1 xN +1 ]T . The predictive distribution for tN +1 is similar but with the addition of σ 2 to the variance.
Basis Functions and Similarity Kernels Although considering only linear functions might seem overly restrictive, linear regression actually gives us the basic tools we need to solve this problem for more general classes of functions. Many classes of functions can be described as linear combinations of a small set of basis functions. For example, all kth degree polynomials are linear combinations of functions of the form 1 (the constant function), x, x 2 ,. . . , x k . Letting φ (1) , . . . , φ (k) denote a set of basis functions, we can define a prior on the class of functions that are linear combinations of this basis by expressing such functions in the form f (x) = b0 + φ (1) (x)b1 + · · · + φ (k) (x)bk and defining a prior on the vector of weights b. If we take the prior to be Gaussian, we reach the same solution as outlined in the previous paragraph, substituting = [1N φ (1) (xN ) . . . φ (k) (xN )] for X and [1 φ (1) (xN +1 ) . . . φ (k) (xN +1 ) for [1 xN +1 ], where φ(xN ) = [φ(x1 ) . . . φ(xN )]T . If our goal were merely to predict yN +1 from xN +1 , yN , and xN , we might consider a different approach, by simply defining a joint distribution on yN +1 given xN +1 and conditioning on yN . For example, we might take yN +1 to be jointly Gaussian, with covariance matrix KN kN ,N +1 KN +1 = kTN ,N +1 kN +1 where KN depends on the values of xN , kN ,N +1 depends on xN and xN +1 , and kN +1 depends only on xN +1 . If we condition on yN , the distribution of yN +1 is Gaussian with mean kTN ,N +1 K−1 N y k . This and variance kN +1 − kTN ,N +1 K−1 N N ,N +1
approach to prediction is often referred to as using a Gaussian process, since it assumes a stochastic process that induces a Gaussian distribution on y based on the values of x. This approach can also be extended to allow us to predict yN +1 from xN +1 , tN , and xN by adding σt2 IN to KN , where IN is the n × n identity matrix, to take into account the additional variance associated with the observations tN . The covariance matrix KN +1 is specified using a function whose argument is a pair of stimuli known as a kernel, with Kij = K (xi , xj ). Any kernel that results in an appropriate (symmetric, positive-definite) covariance matrix for all x can be used. One common kernel is the radial basis function, with 1 K (xi , xj ) = θ12 exp − 2 (xi − xj )2 θ2 indicating that values of y for which values of x are close are likely to be highly correlated. See Schölkopf and Smola (2002) for further details regarding kernels. Gaussian processes thus provide an extremely flexible approach to regression, with the kernel being used to define which values of x are likely to have similar values of y. Some examples are shown in Figure 9.5. Just as we can express a covariance matrix in terms of its eigenvectors and eigenvalues, we can express a given kernel K (xi , xj ) in terms of its eigenfunctions φ and eigenvalues λ, with K (xi , xj ) =
∞
λk φ (k) (xi )φ (k) (xj )
k=1
for any xi and xj . Using the results from the previous paragraph, any kernel can be viewed as the result of performing Bayesian linear regression with a set of basis functions corresponding to its eigenfunctions, and a prior with covariance matrix b = diag(λ). These equivalence results establish an important duality between Bayesian linear regression and Gaussian processes: For every prior on functions, there exists a kernel that defines the similarity between values of x, and for every kernel, there exists a corresponding prior on functions that yields the same predictions. This result is a consequence of Mercer’s theorem (Mercer, 1909). Thus, Bayesian linear regression and prediction with Gaussian processes are just two views of the same class of solutions to regression problems.
structure and flexibility in bayesian models of cognition
201
B
3
3
2
2
1
1
Output (y)
Output (y)
A
0
0
−1
−1
−2
−2
−3
−1.5
−1
−0.5
0
0.5
1
1.5
Input (x)
−3
−1.5
−1
−0.5
0
0.5
1
1.5
Input (x)
Fig. 9.5 Modeling functions with Gaussian processes. The data points (crosses) are the same in both panels. (A) Inferred posterior using a radial basis covariance function (Eq. 2) with θ2 = 1/4. (B) Same as panel A, but with θ1 = 1 and θ2 = 1/8. Notice that as θ2 gets smaller, the posterior mean more closely fits the observed data points, and the posterior variance is larger in regions far from the data.
Modeling Human Function Learning The duality between Bayesian linear regression and Gaussian processes provides a novel perspective on human function learning. Previously, theories of function learning had focused on the roles of different psychological mechanisms. One class of theories (e.g., Carroll, 1963; Brehmer, 1974; Koh and Meyer, 1991) suggests that people are learning an explicit function from a given class, such as the polynomials of degree D. This approach attributes rich representations to human learners, but has traditionally given limited treatment to the question of how such representations could be acquired. A second approach (e.g., DeLosh, Busemeyer, & McDaniel, 1997) emphasizes the possibility that people could simply be forming associations between similar values of variables. This approach has a clear account of the underlying learning mechanisms, but it faces challenges in explaining how people generalize beyond their experience. More recently, hybrids of these two approaches have been proposed (e.g., Kalish, Lewandowsky, & Kruschke, 2004; McDaniel and Busemeyer, 2005). For example, the population of linear experts (POLE; Kalish et al. 2004) uses associative learning to learn a set of linear functions 202
higher level cognition
and their expertise over regions of dimensional space. Bayesian linear regression resembles explicit rule learning, estimating the parameters of a function, whereas the idea of making predictions based on the similarity between predictors (as defined by a kernel) that underlies Gaussian processes is more in line with associative accounts. The fact that, at the computational level, these two ways of viewing regression are equivalent suggests that these competing mechanistic accounts may not be as far apart as they once seemed. Just as viewing category learning as density estimation helps to understand that prototype and exemplar models correspond to different types of solutions of the same statistical problem, viewing function learning as regression reveals the shared assumptions behind rule learning and associative learning. Gaussian process models also provide a good account of human performance in function learning tasks. Griffiths et al. (2009) compared a Gaussian process model with a mixture of kernels (linear, quadratic, and radial basis) to human performance. Figure 9.6 shows mean human predictions when trained on a linear, exponential, and quadratic function (from DeLosh et al., 1997), together with
(a)
(b)
Linear
Exponential
Quadratic
Function Human/Model
Fig. 9.6 Extrapolation performance in function learning. Mean predictions on linear, exponential, and quadratic functions for (a) human participants (from DeLosh et al. 1997) and (b) a Gaussian process model with linear, quadratic, and nonlinear kernels. Training data were presented in the region between the vertical lines, and extrapolation performance was evaluated outside this region. Figure reproduced from Griffiths et al. (2009).
the predictions of the Gaussian process model. The regions to the left and right of the vertical lines represent extrapolation regions, being input values for which neither people nor the model were trained. Both people and the model extrapolate near optimally on the linear function, and reasonably accurate extrapolation also occurs for the exponential and quadratic function. However, there is a bias toward a linear slope in the extrapolation of the exponential and quadratic functions, with extreme values of the quadratic and exponential function being overestimated.
Conclusions Probabilistic models form a promising framework for explaining the impressive success that people have in solving different inductive problems. As part of performing these feats, the mind constructs structured representations that flexibly adapt to the current set of stimuli and context. In this chapter, we reviewed how these problems can be described from a statistical viewpoint. To define models that infer representations that are both flexible and structured, we described three main classes of nonparametric Bayesian models and how the format of the observed stimuli determines
which of the three classes of models should be used. Our presentation of these three classes of models is only the beginning of an ever-growing literature using nonparametric Bayesian processes in cognitive science. Each class of model can be thought of as providing primitive units, which can be composed in various ways to form richer models. For example, the IBP can be interpreted as providing primitive units that can be composed together using logical operators to define a categorization model, which learns its own features along with a propositional rule to define a category. Figure 9.7 presents some of these applications and Box 2 provides a detailed discussion of them.
Concluding remarks 1. Although it is possible to define probabilistic models that can infer any desired structure, due to the bias-variance trade-off, prior expectations over a rich class of structures are needed to capture the structures inferred by people when given a limited number of observations. 2. Nonparametric Bayesian models are probabilistic models over arbitrarily complex structures that are biased toward simpler structures.
structure and flexibility in bayesian models of cognition
203
Representational primitive
Key Psychological Application
Partitions Probability distributions over competing discrete units Probability distributions over independent discrete units Distributions over continuous units
Category assignment Chinese restaurant process (CRP) Category learning/Density estiDirichlet process (DP) mation Indian buffet process (IBP)/Beta proFeature assignment cess (BP)1
Aldous (1985)
Function learning
Rasmussen and Williams (2006)
Gaussian process (GP)
Key References
Ferguson (1973) Griffiths and Ghahramani (2005); Thibaux and Jordan (2007)
Griffiths et al. (2008b); Blei et al. (2010) Jointly learning categories and CRP + IBP, nested CRP + hierarchical Austerweil and Griffiths (2013); features DP topic model Salakhutdinov et al. (2012) CRP over entities embedded inside Cross-cutting category learning Shafto et al. (2011) CRP over attributes Product of CRPs over multiple entity Relational category learning Kemp et al. (2006, 2010) types Property induction GP over latent graph structure Kemp and Tenenbaum (2009) GP/CRP/IBP + grammar over model Kemp and Tenenbaum (2008); Domain structure learning forms Grosse et al. (2012) Hierarchical category learning
Composites
Nonparametric Bayesian Process
Nested CRP
Fig. 9.7 Different assumptions about the type of structure generating the observations from the environment results in different types of nonparametric Bayesian models. The most basic nonparametric models define distributions over core representational primitives, while more advanced models can be constructed by composing these primitives with each other and with other probabilistic modeling and knowledge representation elements (see Box 2). Typically, researchers in cognitive science do not distinguish between the CRP and DP, or the IBP and BP. However, they are all distinct mathematical objects, where the CRP and IBP are distributions over the assignment of stimuli to units and the DP and BP are distributions over the assigment of stimuli to units and the parameters associated with those units. The probability distribution given by only considering the number of stimuli assigned to each unit by a DP and BP yields a distribution over assignments equivalent to the CRP and IBP, respectively.
Thus, they form a middle ground between the two extremes of models that infer overly simple structures (parametric models) and models that infer overly complex structures (nonparametric models). 3. Using different nonparametric models result in different assumptions about the format of the hidden structure. When each stimulus is assigned to a single latent unit, multiple latent units, or continuous units, the Dirichlet process, Beta process, and Gaussian process are appropriate, respectively. These processes are compositional in that they can be combined with each other and other models to infer complex latent structures, such as relations and hierarchies.
Some Future Questions 1. How similar are the inductive biases defined by nonparametric Bayesian models to those people use when inferring structured representations? 2. What are the limits on the complexity of representations that people can learn? Are nonparametric Bayesian models too powerful? 3. How do nonparametric Bayesian models compare to other computational frameworks that adapt their structure with experience, such as neural networks? 204
higher level cognition
Notes 1. Note that we define parametric, nonparametric, and other statistical terms from the Bayesian perspective. We refer the reader interested in the definition of these terms from the frequentist perspective and a comparison of frequentist and Bayesian perspectives to Young and Smith (2010). 2. Our definition of “density estimation” includes estimating any probability distribution over a discrete or continuous space, which is slightly broader than its standard use in statistics, estimating any probability distribution over a continuous space. 3. Our formulation departs from Anderson’s by adopting the notation typically used in the statistics literature. However, the two formulations are equivalent. 4. We have written the posterior probability as proportional to the product of three terms because the normalizing constant (the denominator) for this example is intractable to compute when there is an infinite repository of features. 5. Technically, to ensure that the infinite limit of P(Z|α) is valid requires defining all feature ownership matrices that differ only in the order of the columns to be equivalent. This is due to identifiability issues and is analogous to the arbitrariness of the cluster (or table) labels in the CRP. 6. Austerweil and Griffiths (2011) tested whether people represent the objects with the parts as features by seeing if they were willing to generalize a property of the set of objects (being found in a cave on Mars) to a novel combination of three of the six parts used to create the images. See Austerweil and Griffiths (2011) and Austerweil and Griffiths (2013) for a discussion of this methodology and the theoretical implications of these results. 7. Although we collapsed the distinction between IBP and BP, they are distinct nonparametric Bayesian processes. See the caption of Figure 2 and the glossary for more details.
Glossary Beta process: a stochastic process that assigns a real number between 0 and 1 to a countable set of units, which makes it a natural prior for latent feature models (interpreting the number as a probability) bias: the error between the true structure and the average structure inferred based on observations from the environment bias-variance trade-off: to reduce the error in generalizing a structure to new observations, an agent has to reduce both its bias and variance Chinese restaurant process: a culinary metaphor that defines a probability distribution over partitions, which yields an equivalent distribution on partitions as the one implied by a Dirichlet process when only the number of stimuli assigned to each block is considered computational level: interpreting the behavior of a system as the solution to an abstract computational problem posed by the environment
particle filtering: a sequentially adapting importance sampler where the surrogate distribution is based on the approximated posterior at the previous time step partition: division of a set into nonoverlapping subsets posterior probability: an agent’s belief in a structure or hypothesis after some observations prior probability: an agent’s belief in a structure or hypothesis before any observations rational analysis: interpreting the behavior of a system as the ideal solution to an abstract computational problem posed by the environment usually with respect to some assumptions about the system’s environment rational process models: a process model that is a statistical approximation to the ideal solution given by probability theory variance: the degree that the inferred structure changes across different possible observations from the environment
consistency: given enough observations, the statistical model infers the true structure producing the observations Dirichlet process: a stochastic process that assigns a set of non-negative real numbers that sum to 1 to a countable set of units, which makes it a natural prior for latent class models (interpreting the assigned number as the probability of that unit) exchangeability: a sequence of random variables is exchangeable if and only if their joint probability is invariant to reordering (does not change) Gaussian process: a stochastic process that defines a joint distribution on a set of variables that is Gaussian, which makes it a natural prior for function learning models importance sampling: approximating a distribution by sampling from a surrogate distribution and then reweighting the samples to compensate for the fact that they came from the surrogate rather than the desired distribution Indian buffet process: a culinary metaphor for describing the probability distribution over the possible assignments of observations to multiple discrete units, which yields an equivalent distribution on discrete units as the one implied by a Beta process when only the number of stimuli assigned to each unit is considered inductive inference: a method for solving a problem that has more than one logically possible solution likelihood: the probability of some observed data given a particular structure or hypothesis is true Markov chain Monte Carlo: approximating a distribution by setting up a Markov chain whose limiting distribution is that distribution Monte Carlo: using random number sampling to solve numerical problems nonparametric: a model whose possible densities belongs to a family that includes arbitrary distributions parametric: a model that assumes possible densities belongs to a family that is parameterized by a fixed number of variables
References Aldous, D. (1985). Exchangeability and related topics. In École d’Été de Probabilités de Saint-Flour XIII, pp. 1–198. Berlin: Springer. Anderson, J. R. (1990). The adaptive character of thought. Hillsdele, NJ: Erlbaum. Anderson, J. R. (1991). The adaptive nature of human categorization. Psychological Review 98(3), 409–429. Antoniak, C. (1974). Mixtures of Dirichlet processes with applications to Bayesian nonparametric problems. The Annals of Statistics, 2, 1152–1174. Ashby, F. G. & Alfonso-Reese, L. A. (1995). Categorization as probability density estimation. Journal of Mathematical Psychology, 39, 216–233. Austerweil, J. L. & Griffiths, T. L. (2010a). Learning hypothesis spaces and dimensions through concept learning. In S. Ohlsson & R. Camtrabone, (Ed.). Proceedings of the 32nd Annual Conference of the Cognitive Science Society, 73–78. Austin, TX: Cognitive Science Society. Austerweil, J. L. & Griffiths, T. L. (2010b). Learning invariant features using the transformed indian buffet process. In R. Zemel & J. Shawne-Taylor, (Ed.). Advances in neural information processing systems (Vol. 23, pp. 82–90. Cambridge, MA. MIT Press. Austerweil, J. L. & Griffiths, T. L. (2011). A rational model of the effects of distributional information on feature learning. Cognitive Psychology, 63, 173–209. Austerweil, J. L. & Griffiths, T. L. (2013). A nonparametric Bayesian framework for constructing flexible feature representations. Psychological Review, 120, 817–851. Bernardo, J. M. & Smith, A. F. M. (1994). Bayesian theory. New York, NY: Wiley. Bickel, P. J. & Doksum, K. A. (2007). Mathematical statistics: basic ideas and selected topics. Upper Saddle River, NJ: Pearson. Blackwell, D. & MacQueen, J. (1973). Ferguson distributions via Polya urn schemes. The Annals of Statistics, 1, 353–355.
structure and flexibility in bayesian models of cognition
205
Blei, D. M., Griffiths, T. L., & Jordan, M. I. (2010). The nested Chinese restaurant process and Bayesian nonparametric inference of topic hierarchies. Journal of the ACM, 57, 1–30. Bouton, M. (2004). Context and behavioral processes in extinction. Learning & Memory, 11(5):485–494. Bowers, J. S. & Davis, C. J. (2012). Bayesian just-so stories in psychology and neuroscience. Psychological Bulletin, 138(3):389–414. Braddick, O. (1993). Segmentation versus integration in visual motion processing. Trends in neurosciences, 16(7), 263–268. Brehmer, B. (1971). Subjects’ ability to use functional rules. Psychonomic Science, 24, 259–260. Brehmer, B. (1974). Hypotheses about relations between scaled variables in the learning of probabilistic inference tasks. Organizational Behavior and Human Decision Processes, 11, 1–27. Brogden, W. (1939). Sensory pre-conditioning. Journal of Experimental Psychology, 25(4), 323–332. Carroll, J. D. (1963). Functional learning: The learning of continuous functional mappings relating stimulus and response continua. Princeton, NJ: Education Testing Service. Chater, N., Goodman, N., Griffiths, T. L., Kemp, C., Oaksford, M., & Tenenbaum, J. B. (2011). The imaginary fundamentalists: The unshocking truth about Bayesian cognitive science. Behavioral and Brain Sciences, 34(4), 194– 196. Chater, N. & Oaksford, M. (2008). The probabilistic mind: Prospects for Bayesian cognitive science. New York, NY: Oxford University Press. Cheng, P. (1997). From covariation to causation: A causal power theory. Psychological Review, 104, 367–405. Clapper, J. & Bower, G. (1994). Category invention in unsupervised learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20(2), 443–460. Courville, A., Daw, N., & Touretzky, D. (2006). Bayesian theories of conditioning in a changing world. Trends in Cognitive Sciences, 10(7), 294–300. Dayan, P., Kakade, S., & Montague, P. R. (2000). Learning and selective attention. Nature Neuroscience, 3, 1218–1223. DeLosh, E. L., Busemeyer, J. R., & McDaniel, M. A. (1997). Extrapolation: the sine qua non for abstraction in function learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23(4), 968–986. Ferguson, T. (1973). A Bayesian analysis of some nonparametric problems. The Annals of Statistics, 1, 209–230. Freedman, D. & Diaconis, P. (1983). On inconsistent Bayes estimates in the discrete case. Annals of Statistics, 11(4), 1109–1118. Geman, S., Bienenstock, E., & Doursat, R. (1992). Neural networks and the bias-variance dilemma. Neural Computation, 4, 1–58. Geman, S. & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741. Gershman, S., Blei, D., & Niv, Y. (2010). Context, learning, and extinction. Psychological Review, 117(1), 197–209. Gershman, S. & Niv, Y. (2012). Exploring a latent cause theory of classical conditioning. Learning & Behavior, 40(3), 255– 268.
206
higher level cognition
Gershman, S. J. & Blei, D. M. (2012). A tutorial on bayesian nonparametric models. Journal of Mathematical Psychology, 56(1), 1–12. Ghosal, S. (2010). The Dirichlet process, related priors, and posterior asymptotics. In N. L. Hjort, C. Holmes, P. Müller, & S. G. Walker, (Eds.), Bayesian nonparametrics (pp. 35–79). Cambridge UK, Cambridge University Press. Goldmeier, E. (1972). Similarity in visually perceived forms. Psychological Issues, 8, 1–136. Original written in German and published in 1936. Goldwater, S., Griffiths, T. L., & Johnson, M. (2009). A Bayesian framework for word segmentation: Exploring the effects of context. Cognition, 112, 21–54. Goodman, N. (1972). Seven strictures on similarity. In N. Goodman, (Ed.), Problems and projects. New York, NY: Bobbs-Merrill. Gordon, N., Salmond, J., & Smith, A. (1993). A novel approach to non-linear/non-Gaussian Bayesian state estimation. IEEE Proceedings on Radar and Signal Processing, 140, 107–113. Görür, D., Jäkel, F., & Rasmussen, C. E. (2006). A choice model with infinitely many latent features. In Proceedings of the 23rd International Conference on Machine Learning (ICML 2006), pages 361–368, New York. ACM Press. Griffiths, T., Chater, N., Kemp, C., Perfors, A., & Tenenbaum, J. (2010a). Probabilistic models of cognition: exploring representations and inductive biases. Trends in Cognitive Sciences, 14, 357–364. Griffiths, T., Steyvers, M., & Tenenbaum, J. (2007). Topics in semantic representation. Psychological Review, 114(2), 211–244. Griffiths, T. L. (2010). Bayesian models as tools for exploring inductive biases. In Banich, M. & Caccamise, D., editors, Generalization of knowledge: Multidisciplinary perspectives. Psychology Press, New York. Griffiths, T. L., Chater, N., Kemp, C., Perfors, A., & Tenenbaum, J. B. (2010b). Probabilistic models of cognition: exploring representations and inductive biases. Trends in Cognitive Sciences, 14(8), 357–364. Griffiths, T. L., Chater, N., Norris, D., & Pouget, A. (2012). How the Bayesians got their beliefs (and what those beliefs actually are): Comment on Bowers and Davis (2012). Psychological Bulletin, 138(3), 415–422. Griffiths, T. L. & Ghahramani, Z. (2005). Infinite latent feature models and the Indian buffet process. (Technical Report 2005-001, Gatsby Computational Neuroscience Unit). Griffiths, T. L. & Ghahramani, Z. (2011). The Indian buffet process: An introduction and review. Journal of Machine Learning Research, 12, 1185–1224. Griffiths, T. L., Kemp, C., & Tenenbaum, J. B. (2008a). Bayesian models of cognition. In R. Sun, (Ed.). Cambridge handbook of computational cognitive modeling. Cambridge, England: Cambridge University Press. Griffiths, T. L., Lucas, C., Williams, J. J., & Kalish, M. L. (2009). Modeling human function learning with Gaussian processes. In Y. Bengio, D. Schuurmans, J. D. Lafferty, C. K. I. Williams & A. Culotta, (Eds.). Advances in Neural Information Processing Systems 21, pp. 556–563. Red Hook, NY: Curran. Griffiths, T. L., Sanborn, A. N., Canini, K. R., & Navarro, D. J. (2008b). Categorization as nonparametric Bayesian
density estimation. In N. Chater, & M. Oaksford, (Eds.). The probabilistic mind. Oxford, England: Oxford University Press. Griffiths, T. L. & Yuille, A. (2006). A primer on probabilistic inference. Trends in Cognitive Sciences, 10(7), 1–11. Grosse, R. B., Salakhutdinov, R., Freeman, W. T., & Tenenbaum, J. B. (2012). Exploiting compositionality to explore a large space of model structures. In N. de Freitas & K. Murphy, (Eds.). Conference on Uncertainty in Artificial Intelligence, pp. 306–315. Corvallis, OR: AUAI Press. Hjort, N. L. (1990). Nonparametric Bayes estimators based on Beta processes in models for life history data. Annals of Statistics, 18, 1259–1294. Hu, Y., Zhai, K., Williamson, S., & Boyd-Graber, J. (2012). Modeling images using transformed Indian buffet processes. International Conference of Machine Learning, Edinburgh, UK. Huber, J., Payne, J. W., & Puto, C. (1982). Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal of Consumer Research, 9(1), 90–98. Hyvarinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York, NY: Wiley. Jaynes, E. T. (2003). Probability theory: The logic of science. Cambridge, England: Cambridge University Press. Jolliffe, I. T. (1986). Principal component analysis. New York, NY: Springer. Jones, M. & Love, B. C. (2011). Bayesian fundamentalism or enlightenment? on the explanatory status and theoretical contributions of Bayesian models of cognition. Behavioral and Brain Sciences, 34, 169–231. Kahneman, D., Slovic, P., & Tversky, A., editors (1982). Judgment under uncertainty: Heuristics and biases. Cambridge, England: Cambridge University Press. Kalish, M. L., Lewandowsky, S., & Kruschke, J. K. (2004). Population of linear experts: knowledge partitioning and function learning. Psychological Review, 111(4), 1072–1099. Kaplan, A. & Murphy, G. (1999). The acquisition of category structure in unsupervised learning. Memory & Cognition, 27(4), 699–712. Kemp, C., Perfors, A., & Tenenbaum, J. B. (2007). Learning overhypotheses with hierarchical bayesian models. Developmental Science, 10(3), 307–321. Kemp, C., Tenenbaum, J., Niyogi, S., & Griffiths, T. (2010). A probabilistic model of theory formation. Cognition, 114(2), 165–196. Kemp, C. & Tenenbaum, J. B. (2008). The discovery of structural form. Proceedings of the National Academy of Sciences, 105(31), 10687–10692. Kemp, C. & Tenenbaum, J. B. (2009). Structured statistical models of inductive reasoning. Psychological Review, 116(1), 20–58. Kemp, C., Tenenbaum, J. B., Griffiths, T. L., Yamada, T., & Ueda, N. (2006). Learning systems of concepts with an infinite relational model. In Y. Gil & R. J. Mooney, (Eds.). Proceedings of the 21st National Conference on Artificial Intelligence, pp. 381–388. Menlo Park, CAAAAI Press. Koh, K. & Meyer, D. E. (1991). Function learning: induction of continuous stimulus–response relations. Journal of Experimental Psychology: Learning, Memory, and Cognition, 17(5), 811–836.
Kruschke, J. (2008). Bayesian approaches to associative learning: From passive to active learning. Learning & Behavior, 36(3), 210–226. Love, B. C., Medin, D. L., & Gureckis, T. M. (2004). SUSTAIN: A network model of category learning. Psychological Review, 111, 309–332. Marr, D. (1982). Vision. San Francisco, CA: WH Freeman. McClelland, J. L., Botvinick, M. M., Noelle, D. C., Plaut, D. C., Rogers, T. T., Seidenberg, M. S., & Smith, L. B. (2010). Letting structure emerge: connectionist and dynamical systems approaches to cognition. Trends in Cognitive Sciences, 14(8), 348–356. McDaniel, M. A. & Busemeyer, J. R. (2005). The conceptual basis of function learning and extrapolation: Comparison of rule-based and associative-based models. Psychonomic Bulletin & Review, 12(1), 24–42. McKinley, S. C. & Nosofsky, R. M. (1995). Investigations of exemplar and decision bound models in large, ill-defined category structures. Journal of Experimental Psychology: Human Perception and Performance, 21(1), 128–148. Medin, D. L. & Schaffer, M. M. (1978). Context theory of classification learning. Psychological Review, 85, 207–238. Mercer, J. (1909). Functions of positive and negative type and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society A, 209, 415– 446. Miller, K. T., Griffiths, T. L., & Jordan, M. I. (2008). The phylogenetic Indian Buffet Process: A non-exchangeable nonparameteric prior for latent features. In D. McAllester & P. Myllymaki, (Eds.). Proceedings of the Twenty-Fourth Conference on Uncertainity in Artificial Intelligence, (pp. 403– 410). Corvallis, Oregon: AUAI Press. Murphy, G. L. & Medin, D. L. (1985). The role of theories in conceptual coherence. Psychological Review, 92, 289–316. Neal, R. M. (2000). Markov chain sampling methods for dirichlet process mixture models. Journal of Computational and Graphical Statistics, 9(2), 249–265. Nosofsky, R. M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115(1), 39–57. Oaksford, M. & Chater, N. (2007). Bayesian rationality: The probabilistic approach to human reasoning. New York, NY: Oxford University Press. Palmer, S. E. (1999). Vision Science. Cambridge, MA: MIT Press. Pearl, J. (1988). Probabilistic reasoning in intelligent systems. San Francisco, CA: Morgan Kaufmann. Posner, M. I. & Keele, S. W. (1968). On the genesis of abstract ideas. Journal of Experimental Psychology, 77(3p1), 353. Pothos, E. & Chater, N. (2002). A simplicity principle in unsupervised human categorization. Cognitive Science, 26(3), 303–343. Rasmussen, C. E. & Williams, C. K. (2006). Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press. Reed, S. K. (1972). Pattern recognition and categorization. Cognitive Psychology, 3, 393–407. Rescorla, R. A. & Wagner, A. R. (1972). A theory of of Pavlovian conditioning: variations in the effectiveness of reinforcement and nonreinforcement. In A. Black, & W. Prokasy, (Eds.), Classical conditioning II: Current research
structure and flexibility in bayesian models of cognition
207
and theory (pp. 64–99). New York, NY: Appleton-CenturyCrofts. Robert, C. P. (1994). The Bayesian choice: A decision-theoretic motivation. New York, NY: Springer. Roe, R. M., Busemeyer, J. R., & Townsend, J. T. (2001). Multialternative decision field theory: A dynamic connectionist model of decision making. Psychological Review, 108(2), 370–392. Rumelhart, D. & Greeno, J. (1971). Similarity between stimuli: An experimental test of the Luce and Restle choice models. Journal of Mathematical Psychology, 8, 370–381. Salakhutdinov, R., Tenenbaum, J. B., & Torralba, A. (2012). Learning to learn with compound hierarchicaldeep models. In Advances in Neural Information Processing Systems. Sanborn, A. N., Griffiths, T. L., & Navarro, D. J. (2010). Rational approximations to rational models: alternative algorithms for category learning. Psychological Review, 117(4), 1144–1167. Schölkopf, B. & Smola, A. J. (2002). Learning with kernels: Support vector machines, regularization, optimization, and beyond. MIT Press, Cambridge, MA. Shepard, R. N. (1987). Towards a universal law of generalization for pshychological. Science. 237, 1317–1323. Selfridge, O. G. & Neisser, U. (1960). Pattern recognition by machine. Scientific American, 203, 60–68. Shafto, P., Kemp, C., Manishka, V., & Tenenbaum, J. B. (2011). A probabilistic model of cross-categorization. Cognition, 120(1), 1–25.
208
higher level cognition
Tenenbaum, J. B., Griffiths, T. L., & Kemp, C. (2006). Theorybased Bayesian models of inductive learning and reasoning. Trends in Cognitive Science, 10:309–318. Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011). How to grow a mind: Statistics, structure, and abstraction. Science, 331, 1279–1285. Thibaux, R. & Jordan, M. I. (2007). Hierarchical Beta processes and the Indian buffet process. In Eleventh International Conference on Artificial Intelligence and Statistics (AISTATS 2007), pages 564–571. Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79, 281–299. Tversky, A. (1977). Features of similarity. Psychological Review, 84, 327–352. Werker, J. & Yeung, H. (2005). Infant speech perception bootstraps word learning. Trends in Cognitive Sciences, 9(11), 519–527. Williams, C. K. (1998). Prediction with gaussian processes: From linear regression to linear prediction and beyond. In M. Jordan, (Ed.). Learning in graphical models (pp. 599–621). Cambridge, MA: MIT Press. Wood, F., Griffiths, T. L., & Ghahramani, Z. (2006). A nonparametric Bayesian method for inferring hidden causes. Proceedings of the 22nd Conference in Uncertainty in Artificial Intelligence (UAI ’06), 536–543. Yildirm, I. & Jacobs, R. A. (2012). A rational analysis of the acquisition of multisensory representations. Cognitive Science, 36, 305–332. Young, G. A. & Smith, R. L. (2010). Essentials of statistical inference. Cambridge, UK: Cambridge University Press.
CHAPTER
10
Models of Decision Making under Risk and Uncertainty
Timothy J. Pleskac, Adele Diederich, and Thomas S. Wallsten
Abstract
Formal models have a long and important history in the study of human decision-making. They have served as normative standards against which to compare real choices, as well as precise descriptions of actual choice behavior. This chapter begins with an overview of the historical development of decision theory and rational choice theory and then reviews how models have been used in their normative and descriptive capacities. Models covered include prospect theory, rank- and sign-dependent utility theories and their descendants, as well as cognitive models of human decision-making like Decision Field Theory and the Leaky Competing Accumulator Model, which are based on basic psychological principles rather than assumptions of rationality. Key Words: subjective probability, utility, rational choice models, choice axioms, cognitive
model, expected utility theory, subjective expected utility theory, prospect theory, decision field theory, risk , uncertainty, independence, transitivity, stochastic dominance
Introduction Formal models serve three separate but intertwined roles in the study of human decisionmaking.1 First, rational choice models, especially when posed as sets of axioms, serve as standards against which to compare real choices made by sentient beings. Indeed, many of the axioms can be cast as putative empirical laws to be tested in suitable experiments (Krantz, 1972). Such comparisons invoke a second, closely related role of formal models, which is to describe actual behavior. Finally, to the extent that a descriptive model is accurate, it provides a means for measuring the values of latent constructs underlying choice. This chapter provides an overview of formal decision and choice models in the first two of these three roles. For illustrations of the third role, see the chapter by Pleskac (in press), which includes examples of how descriptive models of choice have been used to measure
latent constructs involved in decision-making with the aim of characterizing decision-making deficits associated with clinical and neurological disorders. We begin with a historical introduction to the rational choice models ofexpected utility theory (EUT; von Neumann & Morgenstern, 1944) and subjective expected utility theory (SEUT; Savage, 1954), followed by a discussion of empirical tests of the underlying axioms. Descriptive failures of the axioms laid the groundwork for prospect theory, rank- and sign-dependent utility theories, and their descendants. Following a presentation of these newer models, which represent generalizations of EUT and SEUT, we turn to formal models rooted in cognitive psychology that represent alternative approaches to understanding decision behavior (Busemeyer & Diederich, 2010). For conciseness, we refer to these as cognitive models of choice or decisionmaking.2 209
Box 1 Types of uncertainty The terms risk and uncertainty in the title of this chapter arise from a distinction that economists make between risky and uncertain situations, the former when probabilities of the outcomes in a situation are known (and not 0 or 1) and the latter when the probabilities are not well specified (Knight, 1921; Luce & Raiffa, 1957). Applicable rational models are EUT in the former case and SEUT in the latter. A related distinction often found in the behavioral literature is between precise and ambiguous probabilities, the former applying when probability distributions are described or well learned from the environment and the latter when they are not (Ellsberg, 1961). The term ambiguous is really a misnomer in this context, in that something is ambiguous if it can take on one of a number of well-defined meanings, which is not what is intended when decision researchers refer to ambiguity or ambiguous probability. The terms vague or imprecise would be more appropriate, but generally have not taken hold in this literature. See Budescu and Wallsten (1987) for further discussion. A third distinction, this one made by philosophers and risk analysts, is between aleatory and epistemic uncertainty, the former when the uncertainty is due to stochastic factors in the environment and the latter when it is due to lack of knowledge (Hacking, 1975).
Historical Development of Decision Theory The academic study of decision-making under risk can trace its origins to the 18th century when the gentleman-gambler Chevalier de Méré asked prominent mathematicians of the time for a solution to what became known as the Problem of Points (Hacking, 1975; Todhunter, 1865). In the problem, two people are playing a game of chance with a goal of reaching a pre-determined set of points to win a pot of money. Méré was curious about how the stakes sould be divided between the players if the game ended before either one reached the threshold. Blaise Pascal and Pierre de Fermat took up this problem, and in their correspondence, Pascal developed the basic ideas of what would later be recognized as probability theory to solve the puzzle (Hacking, 1975). 210
higher level cognition
Pascal’s solution was that the players should be paid based on their expected earnings based on their play so far, which eventually led to an expectedvalue theory of decision-making. This is the idea that gambles and risky alternatives should be valued according to their expected value and ultimately one should choose according to this value.3 For example, consider an individual deciding whether to take Gamble A1 or A2 , either of which pays off on the basis of a coin flip. The situation is illustrated in the payoff matrix in Table 10.1, in which the rows are the alternatives between which the decision maker (DM) must choose (A1 or A2 ) and the columns are the states of nature that might occur (coin lands heads, s1 , or tails, s2 ). Each state has a probability pj of occurring. The values in the cells cij are the cash payoffs that the individual receives when choosing alternative i and state sj occurs. Which alternative should the DM choose? According to expected value theory, letting p(sj ) denote the probability of state sj , the DM should choose the alternative with the largest expected value, v(Ai ), calculated as p sj · cij . (1) v (Ai ) = j
Structuring the decision problem in this payoff matrix illustrates a common technique in modeling risky decisions, which is to represent a risky decision as a choice between two or more well-defined gambles (Coombs, Dawes, & Tversky, 1970; Savage, 1954). Without a doubt, this quantitative representation facilitated the development of normative and descriptive theories of risky choice. Distilling risky choice into these crucial dimensions also provides a small, constrained world in which one can ask how actual DMs use the dimensions of payoffs and probabilities to make a decision. For example, nearly 100 years after Pascal and Fermat discussed the problem of the points, Daniel Bernoulli (1954; originally 1738) used another lottery puzzle, the St. Petersburg paradox, to demonstrate why valuing a gamble—and by extension a risky alternative—by its objective expectation does not seem quite right. In the St. Petersburg gamble, a fair coin is tossed Table 10.1. Payoff matrix States of nature
Alternatives
Head (s1 )
Tail (s2 )
Gamble A1
c11
c12
Gamble A2
c21
c22
until tails appears. At this point, the player is paid a sum equal to 2n ducats (a gold coin) where n is the number of tosses that occurred. Note that, by substituting in Eq. 1, the expected value of this gamble, ∞ 0.5j · 2j , v (G) = j=1
is infinite. The paradox was, why does intuition suggest that “any fairly reasonable man would sell his chance, with great pleasure, for twenty ducats” (Bernoulli, 1954, p. 31) when its expected value is infinite? Bernoulli’s insight was that the operative values were not the cash amounts, c, but their “moral value,” or what today commonly is termed their utility, u(c). He persuasively argued that the moral value (utility) of wealth is a concave downward function of ducats (monetary value), representing the idea that as wealth increases, there is a diminishing return to its utility. Specifically, based on reasonable assumptions about human behavior, he deduced that the utility function must be logarithmic. This insight can resolve the paradox because taking the expectation of this (logarithmic) moral value leads to a finite price for the St. Petersburg gamble. In other words, according to Bernoulli, the value to the DM of the St. Petersburg gamble G was its expected utility, u(G), calculated as ∞ 0.5j · u 2j . u (G) = j=1
More generally, and moving beyond Bernoulli’s log function, Eq. 1 is modified to p sj · u cij . u (Ai ) = (2) j
Thus, subjective values of wealth entered into economic theory in the form of EUT via lotteries. This concept of a subjective value of a physical stimulus influenced later psychological theorists like Fechner, Weber, and Thurstone (Masin, Zudini, & Antonelli, 2009). Brilliant as it was, a problem with Bernoulli’s expected utility principle was that it explained a phenomenon ex post. Moreover, the explanation depended crucially on the untestable assumption that DMs made choices that maximized expected utility. In a development that provides the modern form of EUT, von Neumann and Morgenstern (1944) provided an ex ante justification based on first principles. Focusing on monetary gambles, they proposed a set of qualitative (i.e., nonnumerical) axioms4 such that if a decision maker’s pattern of choices over selected pairs of lotteries
satisfied the axioms, then a utility value, unique up to a positive linear transformation,5 could be assigned to each outcome, and the DM can be said to have chosen so as to maximize expected utility as represented in Eq. 2, but without the restriction that u(cij ) is a log function or, indeed, that the cij even have cash values at all. Savage (1954) generalized EUT to situations in which the probabilities are not explicitly given, which Knight (1921) called decisions under uncertainty.6 His solution, known as SEUT, provided the axiomatic basis for rational choice among lotteries with unstated or unknown probabilities. A DM whose choices are consistent with Savage’s axioms is acting as though she assigns utilities to outcomes, personal numerical probabilities to the relevant possible states of the world, and is choosing so as to maximize subjectively expected utility. Both von Neumann and Morgenstern and Savage proposed axiom sets that can be taken as bases for rational choice models, not as foundations of models of human behavior. Nevertheless, these developments opened up a larger set of issues concerning whether humans (or other animals) make choices that correspond with the axioms of EUT and SEUT or whether alternative theories are required to describe behavior. This question has been the entry point for the development of models that better describe the choices people actually make under risk and uncertainty. A prominent approach has been to generalize the expected utility expression in Eq. 2. Another approach has been more computational in nature. Instead of relying on maximization models that consist of, or are generalizations of, rational choice models, cognitive models of choice tend to build on psychologically plausible processes that may govern how DMs deliberate among choice alternatives over time. Following a brief discussion of axiom tests, we review these various approaches and their implications.
Empirical Tests of Choice Axioms This chapter is not the place for an extensive review of the experimental literature on the adequacy of the choice axioms as descriptive of human behavior, but a few examples are useful as illustrations of this work, which has so profoundly affected decision modeling. We consider the axioms of independence and of transitivity, because they are utterly foundational to rational models and necessary for a pattern of choices to be considered rational under any axiomatic definitions.7 Readers desiring more complete coverage of this issue should
models of decision making under risk and uncertainty
211
consult Rapoport and Wallsten (1972) for an early discussion of the empirical status of the axioms and Mellers, Schwartz, and Cooke (1998) and Shafir and Le Boeuf (2002) for more recent reviews. The report by MacCrimmon and Larsson (1979) contains a very thorough summary of the von Neumann and Morgenstern as well as the Savage axioms, along with empirical tests of them. For a discussion of the merits of testing decisionmaking theories via critical tests of their axioms, see Birnbaum (2011).
Independence axioms Independence axioms come in many forms, but they all capture the principle that adding identical outcomes to choice alternatives should not alter the direction of preference between the alternatives. One version is Savage’s (1954) sure-thing principle, which states that if gambles A and B provide the same outcome contingent on state sj , then the DM’s preference between A and B is independent of that outcome (Wakker, 2010, p. 115). Maurice Allais provided an early and powerful test of the sure-thing principle with the Allais Paradox (Allais, 1953, 1979). The basic design is illustrated in Table 10.2. Note that Gamble A can be derived from A and B can be derived from B by in each case adding an 89% chance of winning $1M. Thus, according to the sure-thing principle, the directions of preference should be the same for Choices 1 and 2, i.e., either A B and A B or B A and B A .8 To see that EUT (and SEUT) requires precisely one of these two choice patterns, observe that if, e.g., A B, then according to EUT, Eq. 2 with u(0) = 0, it must be the case that u (A) > u (B) =⇒u (1M ) > 0.89u (1M ) + 0.10u(5M ).
Table 10.2. Two choice situations that illustrate the Allais paradox Choice 1
Choice 2
Gamble A * $1M 100% Gamble A ’ Gamble B
$1M $0 $5M
$1M 11% $0 89%
89% Gamble B ’* 1% $0 90% 10% $5M 10%
* = Majority choice, M = Million
212
higher level cognition
Subtracting 0.89u ($1M) from both sides of the inequality yields 0.11u ($1M ) > 0.10u ($5M ) =⇒ u A > u B , implying that A B . But, in fact, the most commonly observed pattern is A B and B A , which is inconsistent with the predictions of EUT (e.g., MacCrimmon & Larsson, 1979). Although the existence, magnitude and even the direction of the Allais effect can be manipulated (e.g., Barron & Erev, 2003), its overall robustness calls into question the generality of EUT or SEUT as descriptive theories of choice. Kahneman and Tversky (1979) introduced prospect theory as a generalization of EUT in response to this and other choice patterns that violate the normative rational models.
The axiom of transitivity Preferences among options A, B, and C are said to be transitive if and only if (A B and B C) =⇒ A C. It is easy to see that Eq. 2 implies transitivity among all triples of options. If it systematically fails, then EUT and SEUT cannot hold, nor for that matter can any model that assumes or implies a weak ordering of preferences. One would think that empirically testing transitivity is a straightforward matter. After all, what could be simpler than establishing a triple of preferences? The difficulty arises from the fact that choice, as with so much of behavior, is stochastic, so that when confronted with the same pair of options, A and B, on two or more ostensibly identical occasions, an individual will not necessarily choose the same option each time. Stochastic behavior often conflicts with the deterministic nature of the axioms, causing statistical difficulties when testing the descriptive validity of the axioms.9 There have been numerous attempts to develop statistical techniques especially suited to axiom testing, but until recently (Regenwetter, Dana, & Davis-Stober, 2011) none has proved satisfactory for a variety of deep reasons (Luce, 1995). We return to Regenwetter et al. shortly, but note first that, early on, researchers distinguished three levels of stochastic transitivity, strong (SST), moderate (MST) and weak (WST). Letting P(i j) denote the probability that i is preferred to j, assume that P (A B) > 0.5 and P (B C) > 0.5. Then, • SST holds when P (A C) > max [P (A B) , P (B C) ] ,
• MST holds when P (A C) > min [P (A B) , P (B C)] ,
and • WST holds when P (A C) > 0.5.
By relaxing the assumptions of Thurstone (1927) Case V Law of Comparative Judgment, Morrison (1963) developed 42 distinct choice models, which differed, among other ways, in the form of stochastic transitivity that each implied. Within that framework, Morrison (1963) pointed out that for n > 3 options in a choice set, it is impossible for preferences to be intransitive for all triples of options. And in fact, as n increases, the maximum proportion of triples that can be intransitive approaches the expected proportion given random responding, making actual tests very difficult. This result caused Morrison (1963) to conclude that a proper demonstration of intransitivity requires one to determine in advance which triples of options will produce the violations. Tversky (1969), in what has become a classic paper, took up that challenge and demonstrated systematic intransitivities precisely where he expected to see them based on an additive-difference choice model. In contrast to expected utility types of models, this model assumed that DMs compare options dimension by dimension, for example, first compare the probabilities, then compare the outcomes, and so on, weight the difference on each dimension according to a weighting function, sum the transformed differences, and choose accordingly. Depending on the nature of the weighting functions, this model can be rendered indistinguishable from SEUT, or it can be very different indeed, to the point of predicting intransitivities. Tversky (1969) constructed sets of five gambles, each displayed on a card as a circular spinner radially divided into a black and a white sector of proportions p and 1 − p, respectively, with a positive dollar amount, x, above the black sector. Thus, the card represented the gamble “win $x with probability p, otherwise nothing.” From cards a through e in a set, values of p increased by units of 1/24, say from 7/24 to 11/24 of the circle, whereas payoffs decreased by small amounts, say from $5.00 to $4.00 in steps of $0.25. Tversky assumed that when presented with pairs of cards, DMs would first compare black sectors, that is, perceived values of p. If they were sufficiently different, then the DM would choose the gamble with the greater probability.
Otherwise, the DM would choose according to the value of x. This lexicographic semi-order (LS) choice rule is a particular form of the additivedifference model and predicted a certain pattern of intransitive choices. Specifically, Tversky (1969) proposed that probabilities in adjacent cards, a and b, b and c, and so on, would be perceptually so similar that respondents would consider them essentially equal and compare the outcomes. As the outcomes were expressed numerically and easily distinguished, they would provide the basis for the choice between adjacent cards in the set. Sufficiently nonadjacent cards, say a and e, would differ noticeably in perceptual probabilities and, therefore, they would provide the basis for choice. Hence a pattern of choices might be a ≺ b, b ≺ c, c ≺ d , d ≺ e but e ≺ a. Using participants pretested for a propensity toward such circular choices, Tversky (1969) used maximum-likelihood chi-squared tests to compare the likelihoods of observed choice patterns under LS and under WST. He interpreted the data to strongly favor the former. This conclusion has predominated despite the fact that Tversky preselected his subjects, leaving open the possibility that the results might not generalize, and that Iverson and Falmagne (1985) pointed out problems with the data analysis stemming from the inappropriateness of the chi-squared distribution in this case. When they used the proper asymptotic distribution, the pattern of results, with one exception, became not significantly different from that expected under WST. Subsequently, Myung, Karabatsos and Iverson (2005) used a Bayesian approach to find greater support for Tversky’s (1969) model than for a variety of other reasonable models. Thus, although it appears that Tversky (1969) obtained the predicted pattern of intransitivities, the supporting data might be best considered as weakly supporting violations of transitivity. Returning now to Regenwetter et al. (2011) they discuss in depth various shortcomings in earlier formulations of tests of transitivity. Their work subsumes and goes substantially beyond Iverson and Falmagne (1985) in proposing new theoretical, statistical, and empirical approaches to investigating the descriptive validity of transitivity and other choice axioms. Using the developments in Regenwetter et al. (2011), Regenwetter, Dana, and Davis-Stober (2010) provided data in support of a particular mixture model of transitive preference, the essence of which is that an individual may have multiple transitive preference states, only
models of decision making under risk and uncertainty
213
one of which is active at any given moment. A set of observed choices, therefore, likely arises from many different states. In this formulation, the preference states, and not the choices they imply, are stochastic and define the parameter space. This model has a good deal of structure and, therefore, is highly constrained, rendering only a small fraction of logically possible response patterns consistent with it. In careful individuallevel analyses of data from 18 respondents, they found little support for intransitive preference states. We, in turn, must conclude that when subjected to rigorous tests, transitivity has not been shown to fail. This is a fortunate outcome, because it is implied by most decision models.
Generalizing Rational Choice Models One common response to the violation of the independence axiom and others has been to seek ways to generalize the rational models to account for human behavior. Indeed Luce (2000) suggested, “. . . any descriptive theory of decision making should always include as special cases any locally rational theory for the same topic” (p. 108). Prospect theory (Kahneman & Tversky, 1979; Tversky & Kahneman, 1992) has been perhaps the most successful theory in this spirit. At a very general level, prospect theory not only introduced psychologically more plausible assumptions about the properties of the utility function10 , but it also did away with the linear treatment of probabilities in estimating the value of risky options. Kahneman and Tversky (1979) initially expressed prospect theory in the form of a probability weighting function, generalizing Eq. 2 to (3) w[p sj ] · u cij . u (Ai ) = j
Next, we examine how Kahneman and Tversky generalized the utility and weighting functions first in terms of prospect theory and then in terms of cumulative prospect theory to better account for people’s choices. For more complete coverage of these generalizations readers should consult Luce (2000) and Wakker (2010).
Probability weighting function The assumption of linear sensitivity to probabilities in EUT and SEUT largely relegates all descriptive power to the utility function, ascribing, for example, risk aversion to the concavity of the utility function. However, starting with Preston and 214
higher level cognition
Baratta (1948) psychologists explicitly entertained the hypothesis that an expectation-based descriptive model of risky choice required both a utility function and a function describing the psychological value of probabilities (see also Edwards, 1954). Kahneman and Tversky’s (1979) prospect theory made perhaps the best case for the probability weighting function by illustrating that nonlinear probability weighting can account for the Allais paradox. To see how, according to Eq. 3, the preference of A B in choice 1 in Table 10.2 implies u (A) > u (B) =⇒u ($1M ) > w (0.89) · u ($1M ) + w (0.10) · u($5M ). With some algebra, the right-hand side of the arrow can be rewritten as u ($1M ) w(0.10) > . u($5M ) 1 − w(0.89) The preference B A in choice 2 implies u(B ) > u(A ) =⇒w (0.10) · w ($M ) > w (0.11) · u ($1M ) . The right-hand side of the arrow here can be rewritten as w(0.10) u ($1M ) > w(0.11) u($5M ) Thus, these two inequalities imply 1 > w (0.11) + w(0.89). Kahneman and Tversky (1979) referred to this relationship as subcertainty and it implies that w(p) is regressive with respect to p. That is, the Allais paradox would imply that the weighting function for intermediate levels of probabilities is less sensitive to variations of probability than dictated by the expected utility model. There are other properties of the weighting function besides the subcertainty property. For example, one of the more common properties is overweighting of rare events. Kahneman and Tversky (1979) demonstrated this property when they asked participants to choose between a risky gamble of $5,000 with a probability of 0.001 and a sure thing of $5. Participants expressed a preference for the risky gambles, much as is observed with people playing the lottery. Assuming concavity of the utility function, this preference implies overweighting of rare events: w(0.001) >u(5)/u(5,000) > 0.001. These gambles as well as
w(p)
A 1.0
B
u(c)
.5
0
c
0
.5 p
1.0
Fig. 10.1 Prospect theory’s inverse-S probability weighting function (Panel A) and utility function with loss aversion (Panel B).
others lead to the conclusion that the probability weighting function on average takes an inverse Sshape like the one shown in Figure 10.1A. Analyses at the individual level also support this conclusion (Gonzalez & Wu, 1999). It is important to note that investigations into the shape of the probability weighting function have primarily focused on decisions under risk. An important question is what is the shape of the weighting function for decisions under uncertainty, when events do not have explicitly stated probabilities? Tversky and Fox (1995) showed that the inverse-S shape of the probability weighting function generalized to decisions under uncertainty. In particular, they found that, in evaluating the worth of gambles in domains like sporting events, the stock market, and temperatures, events that turned an impossibility into a possibility or a possibility into a certainty had greater impact than when the same event made a possibility more or less likely (see also Tversky & Wakker, 1995). Psychologically, Tversky and Fox (1995) interpreted these results as consistent with a two-stage process for making decisions under uncertainty (see also Fox & Tversky, 1998). In their theory, people first estimate the probability of an uncertain event based on the support it receives relative to the alternative events. This support can be understood as a measure of the strength of evidence in favor of the event arrived at via memory search or some other heuristic process (Tversky & Koehler, 1994). The probability estimate then is transformed into a decision weight during the decision process. Recently, psychologists have been examining an alternative approach to making decisions under uncertainty, whereby, instead of searching memory
to form a probability estimate, DMs form the estimate by sampling directly from the available alternatives. This process has been called decisions from experience (Barron & Erev, 2003; Hertwig, Barron, Weber, & Erev, 2004). One interesting empirical consequence of making decisions in this manner is that instead of rare events being overweighted, as they appear to be when probabilities are described, they appear to be underweighted, that is, to have less impact on choice than they deserve according to their objective probabilities. In other words, the probability weighting function seems to take a different form for decisions made from experience than from description (see Box 2).
Box 2 Experience-Based Decision Making Often DMs do not have convenient descriptions of the probabilities and payoffs associated with different choice options. Instead they must use their experience with the options to form a statistical estimate of these values. A natural question is do decisions made from descriptions differ from decisions made on the basis of samples of information, decisions from experience? Choice 1
Choice 2
Gamble A
$4 $0
80% 20%
Gamble A’
$4 $0
20% 80%
Gamble B
$3
100% Gamble B’
$3 $0
25% 75%
To investigate this question, it is necessary to investigate choices in description and experience
models of decision making under risk and uncertainty
215
Box 2 Continued conditions. In the description condition, people are presented with verbal descriptions of options as in Choice 1. In the experience condition, the descriptions of the gambles are replaced with computerized buttons. Clicking either button produces a random draw from the respective unknown gamble. This change in presentation impacts preferences. For example, when faced with Choice 1 in the description condition approximately 70 to 80% of people choose the sure thing (Option B). In comparison, when the same gambles are learned via experience, then 70 to 80% of people prefer the gamble (Option A) (see for example Hertwig et al., 2004). Moreover, when the probabilities of the nonzero payoffs are divided by 4 (see Choice 2), then people in the description condition tend to prefer Option A , but people in the experience condition tend to prefer Option B . Because the probabilities between the two choices are a common ratio of each other preferences should be consistent, so that, for example, if A is preferred to B then A should be preferred to B . Thus, the choice patterns in both the description and experience conditions are each violations of the independence axiom of expected utility theory. Like the Allais paradox the switch in preference can be understood in terms of a non-linear probability weighting function. However, they imply differently shaped functions: Choices in the description condition imply overweighting of rare events (Kahneman & Tversky, 1979) and the choices in the experience condition imply underweighting of rare events (Hertwig et al., 2004). A question of great interest is what can account for this apparent gap between description and experience. Part of the gap is due to the sampling error that occurs in the experience condition (Fox & Hadar, 2006). However, there are now several studies that show even after reducing sampling error (Hau, Pleskac, & Hertwig, 2010) or controlling for it (Ungemach et al., 2009) the gap remains. The source of the gap is still an active area of research, but current research points to learning and memory processes, estimation error of the event probabilities, and format dependence of the decision algorithm used (see Hertwig & Erev, 2009).
216
higher level cognition
Utility function Prospect theory also introduced several psychological principles regarding the utility function. One, which is foundational, is that the function is defined over gains and losses relative to a reference point (usually the status quo—see Panel B of Figure 10.1), not over total final wealth. For example, consider the two choice problems in Table 3 given to two different groups of participants. A majority of the participants who received Choice 1 chose the sure thing, Gamble A. In contrast, a majority of the participants who received Choice 2 chose the risky option, Gamble D. As one can see, if participants made their choice based on final wealth, as required by the utility theories, the choices should have been the same. Instead the actual choices imply that the DMs treated the outcomes as gains or losses relative to a reference point (in this case, and often, the status quo). Although historically the valuation of outcomes has been often modeled as changes from a reference point, disregarding initial wealth, prospect theory formally included this property as a core principle (see Wakker, 2010, p. 234; Luce, 2000, p. 1). In prospect theory, the distinction between gains and losses brought with it two further psychological implications in terms of the shape of the utility function. One is that the function is concave over gains and convex over losses (see Panel B in Figure 1), reflecting the hypothesis that as the magnitude of gains and losses increase, people become increasingly less sensitive to changes. Originally, support for this hypothesis came from choices among gambles like the one in Table 10.4, which Kahneman and Tversky (1979) presented to participants. Using the model in Eq. 3, the majority choices among these gambles implies that u(6, 000) < u(4, 000) + u(2, 000) and u(−6, 000) > u( − 4, 000) + u( − 2, 000). However, as we will see shortly, this interpretation of the preferences in terms of Eq. 3 has given way to a slightly different version called rank dependent utility models. The rank dependent models do not allow such a clean interpretation of preferences in Table 10.4 (see Luce, 2000, p. 82). Nevertheless, analyses of a wide range of situations reveal that there is a mixture of concave and convex utility functions over gains and losses, but the most common pattern is concave over gains and convex over losses (Fishburn & Kochenberger, 1979). A second property that emerges from prospect theory’s distinction between gains and losses is loss aversion, the hypothesis that losses loom
Table 10.3. Two Choice Situations Used to Illustrate Reference Dependence Choice 1 In addition to whatever you own, you have been given $1,000
Choice 2 In addition to whatever you own, you have been given 2,000. You are now asked to choose between
Gamble A*
$500
100%
Gamble C
$-500
100%
Gamble B
$1000 $0
50% 50%
Gamble D*
$-1000 $0
50% 50%
* = majority choice
larger than the equivalent gains in risky choices. Thus, as Kahneman and Tversky (1979) argued, “. . . most people find symmetric bets of the form (x, .50; −x, .50) distinctly unattractive” (p. 279). Formally, loss aversion results in the utility function having a steeper slope for losses than gains (see Figure 1 Panel B). Perhaps the strongest evidence in support of loss aversion comes from the endowment effect, in which individuals demand significantly more to sell a good that they own than they would be willing pay to buy the same good when they do not own it (Kahneman, Knetsch, & Thaler, 1990). In risky contexts, the certainty equivalents people place on mixed gambles suggest loss aversion (Tom, Fox, Trepel, & Poldrack, 2007; Tversky & Kahneman, 1992). However, the evidence is a bit more mixed when participants are given the option to choose between symmetric bets. Battalio, Kagel, and Jiranyakul (1990) reported mixed results with a majority of people choosing the gamble over a zero change alternative when the gamble was for $10, but more people choosing the zero change option when the gamble was for $20 (see also Ert & Erev, 2008). Such mixed evidence has led to calls for alternative views of losses as more modulators of attention, rather than some sort of loss aversion in the valuation of prospects (Yechiam & Hochman, 2013).
Cumulative Prospect Theory A potential drawback of original prospect theory or any other theory that assumes a subadditive weighting function is that they predict violations of stochastic dominance. Gamble A stochastically dominates Gamble B when A’s probability distribution over outcomes is as favorable or more favorable than (i.e., dominates) B’s. Formally, for two nonidentical gambles A and B, A stochastically dominates B if and only if P (x > t | A) P(x > t|B) for all t, where P(x > t|A) is the probability that an outcome of Gamble A exceeds t. Prospect theory originally described choices as occurring in two phases: first editing and then evaluation. The editing phase provides for, among other operations, the removal of dominated alternatives from consideration. Without this editing phase, violations in stochastic dominance can occur. For example, consider the following numerical example from Weber (1994) shown in Table 10.5. Gamble A stochastically dominates Gamble B. However, by allowing subadditivity in the weighting function, prospect theory permits w(0.8) > w(0.4) + w(0.4). Consequently, so long as the difference between w(0.8) and w(0.4) + w(0.4) outweighs the greater utility difference between $200 and $210, prospect theory will predict a greater value for Gamble B than A and predict a violation in stochastic dominance. More generally, violations of stochastic dominance can occur when
Table 10.4. Gambles illustrating diminishing sensitivity in gains and losses. Choice 1 Gamble A
Choice 2
$6000 25% Gamble C* $-6000 25% $0 75% $0 75%
Gamble B* $4000 25% Gamble D $2000 25% $0 50% * = majority choice
$-4000 25% $-2000 25% $0 50%
Table 10.5. Gambles illustrating prospect theory’s violation of stochastic dominance. Gamble A
$210 $200 $0
40% 40% 20%
Gamble B
$200 $0
80% 20%
models of decision making under risk and uncertainty
217
the decision weights for a set of outcomes do not sum to 1 (Fishburn, 1978). Empirically there is evidence that when dominance is not transparent, people do make choices that violate stochastic dominance(Birnbaum & Navarrete, 1998; Lopes, 1984; Mellers, Weiss, & Birnbaum, 1992; Tversky & Kahneman, 1986). In fact, later in this chapter, we review how decision field theory predicts violations of stochastic dominance, and empirical data support the prediction. Nevertheless, a choice theory that does not violate stochastic dominance seemed desirable. One reason was that stochastic dominance (as well as transitivity) is among the properties that in the words of Quiggin (1982) commanded “. . . virtually unanimous assent even among those who violated them” (p. 325). A second reason was that adherence to dominance principles is assumed by a large body of empirically supported economic theories of decision-making under certainty, and thus a theory of decision-making under risk and uncertainty that was consistent with these theories also seemed useful. On a separate note, when there are a large number of outcomes or even a continuous distribution over outcomes, the application of a weighting function applied to individual event probabilities is cumbersome. These reasons led to the development of rank-dependent utility theories (Quiggin, 1982). The key idea behind rank-dependent utility theories is that the weighting function is not applied to the single event probabilities but to the (de)cumulative probabilities, that is, the probability of obtaining a given outcome or worse (better) from a specific alternative. The transformed (de)cumulative probabilities are then used to determine the weight each outcome receives (i.e., the marginal contribution of each outcome) in assessing the value of the alternative. Cumulative prospect theory is one instance of a rank dependent utility theory (Tversky & Kahneman, 1992). For a broader review of rank dependent utility theories see Luce (2000). Cumulative prospect theory applies rank-depen dent transformation separately to gains and losses, when they are present in a specific gamble, and then sums the two to form an overall evaluation of a gamble. More formally, suppose the possible outcomes for Gamble A are x1 ≤ · · · ≤ xk ≤ 0 ≤ xk+1 ≤ · · · ≤ x?, , which occur with probability p1 , . . . pk , pk+1 , . . . , pn . According to cumulative prospect theory, the value of A is k n u (A) = πi− · u (xi ) + π + · u (xi ). i=1 i=k+1 i (4) 218
higher level cognition
The decision weights πi− and πi+ are found according to the following formulas π1− = w− p1 , πi− = w− p1 + · · · + pi − w− p1 + · · · + pi−1 , 2 ≤ i ≤ k πn+ = w+ pn , πi+ = w+ pi + · · · + pn − w+ pi+1 + · · · + pn , k + 1 ≤ i ≤ n. The probability weighting functions w− and w+ are the same inverse S-shaped functions shown in Figure 1 Panel A. In words, as mentioned earlier, according to cumulative prospect theory, gains and losses are evaluated separately. For losses, the cumulative probabilities (i.e., the probability of obtaining an outcome as bad or worse) are transformed via a probability weighting function w− . Then, the decision weights are estimated by taking the difference between the transformed cumulative probability of obtaining the outcome of i and of obtaining i −1. For gains, the decumulative probabilities (i.e., the probability of obtaining an outcome as least as good as a specific value) are first transformed and then the decision weights are found via the difference between the transformed decumulative probability of obtaining outcome i and outcome i + 1. In the end, the decision weights πi− and + πi can be understood as controlling the marginal contribution of each outcome to the overall value of the alternative. When the probability weighting functions w− and w+ are identity functions, then the decision weights are the probabilities associated with each outcome. Thus, expected utility theory (Eq. 2) is a special case of cumulative prospect theory (Eq. 4). The psychological intuition behind this generalization is that the impact an outcome has to the overall value of the risky alternative is a function of both the probability of the outcome occurring and its rank relative to the other possible outcomes (Diecidue & Wakker, 2001). To see how cumulative prospect theory works, consider the gambles in the Allais paradox (see Table 10.1). The majority preference in Choice 1 implies u (A) > u (B) =⇒ w+ (1) · u ($1M ) > w+ (0.10) ·
u ($5M ) + w+ (0.99) − w+ (0.10) ·
u ($1M ) + w+ (1.0) − w+ (0.99) · u (0) . First, note that the right-hand side of the inequality demonstrates that the decision weights sum to 1, π + (0.10) + π + (0.89) + π + (0.01)
= w+ (0.10) + w+ (0.99) − w+ (0.10)
+ w+ (1.0) − w+ (0.99) +
= w (1.0) = 1.0 Thus, revealing how cumulative prospect theory and rank dependent utility theories obey stochastic dominance. Even with this constraint, cumulative prospect theory can account for the Allais paradox. It does so via the nonlinearity of the probability weighting function. To see how, the preceding inequality can be rewritten so that u ($1M ) w+ (0.10) > + u ($5M ) w (1.0) − w+ (0.99) + w+ (0.10) Choice 2 in the Allais paradox implies u B > u A =⇒ w+ (0.10) · u ($5M ) > w+ (0.11) · u ($1M ) . This inequality in turn implies w+ (0.10) u ($1M ) > w+ (0.11) u($5M ) Thus, according to the majority preferences in Choice 1 and Choice 2 of the paradox w+ (0.10) w+ (0.10) > w+ (0.11) w+ (1.0) − w+ (0.99) + w+ (0.10) which in turn implies +
w+ (0.11) − w+ (0.10) < w (1.0) − w+ (0.99) . In words, cumulative prospect theory accounts for the Allais paradox by allowing the change from w+ (0.99) to w+ (1.0) to exceed the change from w+ (0.10) to w+ (0.11), attributing the Allais paradox and the certainty effect to the steepness of the weighting function at p = 1. This property can obviously occur with the convex component of the inverse S-shape of the probability weighting function. Cumulative prospect theory has been tested empirically in different fashions. One approach has been to explore the shapes of the utility and weighting functions (Gonzalez & Wu, 1999; Tversky & Kahneman, 1992). A second approach has been to further test critical properties of the theory. For tests of an independence type assumption that rank-dependent theories like cumulative prospect theory make, see Wakker, Erev, and Weber (1994). A specific focus has been given to contexts in which people tend to, in fact, violate stochastic dominance (Birnbaum, 2004, 2008c; Diederich & Busemeyer, 1999). These violations are consistent with alternative descriptive accounts of choice.
One of those accounts is a cognitive model of decision-making called decision field theory (DFT; Busemeyer & Townsend, 1993), which will be described in the next section. Another account that successfully predicts specific violations of stochastic dominance is a different generalization of expected utility theory from that described earlier called the transfer of attention exchange model (TAX; Birnbaum & Naverette, 1998). The TAX model treats the utility of an alternative as a weighted average of the utilities of the possible outcomes. Similar, to cumulative prospect theory, the weights are a function of both the probability of obtaining the specific outcome and the rank of that outcome, with lower valued outcomes having some of the weight transferred from higher ranked outcomes. For more details, see Birnbaum (2008b). The more important point is that although cumulative prospect theory appears to provide a fairly comprehensive account of a range of decisionmaking phenomena, there still is room for improvement.
Cognitive Models of Decision Making Now, we review some cognitive models of human decision-making under risk and uncertainty. Rather than being grounded in compact sets of axioms, these models are developed from cognitive process theories. Consequently, the variety of models mirrors the variety of available approaches and theories. One cognitive approach to understanding decision-making has been via choice heuristics. These accounts began primarily as verbal theories (see Shah & Oppenheimer, 2008), but increasingly have been implemented as computational and/or mathematical models. Examples of computational implementation of heuristic choice theories are by Brandstätter, Gigerenzer, & Hertwig, 2006; Gigerenzer, Hertwig, & Pachur, 2011; Payne, Bettman, & Johnson, 1993; Thorngate, 1980; examples of mathematical models are by Rieskamp (2008) and Tversky (1972). For discussion of issues that arise in testing these models, see Brandstätter, Gigerenzer, & Hertwig (2008); Birnbaum (2008a); Johnson, Schulte-Mecklenbeck, & Willemsen (2008); Pachur, Hertwig, Geigerenzer, & Brandstätter (2013) and Rieger & Wang (2008). Dynamic decision models represent theories of how decision processes unfold over time. Examples include decision-by-sampling models (Stewart, 2009; Stewart, Chater, & Brown, 2006) and more all-encompassing models based on complete cognitive architectures such as ACT-R (e.g., Gonzalez
models of decision making under risk and uncertainty
219
Decision Field Theory Decision field theory (DFT) can be understood as a dynamic generalization of the constituents of the expectation framework (Eq. 2). To see this, we start with a binary choice problem similar to the one presented in Table 10.1, each choice characterized by two attributes, with possible consequences c, for example, winning c11 or losing c12 when choosing alternative A1 and winning c21 or losing c22 when choosing alternative A2 . The subjective values of the consequences are denoted as m cij = mij . Note that these subjective values are very similar to utility values. However, we use different notation to denote the fact that these values are not grounded in an axiomatic system. The subjective probability pij assigned to an event associated with consequence cij , or more precisely, its probability weighting function, w pij , is interpreted psychologically as the amount of attention given to the possible consequence. Importantly, the attention weight is assumed to be a random variable W and moreover, a dynamic random variable. The idea is that during the course of making a decision, the DM focuses attention on different attributes or events; the amount of attention during deliberation is indexed by W. This dynamic process of deliberation gives rise to a sequential sampling process according to which the subjective values of the consequences mij are accumulated sequentially over time until a preset criterion is reached and a response is initiated. Figure 10.2 shows the process for three trials. The graphs are called trajectories and they reflect the randomness in accumulation process toward preset criteria for alternatives A and B. More formally, we represent the momentary attention allocated to attribute j at time t as Wj (t). Attention is focused at time t on attribute 1 if 220
higher level cognition
Criterion for choosing alternative A Accumulation Process P(t)
& Dutt, 2011). For the remainder of this chapter, we will focus on one specific dynamic decision model called decision field theory (DFT; Busemeyer & Townsend, 1993). We focus on this model for several reasons. First, as we will show shortly, DFT can be understood as a reinterpretation of the earlier expectation-based models (Eq. 2). Second, DFT goes beyond the expectation models and predicts distributions of response times, and it can be extended to account for other measures of preference such as willingness to sell or buy. Finally, the DFT framework is arguably the most comprehensive framework for modeling choice covering not only decisions under risk and uncertainty, but also decisions under certainty.
0
time (t)
Criterion for choosing alternative B Fig. 10.2 Sequential sampling process to describe the dynamic process of deliberation when deciding between two choice alternatives A and B. Each trajectory refers to the preference process of a single trial. Note, that for one trial A is chosen, for another B is chosen and for the third trial no decision criterion has yet been reached.
W1 (t) > W2 (t) and if one moment (t + τ ) later, W2 (t + τ ) > W1 (t + τ ), then attention is focused on attribute 2. It is easy to see that with the Wj as random variables, Ui(t) = W1 (t) · mi1 + W2 (t) · mi2
(5)
is the stochastic version of the classic weighted additive utility model (Eq. 2), that is, the attention weights are stochastic rather than deterministic and thus illustrates how DFT can be understood as a dynamic generalization of the expectation models described earlier. The expected
of the atten values tion weights over time, E Wj (t) , correspond to the deterministic weights of the weighted additive model, wj , and Equation 5 can be written as wj (t) · mij + εj (t), (6) Ui(t) = j
where εj refers to the random component. Alternative A1 is chosen over alternative A2 if the value (utility) for A1 is higher than the value for A2 , that is, if the difference in value for alternatives A1 and A2 , V = U1 − V2 , is positive. If the difference is negative, alternative A2 is chosen. In DFT the difference in value, called valence, changes from moment to moment: V (t) = U1 (t) − U2 (t). A positive (negative) valence V (t) indicates an advantage (disadvantage) for alternative A1 under the current focus of attention. The valence is integrated over time into a preference state (t + τ ).11 That is, the difference in utility is not static as for the previous models but evolves dynamically. This dynamic property
allows DFT to account for response times in addition to choices. It is also consistent with recent behavioral and neurological data that suggests an evidence accumulation process underlies decisionmaking (see for example Gold & Shadlen, 2007; Krajbich & Rangel, 2011; Liu & Pleskac, 2011). DFT also specifies how memory and motivational processes can impact the preference state. It does so by weighting the previous preference state P (t) in the accumulation process by a constant γ , such that P (t + τ ) = (1 − τ γ ) P (t) + V (t + τ ) . Thus, the new preference state P (t + τ ) is a weighted combination of the previous state of preference and the new input valence. When γ = 0 then the model is a standard drift diffusion process. When 0 < γ τ < 1, then the accumulation process slows down (relative to when γ = 0) because the increments to the preference state have less impact with increasing position in the preference state. This can be understood as a recency effect or avoidance behavior or both. Technically, when 0 < γ τ < 1, this process is an Ornstein-Uhlenbeck process with drift (e.g., Bhattacharya & Waymire, 1990). When γ τ < 0, then the stochastic process speeds up producing primacy effects or approach behavior or both.12 For more details on separating the memory and motivational components see Busemeyer and Townsend (1993). An important characteristic of any informationprocessing system is its stopping rule: How and when does one stop information search and make a choice (see also Townsend & Ashby, 1983)? Two stopping rules (and some combinations of them) have been proposed for dynamic systems of choice. One is a fixed stopping time, also called externally controlled decision threshold, in which the time to make the decision is predetermined, for example, a deadline. The other is an optional stopping time, also called internally controlled decision threshold, in which the DM makes a choice when preference meets or exceeds the threshold, P(t) ≥ |θ |. Alternative A1 is chosen when P(t) ≥ θ and alternative A2 is chosen when P(t) ≤ −θ . In this case, the decision time is a random variable, and, thus, DFT predicts choice proportions and response-time distributions. Mathematical formulas for computing choice probabilities and distributions of response times have been derived (see Busemeyer & Townsend, 1992; Diederich & Busemeyer, 2003). The model can also be simulated with Monte Carlo
computer simulation methods to generate predictions. DFT has been applied to a variety of traditional decision making problems including decisionmaking under risk and uncertainty (Busemeyer & Townsend, 1993), selling prices and certainty equivalents and preference reversal phenomena (Busemeyer & Goldstein, 1992; Johnson & Busemeyer, 2005). DFT has been extended to account for phenomena that arise in multiattribute (Diederich, 1997) and multialternative (Roe, Busemeyer, & Townsend, 2001) choice problems including changes in preference under time pressure (Diederich, 2003) and violation of stochastic dominance (Diederich & Busemeyer, 1999) shown next.
Example: Multi-attribute Decision Field Theory (Predicted and Observed Violations in Stochastic Dominance) Most decision alternatives are not unidimensional, for example, purely monetary. Instead most alternatives are made up of multiple, sometimes conflicting, dimensions. Keeney and Raiffa (1993) provided an axiomatic foundation for extending expected utility theories to these situations. Cognitive models like DFT also can be extended to these more realistic choice alternatives and, in fact, doing so reveals new predictions about where rational models fail. For example, Diederich and Busemeyer (1999) showed an extension to multi-attribute alternatives that reveals systematic violations in stochastic dominance. To illustrate the violation, consider the two hypothetical medical decisions in Table 6. Each decision offers a choice between two treatments whose outcomes depend on the state of the world that occurs. Focusing first on the Situation 1, one can see that almost certainly in terms of preference x1 = 190 days of recovery; $255, 000 cost ≺ x2 = 180 days of recovery; $250, 000 cost ≺ x3 = 149 days of recovery; $5, 500 cost ≺ x4 = 7 days of recovery; $5, 000 cost . We also know the probability of obtaining a consequence as good as xi or better for treatment A, GA , and treatment B, GB . They are as follows: GA (x1 ) = GB (x1 ) = 1.0, GA (x2 ) = 1.0 > GB (x2 ) = 0.5, GA (x3 ) = GB (x3 ) = 0.5, GA (x4 ) = 0.5 > GB (x4 ) = 0.
models of decision making under risk and uncertainty
221
Thus, treatment A stochastically dominates treatment B. The same is true for Situation 2. Because the two situations are identical with respect to the probabilities of the preference-ordered list, cumulative prospect theory (Eq. 4) and other rankdependent utility theories predict that in both cases treatment A should be chosen over B. However, upon closer inspection, the situations seem psychologically quite different: Situation 1 appears more conflicted than Situation 2. That is because in Situation 1 the treatments are negatively correlated in terms of the outcomes that can be obtained for a given state. For example, for state S1 , treatment A has low cost and low recovery times, whereas treatment B has high cost and high recovery times and vice versa for state S2 . This negative correlation between outcomes would suggest that if DMs focus on a particular state to evaluate the two treatments, then they would be conflicted between the two possible actions. In comparison, in Situation 2 the actions are positively correlated in terms of the outcomes that can be obtained for a given state so that, for Si , treatments A and B had both low (high) cost and short (long) recovery times. This would imply that, if DMs focus on a given state to evaluate the possible treatments, then the choice will be straightforward regardless of the state that occurs and, thus, the DM should experience less conflict in this situation. Multi-attribute DFT, in fact, instantiates this idea of conflict and predicts that stochastic dominance is perfectly satisfied for Situation 2 (positively correlated, no-conflict, condition) but violations are expected for Situation 1 (negatively correlated, high-conflict, condition). This is because in DFT the valence V (t) fluctuates as the DM changes attention from moment to moment, switches back and forth between the states S1 and S2 , from attribute “treatment cost” to attribute “recovery duration” during the deliberation process. In Situation 1, the valence V (t) changes back and forth from positive to negative valence as attention switches from state S1 , favoring alternative A, to state S2 , favoring alternative B. In Situation 2, the valence is always such that it favors alternative A. An experiment with real consequences (money to win or lose the duration of a noisy sound) and probabilities learned from experience showed that stochastic dominance was indeed violated for Situation 1 but not for Situation 2, as predicted by the model (Diederich & Busemeyer, 1999). Moreover, in the same experiment, response times for the choices were collected and multi-attribute 222
higher level cognition
DFT was shown to account for the distributions of response times as well. This ability to provide an account not only of choice but also of response times demonstrates an added advantage of this dynamic approach to choice over and above the static theories like prospect theory and cumulative prospect theory.
Decision Field Theory for multialternative choice problems DFT’s grounding in basic cognitive processes also allows it to be extended to multi-attribute choice during decisions under certainty, thus providing an account of not only decisions made under risk and uncertainty but also decisions under certainty. This extension also shows how DFT can be extended to model choices with more than two alternatives. To see how, we start with a binary choice situation with optional stopping times. The decision process for a binary choice situation stops when P (t + τ ) ≥ |θ|, with Alternative 1 chosen if P (t + τ ) ≥ θ and Alternative 2 chosen if P (t + τ ) ≤ −θ .This assumption is equivalent to assuming that the accumulation process is represented by a two-dimensional diffusion process of the form (cf. Heath, 1981) P1 (t + τ ) = [ (1 − τ γ1 ) P1 (t) − τ γ2 P2 (t) ] + V1 (t + τ ) for choosing Alternative 1 and P2 (t + τ ) = [−τ γ2 P1 (t) + (1 − τ γ1 ) P2 (t)] + V2 (t + τ )
(7)
for choosing Alternative 2. Each choice alternative is evaluated relative to the other one, and the valences for the two alternatives sum to V1 + V2 = 0. Accordingly, for a choice problem with n > 2 alternatives the valence for alternative i is produced by contrasting the utility (Eq. 5) of alternative i against the average utility of the remaining n − 1 alternatives, n−1 k =i Uk (t) . (8) Vi (t) = Ui (t) − n−1 The valence vector V can then be written as the product of three matrices, V (t) = CMW (t) , where the elements of C are defined as cii = 1 and 1 for i ≤ j. The matrix M contains the cij = n−1 subjective values mij of alternative i on attribute j, and the matrix W(t) contains the momentary attention weights allocated to attribute j at
Table 10.6. Two choice situations with different correlations between outcomes. State Situation
Treatment
S1 (p = 0.50)
S2 (p = 0.50)
1: Negatively correlated outcomes
A
$ 5,000 cost 7 days of recovery
$ 250,000 cost 180 days of recovery
B
$ 255,000 cost 190 days of recovery
$ 5,500 cost 14 days of recovery
A
$ 5,000 cost 7 days of recovery
$ 250,000 cost 180 days of recovery
B
$ 5,500 cost 14 days of recovery
$ 255,000 cost 190 days of recovery
2: Positively correlated outcomes
time t. The preference state at time t + τ for all choice alternatives can be written as P (t + τ ) = S · P (t) + V (t + τ ) , with S = [I − τ ], where I is the identity matrix and is the matrix containing the γ ’s. The diagonal elements of S provide, for instance, memory for previous states of the system (see earlier). The offdiagonal values allow for competitive interactions among competing alternatives that are a function of the distance between the alternatives in a psychological space (see Hotalling, Busemeyer, & Li, 2010). See Busemeyer and Diederich (2002) and Diederich and Busemeyer (2003) for derivations. The extended DFT adds theoretical power as it reveals how the same deliberation system can account for multi-alternative context effects such as similarity, compromise, and attraction effects.
Box 3 A Connectionist Interpretation of Decision Field Theory DFT can be interpreted as a connectionist model with four layers (Roe et al., 2001). The first layer, the input, corresponds to the evaluations of each alternative on each attribute (M), feeding into a second layer via the attentional weights (W), and activation is determined by Eq. 6. Note that, in this context, at any time step, wj is either 1 or 0, depending on which attribute the decision maker attends to. The third layer computes the valences by comparing the
weighted value for alternativei with the average of the other (n − 1) alternatives (C) via Eq. 8. Since the weights are random variables and fluctuate over time, the valences for the choice alternatives at any time fluctuate as well. The fourth layer is a recursive network with positive self-recurrence for each choice alternative and negative lateral inhibition between choice alternatives (S), and competition between choice alternatives takes place here. Figure 4 shows the architecture of this network. The preference state for alternative i from a set of n alternatives is computed according to (compare Equation 7) Pi (t +τ ) = sii ·Pi (t)−
n−1
sik · Pk (t) + Vi (t + τ ) ,
k =i
(9) where sii reflects self-recurrence (the decay or leakage) and sik lateral inhibition. Note, thestrength of the lateral inhibition connection is a decreasing function of the dissimilarity (distance) between a pair of alternatives. See Hotaling, Busemeyer, and Li (2010) for a specification of the function.
There are now several different dynamic decision models that model decision-making as a sequential sampling process. Another strong competitor to DFT is the leaky competing accumulator (Usher & McClelland, 2001; Usher & McClelland, 2004). The model shares a number of similar assumptions with DFT (see Box 4). More recently, three other models have been proposed that also share similar
models of decision making under risk and uncertainty
223
Attribute 2
at the third layer of LCA, the advantage/disadvantage of alternative i relative to all remaining alternatives j, j ≤ i, Ii , is F (dij ) + I0 , (10) Ii =
S A D
++ + 0
C
–
j =i
B –– –
+ 0 Attribute 1
Fig. 10.3 Choice alternatives characterized by two attributes. Weights
Contrasts
Valences
Preferences
A
A
A
B
B
B
C
C
C
1
2
Fig. 10.4 Connectionist network for decision field theory. The first layer represents the subjective values of the consequences, mij , for the i = A, B, C alternatives with the j = 1, 2 attributes. In the fourth layer inhibition takes place symbolized by the circled headed lines.
properties: The 2N -ary choice tree model for N -alternative preferential choice (Wollschläger & Diederich, 2012), the association accumulation model (Bhatia, 2013), and the multi-attribute linear ballistic accumulator (MLBA) model (Trueblood, Brown, & Heathcote, 2014). For details and a comparison of some of these models, see Marley and Regenwetter (in press).
Box 4 The Relationship Between the Leaky Competing Accumulator Model and Decision Field Theory The leaky competing accumulator model (LCA) (Usher & McClelland, 2001; Usher & McClelland, 2004) shares many assumptions with DFT, but differs from it on a few important points. The model is defined as a four-layered connectionist network as well. The first two layers of both models are identical to DFT, and the main differences between DFT and LCA are with respect to the third (i.e., the evaluation of alternatives) and fourth layer (i.e., the dynamics of the choice process). Figure 5 shows the architecture of the network. Each of the choice alternatives serves as a reference point in evaluating each of the other alternatives, and
224
higher level cognition
where dij is the advantage or advantage of alternative i to alternative j with respect to the chosen attribute (dimension) (only one dimension is active at a time, compare to wj = 1 or 0 in DFT), F is a nonlinear function accounting for loss aversion and I0 is a positive constant promoting the alternatives in the choice process and preventing Ii of inferior alternatives to become too negative. The function F is consistent with Tversky and Kahneman’s (1991) referencedependent model and Tversky and Simonson’s (1993) context-advantage model in which losses and disadvantages have greater impact on preferences than gains and advantages. In particular, F (x) =
log (1 + x) , −[log (1 + |x|) + log (1 + |x|) 2 ],
x>0 x<0 (11)
with (x > 0) for relative advantages and (x < 0) for relative disadvantages. The fourth layer determines the response activation (the preference state in DFT) for each choice alternative following an iterative procedure: Ai (t + 1) = λAi (t) + (1 − λ) × Ii(t) − β
j =i
Aj (t) + ξi(t) ,
where λ is a neural decay constant (the leakage), β is a global inhibition parameter, and ξ is a noise term. LCA also accounts simultaneously for the multi-alternative context effects, that is, similarity, compromise and attraction shown as an example later. To see the similarity and difference between DFT and LCA, we give an example for choosing alternative A1 among three possible choice alternatives. For convenience we take the DFT notation: DFT 1 (t + 1) = s11 P1 (t) − s12 P2 (t) − s13 P3 (t) +
Box 4 Continued V1 (t + 1) = (1 − γ11 ) P1 (t) − (1 − γ12 ) P2 (t) − (1 − γ13 ) P3 (t) + U1 (t + 1) −
The positive correlation between the similar alternatives A and S lead to high activations for both and therefore, the choice between them are shared. For a formal implementation of the effects into the model and numerical examples see Usher and McClelland (2004).
U2 (t + 1) + U3 (t + 1) 2
and LCA1 (t + 1) = γ P1 (t) + (1 − γ ) {V 1 (t) − β[P2 (t) + P3 (t) ] + ε1 (t)} = γ P1 (t) − (1 − γ ) βP2 (t) − (1 − γ ) βP3 (t) + (1 − γ )F [U1 (t) − U2 (t) ] + F [U1 (t) − U3 (t) ] + ε1 (t) As seen from the equations, the decay γ in DFT is local, separate for each choice alternative whereas for LCA it is global, the same for all choice alternatives. In DFT, the choice alternatives are contrasted by determining the difference between the value of the alternative and the average value of the remaining alternatives with respect to the attribute under consideration. In LCA, the advantages and disadvantages between choice alternatives are determined and transformed by a function accounting for risk attitudes. For a detailed comparison, see Busemeyer, Townsend, Diederich, and Barkan (2005) and Tsetsos, Usher, and Chater (2010). The LCA model explains the compromise and attraction effects as adirect outcome of the loss-aversion advantage function. In both conditions, the asymmetric function (Eq. 11) penalizes the distant alternatives (B in the similarity condition and both A and B in the compromise condition). That is, the advantages of the more distant alternative decrease (Eq. 10) and alternative C, which has two shorter distance alternatives, is preferred in the compromise condition. In the attraction condition, A is preferred over B, which has two longer distance alternatives (A and D), as opposed to one for A (a longer distance to B and a short one to D). The explanation of the similarity effect is similar to that of DFT.
Example: Context effects - similarity, attraction, and compromise Accounting simultaneously for three specific context effects has become a criterion for evaluating multi-alternative choice models. Next we briefly review the three effects and how DFT gives an account of all three effects. We do this to demonstrate an important attribute of DFT and cognitive models in general is that they offer the possibility of providing a parsimonious account of a range of decision-making phenomena. The three effects—similarity, compromise, and attraction— refer to effects on choice probabilities produced by adding a third alternative to an existing choice set of two. The effect that occurs depends on the characteristics of the third alternative compared to the other two alternatives. Suppose that in a binary choice between A and B, the two alternatives, characterized by different levels of the same two attributes,are about equally attractive and are chosen equally frequently, Pr[A | {A, B}] = Pr[B | {A, B}]. The similarity effect is produced by adding an alternative, S, to the choice set that is similar to either of the two alternatives of the earlier choice set, say A. In this case, the probability to choose A decreases because the new choice alternative S takes away shares from its neighbor A; the probability of choosing B remains unaffected and
Weights
Differences
Preferences
A
A
B
B
C
C
1
2
Fig. 10.5 Connectionist network for leaky competing accumulator model. The first layer represents the subjective values of the consequences, mij , for the i = A, B, C alternatives with the j = 1, 2 attributes. In the fourth layer inhibition takes place symbolized by the circled headed lines.
models of decision making under risk and uncertainty
225
Pr[B|{A, B, S}] > Pr[A|{A, B, S}]. This result violates the principle of independence from irrelevant alternatives (see Tversky, 1972 for a discussion). The attraction effect is produced by adding an alternative, D, to the choice set that is similar to but dominated by one of the two alternatives of the earlier choice set, say A.In this case, the probability to choose A increases, Pr[A|{A, B, D}] > Pr[A|{A, B}] (Huber, Payne, & Puto, 1982), violating the principle ofregularity, which asserts that the probability of choosing an alternative from a givenset cannot be increased by enlarging the offered set (Tversky, 1972, p. 289). The compromise effect is produced by adding an alternative, C, midway between alternatives A and B,to the choice set that is. Suppose, that all the binary choices are equal so that Pr[A | {A,B)] = Pr[A | {A,C)] = Pr[B | {B,C)] = 0.50. When presenting all three alternatives together, the probability of choosing C increases such that Pr[C|{A, B, C}] > Pr[A|{A, B, C}] = Pr[B|{A, B, C}]. This result also violates the principle of independence from irrelevant alternatives (Simonson, 1989). Elimination by aspects (Tversky, 1972) can account for the similarity effect, and the contextdependent preference model (Tversky & Simonson, 1993) can account for the attraction and compromise effect, but neither of these models can explain all three effects simultaneously. In contrast, both the DFT and the LCA model can handle all three effects. To see how they accomplish this feat, it is helpful to consider the alternatives presented in a two-dimensional space with negative (−) to positive (+) attribute values for each attribute (Figure 9.3). Each choice alternative is represented as a point in this two-dimensional space. A minus sign (−) indicates negative values and a plus sign (+) indicates positive values for the attribute level. Alternatives S and D are similar to alternative A, with A dominating D but not S. For the similarity effect consider the choice alternatives A, B, and S, in Figure 3. Multi alternative DFT explains the effect as follows. Whenever attention is focused on attribute 1 alternative B gets an advantage, and both alternatives A and S get disadvantages; whenever attention is focused on attribute 2, then both alternatives A and S get advantages and alternative B gets a disadvantage. The valences of A and S are positively correlated with each other and negatively correlated with B. When focusing attention more on attribute 1, alternative B is chosen; when focusing more on attribute 2, alternative A or S is 226
higher level cognition
chosen. That is, adding S to the choice set takes choices of A but not of alternative B. For the attraction effect, consider the choice alternatives A, B, and D, in Figure 3. Comparison of the dominated alternative D with the average ofthe other two alternatives (Eq. 8) results in a negative valence, VD (t) , and eventually, during deliberation, in a negative preference state PD (t) for D (Eq. 9). The negative preference state PD (t) feeds through a negative inhibitory connection to the similar but dominant alternative, A, producing a positive (bolstering) effect on it. Thus, the dominant alternative A looks better in light of the dominated alternative D. The effect does not work for alternative B since B is too far away from D and, therefore, lateral inhibition is not strong enough. Although the attention-switching process is mainly responsible for producing the similarity effect and the lateral-inhibition process is predominately responsible for producing the attraction effect, they nevertheless operate in synchrony to generate both effects. This interaction is essential for producing the compromise effect. If attention is focused on some irrelevant features favoring the alternative C, the compromise, lateral inhibition is sent to alternatives A and B, decreasing their strength and simultaneously increasing the strength for C. For a formal implementation of the effects into the model and for numerical examples see Roe et al. (2001).
Final Thoughts It is clear from this review that modern behavioral decision theory has roots in both rational choice theory and cognitive psychology. Rational choice theory, in its axiomatic form, provides a set of normative principles against which to evaluate actual behavior. Descriptive failures of axiomatic choice theory, specifically of EUT and SEUT, resulted in their being generalized to other models, such as prospect and cumulative theory, rankand sign-dependent theories, and TAX. We barely touched on stochastic versions of these theories, but see the review by Marley and Regenwetter (in press). However, even these generalizations are insufficient to understand fully the descriptive failures of various normative and, arguably more importantly, to understand how observed choice behavior arises. Models, especially dynamic stochastic models, that apply what is known about cognition to humanchoice behavior address this gap and go beyond it to examine other variables, such as choice time and
confidence ratings (Pleskac & Busemeyer, 2010), in addition to choice probabilities.
Acknowledgments We thank Shuli Yu for her detailed comments on an earlier draft. We express our gratitude to Mitchell Uitvlugt and Kyle Bort for theireditorial assistance in preparing this chapter. A grant from the National Science Foundation (0955410) supported TJP while writing this chapter.
Notes 1. Terms that may be unfamiliar to readers new to this field are italicized the first time we introduce them and are defined in the glossary. 2. We recognize that not all cognitive models are formal in nature. But in this chapter, we consider only those that are. 3. Christiaan Huygens(1629–1695) is responsible for using the word expectation when speaking of the value of a gamble (Hacking, 1975). Huygens, in fact, wrote the first text of probability theory, developing a set of axioms for the science of mathematical expectation with the sole purpose of finding the fair price of a gamble (David, 1962). 4. A particularly readable description of the axioms can be found in Chapter 2 of Luce and Raiffa (1957) or in Kreps (1988). 5. A positive linear transformation is of the form u (xi ) = a + b · u(xi ) with b > 0. Commonly, the value of a is fixed at 0 by setting u (0) = u(0) = 0, but that is not necessary. 6. See uncertain situations in the glossary. 7. Only patterns of choices, never an individual choice, can be considered rational or irrational within the framework of EUT or SEUT. For more on this point, see Wakker (2010). 8. The symbol denotes empirical preference, in contrast to the symbol >, which denotes numerical order, i.e. larger than. Thus, A B is read as “A is preferred to B.” 9. The stronger the violation of the axiom, i.e., the bigger the effect, the less problematic is the presence of an error term. Thus, for example, the stochastic nature of choice does not hinder the demonstration of the Allais paradox in most cases. 10. Note Kahneman and Tversky (1979) used the term value function instead of utility function, presumably to highlight the different psychological properties, such as reference dependence, of the two functions. However, to maintain consistency across this chapter and echoing Wakker (2010) we will refer to it as a utility function. 11. The parameter τ is an arbitrary small time unit. The preference state approximates to a diffusion process as τ approaches zero. 12. Strictly speaking when, γ τ < 0, this process can be unstable in the long run and is not an Ornstein-Uhlenbeck process with drift.
Glossary Axiom: A proposition or principle that is assumed without proof and taken as self-evident. Axiomatic model: A model that derives necessary consequences from a set of independent and non-conflicting axioms. (See model in this glossary.)
Choice: See decision in this glossary. Cognitive architecture: A unified theory of cognition underlying intelligent behavior, often implemented as a computational model that provides an integrated account of the structure of the mind. Cognitive model: A formal model of one or more basic cognitive processes that describes the flow of information when accomplishing a task. Computational model: A formal model applying computer algorithms and programs, implementing them on a computer and performing computer simulations to derive the predictions of the model and also to generate data. (See model in this glossary.) Connectionist model: a model that assumes neural systemspass activation among simple, interconnected processing units. These models are also referred to as neural network models. Decision: A discrete response that explicitly identifies an action selected from a larger set of possible actions, each of which has consequences in the form of an outcome or set of outcomes. Deterministic model: A model that has no randomness in its representation so that it always produces the same output from a given set of inputs. Dynamic model: A model of the time-dependent changes in the state of the system. Empirical law: An observed regularity independent and dependent variables.
between
Expected utility theory: (EUT) A special case of rational choice models. See relational choice models. Formal model: A model that uses mathematical and statistical methods, formal logic, or computer simulation to represent a phenomenon, situation, or system in the real world. (See model in this glossary.) Heuristics: Simple procedures or rules of thumbs that provide approximate solutionsto a problem. Independence axiom: A principle that adding identical outcomes to choice alternatives does not alter the direction of preference between the alternatives. Latent construct: A theoretical dimension or variable that affects behavior but cannot be observed directly and can be measured via a formal model that links the dimension to observable behavior.Utility and subjective probability are two important latent constructs in decision theory. Mathematical model: A formal model taking on various structures such as algebraic or geometric and various forms such as axiomatic and systems of equations. Model: An abstraction representing some aspects of a phenomenon, situation, or system in the real world. Preference: An ordered relation between two or more outcomes or choice alternatives in terms of their desirability or value. Probability weighting function: A nonlinear transformation of probabilities into decision weights capturing the impact of probabilities on decisions made under risk and uncertainty.
models of decision making under risk and uncertainty
227
Glossary Probabilistic model: A model that incorporates randomness in its representation of a complex phenomenon or situation so that inputs result in probability distributions over outputs or actions. Outputs are not deterministic. (See, also, stochastic model.) Prospect theory: A generalization of expected utility theory in which the utilities are over gains or losses relative to a frame of reference, usually the status quo, and the probabilities map into nonadditive decision weights. Rank- and sign-dependent choice models: Generalizations of expectation-based models, such as expected utility, subjective expected utility and prospect theories, in which decisions weights are associated with decumulative probability distributions and moreover, the weights are different for losses than for gains relative to a frame of reference, usually the status quo. Rational choice models: Models derived from axioms sufficient to prescribe patterns of choices such that, if satisfied, a utility value, unique up to a positive linear transformation, can be assigned to each outcome, and the decision maker can be said to have chosen so as to maximize expected utility (when the probability distribution over states is known) or subjectively expected utility (when it is not known). Response time: Duration between a specified start point, usually the onset of the stimulus, and an observer’s response, sometimes referred to as latency. Risky situation: A situation in which the probabilities of different events occurring are known and are not 0 or 1. Static model: A time-insensitive model of a system. Stochastic dominance: Arelation between nonidentical gambles such that the probability distribution over outcomes of one gamble is consistently at least as favorable as that of the other gamble. Formally, A stochastically dominates B iff F ( X |A ) ≥F ( X |B). Stochastic: the property of a system that is nondeterministic, so that its state at any given time is determined probabilistically. Subjective probability: This term has different, but related meanings in rational and behavioral decision models. In rational choice models subjective probability refers to the belief structure of a rational choice agent. In behavioral decision models, subjective probability refers to an individual’s underlying belief structure. If that belief structure is coherent, that is, conforms to the axioms of probability theory, then it can be considered rational within the framework of rational choice theory. Subjectively expected utility theory: (SEUT) A special case of rational choice models. See that entry. Transitivity axiom: A principle of most models of decisionmaking that states, that when option x is preferred to y and y is preferred to z, then x must be preferred to z. Uncertain situation: A situation in which the probability of different events occurring are not known or not given. Utility: The subjective value of outcomes obtained from choices or actions. In axiomatic choice models, utilities are inferred from the pattern of choices.
228
higher level cognition
Utility function: A function that maps objective into subjective valueswithin the context of EUT and SEUT. Value function: A function that maps objective into subjective changes in valuesrelative to the status quo (or other frame of reference) within the context of prospect theory. In this chapter, following Wakker (2010), we use the term utility function to serve that purpose. See footnote 10.
References Allais, M. (1953). Le comportement de l’homme rationnel devant le risque: Critique des postulats et axiomes de l’ecole Americaine. Econometrica, 21 (4), 503–546. doi:10.2307/1907921 Allais, M. (1979). The so-called Allais paradox and rational decisions under uncertainty. In M. Allais & O. Hagen (Eds.), Expected utility hypotheses and the Allais paradox: Contemporary discussions of decisions under uncertainty with Allais’ rejoinder (pp. 437–681). Dordrecht, Netherlands: D. Reidel. Barron, G., & Erev, I. (2003). Small feedback-based decisions and their limited correspondence to description-based decisions. Journal of Behavioral Decision Making, 16 (3), 215–233. doi:10.1002/bdm.443 Battalio, R. C., Kagel, J. H., & Jiranyakul, K. (1990). Testing between alternative models of choice under uncertainty: Some initial results. Journal of Risk and Uncertainty, 3(1), 25–50. doi:10.1007/bf00213259 Bhatia, S. (2013). Associations and the accumulation of preference. Psychological Review, 120(3), 522–543. doi: 10.1037/a0032457 Bernoulli, D. (1954/1738). Exposition of a new theory on the measurement of risk. Econometrica, 22(1), 23–36. doi:10.2307/1909829 Bhattacharya, R. N., & Waymire, E. C. (1990). Stochastic processes with applications. New York, NY: Wiley. Birnbaum, M. H. (2004). Causes of Allais common consequence paradoxes: An experimental dissection. Journal of Mathematical Psychology, 48(2), 87–106. doi:10.1016/j.jmp.2004 .01.001 Birnbaum, M. H. (2008a). Evaluation of the priority heuristic as a descriptive model of risky decision making: Comment on Brandstätter, Gigerenzer, and Hertwig (2006). Psychological Review, 115(1), 253–260. doi: Doi 10.1037/0033295x.115.1.253 Birnbaum, M. H. (2008b). New paradoxes of risky decision making. Psychological Review, 115(2), 463–501. doi:10.1037/0033-295X.115.2.463 Birnbaum, M. H. (2008c). New tests of cumulative prospect theory and the priority heuristic: Probability-outcome tradeoff with branch splitting. Judgment and Decision Making Journal, 3(4), 304–316. Birnbaum, M. H. (2011). Testing theories of risky decision making via critical tests. Frontiers in Psychology, 2(315), 1–3. doi:10.3389/fpsyg.2011.00315 Birnbaum, M. H., & Navarrete, J. B. (1998). Testing descriptive utility theories: Violations of stochastic dominance and
cumulative independence. Journal of Risk and Uncertainty, 17 (1), 49–79. doi:10.1023/a:1007739200913 Brändstatter, E., Gigerenzer, G. & Hertwig, R. (2006). The priority heusistic-making choices without-trade-offs. Psychological Review 113, 409–432. doi:10.1037/0033295X.113.2.409 Brändstatter, E., Gigerenzer, G. & Hertwig, R. (2008). Postscript: Rejoinder to Johnson et al. (2008) and Birnbaum (2008). Pscyhological Review 115, 289–290. doi:10.1037/0033-295X.115.1.289 Budescu, D. V., & Wallsten, T. S. (1987). Subjective estimation of precise and vague uncertainties. In G. Wright & P. Ayton (Eds.), Judgmental forecasting (pp. 63–81). Chichester, England: Wiley. Busemeyer, J. R., & Diederich, A. (2002). Survey of decision field theory. Mathematical Social Sciences, 43(3), 345–370. doi:10.1016/S0165-4896(02)00016-1 Brandstätter E, Gigerenzer G, & Hertwig R. (2006). The Priority Heuristic: Making choices without trade-offs. Psychological Review, 113, 409–432. doi: 10.1037/0033295X.113.2.409 Brandstätter E, Gigerenzer G, & Hertwig R. (2008). Postscript: Rejoinder Johnson et al. and Birnbaum (2008). Psychological Review, 115, 289–290. doi: 10.1037/0033295X.115.1.289 Busemeyer, J. R. & Townsend, J. T. (1992). Fundamental derivations from decision field theory. Mathematical Social Sciences, 23, 255–282. doi: 10.1016/0165-4896(92)9043-5 Busemeyer, J. R., & Diederich, A. (2010). Cognitive modeling. Thousand Oaks, CA: SAGE. Busemeyer, J. R., & Goldstein, W. M. (1992). Linking together different measures of preference: A dynamic model of matching derived from decision field theory. Organizational Behavior and Human Decision Processes, 52(3), 370–396. doi:10.1016/0749-5978(92)90026-4 Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review, 100(3), 432–459. doi:10.1037//0033-295X.100.3.432 Busemeyer, J.R., Townsend, J.T., Diederich, A., & Barkan, R. (2005). Contrast effects of loss aversion? Comment on M.Usher and J.L.McClelland’s (2004) ’Loss aversion and inhibition in dynamical models of multi-alternative choice. Psychological Review, 112(1), 253–255. doi:10.1037/0033295X.112.1.253 Busemeyer, J.R., & Townsend, J.T. (1992). Fundamental derivations of decision field theory. Mathematical Social Sciences, 23, 255–282 doi:10.1016/0165-4896(92) 90043-5 Coombs, C. H., Dawes, R. M., & Tversky, A. (1970). Mathematical psychology: An elementary introduction. Englewood Cliffs, NJ: Prentice-Hall. David, F. N. (1962). Games, gods, and gambling: The origins and history of probability and statistical ideas from the earliest times to the Newtonian era. London, England: Griffin. Diecidue, E., & Wakker, P. P. (2001). On the intuition of rankdependent utility. Journal of Risk and Uncertainty, 23(3), 281–298. doi:10.1023/a:1011877808366
Diederich, A. (1997). Dynamic stochastic models for decision making under time constraints. Journal of Mathematical Psychology, 41(3), 260–274. doi:10.1006/jmps.1997.1167 Diederich, A. (2003). MDFT account of decision making under time pressure. Psychonomic Bulletin & Review, 10(1), 157– 166. doi:10.3758/bf03196480 Diederich, A., & Busemeyer, J. R. (1999). Conflict and the stochastic-dominance principle of decision making. Psychological Science, 10(4), 353–359. doi:10.1111/14679280.00167 Diederich, A., & Busemeyer, J. R. (2003). Simple matrix methods for analyzing diffusion models of choice probability, choice response time, and simple response time. Journal of Mathematical Psychology, 47 (3), 304–322. doi:10.1016/s0022-2496(03)00003-8 Edwards, W. (1954). The theory of decision making. Psychological Bulletin, 51(4), 380–417. doi:10.1037/h0053870 Ellsberg, D. (1961). Risk, ambiguity, and the savage axioms. Quarterly Journal of Economics, 75(4), 643–669. doi:10.2307/1884324 Ert, E., & Erev, I. (2008). The rejection of attractive gambles, loss aversion, and the lemon avoidance heuristic. Journal of Economic Psychology, 29(5), 715–723. doi:10.1016/j.joep.2007.06.003 Fishburn, P. C. (1978). On Handa’s “New theory of cardinal utility” and the maximization of expected return. Journal of Political Economy, 86 (2), 321–324. doi:10.1086/260670 Fishburn, P. C., & Kochenberger, G. A. (1979). Two-piece von Neumann-Morgenstern utility functions. Decision Sciences, 10(4), 503–518. doi:10.1111/j.1540-5915.1979.tb00043.x Fox, C. R., & Hadar, L. (2006). “Decisions from experience” = sampling error plus prospect theory: Reconsidering Hertwig, Barron, Weber & Erev (2004). Judgment and Decision Making Journal, 1(2), 159–161. Fox, C. R., & Tversky, A. (1998). A belief-based account of decision under uncertainty. Management Science, 44(7), 879. doi:10.1287/mnsc.44.7.879 Gigerenzer, G., Hertwig, R., & Pachur, T. (2011). Heuristics: The foundations of adaptive behavior. Oxford, England: Oxford University Press. Gold, J. I., & Shadlen, M. N. (2007). The neural basis of decision making. Annual Review of Neuroscience, 30, 535–574. doi:10.1146/annurev.neuro.29.051605.113038 Gonzalez, C., & Dutt, V. (2011). Instance-Based Learning: Integrating Sampling and Repeated Decisions From Experience. Psychological Review, 118(4), 523–551. doi: 10.1037/A0024558 Gonzalez, R., & Wu, G. (1999). On the shape of the probability weighting function. Cognitive Psychology, 38(1), 129–166. doi:10.1006/cogp.1998.0710 Hacking, I. (1975). Emergence of probability: A philosophical study of early ideas about probability, induction, and statistical inference. Cambridge, England: Cambridge University Press. Hau, R., Pleskac, T. J., & Hertwig, R. (2010). Decisions from experience and statistical probabilities: Why they trigger different choices than a priori probabilities. Journal of Behavioral Decision Making, 23(1), 48-68. doi: 10.1002/bdm.665 Heath, R.A. (1981). A tandem random walk model for psychological discrimination. British Journal of Mathematical
models of decision making under risk and uncertainty
229
and Statistical Psychology, 34, 79–92. doi:10.111/j.20448317.1981.tb00619.X Hertwig, R., Barron, G., Weber, E. U., & Erev, I. (2004). Decisions from experience and the effect of rare events in risky choice. Psychological Science, 15(8), 534–539. doi:10.1111/j.0956-7976.2004.00715.x Hertwig, R., & Erev, I. (2009). The description-experience gap in risky choice. Trends in Cognitive Sciences, 13(12), 517– 523. doi:10.1016/j.tics.2009.09.004 Hotaling, J. M., Busemeyer, J. R., & Li, J. (2010). Theoretical developments in decision field theory: Comment on Tsetsos, Usher, and Chater (2010). Psychological Review, 117 (4), 1294–1298. doi:10.1037/a0020401 Huber, J., Payne, J. W., & Puto, C. (1982). Adding asymmetrically dominated alternatives: Violations of regularity and the similarity hypothesis. Journal of Consumer Research, 9(1), 90–98. doi:10.1086/208899 Iverson, G., & Falmagne, J.-C. (1985). Statistical issues in measurement. Mathematical Social Sciences, 10(2), 131–153. doi:10.1016/0165-4896(85)90031-9 Johnson, E. J., Schulte-Mecklenbeck, M., & Willemsen, M. C. (2008). Process models deserve process data: Comment on Brandstätter, Gigerenzer, and Hertwig (2006). Psychological Review, 115(1), 263–272. doi:10.1037/0033295x.115.1.263 Johnson, J. G., & Busemeyer, J. R. (2005). A dynamic, stochastic, computational model of preference reversal phenomena. Psychological Review, 112(4), 841–861. doi:10.1037/0033295X.112.4.841 Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1990). Experimental tests of the endowment effect and the Coase theorem. Journal of Political Economy, 1325–1348. doi:10.1086/261737 Kahneman, D., & Tversky, A. (1979). Prospect theory: An analysis of decision under risk. Econometrica, 47 (2), 263– 292. doi:10.2307/1914185 Keeney, R. L., & Raiffa, H. (1993). Decisions with multiple objectives: Preferences and value tradeoffs. Cambridge, England: Cambridge University Press. Knight, F. H. (1921). Risk, uncertainty, and profit. New York, NY: Sentry Press. Krajbich, I., & Rangel, A. (2011). Multialternative driftdiffusion model predicts the relationship between visual fixations and choice in value-based decisions. Proceedings of the National Academy of Sciences of The United States of America, 108(33), 13852–13857. doi:10.1073/Pnas. 1101328108 Krantz, D. H. (1972). Measurement structures and psychological laws. Science, 175(4029), 1427–1435. doi:10.1126/science .175.4029.1427 Kreps, D. M. (1988). Notes on the Theory of Choice. Boulder, CO: Westview Press. Liu, T., & Pleskac, T. J. (2011). Neural correlates of evidence accumulation in a perceptual decision task. Journal of Neurophysiology, 106, 2383–2398. doi: 10.1152/jn.0041 3.2011 Lopes, L. L. (1984). Risk and distributional inequality. Journal of Experimental Psychology: Human Perception and Performance, 10(4), 465–485. doi:10.1037/0096-1523.10.4.465
230
higher level cognition
Luce, R. D. (1995). Four tensions concerning mathematical modeling in psychology. Annual Review of Psychology, 46, 1–27. doi:10.1146/annurev.ps.46.020195.000245 Luce, R. D. (2000). Utility of gains and lossess: Measurementtheoretical and experimental approaches. Mahwah, NJ: Erlbaum. Luce, R. D., & Raiffa, H. (1957). Games and decisions: Introduction and critical survey. New York, NY: Wiley. MacCrimmon, K. R., & Larsson, S. (1979). Utility theory: Axioms versus ’paradoxes’. In M. Allais & O. Hagen (Eds.), Expected utility hypotheses and the Allais paradox: Contemporary discussions of decisions under uncertainty with Allais’ rejoinder (pp. 333–409). Dordrecht, Netherlands: D. Reidel. Marley, A. A. J., & Regenwetter, M. (in press). Choice, preference, and utility: Probabilistic and deterministic representations. In W. Batchelder, H. Colonius, E. Dzhafarov, and J. Myung (Ed.), The new handbook of mathematical psychology: Cambridge, UK. Cambridge University Press. Masin, S. C., Zudini, V., & Antonelli, M. (2009). Early alternative derivations of Fechner’s law. Journal of the History of the Behavioral Sciences, 45(1), 56–65. doi:10.1002/jhbs.20349 Mellers, B., Weiss, R., & Birnbaum, M. (1992). Violations of dominance in pricing judgments. Journal of Risk and Uncertainty, 5(1), 73–90. doi:10.1007/bf00208788 Mellers, B. A., Schwartz, A., & Cooke, A. D. J. (1998). Judgment and decision making. Annual Review of Psychology, 49, 447–477. doi:10.1146/annurev.psych.49.1.447 Morrison, H. W. (1963). Testable conditions for triads of paired comparison choices. Psychometrika, 28(4), 369–390. doi:10.1007/bf02289558 Myung, J., Karabatsos, G., & Iverson, G. (2005). A Bayesian approach to testing decision making axioms. Journal of Mathematical Psychology, 49, 205–225. Pachur, T., Hertwig, R., Gigerenzer, G., & Brandstätter, E. (2013). Testing process predictions of models of risky choice: A quantitative model comparison approach. Frontiers in Psychology, 4. doi: 10.3389/fpsyg.2013.00646 Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. New York, NY: Cambridge University Press. Pleskac, T. J. (in press). Decision making and learning. In G. Keren & G. Wu (Eds.), Handbook of judgment and decision making. Hoboken, NJ: Wiley-Blackwell. Pleskac, T. J., & Busemeyer, J. R. (2010). Two-stage dynamic signal detection: A theory of choice, decision time, and confidence. Psychological Review, 117 (3), 864–901. doi:10.1037/a0019737 Preston, M. G., & Baratta, P. (1948). An experimental study of the auction-value of an uncertain outcome. American Journal of Psychology, 61(2), 183–193. doi:10.2307/1416964 Quiggin, J. (1982). A theory of anticipated utility. Journal of Economic Behavior & Organization, 3(4), 323–343. doi:10.1016/0167-2681(82)90008-7 Rapoport, A., & Wallsten, T. S. (1972). Individual decision behavior. Annual Review of Psychology, 23, 131–176. doi:10.1146/annurev.ps.23.020172.001023 Regenwetter, M., Dana, J., & Davis-Stober, C. P. (2010). Testing transitivity of preferences on two-alternative forced
choice data. Frontiers in Psychology, 1(148), 1–15. doi:10.3389/fpsyg.2010.00148 Regenwetter, M., Dana, J., & Davis-Stober, C. P. (2011). Transitivity of preferences. Psychological Review, 118(1), 42– 56. doi:10.1037/a0021150 Rieger, M. O., & Wang, M. (2008). What is behind the priority heuristic? A mathematical analysis and comment on Brandstätter, Gigerenzer, and Hertwig (2006). Psychological Review, 115(1), 274–280. doi:10.1037/0033-295x.115.1.274 Rieskamp, J. (2008). The Probabilistic Nature of Preferential Choice. Journal of Experimental Psychology: Learning Memory and Cognition, 34(6), 1446–1465. doi: 10.1037/A0013646 Roe, R. M., Busemeyer, J. R., & Townsend, J. T. (2001). Multialternative decision field theory: A dynamic connectionst model of decision making. Psychological Review, 108(2), 370–392. doi:10.1037//0033-295X.108.2.370 Savage, L. J. (1954). The foundations of statistics. New York, NY: Wiley. Shafir, E. & LeBoeuf, R.A. (2002). Raionality, Annual Review of Psychology, 53, 491–517. doi:10.1146/annurev.psych.53.100 901.B2513 Shah, A.K., & Oppenheimer, D.M. (2008). Heuristics made easy: An effort reduction framework. Psychological Bulletin, 134, 207–222. doi: 10.1037/0033-2909.134.2.207 Simonson, I. (1989). Choice based on reasons: The case of attraction and compromise effects. Journal of Consumer Research, 16 (2), 158–174. doi:10.1086/209205 Stewart, N. (2009). Decision by sampling: The role of the decision environment in risky choice. Quarterly Journal of Experimental Psychology, 62(6), 1041–1062. doi:10.1080/17470210902747112 Stewart, N., Chater, N., & Brown, G. D. A. (2006). Decision by sampling. Cognitive Psychology, 53(1), 1–26. doi:10.1016/j.cogpsych.2005.10.003 Thorngate, W. (1980). Efficient decision heuristics, Behavioural Science, 25, 219–225. doi:10.1002/bs.38302506 Thurstone, L. L. (1927). A law of comparative judgment. Psychological Review, 34(4), 273–286. doi:10.1037/h00 70288 Todhunter, I. (1865). A history of the mathematical theory of probability from the time of Pascal to that of Laplace. Cambridge and London, England: Macmillan. Tom, S. M., Fox, C. R., Trepel, C., & Poldrack, R. A. (2007). The neural basis of loss aversion in decision-making under risk. Science, 315(5811), 515–518. doi:10.1126/science.1134239 Townsend, J. T., & Ashby, F. G. (1983). Stochastic modeling of elementary psychological proceses. New York, NY: Cambridge University Press. Trueblood, J. S., Brown, S., & Heathcote, A. (2014). The multi-attribute linear ballistic accumulator model of context effects in multi-alternative choice. 121, 179–205. doi:10.1037/a0030137 Tsetsos, K., Usher, M., & Chater, N. (2010). Preference reversal in multiattribute choice. Psychol Review, 117 (4), 1275–1291. doi:10.1037/a0020580 Tversky, A. (1969). Intransitivity of preferences. Psychological Review, 76 (1), 31–48. doi:10.1037/h0026750
Tversky, A. (1972). Elimination by aspects: A theory of choice. Psychological Review, 79(4), 281–299. doi:10.1037/h0032955 Tversky, A., & Fox, C. R. (1995). Weighing risk and uncertainty. Psychological Review, 102(2), 269–283. doi:10.1037//0033295X.102.2.269 Tversky, A., & Kahneman, D. (1986). Rational choice and the framing of decisions. Journal of Business, 59(4), S251–S278. doi:10.1086/296365 Tversky, A., & Kahneman, D. (1991). Loss aversion in riskless choice: A reference-dependent model. The Quarterly Journal of Economics, 106 (4), 1039–1061. doi:10.2307/2937956 Tversky, A., & Kahneman, D. (1992). Advances in prospect theory: Cumulative representation of uncertainty. Journal of Risk and Uncertainty, 5(4), 297–323. doi:10.1007/bf00122574 Tversky, A., & Koehler, D. J. (1994). Support theory: A nonextensional representation of subjective probability. Psychological Review, 101(4), 547–567. doi:10.1037//0033295X.101.4.547 Tversky, A., & Simonson, I. (1993). Context-dependent preferences. Management Science, 39(10), 1179–1189. doi:10.1287/mnsc.39.10.1179 Tversky, A., & Wakker, P. (1995). Risk attitudes and decision weights. Econometrica, 63(6), 1255–1280. doi:10.2307/21 71769 Ungemach, C., Chater, N., & Stewart, N. (2009). Are Probabilities Overweighted or Underweighted When Rare Outcomes Are Experienced (Rarely)? Psychological Science, 20(4), 473–479. doi: 10.1111/j.1467-9280.2009.02319.x Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108(3), 550–592. doi:10.1037/0033295X.108.3.550 Usher, M., & McClelland, J. L. (2004). Loss aversion and inhibition in dynamical models of multialternative choice. Psychological Review, 111(3), 757–769. doi:10.1037/0033295X.111.3.757 von Neumann, J., & Morgenstern, O. (1944). Theory of games and economic behavior. Princeton, NJ: Princeton University Press. Wakker, P. (2010). Prospect theory for risk and ambiguity. Cambridge, England: Cambridge University Press. Wakker, P., Erev, I., & Weber, E. U. (1994). Comonotonic Independence - the Critical Test between Classical and RankDependent Utility Theories. Journal of Risk and Uncertainty, 9(3), 195–230. doi:10.1007/Bf01064200 Weber, E. U. (1994). From subjective probabilities to decision weights: The effect of asymmetric loss functions on the evaluation of uncertain outcomes and events. Psychological Bulletin, 115, 228–242. doi:10.1037/0033-2909.115.2.228 Wollschläger, L. M., & Diederich, A. (2012). The 2N ary choice tree model for N -alternative preferential choice. Frontiers in Psychology, 3(189), 1–11. doi:10.3389/fpsyg.201 2.00189 Yechiam, E., & Hochman, G. (2013). Losses as modulators of attention: Review and analysis of the unique effects of losses over gains. Psychological Bulletin, 139(2), 497–518. doi:10.1037/a0029383
models of decision making under risk and uncertainty
231
CHAPTER
11
Models of Semantic Memory
Michael N. Jones, Jon Willits, and Simon Dennis
Abstract Meaning is a fundamental component of nearly all aspects of human cognition, but formal models of semantic memory have classically lagged behind many other areas of cognition. However, computational models of semantic memory have seen a surge of progress in the last two decades, advancing our knowledge of how meaning is constructed from experience, how knowledge is represented and used, and what processes are likely to be culprit in disorders characterized by semantic impairment. This chapter provides an overview of several recent clusters of models and trends in the literature, including modern connectionist and distributional models of semantic memory, and contemporary advances in grounding semantic models with perceptual information and models of compositional semantics. Several common lessons have emerged from both the connectionist and distributional literatures, and we attempt to synthesize these themes to better focus future developments in semantic modeling. Key Words: semantic memory, semantic space model, distributional semantics,
connectionist network, concepts, cognitive model, latent semantic analysis
Introduction Meaning is simultaneously the most obvious feature of memory—we can all compute it rapidly and automatically—and the most mysterious aspect to study. In comparison to many areas of cognition, relatively little is known about how humans compute meaning from experience. Nonetheless, a mechanistic account of semantics is an essential component of all major theories of language comprehension, reading, memory, and categorization. Semantic memory is necessary for us to construct meaning from otherwise meaningless words and utterances, to recognize objects, and to interact with the world in a knowledge-based manner. Semantic memory typically refers to memory for word meanings, facts, concepts, and general world knowledge. For example, you know that a panther is a jungle cat, is more like a tiger than a corgi, and you know better than to try to pet one. The 232
two common types of semantic information are conceptual and propositional knowledge. A concept is a mental representation of something, such as a panther, and knowledge of its similarity to other concepts. A proposition is a mental representation of conceptual relations that may be evaluated to have a truth value, for example, that a panther is a jungle cat, or has four legs and knowledge that panthers do not have gills. In Tulving’s (1973) classic modular taxonomy, declarative memory was subdivided into episodic and semantic memory, the former containing memory for autobiographical events, and the latter dedicated to generalized memory not linked to a specific event. Although you may have a specific autobiographical memory of the last time you saw a panther at the zoo, you do not have a specific memory of when you learned that a panther was a jungle cat, was black, or how it is similar to a tiger. In this
sense, semantic memory gained a reputation as the more miscellaneous and mysterious of the memory systems. Although episodic memory could be studied with experimental tasks such as list learning and could be measured quantitatively by counting the number of items correctly recognized or recalled, semantic memory researchers focused more on tasks such as similarity judgments, proposition verification, semantic priming, and free association. Unlike episodic memory, there existed no mechanistic account of how semantic memory was constructed as a function of experience. However, the field has advanced a considerable amount in the past 25 years. A scan of the contemporary literature reveals a large number of formal models that aim to understand the mechanisms that humans use to construct semantic memory from repeated episodic experience. Modern semantic models have made truly impressive progress at elucidating how humans learn and represent semantic information, how semantic memory is recruited and used in cognitive processing, and even how complex functions like semantic composition may be accomplished by relatively simple cognitive mechanisms. Many of the current advances build from classic ideas, but only relatively recently has computational hardware advanced to a scale where we can actually simulate and evaluate these systems. Advances in semantic modeling also are indebted to excellent interdisciplinary collaboration, building in part on developments in computational linguistics, machine learning, and information retrieval. The goal of this chapter is to provide an overview of recent advances in models of semantic memory. We will first provide a brief synopsis of classic models and themes in semantic memory research, but will then focus on computational developments. In addition, the focus of the chapter is on models that have a formal instantiation that may be tested quantitatively. Hence, although there are several exciting new developments in verbal conceptual theory (e.g., Louwerse’s (2011) Symbol Interdependency Hypothesis), we focus exclusively on models that are explicitly expressed by computer code or mathematical expressions. In addition, the chapter assumes a sufficient understanding of the empirical literature on semantic memory. For an overview of contemporary experimental findings, we refer the reader to a companion chapter on semantic memory by McRae and Jones (2014). There are several potential ways to organize a review of the literature, and no single structure will satisfy all theorists. We opt here to follow two
major clusters of cognitive models that have been prominent: distributional models and connectionist models. The division may also be broadly thought of as a division between models that specify how concepts are learned from statistical experience (distributional models), and models that specify how propositions are learned or that use conceptual representations in cognitive processes (connectionist models). Obviously, there are exceptions in both clusters that cross over, but the two literatures have had different foci. Next, we summarize some classic models of semantic memory and common theoretical debates that have extended to the contemporary models. Following the historical trends in the literature, we then discuss advances in connectionist models, followed by distributional models. Finally, we discuss hybrid approaches and new directions in models of grounded semantics and compositional semantics, and attempt to synthesize common lessons that have been learned across the literature.
Classic Models and Themes in Semantic Memory Research The three classic models of semantic memory most commonly discussed are semantic networks, feature-list models, and spatial models. These three models deserve mention here, both because they have each seen considerable attention in the literature, and because features of each have clearly evolved into modern computational models. The semantic network has traditionally been one of the most common theoretical frameworks used to understand the structure of semantic memory. Collins and Quillian (1969) originally proposed a hierarchical model of semantic memory in which concepts were nodes and propositions were labeled links (e.g., the nodes for dog and animal were connected via an “isa” link). The superordinate and subordinate structure of the links produced a hierarchical tree structure (animals were divided into birds, fish, etc., and birds were further divided into robin, sparrow, etc.), and allowed the model to explain both conceptual and propositional knowledge within a single framework. Accessing knowledge required traversal of the tree to the critical branch, and the model was successful in this manner of explaining early sentence verification data from humans (e.g., the speed to verify that “a canary can sing”). A later version of the semantic network model proposed by Collins and Loftus (1975) deemphasized the hierarchical nature of the network in favor of the process of spreading models of semantic memory
233
activation through all network links simultaneously to account for semantic priming phenomena—in particular, the ability to produce fast negative responses. Early semantic networks can be seen as clear predecessors to several modern connectionist models, and features of them can also be seen in modern probabilistic and graphical models as well. A competing model was the feature-comparison model of Rips, Shoben, and Smith (1973). In this model, a word’s meaning is encoded as a list of binary descriptive features, which were heavily tied to the word’s perceptual referent. For example, the feature would be turned on for a robin, but off for a beagle. Smith, Shoben, and Rips (1974) proposed two types of semantic features: defining features that all concepts have, and characteristic features that are typical of the concept, but are not present in all cases. For example, all birds have wings, but not all birds fly. Processing in the model was accomplished by computing the feature overlap between any two concepts, and the features were allowed to vary in their contribution of importance to the concept, although how particular features came to be and how they were ranked was not fully specified. Modern versions of feature-list models use aggregate data collected from human raters in property generation tasks (e.g., McRae, de Sa, & Seidenberg, 1997). A third type was the spatial model, which emerged from Osgood’s (1952, 1971) early attempts to empirically derive semantic features using semantic differential ratings. Osgood had humans rate words on a Likert scale against a set of polar opposites (e.g., rough-smooth, heavy-light), and a word’s meaning was then computed as a coordinate in a multidimensional semantic space. Distance between words in the space was proposed as a process for semantic comparison.1 Featural and spatial representations have been contrasted as models of human similarity judgments (e.g., Tversky & Gati, 1982), and the same contrast applies to spatial versus featural representations of semantic representations. We will see the feature versus space debate emerge again with modern distributional models. Early spatial models can be seen as predecessors of modern semantic space models of distributional semantics (but co-occurrences in text corpora are used as the data on which the space is constructed rather than human ratings). One issue with all three of these classic models is that none ever did actually learn anything. Each model relied on representations that were hand 234
higher level cognition
coded based on the theorist’s intuition (or subjective ratings) of semantic structure, but none formally specified the cognitive mechanisms by which the representations were constructed. As Hummel and Holyoak (2003) have noted, this type of intuitive modeling may have serious consequences: “The problem of hand-coded representations is the most serious problem facing computational modeling as a scientific enterprise. All models are sensitive to their representation, so the choice of representation is among the most powerful wildcards at the modeler’s disposal” (p. 247). As we will see later in the chapter, this is exactly the concern that modern distributional models address.
Connectionist Models of Semantic Memory Connectionist models were among the first to specify how semantic representations might come to be learned, and how those representations might interact with other cognitive processes. Modern connectionism is a framework used to model mental and behavioral phenomena as an emergent process—one that arises out the behavior of networks of simple interconnected units (Rumelhart, McClelland, & the PDP Group, 1986). Connectionism is a very broad enterprise. Connectionist models can be used to explicitly model the interaction of different brain regions or neural processes (O’Reilly, Munakata, Frank, Hazy, & Contributors, 2012) or they can be used to model cognition and behavior from a “neurally inspired” perspective, which values the way in which the models exhibit parallel processing, interactivity, and emergentism (Rumelhart et al., 1986; Rogers & McClelland, 2006). Connectionist models have made a very large contribution to simulating and understanding the dynamic nature of semantic knowledge and how semantic knowledge interacts with other cognitive processes. Connectionist models represent knowledge in terms of weighted connections between interconnected units. A model’s set of units, its connections, and how they are organized is called the model’s architecture. Research involving connectionist models has studied a wide range of architectures, but most connectionist models share a few common features. Most models have at least one set of units designated as input units, as well as at least one set of units designated as target or output units. Most connectionist models also have one or more sets of intervening units between the input and output units, which are often referred to as hidden layers.
A connectionist model represents knowledge in terms of the strength of the weighted connections between units. Activation is fed into the input units, and that activation in turn activates (or suppresses) the units to which the input units are connected, as a function of the weighted connection strength between the units. Activation eventually propagates to the output units, with one important question of interest being: What output units will a connectionist model activate given a particular input? In this sense, the knowledge in connectionist models is typically thought of as representing the function or relationship between a set of inputs and a set of outputs. Connectionist models should not, however, be confused with models that map simple stimulus-response relationships; The hidden layers between input and output layers in connectionist networks allow them to learn very complex internal representations. Models with an architecture such as the one just described, where activation flows from input units to hidden units to output units, are typically referred to as feed-forward networks. A key aspect of connectionist models is that they are often used to study the learning process itself. Typically, the weights between units in a connectionist network are initialized to a random state. The network is then provided with a training phase, in which the model is provided with inputs (typically involving some sort of expected input from the environment), and the weights are adjusted as a function of the particular inputs the network received. Learning (adjusting the weights) is accomplished in either an unsupervised or a supervised fashion. In unsupervised learning, weights are typically adjusted according some sort of associative principle, such as Hebbian learning (Grossberg, 1976; Hebb, 1946), where weights between units are increased the more often the two units are active at the same time. In supervised learning, weights are adjusted by observing which output units the network activated given a particular input pattern, and comparing that to some goal or target output given those inputs. The weights are then adjusted so as to reduce the amount of error the network makes in terms of its activation of the “correct” and “incorrect” outputs (Kohonen, 1982; Rosenblatt, 1959; Rumelhart, Hinton, & Williams, 1986; Widrow & Hoff, 1960).
Rumelhart Networks An illustrative example of a connectionist model of semantic memory (shown in Figure 11. 1a) was first presented by Rumelhart & Todd (1993) and
studied in detail by Rogers and McClelland (2006). This network has two sets of input units: (1) a set of units meant to represent words or concepts (e.g., robin, canary, sunfish, etc.), and (2) a set of units meant to represent different types of relations (e.g., is-a, can, has, etc.). The network learns to associate conjunctions of those inputs (e.g., robin+can) with outputs representing semantic features (e.g. fly, move, sing, grow, for robin+can). The model accomplishes this using supervised learning, having robin+can activated as inputs, observing what a randomly initialized version of the model produces as an output, and then adjusting the weights so as to make the activation of the correct outputs more likely. The model is not merely learning associations between inputs and outputs; in the Rumelhart network, the inputs and outputs are mediated by two sets of hidden units, which allow the network to learn complex internal representations for each input. A critical property of connectionist architectures using hidden layers is that the same hidden units are being used to create internal representations for all possible inputs. In the Rogers et al. example, robin, oak, salmon, and daisy all use the same hidden units; what differentiates their internal representations is that they instantiate different distributed patterns of activation. But because the network is using overlapping distributed representations for all the concepts, this means that during the process of learning, changing the connection weights as a result of learning about one input could potentially affect how the network represents all other items. When the network learns an internal representation (i.e., hidden unit activation state) for the input robin+can, and learns to associate the outputs sing and fly with that internal representation, this will mean that other inputs whose internal representations are similar to robin (i.e., have similar hidden unit activation states, such as canary) will also become more associated with sing and fly. This provides these networks with a natural mechanism for categorization, generalization, and property induction. The behavior allows researchers using connectionist models to study how these networks categorize, and to compare the predictions of the model to human behaviors. Rogers and McClelland (2006) extensively studied the behavior of the Rumelhart networks, and found that the model provides an elegant account of a number of aspects of human concept acquisition and representation. For example, they found that as the model acquires concepts through increasing models of semantic memory
235
pine oak rose daisy robin canary sunfish salmon Item
Hidden 1
ISA is can has Query
Hidden 2
living thing plant animal tree flower bird flower pine oak rose daisy robin canary sunfish salmon
(b)
pretty tall living green red yellow
0.0
grow move swim fly sing
canary
daisy 0.5
robin salmon rose
Second component
(a)
sunfish –0.5
oak bark petals wings feathers scales gills roots skin
pine
–1.0 –1.0
–0.5
0.0
0.5
First Component
Fig. 11.1 (left). The network architecture used by Rogers and McClelland (2006), showing the output activation of appropriate semantic features given an input concept and an input relation. Fig. 11.1b (right). A graph of the network’s learning trajectory, obtained by performing a multidimensional scaling on the network’s hidden unit activations for each input. As the network obtains more experience with each concept, it progressively learns to make finer and finer distinctions between categories. c Massachusetts Institute of Technology, by permission of The MIT Press. Timothy T. Rogers and James L. McClelland, Semantic Cognition: A Parallel Distributed Processing Approach, figures 2.2 & 3.3,
amounts of experience, the internal representations for the concepts show progressive differentiation, learning broader distinctions first and more finegrained distinctions later, similar to the distinctions children show (Mandler, Bauer, & MoDonough, 1991). In the model, this happens because the network is essentially performing something akin to a principal component analysis, learning the different features in the order of the amount of variance in the input that they explain. Rogers and McClelland argued that this architecture, which combines simple learning principles with the expected structure of the environment, can be used to understand how certain features (those that have rich covariational structure) become the features that organize categories, and how conceptual structure can become reorganized over the course of concept acquisition. The general (and somewhat controversial) conclusion that Rogers and McClelland draw from their study of this model is that a number of properties of the semantic system, such as the taxonomic structure of categories (Bower, 1970) and role of causal knowledge in semantic reasoning (Keil, 1989), can be explained as an emergent consequence of simple learning mechanisms combined with the expected structure of the environment, and that these structural factors do not necessarily need to be explicitly built into models of semantic memory. Feed-forward connectionist models have only been used in a limited fashion to study the actual structure of semantic memory. However, these models have been used extensively to study how semantic structure interacts with various other cognitive processes. For example, feed-forward models have been used to simulate and understand the word learning process (Gasser & Smith, 1998; Regier, 2005). These word-learning models have been used to show that many details about the representation of word meanings (like hierarchical structure), learning constraints (such as mutual exclusivity and shape bias), and empirical phenomena (such as the vocabulary spurt that children show around two years of age) emerge naturally from the structure of environment with a simple learning algorithm, and do not need to be explicitly built into the model. Feed-forward models have also been used to model consequences of brain damage (Farah & McClelland, 1991; Rogers et al., 2004; Tyler, Durrant-Peatfield, Levy, Voice, & Moss, 2000), Alzheimer’s disease (Chan, Salmon, & Butters, 1998), schizophrenia (Braver, Barch, & Cohen, 1999; Cohen and Servan-Schreiber,
1992; Nestor et al., 1998), and a number of other disorders that involve impairments to semantic memory (see Aakerlund & Hemmingsen, 1998, for a review). These models typically study brain disorders by lesioning the network (i.e., removing units or connections), or otherwise causing the network to behave in suboptimal ways, and then studying the consequences of this disruption. Connectionist models provide accounts of a wide range of impairments and disorders, and have also been used to show that many semantic consequences of impairments and disorders, such as the selective impairment of certain categories, can be explained in terms of emergent processes deriving from the interaction of low-level features, rather than requiring explicit instantiations in the model (such as creating modular memory systems for living and nonliving things, see McRae and Cree, 2002, for a review).
Dynamic Attractor Networks In addition to feed-forward models such as the Rumelhart network, a considerable amount of semantic memory research has explored the use of dynamical connectionist models (Hopfield, 1982). A connectionist model becomes a dynamical model when its architecture involves some sort of bi-directionality, feedback, or recurrent connectivity. Dynamical networks allow investigations into how the activation of representations may change over time, as well as how semantic representations interact with other cognitive processes in an online fashion. For example, Figure 11.2a shows McLeod, Shallice, and Plaut’s (2000) dynamical network for pronouncing printed words. The network has a layer of units for encoding orthographic representations (grapheme units), a layer of units for encoding phonological representations (phoneme units), and an intervening layer between the two that encodes the words’ semantic features (sememe units), as well as additional layers of hidden units between each of these layers. Critically, the activation in this network is allowed to flow in both directions, from phonemes to sememes to graphemes, and from graphemes to sememes to phonemes. The network also has recurrent connections (the loops in Figure 11.2a) connecting the grapheme, sememe, and phoneme layers to themselves. The combination of the bidirectional connections and recurrent connectivity allows the McLeod et al. (2000) network to establish a dynamical system where the activation at the various levels will feed back models of semantic memory
237
(a)
33 phoneme units
(b)
normal trajectory error trajectory
initial semantic pattern
40 hidden units
68 sememe units initial influence of orthography
enegry
40 hidden units mask onset 28 grapheme units cat
dog
log
Fig. 11.2 (left). A prototypical example of a semantic attractor network, from McLeod, Shallice, & Plaut (2000). Fig. 11.2b (right). An example of the network’s behavior, simulating the experience of a person reading the word “dog”. The weights of the network have created a number of attractor spaces, determined by words’ orthographic, phonological, and semantic similarity. Disrupting the input (such as presenting the participant with a stimulus mask) at different stages has different effects. Early masking leads to a higher likelihood of falling into the wrong perceptual attractor (LOG instead of DOG). Later masking leads to a higher likelihood of falling into the wrong semantic attractor (CAT instead of DOG). c (left) After McLeod, Shallice, and Plaut (2000), Cognition Elsevier Inc.
and forth eventually settling into a stable attractor state. The result is that these attractor networks can allow multiple constraints (e.g. the weights that establish the network’s knowledge of the links between orthography and semantics, and semantics and phonology) to compete, eventually settling into a state that satisfies the most likely constraints for a given input. As an illustration of how this works, consider an example using the McLeod et al. network, shown in Figure 11.2b. Here, the network is simulating the experience of a person reading words. The figure depicts a three-dimensional space, where the vertical direction (labeled “energy”) represents the stability of the network’s current state (versus its likelihood to switch to a new state) as activity circulates through the network. In an attractor network, only a small number of possible states are stable. These stable states are determined by the network’s knowledge about the likelihood of certain orthographic, phonological, and semantic states to co-occur. And given any input, the network will eventually settle into one of these stable states. For example, if the network receives a clear case of the printed word DOG as input, and this input is not disrupted, the network will quickly settle into the corresponding DOG state in its orthographic, phonological, and semantic layers. Alternatively, if the network received a nonword like DAG as an input, it would eventually settle into a neighboring attractor state (like DOG or DIG or DAD). Similarly, if the network receives DOG as an input, but this 238
higher level cognition
input is impoverished (e.g., noisy, with errors in the input signal), or disrupted (simulating masking such as might happen in a psychology experiment), this can affect the network’s ability to settle into the correct attractor. In a manner corresponding well to the disruption effects that people show in behavioral experiments, an early disruption (before the network has had a chance to settle into an orthographic attractor basin) can lead the network to make a form-based error (settling into the LOG basin instead). A later disruption—happening after the orthographic layer has settled into its basin but before the semantic layer has done so—can lead the network to make a semantic error, activating a code of semantic features corresponding to CAT. Attractor networks have been used to study a very wide range of semantic-memory related phenomena. Rumelhart et al. (1986) used an attractor network to show how schemas (e.g., one’s representations for different rooms) can emerge naturally out of the dynamics of co-occurrence of lower-level objects (e.g., items in the rooms), without needing to build explicit schema representations into the model (see also Botvinick & Plaut, 2004). Like the McLeod example already described, attractor networks have been extensively used to study how semantic memory affects lexical access (Harm & Seidenberg, 2004; McLeod et al., 2000) as well as to model semantic priming (Cree, McRae, & McNorgan, 1999; McRae, et al., 1997; Plaut & Booth, 2000). Dynamical models have also been used to study the organization
and development of the child lexicon (Horst, McMurray, & Samuelson, 2006; Li, Zhao, & MacWhinney, 2007), the bilingual lexicon (Li, 2009), and children’s causal reasoning using semantic knowledge (McClelland & Thompson, 2007), and how lexical development differs in typical and atypical developmental circumstances (Thomas & Karmiloff-Smith, 2003). Dynamical connectionist models have also simulated various ways that semantic knowledge impacts and interacts with sentence production and comprehension, including how semantic constraints impact the grammaticality of sentences (Allen & Seidenberg, 1999; Dell, Chang, & Griffin, 1999; McClelland, St. John, & Taraban, 1989; Tabor & Tanenhaus, 1999; Taraban & McClelland, 1988), and how semantic knowledge assists in the learning of linguistic structure (Borovsky & Elman, 2006; Chang, Dell, & Bock, 2006; Rohde & Plaut, 2000). As with feed-forward models, dynamical models have been also used to extensively study many developmental and brain disorders such as dyslexia and brain damage (Devlin, Gonnerman, Anderson, & Seldenberg, 1998; Hinton & Shallice, 1991; Kinder & Shanks, 2003; Lambon Ralph, MoClelland, Patterson, Galton, & Hodges, 2001; Plaut, 1999, 2002).
company it keeps,” and this idea was further developed by Harris (1970) into the distributional hypothesis of contextual overlap. For example, robin and egg may become related because they tend to co-occur frequently with each other. In contrast, robin and sparrow become related because they are frequently used in similar contexts (with the same set of words), even if they rarely co-occur directly. Ostrich may be less related to robin due to a lower overlap of their contexts compared to sparrow, and stapler is likely to have very little contextual overlap with robin. Formal models of distributional semantics differ in their learning mechanisms, but they all have the same overall goal of formalizing the construction of semantic representations from statistical redundancies in language. A taxonomy of distributional models is very difficult now given the large number of them and range of learning mechanisms. The models can be loosely clustered based on their notion of context (e.g., documents, words, time, etc.), or the learning mechanism they employ. We opt for the latter organization here, and just present some standard exemplars of each model type; an exhaustive description of all models is beyond the scope of this chapter (for reviews, see Bullinaria & Levy, 2007; Riordan & Jones, 2011; Turney & Pantel, 2010).
Distributional Models of Semantic Memory
Latent Semantic Analysis
There are now a large number of computational models in the literature that may be classified as distributional. Other terms commonly used to refer to these models are corpus-based, semantic-space, or co-occurrence models, but distributional is the most appropriate term common to all the models in that it fairly describes the environmental structure all learning mechanisms capitalize on (i.e., not all are truly spatial models, and most do not capitalize merely on direct co-occurrences). The various models differ greatly in the cognitive mechanisms they posit that humans use to construct semantic representations, ranging from Hebbian learning to probabilistic inference. But the unifying theme common to all these models is that they hypothesize a formal cognitive mechanism to learn semantics from repeated episodic experience in the linguistic environment (typically a text corpus). The driving theory behind modern distributional models of semantic representation is certainly not a new one, and dates back at least to Wittgenstein (1953). The most famous and commonly used phrase to summarize the approach is Firth’s (1957) “you shall know a word by the
Perhaps the best-known distributional model is Latent Semantic Analysis (LSA; Landauer & Dumais, 1997). LSA begins with a term-by-document frequency matrix of a text corpus, in which each row vector is a word’s frequency distribution over documents. A document is simply a “bag-of-words” in which transitional information is not represented. Next, a word’s row vector is transformed by its log frequency in the document and its information entropy over documents (− p (x) log2 p(x); cf. Salton & McGill, 1983). Finally, the matrix is factorized using singular-value decomposition (SVD) into three component matrices, U, , and V. The U matrix represents the orthonormal basis for a space in which each word is a point, V represents an analogous orthonormal document space, and
is a diagonal matrix of singular values (cf. an eigenvector) weighing dimensions in the space (see Landauer, McNamara, Dennis, & Kintsch, 2007 for a tutorial). The original transformed term-bydocument matrix, M, may be reconstructed as: M = U V T , where
VT
(1)
is the transpose of V.
models of semantic memory
239
More commonly, only the top N singular values of are retained, where N is usually around 300. This dimension reduction allows an approximation of the original “episodic” matrix to be reconstructed, and has the effect of bringing out higher-order statistical relationships among words more sophisticated than mere direct co-occurrence. A word’s semantic representation is then a pattern across the N latent semantic dimensions, and is often projected as a point in N -dimensional semantic space (cf. Osgood, 1952). Even though two words (e.g., boat and ship) might have had zero similarity in the original M matrix, indicating that they do not co-occur in the same documents, they may nonetheless be proximal in the reduced space reflecting their deeper semantic similarity (contextual similarity but not necessarily contextual overlap). The application of SVD in LSA is quite similar to common uses of principal component analysis (a type of SVD) in questionnaire research. Given a pattern of observable scale responses to items on a personality questionnaire, for example, the theorist may apply SVD to infer a small number of latent components (e.g., extroversion, neuroticism) that are causing the larger number of observable response patterns. Similarly, LSA uses SVD to infer a small number of latent semantic components in language that explain the pattern of observable word co-occurrences across contexts. In this sense, LSA was the first model to successfully specify a function mapping semantic memory to episodic context. Landauer and Dumais (1997) were careful not to claim that humans use exactly SVD as a learning mechanism, but rather that the brain uses some dimensional reduction mechanism akin to SVD to create abstract semantic representations from experience. The semantic representations constructed by LSA have demonstrated remarkable success at simulating a wide range of human behavioral data, including judgments of semantic similarity (Landauer & Dumais, 1997), word categorization (Laham, 2000), and discourse comprehension (Kintsch, 1998), and the model has also been applied to the automated scoring of essay quality (Landauer, Lahma, Rehder, & Schreiner, 1997). One of the most publicized feats of LSA was its ability to achieve a score on the Test of English as a Foreign Language (TOEFL) that would allow it entrance into most U.S. colleges (Landauer & Dumais, 1997). A critically important insight from the TOEFL simulation was that the model’s 240
higher level cognition
performance peaked at the reduced 300 dimensions compared to fewer or even the full dimensionality of the matrix. Even though the ability of the model (from an algebraic perspective) to reconstruct the original M matrix diminishes monotonically as dimensionality is reduced, its ability to simulate the human semantic data was better at the reduced dimensionalities. This finding supports the notion that semantic memory may simply be supported by a mental dimension reduction mechanism applied to episodic contexts. The dimension reduction operation brings out higher-order abstractions by glossing over variance that is idiosyncratic to specific contexts. The astute reader will note the similarity of this notion to the emergent behavior of the hidden layers of a connectionist network that also performs some dimensional reduction operation; we will return to this similarity in the discussion. The influence of LSA on the field of semantic modeling cannot be overstated. Several criticisms of the model have emerged over the years (see Perfetti, 1998), including the lack of incremental learning, neglect of word-order information, issues about what exact cognitive mechanisms would perform SVD, and concerns over its core assumption that meaning can be represented as a point in space. However, LSA clearly paved the way for a rapid sequence of advances in semantic models in the years since its publication.
Moving Window Models An alternative approach to learning distributional semantics is to slide an N -word window across a text corpus, and to apply some lexical association function to the co-occurrence counts within the window at each step. Although LSA represents a word’s episodic context as a document, moving-window models operationalize a word’s context in terms of the other words that it is commonly seen with in temporal contexts. Compared to LSA’s batch-learning mechanism, this allows moving-window models to gradually develop semantic structure from simple co-occurrence counting (cf. Hebbian learning), because a text corpus is experienced in a continuous fashion. In addition, several of these models inversely weight co-occurrence by how many words intervene between a target word and its associate, allowing them to capitalize on word-order information. The prototypical exemplar of a moving-window model is the Hyperspace Analogue to Language model (HAL; Lund & Burgess, 1996). In HAL,
a co-occurrence window (typically, the 10 words preceding and succeeding the target word) is slid across a text corpus, and a global word-by-word co-occurrence matrix is updated at each one-word increment of the window. HAL uses a ramped window in which co-occurrence magnitudes are weighted inversely proportional to distance from the target word. A word’s semantic representation in the model is simply a concatenation of its row and column vectors from the global co-occurrence matrix. The row and column vectors reflect the weighted frequency with which each word preceded and succeeded, respectively, the target word in the corpus. Obviously, the word vectors in HAL are both high dimensional and very sparse. Hence, it is common to only use the column vectors with the highest variance (typically about 10% of all words are then retained as ‘context’ words; Lund & Burgess, 1996). Considering its simplicity, HAL has been very successful at accounting for human behavior in semantic tasks, including semantic priming (Lund & Burgess, 1996), and asymmetric semantic similarity as well as higher-order tasks such as problem solving (Burgess & Lund, 2000). In HAL, words are most similar if they have appeared in similar positions relative to other words (paradigmatic similarity; e.g., bee-wasp). In fact, Burgess and Lund (2000) have suggested that the structure learned by HAL is very similar to what an SRN (Elman, 1990) would learn if it could scale up to such a large linguistic dataset. In contrast, it is known that LSA gives stronger weight to syntagmatic relations (e.g., bee-honey) than does HAL, since LSA ignores word order, and both types of similarity are important factors in human semantic representation (Jones, Kintsch, & Mewhort, 2006). Several recent modifications to HAL have produced models with state-of-the-art performance at simulating human data. One concern in the original model was that chance frequencies can produce spurious similarities in the global matrix: A higher frequency word has a greater chance of randomly occurring with any other word and, hence, highfrequency words end up being more semantically similar to a target independent of semantic similarity. Recent versions of HAL, such as Hidex (Shaoul & Westbury, 2006) factor out chance occurrence by weighting co-occurrence by inverse frequency of the target word, which is similar to LSA’s application of log-entropy weighting but after learning the matrix. A second modification to HAL was proposed by Rohde, Gonnerman, and Plaut (2004) in their
COALS model (Correlated Occurrence Analogue to Lexical Semantics). In COALS, there is no preceding/succeeding distinction within the moving window, and the model uses a co-occurrence association function based on Pearson’s correlation to factor out the confounding of chance cooccurrence due to frequency. Hence, the similarity between two words is their normalized covariational pattern over all context words. In addition, COALS performs SVD on this matrix. Although these are quite straightforward modifications to HAL, COALS heavily outperforms its predecessor on human tasks such as semantic categorization (Riordan & Jones, 2011). A similar moving window model was used by McDonald and Lowe (1998) to simulate semantic priming. In their model, there is no predecessor/successor distinction, but all words are simply represented by their co-occurrence in the moving window with a small number of predefined “context words.” Although many applications of HAL tabulate the entire matrix and then discard the 90% of column vectors with the least amount of variance, McDonald and Lowe’s context-word approach specifies the context words (columns) a priori, and it tabulates row vectors for each target word but only in relation to the predefined context words. This context word approach, in which as few as 100 context words are used as the columns, has also been successfully used by Mitchell et al. (2008) to predict fMRI brain activity associated with humans making semantic judgments about nouns. Slightly more modern versions of these context-word models use log likelihood or log odds rather than raw co-occurrence frequency as matrix elements (Lowe & McDonald, 2000), and some even apply SVD to the word-by-word matrix (e.g., Budi, Royer, & Pirolli, 2007) to bring out latent word relationships. Moving window models such as HAL have surprised the field with the array of “deep” semantic tasks they can explain with relatively simple learning algorithms based on counting repetitions. They also tie large-scale models of statistical semantics with other learning models such as compound cuing (McKoon & Ratcliff, 1992) and cross-situational word learning (Smith & Yu, 2008).
Random Vector Models An entirely different take on contextual representation is seen in models that use random representations for words that gradually develop semantic structure through repeated episodes of the models of semantic memory
241
word in a text corpus. The mechanisms used by these models are theoretically tied to mathematical models of associative memory. For this reason, random vector models tend to capitalize on both contextual co-occurrence as LSA does, and also associative position relative to other words as models like HAL and COALS do, representing both in a composite vector space. In the Bound Encoding of the Aggregate Language Environment model (BEAGLE; Jones & Mewhort, 2007), semantic representations are gradually acquired as text is experienced in sentence chunks. The model is based heavily on mechanisms from Murdock’s (1982) theory of item and associative memory. The first time a word is encountered, it is assigned a random initial vector known as its environmental vector, ei . This vector is the same each time the word is experienced in the text corpus, and is assumed to represent the relatively stable physical characteristics of perceiving the word (e.g., its visual form or sound). The random vector assumption is obviously an oversimplification, assuming that all words are equally similar to one another in their environmental form (e.g., dog is as similar to dug as it is to carburetor), but see Cox, Kachergis, Recchia, and Jones (2010) for a version of the model that builds in preexisting orthographic structure. In BEAGLE, each time a word is experienced in the corpus, its memory vector, mi , is updated as the sum of the random environmental vectors for the other words that occurred in context with it, ignoring high-frequency function words. Hence, in the short phrase “A dog bit the mailman,” the memory representation for dog is updated as mdog = ebit + emailman . In the same sentence, mbit = edog + emailman and mmailman = edog + ebit are encoded. Even though the environmental vectors are random, the memory vectors for each word in the phrase have some of the same random environmental structure summed into their memory representations. Hence, mdog , mbit , and mmailman all move closer to one another in memory space each time they directly co-occur in contexts. In addition, latent similarity naturally emerges in the memory matrix; even if dog and pitbull never directly cooccur with each other, they will become similar in memory space if they tend to occur with the same words (i.e., similar contexts). This allows higherorder abstraction, achieved in LSA by SVD, to emerge in BEAGLE naturally from simple Hebbian summation. Rather than reducing dimensionality after constructing a matrix, BEAGLE sets dimensionality a priori, and the semantic information is 242
higher level cognition
distributed across dimensions evenly. If fewer or more dimensions are selected (provided a critical mass is used), the information is simply distributed over fewer or more dimensions. Multiple runs of a model on the same corpus may produce very different vectors (unlike LSA or HAL), but the overall similarity structure of the memory matrix on multiple runs will be remarkably similar. In this sense, BEAGLE has considerable similarity to unsupervised connectionist models. The use of random environmental representations allows BEAGLE to learn information as would LSA, but in a continuous fashion and without the need for SVD. But the most interesting aspect of the model is that the random representations allow the model to encode word order information in parallel by applying an operation from signal processing known as convolution to bind together vectors for words in sequence. Convolution-based memory models have been very successful as models of both vision and paired-associate memory, and BEAGLE extends this mechanism to encode n-gram chunk information in the word’s representation. The model uses circular convolution, which binds together two vectors, with dimensionality n, into a third vector of the same dimensionality: for i = 0 to n−1: z i =
n−1
xjmodn ∗ y(i−j)modn (2)
j=0
BEAGLE applies this operation recursively to create an order vector representing all the environmental vectors that occur in sequences around the target word, and this order vector is also summed into the word’s memory vector. Hence, the memory vector becomes a pattern of elements that reflects the word’s history of co-occurrence with, and position relative to, other words in sentences. Words that appear in similar contexts and similar syntactic roles within sentences will become progressively more similar. Jones, et al. (2006) have demonstrated how this integration of context and order information in a single vector representation allows the model to better account for patterns in semantic priming data. An additional benefit of having order information encoded in a word’s memory vector is that the convolution mechanism used to encode sequence information may be inverted to decode sequential expectancies for a word from its learned history. This decoding operates in a similar fashion to how Murdock (1982) retrieves an associated target given a cue in paired-associate learning. The model can make inferences about likely transitions preceding
or following a word and can build up expectancies for which words should be upcoming in sentence processing tasks using the same associative mechanism it uses for learning (see Jones & Mewhort, 2007). Although it only learns lexical semantic structure, BEAGLE naturally displays complex rulelike syntactic behavior as an emergent property of its lexicon. Further, it draws a theoretical bridge between models of lexical semantics and associative memory suggesting that they may be based on the same cognitive mechanisms. A similar approach to BEAGLE, known as random indexing, has been taken by Kanerva and colleagues (Kanerva, 2009; Kanerva, Kristoferson, & Holst, 2000). Random indexing uses similar principles to BEAGLE’s summation of random environmental vectors, but is based on Kanerva’s (1988) theory of sparse distributed memory. The initial vector for a word in random indexing is a sparse binary representation, a very high dimensional vector in which most elements are zeros with a small number of random elements switched to ones (a.k.a., a “spatter code”). A word’s memory representation is then a sum of initial vectors for the other words with which it has appeared in contexts. Words that are semantically similar will tend to be additive on the random elements that they share nonzero values on, which leads to a similarity structure remarkably similar to LSA, but without the need for SVD. Sahlgren, Holst, & Kanerva (2008) have extended random indexing to encode order information as does BEAGLE in their Random Permutation Model (RPM). The RPM encodes contextual information the same way as standard random indexing. Rather than convolution, it uses a permutation function to encode the order of words around a target word. The permutation function may be applied recursively to encode multiple words at multiple positions around the target word, and this order vector is also added to the word’s memory representation. Like BEAGLE, a word’s memory vector is a distributed pattern that contains information about its co-occurrence with and position relative to other words. However, in RPM this representation is a sparse hyperdimensional vector that contains less noise than does BEAGLE’s dense Gaussian vectors. In comparative simulations, RPM has been shown to outperform BEAGLE on simple associative tasks (Recchia, Jones, Sahlgren, & Kanerva, 2010). Howard and colleagues (e.g., Howard, Shakar, & Jagadisan, 2011) have taken a different approach
to learning semantic representations, binding local item representations to a gradually changing representation of context by modifying the Temporal Context Model (TCM; Howard & Kahana, 2002) to learn semantic information from a text corpus. The TCM uses static vectors representing word form, similar to RPM’s initial vectors or BEAGLE’s environmental vectors. However, the model binds words to temporal context, a representation that changes gradually with time, similar to oscillatorbased systems. In this sense, the model is heavily inspired by hippocampal function. Encountering a word reinstates its previous temporal contexts when encoding its current state in the corpus. Hence, whereas LSA, HAL, and BEAGLE all treat context as a categorical measure (documents, windows, and sentences, respectively, are completely different contexts), TCM treats context as a continuous measure that is gradually changing over time. In addition, although all the aforementioned models are essentially batch learners or ignore previous semantic learning when encoding a word, a word’s learned history in TCM contributes to its future representation. This is a unique and important feature of TCM compared to other models. Howard et al. (2011) trained a predictive version of TCM (pTCM) on a text corpus to compare to established semantic models. The pTCM continuously attempts to predict upcoming words based on reinstated temporal context. In this sense, the model has many features in common with both BEAGLE and SRNs (Elman, 1990), allowing it to represent both context and order information within the same composite representation. Howard et al. demonstrate impressive performance from pTCM on linguistic association tasks. In addition, the application of TCM in general to semantic representation makes a formal link to mechanisms of episodic memory (which at its core, TCM is), as well as findings in cognitive neuroscience (see Polyn & Kahana, 2008).
Probabilistic Topic Models Considerable attention in the cognitive modeling literature has recently been placed on Bayesian models of cognition (see Austerweil, et al., this volume), and mechanisms of Bayesian inference have been successfully extended to semantic memory as well. Probabilistic topic models (Blei, Ng, & Jordan, 2003; Griffiths, Steyvers, & Tenenbaum, 2007) operate in a similar fashion to LSA, performing statistical inference to reduce the dimensionality of a term-by-document matrix. However, the models of semantic memory
243
theoretical mechanisms behind the inference and representation in topic models differ markedly from LSA and other spatial models. An assumption of a topic model is that documents are generated by mixtures of latent “topics,” in which a topic is a probability distribution over words. Although LSA makes a similar assumption that latent semantic components can be inferred from observable co-occurrences across documents, topic models go a step further, specifying a fully generative model for documents (a procedure by which documents may be generated). The assumption is that when constructing documents, humans are sampling a distribution over universal latent topics. For example, one might construct a document about a recent beetle infestation by mixing topics about insects, forests, the ecosystem, etc., with varying weights. To generate each word within this document, one samples a topic according to the document’s mixture weights, and then samples words from that topic’s probability distribution over words. To train the model, Bayesian inference is used to reverse the generative process: Assuming that topic mixing is what generates documents, the task of the model is to invert the process and statistically infer the set of topics that were responsible for generating a given set of documents. The formal instantiation of a topic model can be technically intimidating to the novice modeler—based on Latent Dirichlet Allocation algorithms, Markov Chain Monte Carlo algorithms etc. (see Griffiths et al., 2007; Griffiths, Steyvers, Blei, & Tenenbaum, 2005). However, it is important to note that the theoretical ideas underlying the model are actually quite simple and elegant and are based on the same ideas posited for how children infer unseen causes for observable events (Tenenbaum, Kemp, Griffiths, & Goodman, 2011). Consider the analogy of a dermatologist: Given that disease X is present, symptoms A, B, and C are expected to manifest. The task of a dermatologist is one of causal inference, however—given a set of co-occurring symptoms she must infer the unseen disease or diseases that produced the observed data. Over many instances of the same co-occurring symptoms, she can infer the likelihood that they are the result of a common cause. The topic model works in an analogous way, but on a much larger scale of inference and with mixtures of causal variables. Given that certain words tend to co-occur in contexts and this pattern is consistent over many contexts, the model infers the likely latent 244
higher level cognition
“topics” that are responsible for generating the cooccurrence patterns, in which each document is a probabilistic mixture of these topics. Each topic is a probability distribution over words, and a word’s meaning can be captured by the probability that it was generated by each topic (just as each disease would be a probability distribution over symptoms, and a symptom is a probability distribution over possible diseases that generated it). Figure 11.3, reproduced from Steyvers and Griffiths (2007), illustrates this process. Assuming that document co-occurrences are being generated by the process on the left, the topic model attempts to statistically infer (on the right) the most likely topics and mixtures that would have generated the observed data. It is important to note that, like LSA, topic models tend to assume a simple bag-of-words representation of a document, neglecting wordorder information (but see Andrews & Vigliocco, 2010; Griffiths, et al., 2005). Similar to LSA, each document in the original co-occurrence matrix may be reconstructed by determining the document’s distribution over N topics (reflecting its gist, g), using this distribution to select a topic for each word wi , and then generating a word from the distribution of words conditioned on the topic: N P(wi |zi )P zi g , P wi g =
(3)
zi =1
where w is the distribution of words over topics, and z is the distribution of topics over words. In practice, topic models construct a prior value on the degree of mixing of topics in a document, and then estimate the probability distributions of topics over words and documents over topics using Gibbs sampling (Griffiths & Steyvers, 2004). The probabilistic inference machinery behind topic models results in at least three major differences in topic models when compared to other distributional models. First, as mentioned earlier, topic models are generative. Second, it is often suggested that the topics themselves have a meaningful interpretation, such as finance, medicine, theft, and so on, whereas the components of LSA are difficult to interpret, and the components of models like BEAGLE are purposely not interpretable in isolation from the others. It is important to note, however, that since the number of topics (and the value of the priors) is set a priori by the theorist, there is often a considerable amount of hand-fitting and intuition that can go into constructing topics that are meaningful (similar to “labeling” factors
PROBABILISTIC GENERATIVE PROCESS ey
mo
1.0
k
loan ba n
bank
loan
ne mon y bank loan
ri v er
TOPIC 1
.5
DOC2: money1 bank1 bank2 river2 loan1 stream2 bank1 money1
1.0
DOC3: river2 bank2 stream2 bank2 river2 river2 stream2 bank2
river
s t r eam ban k
r
DOC1: money? bank? loan? bank? money? money? bank? loan?
?
.5
nk
ba m a stre iver
DOC1: money1 bank1 loan1 bank1 money1 money1 bank1 loan1
STATISTICAL INFERENCE
TOPIC 2
TOPIC 1
?
?
DOC2: money? bank? bank? river? loan? stream? bank? money? DOC3: river? bank? stream? bank? river? river? stream? bank?
TOPIC 2
Fig. 11.3 Illustration of the generative process (left) and the problem of statistical inference (right) underlying topic models. (Reproduced from Steyvers & Griffiths, 2007). Steyvers, M., & Griffiths, T. (2008). Probabilistic topic models. In T. Landauer, D. McNamara, S. Dennis, & W. Kintsch (Eds.) Handbook of Latent Semantic Analysis, NJ: Erlbaum. With kind permission from the Taylor and Francis Group.
Box 1 So Which Model Is Right? It is tempting to think of different distributional models as competing “brands.” However, a potentially more fruitful approach is to consider each specific model as a point in a parameter space, as one would with other cognitive models. Each model is really just a particular set of decisions made to formalize the distributional theory of “knowing a word by the company it keeps” (Firth, 1957), and no single model has emerged victorious at accounting for the wide range of semantic behavioral data. Each model has its merits and shortcomings. How should distributional models be compared? If a model is being proposed as a psychological model, it is important to identify the model’s subprocesses. How do those subprocesses contribute to how the model works? How are they related to other psychological theories? And how do they contribute to the model’s ability to predict behavioral data? For example, LSA and HAL vary in a large number of ways (see Table 1). Studies that perform simple model comparisons end up confounding these differences, leaving us unsure what underlying psychological claim is being tested. Most model differences can be ascribed to one of three categories, each corresponding to important differences in the underlying psychological theory: 1. Representational structure: What statistical information does the model pay
attention to, and how is this information initially represented? 2. Representational transformations: By what function are the representations transformed to produce a semantic space? 3. Comparison process: How is the semantic space queried, and how is the semantic information, relations, or similarity used to model behavioral data? The HAL model defines its representations in terms of a word-by-word co-occurrence matrix, whereas the LSA model defines its representation in terms of a word-counts-withindocuments matrix. This difference corresponds to a long tradition of different psychological theories. HAL’s word-word co-occurrences are akin to models that propose representations based on associations between specific stimuli (such as classical associationist theories of learning). In contrast, LSA’s word-by-document representation proposes representations based associations between a stimuli and abstract pointers to the event in which those stimuli participate (similar to classic context association theories of learning). A number of studies have begun comparing model performance as a function of differences in these subprocesses (e.g. Bullinaria & Levy, 2012; Shaoul & Westbury, 2010), but much more research is needed before any firm conclusions can be made.
models of semantic memory
245
in a factor analysis). Third, words in a topic model are represented as probability distributions rather than as points in semantic space; this is a key distinction between topic models and the earlier spatial models. It allows topic models to naturally display asymmetric associations, which are commonly seen in free association data but require additional assumptions to explain with spatial models (Griffiths, et al., 2007; but see Jones, Gruenenfelder, & Recchia, 2011). Representing a word’s meaning with a probability distribution also naturally allows for polysemy in the representation compared to vector representation models that collapse multiple meanings to a single point. For these reasons, topic models have been shown to produce better fits to free association data than LSA, and they are able to account for disambiguation, word-prediction, and discourse effects that are problematic for LSA (Griffiths et al., 2007).
Box 2 Semantic Memory Modeling Resources A chapter on semantic models would seem incomplete without some code! Testing models of semantic memory has become much easier due to an increase in semantic modeling resources. There are now a wide variety of software packages that provide the ability to construct and test semantic models. The software packages vary in terms of their easy of installation and use, flexibility, and performance. In addition to the software packages, a limited number of Web-based resources exist for doing simple comparisons online. You may test models on standardized datasets, train them on your own corpora for semantic exploration, or use them for generating stimuli. Software Packages • HiDEx (http://www.psych.ualberta.ca/ westburylab/downloads/HiDEx.download. html): A C++ implementation of the HAL model; it is useful for constructing large word-by-word co-occurrence matrices and testing a wide variety of possible parameters. • SuperMatrix (http://semanticore.org/ supermatrix/): A python implementation of a large number of semantic space model transformations (including PCA/SVD, Latent Dirichlet Allocation, and Random Vector Accumulation) on both word-by-word
246
higher level cognition
and word-by-document spaces. SuperMatrix was designed to emphasize the exchangeability of various sub-processes within semantic models (see Box 1), to allow isolation and testing the effects of specific model components. • GenSim (http://radimrehurek.com/ gensim/): A python module that is very fast and efficient for constructing and testing word-by-document models, including LSA (reduced using SVD) and Topics (reduced using Latent Dirichlet Allocation). • S-Space (https://github.com/ fozziethebeat/S-Space): A Java-based implementation of a large number of semantic space models, including HAL, LSA, BEAGLE, and COALS. • SEMMOD (http://mall.psy.ohio-state. edu/wiki/index.php/Semantic_Models_Package_ (SEMMOD)): A python package to implement and compare many of the most common semantic models. • Word-Similarity (https://code.google. com/p/wordsimilarity/wiki/train): A tool to explore and visualize semantic spaces, displayed as directed graphical networks. Web-Based Resources • http://lsa.colorado.edu: The original LSA website provides the ability to exploreLatent Semantic Analysis with a wide variety of different metrics, including word-word similarities, similarities of passages of text to individual words, and similarities of passages of texts to each other. • http://semanticore.org: The Semanticore website is a web portal designed to bring data from many semantic models and psycholinguistic databases under one roof. Users can obtain frequency and co-occurrence statistics from a wide variety of corpora, as well as semantic similarities from a number of different semantic memory models, including HAL, LSA, BEAGLE, and Probabilistic Topics Models.
Retrieval-Based Semantics Kwantes (2005) proposed an alternative approach to modeling semantic memory from distributional structure. Although not named in his
publication, Kwantes’s model is commonly referred to as the constructed semantics model (CSM), a name that is paradoxical given that the model posits that there is no such thing as semantic memory. Rather, semantic behavior exhibited by the model is an emergent artifact of retrieval from episodic memory. Although all other models put the semantic abstraction mechanism at encoding (e.g., SVD, Bayesian inference, vector summation), CSM actually encodes the episodic matrix and performs abstraction as needed when a word is encountered. CSM is based heavily on Hintzman’s (1986) Minerva 2 model which was used as an existence proof that a variety of behavioral effects that had been used to argue for two distinct memory stores (episodic and semantic) could naturally be produced by a model that only had memory for episodes. So-called “prototype effects” were simply an artifact of averaging at retrieval in the model, not necessarily evidence of a semantic store. CSM extends Minerva 2 almost exactly to a text corpus. In CSM, the memory matrix is the term-bydocument matrix (i.e., it assumes perfect memory of episodes). When a word is encountered in the environment, its semantic representation is constructed as an average of the episodic memories of all other words in memory, weighted by their contextual similarity to the target. The result is a vector that has higher-order semantic similarities accumulated from the lexicon. This semantic vector is similar in structure to the memory vector learned in BEAGLE by context averaging, but the averaging is done on the fly, it is not encoded or stored. Although retrieval-based models have received less attention in the literature than models like LSA, they represent a very important link to other instance-based models, especially exemplar models of recognition memory and categorization (e.g., Nosofsky, 1986). The primary reason limiting their uptake in model applications is likely due to the heavy computational expense required to actually simulate their process (Stone, Dennis, & Kwantes, 2011).
Grounding Semantic Models Semantic models, particularly distributional models, have been criticized as psychologically implausible because they learn from only linguistic information and do not contain information about sensorimotor perception contrary to the grounded cognition movement (for a review, see de Vega, Glenberg, & Graesser, 2008). Hence, representations in distributional models are not a replacement
for feature norms. Feature-based representations contain a great deal of sensorimotor properties of words that cannot be learned from purely linguistic input, and both types of information are core to human semantic representation (Louwerse, 2008). Riordan and Jones (2011) recently compared a variety of feature-based and distributional models on semantic clustering tasks. Their results demonstrated that whereas there is information about word meaning redundantly coded in both feature norms and linguistic data, each has its own unique variance and the two information sources serve as complimentary cues to meaning. Research using recurrent networks trained on child-directed speech corpora has found that pretraining a network with features related to children’s sensorimotor experience produced significantly better word learning when subsequently trained on linguistic data (Howell, Jankowicz, & Becker, 2005). Durda, Buchanan, and Caron (2009) trained a feed-forward network to associate LSAtype semantic vectors with their corresponding activation of features from McRae, Cree, Seidenberg, and McNorgan’s (2005) norms. Given the semantic representation for dog, the model attempts to activate correct output features such as and inhibit incorrect output featuressuch as. After training, the network was able to infer the correct pattern of perceptual features for words that were not used in training because of their linguistic similarity to words that were learned. Several recent probabilistic topic models have also explored parallel learning of linguistic and featural information (Andrews, Vigliocco, & Vinson, 2009; Baroni, Murphy, Barba, & Poesio, 2010; Steyvers, 2009). Given a word-by-document representation of a text corpus and a word-byfeature representation of feature production norms, the models learn a word’s meaning by simultaneously considering inference across documents and features. This enables learning from joint distributional information: If the model learns from the feature norms that sparrows have beaks, and from linguistic experience that sparrows and mockingbirds are distributionally similar, it will infer that mockingbirds also have beaks, despite having no feature vector for mockingbird. Integration of linguistic and sensorimotor information allows the models to better fit human semantic data than a model trained with only one source (Andrews et al., 2009). This information integration is not unique to Bayesian models but can also be accomplished models of semantic memory
247
Table 11.1. Highly cited semantic models and the specific subprocesses that comprise the models. Model
HAL COALS LSA Topic Models BEAGLE
Representational Structure
Representational Transformation
Comparison Process
Unit Type
Unit Span
Normalization
Abstraction
Ordered word-by-word co-occurrence matrix Ordered word-by-word co-occurrence matrix Unordered word-by-document count matrix Unordered word-by-document count matrix Ordered word-by-word matrix
Distance weighted 10word window 10-word window
Conditional probabilities (matrix row sum) Correlational normalization
None
City block distance similarity
Principle components analysis
Correlational similarity
Predefined document
Log entropy
Singular value decomposition
Cosine similarity
Predefined document
None
Latent Dirichlet allocation
Inner product similarity
Sentence window
None
Random vector accumulation
Cosine similarity
within random vector models (Jones & Recchia, 2010; Vigliocco, Vinson, Lewis, & Garrett, 2004) and retrieval-based models (Johns & Jones, 2012).
Compositional Semantics The models we have considered thus far are designed to extract the meaning of individual terms. However, the sentence “John loves Mary” is not just the sum of the words it contains. Rather “John” is bound to the role LOVER and “Mary” is bound to the role LOVEE. The study of how sentence structure determines these bindings is called compositional semantics. Recent work has begun to explore mechanisms for compositional semantics by applying learning mechanisms to the already learned lexicon of a distributional model (Mitchell & Lapata, 2010). Dennis (2004, 2005) argued that extracting propositional structure from sentences revolves around the distinction between syntagmatic and paradigmatic associations. Syntagmatic associations occur between words that appear together in utterances (e.g., run fast). Paradigmatic associations occur between words that appear in similar contexts, but not necessarily in the same utterances (e.g., deep and shallow). The syntamatic paradigmatic model proposes that syntagmatic associations are used to determine that words could have filled a particular slot within a sentence. The set of these words form role vectors that are then bound to fillers by paradigmatic associations to form a propositional representation of the sentence. The syntagmatic paradigmatic mechanism has been shown to be capable of accounting for a wide range of sentence-processing phenomena. Furthermore, it is capable of taking advantage of regularities in the overlap of role patterns to create implicit inferences that Dennis (2005) claimed are responsible for the robustness of human commonsense reasoning.
Common Lessons and Future Directions Models of semantic memory have seen impressive developments over the past two decades that have greatly advanced our understanding of how humans create, represent, and use meaning from experience. These developments have occurred thanks in part to advances in other areas, such as machine learning and to better large-scale norms of semantic data on which to fit and compare the models. In general, distributional models have been successfully used to better explore the statistical
structure of the environment and to understand the mechanisms that may be used to construct semantic representations. Connectionist models are an excellent compliment, elucidating our understanding of semantic processing, and how semantic structure interacts with other cognitive systems and tasks. An obvious and important requirement for the future is to start to bring these insights together, and several hybrid models are now emerging in the literature. Several important themes have emerged that are common to both the connectionist and distributional literatures. The first is data reduction. Whatever specific mechanism humans are using to construct conceptual and propositional knowledge from experience, it is likely that this mechanism learns by focusing on important statistical factors that are constant across contexts, and by throwing away factors that are idiosyncratic to specific contexts. In a sense, capacity constraints on human encoding, storage, and retrieval may give rise to our incredible ability to construct and use meaning. A second common theme is the importance of data scale in semantic modeling. In both connectionist and distributional models, the issue of data scale versus mechanistic complexity has been brought to the forefront of discussion in the literature. A consistent emerging theme is that simpler models tend to give the best explanation of human data, both in terms of parsimony and quantitative fit to the data, when they are trained on linguistic data that is on a realistic scale to what humans experience. For example, although simple context-word moving-window models are considerably simpler than LSA and do not perform well at small data scales, they are capable of scaling up to learn from human-scale amounts of linguistic data (a scale not necessarily possible to learn with LSA), and consistently outperform more complex models such as LSA with large data (e.g., Louwerse, 2011; Recchia & Jones, 2009). This leads to potential concern that earlier theoretical advancements with models trained on so-called “toy datasets” (artificial language corpora constructed to test the model’s structural learning) may have been overly complex. To fit human behavioral data with a corpus that is far smaller and without the deep complexities inherent in real language, the model must necessarily be building complexity into the architecture and mechanism whereas humans may be using a considerably simpler mechanism, offloading considerable statistical complexity already present in their linguistic environment. models of semantic memory
249
A third common theme is that complex semantic structures and behaviors may be an emergent property of the lexicon. Emergence is a key property of connectionist models, and we have seen that complex representations of schemata, hierarchical categories, and syntactic processing may be emergent consequences of many connectionist models (e.g., Rogers & McClelland, 2004). But emergence is also a natural consequence of distributional models. In several cases, the same mechanisms used to learn semantic representations may be applied to the learned representations to simulate complex behaviors, such as BEAGLE’s ability to model sentence comprehension as an emergent property of order information distributed across the lexicon (Jones & Mewhort, 2007). Topic models also possess a natural mechanism for producing asymmetric similarity and polysemous processing through conditional inference. Learning to organize the mental lexicon is one of the most important cognitive functions across development, laying the fundamental structure for future semantic learning and communicative behavior. Semantic modeling has a very promising future, with potential to further our understanding of basic cognitive mechanisms that give rise to complex meaning structures, and how these mental representations are used in a wide range of higherorder cognitive tasks.
Acknowledgments This work was supported by National Science Foundation grant BCS-1056744 and National Institutes of Health grant R01MH096906 to MNJ, and Defence Research and Development Canada grant W7711-067985 to SD. JW was supported by postdoctoral training grant NIH T32 DC000012.
Note 1. One interpretation of feature comparison given by Rips et al., 1973 was also spatial distance.
Glossary Compositional Semantics: The process by which a complex expression (e.g., a phrase or sentence) is constructed from the meanings of its constituent concepts. Concept: A mental representation generalized from particular instances, and knowledge of its similarity to other concepts.
250
higher level cognition
Connectionist Model: A model that represents knowledge as weighted network of interconnected units. Behavioral phenomena are an emergent process of the full network. Context: In semantic models, context is typically considered the “window” within which two words may be considered to co-occur, and it is one of the major theoretical differences between distributional models. Context may be considered to be discrete units, such as sentences or documents, or it may be more continuous, such as in moving-window or temporal context models. Distributional Model: A general approach to concept learning and representation from statistical redundancies in the environment. Dynamic Network: A connectionist network whose architecture involves bi-directionality, feedback, or recurrent connectivity. Feature Comparison Model: A classic model of semantic memory that represents concepts as vectors of binary features representing the presence or absence of features. For example, the has_wings element would be turned on for ROBIN, but off for GOLDEN RETRIEVER. Paradigmatic and Syntagmatic Relations: Paradigmatic similarity between two words emphasizes their synonymy or substitutability (bee-wasp), whereas syntagmatic similarity emphasizes associative or event relations (e.g., bee-honey, wasp-sting). Proposition: A mental representation of conceptual relations that may be evaluated to have a truth value, for example, knowledge that birds have wings. Random Vector Model: A semantic model that begins with some sort of randomly generated vector to initially represent a concept. Over linguistic experience, an aggregating function gradually produces similar vector patterns among words that are semantically related. They allow for study of the time course of semantic acquisition. Semantic Memory: Memory for word meanings, facts, concepts, and general world knowledge. Typically not tied to a specific event. Semantic Network: A classic graphical model of semantic memory that represents concepts as nodes and semantic relations as labeled edges between the nodes. Often, the hypothetical process of spreading activation is used to simulate behavioral data such as semantic priming from a semantic network. Singular-Value Decomposition: A statistical method of factorizing an m × n matrix, M, into an m × m unitary matrix, U, an m × n diagonal matrix, , with diagonal entries that are the singular values, and an n × n unitary matrix, V. The original matrix may be recomposed as M = UVT , where VT is the transpose of V. Spatial Model: A model that represents word meaning as a point in a multidimensional space, and that typically applies a geometric function to express conceptual similarity. Supervised and Unsupervised Learning: Supervised learning typically trains the model on a set of labeled exemplars (i.e., the true output of each training exemplar is known), whereas in unsupervised learning the model must
Glossary discover structure in the data without the benefit of known labels. Topic Model: A generative probabilistic model that uses Bayesian inference to abstract the mental “topics” used to compose a set of documents.
References Aakerlund, & L., Hemmingsen, R. (1998). Neural networks as models of psychopathology. Biological Psychiatry, 43, 471– 482. Allen, J., & Seidenberg, M. S. (1999). The emergence of grammaticality in connectionist networks. In B. MacWhinney (Ed.), Emergentist approaches to language: proceedings of the 28th Carnegie symposium on cognition (pp. 115–151). Hillsdale, NJ: Earlbaum. Andrews, M., & Vigliocco, G. (2010). The hidden-Markov topic model: A probabilistic model of semantic representation. Topics in Cognitive Science, 2, 101–113. Andrews, M., Vigliocco, G. & Vinson, D. P. (2009). Integrating experiential and distributional data to learn semantic representations. Psychological Review, 116, 463–498. Baroni, M., Murphy, B., Barbu, E., & Poesio, M. (2010). Strudel: A corpus-based semantic model based on properties and types. Cognitive Science, 34, 222–254. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. the Journal of machine Learning research, 3, 993– 1022. Borovsky, A., & Elman, J. (2006). Language input and semantic categories: A relation between cognition and early word learning. Journal of Child Language, 33(04), 759–790. Botvinick, M., & Plaut, D. (2004). Doing without schema hierarchies: A recurrent connectionist approach to normal and impaired sequential action. Psychological Review, 111, 395–429. Bower, G. H. (1970). Organizational factors in memory. Cognitive Psychology, 1, 18–46. Braver, T. S., Barch, D. M., Cohen, J. D. (1999). Cognition and control in schizophrenia: a computational model of dopamine and prefrontal function. Biological Psychiatry, 46, 312–328. Budiu, R., Royer, C., Pirolli, P. L. (2007). Modeling information scent: A comparison of LSA, PMI and GLSA similarity measures on common tests and corpora. Proceedings of Recherche d’Information Assistée par Ordinateur (RIAO). Pittsburgh, PA, 8. Bullinaria, J., A., & Levy, J., P. (2007). Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior Research Methods, 39, 510–526. Burgess, C. & Lund, K. (2000) The dynamics of meaning in memory. In E. Dietrich & A. B. Markman (Eds.), Cognitive dynamics: Conceptual and representational change in humans and machines (pp. 117–156). Mahwah, NJ: Erlbaum. Chan, A. S., Salmon, D. P., & Butters, N. (1998). 16 Semantic Network Abnormalities in Patients with Alzheimer’s Disease. Fundamentals of neural network modeling: Neuropsychology and cognitive neuroscience, 381.
Chang, F., Dell, G. S., & Bock, K. (2006). Becoming syntactic. Psychological Review, 113(2), 234–272. Collins, A. M., & Loftus, E. F. (1975). A spreading-activation theory of semantic processing. Psychological Review, 82, 407– 428. Collins, A. M., & Quillian, M. R. (1969). Retrieval time from semantic memory. Journal of Verbal Learning & Verbal Behavior, 8(2), 240–247. Cox, G. E., Kachergis, G., Recchia, G., & Jones, M. N. (2011). Toward a scalable holographic word-form representation. Behavior research methods, 43(3), 602–615. Cree, G. S., McRae, K., & McNorgan, C. (1999). An attractor model of lexical conceptual processing: Simulating semantic priming. Cognitive Science, 23, 371–414. Cohen, J. D., Servan-Schreiber, D. (1992). Context, cortex, and dopamine: A connectionist approach to behavior and biology in schizophrenia. Psychological Review, 99, 45–77. Dell, G. S., Chang, F., & Griffin, Z. M. (1999). Connectionist models of language production: Lexical access and grammatical encoding. Cognitive Science, 23( 4), 517–542. Dell, G. S., Juliano, C., & Govindjee, A. (1993). Structure and content in language production: A theory of frame constraints in phonological speech errors. Cognitive Science, 17, 149–195. Dennis, S. (2004). An unsupervised method for the extraction of propositional information from text. Proceedings of the National Academy of Sciences of the United States of America, 101(Suppl 1), 5206–5213. Dennis, S. (2005). A memory-based theory of verbal cognition. Cognitive Science, 29, 145–193. De Vega, M., Glenberg, A. M., & Graesser, A. C. (2008). Symbols and embodiment: debates on meaning and cognition. Oxford, UK: Oxford University Press. Devlin, J. T., Gonnerman, L. M., Anderson, E. S., & Seidenberg, M. S. (1998). Category specific semantic deficits in focal and widespread brain damage: A computational account. Journal of Cognitive Neuroscience, 10, 77–94. Durda, K., Buchanan, L., & Caron, R. (2009). Grounding co-occurrence: Identifying features in a lexical co-occurrence model of semantic memory. Behavior Research Methods, 41, 1210–1223. Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179–211. Farah, M. J., & McClelland, J. L. (1991). A computational model of semantic memory impairment: Modality- specificity and emergent category-specificity. Journal of Experimental Psychology: General, 120, 339–357. Firth, J. R. (1957). A synopsis of linguistic theory, 1930-1955. In Philological Society (Great Britain) (Ed.), Studies in linguistic analysis. Oxford, England: Blackwell. Gasser, M., & Smith, L. B. (1998). Learning nouns and adjectives: A connectionist account. Language and cognitive processes, 13(2-3), 269–306. Griffiths, T. L., & Steyvers, M. (2004). Finding scientific topics. Proceedings of the National Academy of Sciences, 101, 5228– 5235. Griffiths, T. L., Steyvers, M., Blei, D. M., & Tenenbaum, J. B. (2005). Integrating topics and syntax. Advances in Neural Information Processing Systems, 17, 537–544.
models of semantic memory
251
Griffiths, T. L., Steyvers, M., & Tenenbaum, J. B. (2007). Topics in semantic representation. Psychological Review, 114, 211– 244. Grossberg, S. (1976). Adaptive pattern classification and universal recoding, I: Parallel development and coding of neural feature detectors. Biological Cybernetics, 23, 121–134. Harm, M. W., & Seidenberg, M. S. (2004). Computing the meanings of words in reading: Cooperative division of labor between visual and phonological processes. Psychological Review, 111, 662–720. Harris, Z. (1970). Distributional structure. In Papers in Structural and Transformational Linguistics (pp. 775–794). Dordrecht, Holland: D. Reidel. Hebb, D. (1946). The Organization of Learning. Hinton, G. E. & Shallice, T. (1991). Lesioning an attractor network: Investigations of acquired dyslexia. Psychological Review, 98, 74–95. Hintzman, D. L. (1986). "Schema abstraction" in a multiple˝ trace memory model. Psychological Review, 93, 411U428. Horst, J. S., McMurray, B., and Samuelson, L. K. (2006) Online processing is essential for learning: Understanding fast mapping and word learning in a dynamic connectionist architecture. In R. Sun (Ed) Proceedings of the 28th meeting of the Cognitive Science Society (pp. 339–334). Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, 79, 2554–2558. Howard, M. W., & Kahana, M. J. (2002). A distributed representation of temporal context. Journal of Mathematical Psychology, 46, 269–299. Howard, M. W., Shankar, K. H., & Jagadisan, U. K. (2011). Constructing semantic representations from a gradually changing representation of temporal context. Topics in Cognitive Science, 3, 48–73. Howell, S., Jankowicz, D., & Becker, S. (2005). A model of grounded language acquisition: Sensorimotor features improve lexical and grammatical learning. Journal of Memory and Language, 53, 258–276. Hummel, J. E., & Holyoak, K. J. (2003). A symbolicconnectionist theory of relational inference and generalization. Psychological Review, 110, 220–264. Johns, B. T., & Jones, M. N. (2012). Perceptual inference from global lexical similarity. Topics in Cognitive Science, 4, 1, 103– 120. Jones, M. N., Kintsch, W., & Mewhort, D. J. K. (2006). Highdimensional semantic space accounts of priming. Journal of Memory and Language, 55, 534–552. Jones, M. N., Gruenenfelder, T. M., & Recchia, G. (2011). In defense of spatial models of lexical semantics. In Proceedings of the 33rd annual conference of the cognitive science society (pp. 3444–3449). Jones, M. N., & Recchia, G. (2010). You can’t wear a coat rack: A binding framework to avoid illusory feature migrations in perceptually grounded semantic models. Proceedings of the 32nd Annual Cognitive Science Society, 877–882. Jones, M. N., & Mewhort, D. J. K. (2007). Representing word meaning and order information in a composite holographic lexicon. Psychological Review, 114, 1–37. Kanerva, Pentti. Sparse distributed memory. MIT press, 1988.
252
higher level cognition
Kanerva, P. (2009). Hyperdimensional computing: An introduction to computing in distributed representations with high-dimensional random vectors. Cognitive Computation, 1, 139–159. Keil, F. C. (1989). Concepts, kinds, and cognitive development. Cambridge, MA: MIT Press. Kinder, A., & Shanks, D. R. (2003). Neuropsychologicla dissociations between priming and recognition: A singlesystem connectionist account. Psychological Review, 110, 728–744. Kintsch, W. (1998). Comprehension: A paradigm for cognition. New York, NY: Cambridge University Press. Kohonen, T. (1982). Self-organized formation of topologically correct feature maps. Biological Cybernetics, 43, 59–69. Kwantes, P. J. (2005). Using context to build semantics. Psy Bull & Review, 12, 703–710. Landauer, T., & Dumais, S. (1997). A solution to Plato’s problem: The Latent Semantic Analysis theory of the acquisition, induction, and representation of knowledge. Psychological Review, 104, 211–240. Laham, R. D. (2000). Automated content assessment of text using latent semantic analysis to simulate human cognition (Doctoral dissertation, University of Colorado). Landauer, T. K., McNamara, D. S., Dennis, S., & Kintsch, W. (Eds.). (2013). Handbook of latent semantic analysis. Psychology Press. Landauer, T. K., Laham, D., Rehder, B., & Schreiner, M. E. (1997). How well can passage meaning be derived without using word order? A comparison of latent semantic analysis and humans. In M. G. Shafto & P. Langley (Eds.), Proceedings of the 19th Annual Meeting of the Cognitive Science Society (pp. 412–417). Mahwah, NJ: Erlbaum. Li, P. (2009). Lexical organization and competition in first and second languages: Computational and neural mechanisms. Cognitive science, 33(4), 629–664. Li, P., Zhao, X., & Mac Whinney, B. (2007). Dynamic Self−Organization and Early Lexical Development in Children. Cognitive science, 31(4), 581–612. Lowe, W. and McDonald, S. (1998). Modelling functional priming and the associative boost. In M. A.Gernsbacher & S. D. Derry, (Eds.). Proceedings of the 20th Annual Meeting of ˝ the Cognitive Science Society, (pp. 675U680). Hillsdale, NJ: Lawrence Erlbaum Associates. Louwerse, M. M. (2008). Embodied relations are encoded in language.Psychonomic Bulletin & Review, 15 (4), 838–844. Louwerse, M. M. (2011). Symbol interdependency in symbolic and embodied cognition. Topics in Cognitive Science, 3, 273–302. Lund, K., & Burgess, C. (1996). Producing high-dimensional semantic spaces from lexical co-occurrence. Behavioral Research Methods, Instrumentation, and Computers, 28, 203– 208. Mandler, J. M., Bauer, P. J. & McDonough, L. (1991) Separating the sheep from the goats: Differentiating global categories. Cognitive Psychology, 23, 263–298. McClelland, J. L. & Thompson, R. M. (2007). Using domaingeneral principles to explain children’s causal reasoning abilities. Developmental Science, 10(3), 333–356.
McClelland, J. L., St. John, M., & Taraban, R. (1989). Sentence comprehension: A parallel distributed processing approach. Language and Cognitive Processes, 4, 287–335. McDonald, S. & Lowe, W. (1998). Modelling functional priming and the associative boost. Proceedings of the 20th Annual Conference of the Cognitive Science Society (pp. 667– 680). McKoon, G., & Ratcliff, R. (1992). Spreading activation versus compound cue accounts of priming: Mediated priming revisited. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 1155–1172. McLeod, P., Shallice, T., & Plaut, D. C. (2000). Visual and semantic influences in word recognition: Converging evidence from acquired dyslexic patients, normal subjects, and a computational model. Cognition, 74, 91–114. McRae, K., & Cree, G. (2002). Factors underlying categoryspecific semantic deficits. In E. M. E. Forde, & G. Humphreys (Eds.), Category-specificity in mind and brain. East Sussex, England: Psychology Press. McRae, K., Cree, G., Seidenberg, M. S., & McNorgan, C. (2005). Semantic feature production norms for a large set of living and nonliving things. Behavior Research Methods, 37, 547–559. McRae, K., de Sa, V., & Seidenberg, M. S. (1997). On the nature and scope of featural representations of word meaning. Journal of Experimental Psychology: General, 126, 99–130 McRae, K., & Jones, M. N. (2014). Semantic memory. In D. Reisberg (Ed.) The Oxford Handbook of Cognitive Psychology. Mitchell, J., & Lapata, M. (2010). Composition in distributional models of semantics. Cognitive science, 34(8), 1388–1429. Mitchell, T. M., Shinkanerva, S. V., Carlson, A., Chang, K., Malave, V. L., Mason, R. A., & Just, M. A. (2008). Predicting human brain activity associated with the meanings of nouns. Science, 320, 1191–1195. Murdock, B. B. (1982). A theory for the storage and retrieval of item and associative information. Psychological Review, 89, 609–626. Nestor, P. G., Akdag, S. J., O’Donnell, B. F., Niznikiewicz, M., Law, S., Shenton, M. E., & McCarley, R. W. (1998). Word recall in schizophrenia: a connectionist model. American Journal of Psychiatry, 155(12), 1685–1690. Nosofsky, R. M. (1986). Attention, similarity, and the identification–categorization relationship. Journal of experimental psychology: General, 115(1), 39. O’Reilly, R. C., Munakata, Y., Frank, M. J., Hazy, T. E., and Contributors (2012). Computational Cognitive Neuroscience. Wiki Book, 1st Edition. URL: http://ccnbook.colorado.edu. Osgood, C. E. (1952). The nature and measurement of meaning. Psychological Review, 49, 197–237. Osgood, C. E. (1971). Explorations in semantic space: A personal diary. Journal of Social Issues, 27, 5–62. Perfetti, C. (1998). The limits of co-occurrence: Tools and theories in language research. Discourse Processes, 25, 363– 377. Plaut, D. C. (1999). A connectionist approach to word reading and acquired dyslexia: Extension to sequential processing. Cognitive Science, 23, 543–568.
Plaut, D. C. (2002). Graded modality-specific specialisation in semantics: A computational account of optic aphasia. Cognitive Neuropsychology, 19 (7), 603–639. Plaut, D. C., and Booth, J. R. (2000). Individual and developmental differences in semantic priming: Empirical and computational support for a single-mechanism account of lexical processing. Psychological Review, 107, 786–823. Polyn, S. M., & Kahana, M. J. (2008). Memory search and the neural representation of context. Trends in cognitive sciences, 12(1), 24-30. Recchia, G. L., & Jones, M. N. (2009). More data trumps smarter algorithms: Training computational models of semantics on very large corpora. Behavior Research Methods, 41, 657–663. Recchia, G., Jones, M., Sahlgren, M., & Kanerva, P. (2010). Encoding sequential information in vector space models of semantics: Comparing holographic reduced representation and random permutation. Regier, T. (2005). The emergence of words: Attentional learning of form and meaning. Cognitive Science, 29, 819–865. Riordan, B., & Jones, M. N. (2011). Redundancy in linguistic and perceptual experience: Comparing distributional and feature-based models of semantic representation. Topics in Cognitive Science, 3, 1–43. Rips, L. J., Shoben, E. J., & Smith, E. E. (1973). Semantic distance and the verification of semantic relations. Journal of Verbal Learning & Verbal Behavior, 12, 1–20. Rogers, T. T., & McClelland, J. L. (2006). Semantic Cognition. Cambridge, MA: MIT Press. Rogers, T. T., Lambon Ralph, M. A., Garrard, P., Bozeat, S., McClelland, J. L., Hodges, J. R., & Patterson, K. (2004). The structure and deterioration of semantic memory: A neuropsychological and computational investigation. Psychological Review, 111, 205–235. Rohde, D. L. T., and Plaut, D. C. (1999). Language acquisition in the absence of explicit negative evidence: How important is starting small? Cognition, 72, 67–109. Rhode, D. L., Gonnerman, L. M., & Plaut, D. C. (2004). An Introduction to COALS. Retrieved February, 4, 2008. Rosenblatt, F. (1959). Two theorems of statistical separability in the perceptron. In M. Selfridge (Ed.), Mechanisation of thought processes: Proceedings of a symposium held at the National Physical Laboratory . London: HM Stationery Office Rumelhart, D. E., Hinton, G. E. & Williams, R. J. (1986) Learning representations by back-propagating errors. Nature, 323, 533–536. Rumelhart, D. E., McClelland, J. L., & the PDP research group. (1986). Parallel distributed processing: Explorations in the microstructure of cognition. Volume I. Cambridge, MA: MIT Press. Rumelhart, D. E. & Todd, P. M. (1993) Learning and connectionist representations. In D. E. Meyer & S. Kornblum’s (Eds.) Attention and performance XIV: Synergies in experimental psychology, artificial intelligence, and cognitive neuroscience (pp. 3–30). Cambridge, MA: MIT Press. Sahlgren, M., Holst, A., & Kanerva, P. (2008). Permutations as a means to encode order in word space. Proceedings
models of semantic memory
253
of the 30th Conference of the Cognitive Science Society, pp. 1300–1305. Salton, G., & McGill, M. (1983). Introduction to modern information retrieval. New Yark, NY: McGraw-Hill. Shaoul, C., & Westbury, C. (2006). Word frequency effects in highdimensional co-occurrence models: A new approach. Behavior Research Methods, 38, 190–195. Smith, E. E., Shoben, E. J., & Rips, L. J. (1974). Structure and process in semantic memory: A featural model for semantic decisions. Psychological Review, 81, 214–241. Smith, L., & Yu, C. (2008). Infants rapidly learn wordreferent mappings via cross-situational statistics. Cognition, 106, 1558–1568. Steyvers, M. (2009). Combining feature norms and text data with topic models. Acta Psychologica, 133, 234–243. Steyvers, M., & Griffiths, T. (2007). Probabilistic topic models. In T. Landauer, D. McNamara, S. Dennis, & W. Kintsch (Eds.) Handbook of latent semantic analysis. Mahwah, NJ: Erlbaum. Stone, B., Dennis, S. & Kwantes, P. J. (2011). Comparing methods for single paragraph similarity analysis. Topics in Cognitive Science. 3(1). 92–122. Tabor, W. & Tanenhaus, M. K. (1999). Dynamical theories of sentence processing. Cognitive Science, 23, 491–515. Taraban, R. & McClelland, J. L. (1988). Constituent attachment and thematic role assignment in sentence processing: Influences of content-based expectations. Journal of Memory & Language, 27, 597–632.
254
higher level cognition
Tenenbaum, J. B., Kemp, C., Griffiths, T. L., & Goodman, N. D. (2011). How to grow a mind: Statistics, structure, and abstraction. science, 331(6022), 1279– 1285. Thomas, M. S. C., & Karmiloff-Smith, A. (2003). Modeling language acquisition in atypical phenotypes. Psychological Review, 110(4), 647–682. Turney, P. D. & Pantel, P. (2010). From frequency to maning: Vector space models of semantics. Journal of Artificial Inteligence Research, 37, 141–188. Tulving, E. (1972). Episodic and semantic memory 1. Organization of Memory. London: Academic, 381, e402. Tversky, A. & Gati, I. (1982). Studies of similarity. In E. Rosch & B. Lloyd Eds.), On the nature and principle of formation of categories (pp. 79–98). Hillsdaley, NJ: Erlbaum. Tyler, L. K., Durrant-Peatfield, M. R., Levy, J. P., Voice, J. K., & Moss, H. E. (1996). Distinctiveness and correlations in the structure of categories: Behavioral data and a connectionist model. Brain and Language, 55, 89–91. Vigliocco, G., Vinson, D. P., Lewis, W., & Garrett, M. (2004). Representing the meanings of object and action words: The featural and unitary semantic space hypothesis. Cognitive Psychology, 48(4), 422–488. Widrow, B., & Hoff, M. E., Jr. (1960). Adaptive switching circuits. In 1960 IRE WESCON Convention Record, Part 4 (pp. 96–104). New York, NY: IRE. Wittgenstein, Ludwig (1953). Philosophical Investigations. Blackwell.
CHAPTER
12
Shape Perception
Tadamasa Sawada, Yunfeng Li, and Zygmunt Pizlo
Abstract This chapter provides a review of topics and concepts that are necessary to study and understand 3D shape perception. This includes group theory and their invariants; model-based invariants; Euclidean, affine, and projective geometry; symmetry; inverse problems; simplicity principle; Fechnerian psychophysics; regularization theory; Bayesian inference; shape constancy and shape veridicality; shape recovery; perspective and orthographic projections; camera models; as well as definitions of shape. All concepts are defined and illustrated, and the reader is provided with references providing mathematical and computational details. Material presented here will be a good starting point for students and researchers who plan to study shape, as well as for those who simply want to get prepared for reading the contemporary literature on the subject. Key Words: 3-dimensional shape perception, shape constancy, perspective projection, simplicity constraints, inverse problems, symmetry, shape veridicality, regularization methods, Bayesian Model, definition of shape
Introduction: Uniqueness of Shape Shape is unique among other perceptual features such as color, position, orientation, size, and speed because shape is characterized by high degree of (a) complexity and (b) spatial regularity. These two characteristics, complexity and regularity, allow the visual system to apply very sophisticated and very effective computations resulting in veridical perception. By “veridical” we mean that we see shapes the way they are “out there.” Although all other perceptual features are represented in a space of a few dimensions: 3 dimensions for color, 3 dimensions for position, 3 dimensions for orientation, 1 dimension, for size and 1 dimension, for speed, shape requires dozens if not hundreds of dimensions.Consider first a 2dimensional (2D) polygonal shape. The position of each vertex of a polygon is represented by two coordinates. Hence, a polygon can be represented as a point in a 2N dimensional space and its
shape in a (2N – 4) dimensional space, where N is the number of vertices of the polygon (here, by shape we mean a figure whose 2D position, orientation, and size are irrelevant). Clearly, to represent an arbitrary 2D polygon, one will need a high-dimensional space. To represent an arbitrary smooth curve, an infinitely dimensional space is needed. Obviously, the dimensionality will also be high with 3-dimensional (3D) shapes. Complexity of shape is essential in recognizing objects from their retinal or camera images. We will never confuse a horse with a bookshelf when presented with a single 2D image of one of these two objects, regardless from which viewing direction the image is taken. A horse and a bookshelf will never produce identical retinal images because of the complexity of their shapes. Confusion, on the other hand is very likely with very simple figures such as ellipses or triangles. The second unique aspect of 3D shapes is that they are characterized by spatial regularities.
255
Natural objects are not a random scatter of points. Objects are volumetric, whose surfaces are piecewise smooth. Even more importantly, natural objects are characterized by symmetries. Plants are symmetrical because of the way they grow. Animals are symmetrical because of the way they move. Man-made objects are symmetrical because of the functions they serve. Until very recently, veridical perception of 3D shapes was considered an unsolved problem. In fact, many, if not most, students of vision have assumed that veridical perception of shape never actually happens. The difficulty of the problem is related to the fact that the depth dimension is lost in the projection from a 3D space to the 2D retinal or camera image. It follows that the task of recovering a 3D shape from its 2D retinal image is underconstrained: a 2D image allows for infinitely many 3D interpretations. Yet, we always perceive 3D natural shapes veridically and our percept does not depend on the direction from which we view the object. When the viewing direction changes, the shape of the 2D retinal image changes, as well. The fact that our percept of the 3D shape does not change is called shape constancy. In this chapter we will describe the contemporary approach to study 3D shape perception. We will review studies of shape constancy (Review of Psychophysical Results on Shape Constancy) and will explain conventional (Fechnerian Paradigm Is Inadequate: Shape Perception Should Be Viewed as an Inverse Problem) and new (Solving Inverse Problems: The Role of Constraints in Regulation and Bayesian Methods) paradigms of shape perception, geometry of projections from 3D to 2D space (Geometry of the Eye and of the Camera) and invariants of these projections (Invariants of Perspective and Orthographic Projection). Then, we will show how shape perception can be studied using the new approach (Symmetry of a 3D Shape and its 2-dimensional Orthographic and Perspective Projections, Any 2D Image Is Consistent with a 3D Symmetrical Interpretation, and Models of Recovering a 3-dimensional MirrorSymmetrical Shape from One or Two 2D Images). Finally, we present our new definition of shape (Conclusion: a new definition of shape).
Review of Psychophysical Results on Shape Constancy Shape constancy, which is illustrated in Figure 12.1, is of fundamental importance in our everyday life. As an example, imagine going to a car dealer after checking photographs of cars that are available in your favorite magazine. You 256
higher level cognition
can easily recognize the cars when you see them at the dealership after you had seen them in the photos. This phenomenological observation has been confirmed in well-controlled psychophysical experiments. In a typical shape constancy experiment, the subject is shown a single object twice or two different objects sequentially. The subject is asked to judge whether the shape was the same in the two presentations. Note that when the same shape is shown twice, the 3D orientation of the shape in the second presentation must be different from that in the first presentation so that the 2D images are different (see Figure 12.1). This is accomplished by rotating the object in depth. A rotation around the line of sight does not change the 2D retinal shape.It only affects the 2D orientation of the retinal shape. As a result, such a rotation does not allow testing shape constancy. Some parts of the object that are visible in the first view become invisible in the second view, when the object is rotated in depth, and other parts of the object appear only in the second view. If the rotation in depth is small, the 2D images of the object are similar. If the rotation in depth is large, the 2D images will be very different. The first experiment reporting perfect constancy with 3D shapes was performed only 20 years ago (Biederman & Gerhardstein, 1993). Their 3D objects were composed of simple parts that were 3D mirror- or translationally-symmetrical (see section Symmetry of a 3D Shape and Its 2D Orthographic and Perspective Projections). Shape constancy was perfect even with very large rotations in depth, as long as the 3D parts were at least partially visible. Very reliable shape constancy was also demonstrated with 3D mirror-symmetrical objects that did not have distinctive parts (Stevenson, Li, Chan, & Pizlo, 2006; Li & Pizlo, 2011; Pizlo & Stevenson, 1999). Shape constancy fails, however, with irregular and unstructured 3D objects. The first systematic study of the role of spatial regularities in shape constancy was performed by Pizlo & Stevenson (1999). Their stimuli are shown in Figure 12.2. Some of these 3-dimensional stimuli were mirror-symmetrical, some of them were volumetric, and some of them had piecewise planar contours. Shape constancy was reliable with 3D mirror-symmetrical polyhedra whose faces were planar (Figure 12.2a) and it completely failed with 3D asymmetrical polyhedra whose faces were nonplanar (Figure 12.2d), as well as with a 3D irregular polygonal lines (Figure 12.2e). For other
(a) (b) (c)
(e)
(d)
Fig. 12.1 Five images of a 3D shape. If you see the same 3D shape, you achieved shape constancy.
stimuli, the degree of shape constancy was intermediate. Interestingly, monocular and binocular performances were highly correlated, suggesting a common underlying mechanism. In particular, binocular shape constancy was not reliable, unless monocular performance was reliable, as well. These results indicate that 3D mirror-symmetry, volume, and planarity of faces play a critical role in shape perception. Note that all these three characteristics refer to abstract aspects of objects, rather than to particular exemplars of objects. Also, these characteristics can be found in natural 3D objects, but not in the 2D retinal images. It follows that symmetry, volume, and planarity are used by the human visual system as constraints, not as visual cues. The distinction between visual cues and constraints is best expressed in the language of inverse problems theory.
(a)
(b)
(c)
(d)
(e)
(f )
Left
Right
Left
Fig. 12.2 Stereoscopic images of the six types of stimuli used in our prior studies on shape constancy. The left and the center images are used for uncrossed fusion and the center and the right image are used for crossed fusion. (a) Mirror-symmetrical polyhedron with planar faces. (b) Mirrorsymmetrical polyhedron with nonplanar faces. (c) Asymmetrical polyhedron with planar faces. (d) Asymmetrical polyhedron with nonplanar faces. (e) irregular polygonal 3D contour containing no volume. (f ) 16 points, the vertices of a 3D symmetrical polyhedron.
Fechnerian Paradigm is Inadequate: Shape Perception Should Be Viewed as an Inverse Problem According to the commonly accepted paradigm attributed to Gustav Theodor Fechner (1860/1966), the “percept” can be best understood as the result of a causal chain of events. In the case of 3dimensional shape perception, the chain begins with the shape of an object “out there” (called shape perception
257
a distal stimulus), followed by forming the 2dimensional retinal image (called a proximal stimulus), transduction, which changes the light energy into electrical energy in the nervous system, brain processes, and finally the percept of the distal stimulus. In this view, the percept is assumed to be a mental measurement of the physical stimulus. Although it is easy to treat brightness or loudness perception as a mental measurement of light or sound intensity, perception of a 3D shape produced by looking at a single 2D image can only be viewed as an inference based on incomplete data, not as a measurement. The visual system must go beyond the information given in the 2D retinal image. How this can be done is described in a theory of inverse problems (Pizlo, 2001; Poggio, Torre, & Kach, 1985; Tikhonov & Arsenin, 1977). Consider an orthographic projection from a 3dimensional space to a 2-dimensional image. This type of projection will be explained in detail in the section Geometry of the Eye and of the Camera. For now, the illustration in Figure 12.3 as well as the Box 1 should be sufficient. Let a set of 3D points represent a 3D object. The coordinates of these 3D points form a matrix P 3xN . The 2D orthographic image of these points forms a matrix p2xN . The projection from P 3xN to p2xN is called a forward problem. It is accomplished by dropping the zcoordinates of the points. In matrix form, this is expressed as follows: 1 0 0 P3×N , (1) p2×N = 0 1 0 The binary matrix A2x3 represents an orthographic projection. It is obvious that this forward problem always has a solution: given P 3xN , it is easy to compute a unique p2xN . Now, the 3D percept produced by a 2D image is equivalent to solving an inverse problem. The name is related to the fact that if A2x3 were a square nonsingular matrix, its inverse A2x3 −1 would have existed, and P 3xN could have been computed from p2xN simply by premultiplying both sides of Eq. (1) by A2x3 −1 . This cannot be done in our case because A2x3 is not a square matrix. This inverse problem has infinitely many solutions represented by assigning arbitrary z-coordinates to each of the N retinal points. This fact had already been known to the Bishop Berkeley (1709/1910), but it did not receive attention it deserves until the second half of the 20th century. This inverse problem is called ill-posed. A problem is well-posed when the solution exists, it is unique, and it depends continuously on the data. When 258
higher level cognition
one or more of these criteria are not met, the problem is ill-posed. In 3D shape perception, it is the nonuniqueness of the solution that is the reason for ill-posedness. The only way to transform an illposed problem to a well-posed problem is to impose constraints on the family of 3D interpretations. In plain English, the visual system must know something about the 3D shapes “out there” and combine this knowledge with the 2D retinal image in producing a unique and veridical 3D percept. For example, given a 2D image of a 3D rectangular box, a shoebox for example, the 3D shape of the shoebox can be recovered veridically by maximizing the 3D symmetries of the 3D interpretation. It might be a little surprising, but it is true nonetheless, that applying 3D symmetries to the 2D image of a 3D opaque shoebox allows recovering the entire box, its invisible back parts, as well its visible front parts. The next section will describe the method of solving ill-posed inverse problems formally.
Solving Inverse Problems: The Role of Constraints in Regularization and Bayesian Methods Although the fact that the percept is a result of applying simplicity principle or constraint to the retinal image(s) has been known for at least 100 years (Köhler, 1920/1938; Mach, 1906/1939; Werheimer, 1923/1958), a formal theory of how an inverse problem can be solved did not appear in vision literature until 30 years ago (Poggio, Torre & Koch, 1985). According to this new paradigm, visual perception is an inference about (recovery of ) the 3D environment “out there” based on incomplete data provided by a 2D retinal image. Consider forming a 2D retinal image I 2D by a 3D object η3D : I2D = Fprojection (η3D )
(2)
where Fprojection is a function representing the projection from η3D to I 2D . Eq. (2) is a more general case of Eq. (1). Specifically, projection F could be either a perspective or orthographic mapping (see section Geometry of the Eye and of the Camera and Box 1). Note that the retinal image I 2D always has some noise due to uncertainty in identifying points and extracting contours. Hence, Eq. (2) can be modified as follows: I2D = Fprojection (η3D ) + ε2D ,
(3)
where ε 2D is the noise inherent in the retinal image I 2D or in the higher visual representations in the brain hierarchy. The process of recovering the 3D
(b)
4 2
(a)
8
6 6
2
3 5 1
1
7
8 5
4
ΠI
7 3
Fig. 12.3 (a) A 2D orthographic image of a 3D cube. This image consists of three sets of four parallel line segments. (b) Geometry of an orthographic projection of the cube to an image plane I . Here, the image plane is represented by a line. The 3D vertices of the cube are projected to I along projection lines (dotted lines). The projection lines are parallel to one another and orthogonal to I .
object η3D from the 2D retinal image I 2D can be written as follows: −1 (I2D ). η3D = Fprojection
(4)
Recall that there are infinitely many 3D interpretations for any given I 2D . It is impossible to determine Fprojection −1 uniquely unless some additional information is available. Hence, recovering the 3D object from the 2D retinal image is an ill-posed problem. Before an ill-posed inverse problem is solved, it must be transformed into a well-posed problem by imposing a priori constraints on the family of possible interpretations. For the percept to be veridical, the constraints must reflect regularities present in the natural world. What is the nature of these regularities? Symmetries are arguably the most important regularities present in our natural world. As already pointed out in the beginning of this chapter, all natural objects are symmetrical. The main idea behind a regularization method of solving inverse problems is fairly simple. We look for a 3D shape η 3D that satisfies two requirements. First, the 3D shape η 3D has to be geometrically consistent with the retinal image I 2D . In other words, the 2D image I 2D can be produced by η 3D . Second, the 3D shape η 3D has to satisfy the a priori constraints. Typically, there is no 3D shape that exactly satisfies these two requirements. This may happen because of the noise ε 2D in the retinal image (e.g., a 2D digital image of a perfect sphere is never a perfect ellipse) or because the original 3D shape η3D does not exactly satisfy the constraints (e.g., no human face is perfectly mirror-symmetrical). A good (perhaps the best) way to resolve this compromise is to use a cost function consisting of two terms representing these two requirements
(Tikhonov & Arsenin, 1977): '2 ' , I2D ) = 'Fprojection (η3D ) − I2D ' Etotal (η3D ' ' '2 +λ 'econstraint (η3D ) , (5) where λ is a regularization parameter. The first term in the cost function E evaluates how consistent the projected image of η 3D is with the retinal image I 2D . The second term evaluates how well η 3D satisfies the a priori constraints. If the retinal image (visual data) is reliable, λ should be small. Otherwise λ should be large. Recovering a 3D shape that best satisfies both these requirements is equivalent to finding the global minimum of Etotal (η 3D , I 2D ) in the space of all 3D shapes. Sometimes there are two (or more) local minima of the cost function. There is a stochastic version of the regularization theory involving Bayes’s rule (Bouman & Sauer, 1993; Kersten, 1999; Knill & Richards, 1996; Pizlo, 2001; Poggio et al., 1985). Bayes’s rule allows the computation of the posterior probability p(η 3D |I 2D ) as follows: | I2D ) = p(η3D
)p(η ) p(I2D |η3D 3D . p(I2D )
(6)
where p(η 3D ) is the prior probability distribution for the 3D shape η 3D and it represents a priori knowledge about shapes in the environment. p(I 2D |η 3D ) is the likelihood function for η 3D . This function is a probability of producing the retinal image I 2D by a 3D shape η 3D . Finally, p(I 2D ) is the probability of obtaining I 2D . The inverse problem of determining the 3D shape based on the information provided by the 2D retinal image can be solved by finding η 3D that maximizes the posterior probability, p(η 3D |I 2D ). Such an shape perception
259
η 3D is called a maximum a posteriori (MAP) estimator. Note that p(I 2D ) is a constant for the given image I 2D in Eq. (6), and so it can be ignored in solving the inverse problem. Also note that, typically, priors are included in both the prior probability distribution and in the likelihood function. If a prior is expected to be satisfied exactly, it is put in the likelihood function and called an assumption, or an implicit constraint. Otherwise, the prior is called an explicit constraint. The relation between Eq. (5) and (6) can be seen by taking the logarithm of both sides of Eq. (6) (ignoring the term p(I 2D )):
Box 1 Projections (from a 3D Space to a 2D Image): Orthographic: The projecting lines are orthogonal to the image plane (http://shapebook. psych.purdue.edu/3.1/). x=X y=Y Perspective: The projecting lines intersect at the center of perspective projection (http://shapebook.psych.purdue.edu/3.2/). x = f (X /Z ) y = f (Y /Z )
− log p(η3D | I2D ) = − log p(I2D |η3D ) − log p(η3D ).
(7) Note that Eq. (7) is analogous to Eq. (5). The logarithm of the likelihood function p(I 2D |η 3D ) corresponds to the first term of Eq. (5), which evaluates how consistent η 3D is with I 2D . The logarithm of the prior probability distribution p(η 3D ) corresponds to the second term of Eq. (5), which evaluates how well η 3D satisfies the constraints. The regularization parameter λ in Eq. (5) is implicitly represented in Eq. (7) by the ratio of the variances of p(I 2D |η 3D ) and p(η 3D ). In fact, under some assumptions about the probability distributions, maximizing the posterior probability p(η 3D |I 2D ) is equivalent to minimizing the cost function Etotal (η 3D , I 2D ) (Pizlo, 2001). Before we show in the section Models of Recovering a 3-Dimensional Shape from One or Two 2-Dimensions Images how the cost function (Eq. 5) is used to recover 3D shapes, we will describe the geometry of the eyeball and the rules of geometrical optics representing the forward problem. This will be followed by description of invariants of perspective and orthographic projection. These invariants are essential in determining if and how the a priori constraints can be applied. The section Symmetry of a 3D Shape and Its 2D Orthographic and Perspective Projections will describe 3D symmetries of shapes, as well as three other a priori shape constraints.
Geometry of the Eye and of the Camera Geometry of a human eye is complex because the eye is anatomically composed of several optical parts and some of them are even flexible. Hence, it is quite common to use simplified models. Two different models of the human eye are particularly common: a pinhole camera with (a) a spherical and with (b) a planar retina. These two models are 260
higher level cognition
essentially equivalent because the 2D coordinates of the position on the spherical and the planar retina can be uniquely transformed to one another (Figure 12.4a). In most geometrical analyses, the planar retina leads to simpler equations. So, we will use the pinhole camera model with a planar retina in this chapter. The geometry of the pinhole camera is characterized by its focal length and the principal point. The focal length is a distance between the center of perspective projection (the pinhole) and the image plane (Figure 12.4). The principal point is the center of the image plane. In the human eye, the principal point is the center of the fovea. The line connecting the principal point and the center of projection is normal to the image plane, and this line is called the principal axis. According to the rules of geometrical optics, the center of projection (the pinhole) is always on the line connecting a point in a 3D scene and its 2D image. This is called a projection line. Let [0, 0, 0]t be the center of perspective projection and Z = f be the image plane where |f | is the focal length of the camera. The x- and y-axes of the 2D Cartesian coordinate system on the image plane are parallel to the X and Y -axes of the 3D Cartesian coordinate system. The origin of the 2D coordinate system is set at [0, 0, f ]t . The principal point has 2D coordinates [0, 0]t . Then, a point V 3D = [X 3D , Y 3D , Z 3D ]t in a 3D scene projects to a point v2D = [x 2D , y2D ]t on the image plane according to the following matrix equation: !
t fY3D t 3D . (8) x2D y2D = fX Z Z 3D
3D
Equation (8) represents the rules of a perspective projection. According to the rules of optics, the 3D point should be on the opposite side of the center
V3D
(a)
V3D
(b) Principal point
C
v2D
Principal axis
vretina
Principal point
|f| |f|
v2D
Principal axis
ΠI
C
ΠI
Fig. 12.4 Geometry of a perspective projection. A point V3D in a 3D scene projects to v2D on an image plane I through a center of projection C. (a) Models with a spherical and planar “retina,” which are behind the center of projection C. Note that the correspondence between vretina on the spherical image and v2D on the planar image is unique for a given center of projection C. This means that these two models of the eye are essentially equivalent. (b) The image plane I is placed in front of C. This setting is often used in computational models.
of projection compared to the image plane (Figure 12.4a): (9) fZ3D < 0. The signs of x 2D and y2D are always opposite to those of X 3D and Y 3D respectively. This reflects the fact that the 2D retinal image of the 3D scene is always upside down (more precisely, 180◦ rotated) in the human eye or in a camera. When fZ 3D > 0, the 3D point is behind the eye and becomes invisible. It is quite common to “put” the image plane in front of the center of projection, in computational models of a camera. This simplifies the computations in the sense that the computed image is right sight up (see Figure 12.4b). We will follow this convention here. Perspective projection is a nonlinear transformation, which makes it difficult to use. In particular, matrix algebra cannot be directly applied to a perspective projection. Furthermore, even if we restrict a perspective projection to a one-to-one mapping by considering the case of an image of a planar figure slanted relative to the eye, perspective projection is not a group, and so it does not have invariants (Pizlo & Rosenfeld, 1992). As a reminder, a set of transformations is a group if four axioms are satisfied (see Glossary entry Group of Transformations): (a) an inverse transformation is in the set, (b) identity transformation is in the set, (c) a composition of two transformations from the set belongs to the set, and (d) transformations are associative. A perspective projection between two planes is not a group because it violates the closure axiom (c). The smallest group of transformations
containing a perspective projection is a projective transformation (Coxeter, 1987; Mundy & Zisserman, 1992). It turns out that a projective transformation corresponds to an uncalibrated camera, that is a camera whose focal length and principal point are unknown (they are free parameters). It is important to emphasize that the projective group applies only to the case of camera images of planar figures because this case is a one-to-one mapping. It is customary to use the terminology of projective geometry in the case of a 3D to 2D mapping, as well, even though this mapping is not one-to-one. The 3D to 2D mapping does not have a unique inverse and so it is not a group. It follows that this mapping is not a projective transformation. Before we describe the relation between the eye and a projective transformation in some more detail, we will introduce an orthographic projection that is an important and useful approximation to a perspective projection. Using an orthographic approximation to a perspective projection is a way to circumvent the problems of nonlinearity and of nonexistence of invariants. Practically, an orthographic projection is a good approximation to a perspective projection when the range in depth of an object is less than 10% of the viewing distance. Formally, an orthographic projection can be introduced as follow. Consider moving the center of perspective projection to infinity along the Z -axis, relative to the image plane and to the 3D object. This is equivalent to translating both the image plane and the 3D object together to infinity in the opposite direction. As a result, the projecting lines become parallel to the principal shape perception
261
axis (the Z -axis) and perpendicular to the image plane (Figure 12.3). In such a case, Eq. 8 takes the following form:
t (f + Ztranslation )X3D x2D y2D = lim |Ztranslation |→∞ (Z3D + Zt ranslation ) t (f + Ztranslation )Y3D = [X3D Y3D ]t , (10) (Z3D + Ztranslation ) where Z translation is the translation along the Z axis. Note that Eq. (10) is equivalent to Eq. (1). The relation between a perspective and an orthographic projection is illustrated in our demo: http://shapebook.psych.purdue.edu/3.2/ (from Pizlo, Li, Sawada, & Steinman, 2014). You see a 3D object (a chair) inside a rectangular box defining the x, y, z directions. The center of perspective projection is at F whose position can be changed using a slider in the bottom. Use your mouse to rotate the “entire scene” relative to you, or to rotate the “inner object” relative to the image plane. Moving F toward infinity makes the perspective projection closer and closer to an orthographic projection. The orthographic projection is also illustrated in another demo: http://shapebook.psych.purdue.edu/3.1/. Next, consider a general case of the viewing geometry where the coordinate system of the 3D scene and the coordinate system of the image plane are not necessarily aligned. We assume a perspective projection, which means that we will elaborate Eq. (8). This equation will take a more complex form because both the camera parameters (called intrinsic parameters) and the information about the orientation of the 3D object relative to the camera (extrinsic parameters) must be used on the right-hand side. This is done by putting all these parameters in a “camera matrix” P perspective , which is a 3 × 4 matrix. The projection from a 3D point V = [X3D , Y3D , Z3D ]t to its 2D perspective projection v = [x2D , y2D ]t can be written as follows: v∗ = Pperspective V ∗
(11)
where V ∗ = [X∗3 D ,Y∗3 D , Z∗3 D ,W∗3 D ]t = W∗3 D [X∗3 D , Y∗3 D , Z∗3 D , 1]t and v∗ = [x∗2 D , y∗2 D , w∗2 D ]t = w∗2 D [x∗2 D , y∗2 D , 1]t . The vectors V and v are called the homogeneous coordinates of V ∗ and v∗ . The homogeneous coordinates allow expressing a nonlinear perspective projection by using matrix notation. In homogenous coordinates, the last value of a vector can be an arbitrary non-zero number. If the last value is equal to zero, the vector represents a point at infinity. In equation (11), 262
higher level cognition
1/αx
1/αy
y
x
1 pixel
s/αxαy
Fig. 12.5 Geometry of a camera pixel.
W∗3 D of V ∗ can usually be set to 1. This way, 3D homogenous coordinates are trivially obtained from 3D Euclidean coordinates (and vice versa). Note that Eq. (11) is equivalent to Eq. (8) when: ⎡ ⎤ f 0 0 0 Pperspective = ⎣ 0 f 0 0 ⎦ (12) 0 0 1 0 In this chapter, some equations use both homogenous and Euclidean coordinates. To avoid confusion, symbols with asterisks represent the homogenous coordinates and those without asterisks represent the Euclidean coordinates. The “camera matrix” P perspective can be decomposed into the following two matrices: ⎡ ⎤ αx s u0 Kperspective = ⎣ 0 αy v0 ⎦ and 0 0 1
Q = R3x3 −R3x3 C3x1 (13) where P perspective = Kperspective Q. The matrix K perspective is called the “intrinsic matrix,” which defines the camera’s intrinsic properties, such as focal length. The position of the principal point is [μ0 , ν 0 ]T on the image plane. If the pixels in the camera are squares, α x = α y and s = 0. The focal length in our representation is equal to α x and to α y . If s = 0, the coordinate system is skewed (Figure 12.5) and s is equal to cosine of the angle between the x and y axes. Matrix Q is called the “extrinsic matrix,” and it represents the transformation between the world coordinate system and the camera coordinate system. This transformation consists of a 3D translation (−C3x1 ) followed by a 3D rotation (R3x3 ). Note that the vector C 3x1 represents the position of the center of projection expressed in the world coordinate system. The “camera matrix” for the case of an orthographic projection can be defined in a similar
manner: ∗
v = Porthographic V = Korthographic QV where:
Korthographic =
αx 0
s αy
0 0
∗
(14)
.
(15)
W∗3 D of V ∗ should be set to 1 and the homogeneous coordinate of v is ignored. The extrinsic matrix Q is the same as in Eq. (13). Estimation of Q and K perspective (or K perspective ) is called camera calibration and is typically performed based on multiple images of a reference scene, whose geometry is known (see Hartley & Zisserman, 2003, for more details of camera calibration). When a pair of identical cameras form a binocular (stereo) system, such a system can be calibrated in an analogous way.
Invariants of Perspective and Orthographic Projection Box 2 Transformation Groups (2D Examples) 1. Rigid motion: rotation and translation. x = xcosϕ−ysinϕ+dx y = xsinϕ+ycosϕ+dy Invariants: distances, angle, area, volume. 2. Similarity: rigid motion plus uniform size scaling. x = (xcosϕ−ysinϕ)s+dx y = (xsinϕ+ycosϕ)s+dy Invariants: angle, ratio of lengths of line segments, ratio of areas, ratio of volumes. 3. Affine: Similarity plus stretch along an arbitrary direction. x = ax+by+c y = dx+ey+f Invariants: ratio of length of parallel line segments, ratio of areas, ratio of volumes. 4. Projective: a continuous transformation that maps straight lines into straight lines and planes into planes. x =(ax+by+c)/(gx+hy+i) y =(dx+ey+f )/(gx+hy+i) Invariants: Cross-ratio of four collinear points, cross-ratio of areas and of volumes.
Projection from a 3D space to a 2D retinal image removes information about depth, but it
does not eliminate information about 3D shape. As pointed out in the first section, Introduction: Uniqueness of Shape, the more complex the 3D shape is, the more information about the 3D shape is preserved in the 2D image. Mathematically, this is represented by invariants of perspective projection. There are two main types of invariants: conventional (or group) invariants and model-based invariants (Mundy & Zisserman, 1992, 1993; Pizlo, 2008; Rothwell, 1995; Weiss, 1988, 1993). Conventional invariants are defined for groups of transformations. It follows that the transformation for which we seek invariants must be a one-toone mapping: a line to a line, a plane to a plane or a 3D space to a 3D space. So, the most interesting case representing vision does not belong here because 3D vision involves a mapping between a 3D space and a 2D plane. This mapping can only be handled by model-based invariants (see later). We first discuss invariants of an orthographic projection of a planar (2D) figure to a 2D image. This projection can be described by a group of 2D affine transformations, which is composed of a rotation, translation, size scaling and stretch. Using matrix notation, a 2D affine transformation can be written as follows: gA11 gA12 x gA13 x = + y y gA21 gA22 gA23 (16) In this transformation, parallel lines remain parallel, the ratio of lengths of two parallel line segments stays the same as does the ratio of areas of two regions. These three properties are invariants of a 2D affine transformation and of an orthographic projection of a planar figure. This is illustrated in Figures 12.6a and b where a regular checkerboard and its orthographic image are shown. The square regions of the checkerboard project to parallelograms. All these parallelograms are identical, which represents the operation of affine invariants. Next, consider a 2D perspective image of a planar object slanted relative to the camera. The focal length of the camera (or the eye) and the principal point are fixed and known. This is the case relevant to human vision. In such a case, a perspective projection does not form a group because a composition of two perspective projections is not a perspective projection. Instead, it is a projective transformation (Springer, 1964). It follows that a perspective projection does not have its own invariants. Specifically, properties that are shape perception
263
(a) (c) (b)
Fig. 12.6 (a) A square with a regular (checkerboard) pattern. (b) An orthographic projection of (a) with a slant of 60 degrees. The resulting pattern is not rectangular, but parallel lines remain parallel and all regions that are images of squares are parallelograms. (c) A perspective projection of (a). Parallel lines do not project to parallel lines and the regions that are images of the squares are not identical.
invariant under a 2D perspective projection are also invariant under a larger set of transformations that form a 2D projective group. The 2D projective transformation is represented by the following equation using homogeneous coordinates: ⎡ ⎤ ⎤ gP13 gP12 x gP22 ⎦ + ⎣ gP23 ⎦ . y gP32 gP33 (17) The Euclidean coordinates of the point [x , y ]t after the transformation are obtained as follows: [x , y ]t = [x ∗ /w ∗ , y ∗ /w ∗ ]t . Under the 2D projective transformation, the parallelism of lines is not invariant (Figure 12.6c). Neither is the ratio of areas. What is invariant under a 2D projective transformation is a ratio of ratios (cross ratio) of 4 areas and a ratio of ratios of lengths of line segments defined by 4 collinear points (called cross ratio of 4 collinear points). The fact that there are no invariants characterizing a perspective projection between two planes could be considered only a minor inconvenience because one could always use 2D projective invariants, which are preserved under 2D perspective projection, as well. Mathematically this is true, but perceptually it is not. Namely, the human visual system achieves shape constancy for perspective but not for projective images of planar figures (Pizlo, 1994; 2008). This empirical fact has played a critical role in the recent history of research on human vision because it attracted attention of the vision community to another type of invariants, called model-based invariants. Modelbased invariants are of fundamental importance in vision because these are the only invariants that can be applied to a 2D image of a 3D scene, the most relevant type of projection in vision. A 3D to 2D projection violates all group axioms and, as a result, it does not have conventional invariants ⎡
⎤ ⎡ x ∗ gP11 ⎣ y ∗ ⎦ = ⎣ gP21 w ∗ gP31
264
higher level cognition
(Burns, Weiss, & Riseman 1990). Specifically, there is nothing that can be computed from a set of N points in a 3D space that remains invariant in a 2D image (orthographic or perspective) of these points, except for the number of these points. Unlike a perspective projection between two planes, where one can use invariants of a larger, projective set, such invariants do not exist in a 3D to 2D projection. If one is willing to restrict the class of objects, model-based invariants can be derived for a perspective projection between planes, as well as for a perspective and orthographic projection from a 3D space to a 2D image. Consider a perspective image of a planar shape. When the shape and its perspective image are represented using a contour-based descriptor, called the psi function, certain ratios in this representation are approximately invariant (Pizlo, 1991; Pizlo & Rosenfeld, 1992). Most importantly, these ratios are not invariant under an arbitrary projective or orthographic image. Pizlo called these ratios “perspective invariants” and he showed that the human visual system uses them in achieving shape constancy (Pizlo, 1994). Next, consider a projection from a 3D space to a 2D image. Take a set of lines that are parallel to one another in a 3D scene and not necessarily coplanar. Their 2D perspective projections are lines that converge at a vanishing point in the image (see an example in Figure 12.7). Note that we can say that parallel lines “out there” converge at a point in infinity. This means that lines that intersect at a single point in the 3D space, will intersect at a single point in a 2D image, as well. So, the intersection point is an invariant of the projection. It is a model-based invariant because this invariant cannot be used with an arbitrary configuration of points and lines out there. This invariant can only be used when a set of lines intersect at a single
Fig. 12.7 Converging 2D lines that are 2D projections of parallel lines in a 3D scene.
point, in particular, when all lines are parallel in a 3D space. We are all familiar with images like that in Figure 12.7. Vanishing points have been used by painters since the Renaissance, or more precisely, since the formal introduction of the rules of perspective projection by Brunelleschi in 1413. Conventionally, perception textbooks refer to this invariant as a “perspective cue.” The label “cue” is less than ideal, but it is justified by the fact that lines that converge in the plane of the image are interpreted by human observers as parallel lines “out there.” It is easy to see that parallelism of lines is induced by a mirrorsymmetry in the 3D configuration. Specifically, in every 3D mirror-symmetrical object, pairs of corresponding (symmetrical) points form parallel line segments. It follows that the vanishing point in a 2D-perspective image is a model-based invariant of a 3D to 2D projection of a mirrorsymmetrical object out there. It is “model-based” because this invariant can only be used with a restricted set of objects that are mirror-symmetrical. It cannot be used with an arbitrary object. In shape perception of natural objects, the “symmetry restriction” does not really restrict its use because all objects out there are symmetrical. Animal bodies and many man-made objects are
mirror-symmetrical. Limbs of animal bodies and parts of man-made objects are translationally symmetrical. Flowers are rotationally symmetrical. Each type of symmetry has its own model-based invariant represented by eigenvectors characterizing the symmetry transformation (Li, Sawada, Shi, Steinman, & Pizlo, 2013). In this chapter we focus on 3D mirror-symmetrical shapes, arguably the most important type of symmetry. However, the same approach applies to translational and rotational symmetries as well. The next section will summarize what we know about mirror-symmetrical objects and their images.
Symmetry of a 3D Shape and Its 2D Orthographic and Perspective Projections Consider a 3D mirror-symmetrical object, whose symmetry plane s has the following equation: [X, Y, Z ]t · Ns + ds = 0 where ds is a constant scalar, Ns = [XNs , YNs , ZNs ]t is the normal to s and |Ns |= 1. A line segment connecting a pair of symmetrical points is bisected by s and is normal to s (Figure 12.8). This line segment is called a symmetry line segment. Then, any point Vi = [Xi , Yi , Zi ]t of the object corresponds to its mirror-symmetrical counterpart Vpair (i) = [Xpair (i) , Ypair (i) , Zpair (i) ]t that satisfies shape perception
265
(a) (c) (b)
Fig. 12.8 Three different orthographic views of a 3D mirror-symmetrical object. Pairs of corresponding points are connected by dotted line segments, which are projections of symmetry line segments. The symmetry line segments are bisected by the symmetry plane (grey plane) and are normal to the symmetry plane.
(a)
the following equation: Vpair(i) = Vi − 2 (Vi · Ns + ds ) Ns
(18)
The symmetry line segments are parallel to Ns and (Vi ·Ns + ds ) is equal to the distance betweenVi and s along Ns . If a 3D object is mirror-symmetrical and its 2D image is given, a “virtual image” of the same 3D object from a different viewpoint can be computed directly from the original (real) 2D image (Vetter & Poggio, 1994). The virtual image of the 3D mirror-symmetrical object is computed by reflecting the original 2D image with respect to an arbitrary line (Figure 12.9). It is important to point out that the virtual image is computed from the given real 2D image without knowing the 3D object. Note that, the original and virtual images can be used to recover the object using multiple-view geometry (see section Models of Recovering a 3D MirrorSymmetrical Shape Frome One or Two 2D Images for computational details). The symmetry line segments are parallel to one another in the 3D representation. It follows that the images of symmetry line segments are also parallel in an orthographic image and they converge at a vanishing point in a perspective image. Note that in order to verify whether the symmetry lines segments are parallel or intersect at a single point, one has to solve “symmetry correspondence problem,” which refers to establishing which pairs of points in the 2D image are images of pairs of symmetrical points in the 3D space. We already know that the visual 266
higher level cognition
(b)
Fig. 12.9 An image (a) of a 3D mirror-symmetrical object and its virtual image (b). The virtual image was produced by reflecting the original image with respect to a vertical line.
system solves this problem in the case of curves, but not in the case of unconnected points (Sawada & Pizlo, 2008; Pizlo, Sawada, Li, Kropatch, & Steinman, 2010; Wagemans, 1992, 1993). This fact is illustrated in Figure 12.10. The number of pairs of dots that must be tried in order to solve symmetry correspondence problem in a dotted stimulus grows very quickly with the number of dots. In contrast,
(a)
(b) (c)
(d)
Fig. 12.10 It is almost impossible to see any object when only dots representing vertices of a polygon in (b) and polyhedron in (d) are shown, as in (a) and (c).
the number of pairs of dots (feature points) that must be tried in a stimulus composed of one or more closed contours is proportional to n, where n is the number of feature points such as corners or points of maximal curvature. Once symmetry correspondence problem is solved, the 3D symmetrical shape can be recovered. However, before we describe how to recover 3D symmetrical shape, we will discuss limitations of the symmetry a priori constraint.
Any 2D Image Is Consistent with a 3D Symmetrical Interpretation. Look at the following demo: http://shapebook. psych.purdue.edu/4.1/ The curves do not look three-dimensional and they do not look mirror-symmetrical. This makes sense because they were drawn in a haphazard way on the 2D image. Interestingly, and completely surprisingly, there is a pair of 3D mirrorsymmetrical curves that can produce this 2D image. Use your mouse to rotate the 3D interpretation. It turns out that there is always a 3D symmetrical interpretation no matter how the 2D curves are drawn. This statement was formulated as a theorem and proven by Sawada, Li, & Pizlo, (2011). This means that symmetry constraint is not sufficient to recover a unique 3D symmetrical shape because every pair of curves on the retinal image and any correspondence may lead to a 3D symmetrical interpretation. Using the terminology of signal detection, symmetry constraint would lead to hits with probability 1, but also to false alarms with probability 1. This is obviously unacceptable. Fortunately, this is not what the human visual
system does. The fact that you do not see 3D symmetrical curves when you look at the original image in this demo illustrates that the visual system does not recover symmetry from every 2D retinal image. Two questions arise. First, how does the visual system do this? Second, how well does it do it? These two questions will be answered next. Now look at the next demo: http://shapebook. psych.purdue.edu/4.2/ Again, you see two unrelated 2D curves, but now you know that a 3D symmetrical interpretation of these 2D curves exist. You can see this interpretation by using your mouse to rotate the 3D curves. You see that the 3D symmetry was hidden in the depth direction. Indeed, so much information is lost in the projection from a 3D space to a 2D image that one can hide almost anything there. In this case, one can always hide 3D symmetry. The fact that 3D symmetry is hidden in the depth direction is related to the concept of a degenerate view. A degenerate view is a view in which two or more points or edges “out there” project to a single point or edge in the 2D image. In other words, a point or an edge can hide behind another point or edge. This fact has been known for a while, but now it receives a new meaning and new importance. A degenerate views have not received a lot of attention in the past because they practically never happen. By “never” it is meant “probability zero”. Indeed, the set of viewing directions from which two points in a 3D space project to the same point in the 2D retinal image is of measure zero, and thus, for a wide range of probability distribution of viewing directions, the probability shape perception
267
of a degenerate view is zero. So, degenerate views hide some information entirely, but because they almost never happen, the visual system, human or machine, need not worry about them. However, our case is different because we can always compute a 3D symmetrical interpretation that corresponds to a degenerate view when the actual 3D object is asymmetrical. In other words, when the 3D stimulus is asymmetrical, we can always produce an incorrect (spurious) 3D symmetrical interpretation and this interpretation will correspond to a degenerate view. This will happen not only when the 3D object is asymmetrical, but also when unrelated parts of the 3D scene are incorrectly interpreted as a single 3D object, or when the symmetry correspondence is not established correctly. This brings us to the next demo: http://shapebook. psych.purdue.edu/4.3/ The two examples show two different 3D symmetrical interpretations computed from the same 2D image. These two different interpretations were computed by assuming two different symmetry correspondences. In the example on the left, the pairs of corresponding points form horizontal lines, whereas in the example on the right, they form a -25 degree angle with the horizontal axis. It follows that we can compute infinitely many different 3D symmetrical interpretations just by choosing symmetry correspondence arbitrarily. However, you, as an observer, perceive only one 3D symmetrical interpretation, and what you see is what we show you on the left in the demo. The 3D interpretation that you see has two characteristics: (1) it does not correspond to a degenerate view, and (2) it is simple. By “simple” we mean, in this case, a pair of planar curves. A planar curve is almost always simpler than a 3D curve because when a curve is planar, the torsion of the curve is zero everywhere along this curve (Hilbert & Cohn-Vossen, 1952). Using the intuitions behind Attneave’s (1954) use of information theory to describe shapes, a 3D curve has more uncertainty than a planar curve and more uncertainty translates to more complex description. The next demo suggests that the concept of simplicity of the 3D interpretation may be more general than the concept of a degenerate view: http://shapebook.psych.purdue.edu/4.4/ When you use your mouse to rotate the 3D symmetrical interpretation, you realize that nothing was hidden in the original view. The original view was not degenerate, but you could not see the 3D symmetry that we produced from this 2D image. The only way to explain why the visual system does 268
higher level cognition
not produce a 3D symmetrical interpretation in this case is that such an interpretation is too complex. At this point, we cannot offer a general definition of simplicity, a definition that would explain all human 3D percepts, but we think that a planarity constraint is a good start. This is explained next. Consider a pair of planar mirror-symmetrical curves in a 3D space. It follows that these two curves are identical up to a 3D reflection. As a result, their 2D perspective images are related by a 2D projective transformation, as illustrated by the diagram shown in Figure 12.11. More specifically, they are related by a 5-parameter subset of a 2D projective transformation. If the image is produced by an orthographic, rather than a perspective projection, the curves in the image are related by a 4-parameter subset of affine transformation. We will explain this fact in some detail, next. First, consider an orthographic projection. The 3D mirror-symmetrical pair of the planar contours project to a pair of 2D contours that are related by a 2D affine transformation. This is obvious, considering the fact that the two planar curves “out there” are identical, up to a 3D rigid motion, which includes reflection. The lines connecting pairs of corresponding points between the 2D contours are parallel to one another because they are 2D projections of the symmetry line segments that are parallel in a 3D scene. This parallelism is preserved only by a subset of a 2D affine transformation that is composed of stretch, shear, and translation along a direction of the 2D parallel lines (Figure 12.12). This subset of 2D affine transformations can be written as follows: xpair(i) mo11 mo12 t t (θ ) R2D = R2D (θ ) ypair(i) 0 1 xi mo13 (19) · + yi 0 cos θ − sin θ R2D (θ ) = sin θ cos θ where vi = [xi , yi ]t and vpair (i) = [xpair (i) , ypair (i) ]t are a pair of corresponding points on the 2D curves, θ is an orientation of a line connecting vi and vpair (i) and R2D is a 2D rotation. The three components of the subset of a 2D affine transformation are represented by mo11 (stretch), mo12 (shear) and mo13 (translation) and the orientation of the transformation is θ . Note that θ represents the orientation of the 2D parallel lines which are the projections of the symmetry line segments. We showed that Eq. (19) is the necessary and sufficient
Planar Curve in 3D
3D reflection
Planar Curve in 3D
Perspective projection
Perspective projection
2D projectivity with 5df
2D image curve
2D image curve
Fig. 12.11 A pair of planar mirror-symmetrical curves in a 3D space project to two curves that are related by a subset of 2D projective transformation.
condition for a pair of 2D curves to have a 3D interpretation that is a 3D symmetrical pair of planar curves (Sawada, Li, & Pizlo, 2014). The fact that the two curves in the image are related by an affine transformation implies that one can use affine invariants such as ratio of lengths of parallel line segments and ratios of areas. Affine invariance provides a necessary condition for satisfying the relation [see Eq. (19)]. When a perspective projection is used, the resulting transformation on the 2D image is a 5parameter projectivity. Let the focal length and the principal point of the image be f and [0, 0]t , respectively. The lines connecting the corresponding pairs of points converge at a vanishing point vvp = [xvp , yvp ]t . The relation between the2D contours that are images of a pair of planar mirror-symmetrical curves in 3D is as follows (see Sawada, Li & Pizlo, 2014, for derivation): ⎡ ∗ ⎤ T T (θY )R3DZ (θZ ) ⎣ R3DY
⎛
mp11 =⎝ 0 0
mp12 1 0
xpair(i) ∗ ypair(i) ∗ wpair(i)
⎦
⎞ ⎡ ∗ ⎤ mp13 xi T (θ )R T (θ ) ⎣ y∗ ⎦ 0 ⎠ R3DY Y 3DZ Z i wi∗ 1 ⎞ (20) ⎛
cos θZ − sin θZ 0 cos θZ 0 ⎠, R3DZ (θZ ) = ⎝ sin θZ 0 0 1 ⎛ ⎞ cos θY 0 sin θY 0 1 0 ⎠, R3DY (θY ) ⎝ − sin θY 0 cos θY
where vi = [xi , yi ]t = [fxi ∗ /wi ∗ , fyi ∗ /wi ∗ ]t , vpair(i) = [fx pair(i) ∗ /wpair(i) ∗ , fypair(i) ∗ /wpair(i) ∗ ]t , cosθ Z = xvp /|vvp |, sin θ Z = yvp /|vvp |and θ Y = -atan(f /|vvp |). Again, one can use projective invariants as a necessary condition for two curves
to satisfy the relation (20). The 5th parameter in a perspective image represents |vvp |, which is the distance of the vanishing point vvp from the principal point of the image. There is a simple, but an interesting qualitative invariant implied by the relations [Eq. (19) and Eq. (20)]. When we consider a pair of curves on a 2D image, the curvatures must have the same (or opposite) sign for all pairs of corresponding points, if these 2D curves are images of 3D planar symmetrical curves (see Figure 12.2). This qualitative criterion provides only a necessary condition for the existence of a 3D interpretation consisting of a pair of planar curves. The sufficient condition is provided by Eqs. (19) and (20). The advantage of the qualitative criterion is that it can be used with images of approximately planar 3D symmetrical curves as well. Now that you know that symmetry constraint should not be applied to an arbitrary 2D image with an arbitrary correspondence of points, we will explain how this constraint actually works.
Models of Recovering a 3D Mirror-Symmetrical Shape From One or Two 2D Images Let’s begin with a demo illustrating the 3D shape recovery: http://shapebook.psych.purdue.edu/1.2/ jeep/ On the bottom left, you see a 3D synthetic model of a jeep. On top left you see a single 2D orthographic image of the jeep. The jeep is opaque, so the contours and surfaces on the back are not visible. On top right you see 2D contours extracted by hand. Only the front, visible contours have been marked. On the bottom right, you see the 3D contours recovered from the 2D image on top right. The recovery is perfect! Note that we recovered the back invisible contours as well as the front visible ones. This shows that one of the main claims of David Marr (1982) that we see only the front visible surfaces (2.5D sketch) is not necessarily true. Our model can see the entire 3D shape, and we are sure that you do, too. Go here to see a few more examples: http://shapebook.psych.purdue.edu/1.2/ Now, we will explain how our model recovers a 3D symmetrical shape from a single 2D image. The model uses four constraints: 3D mirror-symmetry, planarity of contours, maximum 3D compactness, and minimum surface area. We will focus on recovering a 3D mirror-symmetrical shape from its single 2D orthographic image. An orthographic image is an approximation to the true perspective shape perception
269
(a)
(b)
(c)
(d)
Fig. 12.12 Pairs of 2D contours whose relation can be represented by stretch (a, b), shear (c) and translation (d) along a horizontal direction. The stretch and shear transformations were applied in all four cases. The stretch factor is positive in (a) and negative in (b).
image, and it is also mathematically less informative than a perspective image. The orthographic image is computationally more interesting because most real images of real objects are so close to orthographic images that it is difficult to capitalize on the mathematical advantage of a perspective projection. Let Z = 0 be the plane of the image in the 3D scene, and let the x- and yaxes of the 3D Cartesian coordinate system be the 2D coordinate system on the image plane. We assume that symmetry correspondence has been solved (see Li, Sawada, Latecki, Steinman, & Pizlo, 2012, for an algorithm that can do this). Recall that symmetry line segments connecting mirror-symmetrical pairs of points of the 3D object are parallel to one another. The symmetry line segments project to line segments that are also parallel to one another in the 2D image. Let’s set the direction of the x-axis so that it is parallel to the projections of the symmetry line segments. Note that the symmetry line segments in the image may not be exactly parallel if the object is not exactly mirror-symmetrical, if the image has visual noise, or if the image was produced by a perspective projection. In such cases, the x-axis is set to a direction that is as parallel to the 2D line segments as possible in the least squares sense: min
yi − ypair(i)
2
(21)
i
where v i = [x i , y i ]t and v pair(i) = [x pair(i) , y’pair(i) ]t are projections of a mirror-symmetrical pair of points i and pair(i) of the 3D object. The y-coordinates of the mirror-symmetrical pairs of the points in the image are modified (corrected) so that the line segments connecting them become parallel 270
higher level cognition
to one another (Zabrodsky & Weinshall, 1997): xi xi xpair(i) , vpair(i) = = yi +ypair(i) vi = yi ypair(i) 2 xpair(i) = yi +ypair(i) , (22) 2
where vi = [xi , yi ]t and vpair (i) = [x pair(i) , ypair(i) ]t are the projections of the points i and pair(i) after the correction. Note that yi = y pair(i) . If the projections of the symmetry line segments are exactly parallel to one another in the image, vi = v i and vpair(i) = v’pair(i) . Now, the corrected orthographic image is consistent with an interpretation of a perfectly mirror-symmetrical 3D object. Let the 3D coordinates of the points i and pair(i) be Vi = [Xi , Yi , Zi ]t and V pair(i) = [X pair(i) , Y pair(i) , Z pair(i) ]t . Note that the X - and Y coordinates of Vi and V pair(i) are identical to those of vi and vpair(i) respectively under the orthographic projection. Then, the 2D symmetry line segments are perpendicular to the Y -axis because Yi = yi = ypair(i) = Y pair(i) . Note that the symmetry line segments of the 3D object are perpendicular to the symmetry plane s of the 3D object and are bisected by s . So, the tilt τ s of the symmetry plane s is zero and s is represented by the following equation: (23) Z = X tan σs + ds where σ s and ds are the slant and the depth of s . The symmetry line segments are normal to s . Hence, they are parallel to a vector Ns = [XNs , YNs , ZNs ]t = [1, 0, -cosσ s /sinσ s ]t that is normal to s . A 2D vector from vpair(i) to vi is li [1, 0]t where li = xi - x pair(i) and |li | is the length of the 2D vector. Then, a 3D vector from V pair(i) to Vi can be recovered as: (
t li Ns = li 1 0 −cos σs sin σs (24)
Let a midpoint of the 3D symmetry line segment between Vi and V pair(i) be Mi = [XMi , YMi , ZMi ]t . The midpoint Mi projects to a midpoint mi = [xmi , ymi ]t between vi and vpair(i) . The X - and Y -coordinates of Mi are also identical with those of mi under the orthographic projection. Hence, mi = [(xi + xpair(i) )/2, yi ]t and Mi = [(xi + xpair(i) )/2, yi , (Zi + Zpair(i) )/2]t . Note Mi is on s because the symmetry line segment is bisected by s (see Figure 12.8). Hence, the Z -coordinate of Mi can be recovered from mi as: ZMi = xmi
sin σs + ds cos σs
(25)
The 3D coordinates of the points i and pair(i) can be recovered from their midpoint Mi and the 3D vector from V pair(i) to Vi : Vi = Mi + li Ns /2 and V pair(i) = Mi − li Ns /2. Particularly, the Z coordinates of Vi and V pair(i) can be recovered as1 : Zi = ZMi + ZNs
li sin σs li cos σs + ds − = xmi 2 cos σs 2 sin σs
xpair(i) − xi cos 2σs + ds Zpair(i) sin 2σs li sin σs li cos σs = ZMi − ZNs = xmi + ds + 2 cos σs 2 sin σs xi − xpair(1) cos 2σs = +d sin 2σs =
(26) Note that σ s and ds are free parameters in Eq. (26). ds does not affect the recovered 3D object but only its position along the Z -axis. So, we will ignore this parameter from now on. Slant σ s affects the aspect ratio of the recovered 3D shape through its effect on Zi and Z pair(i) . It means that mirrorsymmetrical 3D objects that are consistent with the 2D orthographic image form a one-parameter family controlled by σ s . This family is illustrated on one example in the following demo: http://shapebook. psych.purdue.edu/2.2/ The slant σ s can be any value in the range between 0 and 90 degrees. The 3D recovery cannot be performed when σ s = 0 degrees or 90 degrees. Before we choose a unique 3D shape from the one parameter family, we have to recover points whose symmetrical counterparts are not visible. To recover these points, additional constraints are required, for example, planarity of faces (Li et al., 2009; Pizlo et al., 2010). In order to use a planarity constraint, at least three points of a plane on which
Fig. 12.13 A 3D mirror-symmetrical object with planar faces. The position and the orientation of the planar face enclosed by a dotted contour can be determined if the 3D coordinates of three points on the plane are known. In this image, five points on the planar face can be recovered with their visible counterparts (open circles) using Eq. (26). The remaining points on this face can be recovered as intersections of the projection lines and this plane.
the visible point is located have to be recovered first. These three points define the plane, which means that the Z -coordinate of the visible point can be computed as an intersection of this plane and the projection line emanating from the image of this visible point (see Figure 12.13). The hidden counterpart of the visible point is recovered as a mirror-reflection with respect to the symmetry plane of the 3D shape. Now, we are ready to select a unique 3D shape from the one-parameter family. This is done by using maximum 3D compactness and minimum surface area of the 3D shape as additional constraints. 3D compactness is defined as follows (Hildebrandt & Tromba, 1996): ( (27) Volume2 (η) Surface3 (η) where η represents the 3D shape, Volume (η) and Surface (η) are the volume and the surface area of η. The next demo shows 8 members from the one-parameter family of 3D shapes produced from the 2D line drawing shown in the center: http://shapebook.psych. purdue.edu/2.3/ Some of the shapes are tall, whereas others are wide. The shape at the 3-oclock has maximal 3D compactness, which essentially means that it has most volume for a given surface area. We showed that human monocular performance in recovering 3D mirror-symmetrical shapes is best simulated by selecting a 3D object that maximizes the following criterion (Li et al., 2009, 2011): ( (28) Volume(η) Surface3 (η) This criterion is a combination of (or a compromise between) maximizing 3D compactness of the 3D shape perception
271
object and minimizing its surface area. In other words, the observer’s percept corresponds to a 3D shape that is slightly thinner in the depth direction compared to the maximally compact 3D shape. In the demo you just saw, the shape in the bottom right corner maximizes criterion (28). If it is difficult to determine the surface and the volume of the object, the compactness criterion (28) is applied to the 3D convex hull containing the entire object. This method was used to recover a 3D shape from a hand drawing of a real object (praying mantis): http://shapebook.psych.purdue.edu/1.2/mantis/ If the original image was corrected using Eq. (22), the recovered object is not consistent with the original image. This means that the recovered object must be distorted (un-corrected) so that the object agrees with the original image: V i = Vi + i where i is a 3D distortion and V i is the position of the point i after the distortion. Let the 3D coordinates of i be [X i , Y i , Z i ]t . From Eq. (22), i = [0, (yi - ypair(i) )/2, Z i ]t , where Z i can be arbitrary. The magnitude of 3D distortion |i | is minimized when Z i = 0. The result is a maximally symmetrical 3D shape consistent with the 2D image.2 Next, we will explain binocular shape recovery. In order to motivate the formulation of the binocular model, look at Figure 12.14a, which shows results of monocular and binocular 3D shape recovery of one of our subjects (Li et al., 2011). The abscissa shows the slant of a stationary “reference” 3D shape that was viewed monocularly or binocularly, depending on the viewing condition. We used 3D symmetrical polyhedral objects consisting of 16 vertices. The subject’s task was to adjust the aspect ratio of a “test” 3D shape so
(a)
that it perceptually matched the 3D reference shape. The test shapes were taken from the one-parameter family of 3D shapes produced from the image of the reference shape. The test shape was viewed monocularly and was rotating around the vertical axis in order to show to the subject many views of this test shape. The ordinate shows the error of the match on a log scale. Zero means perfect (veridical) match. Negative (or positive) 1.0 means that the adjusted 3D shape had an aspect ratio that was different from the true aspect ratio by a factor of 2. Monocular adjustment of the aspect ratio is close to veridical when the slant of the symmetry plane is not too far from 45 degrees. For slants 15 degrees and 75 degrees the errors were larger in the direction of 3D shapes that were thinner along the depth direction. It is important to point out, however, that the 3D shapes in our experiment were characterized by as many as 20 parameters (8 vertices characterizing one-half of the object, times 3 coordinates (x, y, z), plus 3 parameters for the orientation of the symmetry plane, minus 7 parameters characterizing rigid motion in a 3D space and size scaling). The fact that the subjects were able to match the 3D shapes perceptually means that they recovered 19 of the 20 parameters veridically and the error existed only in the aspect ratio. Now, look at the black curve in the left panel of Figure 12.14. This curve, which shows binocular performance of our subject, essentially coincides with the x-axis. The binocular shape percept was absolutely veridical for all slants tested! This is the first such result in the entire history of research on 3D shape perception. The main reason for why we
(b)
2
2 YS: Model
YS 1 Dissimilarity
Dissimilarity
1
0
–1
–1
–2
0
Monocular Binocular
Monocular Binocular –2 15
30
45 Slant (°)
60
75
15
30
45 Slant (°)
60
75
Fig. 12.14 Results of a subject, YS, (a) and results of our “YS: Model” (b) that simulated the performance of this subject. Monocular performance is shown in grey and binocular performance is shown in black.
272
higher level cognition
observed veridical perception, while all others did not, is that we, but no one else, used symmetrical objects presented at nondegenerate viewing directions. Using Brunswik’s (1956) terminology, our stimuli were ecologically valid: they captured all important characteristics of natural objects in our natural environment. Natural objects are symmetrical and volumetric. Unstructured stimuli such as bent paper clips (among others) that dominated prior research have no resemblance to natural objects. Results obtained with such unnatural stimuli do not allow making any generalizations, whatsoever, to how we see in everyday life. The argument is the same as that used in explaining why responses of a nonlinear system to one set of inputs does not allow predicting the response of the system to other inputs. Everyone knows that the human visual system is highly nonlinear, but somehow this fact has been ignored when “shape” stimuli were designed in prior laboratory experiments and generalizations to natural shapes were drawn from such limited experimental results. Formulating a binocular model that achieves perfect performance is not easy. It is much more difficult than formulating a model that performs poorly. There are two types of binocular abilities: judging depth intervals and judging depth order. The former is very sloppy, but the latter is excellent. Judging depth order belongs to hyperacuities because it is performed with precision that is an order of magnitude better than the distance between the retinal receptors (Blakemore, 1970; Westheimer, 1979; Regan, 2000). All prior models of binocular shape perception were based on binocular depth intervals. Our model uses the depth order. How can a depth order lead to veridical perception of a metric shape? The way we do it is not very different from the way multidimensional scaling (MDS) works. We combine depth order with an a priori constraint. Whereas MDS used a fairly weak constraint, namely that the ordinal judgments come from a metric space, we used a much stronger constraint of mirrorsymmetry. The way 3D symmetry interacts with depth order is shown in the following interactive demo: http://shapebook.psych.purdue.edu/5.2/ Use your mouse to “walk” through the one parameter family of 3D symmetrical shapes produced from the image shown on the left. All these 3D shapes produce the same image, which is illustrated by the fact that the horizontal projection lines do not change. By changing the slant of the symmetry plane, the aspect ratio of the 3D
shape changes resulting in a frequent change of the depth order of the vertices. The depths of the vertices are marked by colored vertical lines. If the depth order is correctly perceived by the observer, the aspect ratio of the 3D shape is restricted to a very small range. Our Bayesian model puts the symmetry and planarity priors in the likelihood function which also evaluates the probability of the depth order of many pairs of vertices. The prior is a Gaussian distribution whose maximum corresponds to the maximum of the criterion shown in formula (28). The product of these two functions is a posterior whose maximum is taken as the model’s “percept.” When the information about depth order is less reliable, for example due to a larger viewing distance, the maximum of the posterior gets closer to the maximum of the prior. In the extreme case, when the model “views” the shape monocularly, the posterior is equivalent to the prior. Both the monocular and binocular performance of our model is shown in Figure 12.14b. Note how close the model’s performance is to the performance of our subject. Recovery of a 3D shape is mathematically simpler when a 2D perspective, rather than orthographic image is used. After solving the symmetry correspondence problem in the 2D image, the vanishing point for the symmetry line segments is estimated. This vanishing point allows a unique 3D shape recovery, which means that the compactness constraint is not needed anymore or it is less critical. However, reliable estimation of the vanishing point is computationally difficult, unless other constraints such as the direction of gravity and the position of the horizon in the 2D image are used (Li et al., 2012).
Conclusion: a new definition of shape All natural objects in our natural environment have a great deal of symmetry. Once one appreciates this fact, it follows that any meaningful definition of shape must be based on the concept of symmetry. We claim that symmetry is the sine qua non of shape (Li et al., 2013; Pizlo, Li, Sawada, & Steinman, 2014). In plain English, there is only as much shape in an object as there is symmetry in it. It implies that some objects have more shape than others and there may be objects with no shape, whatsoever. Consider completely irregular objects, such as a crumpled newspaper and a bent paperclip, or a random set of points. Such stimuli were quite common in prior studies on shape perception. These studies showed that shape constancy fails with those objects shape perception
273
(see Pizlo, 2008 for review). This is not surprising because these objects did not have shape to begin with. Symmetry refers to spatial self-similarity, which means that one part of an object is produced by copying, transforming, and pasting another part. If the transformation is a reflection, we produce mirror-symmetry, the type of symmetry characterizing most animal bodies. If the transformation is a rotation, we produce rotational symmetry, the type of symmetry characterizing flowers. If the transformation is a translation, we produce generalized cones, like those used by Biederman (1987) to define his geons. As a reminder, a generalized cone is produced by sweeping a planar cross-section in a 3D space along an axis. The size of the cross-section does not have to be constant, hence the name “cone.” A circular cone is produced by sweeping a circle along a straight line segment and reducing linearly the circle’s diameter. If a square is used instead of a circle, we produce a square pyramid. And so on. If the cross-section is rotated around and translated along the axis, we produce a spiral symmetry like that characterizing shells of snails. Spiral symmetry is also present in the arrangement of leaves around the stem of a flower. Note that no real object is exactly symmetrical. In such a case, the shape of the object can be represented as: symmetry + distortion. One-half of a human face is never an exact reflection of the other half. The difference is usually so small that it is hard to be detected. But when a person makes an asymmetrical grimace, we do not seize to see the face as symmetrical in the sense that we know what needs to be changed in the face to make it perfectly symmetrical, again. In fact, we perceive the face as the same even when the grimace changes in time, and we perceive the person as the same even when the person walks or runs. As long as symmetry is not completely removed from a 3D nonrigid object, the shape can be recognized as the same. This is illustrated in the following three demos: http://shapebook.psych.purdue.edu/1.1/. It should be obvious to the reader at this point that using symmetry to define shape has great advantages because it allows us to use mathematical tools of group theory, including group transformations and their invariants to describe real objects that we encounter in our natural environment. The same tools will surely be used to explain how our visual system recognizes shapes, remembers them, and uses them to identify objects and their functions. The diagram in Figure 12.11 illustrated 274
higher level cognition
how a symmetry group characterizing the 3D shape “out there” is transformed into another symmetry group characterizing model-based invariants in the 2D retinal image. This is just a glimpse of what and how much can be done when shape is defined the way we propose to do it, namely by the object’s symmetries. Finally, and most important, symmetry is a natural prior in the task of recovery of 3D shapes from 2D images. It is the kind of prior that need not be learned and need not be updated during our experience with real objects in our natural environment. It allows us to see 3D shapes of objects veridically regardless of whether the objects are familiar or not. As a mathematical concept, symmetry exists, so to speak, independently from real objects, and there is no reason to believe that it can be improved or “updated” during our interaction with the physical environment.
Notes 1. It is also possible to derive the 3D coordinates using multiple-view geometry (Vetter & Poggio, 1994; Li, Pizlo, & Steinman, 2009). Assume a single 2D orthographic image of a 3D mirror-symmetric shape is given. Then, a virtual image of the same 3D shape from a different viewpoint can be computed by reflecting the original 2D image with respect to an arbitrary line (Vetter & Poggio, 1994; see the section Symmetry of a 3D Shape and Its 2D Orthographic and Perspective Projections). The 3D shape can be computed from these two 2D images of the shape using multiple-view geometry resulting in Eq. (26) (Sawada, 2010). 2. You can see the application of this method here: http://shapebook.psych.purdue.edu/2.4/ In these three examples, we recovered the skeleton of a human body. The compactness criterion (24) was applied to the 3D convex hull containing the skeleton.
Glossary Bayesian method: of solving inverse problems. The posterior is proportional to the product of a likelihood function and a prior. The likelihood function is a probability that the 3D object can produce the 2D image. The prior is the a priori probability of the 3D object. The 3D object for which the posterior has a global maximum is taken as the solution. Degenerate view: Two or more 3D points project to the same 2D point. Forward problem: Mapping from a 3D object (model) to a 2D image (data). Forward problem is well-posed: solution exists, is unique, and depends continuously on the data. Group of transformations: A set of transformations is a group if the following four axioms are satisfied: 1. Identity transformation is in the set. 2. Any transformation in the set can be undone by an inverse transformation.
Glossary 3. The composition of two transformations from the set is also in this set (closure). 4. The result of a composition of three transformations does not depend on how we group these operations (associativity). Group invariants: A set of transformations has invariants if and only if the set is a group. An invariant is a property that remains unchanged under the transformation. Invariants exist only if the transformations are one-to-one. Homogeneous coordinates: Generalizations of Euclidean coordinates that make it possible to use matrix operations in the analysis of a perspective and projective transformations. Inverse problem: Mapping from a 2D image (data) to a 3D object (model). Inverse problem is ill-posed. Solving inverse problem is equivalent to an inference problem. Model-based invariants: Invariants of a set of transformations that is not a group. These invariants are typically formulated by restricting the types of objects. Regularization method: of solving inverse problems. A cost function evaluates how close the 3D object is to the 2D image and how well the 3D object satisfies a priori simplicity constraints. The 3D object for which the cost function has a global minimum is taken as the solution. Shape: Refers to all spatially global self-similarities (symmetries) of an object. Shape constancy: Refers to the fact that the perceived shape of a 3D object is constant despite changes in the shape of the 2D retinal image produced by changes in the 3D viewing direction. Symmetry: Refers to all spatial self-similarities of an object. There are three common types of symmetries in the natural objects: mirror (reflectional), rotational and translational. Symmetry correspondence: problem: Establishing which pairs (more generally which sets) of 2D points, features, or contours form 3D symmetrical configurations. Symmetry lines: In a mirror-symmetrical object symmetry lines connect pairs of symmetrical points. These lines are parallel in the 3D object and they are bisected by the plane of symmetry. In a 2D orthographic image symmetry lines are also parallel. In a 2D perspective image symmetry lines intersect at the vanishing point, which is a projection of a point at infinity). Vanishing point: In a 2D perspective image the vanishing point is a projection of a 3D point in infinity that represents the intersection of parallel lines in the 3D space. Veridical perception: Refers to the fact that we see the 3D objects and 3D scenes the way they are “out there.”
References Attneave, F. (1954). Some informational aspects of visual perception, Psychological Review, 61, 183–193. Biederman, I. (1987). Recognition-by-components: A theory of human image understanding. Psychological Review, 94, 115–147.
Biederman, I. & Gerhardstein, P. C. (1993). Recognizing depth-rotated objects: Evidence and conditions from threedimensional viewpoint invariance. Journal of Experimental Psychology: Human Perception & Performance, 19, 1162–82. Berkeley, G. (1910). A new theory of vision. New York, NY: Dutton. Blakemore, C. (1970). The range and scope of binocular depth discrimination in man. Journal of Physiology, 211, 599–622. Bouman, C. & Sauer, K. (1993). A Generalized Gaussian image model for edge-preserving MAP estimation. IEEE Transactions on Image Processing, 2, 296–310. Brunswik, E. (1956). Perception and the representative design of psychological experiments. Berkeley, CA: University of California Press. Burns, J. B., Weiss, R. & Riseman, E. M. (1990) View variation of point set and line segment features. Proceedings of DARPA Image Understanding Workshop, 650–659. Chan, M. W., Stevenson, A. K., Li, Y. & Pizlo, Z. (2006) Binocular shape constancy from novel views: the role of a priori constraints. Perception & Psychophysics, 68, 1124– 1139. Coxeter, H. S. M. (1987). Projective geometry (second edition). New York, NY: Springer. Fechner, G. (1860/1966). Elements of psychophysics. New York, NY: Holt, Rinehart & Winston. Hartley, R. & Zisserman, A. (2003). Multiple view geometry in computer vision. Cambridge, MA: Cambridge University Press. Hilbert, D., & Cohn-Vossen, S. (1952). Geometry and the imagination; New York, NY: Chelsea. Hildebrandt, S., & Tromba, A. (1996). The parsimonious universe. New York: Springer. Kersten, D. (1999). High level vision as statistical inference. In M. S. Gazzaniga (Ed.), The new cognitive neurosciences (pp. 353–363). Cambridge, MA: MIT Press. Knill, D. C. & Richards, W. (1996). Perception as Bayesian inference. Cambridge, MA: Cambridge University Press. Köhler, W. (1920/1938). Physical Gestalten. In: W. D. Ellis, (Ed.) A source book of Gestalt psychology (pp. 17–54). New York, NY: Humanities Press. Li, Y., Pizlo, Z. & Steinman, R. M. (2009). A computational model that recovers the 3D shape of an object from a single 2D retinal representation. Vision Research, 49, 979–991. Li, Y. & Pizlo, Z. (2011). Depth cues vs. simplicity principle in 3D shape perception. Topics in Cognitive Science 3, 667–685. Li, Y., Sawada, T., Shi, Y., Kwon, T. & Pizlo, Z. (2011). A Bayesian model of binocular perception of 3D mirror symmetric polyhedra. Journal of Vision, 11(4), 1–20. Li, Y., Sawada, T., Shi, Y., Steinman, R. M. & Pizlo, Z. (2013). Symmetry is the sine qua non of shape. In: S. Dickinson & Z. Pizlo (Eds.), Shape perception in human and computer vision, New York, NY: Springer. Li, Y., Sawada, T., Latecki, L. M., Steinman, R. M. & Pizlo, Z. (2012). Visual recovery of the shapes and sizes of objects, as well as distances among them, in a natural 3D scene. Journal of Mathematical Psychology, 56, 217–231. Mach, E. (1906/1959). The Analysis of Sensations. New York, NY: Dover. Marr, D. (1982). Vision. New York, NY: W.H. Freeman.
shape perception
275
Mundy, J. L. & Zisserman, A. (1992). Geometric invariance in computer vision. Cambridge, MA: MIT Press. Mundy, J.L., & Zisserman, A. & Forsyth, D. (1993). Applications of invariance in computer vision. New York, NY: Springer-Verlag. Pizlo, Z. (1991). Shape constancy in human beings and computers based on a perspective invariant. Doctoral dissertation. Department of Psychology, University of Maryland, College Park, MD. Pizlo, Z. (1994). A theory of shape constancy based on perspective invariants. Vision Research, 34, 1637–1658. Pizlo, Z. (2001). Perception viewed as an inverse problem. Vision Research, 41, 3145–3161. Pizlo, Z. (2008). 3D shape: its unique place in visual perception. Cambridge, MA: MIT Press. Pizlo, Z., Li, Y., Sawada, T. & Steinman, R.M. (2014). Making a machine that sees like us. New York, NY: Oxford University Press. Pizlo, Z. & Rosenfeld, A. (1992). Recognition of planar shapes from perspective images using contour-based invariants. Computer Vision, Graphics, and Image Processing: Image Understanding, 56, 330–350. Pizlo, Z. & Stevenson, A.K. (1999). Shape constancy from novel views. Perception & Psychophysics, 61, 1299–1307. Pizlo, Z., Sawada, T., Li, Y., Kropatsch, W., & Steinman, R.M. (2010). New approach to the perception of 3D shape based on veridicality, complexity, symmetry and volume. Vision Research, 50, 1–11. Poggio, T., Torre, V. & Koch, C. (1985). Computational vision and regularization theory. Nature 317, 314–319. Regan, D. (2000). Human perception of objects. Sunderland, MA: Sinauer. Rothwell, C. A. (1995). Object recognition through invariant indexing. Oxford, England: Oxford University Press. Sawada, T. (2010). Visual detection of symmetry in 3D shapes. Journal of Vision, 10(6), 4, 1–22.
276
higher level cognition
Sawada, T. & Pizlo, Z. (2008). Detection of skewed symmetry. Journal of Vision, 8(5), pp. 1–18. Sawada, T., Li, Y. & Pizlo, Z. (2011). Any pair of 2D curves is consistent with a 3D symmetric interpretation. Symmetry 3, 365–388. Sawada T., Li Y. & Pizlo Z. (2014). Detecting 3-D Mirror Symmetry in a 2-D Camera Image for 3-D Shape Recovery. Proceedings of the IEEE, 102, 1588–1606. Springer, C. (1964). Geometry and analysis of projective spaces. New York, NY: W.H. Freeman. Tikhonov, A. N. & Arsenin, V. Y. (1977). Solutions of ill-posed problems. New York, NY: Wiley. Vetter, T. & Poggio, T. (1994). Symmetric 3D objects are an easy case for 2D object recognition. Spatial Vision, 8 (4), 443–453. Wagemans, J. (1992). Perceptual use of nonaccidental properties. Canadian Journal of Psychology, 46, 236–279. Wagemans, J. (1993). Skewed symmetry: A nonaccidental property used to perceive visual forms. Journal of Experimental Psychology: Human Perception and Performance, 19, 364–380. Weiss, I. (1988). Projective invariants of shapes. Proceedings of DARPA image understanding warkshop (pp. 1125–1134) Cambridge, MA. Weiss, I. (1993). Geometric invariants and object recognition. International Journal of Computer Vision, 10, 207–231. Wertheimer, M. (1923/1958). Principles of perceptual organization. In: D. C. Beardslee & M. Wertheimer (Eds.) Readings in Perception (pp. 115–135). New York, NY: van Nostrand. Westheimer, G. (1979). Cooperative neural processes involved in stereoscopic acuity. Experimental Brain Research, 36, 585– 597. Zabrodsky, H., & Weinshall D. (1997). Using bilateral symmetry to improve 3D reconstruction from image sequences. Computer Vision and Image Understanding, 67, 48–57.
PART
New Directions
IV
CHAPTER
13
Bayesian Estimation in Hierarchical Models
John K. Kruschke and Wolf Vanpaemel
Abstract Bayesian data analysis involves describing data by meaningful mathematical models, and allocating credibility to parameter values that are consistent with the data and with prior knowledge. The Bayesian approach is ideally suited for constructing hierarchical models, which are useful for data structures with multiple levels, such as data from individuals who are members of groups which in turn are in higher-level organizations. Hierarchical models have parameters that meaningfully describe the data at their multiple levels and connect information within and across levels. Bayesian methods are very flexible and straightforward for estimating parameters of complex hierarchical models (and simpler models too). We provide an introduction to the ideas of hierarchical models and to the Bayesian estimation of their parameters, illustrated with two extended examples. One example considers baseball batting averages of individual players grouped by fielding position. A second example uses a hierarchical extension of a cognitive process model to examine individual differences in attention allocation of people who have eating disorders. We conclude by discussing Bayesian model comparison as a case of hierarchical modeling. Key Words: Bayesian statistics, Bayesian data analysis, Bayesian modeling, hierarchical model, model comparison, Markov chain Monte Carlo, shrinkage of estimates, multiple comparisons, individual differences, cognitive psychometrics, attention allocation
The Ideas of Hierarchical Bayesian Estimation Bayesian reasoning formalizes the reallocation of credibility over possibilities in consideration of new data. Bayesian reasoning occurs routinely in everyday life. Consider the logic of the fictional detective Sherlock Holmes, who famously said that when a person has eliminated the impossible, then whatever remains, no matter how improbable, must be the truth (Doyle, 1890). His reasoning began with a set of candidate possibilities, some of which had low credibility a priori. Then he collected evidence through detective work, which ruled out some possibilities. Logically, he then reallocated credibility to the remaining possibilities. The complementary logic of judicial exoneration is also
commonplace. Suppose there are several unaffiliated suspects for a crime. If evidence implicates one of them, then the other suspects are exonerated. Thus, the initial allocation of credibility (i.e., culpability) across the suspects was reallocated in response to new data. In data analysis, the space of possibilities consists of parameter values in a descriptive model. For example, consider a set of data measured on a continuous scale, such as the weights of a group of 10-year-old children. We might want to describe the set of data in terms of a mathematical normal distribution, which has two parameters, namely the mean and the standard deviation. Before collecting the data, the possible means and standard deviations have some prior credibility, about which 279
we might be very uncertain or highly informed. After collecting the data, we reallocate credibility to values of the mean and standard deviation that are reasonably consistent with the data and with our prior beliefs. The reallocated credibilities constitute the posterior distribution over the parameter values. We care about parameter values in formal models because the parameter values carry meaning. When we say that the mean weight is 32 kilograms and the standard deviation is 3.2 kilograms, we have a clear sense of how the data are distributed (according to the model). As another example, suppose we want to describe children’s growth with a simple linear function, which has a slope parameter. When we say that the slope is 5 kilograms per year, we have a clear sense of how weight changes through time (according to the model). The central goal of Bayesian estimation, and a major goal of data analysis generally, is deriving the most credible parameter values for a chosen descriptive model, because the parameter values are meaningful in the context of the model. Bayesian estimation provides an entire distribution of credibility over the space of parameter values, not merely a single “best” value. The distribution precisely captures our uncertainty about the parameter estimate. The essence of Bayesian estimation is to formally describe how uncertainty changes when new data are taken into account.
Hierarchical Models Have Parameters with Hierarchical Meaning In many situations, the parameters of a model have meaningful dependencies on each other. As a simplistic example, suppose we want to estimate the probability that a type of trick coin, manufactured by the Acme Toy Company, comes up heads. We know that different coins of that type have somewhat different underlying biases to come up heads, but there is a central tendency in the bias imposed by the manufacturing process. Thus, when we flip several coins of that type, each several times, we can estimate the underlying biases in each coin and the typical bias and consistency of the manufacturing process. In this situation, the observed heads of a coin depend only on the bias in the individual coin, but the bias in the coin depends on the manufacturing parameters. This chain of dependencies among parameters exemplifies a hierarchical model (Kruschke, 2015, Ch. 9). 280
new directions
As another example, consider research into childhood obesity. The researchers measure weights of children in a number of different schools that have different school lunch programs, and from a number of different school districts that may have different but unknown socioeconomic statuses. In this case, a child’s weight might be modeled as dependent on his or her school lunch program. The school lunch program is characterized by parameters that indicate the central tendency and variability of weights that it tends to produce. The parameters of the school lunch program are, in turn, dependent on the school’s district, which is described by parameters indicating the central tendency and variability of school-lunch parameters across schools in the district. This chain of dependencies among parameters again exemplifies a hierarchical model. In general, a model is hierarchical if the probability of one parameter can be conceived to depend on the value of another parameter. Expressed formally, suppose the observed data, denoted D, are described by a model with two parameters, denoted α and β. The probability of the data is a mathematical function of the parameter values, denoted by p(D|α, β), which is called the likelihood function of the parameters. The prior probability of the parameters is denoted p(α, β). Notice that the likelihood and prior are expressed, so far, in terms of combinations of α and β in the joint parameter space. The probability of the data, weighted by the probability of the parameter values, is the product, p(D|α, β)p(α, β). The model is hierarchical if that product can be factored as a chain of dependencies among parameters, such as p(D|α, β)p(α, β) = p(D|α)p(α|β)p(β). Many models can be reparameterized, and conditional dependencies can be revealed or obscured under different parameterizations. The notion of hierarchical has to do with a particular meaningful definition of a model structure that expresses dependencies among parameters in a meaningful way. In other words, it is the semantics of the parameters when factored in the corresponding way that makes a model hierarchical. Ultimately, any multiparameter model merely has parameters in a joint space, whether that joint space is conceived as hierarchical or not. Many realistic situations involve natural hierarchical meaning, as illustrated by the two major examples that will be described at length in this chapter. One of the primary applications of hierarchical models is describing data from individuals within
groups. A hierarchical model may have parameters for each individual that describe each individual’s tendencies, and the distribution of individual parameters within a group is modeled by a higherlevel distribution with its own parameters that describe the tendency of the group. The individuallevel and group-level parameters are estimated simultaneously. Therefore, the estimate of each individual-level parameter is informed by all the other individuals via the estimate of the group-level distribution, and the group-level parameters are more precisely estimated by the jointly constrained individual-level parameters. The hierarchical approach is better than treating each individual independently because the data from different individuals meaningfully inform one another. And the hierarchical approach is better than collapsing all the individual data together because collapsed data may blur or obscure trends within each individual.
are inherently designed to provide clear representations of uncertainty. A thorough critique of frequentist methods such as p values would take us too far afield. Interested readers may consult many other references, such as articles by Kruschke (2013) or Wagenmakers (2007).
Some Mathematics and Mechanics of Bayesian Estimation The mathematically correct reallocation of credibility over parameter values is specified by Bayes’ rule (Bayes & Price, 1763): p(α|D) = p(D|α) p(α) /p(D) ) *+ , ) *+ , )*+, posterior likelihood prior
(1)
where p(D) =
d α p(D|α)p(α)
(2)
Advantages of the Bayesian Approach Bayesian methods provide tremendous flexibility in designing models that are appropriate for describing the data at hand, and Bayesian methods provide a complete representation of parameter uncertainty (i.e., the posterior distribution) that can be directly interpreted. Unlike the frequentist interpretation of parameters, there is no construction of sampling distributions from auxiliary null hypotheses. In a frequentist approach, although it may be possible to find a maximum-likelihood estimate (MLE) of parameter values in a hierarchical nonlinear model, the subsequent task of interpreting the uncertainty of the MLE can be very difficult. To decide whether an estimated parameter value is significantly different from a null value, frequentist methods demand construction of sampling distributions of arbitrarily-defined deviation statistics, generated from arbitrarily-defined null hypotheses, from which p values are determined for testing null hypotheses. When there are multiple tests, frequentist decision rules must adjust the p values. Moreover, frequentist methods are unwieldy for constructing confidence intervals on parameters, especially for complex hierarchical nonlinear models that are often the primary interest for cognitive scientists.1 Furthermore, confidence intervals change when the researcher intention changes (e.g., Kruschke, 2013). Frequentist methods for measuring uncertainty (as confidence intervals from sampling distributions) are fickle and difficult, whereas Bayesian methods
is called the “marginal likelihood” or “evidence.” The formula in Eq. 1 is a simple consequence of the definition of conditional probability (e.g., Kruschke, 2015), but it has huge ramifications when applied to meaningful, complex models. In some simple situations, the mathematical form of the posterior distribution can be analytically derived. These cases demand that the integral in Eq. 2 can be mathematically derived in conjunction with the product of terms in the numerator of Bayes’ rule. When this can be done, the result can be especially pleasing because an explicit, simple formula for the posterior distribution is obtained. Analytical solutions for Bayes’ rule can rarely be achieved for realistically complex models. Fortunately, instead, the posterior distribution is approximated, to arbitrarily high accuracy, by generating a huge random sample of representative parameter values from the posterior distribution. A large class of algorithms for generating a representative random sample from a distribution is called Markov chain Monte Carlo (MCMC) methods. Regardless of which particular sampler from the class is used, in the long run they all converge to an accurate representation of the posterior distribution. The bigger the MCMC sample, the finer-resolution picture we have of the posterior distribution. Because the sampling process uses a Markov chain, the random sample produced by the MCMC process is often called a chain.
bayesian estimation in hierarchical models
281
Box 1 MCMC Details Because the MCMC sampling is a random walk through parameter space, we would like some assurance that it successfully explored the posterior distribution without getting stuck, oversampling, or undersampling zones of the posterior. Mathematically, the samplers will be accurate in the long run, but we do not know in advance exactly how long is long enough to produce a reasonably good sample. There are various diagnostics for assessing MCMC chains. It is beyond the scope of this chapter to review their details, but the ideas are straightforward. One type of diagnostic assesses how “clumpy” the chain is, by using a descriptive statistic called the autocorrelation of the chain. If a chain is strongly autocorrelated, successive steps in the chain are near each other, thereby producing a clumpy chain that takes a long time to smooth out. We want a smooth sample to be sure that the posterior distribution is accurately represented in all regions of the parameter space. To achieve stable estimates of the tails of the posterior distribution, one heuristic is that we need about 10,000 independent representative parameter values (Kruschke, 2015, Section 7.5.2). Stable estimates of central tendencies can be achieved by smaller numbers of independent values. A statistic called the effective sample size (ESS) takes into account the autocorrelation of the chain and suggests what would be an equivalently sized sample of independent values. Another diagnostic assesses whether the MCMC chain has gotten stuck in a subset of the posterior distribution, rather than exploring the entire posterior parameter space. This diagnostic takes advantage of running two or more distinct chains, and assessing the extent to which the chains overlap. If several different chains thoroughly overlap, we have evidence that the MCMC samples have converged to a representative sample.
It is important to understand that the MCMC “sample” or “chain” is a huge representative sample of parameter values from the posterior distribution. The MCMC sample is not to be confused with the sample of data. For any particular analysis, there is a single fixed sample of data, and there is a single underlying mathematical posterior distribution 282
new directions
that is inferred from the sample of data. The MCMC chain typically uses tens of thousands of representative parameter values from the posterior distribution to represent the posterior distribution. Box 1 provides more details about assessing when an MCMC chain is a good representation of the underlying posterior distribution. Contemporary MCMC software works seamlessly for complex hierarchical models involving nonlinear relationships between variables and nonnormal distributions at multiple levels. Modelspecification languages such as BUGS (Lunn, Jackson, Best, Thomas, & Spiegelhalter, 2013; Lunn, Thomas, Best, & Spiegelhalter, 2000), JAGS (Plummer, 2003), and Stan (Stan, 2013) allow the user to specify descriptive models to satisfy theoretical and empirical demands.
Example: Shrinkage and Multiple Comparisons of Baseball Batting Abilities American baseball is a sport in which one person, called a pitcher, throws a small ball as quickly as possible over a small patch of earth, called home plate, next to which is standing another person holding a stick, called a bat, who tries to hit the ball with the bat. If the ball is hit appropriately into the field, the batter attempts to run to other marked patches of earth arranged in a diamond shape. The batter tries to arrive at the first patch of earth, called first base, before the other players, called fielders, can retrieve the ball and throw it to a teammate attending first base. One of the crucial abilities of baseball players is, therefore, the ability to hit a very fast ball (sometimes thrown more than 90 miles [145 kilometers] per hour) with the bat. An important goal for enthusiasts of baseball is estimating each player’s ability to bat the ball. Ability can not be assessed directly but can only be estimated by observing how many times a player was able to hit the ball in all his opportunities at bat, or by observing hits and at-bats from other similar players. There are nine players in the field at once, who specialize in different positions. These include the pitcher, the catcher, the first base man, the second base man, the third base man, the shortstop, the left fielder, the center fielder, and the right fielder. When one team is in the field, the other team is at bat. The teams alternate being at bat and being in the field. Under some rules, the pitcher does not have to bat when his team is at bat. Because different positions emphasize different skills while on the field, not all players are prized
for their batting ability alone. In particular, pitchers and catchers have specialized skills that are crucial for team success. Therefore, based on the structure of the game, we know that players with different primary positions are likely to have different batting abilities.
The Data The data consist of records from 948 players in the 2012 regular season of Major League Baseball who had at least one at-bat.2 For player i, we have his number of opportunities at bat, ABi , his number of hits Hi , and his primary position when in the field pp(i). In the data, there were 324 pitchers with a median of 4.0 at-bats, 103 catchers with a median of 170.0 at-bats, and 60 right fielders with a median of 340.5 at-bats, along with 461 players in six other positions.
The Descriptive Model with Its Meaningful Parameters We want to estimate, for each player, his underlying probability θi of hitting the ball when at bat. The primary data to inform our estimate of θi are the player’s number of hits, Hi , and his number of opportunities at bat, ABi . But the estimate will also be informed by our knowledge of the player’s primary position, pp(i), and by the data from all the other players (i.e., their hits, atbats, and positions). For example, if we know that player i is a pitcher, and we know that pitchers tend to have θ values around 0.13 (because of all the other data), then our estimate of θi should be anchored near 0.13 and adjusted by the specific hits and at-bats of the individual player. We will construct a hierarchical model that rationally shares information across players within positions, and across positions within all major league players.3 We denote the ith player’s underlying probability of getting a hit as θi . (See Box 2 for discussion of assumptions in modeling.) Then the number of hits Hi out of ABi at-bats is a random draw from a binomial distribution that has success rate θi , as illustrated at the bottom of Figure 13.1. The arrow pointing to Hi is labeled with a “∼” symbol to indicate that the number of hits is a random variable distributed as a binomial distribution. To formally express our prior belief that different primary positions emphasize different skills and hence have different batting abilities, we assume that the player abilities θi come from distributions specific to each position. Thus, the θi ’s for the 324
Box 2 Model Assumptions For the analysis of batting abilities, we assume that a player’s batting ability, θi , is constant for all at-bats, and that the outcome of any at-bat is independent of other at-bats. These assumptions may be false, but the notion of a constant underlying batting ability is a meaningful construct for our present purposes. Assumptions must be made for any statistical analysis, whether Bayesian or not, and the conclusions from any statistical analysis are conditional on its assumptions. An advantage of Bayesian analysis is that, relative to 20th century frequentist techniques, there is greater flexibility to make assumptions that are appropriate to the situation. For example, if we wanted to build a more elaborate analysis, we could incorporate data about when in the season the at-bats occurred, and estimate temporal trends in ability due to practice or fatigue. Or, we could incorporate data about which pitcher was being faced in each atbat, and we could estimate pitcher difficulties simultaneously with batter abilities. But these elaborations, although possible in the Bayesian framework, would go far beyond our purposes in this chapter. pitchers are assumed to come from a distribution specific to pitchers, that might have a different central tendency and dispersion than the distribution of abilities for the 103 catchers, and so on for the other positions. We model the distribution of θi ’s for a position as a beta distribution, which is a natural distribution for describing values that fall between zero and one, and is often used in this sort of application (e.g., Kruschke, 2015). The mean of the beta distribution for primary position pp is denoted μpp , and the narrowness of the distribution is denoted κpp . The value of μpp represents the typical batting ability of players in primary position pp, and the value of κpp represents how tightly clustered the abilities are across players in primary position pp. The κ parameter is sometimes called the concentration or precision of the beta distribution.4 Thus, an individual player whose primary position is pp(i) is assumed to have a batting ability θi that comes from a beta distribution with mean μpp(i) and precision κpp(i) . The values of μpp and κpp are estimated simultaneously with all the θi . Figure 13.1 illustrates this aspect of the model by showing an arrow pointing to θi
bayesian estimation in hierarchical models
283
from a beta distribution. The arrow is labeled with “∼ . . . i” to indicate that the θi have credibilities distributed as a beta distribution for each of the individuals. The diagram shows beta distributions as they are conventionally parameterized by two shape parameters, denoted app and bpp , that can be algebraically redescribed in terms of the mean μpp and precision κpp of the distribution: app = μpp κpp and bpp = (1 − μpp )κpp . To formally express our prior knowledge that all players, from all positions, are professionals in major league baseball, and, therefore, should mutually inform each other’s estimates, we assume that the nine position abilities μpp come from an overarching beta distribution with mean μμpp and precision κμpp . This structure is illustrated in the upper part of Figure 13.1 by the split arrow, labeled with “∼ . . . pp”, pointing to μpp from a beta distribution. The value of μμpp in the overarching distribution represents our estimate of the batting ability of major league players generally, and the value of κμpp represents how tightly clustered the abilities are across the nine positions. These across-position parameters are
estimated from the data, along with all the other parameters. The precisions of the nine distributions are also estimated from the data. The precisions of the position distributions, κpp , are assumed to come from an overarching gamma distribution, as illustrated in Figure 13.1 by the split arrow, labeled with “∼ . . . pp”, pointing to κpp from a gamma distribution. A gamma distribution is a generic and natural distribution for describing nonnegative values such as precisions (e.g., Kruschke, 2015). A gamma distribution is conventionally parameterized by shape and rate values, denoted in Figure 13.1 as sκpp and rκpp . We assume that the precisions of each position can mutually inform each other; that is, if the batting abilities of catchers are tightly clustered, then the batting abilities or shortstops should probably also be tightly clustered, and so forth. Therefore the shape and rate parameters of the gamma distribution are themselves estimated. At the top level in Figure 13.1 we incorporate any prior knowledge we might have about general properties of batting abilities for players in the
Fig. 13.1 The hierarchical descriptive model for baseball batting ability. The diagram should be scanned from the bottom up. At the bottom, the number of hits by the i th player, Hi , are assumed to come from a binomial distribution with maximum value being the at-bats, ABi , and probability of getting a hit being θi . See text for further details.
284
new directions
major leagues, such as evidence from previous seasons of play. Baseball aficionados may have extensive prior knowledge that could be usefully implemented in a Bayesian model. Unlike baseball experts, we have no additional background knowledge, and, therefore, we will use very vague and noncommittal top-level prior distributions. Thus, the top-level beta distribution on the overall batting ability is given parameter values A = 1 and B = 1, which make it uniform over all possible batting abilities from zero to one. The top-level gamma distributions (on precision, shape, and rate) are given parameter values that make them extremely broad and noncommittal such that the data dominate the estimates, with minimal influence from the top-level prior. There are 970 parameters in the model altogether: 948 individual θi , plus μpp , κpp for each of nine primary positions, plus μμ , κμ across positions, plus sκ and rκ . The Bayesian analysis yields credible combinations of the parameters in the 970-dimensional joint parameter space. We care about the parameter values because they are meaningful. Our primary interest is in the estimates of individual batting abilities, θi , and in the position-specific batting abilities, μpp . We are also able to examine the relative precisions of abilities across positions to address questions such as, Are batting abilities of catchers as variable as batting abilities of shortstops? We will not do so here, however.
Results: Interpreting the Posterior Distribution We used MCMC chains with total saved length of 15,000 after adaptation of 1,000 steps and burnin of 1,000 steps, using 3 parallel chains called from the runjags package (Denwood, 2013), thinned by 30 merely to keep a modest file size for the saved chain. The diagnostics (see Box 1) assured us that the chains were adequate to provide an accurate and high-resolution representation of the posterior distribution. The effective sample size (ESS) for all the reported parameters and differences exceeded 6,000, with nearly all exceeding 10,000. check of robustness against changes in top-level prior constants Because we wanted the top-level prior distribution to be noncommittal and have minimal influence on the posterior distribution, we checked whether the choice of prior had any notable effect on the posterior. We conducted the analysis with
different constants in the top-level gamma distributions, to check whether they had any notable influence on the resulting posterior distribution. Whether all gamma distributions used shape and rate constants of 0.1 and 0.1, or 0.001 and 0.001, the results were essentially identical. The results reported here are for gamma constants of 0.001 and 0.001. comparisons of positions We first consider the estimates of hitting ability for different positions. Figure 13.2, left side, shows the marginal posterior distributions for the μpp parameters for the positions of catcher and pitcher. The distributions show the credible values of the parameters generated by the MCMC chain. These marginal distributions collapse across all other parameters in the high-dimensional joint parameter space. The lower-left panel in Figure 13.2 shows the distribution of differences between catchers and pitchers. At every step in the MCMC chain, the difference between the credible values of μcatcher and μpitcher was computed, to produce a credible value for the difference. The result is 15,000 credible differences (one for each step in the MCMC chain). For each marginal posterior distribution, we provide two summaries: Its approximate mode, displayed on top, and its 95% highest density interval (HDI), shown as a black horizontal bar. A parameter value inside the HDI has higher probability density (i.e., higher credibility) than a parameter value outside the HDI. The total probability of parameter values within the 95% HDI is 95%. The 95% HDI indicates the 95% most credible parameter values. The posterior distribution can be used to make discrete decisions about specific parameter values (as explained in Box 3). For comparing catchers and pitchers, the distribution of credible differences falls far from zero, so we can say with high credibility that catchers hit better than pitchers. (The difference is so big that it excludes any reasonable ROPE around zero that would be used in the decision rule described in Box 3.) The right side of Figure 13.2 shows the marginal posterior distributions of the μpp parameters for the positions of right fielder and catcher. The lower-right panel shows the distribution of differences between right fielders and catchers. The 95% HDI of differences excludes a difference of zero, with 99.8% of the distribution falling above zero. Whether or not we reject zero as a credible
bayesian estimation in hierarchical models
285
Right Field
Catcher
mode = 0.26
mode = 0.241
95% HDI 0.233 0.25 0.15
0.20
0.25
95% HDI 0.251 0.23
0.24
μpp=2
0.25 0.26 μpp=9
0.27
0.28
0.27
0.28
Catcher
Pitcher mode = 0.13
mode = 0.241
95% HDI
95% HDI 0.12 0.141
0.233
0.15
0.20
0.25
0.23
0.25 0.24
μpp=1
0.25 0.26 μpp=2
Difference
Difference mode = 0.111 0% < 0 < 100% 0% in ROPE 95% HDI 0.0976 0.125 0.00
0.269
0.04 0.08 μpp = 2−μpp = 1
0.12
mode = 0.0183 0.2% < 0 < 99.8% 10% in ROPE 95% HDI 0.0056 0.00
0.01
0.031 0.02 0.03 μpp=9−μpp=2
0.04
Fig. 13.2 Comparison of estimated batting abilities of different positions. In the data, there were 324 pitchers with a median of 4.0 at-bats, 103 catchers with a median of 170.0 at-bats, and 60 right fielders with a median of 340.5 at-bats, along with 461 players in six other positions. The modes and HDI limits are all indicated to three significant digits, with a trailing zero truncated from the display. In the lowest row, a difference of 0 is marked by a vertical dotted line annotated with the amount of the posterior distribution that falls below or above 0. The limits of the ROPE are marked with vertical dotted lines and annotated with the amount of the posterior distribution that falls inside it. The subscripts such as “pp = 2” indicate arbitrary indexical values for the primary positions, such as 1 for pitcher, 2 for catcher, and so forth.
difference depends on our decision rule. If we use a ROPE from −0.01 to +0.01, as shown in Figure 13.2, then we would not reject a difference of zero because the 95% HDI overlaps the ROPE. The choice of ROPE depends on what is practically equivalent to zero as judged by aficionados of baseball. Our choice of ROPE shown here is merely for illustration. In Figure 13.2, the triangle on the x-axis indicates the ratio in the data of total hits divided by total at-bats for all players in that position. Notice that the modes of the posterior are not centered exactly on the triangles. Instead, the modal estimates are shrunken toward the middle 286
new directions
between the pitchers (who tend to have the lowest batting averages) and the catchers (who tend to have higher batting averages). Thus, the modes of the posterior marginal distributions are not as extreme as the proportions in the data (marked by the triangles). This shrinkage is produced by the mutual influence of data from all the other players, because they influence the higher-level distributions, which in turn influence the lower-level estimates. For example, the modal estimate for catchers is 0.241, which is less than the ratio of total hits to total at-bats for catchers. This shrinkage in the estimate for catchers is caused by the fact that there are 324 pitchers who, as a group, have relatively low batting
Box 3 Decision Rules for Bayesian Posterior Distribution The posterior distribution can be used for making decisions about the viability of specific parameter values. In particular, people might be interested in a landmark value of a parameter, or a difference of parameters. For example, we might want to know whether a particular position’s batting ability exceeds 0.20, say. Or we might want to know whether two positions’ batting abilities have a non-zero difference. The decision rule involves using a region of practical equivalence (ROPE) around the null or landmark value. Values within the ROPE are equivalent to the landmark value for practical purposes. For example, we might declare that for batting abilities, a difference less than 0.04 is practically equivalent to zero. To decide that two positions have credibly different batting abilities, we check that the 95% HDI excludes the entire ROPE around zero. Using a ROPE also allows accepting a difference of zero: If the entire 95% HDI falls within the ROPE, it means that all the most credible values are practically equivalent to zero (i.e., the null value), and we decide to accept the null value for practical purposes. If the 95% HDI overlaps the ROPE, we withhold decision. Note that it is only the landmark value that is being rejected or accepted, not all the values inside the ROPE. Furthermore, the estimate of the parameter value is given by the posterior distribution, whereas the decision rule merely declares whether the parameter value is practically equivalent to the landmark value. We will illustrate use of the decision rule in the results from the actual analyses. In some cases we will not explicitly specify a ROPE, leaving some nonzero width ROPE implicit. In general, this allows flexibility in decision-making when limits of practical equivalence may change as competing theories and instrumentation change (Serlin & Lapsley. 1993). In some cases, the posterior distribution falls so far away from any reasonable ROPE that it is superfluous to specify a specific ROPE. For more information about the application of a ROPE, under somewhat different terms of “range of equivalence,” “indifference zone,” and “good-enough belt,” see e.g., Carlin and Louis (2009); Freedman, Lowe, and Macaskill (1984); Hobbs and Carlin (2008); Serlin and
Lapsley (1985, 1993); Spiegelhalter, Freedman, and Parmar (1994). Notice that the decision rule is distinct from the Bayesian estimation itself, which produces the complete posterior distribution. We are using a decision rule only in case we demand a discrete decision from the continuous posterior distribution. There is another Bayesian approach to making decisions about null values that is based on comparing a “spike” prior on the landmark value against a diffuse prior, which we discuss in the final section on model comparison, but for the purposes of this chapter we focus on using the HDI with ROPE.
ability, and pull down the overarching estimate of batting ability for major-league players (even with the other seven positions taken into account). The overarching estimate in turn affects the estimate of all positions, and, in particular, pulls down the estimate of batting ability for catchers. We see in the upper right of Figure 13.2 that the estimate of batting ability for right fielders is also shrunken, but not as much as for catchers. This is because the right fielders tend to be at bat much more often than the catchers, and, therefore, the estimate of ability for right fielders more closely matches their data proportions. In the next section we examine results for individual players, and the concepts of shrinkage will become more dramatic and more clear. comparisons of individual players In this section we consider estimates of the batting abilities of individual players. The left side of Figure 13.3 shows a comparison of two individual players with the same record, 1 hit in 3 at-bats, but who play different positions, namely catcher and pitcher. Notice that the triangles are at the same place on the x-axes for the two players, but there are radically different estimates of their probability of getting a hit because of the different positions they play. The data from all the other catchers inform the model that catchers tend to have values of θ around 0.241. Because this particular catcher has so few data to inform his estimate, the estimate from the higher-level distribution dominates. The same is true for the pitcher, but the higher-level distribution says that pitchers tend to have values of θ around 0.130. The resulting distribution of differences, in the lowest panel, suggests that these two players have
bayesian estimation in hierarchical models
287
Tim Federowicz (Catcher) 1 Hits/3 At Bats
Mike Leake (Pitcher) 18 Hits/61 At Bats mode = 0.157
mode = 0.241
0.191 0.05
95% HDI
95% HDI 0.297
0.15
0.25
0.119 0.35
0.05
0.10
0.209 0.15
0.20
θ263
Casey Coleman (Pitcher) 1 Hits/3 At Bats
mode = 0.112
95% HDI 0.0905 0.176
95% HDI 0.0825 0.156
0.15
0.25
0.35
0.05
0.10
0.15
0.30
mode = 0.0365 5.6% < 0 < 94.4% 46% in ROPE
0.1% < 0 < 99.9%
2% in ROPE
95% HDI
0.05
0.25
Difference
Difference
mode = 0.111
−0.05
0.20 θ754
θ169
0.0419
0.30
Wandy Rodriguez ( Pitcher) 4 Hits/61 At Bats
mode = 0.132
0.05
0.25
θ494
95% HDI 0.178
0.10 0.15 θ263−θ169
0.20
−0.0151 0.25
−0.05
0.00
0.105 0.05
0.10
0.15
0.20
θ494−θ754
Fig. 13.3 Comparison of estimated batting abilities of different individual players. The left column shows two players with the same actual records of 1 hit in 3 at-bats, but very different estimates of batting ability because they play different positions. The right column shows two players with rather different actual records (18/61 and 4/61) but similar estimates of batting ability because they play the same position. Triangles show actual ratios of hits/at-bats. Bottom histograms display an arbitrary ROPE from −0.04 to +0.04; different decision makers might use a different ROPE. The subscripts on θ indicate arbitrary identification numbers of different players, such as 263 for Tim Federowicz.
credibly different hitting abilities, even though their actual hits and at-bats are identical. In other words, because we know the players play these particular different positions, we can infer that they probably have different hitting abilities. The right side of Figure 13.3 shows another comparison of two individual players, both of whom are pitchers, with seemingly quite different batting averages of 18/61 and 4/61, as marked by the triangles on the x-axis. Despite the players’ different hitting records, the posterior estimates of their hitting probabilities are not very different. Notice the dramatic shrinkage of the estimates toward the mode of players who are pitchers.
288
new directions
Indeed, in the lower panel, we see that a difference of zero is credible, as it falls within the 95% HDI of the differences. The shrinkage is produced because there is a huge amount of data, from 324 pitchers, informing the position-level distribution about the hitting ability of pitchers. Therefore, the estimates of two individual pitchers with only modest numbers of at-bats are strongly shrunken toward the group-level mode. In other words, because we know that the players are both pitchers, we can infer that they probably have similar hitting abilities. The amount of shrinkage depends on the amount of data. This is illustrated in Figure 13.4,
Andrew McCutchen ( Center Field) 194 Hits/593 At Bats
ShinSoo Choo ( Right Field) 169 Hits/598 At Bats mode = 0.276
mode = 0.304
95% HDI
95% HDI 0.335 0.274 0.15
0.20
0.25
0.30
0.35
0.302
0.246 0.22
0.24
0.26
0.28
0.30
0.32
θ159
θ573
Brett Jackson (Center Field) 21 Hits/120 At Bats
Ichiro Suzuki ( Right Field) 178 Hits/629 At Bats
mode = 0.233
mode = 0.275
95% HDI
95% HDI 0.194 0.15
0.20
0.248
0.278 0.25
0.30
0.35
0.22
0.24
0.304 0.26
θ428
0.30
0.32
Difference
Difference
mode = −0.00212 50.8% < 0 < 49.2% 95% in ROPE
mode = 0.0643 0.5% < 0 < 99.5% 14% in ROPE
95% HDI
95% HDI 0.0171 0.00
0.28
θ844
−0.0398
0.122 0.05 0.10 θ573−θ428
0.15
−0.05
0.039 0.00 θ159− θ 844
0.05
Fig. 13.4 The left column shows two individuals with rather different actual batting ratios (194/593 and 21/120) who both play center field. Although there is notable shrinkage produced by playing the same position, the quantity of data is sufficient to exclude a difference of zero from the 95% HDI on the difference (lower histogram); although the HDI overlaps the arbitrary ROPE shown here, different decision makers might use a different ROPE. The right column shows two right fielders with very high and nearly identical actual batting ratios. The 95% HDI of their difference falls within the ROPE in the lower right histogram. Note: Triangles show actual batting ratio of hits/at-bats.
which shows comparisons of players from the same position, but for whom there are much more personal data from more at-bats. In these cases, although there is some shrinkage caused by position-level information, the amount of shrinkage is not as strong because the additional individual data keep the estimates anchored closer to the data. The left side of Figure 13.4 shows a comparison of two center fielders with 593 and 120 at-bats, respectively. Notice that the shrinkage of estimate for the player with 593 at-bats is not as extreme as the player with 120 at-bats. Notice also that the width of the 95% HDI for the player with 593 at-bats is narrower than for the player with 120
at-bats. This again illustrates the concept that the estimate is informed by both the data from the individual player and by the data from all the other players, especially those who play the same position. The lower left panel of Figure 13.4 shows that the estimated difference excludes zero (but still overlaps the particular ROPE used here). The right side of Figure 13.4 shows right fielders with huge numbers of at-bats and nearly the same batting average. The 95% HDI of the difference falls almost entirely within the ROPE, so we might decide to declare that players have identical probability of getting a hit for practical purposes, that is, we might decide to accept the null value of zero difference.
bayesian estimation in hierarchical models
289
Shrinkage and Multiple Comparisons In hierarchical models with multiple levels, there is shrinkage of estimates within each level. In the model of this section (Figure 13.1), there was shrinkage of the player-position parameters toward the overall central tendency, as illustrated by the pitcher and catcher distributions in Figure 13.2, and there was shrinkage of the individual-player parameters within each position toward the position central tendency, as shown by various examples in Figures 13.3 and Figure 13.4. The model also provided some strong inferences about player abilities based on position alone, as illustrated by the estimates for individual players with few at bats in the left column of Figure 13.3. There were no corrections for multiple comparisons. We conducted all the comparisons without computing p values, and without worrying whether we might intend to make additional comparisons in the future, which is quite likely given that there are 9 positions and 948 players in whom we might be interested. It is important to be clear that Bayesian methods do not prevent false alarms. False alarms are caused by accidental conspiracies of rogue data that happen to be unrepresentative of the true population, and no analysis method can fully mitigate false conclusions from unrepresentative data. There are two main points to be made with regard to false alarms in multiple comparisons from a Bayesian perspective. First, the Bayesian method produces a posterior distribution that is fixed, given the data. The posterior distribution does not depend on which comparisons are intended by the analyst, unlike traditional frequentist methods. Our decision rule, using the HDI and ROPE, is based on the posterior distribution, not on a false alarm rate inferred from a null hypothesis and an intended sampling/testing procedure. Second, false alarms are mitigated by shrinkage in hierarchical models (as exemplified in the right column of Figure 13.3). Because of shrinkage, it takes more data to produce a credible difference between parameter values. Shrinkage is a rational, mathematical consequence of the hierarchical model structure (which expresses our prior knowledge of how parameters are related) and the actually observed data. Shrinkage is not related in any way to corrections for multiple comparisons, which do not depend on the observed data but do depend on the intended comparisons. Hierarchical modeling is possible with non-Bayesian estimation, 290
new directions
but frequentist decisions are based on auxiliary sampling distributions instead of the posterior distribution.
Example: Clinical Individual Differences in Attention Allocation Hierarchical Bayesian estimation can be applied straightforwardly to more elaborate models, such as information processing models typically used in cognitive science. Generally, such models formally describe the processes underlying behavior in tasks such as thinking, remembering, perceiving, deciding, learning and so on. Cognitive models are increasingly finding practical uses in a wide variety of areas outside cognitive science. One of the most promising uses of cognitive process models is the field of cognitive psychometrics (Batchelder, 1998; Riefer, Knapp, Batchelder, Bamber, & Manifold, 2002; Vanpaemel, 2009), where cognitive process models are used as psychometric measurement models. These models have become important tools for quantitative clinical cognitive science (see Neufeld chapter 16, this volume). In our second example of hierarchical Bayesian estimation, we use data from a classification task and a corresponding cognitive model to assess young women’s attention to other women’s body size and facial affect, following the research of Treat, Nosofsky, McFall, & Palmeri, (2002). Rather than relying on self-reports, Viken et a1. (2002) collected performance data in a prototype-classification task involving photographs of women varying in body size and facial affect. Furthermore, rather than using generic statistical models for data analysis, the researchers applied a computational model of category learning designed to describe underlying psychological properties. The model, known as the multiplicative prototype model (MPM: Nosofsky, 1987; Reed, 1972), has parameters that describe how much perceptual attention is allocated to body size or facial affect. The modeling made it possible to assess how participants in the task allocated their attention. To measure attention allocation, Viken et al. (2002) tapped into women’s perceived similarities of photographs of other women. The women in the photographs varied in their facial expressions of affect (happy to sad) and in their body size (light to heavy). We focus here on a particular categorization task in which the observer had to classify a target photo as belonging with reference photo X or with reference photo Y. In one version
Sad
Affect Sad Happy
Happy
t
Affect
X
X
t Y
Y Light Heavy Body Size
Light
Heavy Body Size
Fig. 13.5 The perceptual space for photographs of women who vary on body size (horizontal axis) and affect (vertical axis). Photo X shows a prototypical light, happy woman and photo Y shows a prototypical heavy, sad woman. The test photo, t, is categorized with X or Y according to its relative perceptual proximity to those prototypes. In the left panel, attention to body size (denoted w in the text) is low, resulting in compression of the body size axis, and, therefore, test photo t tends to be classified with prototype X. In the right panel, attention to body size is high, resulting in expansion of the body size axis, and, therefore, test photo t tends to be classified with prototype Y.
of the experiment, reference photo X was of a light, happy woman and reference photo Y was of a heavy, sad woman. In another version, not discussed here, the features of the reference photos were reversed. Suppose the target photo t showed a heavy, happy woman. If the observer was paying attention mostly to affect, then photo t should tend to be classified with reference photo X, which matched on affect. If the observer was paying attention mostly to body size, then photo t should tend to be classified with reference photo Y, which matched on body size. A schematic representation of the perceptual space for photographs is shown in Figure 13.5. In the actual experiment, there were many different target photos from throughout the perceptual space. By recording how each target photo was categorized by the observer, the observer’s attention allocation can be inferred. Viken et al. (2002) were interested in whether women suffering from the eating disorder, bulimia, allocated their attention differently than normal women. Bulimia is characterized by bouts of overconsumption of food with a feeling of loss of control, followed by self-induced vomiting or abuse of laxatives to prevent weight gain. The researchers were specifically interested in how bulimics allocated their attention to other women’s facial affect and body size, because perception of body size has been the focus of past research into eating disorders, and facial affect is relevant to social perception but is not specifically implicated in eating disorders. An understanding of how bulimics allocate attention
could have implications for both the etiology and treatment of the disease. Viken et al. (2002) collected data from a group of woman who were high in bulimic symptoms, and from a group that was low. Viken et al. then used likelihood-ratio tests to compare a model that used separate attention weights in each group to a model that used a single attention weight for both groups. Their model-comparison approach revealed that high-symptom women, relative to low-symptom women, display enhanced attention to body size and decreased attention to facial affect. In contrast to their non-Bayesian, nonhierarchical, nonestimation approach, we use a Bayesian hierarchical estimation approach to investigate the same issue. The hierarchical nature of our approach means that we do not assume that all subjects within a symptom group have the same attention to body size. Bayesian inference and decision-making implies that we do not require assumptions about sampling intentions and multiple tests that are required for computing p values. Moreover, our use of estimation instead of only model comparison ensures that we will know how much the groups differ.
The Data Viken et a1. (2002) obtained classification judgments from 38 women on 22 pictures of other women, varying in body size (light to heavy) and facial affect (happy to sad). Symptoms of bulimia were also measured for all of the women. Eighteen of these women had BULIT scores exceeding 88, which is considered to be high in bulimic symptoms (Smith & Thelen, 1984). The remaining 20 women had BULIT scores lower than 45, which is considered to be low in bulimic symptoms. Each woman performed the classification task described earlier, in which she was instructed to freely classify each target photo t as one of two types of women exemplified by reference photo X and reference photo Y. No corrective feedback was provided. Each target photo was presented twice, hence, for each woman i, the data include the frequency of classifying stimulus t as a type X, ranging between 0 and 2. Our goal is to use these data to infer a meaningful measure of attention allocation for each individual observer, and simultaneously to infer an overall measure of attention allocation for women high in bulimic symptoms and for women low in bulimic symptoms. We will rely
bayesian estimation in hierarchical models
291
on a hierarchical extension of the MPM, as described next.
function of distance between t and X , dtX , in the psychological space: stX = exp (−ci dtX )
The Descriptive Model with Its Meaningful Parameters Models of categorization take perceptual stimuli as input and generate precise probabilities of category assignments as output. The input stimuli must be represented formally, and many leading categorization models assume that stimuli can be represented as points in a multidimensional space, as was suggested in Figure 13.5. Importantly, the models assume that attention plays a key role in categorization, and formalize the attention allocated to perceptual dimensions as free parameters (for a review see, e.g., Kruschke, 2008). In particular, the MPM (Nosofsky, 1987) determines the similarity between a target item and a reference item by multiplicatively weighting the separation of the items on each dimension by the corresponding attention allocated to each dimension. The higher the similarity of a stimulus to a reference category prototype, relative to other category prototypes, the higher the probability of assigning the stimulus to the reference category. For each trial in which a target photo t is presented with reference photos X and Y, the MPM produces the probability, pi (X |t), that the i th observer classifies stimulus t as category X . This probability depends on two free parameters. One parameter is denoted wi , which indicates the attention that the i th observer pays to body size. The value of wi can range from 0 to 1. Attention to affect is simply 1 − wi . The second parameter is denoted ci and called the “sensitivity” of observer i. The sensitivity can be thought of as the observer’s decisiveness, which is how strongly the observer converts a small similarity advantage for X into a large choice advantage for X. Note that attention and sensitivity parameters can differ across observers, but not across stimuli, which are assumed to have fixed locations in an underlying perceptual space. Formally, the MPM posits that the probability that photo t will be classified with reference photo X instead of reference photo Y is determined by the similarity of t to X relative to the total similarity: pi (X |t) = stX /(stX + stY ).
(3)
The similarity between target and reference is, in turn, determined as a nonlinearly decreasing 292
new directions
(4)
where ci > 0 is the sensitivity parameter for observer i. The psychological distance between target t and reference X is given by the weighted distance between the corresponding points in the 2-dimensional psychological space:
1/2 dtX = wi |xtb − xXb |2 + (1 − wi ) |xta − xXa |2 , (5) where xta denotes the position of the target on the affect dimension, and xtb denotes the position of the target on the body-size dimension. These positions are normative average ratings of the photographs on two 10-point scales: body size (1 = underweight, 10 = overweight), and affect (1 = unhappy, 10 = happy), as provided by a separate sample of young women. The free parameter 0 < wi < 1 corresponds to the attention weight on the body size dimension for observer i. It reflects the key assumption of the MPM that the structure of the psychological space is systematically modified by selective attention (see Figure 13.5). hierarchical structure We construct a hierarchical model that has parameters to describe each individual, and parameters to describe the overall tendencies of the bulimic and normal groups. The hierarchy is analogous to the baseball example discussed earlier: Just as individual players were nested within fielding positions, here individual observers are nested within bulimic symptom groups. (One difference, however, is that we do not build an overarching distribution across bulimic-symptom groups because there are only two groups.) With this hierarchy, we express our prior expectation that bulimic women are similar but not identical to each other, and nonbulimic women are similar but not identical to each other, but the two groups may be different. The hierarchical model allows the parameter estimates for an individual observer to be rationally influenced by the data from other individuals within their symptom group. In our model, the individual attention weights are assumed to come from an overarching distribution that is characterized by a measure of central tendency and of dispersion. The overarching distributions for the high-symptom and low-symptom groups are estimated separately. As the attention weights wi are constrained to range between 0 and 1,
we assume the parent distribution for the wi ’s is [g] a beta distribution, parameterized by mean μw [g] and precision κw , where [g] indexes the group membership (i.e., high symptom or low symptom). The individual sensitivities, ci , are also assumed to come from an overarching distribution. Since the sensitivities are non-negative, a gamma distribution is a convenient parent distribution, parameterized [g] [g] by mode moc and standard deviation σc , where [g] again indicates the group membership (i.e., high symptom or low symptom). The group[g] [g] [g] [g] level parameters (i.e., μw , moc , κw and σc ) are assumed to come from vague, noncommittal uniform distributions. There are 84 parameters altogether, including wi and ci for 38 observers and the 8 group level parameters. Figure 13.6 summarizes the hierarchical model in an integrated diagram. The caption provides details. The parameters of most interest are the group[g] level attention to body size, μw , for g ∈ {low, high}. Other meaningful questions could focus on the relative variability among groups in attention, [g] which would be addressed by considering the κw parameters, but we will not pursue these here.
Results: Interpreting the Posterior Distribution The Bayesian hierarchical approach to estimation yields attention weights for each observer, informed by all the other observers in the group. At the same time, it provides an estimate of the attention weight at the group level. Further, for every individual estimate and the group level estimates, a measure of uncertainty is provided, in the form of a credible interval (95% HDI), which can be used as part of a decision rule to decide whether or not there are credible differences between individuals or between groups. The MCMC process used 3 chains with a total of 100,000 steps after a burn-in of 4,000 steps. It produced a smooth (converged) representation of the 84-dimensional posterior distribution. We use the MCMC sample as an accurate and high-resolution representation of the posterior distribution. check of robustness against changes in top-level prior constants We conducted a sensitivity analysis by using different constants in the top-level uniform distributions, to check whether they had any notable influence on the resulting posterior distribution.
unif
unif
unif
unif
moc, σc
μw, κw beta
gamma ...i
...i
MPM (t, wi, ci ) = ...i pi (X/t)
Bernoulli ...t/i Xt/i Fig. 13.6 The hierarchical model for attention allocation. At the bottom of the diagram, the classification data are denoted as Xt|i = 1 if observer i says “X ” to target t, and Xt|i = 0 otherwise. The responses come from a Bernoulli distribution that has its success parameter determined by the MPM, as defined in Eqs. 3, 4, and 5 in the main text. The ellipsis on the arrow pointing to the response indicates that this relation holds for all targets within every individual. Scanning up the diagram, the individual attention parameters, wi , come from an overarching group-level beta distribution that has mean μw and concentration κw (hence shape parameters of aw = μw κw and bw = (1 − μw )κw , as was indicated explicitly for the beta distributions in Figure 13.1). The individual sensitivity parameters ci come from an overarching group-level gamma distribution that has mode moc and standard deviation σc (with shape and rate parameters that are algebraic combinations of moc and σc ; see Kruschke, 2015, Section 9.2.2). The group-level parameters all come from noncommittal, broad uniform distributions. This model is applied separately to the high-symptom and low-symptom observers.
Whether all uniform distributions assumed an upper bound of 10 or 50, the results were essentially identical. The results reported here are for an upper bound of 10. comparison across groups of attention to body size Figure 13.7 shows the marginal posterior distribution for the group-level parameters of most interest. The left side shows the distribution of the central tendency of attention to body size for each group as well as the distribution of their difference. In particular, the bottom left histogram shows that the low-symptom group has an attention weight on body size about 0.36 lower than the high-symptom
bayesian estimation in hierarchical models
293
mode = 0.526
mode = 0.606
95% HDI
95% HDI
0.335 0.0
0.718
0.2
0.4
0.6
0.0637 0.8
1.0
0.851
0.0
0.5
1.0
μlow w mode = 0.919
0.79 0.0
0.2
0.4
0.6
mode = 0.573
95% HDI 0.976
0.8
95% HDI 0.432 0.685 0.0
1.0
0.5
μhigh w
55.1% < 0 < 44.9%
95% HDI
95% HDI −0.15
−0.6
−0.4 μlow w
−
1.5
mode = 0.0347
99.9% < 0 < 0.1%
−0.8
1.0 mochigh
mode = −0.36
−0.583
1.5
molow c
−0.2
−0.505 0.0
0.325
−0.5
0.0
0.5
molow − mohigh c c
μhigh w
1.0
Fig. 13.7 Marginal posterior distribution of group-level parameters for the prototype classification task. The left column shows the [g]
group-level central tendency of the attention weight on body size, μw . The bottom-left histogram reveals a credibly nonzero difference between groups, with low-symptom observers allocating about 0.36 less attention to body-size than high-symptom observers. The 95% HDI is so far away from a difference of zero that any reasonable ROPE would be excluded; therefore, we do not specify a particular [g]
ROPE. The right column shows the group-level central tendency of the sensitivity parameter, moc . The bottom-right histogram shows that zero difference is squarely among the most credible differences.
group, and this difference is credibly nonzero. The right side shows that the most credible difference of sensitivities is near zero. The conclusions from our hierarchical Bayesian estimation agree with those of Viken et al. (2002), who took a non-Bayesian, nonhierarchical, modelcomparison approach. We also find that highsymptom women, relative to low-symptom women, show enhanced attention to body size and decreased attention to facial affect, but no differences in their sensitivities. However, our hierarchical Bayesian estimation approach has provided explicit distributions on the credible differences between the groups. 294
new directions
comparisons across individual women’s attention to body size Although the primary question of interest involves the group-level central tendencies, hierarchical Bayesian estimation also automatically provides estimates of the attention weights of individual women. Figure 13.8 shows the estimates of individual attention weights wi for three women, based on the hierarchical Bayesian estimation that shares information across all observers to inform the estimate of each individual observer. Figure 13.8 also shows the individual estimates from a nonhierarchical MLE, which derives each individual estimate from the data of a single observer only.
mode = 1
mode = 0.999
95% HDI 0.821 1 0.0
0.2
0.4 0.6 w33
0.8
1.0
95% HDI 0.837 1 0.0
0.2
0.4 0.6 w16
0.8
1.0
mode = 0.00012
95% HDI 1.76e−53 0.172 0.0
0.2
0.4 0.6 w12
0.8
1.0
Fig. 13.8 Posterior of attention weights wi of three individual observers. The vertical mark on the HDI indicates the MLE of the attention weight based on the individual’s data only. Observer 33 is a high-symptom woman, whose estimate is shrunk upward (toward one). Observers 16 and 12 are both low-symptom women, whose estimates are shrunken in different directions (upwards for 16, downwards for 12).
Figure 13.8 illustrates that in hierarchical models, data from one individual influence inferences about the parameters of the other individuals. Technically, this happens because each woman’s data influence the group-level parameters, which affect all the individual-level parameter estimates. For example, the hierarchical Bayesian modal estimate of the attention weight for observer 33, a high-symptom woman, is 1, which is larger than the nonhierarchical MLE of 0.89. This shrinkage in the hierarchical estimate is caused by the fact that most other high-symptom women tend to have relatively high attention weights, thereby pulling up the group-level estimate of the attention weight and the estimates for each individual high-symptom woman. Shrinkage also occurs for the low-symptom woman, shown in the other panels of Figure 13.8. The second panel shows that for observer 16, the hierarchical Bayesian modal estimate of the attention weight is 1, which is higher than the nonhierarchical estimate of 0.93. For observer 12, however, shrinkage is in the opposite direction: the hierarchical Bayesian modal estimate of the attention weight is smaller than the MLE based on individual data (0 vs 0.07). These opposite directions in shrinkage of the estimate are caused by the fact that the overarching beta distribution for low-symptom women is bimodal (i.e., have shape parameters less than 1.0), with one mode near 0 and a second mode near 1, indicating that low-symptom women tend to have either a low attention weight or a high attention weight. This bimodality is evident in the data and is not merely an artifact of the model, insofar as many women classify as if paying most attention to either one dimension or the other. Woman with MLE’s close to 0 have hierarchical Bayesian estimates even closer to 0, whereas woman with MLE’s close to 1 have hierarchical Bayesian
estimates even closer to 1. Shrinkage for the lowsymptom women thus highlights that shrinkage is not necessarily inward, toward the middle of the higher-level distribution; it can also be outward, and always toward the modes.
Model Comparison as a Case of Estimation in Hierarchical Models In the examples discussed earlier, Bayesian estimation was the reallocation of credibility across the space of parameter values, for continuous parameters. We can think of each distinct parameter value (or joint combination of values in a multiparameter space) as a distinct model of the data. Because the parameter values are on a continuum, there is a continuum of models. Under this conceptualization, Bayesian parameter estimation is model comparison for an infinity of models. Often, however, people may think of different models as being distinct, discrete descriptions, not on a continuum. This conceptualization of models makes little difference from a Bayesian perspective. When models are discrete, there is still a parameter that relates them to each other, namely an indexical parameter that has value 1 for the first model, 2 for the second model, and so on. Bayesian model comparison is then Bayesian estimation, as the reallocation of credibility across the values of the indexical parameter. The posterior probabilities of the models are simply the posterior probabilities of the indexical parameter values. Bayesian inference operates, mathematically, the same way regardless of whether the parameter that relates models is continuous or discrete. Figure 13.9 shows a hierarchical diagram for comparing two models. At the bottom of the
bayesian estimation in hierarchical models
295
Pm categorical ...m
p(θ|m =1)
p(ϕ|m =2)
p(D|θ, m =1)
p(D|ϕ, m =2)
D Fig. 13.9 Model comparison as hierarchical modeling. Each dashed box encloses a discrete model of the data, and the models depend on a higher-level indexical parameter at the top of the diagram. See text for further details.
diagram are the data, D. Scanning up the diagram, the data are distributed according to likelihood function p(D|θ, m = 1) when the model index m is 1. The likelihood function for model 1 involves a parameter θ, which has a prior distribution specified by p(θ |m=1). All the terms involving the parameter θ are enclosed in a dashed box, which indicates the part of the overall hierarchy that depends on the higher-level indexical parameter, m, having value m = 1. Notice, in particular, that the prior on the parameter θ is an essential component of the model; that is, the model is not only the likelihood function but also the prior. When m = 2, the data are distributed according to the model on the right of the diagram, involving a likelihood function and prior with parameter φ. At the top of the hierarchy is a categorical distribution that specifies the prior probability of each indexical value of m, that is, the prior probability of each model as a discrete entity. This hierarchical diagram is analogous to previous hierarchical diagrams in Figures 13.1 and Figure 13.6, but the top-level distribution is discrete and lower-level parameters and structures can change discretely instead of continuously when the top-level parameter value changes. The sort of hierarchical structure diagrammed in Figure 13.9 can be implemented in the same MCMC sampling software we used for the baseball and categorization examples earlier. The MCMC algorithm generates representative values of the 296
new directions
indexical parameter m, together with representative values of the parameter θ (when m = 1) and the parameter φ (when m = 2). The posterior probability of each model is approximated accurately by the proportion of steps that the MCMC chain visited each value of m. For a hands-on introduction to MCMC methods for Bayesian model comparison, see Chapter 10 of Kruschke (2015) and Lodewyckx et al. (2011). Examples of Bayesian model comparison are also provided by Vandekerckhove, Matzke, and Wagenmakers in chapter 14, this volume. When comparing models, it is crucially important to set appropriately the prior distributions within each model, because the estimation of the model index can be very sensitive to the choice of prior. In the context of Figure 13.9, we mean that it is crucial to set the prior distributions, p(θ|m = 1) and p(φ|m = 2), so that they accurately express the priors intended for each model. Otherwise it is trivially easy to favor one model over the other, perhaps inadvertently, by setting one prior to values that accommodate the data well while setting the other prior to values that do not accommodate the data well. If each model comes with a theory or previous research that specifically informs the model, then that theory or research should be used to set the prior for the model. Otherwise, the use of generic default priors can unwittingly favor one model over the other. When there are not strong theories or previous research to set the priors for each model, a useful approach for setting priors is as follows: Start each model with vague default priors. Then, using some modest amount of data that represent consensually accepted previous findings, update all models with those data. The resulting posterior distributions in each model are then used as the priors for the model comparison, using the new data. The priors, by being modestly informed, have mitigated the arbitrary influence of inappropriate default priors, and have set the models on more equal playing fields by being informed by the same prior data. These and other issues are discussed in the context of cognitive model comparison by Vanpaemel (2010) and Vanpaemel and Lee (2012). A specific case of the hierarchical structure in Figure 13.9 occurs when the two models have the same likelihood function, and hence the same parameters, but different prior distributions. In this case, the model comparison is really a comparison of two competing choices of prior distribution for the parameters. A common application for this specific
case is null hypothesis testing. The null hypothesis is expressed as a prior distribution with all its mass at a single value of the parameter, namely the “null” value, such as θ = 0. If drawn graphically, the prior distribution would look like a spike-shaped distribution. The alternative hypothesis is expressed as a prior distribution that spreads credibility over a broad range of the parameter space. If drawn graphically, the alternative prior might resemble a thin (i.e., short) slab-shaped distribution. Model comparison then amounts to the posterior probabilities of the spike-shaped (null) prior and the slab-shaped (alternative) prior. This approach to null-hypothesis assessment depends crucially on the meaningfulness of the chosen alternative-hypothesis prior, because the posterior probability of the null-hypothesis prior is not absolute but merely relative to the chosen alternative-hypothesis prior. The relative probability of the null-hypothesis prior can change dramatically for different choices of the alternative-hypothesis prior. Because of this sensitivity to the alternative-hypothesis prior, we recommend that this approach to null-hypothesis assessment is used only with caution when it is clearly meaningful to entertain the possibility that the null value could be true and a justifiable alternative-hypothesis prior is available. In such cases, the prior-comparison approach can be very useful. However, in the absence of such meaningful priors, null-value assessment most safely proceeds by explicit estimation of parameter values within a single model, with decisions about null values made according to the HDI and ROPE as exemplified earlier. For discussion of Bayesian approaches to null-value assessment, see, for example, Kruschke (2011), Kruschke (2013, Appendix D), Morey and Rouder (2011), Wagenmakers (2007), and Wetzels, Raaijmakers, Jakab, and Wagenmakers (2009).
Conclusion In this chapter we discussed two examples of hierarchical Bayesian estimation. The baseball example (Figure 13.1) illustrated multiple levels with shrinkage of estimates within each level. We chose this example because it clearly illustrates the effects of hierarchical structure in rational inference of individual and group parameters. The categorization example (Figure 13.6) illustrated the use of hierarchical Bayesian estimation for psychometric assessment via a cognitive model. The parameters are meaningful in the context of the cognitive
model, and Bayesian estimation provides a complete posterior distribution of credible parameter values for individuals and groups. Other examples of hierarchical Bayesian estimation can be found, for instance, in articles by Bartlema, Lee, Wetzels, and Vanpaemel (2014), Lee (2011), Rouder and Lu (2005), Rouder, Lu, Speckman, Sun, and Jiang (2005), and Shiffrin, Lee, Kim, and Wagcnmakers (2008). The hierarchical Bayesian method is very attractive because it allows the analyst to define meaningfully structured models that are appropriate for the data. For example, there is no artificial dilemma of deciding between doing separate individual analyses or collapsing across all individuals, which both have serious shortcomings (Cohen, Sanborn, & Shiffrin: 2008). When collapsing the data across participants in each group, it is implicitly assumed that all participants within a group behave identically. Such an assumption is often untenable. The other extreme of analyzing every individual separately with no pooling across individuals can be highly error prone, especially when each participant contributed only small amounts of data. A hierarchical analysis provides a middle ground between these two strategies, by acknowledging that people are different, without ignoring the fact that they represent a common group or condition. The hierarchical structure allows information provided by one participant to flow rationally to the estimates of other participants. This sharing of information across participants via hierarchical structure occurs in both the classification and baseball examples of this chapter. A second key attraction of hierarchical Bayesian estimation is that software for expressing complex, nonlinear hierarchical models (e.g., Lunn et al. 2000, Plummer 2003, Stan Development Team 2012), produces a complete posterior distribution for direct inferences about credible parameter values without need for p values or corrections for multiple comparisons. The combination of ease of defining specifically appropriate models and ease of direct inference from the posterior distribution makes hierarchical Bayesian estimation an extremely useful approach to modeling and data analysis.
Acknowledgments The authors gratefully acknowledge Rick Viken and Teresa Treat for providing data from Viken, Treat, Nosofsky, McFall, and Palmeri (2002). Appreciation is also extended to E.-J.
bayesian estimation in hierarchical models
297
Wagenmakers and two anonymous reviewers who provided helpful comments that improved the presentation. Correspondence can be addressed to John K. Kruschke, Department of Psychological and Brain Sciences, Indiana University, 1101 E. 10th St., Bloomington IN 47405-7007, or via electronic mail to [email protected]. Supplementary information about Bayesian data analysis can be found at http://www.indiana.edu/∼kruschke/
Notes 1. The most general definition of a confidence interval is the range of parameter values that would not be rejected according to a criterion p value, such as p < 0.05. These limits depend on the arbitrary settings of other parameters, and can be difficult to compute. 2. Data retrieved December 22, 2012 from http://www.baseball-reference.com/leagues/MLB/2012standard-batting.shtml 3. This analysis was summarized at http://doingbayesiandataanalysis.blogspot.com/2012/11/shrinkage-in-multi-level-hierarchical.html 4. In the context of a normal distribution, instead of a beta distribution, the “precision” is the reciprocal of variance. Intuitively, it refers to the narrowness of the distribution for either the normal or beta distributions.
Glossary Hierarchical model: A formal model that can be expressed such that one parameter is dependent on another parameter. Many models can be meaningfully factored this way, for example when there are parameters that describe data from individuals, and the individual-level parameters depend on group-level parameters. Highest density interval (HDI): The highest density interval summarizes the interval under a probability distribution where the probability densities inside the interval are higher than probability densities outside the interval. A 95% HDI includes the 95% of the distribution with the highest probability density. Markov chain Monte Carlo (MCMC): A class of stochastic algorithms for obtaining samples from a probability distribution. The algorithms take a random walk through parameter space, favoring values that have higher probability. With a sufficient number of steps, the values of the parameter are visited in proportion to their probabilities and therefore the samples can be used to approximate the distribution. Widely used examples of MCMC are the Gibbs sampler and the Metropolis-Hastings algorithm. Posterior distribution: A probability distribution over parameters derived via Bayes’ rule from the prior distribution by taking into account the targeted data. Prior distribution: A probability distribution over parameters representing the beliefs, knowledge or assumptions about the parameters without reference to the targeted data. The prior distribution and the likelihood function together define a model.
298
new directions
Region of practical equivalence (ROPE): An interval around a parameter value that is considered to be equivalent to that value for practical purposes. The ROPE is used as part of a decision rule for accepting or rejecting particular parameter values.
References Bartlema, A., Lee, M D., Wetzels, R., & Vanpaemel, W. (2014). A Bayesian hierarchical mixture approach to individual differences: Case studies in selective attention and representation in category learning. Journal of Mathematical Psychology, 59, 132–150. Batchelder, W H. (1998). Multinomial processing tree models and psychological assessment. Psychological Assessment, 10, 331–344. Bayes, T., & Price, R. (1763). An essay towards solving a problem in the doctrine of chances. By the Late Rev. Mr. Bayes, F.R.S. Communicated by Mr. Price, in a Letter to John Canton, A.M.F.R.S. Philosophical Transactions, 53, 370–418. doi: 10.1098/rstl.1763.0053 Carlin, B P., & Louis, T. A. (2009). Bayesian methods for data analysis (3rd ed.). Boca Raton, FL: CRC Press. Cohen, A. L., Sanborn, A. N., & Shiffrin, R, M. (2008). Model evaluation using grouped or individual data. Psychonomic Bulletin & Review, 15, 692–712. Denwood, M. J. (2013). runjags: An R package providing interface utilities, parallel computing methods and additional distributions for MCMC models in JAGS. Journal of Statistical Software, (in review). http://cran.r-project.org/web/ bpackages/runjags/ Doyle, A. C. (1890). The sign of four. London, England: Spencer Blackett. Freedman, L. S., Lowe, D., & Macaskill, P. (1984). Stopping rules for clinical trials incorporating clinical opinion. Biometrics, 40, 575–586. Hobbs, B. P., & Carlin, B. P. (2008). Practical Bayesian design and analysis for drug and device clinical trials. Journal of Biopharmaceutical Statistics, 18(1), 54–80. Kruschke, J. K. (2008). Models of categorization. R. Sun (Ed.), The Cambridge Handbook of Computational Psychology (p. 267–301). New York, NY: Cambridge University Press. Kruschke, J. K. (2011). Bayesian assessment of null values via parameter estimation and model comparison. Perspectives on Psychological Science, 6(3) 299–312. Kruschke, J. K. (2013). Bayesian estimation supersedes the t test. Journal of Experimental Psychology: General, 142(2), 573–603. doi: 10.1037/a0029146 Kruschke, J. K. (2015). Doing Bayesian data analysis, Second edition: A tutorial with R, JAGS, and Stan. Waltham, Academic Press/Elsevier. Lee, M. D. (2011). How cognitive modeling can benefit from hierarchical Bayesian models. Journal of Mathematical Psychology, 55, 1–7. Lodewyckx, T., Kim, W., Lee, M. D., Tuerlinckx, F., Kuppens, P., & Wagenmakers, E. J. (2011). A tutorial on Bayes factor estimation with the product space method. Journal of Mathematical Psychology, 55(5), 331–347.
Lunn, D., Jackson, C., Best, N., Thomas, A., & Spiegelhalter, D. (2013). The BUGS book: A practical introduction to Bayesian analysis. Boca Raton, FL: CRC Press. Lunn, D., Thomas, A., Best, N., & Spiegelhalter, D. (2000). WinBUGS — A Bayesian modelling framework: Concepts, structure, and extensibility. Statistics and Computing, 10(4), 325–337. Morey, R. D., & Rouder, J. N. (2011). Bayes factor approaches for testing interval null hypotheses. Psychological Methods, 16(4), 406–419. Nosofsky, R. M. (1987). Attention and learning processes in the identification and categorization of integral stimuli. Journal of Experimental Psychology: Learning, Memory and Cognition, 13, 87–108. Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. Proceedings of the 3rd International Workshop on Distributed Statistical Computing (DSC 2003). Vienna, Austria. Reed, S. K. (1972). Pattern recognition and categorization. Cognitive Psychology, 3, 382–407. Riefer, D. M., Knapp, B. R., Batchelder, W. H., Bamber, D., & Manifold, V. (2002). Cognitive psychometrics: Assessing storage and retrieval deficits in special populations with multinomial processing tree models. Psychological Assessment, 14, 184–201. Rouder, J. N., & Lu, J. (2005). An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic Bulletin & Review, 12(4), 573–604. Rouder, J. N., Lu, J., Speckman, P., Sun, D., & Jiang, Y. (2005). A hierarchical model for estimating response time distributions. Psychonomic Bulletin & Review, 12(2), 195–223. Serlin, R. C.,& Lapsley, D. K. (1985). Rationality in psychological research: The good-enough principle. American Psychologist, 40(1), 73–83. Serlin, R. C., & Lapsley, D. K. (1993). Rational appraisal of psychological research and the good-enough principle. G. Keren, & C. Lewis (Eds.) A handbook for data analysis
in the behavioral sciences: Methodological issues (pp. 199– 228). Hillsdale, NJ: Erlbaum. Shiffrin, R. M., Lee, M. D., Kim, W., & Wagenmakers, E. J. (2008). A survey of model evaluation approaches with a tutorial on hierarchical Bayesian methods. Cognitive Science, 32(8), 1248–1284. Smith, M. C., & Thelen, M. H. (1984). Development and validation of a test for bulimia. Journal of Consulting and Clinical Psychology, 52, 863–872. Spiegelhalter, D. J., Freedman, L. S., & Parmar, M. K. B. (1994). Bayesian approaches to randomized trials. Journal of the Royal Statistical Society. Series A, 157, 357–416. Stan Development Team. (2012). Stan: A C++ library for probability and sampling, version 1.1. Retrieved from http://mc-stan.org/citations.html Vanpaemel, W. (2009). BayesGCM: Software for Bayesian inference with the generalized context model. Behavior Research Methods, 41(4), 1111–1120. Vanpaemel, W. (2010). Prior sensitivity in theory testing: An apologia for the Bayes factor. Journal of Mathematical Psychology, 54, 491–498. Vanpaemel, W., & Lee, M. D. (2012). Using priors to formalize theory: Optimal attention and the generalized context model. Psychonomic Bulletin & Review, 19, 1047–1056. Viken, R. J., Treat, T. A., Nosofsky, R. M., McFall, R. M., & Palmeri, T, J. (2002). Modeling individual differences in perceptual and attentional processes related to bulimic symptoms. Journal of Abnormal Psychology, 111, 598–609. Wagenmakers, E. J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14(5), 779–804. Wetzels, R., Raaijmakers, J. G. W., Jakab, E., & Wagenmakers, E. J. (2009). How to quantify support for and against the null hypothesis: A flexible WinBUGS implementation of a default Bayesian t test. Psychonomic Bulletin & Review, 16(4), 752–760.
bayesian estimation in hierarchical models
299
CHAPTER
14
Model Comparison and the Principle of Parsimony
Joachim Vandekerckhove, Dora Matzke, and Eric-Jan Wagenmakers
Abstract
According to the principle of parsimony, model selection methods should value both descriptive accuracy and simplicity. Here we focus primarily on Bayes factors and minimum description length, explaining how these procedures strike a balance between goodness-of-fit and parsimony. Throughout, we demonstrate the methods with an application on false memory, evaluating three competing multimonial proces tree models of interference in memory. Key Words: model selection, goodness of fit, parsimony, inference, Akaike’s information
criterion (AIC), Bayesian information criterion (BIC), minimum description length (MDL), Bayes factor (BF), Akaike weights, Jeffreys weights, Rissanen weights, Schwartz weights
Introduction At its core, the study of psychology is concerned with the discovery of plausible explanations for human behavior. For instance, one may observe that “practice makes perfect”: as people become more familiar with a task, they tend to execute it more quickly and with fewer errors. More interesting is the observation that practice tends to improve performance such that most of the benefit is accrued early on, a pattern of diminishing returns that is well described by a power law (Logan, 1988; but see Heathcote, Brown, & Mewhort, 2000). This pattern occurs across so many different tasks (e.g., cigar rolling, maze solving, fact retrieval, and a variety of standard psychological tasks) that it is known as the “power law of practice.” Consider, for instance, the lexical decision task, a task in which participants have to decide quickly whether a letter string is an existing word (e.g., sunscreen) or not (e.g., tolphin). When repeatedly presented with the same stimuli,1 participants show a power law decrease in their mean response latencies; in fact, they show a power law decrease in the entire response time distribution, that is, both the fast
300
responses and the slow responses speed up with practice according to a power law (Logan, 1992). The observation that practice makes perfect is trivial, but the finding that practice-induced improvement follows a general law is not. Nevertheless, the power law of practice only provides a descriptive summary of the data and does not explain the reasons that practice should result in a power law improvement in performance. In order to go beyond direct observation and statistical summary, it is necessary to bridge the divide between observed performance on the one hand and the pertinent psychological processes on the other. Such bridges are built from a coherent set of assumptions about the underlying cognitive processes—a theory. Ideally, substantive psychological theories are formalized as quantitative models (Busemeyer & Diederich, 2010; Lewandowsky & Farrell, 2010). For example, the power law of practice has been explained by instance theory (Logan, 1992, 2002). Instance theory stipulates that earlier experiences are stored in memory as individual traces or instances; upon presentation of a stimulus, these instances race to be retrieved, and the winner of the race initiates
a response. Mathematical analysis shows that, as instances are added to memory, the finishing time of the winning instance decreases as a power function. Hence, instance theory provides a simple and general explanation of the power law of practice. For all its elegance and generality, instance theory has not been the last word on the power law of practice. The main reason is that single phenomena often afford different competing explanations. For example, the effects of practice can also be accounted for by Rickard’s component power laws model (Rickard, 1997), Anderson’s ACTR model (Anderson, 2004), Cohen et al.’s PDP model (Cohen, Dunbar, & McClelland, 1990), Ratcliff ’s diffusion model (Dutilh, Vandekerckhove, Tuerlinckx, & Wagenmakers, 2009; Ratcliff, 1978), or Brown and Heathcote’s linear ballistic accumulator model (Brown & Heathcote, 2005, 2008; Heathcote & Hayes, 2012). When various models provide competing accounts of the same data set, it can be difficult to choose between them. The process of choosing between models is called model comparison, model selection, or hypothesis testing, and it is the focus of this chapter. A careful model comparison procedure includes both qualitative and quantitative elements. Important qualitative elements include the plausibility, parsimony, and coherence of the underlying assumptions, the consistency with known behavioral phenomena, the ability to explain rather than describe data, and the extent to which model predictions can be falsified through experiments. Here we ignore these important aspects and focus solely on the quantitative elements. The single most important quantitative element of model comparison relates to the ubiquitous trade-off between parsimony and goodness-of-fit (Pitt & Myung, 2002). The motivating insight is that the appeal of an excellent fit to the data (i.e., high descriptive adequacy) needs to be tempered to the extent that the fit was achieved with a highly complex and powerful model (i.e., low parsimony). The topic of quantitative model comparison is as important as it is challenging; fortunately, the topic has received—and continues to receive— considerable attention in the field of statistics, and the results of those efforts have been made accessible to psychologists through a series of recent special issues, books, and articles (e.g., Grünwald, 2007; Myung et al., 2000; Pitt & Myung, 2002; Wagenmakers & Waldorp, 2006). Here we discuss several procedures for model comparison, with an emphasis on minimum description length and
the Bayes factor. Both procedures entail principled and general solutions to the trade-off between parsimony and goodness of fit. The outline of this chapter is as follows. The first section describes the principle of parsimony and the unavoidable trade-off with goodness-offit. The second section summarizes the research of Wagenaar and Boer (1987) who carried out an experiment to compare three competing multinomial processing tree models (MPTs; Batchelder & Riefer, 1980); this model comparison exercise is used as a running example throughout the chapter. The third section outlines different methods for model comparison and applies them to Wagenaar and Boer (1987)’s MPT models. We focus on two popular information criteria, the AIC and the BIC, on the Fisher information approximation of the minimum description length principle, and on Bayes factors as obtained from importance sampling. The fourth section contains conclusions and take-home messages.
The Principle of Parsimony Throughout history, prominent philosophers and scientists have stressed the importance of parsimony. For instance, in the Almagest— a famous second-century book on astronomy— Ptolemy writes: “We consider it a good principle to explain the phenomena by the simplest hypotheses that can be established, provided this does not contradict the data in an important way.” Ptolemy’s principle of parsimony is widely known as Occam’s razor (see Box 2); the principle is intuitive as it puts a premium on elegance. In addition, most people feel naturally attracted to models and explanations that are easy to understand and communicate. Moreover, the principle also gives ground to reject propositions that are without empirical support, including extrasensory perception, alien abductions, or mysticism. In an apocryphal interaction, Napoleon Bonaparte asked Pierre-Simon Laplace why the latter’s book on the universe did not mention its creator, only to receive the curt reply “I had no need of that hypothesis.” However, the principle of parsimony finds its main motivation in the benefits that it bestows on those who use models for prediction. To see this, note that empirical data are assumed to be composed of a structural, replicable part and an idiosyncratic, nonreplicable part. The former is known as the signal, and the latter is known as the noise (Silver, 2012). Models that capture all
model comparison and the principle of parsimony
301
the signal and none of the noise provide the best possible predictions to unseen data from the same source. Overly simplistic models, however, fail to capture part of the signal; these models underfit the data and provide poor predictions. Overly complex models, on the other hand, mistake some of the noise for actual signal; these models overfit the data and again provide poor predictions. Thus, parsimony is essential because it helps discriminate the signal from the noise, allowing better prediction and generalization to new data.
Box 1 Occam’s razor Occam’s razor (sometimes Ockham’s) is named after the English philosopher and Franciscan friar Father William of Occam (c.1288– c.1348), who wrote “Numquam ponenda est pluralitas sine necessitate” (plurality must never be posited without necessity), and “Frustra fit per plura quod potest fieri per pauciora” (it is futile to do with more what can be done with less). Occam’s metaphorical razor symbolizes the principle of parsimony: by cutting away needless complexity, the razor leaves only theories, models, and hypotheses that are as simple as possible without being false. Throughout the centuries, many other scholars have espoused the principle of parsimony; the list predating Occam includes Aristotle, Ptolemy, and Thomas Aquinas (“it is superfluous to suppose that what can be accounted for by a few principles has been produced by many”), and the list following Occam includes Isaac Newton (“We are to admit no more causes of natural things than such as are both true and sufficient to explain their appearances. Therefore, to the same natural effects we must, so far as possible, assign the same causes.”), Bertrand Russell, Albert Einstein (“Everything should be made as simple as possible, but no simpler”), and many others. In the field of statistical reasoning and inference, Occam’s razor forms the foundation for the principle of minimum description length (Grunwald, 2000, 2007). In addition, Occam’s razor is automatically accommodated through Bayes factor model comparisons (e.g., Jefferys & Berger, 1992; Jeffreys, 1961; MacKay, 2003). Both minimum description length and Bayes factors feature prominently in this chapter as principled methods to quantify the trade-off between parsimony and goodness of
302
new directions
fit. Note that parsimony plays a role even in classical null-hypothesis significance testing, where the simpler null hypothesis is retained unless the data provide sufficient grounds for its rejection.
Goodness of fit “From the earliest days of statistics, statisticians have begun their analysis by proposing a distribution for their observations and then, perhaps with somewhat less enthusiasm, have checked on whether this distribution is true. Thus over the years a vast number of test procedures have appeared, and the study of these procedures has come to be known as goodness-of-fit” (D’Agostino & Stephens, 1986, p. v). The goodness of fit of a model is a quantity that expresses how well the model is able to account for a given set of observations. It addresses the following question: Under the assumption that a certain model is a true characterization of the population from which we have obtained a sample, and given the best fitting parameter estimates for that model, how well does our sample of data agree with that model? Various ways of quantifying goodness of fit exist. One common expression involves a Euclidean distance metric between the data and the model’s best prediction (the least squared error or LSE metric is the most well-known of these). Another measure involves the likelihood function, which expresses the likelihood of observing the data under the model, and is maximized by the best fitting parameter estimates (Myung, 2000).
Parsimony Goodness of fit must be balanced against model complexity in order to avoid overfitting—that is, to avoid building models that well explain the data at hand, but fail in out-of-sample predictions. The principle of parsimony forces researchers to abandon complex models that are tweaked to the observed data in favor of simpler models that can generalize to new data sets. A common example is that of polynomial regression. Figure 14.1 gives a typical example. The observed data are the circles in both the left and right panels. Crosses indicate unobserved, out-of-sample data points to which the model should generalize. In the left panel, a quadratic function is fit to the 8 observed data points, whereas
(a)
1500
(b) 1500 hidden data visible data 2nd degree polynomial
hidden data visible data 7th degree polynomial 1000
500
500
0
0
criterion
1000
0
10
20
30 40 predictor
50
60
0
10
20
30 predictor
40
50
60
Fig. 14.1 A polynomial regression of degree d is characterized by yˆ = di=0 ai x i . This model has d + 1 free parameters ai ; hence, in the right panel, a polynomial of degree 7 perfectly accounts for the 8 visible data points. This 7th-order polynomial, however, accounts poorly for the out-of-sample data points.
the right panel shows a 7th-order polynomial function fitted to the same data. Since a polynomial of degree 7 can be made to contain any 8 points in the plane, the observed data are perfectly captured by the best-fitting polynomial. However, it is clear that this function generalizes poorly to the unobserved samples, and it shows undesirable behavior for larger values of x. In sum, an adequate model comparison method needs to discount goodness of fit with model complexity. But how exactly can this be accomplished? As we will describe shortly, several model comparison methods are currently in vogue; all resulting from principled ideas on how to obtain measures of generalizability,2 meaning that these methods attempt to quantify the extent to which a model predicts unseen data from the same source (cf. Figure 14.1). Before outlining the details of various model-comparison methods, we now introduce a data set that serves as a working example throughout the remainder of the chapter.
Example: Competing Models of Interference in Memory For an example model comparison scenario, we revisit a study by Wagenaar and Boer (1987) on the effect of misleading information on the recollection of an earlier event. The effect of misleading postevent information was first studied systematically by Loftus, Miller, and Burns (1978); for a review of relevant literature see Wagenaar and Boer (1987) and references therein.
Wagenaar and Boer (1987) proposed three competing theoretical accounts of the effect of misleading postevent information. To evaluate the three accounts, Wagenaar and Boer set up an experiment and introduced three quantitative models that translate each of the theoretical accounts into a set of parametric assumptions that together give rise to a probability density over the data, given the parameters.
Abstract Accounts Wagenaar and Boer (1987) outlined three competing theoretical accounts of the effect of misleading postevent information on memory. Loftus’s destructive updating model (DUM) posits that the conflicting information replaces and destroys the original memory. A coexistence model (CXM) asserts that an inhibition mechanism suppresses the original memory, which nonetheless remains viable though temporarily inaccessible. Finally, a noconflict model (NCM) simply states that misleading postevent information is ignored, except when the original information was not encoded or already forgotten.
Experimental Design The experiment by Wagenaar and Boer (1987) proceeded as follows. In Phase I, a total of 562 participants were shown a sequence of events in the form of a pictorial story involving a pedestrian-car collision. One picture in the story would show a car
model comparison and the principle of parsimony
303
(a)
(b)
Fig. 14.2 A pair of pictures from the third phase (i.e., the recognition test) of Wagenaar and Boer (1987), reprinted with permission, containing the critical episode at the intersection. c After Wagenaar and Boer (1987), ActaPsychologica Elsevier Inc.
at an intersection, and a traffic light that was either red, yellow, or green. In Phase II, participants were asked a set of test questions with (potentially) conflicting information: Participants might be asked whether they remembered a pedestrian crossing the road when the car approached the “traffic light” (in the consistent group), the “stop sign” (in the inconsistent group) or the “intersection” (the neutral group). Then, in Phase III, participants were given a recognition test about elements of the story using picture pairs. Each pair would contain one picture from Phase I and one slightly altered version of the original picture. Participants were then asked to identify which of the pair had featured in the original story. A picture pair is shown in Figure 14.2, where the intersection is depicted with either a traffic light or a stop sign. Finally, in Phase IV, participants were informed that the correct choice in Phase III was the picture with the traffic light, and were then asked to recall the color of the traffic light. By design, this experiment should yield different response patterns depending on whether the conflicting postevent information destroys the original information (destructive-updating model), only suppresses it temporarily (coexistence model), or does not affect the original information unless it is unavailable (no-conflict model).
Concrete Models Wagenaar and Boer (1987) developed a series of MPT models (see Box 3) to quantify the predictions of the three competing theoretical accounts. Figure 14.3 depicts the no-conflict MPT model in the inconsistent condition. The figure is essentially a decision tree that is navigated from left to right. In Phase I of the collision narrative, the traffic 304
new directions
light is encoded with probability p, and if so, the color is encoded with probability c. In Phase II, the stop sign is encoded with probability q. In Phase III, the answer may be known or may be guessed correctly with probability 1/2, and in Phase IV the answer may be known or may be guessed correctly with probability 1/3. The probability of each path is given by the product of all the encountered probabilities, and the total probability of a response pattern is the summed probability of all branches that lead to it. For example, the total probability of getting both questions wrong is (1 − p) × q × 2/3 + (1−p)×(1−q)×1/2×2/3. We would then, under the no-conflict model, expect that proportion of participants to fall in the response pattern with two errors. The destructive updating model (Figure 2 in Wagenaar & Boer, 1987) extends the three-parameter no-conflict model by adding a fourth parameter d : the probability of destroying the traffic-light information, which may occur whenever the stop sign was encoded. The coexistence model (Figure 3 in Wagenaar & Boer, 1987), on the other hand, posits an extra probability s that the traffic light is suppressed (but not destroyed) when the stop sign is encoded. A critical difference between the latter two is that a destruction step will lead to chance accuracy in Phase IV if every piece of information was encoded, whereas a suppression step will not affect the underlying memory and lead to accurate responding. Note here that, if s = 0, the coexistence model reduces to the no-conflict model, as does the destructive-updating model with d = 0. The models only make different predictions in the inconsistent condition, so that, for the consistent and neutral conditions, the trees are identical.
Phase III correct
Phase IV correct
q
c
1–q p
⅓ q ⅔
yes
1–c ⅓ 1–q ⅔
Color encoded?
⅓
no
q ⅔ 1–p
½
⅔
1–q Traffic light encoded?
Stop sign encoded?
⅓
⅓
½ ⅔ Traffic light guessed?
Color guessed?
Fig. 14.3 Multinomial-processing tree representation of the inconsistent condition according to the no-conflict model adapted from Wagenaar and Boer (1987). c After Wagenaar and Boer (1987), ActaPsychologica Elsevier Inc.
Box 2 Popularity of multinomial processing tree models Multinomial processing tree models (Batchelder & Riefer, 1980; Chechile, 1973; Chechile & Meyer, 1976; Riefer & Batchelder, 1988) are psychological process models for categorical data. MPT models are used in two ways: as a psychometric tool to measure unobserved cognitive processes, and as a convenient formalization of competing psychological theories. Over time, MPTs have been applied to a wide range of psychological tasks and processes. For instance, MPT models are available for recognition, recall, source monitoring, perception, priming, reasoning, consensus analysis, the process dissociation procedure, implicit attitude measurement, and many other phenomena. For more information about MPTs, we recommend the review articles by Batchelder and Riefer (1999; 2007), and Erdfelder et al. (2009). The latter review article also discusses different software packages that can be used to fit MPT models. Necessarily missing from that
list is the recently developed R package MPTinR (Singmann & Kellen 2013) with which we have good experiences. As will become apparent throughout this chapter, however, our preferred method for fitting MPT models is Bayesian (Chechile & Meyer, 1976; Klauer, 2010; Lee & Wagenmakers, 2013; Matzke, Dolan, Batchelder, & Wagenmakers, in press; Rouder, Lu, Morey, Sun, & Speckman, 2008; Smith & Batchelder, 2010).
Previous Conclusions After fitting the three competing MPT models, Wagenaar and Boer (1987) obtained the parameter point estimates in Table 14.1. Using a χ 2 model fit index, they concluded that “a distinction among the three model families appeared to be impossible in actual practice” (p. 304), after noting that the noconflict model provides “an almost perfect fit” to the data. They propose, then, “to accept the most parsimonious model, which is the no-conflict model.” In the remainder of this chapter, we re-examine
model comparison and the principle of parsimony
305
Table 14.1. Parameter point estimates from Wagenaar and Boer (1987). p
c
q
d
s
No-conflict model (NCM)
0.50
0.57
0.50
na
na
Destructive-updating model (DUM)
0.50
0.57
0.50
0.00
na
Coexistence model (CXM)
0.55
0.55
0.43
na
0.20
this conclusion using various model comparison methods.
Three Methods for Model Comparison Many model comparison methods have been developed, all of them attempts to address the ubiquitous trade-off between parsimony and goodness of fit. Here we focus on three main classes of interrelated methods: (1) AIC and BIC, the most popular information criteria; (2) minimumdescription length; (3) Bayes factors. Below we provide a brief description of each method and then apply it to the model comparison problem that confronted Wagenaar and Boer (1987).
Information Criteria Information criteria are among the most popular methods for model comparison. Their popularity is explained by the simple and transparent manner in which they quantify the trade-off between parsimony and goodness of fit. Consider for instance the oldest information criterion, AIC (“an information criterion”), proposed by Akaike (1973, 1974a): AIC = −2 ln p y | θˆ + 2k. (1) The first term ln p y | θˆ is the log maximum likelihood that quantifies goodness of fit, where y is the data set and θˆ the maximum-likelihood parameter estimate; the second term 2k is a penalty for model complexity, measured by the number of adjustable model parameters k. The AIC estimates the expected information loss incurred when a probability distribution f (associated with the true data generating process) is approximated by a probability distribution g (associated with the model under evaluation). Hence, the model with the lowest AIC is the model with the smallest expected information loss between reality f and model g, where the discrepancy is quantified by the Kullback-Leibler divergence I (f , g), a distance metric between two probability distributions (for 306
new directions
full details, see Burnham & Anderson, 2002). The AIC is unfortunately not consistent: as the number of observations grows infinitely large, AIC is not guaranteed to choose the true data-generating model. In fact, there is cause to believe that the AIC tends to select complex models that overfit the data (O’Hagan & Forster, 2004; for a discussion see Vrieze, 2012). Another information criterion, the BIC (“Bayesian information criterion”) was proposed by Schwarz (1978): BIC = −2 ln p y | θˆ + k ln n. (2) Here, the penalty term is k ln n, where n is the number of observations.3 Hence, the BIC penalty for complexity increases with sample size, outweighing that of AIC as soon as n ≥ 8. The BIC was derived as an approximation of a Bayesian hypothesis test using default parameter priors (the “unit information prior”; see later for more information on Bayesian hypothesis testing, and see (Raftery, 1995), for more information on the BIC). The BIC is consistent: as the number of observations grows infinitely large, BIC is guaranteed to choose the true data-generating model. Nevertheless, there is evidence that, in practical applications, the BIC tends to select simple models that underfit the data (Burnham & Anderson, 2002). Now consider a set of candidate models, Mi , i = 1, ..., m, each with a specific IC (AIC or BIC) value. The model with the smallest IC value should be preferred, but the extent of this preference is not immediately apparent. For better interpretation we can calculate IC model weights (Akaike, 1974; Burnham & Anderson, 2002; Wagenmakers & Farrell, 2004); First, we compute, for each model i, the difference in IC with respect to the IC of the best candidate model: i = ICi − min IC.
(3)
This step is taken to increase numerical stability, but it also serves to emphasize the point that only differences in IC values are relevant. Next we obtain
Subjective intensity
Subjective intensity
k = 0.4 β = 1.0 k = 0.7 β = 0.5 0.5
0
0
1 2 Objective intensity
3
c = 0.1 b = 2.0 c = 0.5 b = 0.5 0.5
0
0
1 2 Objective intensity
3
Fig. 14.4 Two representative instances of Fechner’s law (left) and Steven’s law (right). Although Fechner’s law is restricted to nonlinear functions that level off as stimulus intensity increases, Steven’s law can additionally take shapes with accelerating slopes.
the model weights by transforming back to the likelihood scale and normalizing: exp ( − i /2) . wi = M m=1 exp ( − m /2)
(4)
The resulting AIC and BIC weights are called Akaike weights and Schwarz weights, respectively. These weights not only convey the relative preference among a set of candidate models (i.e., they express a degree to which we should prefer one model from the set as superior) but also provide a method to combine predictions across multiple models using model averaging (Hoeting, Madigan, Raftery, & Volinsky, 1999). Both AIC and BIC rely on an assessment of model complexity that is relatively crude, because it is determined entirely by the number of free parameters but not by the shape of the function through which they make contact with the data. To illustrate the importance of the functional form in which the parameters participate, consider the case of Fechner’s law and Steven’s law of psychophysics. Both of these laws transform objective stimulus intensity to subjective experience through a twoparameter nonlinear function.4 According to Fechner’s law, perceived intensity W (I ) of stimulus I is the result of the logarithmic function k ln (I + β). Steven’s law describes perceived intensity as an exponential function: S (I ) = cI b . Although both laws have the same number of parameters, Steven’s is more complex because it can cover a larger number of data patterns (see Figure 14.4). application to multinomial-processing tree models In order to apply AIC and BIC to the three competing MPTs proposed by Wagenaar and Boer (1987), we first need to compute the maximum log likelihood. Note that the MPT model parameters determine the predicted probabilities for the different response outcome categories (cf. Figure 14.3
and Box 3); these predicted probabilities are deterministic parameters from a multinomial probability density function. Hence, the maximum log likelihood parameter estimates for an MPT model produce multinomial parameters that maximize the probability of the observed data (i.e., the occurrence of the various outcome categories). Several software packages exist that can help find the maximum log likelihood parameter estimates for MPTs (Singmann & Kellen, 2013). With these estimates in hand, we can compute the information criteria described in the previous section. Table 14.2 shows the maximum log likelihood as well as AIC, BIC, and their associated weights (wAIC and wBIC; from Eq. 4). Interpreting wAIC and wBIC as measures of relative preference, we see that the results in Table 14.2 are mostly inconclusive. According to wAIC, the no-conflict model and coexistence model are virtually indistinguishable, though both are preferable to the destructive-updating model. According to wBIC, however, the no-conflict model should be preferred over both the destructiveupdating model and the coexistence model. The extent of this preference is noticeable but not decisive.
Minimum Description Length The minimum description length principle is based on the idea that statistical inference centers around capturing regularity in data; regularity, in turn, can be exploited to compress the data. Hence, the goal is to find the model that compresses the data the most (Grünwald, 2007). This is related to the concept of Kolmogorov complexity—for a sequence of numbers, Kolmogorov complexity is the length of the shortest program that prints that sequence and then halts (Grünwald, 2007). Although Kolmogorov complexity cannot be calculated, a suite of concrete methods are available based on the idea of model selection through
model comparison and the principle of parsimony
307
Table 14.2. AIC and BIC for the Wagenaar & Boer MPT models. log likelihood
k*
AIC
wAIC
BIC
wBIC
No-conflict model (NCM)
−24.41
3
54.82
0.41
67.82
0.86
Destructive-updating model (DUM)
−24.41
4
56.82
0.15
74.15
0.04
Coexistence model (CXM)
−23.35
4
54.70
0.44
72.03
0.10
Note: k is the number of free parameters.
data compression. These methods, most of them developed by Jorma Rissanen, fall under the general heading of minimum description length (MDL; Rissanen, 1978, 1987, 1996, 2001). In psychology, the MDL principle has been applied and promoted primarily by Grünwald (2000), Grünwald, Myung, and Pitt (2005), and Grünwald (2007), as well as Myung, Navarro, and Pitt (2006), Pitt and Myung (2002), and Pitt, Myung, and Zhang (2002). Here we mention three versions of the MDL principle. First, there is the so-called crude twopart code (Grünwald, 2007); here, one sums the description of the model (in bits) and the description of the data encoded with the help of that model (in bits). The penalty for complex models is that they take many bits to describe, increasing the summed code length. Unfortunately, it can be difficult to define the number of bits required to describe a model. Second, there is the Fisher information approximation (FIA; Pitt et al., 2002; Rissanen, 1996): k n FIA = − ln p y | θˆ + ln 2 2π # + ln det [I (θ)] dθ, (5)
where I (θ ) denotes the Fisher information matrix of sample size 1 (Ly, Verhagen, Grasman, & Wagenmakers, 2014). I (θ) is a k × k matrix whose (i, j)th element is . ∂ ln p y | θ ∂ ln p y | θ , Ii,j (θ ) = E ∂θi ∂θj where E() is the expectation operator. Note that FIA is similar to AIC and BIC in that it includes a first term that represents goodness of fit, and additional terms that represent a penalty for complexity. The second term resembles that of BIC, and the third term reflects a more sophisticated penalty that represents the number of distinguishable probability distributions that a model can generate (Pitt et al., 2002). Hence, FIA differs from AIC and BIC in that it also accounts for functional 308
new directions
form complexity, not just complexity due to the number of free parameters. Note that FIA weights (or Rissanen weights) can be obtained by multiplying FIA by 2 and then applying Eqs. 3 and 4. The third version of the MDL principle discussed here is normalized maximum likelihood (NML; Myung et al., 2006; Rissanen, 2001): ˆ p y | θ(y) . NML = (6) ˆ (x) dx p x | θ X This equation shows that NML tempers the enthusiasm about a good fit to the observed data y (i.e., the numerator) to the extent that the model could also have provided a good fit to random data x (i.e., the denominator). NML is simple to state but can be difficult to compute; for instance, the denominator may be infinite and this requires further measures to be taken (for details see Grünwald, 2007). Additionally, NML requires an integration over the entire set of possible data sets, which may be difficult to define as it depends on unknown decision processes in the researchers (Berger & Berry, 1988). Note that, since the computation of NML depends on the likelihood of data that might have occurred but did not, the procedure violates the likelihood principle, which states that all information about a parameter θ obtainable from an experiment is contained in the likelihood function for θ for the given y (Berger & Wolpert, 1998). application to multinomial processing tree models Using the parameter estimates from Table 14.1 and the code provided by Wu, Myung and Batchelder (2010), we can compute the FIA for the three competing MPT models considered by Wagenaar and Boer (1987).5 Table 14.3 displays, for each model, the FIA along with its associated complexity measure (the other one of its two constituent components, the maximum log likelihood, can be found in Table 14.2). The conclusions from the MDL analysis mirror those from the
Table 14.3. Minimum description length values for the Wagenaar & Boer MPT models. Complexity
FIA
wFIA
No-conflict model (NCM)
6.44
30.86
0.44
Destructive-updating model (DUM)
7.39
31.80
0.17
Coexistence model (CXM)
7.61
30.96
0.39
AIC measure, expressing a slight disfavor for the destructive updating model, and approximately equal preference for the no-conflict model versus the coexistence model.
Bayes Factors In Bayesian model comparison, the posterior odds for models M1 and M2 are obtained by updating the prior odds with the diagnostic information from the data: p(M1 | y) p(M1 ) m(y | M1 ) = × . (7) p(M2 | y) p(M2 ) m(y | M2 ) Equation 7 shows that the change from prior odds p(M1 )/ p(M2 ) to posterior odds p(M1 | y)/ p(M2 | y) is given by the ratio of marginal likelihoods m(y | M1 )/m(y | M2 ) (see later for the definition of the marginal likelihood). This ratio is known as the Bayes factor (Jeffreys, 1961; Kass & Raftery, 1995). The log of the Bayes factor is often interpreted as the weight of evidence provided by the data (Good, 1985; for details see Berger & Pericchi, 1996; Bernardo & Smith, 1994; Gill, 2002; O’Hagan, 1995). Thus, when the Bayes factor BF12 = m(y | M1 )/m(y | M2 ) equals 5, the observed data y are 5 times more likely to occur under M1 than under M2 ; when BF12 equals 0.1, the observed data are 10 times more likely under M2 than under M1 . Even though the Bayes factor has an unambiguous and continuous scale, it is sometimes useful to summarize the Bayes factor in terms of discrete categories of evidential strength. Jeffreys (1961, Appendix B) proposed the classification scheme shown in Table 14.4. We replaced the labels “not worth more than a bare mention” with “anecdotal,” “decisive” with “extreme,” and “substantial” with “moderate.” These labels facilitate scientific communication but should be considered only as an approximate descriptive articulation of different standards of evidence. Bayes factors negotiate the trade-off between parsimony and goodness of fit and implement an automatic Occam’s razor (Jefferys & Berger, 1992;
Table 14.4. Evidence categories for the Bayes factor B F12 (based on Jeffreys, 1961). Bayes factor BF12
Interpretation
>
100
Extreme evidence for M1
30
—
100
Very Strong evidence for M1
10
—
30
Strong evidence for M1
3
—
10
Moderate evidence for M1
1
—
3
Anecdotal evidence for M1
1
No evidence
1/3
—
1
Anecdotal evidence for M2
1/10
—
1/3
Moderate evidence for M2
1/30
—
1/10
Strong evidence for M2
1/100
—
1/30
Very Strong evidence for M2
<
1/100
Extreme evidence for M2
MacKay, 2003; Myung & Pitt, 1997). To see this, consider that the marginal likelihood m(y | M(·) ) can be expressed as p(y | θ , M(·) ) p(θ | M(·) ) dθ : an average across the entire parameter space, with the prior providing the averaging weights. It follows that complex models with high-dimensional parameter spaces are not necessarily desirable— large regions of the parameter space may yield a very poor fit to the data, dragging down the average. The marginal likelihood will be highest for parsimonious models that use only those parts of the parameter space that are required to provide an adequate account of the data (Lee & Wagenmakers, 2013). By using marginal likelihood, the Bayes factor punishes models that hedge their bets and make vague predictions. Models can hedge their bets in different ways: by including extra parameters, by assigning very wide prior distributions to the model parameters, or by using parameters that participate in the likelihood through a complicated functional form. By computing a weighted average likelihood across the entire parameter space, the marginal likelihood (and, consequently, the Bayes factor) automatically takes all these aspects into account.
model comparison and the principle of parsimony
309
Bayes factors represent “the standard Bayesian solution to the hypothesis testing and model selection problems” (Lewis & Raftery, 1997, p. 648) and “the primary tool used in Bayesian inference for hypothesis testing and model selection” (Berger, 2006, p. 378), but their application is not without challenges (Box 3). Next we show how these challenges can be overcome for the general class of MPT models. Then we compare the results of our Bayes factor analysis with those of the other model comparison methods using Jeffreys weights (i.e., normalized marginal likelihoods).
Box 3 Two challenges for Bayes factors Bayes factors (Jeffreys, 1961; Kass & Raftery, 1995) come with two main challenges, one practical and one conceptual. The practical challenge arises because Bayes factors are defined as the ratio of two marginal likelihoods, each of which requires integration across the entire parameter space. This integration process can be cumbersome and hence the Bayes factor can be difficult to obtain. Fortunately, there are many approximate and exact methods to facilitate the computation of the Bayes factor (e.g., Ardia, Ba¸stürk, Hoogerheide, & van Dijk, 2012; Chen, Shao, & Ibrahim, 2002; Gamerman & Lopes, 2006); in this chapter we focus on BIC (a crude approximation), the Savage-Dickey density ratio (applies only to nested models), and importance sampling. The conceptual challenge that Bayes factors bring is that the prior on the model parameters has a pronounced and lasting influence on the result. This should not come as a surprise: the Bayes factor punishes models for needless complexity, and the complexity of a model is determined in part by the prior distributions that are assigned to the parameters. The difficulty arises because researchers are often not very confident about the prior distributions that they specify. To overcome this challenge one can either spend more time and effort on the specification of realistic priors, or else one can choose default priors that fulfill general desiderata (e.g., Jeffreys, 1961; Liang, Paulo, Molina, Clyde, & Berger, 2008). Finally, the robustness of the conclusions can be verified by conducting a sensitivity analysis in which one examines the effect of changing the prior specification (e.g., Wagenmakers, Wetzels, Borsboom, & van der Maas, 2011).
310
new directions
application to multinomial processing tree models In order to compute the Bayes factor we seek to determine each model’s marginal likelihood m(y | M(·) ). As indicated earlier, the marginal likelihood m(y | M(·) ) is given by integrating the likelihood over the prior: m(y | M(·) ) = p y | θ , M(·) p θ | M(·) dθ . (8) The most straightforward manner to obtain m(y | M(·) ) is to draw samples from the prior p(θ | M(·) ) and average the corresponding values for p(y | θ , M(·) ): m(y | M(·) ) ≈
N 1 p y | θi , M(·) , θi ∼ p(θ ). (9) N i=1
For MPT models, this brute force integration approach may often be adequate. An MPT model usually has few parameters, and each is conveniently bounded from 0 to 1. However, brute-force integration is inefficient, particularly when the posterior is highly peaked relative to the prior: in this case, draws from p(θ | M(·) ) tend to result in low likelihoods and only few chance draws may have high likelihood. This problem can be overcome by a numerical technique known as importance sampling (Hammersley & Handscomb, 1964). In importance sampling, efficiency is increased by drawing samples from an importance density g(θ) instead of from the prior p(θ | M(·) ). Consider an importance density g(θ ). Then,
g(θ ) p y | θ, M(·) p θ | M(·) dθ g(θ) p y | θ , M(·) p θ | M(·) g(θ ) dθ = g(θ) N 1 p y | θi , M(·) p θi | M(·) , ≈ N g(θi )
m(y | M(·) ) =
i=1
θi ∼ g(θ ).
(10)
Note that if g(θ ) = p(θ | M(·) ), the importance sampler reduces to the brute-force integration shown in Eq. 9. Also note that if g(θ ) = p(θ | y, M(·) ), a single draw suffices to determine p(y) exactly. In sum, when the importance density equals the prior, we have brute force integration, and when it equals the posterior, we have a zero-variance estimator. However, in order to compute the posterior, we would have to be able to compute the normalizing constant (i.e., the marginal likelihood),
which is exactly the quantity we wish to determine. In practice, then, we want to use an importance density that is similar to the posterior, is easy to evaluate, and is easy to draw samples from. In addition, we want to use an importance density with tails that are not thinner than those of the posterior; thin tails cause the estimate to have high variance. These desiderata are met by the Beta mixture importance density described in Box 4: a mixture between a Beta(1, 1) density and a Beta density that provides a close fit to the posterior distribution. Here we use a series of univariate Beta mixtures, one for each separate parameter, but acknowledge that a multivariate importance density is potentially even more efficient as it accommodates correlations between the parameters. In our application to MPT models, we assume that all model parameters have uniform Beta(1, 1) priors. For most MPT models this assumption is fairly uncontroversial. The uniform priors can be thought of as a default choice; in the presence of strong prior knowledge one can substitute more informative priors. The uniform priors yield a default Bayes factor that can be a reference point for an analysis with more informative priors, if such an analysis is desired (i.e., when reliable prior information is available, such as can be elicited from experts or derived from earlier experiments). monte carlo sampling for the posterior distribution Before turning to the results of the Bayes factor model comparison, we first inspect the posterior distributions. The posterior distributions were approximated using Markov chain Monte Carlo sampling implemented in JAGS (Plummer, 2003) and WinBUGS (Lunn, Jackson, Best, Thomas, & Spiegelhalter, 2012).6 All code is available on the authors’ websites. Convergence was confirmed by visual inspection and the Rˆ statistic (Gelman & Rubin, 1992). The top panel of Figure 14.6 shows the posterior distributions for the no-conflict model. Although there is slightly more certainty about parameter p than there is about parameters q and c, the posterior distributions for all three parameters are relatively wide considering that they are based on data from as many as 562 participants. The middle panel of Figure 14.6 shows the posterior distributions for the destructive-updating model. It is important to realize that when d = 0 (i.e., no destruction of the earlier memory) the destructive-updating model reduces to the noconflict model. Compared to the no-conflict model,
Box 4 Importance sampling for MPT models using the Beta mixture method Importance sampling was invented by Stan Ulam and John von Neumann. Here we use it to estimate the marginal likelihood by repeatedly drawing samples and averaging—the samples are, however, not drawn from the prior (as per Eq. 9, the brute force method), but instead they are drawn from some convenient density g(θ ) (as per Eq. 10; Andrieu, De Freitas, Doucet, & Jordan, 2003; Hammersley & Handscomb, 1964). The parameters in MPT models are constrained to the unit interval, and, therefore, the family of Beta distributions is a natural candidate for g(θ ). The middle panel of Figure 14.5 shows an importance density (dashed line) for MPT parameter c in the noconflict model for the data from Wagenaar and Boer (1987). This importance density is a Beta distribution that was fit to the posterior distribution for c using the method of moments. The importance density provides a good description of the posterior (the dashed line tracks the posterior almost perfectly) and, therefore, is more efficient than the brute force method illustrated in the left panel of Figure 14.5, which uses the prior as the importance density. Unfortunately, Beta distributions do not always fit MPT parameters so well; specifically, the Beta importance density may sometimes have tails that are thinner than the posterior, and this increases the variability of the marginal likelihood estimate. To increase robustness and ensure that the importance density has relatively fat tails, we can use a Beta mixture, shown in the right panel of Figure 14.5. The Beta mixture consists of a uniform prior component (i.e., the Beta(1, 1) prior as in the left panel) and a Beta posterior component (i.e., a Beta distribution fit to the posterior, as in the middle panel). In this example, the mixture weight for the uniform component is w = 0.2. Small mixture weights retain the efficiency of the Beta posterior approach but avoid the extra variability due to thin tails. It is possible to increase efficiency further by specifying a multivariate importance density, but the present univariate approach is intuitive, easy to implement, and appears to work well in practice. The accuracy of the estimate can be confirmed by increasing the number of draws from the importance density, and by varying the w parameter.
model comparison and the principle of parsimony
311
Beta Posterior Method
Beta Mixture Method
Density
Brute Force Method
0.0 0.2 0.4 0.6 0.8 1.0
0.0 0.2 0.4 0.6 0.8 1.0
0.0 0.2 0.4 0.6 0.8 1.0
Probability Fig. 14.5 Three different importance sampling densities (dashed lines) for the posterior distribution (solid lines) of the c parameter in the no-conflict model as applied to the data from Wagenaar and Boer (1987). Left panel: a uniform Beta importance density (i.e., the brute-force method); middle panel: a Beta posterior importance density (i.e., a Beta distribution that provides the best fit to the posterior); right panel: a Beta mixture importance density (i.e., a mixture of the uniform Beta density and the Beta posterior density, with a mixture weight w = 0.2 on the uniform component).
No−Conflict Model p c
q
0.0
0.2
0.4
0.6
0.8
1.0
Destructive Updating Model Density
p q
d
0.0
0.2
c
0.4
0.6
0.8
1.0
0.8
1.0
Coexistence Model p q
s 0.0
0.2
c
0.4 0.6 Probability
Fig. 14.6 Posterior distributions for the parameters of the noconflict MPT model, the destructive updating MPT model, and the coexistence MPT model, as applied to the data from Wagenaar and Boer (1987).
312
new directions
parameters p, q, and c show relatively little change. The posterior distribution for d is very wide, indicating considerable uncertainty about its true value. A frequentist point-estimate yields dˆ = 0 (Wagenaar & Boer, 1987; see also Table 14.1), but this does not convey the fact that this estimate is highly uncertain. The lower panel of Figure 14.6 shows the posterior distributions for the coexistence model. When s = 0 (i.e., no suppression of the earlier memory), the coexistence model reduces to the noconflict model. Compared to the no-conflict model and the destructive-updating model, parameters p, q, and c again show relatively little change. The posterior distribution for s is very wide, indicating considerable uncertainty about its true value. The fact that the no-conflict model is nested under both the destructive-updating model and the no-conflict model allows us to inspect the extra parameters d and s and conclude that we have not learned very much about their true values. This suggests that, despite having tested 562 participants, the data do not firmly support one model over the other. We will now see how Bayes factors can make this intuitive judgment more precise. importance sampling for the bayes factor We applied the Beta mixture importance sampling method to estimate marginal likelihoods for the three models under consideration. The results were confirmed by varying the mixture weight w, by
independent implementations by the authors, and by comparison to the Savage-Dickey density ratio test presented later. Table 14.5 shows the results. From the marginal likelihoods and the Jeffreys weights we can derive the Bayes factors for the pair wise comparisons; the Bayes factor is 2.77 in favor of the no-conflict model over the destructiveupdating model, the Bayes factor is 1.39 in favor of the coexistence model over the no-conflict model, and the Bayes factor is 3.86 in favor of the coexistence model over the destructive-updating model. The first two of these Bayes factors are anecdotal or “not worth more than a bare mention” (Jeffreys, 1961), and the third one just makes the criterion for “moderate” evidence, although any enthusiasm about this level of evidence should be tempered by the realization that Jeffreys himself described a Bayes factor as high as 5.33 as “odds that would interest a gambler, but would be hardly worth more than a passing mention in a scientific paper” (Jeffreys, 1961, pp. 256–257). In other words, the Bayes factors are consistent with the intuitive visual assessment of the posterior distributions: the data do not allow us to draw strong conclusions. We should stress that Bayes factors apply to a comparison of any two models, regardless of whether or not they are structurally related or nested (i.e., where one model is a special, simplified version of a larger, encompassing model). As is true for the information criteria and minimum description length methods, Bayes factors can be used to compare structurally very different models, such as for example REM (Shiffrin & Steyvers, 1997) versus ACT-R (Anderson, 2004), or the diffusion model (Ratcliff, 1978) versus the linear ballistic accumulator model (Brown & Heathcote, 2008). In other words, Bayes factors can be applied to
nested and non-nested models alike. For the models under consideration, however, there exists a nested structure that allows one to obtain the Bayes factor through a computational shortcut. the savage-dickey approximation to the bayes factor for comparing nested models Consider first the comparison between the noconflict model MNCM and the destructive-updating model MDUM . As shown earlier, we can obtain the Bayes factor for MNCM versus MDUM by computing the marginal likelihoods using importance sampling. However, because the models are nested, we can also obtain the Bayes factor by considering only MDUM , and dividing the posterior ordinate at d = 0 by the prior ordinate at d = 0. This surprising result was first published by Dickey and Lientz (1970), who attributed it to Leonard J. “Jimmie” Savage. The result is now generally known as the Savage-Dickey density ratio (e.g., Dickey, 1971; for extensions and generalizations see Chen, 2005; Verdinelli & Wasserman, 1995; Wetzels, Grasman, & Wagenmakers, 2010; for an introduction for psychologists see Wagenmakers, Lodewyckx, Kuriyal, & Grasman, 2010; a short mathematical proof is presented in O’Hagan & Forster, 2004, pp. 174–177).7 Thus, we can exploit the fact that MNCM is nested in MDUM and use the Savage-Dickey density ratio to obtain the Bayes factor: BFNCM,DUM =
m(y | MNCM ) p(d = 0 | y, MDUM ) = . m(y | MDUM ) p(d = 0 | MDUM ) (11)
The Savage-Dickey density ratio test is visualized in Figure 14.7; the posterior ordinate at d = 0 is higher than the prior ordinate at d = 0, indicating
Table 14.5. Bayesian evidence (i.e., the logarithm of the marginal likelihood), Jeffreys weights, and pairwise Bayes factors computed from the Jeffreys weights or through the Savage-Dickey density ratio, for the Wagenaar and Boer MPT models. Bayesian evidence
Jeffreys weight
No-conflict model (NCM)
−30.55
Destructive-updating model (DUM) Coexistence model (CXM) ∗
Bayes factor (Savage-Dickey) Over NCM
Over DUM
Over CXM
0.36
1
2.77 (2.81)
0.72 (0.80)
−31.57
0.13
0.36 (0.36)
1
0.26 (0.28∗ )
−30.22
0.51
1.39 (1.25)
3.86 (3.51∗ )
1
Derived through transitivity: 2.81 × 1/0.80 = 3.51.
model comparison and the principle of parsimony
313
Savage−Dickey Density Ratio
where the second step is allowed because we have assigned uniform priors to both d and s, so that p(d = 0 | MDUM ) = p(s = 0 | MCXM ). Hence, the Savage-Dickey estimate for the Bayes factor between the two non-nested models MDUM and MCXM equals the ratio of the posterior ordinates at d = 0 and s = 0, resulting in the estimate BFCXM,DUM = 3.51, close to the importance sampling result of 3.86.
Density
Parameter d in the destructive updating model
Posterior Prior
Comparison of Model Comparisons 0.0
0.2
0.4 0.6 Probability
0.8
1.0
Fig. 14.7 Illustration of the Savage-Dickey density-ratio test. The dashed and solid lines show the prior and the posterior distribution for parameter d in the destructive updating model. The black dots indicate the height of the prior and the posterior distributions at d = 0.
that the data have increased the plausibility that d equals 0. This means that the data support MNCM over MDUM . The prior ordinate equals 1, and hence BFNCM,DUM simply equals the posterior ordinate at d = 0. A nonparametric density estimator (Stone, Hansen, Kooperberg, & Truong, 1997) that respects the bound at 0 yields an estimate of 2.81. This estimate is close to 2.77, the estimate from the importance sampling approach. The Savage-Dickey density-ratio test can be applied similarly to the comparison between the no-conflict model MNCM versus the coexistence model MCXM , where the critical test is at s = 0. Here, the posterior ordinate is estimated to be 0.80, and, hence, the Bayes factor for MCXM over MNCM equals 1/0.80 = 1.25, close to the Bayes factor obtained through importance sampling, BFCXM,NCM = 1.39. With these two Bayes factors in hand, we can immediately derive the Bayes factor for the comparison between the destructive updating model MDUM versus the coexistence model MCXM through transitivity, that is, BFCXM,DUM = BFNCM,DUM × BFCXM,NCM . Alternatively, we can also obtain BFCXM,DUM by directly comparing the posterior density for d = 0 against that for s = 0: BFCXM,DUM = BFNCM,DUM × BFCXM,NCM =
p(s = 0 | MCXM ) p(d = 0 | y, MDUM ) × p(d = 0 | MDUM ) p(s = 0 | y, MCXM )
=
p(d = 0 | y, MDUM ) , p(s = 0 | y, MCXM ) (12)
314
new directions
We have now implemented and performed a variety of model comparison methods for the three competing MPT models introduced by Wagenaar and Boer (1987): we computed and interpreted the Akaike information criteria (AIC), Bayesian information criteria (BIC), the Fisher information approximation of the minimum description length principle (FIA), and two computational implementations of the Bayes factor (BF). The general tenor across most of the model comparison exercises has been that the data do not convincingly support one particular model. However, the destructive updating model is consistently ranked the worst of the set. Looking at the parameter estimates, it is not difficult to see why this is so: the d parameter of the destructiveupdating model (i.e., the probability of destroying memory through updating) is estimated at 0, thereby reducing the destructive-updating model to the no-conflict model, and yielding an identical fit to the data (as can be seen in the likelihood column of Table 14.2). Since the no-conflict model counts as a special case of the destructive-updating model, it is by necessity less complex from a modelselection point of view—the d parameter is an unnecessary entity, the inclusion of which is not warranted by the data. This is reflected in the inferior performance of the destructive updating model according to all measures of generalizability. Note that the BF judges the support for NCM to be “anecdotal” even though NCM and DUM provide similar fit and have a clear difference in complexity—one might expect the principle of parsimony to tell us that, given the equal fit and clear complexity difference, there is massive evidence for the simpler model, and the BF appears to fail to implement Occam’s razor here. The lack of clear support of the NCM over the DUM is explained by the considerable uncertainty regarding the value of the parameter d : even though the posterior mode is at d = 0, much posterior variability is visible in the middle panel of Figure 14.6. With more data
and a posterior for d that is more peaked near 0, the evidence in favor of the simpler model would increase. The difference between the no-conflict model and the coexistence model is less clear-cut. Following AIC, the two models are virtually indistinguishable—compared to the coexistence model, the no-conflict model sacrifices one unit of log-likelihood for two units of complexity (one parameter). As a result, both models perform equally well under the AIC measure. Under the BIC measure, however, the penalty for the number of free parameters is more substantial, and here the no-conflict model trades a unit of log likelihood for log (N ) = 6.33 units of complexity, outdistancing both the destructive updating model and the coexistence model. The BIC is the exception in clearly preferring the no-conflict model over the coexistence model. The MDL, like the AIC, would have us hedge on the discriminability of the noconflict model and the coexistence model. The BF, finally, allows us to express evidence for the models using standard probability theory. Between any two models, the BF tells us how much the balance of evidence has shifted due to the data. Using two methods of computing the BF, we determined that the odds of the coexistence model over the destructive updating model almost quadrupled (BFCXM,DUM ≈ 3.86), but the odds of the coexistence model over the no-conflict model barely shifted at all (BFCXM,NCM ≈ 1.39). Finally, we can use the same principles of probability to compute Jeffreys weights, which express, for each model under consideration, the probability that it is true, assuming prior indifference. Furthermore, we can easily recompute the probabilities in case we wish to express a prior preference between the candidate models (for example, we might use the prior to express a preference for sparsity, as was originally proposed by Jeffreys, 1961).
Concluding Comments Model comparison methods need to implement the principle of parsimony: goodness-of-fit has to be discounted to the extent that it was accomplished by a model that is overly complex. Many methods of model comparison exist (Myung et al., 2000; Wagenmakers & Waldorp, 2006), and our selective review focused on methods that are popular, easyto-compute approximations (i.e., AIC and BIC) and methods that are difficult-to-compute “ideal” solutions (i.e., minimum description length and Bayes factors). We applied these model comparison
methods to the scenario of three competing MPT models introduced by Wagenaar and Boer (1987). Despite collecting data from 562 participants, the model comparison methods indicate that the data are somewhat ambiguous; at any rate, the data do not support the destructive updating model. This echoes the conclusions drawn by Wagenaar and Boer (1987). It is important to note that the modelcomparison methods discusses in this chapter can be applied regardless of whether the models are nested. This is not just a practical nicety; it also means that the methods are based on principles that transcend the details of a specific model implementation. In our opinion, a method of inference that is necessarily limited to the comparison of nested models is incomplete at best and misleading at worst. It is also important to realize that model comparison methods are relative indices of model adequacy; when, say, the Bayes factor expresses an extreme preference for model A over model B, this does not mean that model A fits the data at all well. Figure 14.8 shows a classic but dramatic example of the inadequacy of simple measures of relative model fit. Because it would be a mistake to base inference on a model that fails to describe the data, a complete inference methodology features both relative and absolute indices of model adequacy. For the MPT models under consideration here, Wagenaar and Boer (1987) reported that the no-conflict model provided “an almost perfect fit” to the data.8 The example MPT scenario considered here was relatively straightforward. More complicated MPT models contain order-restrictions, feature individual differences embedded in a hierarchical framework (Klauer, 2010; Matzke, Dolan, Batchelder, & Wagenmakers, in press), or contain a mixture-model representation with different latent classes of participants (for application to other models see Frühwirth-Schnatter, 2006; Scheibehenne, Rieskamp, & Wagenmakers, 2013). In theory, it is relatively easy to derive Bayes factors for these more complicated models. In practice, however, Bayes factors for complicated models may require the use of numerical techniques more involved than importance sampling. Nevertheless, for standard MPT models the Beta mixture importance sampler appears to be a convenient and reliable tool to obtain Bayes factors. We hope that this methodology will facilitate the principled comparison of MPT models in future applications.
model comparison and the principle of parsimony
315
Anscombe’s Quartet 12
12
10
10
8
8
6
6
r = 0.816
4
r = 0.816
4 5
10
15
20
5
12
12
10
10
8
8
6
10
6
r = 0.816
4
15
20
r = 0.816
4 5
10
15
20
5
10
15
20
Fig. 14.8 Anscombe’s Quartet is a set of four bivariate data sets whose basic descriptive statistics are approximately identical. In all cases, mean of X is 9, variance of X is 11, mean of Y is 7.5, variance of Y is 4.1, and the best fitting linear regression line is yiest = 3 + 0.5xi , which explains R 2 = 66.6% of the variance in Y . However, in two of the four cases, the linear regression is clearly a poor account of the data. The relative measure of model fit (R 2 ) gives no indication of this radical difference between the data sets, and an absolute measure of fit (even one as rudimentary as a visual inspection of the regression line) is required. (Figure downloaded from Flickr, courtesy of Eric-Jan Wagenmakers.)
Notes 1. This work was partially supported by the starting grant “Bayes or Bust” awarded by the European Research Council to EJW, and NSF grant #1230118 from the Methods, Measurements, and Statistics panel to JV. 2. This terminology is due to Pitt and Myung (2002), who point out that measures often referred to as “model fit indices” are in fact more than mere measures of fit to the data—they combine fit to the data with parsimony and hence measure generalizability. We adopt their more accurate terminology here. 3. Note that for hierarchical models, the definition of sample size n is more complicated (Pauler 1998; Raftery 1995). 4. For a more in-depth treatment, see Townsend (1975). 5. Analysis using the MPTinR package by Singmann and Kellen (2013) gave virtually identical results. Technical details for the computation of the NML for MPTs are provided in Appendix B of Klauer and Kellen (2011). 6. The second author used WinBUGS, the first and third authors used JAGS. 7. Note that the Savage-Dickey density ratio requires that when d = 0 the prior for the common parameters p, c, and q is the same for MDUM and MNCM . That is, p(p, c, q | d = 0, MDUM ) = p(p, c, q | MNCM ).
316
new directions
8. We confirmed the high quality of fit in a Bayesian framework using posterior predictives (Gelman & Hill, 2007), results not reported here.
Glossary Akaike’s information criterion (AIC): A quantity that expresses the generalizability of a model, based on the likelihood of the data under the model and the number of free parameters in the model. Akaike weights: A quantity that conveys the relative preference among a set of candidate models, using AIC as a measure of generalizability. Anscombe’s quartet: A set of four bivariate data sets whose statistical properties are virtually indistinguishable until they are displayed graphically, and a canonical example of the importance of data visualization. Bayes factor (BF): A quantity that conveys the degree to which the observed data sway our beliefs towards one or the other model. Under a-priori indifference between two models M1 and M2 , the BF expresses the a-posteriori relative probability of the two.
Bayesian information criterion (BIC): A quantity that expresses the generalizability of a model, based on the likelihood of the data under the model, the number of free parameters in the model, and the amount of data. Fisher information approximation (FIA): One of several approximations used to compute the MDL. Goodness of fit: A quantity that expresses how well a model is able to account for a given set of observations. Importance sampling: A numerical algorithm to efficiently draw samples from a distribution by factoring it into an easy-to-compute function over an easy-to-sample density. Jeffreys weights: A quantity that conveys the relative preference among a set of candidate models, using BF as a measure of generalizability. Likelihood principle: A principle of modeling and statistics that states that all information about a certain parameter that is obtainable from an experiment is contained in the likelihood function of that parameter for the given data. Many common statistical procedures, such as hypothesis testing with p-values, violate this principle. Minimum description length (MDL): A quantity that expresses the generalizability of a model, based on the extent to which the model allows the observed data to be compressed. Monte Carlo sampling: A general class of numerical algorithms used to characterize (i.e., compute descriptive measures) an arbitrary distribution by drawing large numbers of random samples from it. Nested models: Model M1 is nested in Model M2 if there exists a special case of M2 that is equivalent to M1 . Overfitting: A pitfall of modeling whereby the proposed model is too complex and begins to account for irrelevant particulars (i.e., random noise) of a specific data set, causing the model to poorly generalize to other data sets. Parsimony: A strategy against overfitting, and a fundamental principle of model selection: all other things being equal, simpler models should be preferred over complex ones; or: greater model complexity must be bought with greater explanatory power. Often referred to as Occam’s razor. Rissanen weights: A quantity that conveys the relative preference among a set of candidate models, using FIA as a measure of generalizability. Savage-Dickey density ratio: An efficient method for computing a Bayes factor between nested models. Schwartz weights: A quantity that conveys the relative preference among a set of candidate models, using BIC as a measure of generalizability.
References Akaike, H. (1973). Information theory as an extension of the maximum likelihood principle. In: B. N. Petrov & F. Csaki (Eds.), Second international symposium on information theory (267–281). Budapest: Akadémiai Kiadó. Akaike, H. (1974a). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716–723.
Akaike, H. (1974b). On the likelihood of a time series model. The Statistician, 27, 217–235. Anderson, J. R., Bothell, D., Byrne, M. D., Douglass, S., Lebiere, C., & Qin, Y. (2004). An integrated theory of the mind. Psychological Review, 111, 1036–1060. Andrieu, C., De Freitas, N., Doucet, A., & Jordan, M. I. (2003). An introduction to MCMC for machine learning. Machine Learning, 50, 5–43. Ardia, D., Bastürk, N., Hoogerheide, L., & van Dijk, H. K. (2012). A comparative study of Monte Carlo methods for efficient evaluation of marginal likelihood. Computational Statistics and Data Analysis, 56, 3398–3414. Batchelder, W. H., & Riefer, D. M. (1980). Separation of storage and retrieval factors in free recall of clusterable pairs. Psychological Review, 87, 375–397. Batchelder, W. H., & Riefer, D. M. (1999). Theoretical and empirical review of multinomial process tree modeling. Psychonomic Bulletin & Review, 6, 57–86. Batchelder, W. H., & Riefer, D. M. (2007). Using multinomial processing tree models to measure cognitive deficits in clinical populations. In R. W. J. Neufeld (Ed.), Advances in clinical cognitive science: Formal modeling of processes and symptoms (19–50). Washington, DC: American Psychological Association. Berger, J. O. (2006). Bayes factors. S. Kotz, N. Balakrishnan, C. Read, B. Vidakovic, & N. L. Johnson (Eds.), Encyclopedia of statistical sciences (2nd ed.) (Vol. 1, pp 378–386). Hoboken, NJ: Wiley. Berger, J. O., & Berry, D. A. (1988). Statistical analysis and the illusion of objectivity. American Scientist, 76, 159–165. Berger, J. O., & Pericchi, L. R. (1996). The intrinsic Bayes factor for model selection and prediction. Journal of the American Statistical Association, 91, 109–122. Berger, J. O., & Wolpert, R. L. (1988). The likelihood principle (2nd ed.). Hayward, CA: Institute of Mathematical Statistics. Bernardo, J. M., & Smith, A. F. M. (1994). Bayesian theory. New York, NY: Wiley. Brown, S. D., & Heathcote, A. (2005). Practice increases the efficiency of evidence accumulation in perceptual choice. Journal of Experimental Psychology: Human Perception and Performance, 31, 289–298. Brown, S. D., & Heathcote, A. J. (2008). The simplest complete model of choice reaction time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178. Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information–theoretic approach (2nd ed.). New York, NY: Springer Verlag. Busemeyer, J. R., & Diederich, A. (2010). Cognitive modeling. Thousand Oaks, CA: Sage. Chechile, R. A. (1973). The relative storage and retrieval losses in short–term memory as a function of the similarity and amount of information processing in the interpolated task. Unpublished doctoral dissertation, University of Pittsburgh. Chechile, R. A., & Meyer, D. L. (1976). A Bayesian procedure for separately estimating storage and retrieval components of forgetting. Journal of Mathematical Psychology, 13, 269–295. Chen, M.-H. (2005). Computing marginal likelihoods from a single MCMC output. Statistica Neerlandica, 59, 16–29.
model comparison and the principle of parsimony
317
Chen, M.-H., Shao, Q.-M., & Ibrahim, J. G. (2002). Monte Carlo methods in Bayesian computation. New York, NY: Springer. Cohen, J. D., Dunbar, K., & McClelland, J. L. (1990). On the control of automatic processes: A parallel distributed processing account of the Stroop effect. Psychological Review, 97, 332–361. D’Agostino, R. B., & Stephens, M. A. (1986). Goodness-of-fit techniques. New York, NY: Marcel Dekker. Dickey, J. M. (1971). The weighted likelihood ratio, linear hypotheses on normal location parameters. The Annals of Mathematical Statistics, 42, 204–223. Dickey, J. M., & Lientz, B. P. (1970). The weighted likelihood ratio, sharp hypotheses about chances, the order of a Markov chain. The Annals of Mathematical Statistics, 41, 214–226. Dutilh, G., Vandekerckhove, J., Tuerlinckx, F., & Wagenmakers, E.-J. (2009). A diffusion model decomposition of the practice effect. Psychonomic Bulletin & Review, 16, 1026– 1036. Erdfelder, E., Auer, T.-S., Hilbig, B. E., Aßfalg, A., Moshagen, M., & Nadarevic, L. (2009). Multinomial processing tree models: A review of the literature. Zeitschrift für Psychologie, 217, 108–124. Frühwirth–Schnatter, S. (2006). Finite mixture and Markov switching models. New York, NY: Springer. Gamerman, D., & Lopes, H. F. (2006). Markov chain Monte Carlo: Stochastic simulation for Bayesian inference. Boca Raton, FL: Chapman & Hall/CRC. Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge, England: Cambridge University Press. Gelman, A., & Rubin, D. B. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statistical Science, 7, 457–472. Gill, J. (2002). Bayesian methods: A social and behavioral sciences approach. Boca Raton, FL: CRC Press. Good, I. J. (1985). Weight of evidence: A brief survey. J. M. Bernardo, M. H. DeGroot, D. V. Lindley, & A. F. M. Smith, Bayesian statistics 2 (249–269). New York, NY: Elsevier. Grünwald, P. (2000). Model selection based on minimum description length. Journal of Mathematical Psychology, 44, 133–152. Grünwald, P. (2007). The minimum description length principle. Cambridge, MA: MIT Press. Grünwald, P., Myung, I. J., & Pitt, M. A. (Eds). (2005). Advances in minimum description length: Theory and pplications. Cambridge, MA: MIT Press. Hammersley, J. M., & Handscomb, D. C. (1964). Monte Carlo methods. London, England: Methuen. Heathcote, A., Brown, S., & Mewhort, D. J. K. (2000). The power law repealed: The case for an exponential law of practice. Psychonomic Bulletin & Review, 7, 185–207. Heathcote, A., & Hayes, B. (2012). Diffusion versus linear ballistic accumulation: Different models for response time with different conclusions about psychological mechanisms? Canadian Journal of Experimental Psychology, 66, 125–136. Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999). Bayesian model averaging: A tutorial. Statistical Science, 14, 382–417.
318
new directions
Jefferys, W. H., & Berger, J. O. (1992). Ockham’s razor and Bayesian analysis. American Scientist, 80, 64–72. Jeffreys, H. (1961). Theory of probability (3rd ed.). Oxford, England: Oxford University Press. Kass, R. E., & Raftery, A. E. (1995). Bayes factors. Journal of the American Statistical Association, 90, 773–795. Klauer, K. C. (2010). Hierarchical multinomial processing tree models: A latent–trait approach. Psychometrika, 75, 70–98. Klauer, K. C., & Kellen, D. (2011). The flexibility of models of recognition memory: An analysis by the minimum-description length principle. Journal of Mathematical Psychology, 55(6), 430–450. Lee, M. D., & Wagenmakers, E.-J. (2013). Bayesian modeling for cognitive science: A practical course. Cambridge, England: Cambridge University Press. Lewandowsky, S., & Farrell, S. (2010). Computational modeling in cognition: Principles and practice. Thousand Oaks, CA: Sage. Lewis, S. M., & Raftery, A. E. (1997). Estimating Bayes factors via posterior simulation with the Laplace–Metropolis estimator. Journal of the American Statistical Association, 92, 648–655. Liang, F., Paulo, R., Molina, G., Clyde, M. A., & Berger, J. O. (2008). Mixtures of g priors for Bayesian variable selection. Journal of the American Statistical Association, 103, 410–423. Loftus, E. F., Miller, D. G., & Burns, H. J. (1978). Semantic integration of verbal information into a visual memory. Journal of Experimental Psychology: Human Learning and Memory, 4, 19–31. Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95, 492–527. Logan, G. D. (1992). Shapes of reaction–time distributions and shapes of learning curves: A test of the instance theory of automaticity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 883–914. Logan, G. D. (2002). An instance theory of attention and memory. Psychological Review, 109, 376–400. Lunn, D., Jackson, C., Best, N., Thomas, A., & Spiegelhalter, D. (2012). The BUGS book: A practical introduction to Bayesian analysis. Boca Raton, FL: Chapman & Hall/CRC. Ly, A., Verhagen, A. J., Grasman, R. P. P. P., Wagenmakers, E.-J. (2014). A tutorial on Fisher information. Manuscript submitted for publication. MacKay, D. J. C. (2003). Information theory, inference, and learning algorithms. Cambridge, England: Cambridge University Press. Matzke, D., Dolan, C. V., Batchelder, W. H., & Wagenmakers, E.-J. (in press). Bayesian estimation of multinomial processing tree models with heterogeneity in participants and items. Psychometrika. Myung, I. J. (2000). The importance of complexity in model selection. Journal of Mathematical Psychology, 44, 190–204. Myung, I. J., Forster, M. R., & Browne, M. W. (2000). Model selection [Special issue]. Journal of Mathematical Psychology, 44, 1–2. Myung, I. J., Navarro, D. J., & Pitt, M. A. (2006). Model selection by normalized maximum likelihood. Journal of Mathematical Psychology, 50, 167–179.
Myung, I. J., Pitt, & M. A. (1997). Applying Occam’s razor in modeling cognition: A Bayesian approach. Psychonomic Bulletin & Review, 4, 79–95. O’Hagan, A. (1995). Fractional Bayes factors for model comparison. Journal of the Royal Statistical Society B, 57, 99–138. O’Hagan, A., & Forster, J. (2004). Kendall’s advanced theory of statistics, Vol. 2B: Bayesian inference (2nd ed.). London, England: Arnold. Pauler, D. K. (1998). The Schwarz criterion and related methods for normal linear models. Biometrika, 85, 13–27. Pitt, M. A., & Myung, I. J. (2002). When a good fit can be bad. Trends in Cognitive Sciences, 6, 421–425. Pitt, M. A., Myung, I. J., & Zhang, S. (2002). Toward a method of selecting among computational models of cognition. Psychological Review, 109, 472–491. Plummer, M. (2003). JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling. K. Hornik, F. Leisch, & A. Zeileis (Eds.), Proceedings of the 3rd international workshop on distributed statistical computing. Vienna, Austria. Raftery, A. E. (1995). Bayesian model selection in social research. P. V. Marsden (Ed.), Sociological methodology (pp 111–196). Cambridge, England: Blackwells. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108. Rickard, T. C. (1997). Bending the power law: A CMPL theory of strategy shifts and the automatization of cognitive skills. Journal of Experimental Psychology: General, 126, 288–311. Riefer, D. M., & Batchelder, W. H. (1988). Multinomial modeling and the measurement of cognitive processes. Psychological Review, 95, 318–339. Rissanen, J. (1978). Modeling by shortest data description. Automatica, 14, 445–471. Rissanen, J. (1987). Stochastic complexity. Journal of the Royal Statistical Society B, 49, 223–239. Rissanen, J. (1996). Fisher information and stochastic complexity. IEEE Transactions on Information Theory, 42, 40–47. Rissanen, J. (2001). Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory, 47, 1712–1717. Rouder, J. N., Lu, J., Morey, R. D., Sun, D., & Speckman, P. L. (2008). A hierarchical process dissociation model. Journal of Experimental Psychology: General, 137, 370–389. Scheibehenne, B., Rieskamp, J., & Wagenmakers, E.-J., (2013). Testing adaptive toolbox models: A Bayesian hierarchical approach. Psychological Review, 120, 39–64. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464. Shiffrin, R. M., & Steyvers, M. (1997). A model for recognition memory: REM—retrieving effectively
from memory. Psychonomic Bulletin & Review, 4, 145–166. Silver, N. (2012). The signal and the noise: The art and science of prediction. London, England: Allen Lane. Singmann, H., & Kellen, D. (2013). MPTinR: Analysis of multinomial processing tree models with R. Behavior Research Methods, 45, pp 560–575. Smith, J. B., & Batchelder, W. H. (2010). Beta–MPT: Multinomial processing tree models for addressing individual differences. Journal of Mathematical Psychology, 54, 167–183. Stone, C. J., Hansen, M. H., Kooperberg, C., & Truong, Y. K. (1997). Polynomial splines and their tensor products in extended linear modeling (with discussion). The Annals of Statistics, 25, 1371–1470. Townsend, J. T. (1975). The mind–body equation revisited. C. Cheng (Ed.), Philosophical aspects of the mind–body problem (pp. 200–218). Honolulu, Hawaii: Honolulu University Press. Verdinelli, I., & Wasserman, L. (1995). Computing Bayes factors using a generalization of the Savage–Dickey density ratio. Journal of the American Statistical Association, 90, 614–618. Vrieze, S. I. (2012). Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychological methods, 17, 228. Wagenaar, W. A., & Boer, J. P. A. (1987). Misleading postevent information: Testing parameterized models of integration in memory. Acta Psychologica, 66, 291–306. Wagenmakers, E.-J., & Farrell, S. (2004). AIC model selection using Akaike weights. Psychonomic Bulletin & Review, 11, 192–196. Wagenmakers, E.-J., Lodewyckx, T., Kuriyal, H., & Grasman, R. (2010). Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method. Cognitive Psychology, 60, 158–189. Wagenmakers, E.-J., & Waldorp, L. (2006). Model selection: Theoretical developments and applications [Special issue]. Journal of Mathematical Psychology, 50(2), pp 1–2. Wagenmakers, E.-J., Wetzels, R., Borsboom, D., & van der Maas, H. L. J. (2011). Why psychologists must change the way they analyze their data: The case of psi. Journal of Personality and Social Psychology, 100, 426–432. Wetzels, R., Grasman, R. P. P. P., & Wagenmakers, E.-J. (2010). An encompassing prior generalization of the Savage–Dickey density ratio test. Computational Statistics & Data Analysis, 54, 2094–2102. Wu, H., Myung, J. I., & Batchelder, W. H. (2010). Minimum description length model selection of multinomial processing tree models. Psychonomic Bulletin & Review, 17, 275–286.
model comparison and the principle of parsimony
319
CHAPTER
15
Neurocognitive Modeling of Perceptual Decision Making
Thomas J. Palmeri, Jeffrey D. Schall, and Gordon D. Logan
Abstract
Mathematical psychology and systems neuroscience have converged on stochastic accumulator models to explain decision making. We examined saccade decisions in monkeys while neurophysiological recordings were made within their frontal eye field. Accumulator models were tested on how well they fit response probabilities and distributions of response times to make saccades. We connected these models with neurophysiology. To test the hypothesis that visually responsive neurons represented perceptual evidence driving accumulation, we replaced perceptual processing time and drift rate parameters with recorded neurophysiology from those neurons. To test the hypothesis that movement related neurons instantiated the accumulator, we compared measures of neural dynamics with predicted measures of accumulator dynamics. Thus, neurophysiology both provides a constraint on model assumptions and data for model selection. We highlight a gated accumulator model that accounts for saccade behavior during visual search, predicts neurophysiology during search, and provides insights into the locus of cognitive control over decisions. Key Words: accumulator models, decision making, response time, visual search, stop task, countermanding, neurophysiology, computational modeling, neural modeling, frontal eye field, superior colliculus
Introduction We make decisions all the time. Whom to marry? What car to buy? What to eat? Whether to turn left or right? Some are easy. Some are hard. Some involve uncertainty. Some involve risk or reward. Decision-making requires integrating our perceptions of the current environment with our knowledge and past experience and our assessments of uncertainty and risk in order to select a possible action from a set of alternatives. Behavioral research on decision-making has had a long and distinguished history in psychology (e.g., Kahneman & Tversky, 1984). We now have powerful computational and mathematical models of how decisions are made (e.g., Brown & Heathcote, 2008; Busemeyer & Townsend, 1993; Dayan & Daw, 2008; Ratcliff & Rouder, 1998). And we 320
know more about the brain areas involved in a range of decision-making tasks (Glimcher & Rustichini, 2004; Heekeren, Marrett, & Ungerleider, 2008; Schall, 2001; Shadlen & Newsome, 2001). To develop an integrated understanding of decisionmaking mechanisms, new efforts aim to combine behavioral and neural measures with cognitive modeling (e.g., Forstmann, Wagenmakers, Eichele, Brown, & Serences, 2011; Gold & Shadlen, 2007; Palmeri, in press; Smith & Ratcliff, 2004), an approach we aim to illustrate in some detail here. We focus on perceptual decisions. Perceptual decision-making involves perceptually representing the world with respect to current task goals and using perceptual evidence to inform the selection of an action. A broad class of accumulator models of perceptual decision-making assume that perceptual
evidence accumulates over time to a response threshold (e.g., Bogacz, Brown, Moehlis, Holmes, & Cohen, 2006; Brown & Heathcote, 2008; Link, 1992; Nosofsky & Palmeri, 1997; Palmeri, 1997; Ratcliff & Rouder, 1998; Ratcliff & Smith, 2004; Ratcliff & Smith, in press; Smith & Van Zandt, 2000; Usher & McClelland, 2001; see also Nosofsky & Palmeri, 2015). These models have provided excellent accounts of observed behavior, including the choices people make and the time it takes them to decide. Moreover, the observation that the pattern of spiking activity of certain neurons resembles an accumulation to threshold (Hanes & Schall, 1996) has sparked exciting synergies of mathematical and computational modeling with systems neuroscience (e.g., Boucher, Palmeri, Logan, & Schall, 2007a; Churchland & Ditterich, 2012; Cisek, Puskas, & El-Murr, 2009; Ditterich, 2006, 2010; Mazurek, Roitman, Ditterich, & Shadlen, 2003; Purcell, Heitz, Cohen, Schall, Logan, & Palmeri, 2010; Purcell, Schall, Logan, & Palmeri, 2012; Ratcliff, Cherian, & Segraves, 2003; Ratcliff, Hasegawa, Hasegawa, Smith, & Segraves, 2007; Wong, Huk, Shadlen, & Wang, 2007; Wong & Wang, 2006). In this article, we provide a general review of our contributions to these efforts. We use variants of accumulator models to explain neural mechanisms, use neurophysiology to constrain model assumptions, and use neural and behavioral data as a tool for model section. Our specific focus has been on perceptual decisions about where and when to make a saccadic eye movement to objects in the visual field. The first section of this article, Perceptual Decisions by Saccades, provides an overview of behavior, neuroanatomy, and neurophysiology of the primate saccade system, with an emphasis on the frontal eye field (FEF). There are numerous practical advantages to studying perceptual decisions made by saccades over perceptual decisions made by finger, hand, or limb movement and we can also capitalize on over two decades of careful systems neuroscience research with awake behaving monkeys characterizing the response properties of neurons in FEF and the interconnected network of other brain areas involved in saccadic eye movements (Figure 15.1). FEF itself provides physiologists and theoreticians a unique window on perceptual decision-making. FEF receives projections from a wide range of posterior brain areas involved in visual perception, projects to subcortical brain areas involved directly in the production of eye
LIP FEF MT V4 TEO TE
SC Brainstem Fig. 15.1 Illustration of the macaque cerebral cortex. Frontal eye field (FEF) is a key brain area involved in the production of saccadic eye movements and the focus of our recent work. It receives projections from numerous posterior visual areas, including the middle temporal area (MT), visual area V4, inferotemporal areas TE and TEO, and the lateral intraparietal area (LIP). FEF projects to the superior colliculus (SC). Both FEF and SC project to the brainstem saccade generators that ultimately control the muscles of the eyes. Not shown are connections between FEF and prefrontal cortical areas and areas of the basal ganglia. (Adapted from Purcell et al., 2010.)
movements, and is modulated by prefrontal brain areas involved in cognitive control. Indeed, one class of visually responsive neurons in FEF represent task-relevant salience of objects in the visual field, whereas another class of movement-related neurons increase their activity in a manner consistent with accumulation of evidence models and modulate their activity according to changing task demands (e.g., see Schall, 2001, 2004). One form of an accumulator model is illustrated in Figure 15.2. Accumulator models assume that perceptual processing takes some amount of time. The product of perceptual processing is perceptual evidence that is accumulated over time to make a perceptual decision. The rate of accumulation is often called drift rate, and this drift rate can be variable within a trial, across trials, or both (e.g., Brown & Heathcote, 2008; Ratcliff & Rouder, 1998). Variability in the accumulation of perceptual evidence to a threshold is a major contributor to variability in predicted behavior. In their most general form, accumulator models assume drift rates to be free parameters that can be optimized to fit a set of observed behavioral data. There has been concern that unrestricted assumptions about drift rate and its variability may imbue these models with too much flexibility (Jones & Dzhafarov, 2014; but see also Ratcliff,
neurocognitive modeling of perceptual decision making
321
(a)
perceptual processing time
drift θ
TR
motor response
TM
(b)
θ
motor response
TM Time Fig. 15.2 (a) Illustration of a classic stochastic accumulator model of perceptual decision-making, highlighting some of the key free parameters. Perceptual processing of a visual stimulus takes some variable amount time with mean TR. The outcome of perceptual processing is noisy perceptual evidence in favor of competing decisions with some mean drift rate. Perceptual evidence is accumulated over time, originating at some variable starting point (z), and accumulating until some threshold is reached, determined by θ Illustrated here is a drift-diffusion model, but different architectures for the perceptual decision-making process can be assumed (see Figure 15.5). Variability in the accumulation of evidence to a threshold is a key constituent in predicting variability in RT. A motor response is made with some time TM, which for saccadic eye movements is on the order of 10-20ms. (b) Our recent work has tested whether many of the free parameters can be constrained by the observed physiological dynamics of one class of neurons in FEF (see Figure 15.5) and whether predicted model dynamics of the stochastic accumulator can predict observed physiological dynamics of another class of neurons in FEF (see Figure 15.8).
2013). One important step in theory development has been to significantly constrain these models by creating theories of the drift rates driving the accumulation of evidence, linking models of perceptual decision making with models of perceptual processing (e.g., Ashby, 2000; Logan & Gordon, 2001; Mack & Palmeri, 2010, 2011; Nosofsky & Palmeri, 1997; Palmeri, 1997; Palmeri & Cottrell, 2009; Palmeri & Tarr, 2008; Schneider & Logan, 2005, 2009; Smith & Ratcliff, 2009). As a first step toward a neural theory of drift rates, we hypothesized that activity of visually responsive neurons in FEF represent perceptual evidence driving the accumulation to threshold. To test this hypothesis, as described in the section titled A Neural Locus of Drift Rates, we replaced perceptual processing-time and drift-rate parameters directly with recorded neurophysiology from these neurons (see Figures 15.2 and 15.5), testing whether any model architecture for accumulation of perceptual evidence could then quantitatively
322
new directions
account for observed saccade response probabilities and response time distributions. A number of different model architectures have been proposed that all involve some accumulation of perceptual evidence to a threshold (e.g., see Bogacz et al., 2006; Smith & Ratcliff, 2004). For example, as their name implies, independent race models assume that evidence for each alternative decision independently (Smith & Van Zandt, 2000; Vickers, 1970). Drift-diffusion models (Ratcliff, 1978; Ratcliff & Rouder, 1998) and random walk models (Laming, 1968; Link, 1992; Nosofsky & Palmeri, 1997; Palmeri, 1997) assume that perceptual evidence in favor of one alternative counts as evidence against competing alternatives. Competing accumulator models (Usher & McClelland, 2001) assume that support for various alternatives is mutually inhibitory, so as evidence in favor of one alternative grows, it inhibits the others, often in a winner-take-all fashion (Grossberg, 1976). Different models can vary in other respects
as well, such as whether integration of evidence is perfect or leaky. We describe these alternative model architectures and how well they account for observed response probabilities and response time distributions in the section Architectures for Perceptual Decision Making. We also tested the hypothesis that movementrelated neurons in FEF instantiate an accumulator (Hanes & Schall, 1996). As described in the section Predicting Neural Dynamics, we quantitatively compared measured metrics of neural dynamics with predicted metrics of accumulator dynamics. Neurophysiology and modeling are synergistic in that we test quantitatively whether movementrelated neurons have dynamics predicted by accumulator models, and we use the measured neural dynamics of movement-related neurons as an additional tool to select between competing model architectures. Finally, in a complementary way, in the section Control over Perceptual Decisions, we test whether competing hypotheses about cognitive control mechanisms can predict observed behavior as well as the observed modulation of movementrelated neurons dynamics.
Perceptual Decisions by Saccades Significant insights into the neurophysiological basis of perceptual decision-making have come from research on decisions about where and when to move the eyes (e.g., Gold & Shadlen, 2007; Schall, 2001, 2004; Smith & Ratcliff, 2004). Although the majority of human research on perceptual decisions has used manual key-press responses, a neurophysiological focus on saccadic eye movements is justified on several grounds: From the perspective of effect or dynamics and motor control, eye movements have relatively few degrees of freedom, far fewer than limb movements, allowing fairly direct links between neurophysiology and behavior to be established (Scudder, Kaneko, & Fuchs, 2002). Saccadic eye movements are also relatively ballistic, with movement dynamics quite stereotyped depending on the direction, starting point, and distance the eyes need to move (Gilchrist, 2011), unlike limb movement, which can reach the same endpoint using a multitude of different trajectories having vastly different temporal dynamics (Rosenbaum, 2009). Moreover, from the perspective of understanding the mechanisms by which perceptual evidence is used to produce a perceptual decision, the saccade system is also a choice candidate to study because of the Frontal
Eye Field (FEF), an area where visual perception, motor production, and cognitive control come together in the primate brain (Schall & Cohen, 2011). FEF has long been known to play a role in the production of saccadic eye movements (e.g., Bruce, Goldberg, Bushnell, & Stanton, 1985; Ferrier, 1874). This is reflected by its direct and indirect connectivity with the superior colliculus (SC) and brain stem nuclei necessary for the production of saccadic eye movement (e.g., Munoz & Schall, 2004; Scudder et al., 2002; Sparks, 2002), as illustrated in Figure 15.1. Also as illustrated, FEF is innervated by numerous dorsal and ventral stream areas of extrastriate visual cortex (Schall, Morel, King, & Bullier, 1995). Not illustrated are connections between FEF and brain areas implicated in cognitive control, such as medial frontal and dorsolateral prefrontal cortex (e.g., Stanton, Bruce, & Goldberg, 1995) and basal ganglia (GoldmanRakic & Porrino, 1985; Hikosaka & Wurtz, 1983). Neuroanatomically, FEF lies at a juncture of perception, action, and control. This bears out functionally, as various neurons within FEF reflect the importance of objects in the visual field,signal the selection and timing of saccadic eye movements, and modulate in a controlled manner according to changing task conditions (e.g., Heitz & Schall, 2012; Murthy, Ray, Shorter, Schall, & Thompson, 2009; Thompson, Biscoe, & Sato, 2005). At the start of each neurophysiological session, once a neuron in FEF has been isolated, a memoryguided saccade task is used to classify its response properties (Bruce & Goldberg, 1985). As illustrated in Figure 15.3, the monkey fixates a spot in the center of the screen while a target is flashed in the periphery. To earn reward, the monkey must maintain fixation for a variable amount of time after which the fixation spot disappears and then the monkey must make a single saccade to the remembered target location. When the target is in the receptive field of the FEF neuron, that neuron is classified as a visually responsive neuron (or visual neuron) if it shows a vigorous response to the appearance of the target, perhaps with a tonic response during the delay period, but with no significant saccade-related modulation. The neuron is classified as a movement-related neuron (or movement neuron, sometimes referred to as a buildup neuron) if it shows no or very weak modulation to the appearance of the target but pronounced growth of spike rate immediately
neurocognitive modeling of perceptual decision making
323
visual search
(a) Normalized Visual Neuron Activity
memory-guided
e tim
0.5
preceding saccade production. Other neurons in FEF show other response properties (e.g., Sato & Schall, 2003), but our recent work has focused primarily on visual and movement neurons, which we might loosely characterize as the incoming input signal and outgoing output signal from FEF (see also Pouget et al., 2009). Once visually responsive neurons and movementrelated neurons are identified, their response properties can be measured during a primary perceptual decision task. For example, in a visual search task, as illustrated in Figure 15.3, after the monkey fixates a central spot, a search array is shown containing a target (in this case an L) and several distractors (in this case rotated Ts) and the monkey must make a single saccade to the target in order to receive reward. During visual search, visually responsive and movement-related neurons display characteristic dynamics. Figure 15.4 shows the normalized spiking activity of representative neurons recorded during easy and hard visual search trials when the target (solid) or a distractor (dashed) was in the neuron’s receptive field. For some time after the visual search array appears, visually responsive neurons (Figure 15.4a) show no discrimination between a target and a distractor. However, spiking new directions
Normalized Movement Neuron Activity
Hard (target out) Easy (target out) F 155 DSP02a
0 100 200 Time from array onset (ms)
1.0
(c) Easy (target in)
0.5
0.0
Hard (target in)
0 100 200 Time from array onset (ms)
F 250 DSP04a
Fig. 15.3 Illustration of two saccade decision tasks discussed in this article. (a) In a memory-guided saccade task, the monkey fixates a central point while a peripheral target is quickly flashed; the location of the target is guided by the receptive field properties of the isolated neuron for a given experimental session. The monkey is required to maintain fixation for 400–1000ms, after which the fixation spot disappears. To earn reward, the monkey must make a single saccade to the remembered location of the peripheral target. (b) In a visual search task, the monkey first maintains fixation on a central point. An array of visual objects is then presented and to earn reward the monkey must make a single saccade to the target object and not one of the distractor objects. In this case, the reward target was an L and the distractors were variously rotated Ts, with the particular reward target changed from session to session. Various experiments manipulated the number of distractors (set size), the similarity between targets and distractors, and the particular dimensions on which targets and distractors differed (shape, color, or motion).
324
Hard (target in)
0.0 (b)
Easy (target in)
1.0
–100 –50 0 From saccade (ms)
Fig. 15.4 Illustration of response properties of visually responsive and movement-related neurons in FEF (Hanes, Patterson, & Schall, 1998; Hanes & Schall, 1996; Purcell et al., 2010). Recordings were made while monkeys engaged in a visual search task where the target either appeared among dissimilar distractors (easy search) or among similar distractors (hard search). Plots display normalized spike rate as a function of time (ms). Visually responsive neuron activity aligned on visual search array onset time illustrated in panel (a), movement-related neuron activity aligned on visual search array onset time illustrated in panel (b), and movement-related neuron activity aligned on saccade time illustrated in panel (c). Solid lines are trials in which the target was in the visual neuron’s receptive field or movement neuron’s movement field (target in), and dashed lines are trials in which the target was outside the neurons’ response fields (target out). (Adapted from Purcell et al., 2010.)
activity eventually discriminates between target and distractor, with generally faster and more significant discrimination with easy compared to hard visual search trials (Bichot & Schall, 1999; Sato, Murthy, Thompson, & Schall, 2001) and small compared to large set sizes (Cohen, Heitz, Woodman, & Schall, 2009). We note that the particular shape of the trajectories taken to achieve this neural discrimination can be somewhat heterogeneous across different neurons, but virtually all visually responsive neurons discriminate target from distractor over time. We emphasize that this discrimination concerns the “targetness” of the object in the neuron’s receptive field, not particular features or dimensions of the object like its color or shape, except under unique circumstances (Bichot, Schall, & Thompson, 1996). Visually responsive neurons display these same characteristic dynamics regardless of whether a saccade is made, such as when the monkey withholds or cancels an eye movement
because of a stop signal (Hanes, Patterson, & Schall, 1998) or when the monkey is trained to maintain fixation and respond with a limb movement and not an eye movement (Thompson, Biscoe, & Sato, 2005). Normalized activity of a representative movementrelated neuron is shown aligned on the onset time of the visual search array (Figure 15.4b) and aligned on the time of the saccade (Figure 15.4c). When the monkey makes a saccade to the object in the receptive field (movement field) of the neuron, there is a characteristic buildup of activity some time after array onset; there is far less activity when the nonselected object is in the receptive field, although the precise nature of those dynamics varies somewhat from neuron to neuron. We see clearly that, when aligned on saccade initiation time, activity reaches a relatively constant threshold level immediately prior to the eye movement (Hanes & Schall, 1996), and this pattern of activity holds across search difficulty and set size (Woodman, Kang, Thompson, & Schall, 2008). Movement-related neuron activity does not reach threshold if the monkey withholds or cancels an eye movement because of a stop signal (Hanes et al., 1998; Murthy et al., 2009) or makes a response to the target using a limb movement and not an eye movement (Thompson, Biscoe, & Sato, 2005).We discuss more detailed aspects of the temporal dynamics of movement-related neurons later in this article. One of our primary goals has been to develop models that both predict the saccade behavior of the monkey and predict the temporal dynamics of movement-related neurons in FEF.
A Neural Locus of Drift Rates Movement-related neurons increase in spike rate over time and reach a constant level of activity immediately prior to a saccade being initiated (Figure 15.4). The dynamics of movement-related neurons appear consistent with the dynamics of models that assume a stochastic accumulation of perceptual evidence to a threshold (Hanes & Schall, 1996; Ratcliff et al., 2003; Schall, 2001; Smith & Ratcliff, 2004). This insight raises several questions that we have begun to address in our recent work: If movement-related neurons instantiate an accumulator model, what kind of accumulator model do they instantiate? What kind of an accumulator model can predict the fine-grained dynamics of movement-related neurons? What
drives the accumulator model? We begin with the last question. A broad class of models of perceptual decisionmaking assumes that perceptual evidence is accumulated over time to a threshold (Figure 15.2; see also Ratcliff & Smith, 2015). The rate at which perceptual evidence is accumulated, the drift rate, can vary across objects, conditions, and experience. When accumulator models are tested by fitting them to observed behavior, it is not uncommon to assume that different drift rates across different experimental conditions are free parameters that are optimized to maximize or minimize some fit statistic (e.g., Brown & Heathcote, 2008; Boucher et al., 2007a; Ratcliff & Rouder, 1998; Usher & McClelland, 2001). But other theoretical work has aimed to connect models of perceptual decision-making to models of perceptual processing by developing a theory of the drift rates. For example, Nosofsky and Palmeri (1997; Palmeri, 1997) proposed an exemplar-based random walk model (EBRW) that combined the generalized context model of categorization (Nosofsky, 1986) with the instance theory of automaticity (Logan, 1988) to develop a theory of the drift rates driving a stochastic accumulation of evidence. Briefly, EBRW assumes that a perceived object activates previously stored exemplars in visual memory, the probability and speed of exemplar retrieval is governed by similarity, and repeated exemplar retrievals determine the direction and rate of accumulation to a response threshold. EBRW predicts the effects of similarity, experience, and expertise on response probabilities and response times for perceptual decisions about visual categorization and recognition (see Nosofsky & Palmeri, 2015; Palmeri & Cottrell, 2009; Palmeri, Wong, & Gauthier, 2004). Other theorists have similarly connected visual perception and visual attention mechanisms to accumulator models of perceptual decision making by creating theories of drift rate (e.g., Ashby, 2000; Logan, 2002; Mack & Palmeri, 2010; Schneider & Logan, 2005; Smith & Ratcliff, 2009). As a first step toward a neural theory of drift rates, we recently proposed a neural locus of drift rates when decisions are made by saccades (Purcell et al., 2010, 2012). We hypothesize that the accumulation of evidence is reflected in the firing rate of FEF movement-related neurons and the perceptual evidence driving this accumulation is reflected in the firing rate of FEF visually responsive
neurocognitive modeling of perceptual decision making
325
Cell n
Target in RF
Cell 1 Cell 2
k θ 0 100 200 300 0
200
g
vT
mT 0 100 200 300
400 β
u Cell n
Distractor in RF θ
Cell 1 Cell 2
vD
g
mD 0 100 200 300
0 0
200
100 200 300
400
Fig. 15.5 Illustration of simulation model architectures tested in Purcell et al. (2010, 2012). Spike trains were recorded from FEF visually-responsive neurons during a saccade visual search task. Trials were sorted into two populations according to whether the target or a distractor was within the neuron’s response field. Spike trains were randomly sampled from each population to generate a normalized activation function that served as the dynamic model input associated with a target (vT ) and a distractor (vD ) on a given simulated trial, as illustrated. Different architectures for perceptual decision-making were systematically tested. Decision units (mT ) could integrate evidence or not, and they could be leaky (k) or not. Decision units could integrate a difference between the inputs (u) or not, the stochastic input could be gated (g) or not, and the units could compete with one another (β) or not. Here, only two decision units are shown, one for a target and one for a distractor. In Purcell et al. (2012) there were eight accumulators, one for each possible stimulus location in the visual search array.
neurons. One way to test this hypothesis would be to develop a model of the dynamics of visually responsive neurons, a model of how those dynamics are translated into drift rates, and then use those drift rates to drive a model of the accumulation of perceptual evidence. We chose a different approach. Rather than model the dynamics of visually responsive neurons, we used the observed firing rates of those neurons directly as a dynamic neural representation of the perceptual evidence that was accumulated over time. Figure 15.5 illustrates our general approach. Activity of visually responsive neurons was recorded from FEF of monkeys performing a visual search task. In Figure 15.4, we illustrate spike density functions of a representative neuron when a target or distractor appeared in its receptive field during easy or hard visual search. For our modeling, we did not use the mean activity of neurons as input but, instead, generated thousands of simulated spikedensity functions by subsampling from the full set of individually recorded trials of visually responsive neurons. Specifically, on each simulated trial, we first randomly sampled, with replacement, a set of spike trains recorded from individual neurons. We subsampled from trials when the target was 326
new directions
in the receptive fields of the neurons to simulate perceptual evidence in favor of the target location and trials when a distractor was in the receptive field to simulate perceptual evidence in favor of each of the distractor locations. Along its far left, Figure 15.5 illustrates raster plots for example neurons, with individual trials arranged sequentially along the y axis, time along the x axis, and each black dot indicating the incidence of a recorded spike on a given trial for that neuron. The gray thick bars illustrate a random sampling from those recorded neurons. These sampled spike trains were convolved with a temporally asymmetric doubly exponential function (Thompson, Hanes, Bichot, & Schall, 1996), averaged together, and normalized to create dynamic drift rates associated with target and distractor locations (Purcell et al., 2010, 2012), as illustrated in the middle of Figure 15.5; the resulting input functions are mathematically similar to a Poisson shot noise process (Smith, 2010). Different inputs were defined according to the experimental condition under which the visually responsive neurons were recorded on each trial, such as easy versus hard search or small versus large set sizes. Arguably, this approach allows the most direct test of whether the dynamics of visually responsive
neurons provide a sufficient representation of perceptual evidence to predict where and when the monkey moves its eyes. If no model can predict saccade behavior using visually responsive neurons as input, then some other neural signal must be significantly modulating behavior of the monkey. Furthermore, as illustrated by contrasting Figures 2a and 2b, this novel approach imposes significant constraints on possible models by replacing free parameters governing the mean and variability of perceptual processing time, starting point of accumulation, and drift with observed neurophysiology. Finally, because the neurophysiological signal from visually responsive neurons is continuous in time, the models cannot merely assume that perceptual processing and perceptual decisions constitute discrete stages, as typical for many accumulator models.
Architectures for Perceptual Decision-Making Within the broad class of perceptual decisionmaking models assuming an accumulation of perceptual evidence to a threshold, a variety of different model architectures have been proposed (e.g., see Ratcliff & Smith, 2004; Smith & Ratcliff, 2004). We instantiated several of these competing architectures, and using drift rates defined by the recorded spiking activity of visually responsive neurons as inputs, evaluated how well each could fit observed response probabilities and response times of monkeys making saccades during a visual search task (Purcell et al., 2010, 2012). Figure 15.5 illustrates the common architectural framework. Drift rates defined by neurophysiology constitute the input nodes labeled vT (target) and vD (distractor). We assume an accumulator associated with the target location (mT ) and distractor locations (mD ). Figure 15.5 shows only one target and one distractor accumulator (Purcell et al., 2010) but we have extended this framework to multiple accumulators, one for every possible target location in the visual field (Purcell et al., 2012). Each accumulator is governed by the following stochastic differential equation ⎡⎛ ⎞+ dt ⎣⎝ vi (t) − d mi (t) = uvj(t) − g ⎠ τ j =i ⎤ dt − βmk (t) − kmi (t)⎦ + ξ. τ k =i
The mi (t) are rectified to be greater than or equal to zero because we later compare the dynamics of these accumulators to the observed spike rates of movement-related neurons, and those spike rates are greater than zero by definition. ξ represents Gaussian noise intrinsic to each accumulator with mean 0 and standard deviation σ ; in all of our simulations, this intrinsic accumulator variability could be assumed to be quite small relative to the variability of the visual inputs vi (t). All accumulators, mi (t), are assumed to race against one another to be the first to reach their threshold θ . The winner of that race between accumulators determines which saccade response is made on that simulated trial and the response time is given by the time to reach threshold plus a small ballistic time of 10–20ms. If k > 0, these are leaky accumulators, otherwise they are perfect integrators. If β = 0 and u = 0, we have a version of a simple horse race model. If β > 0, these are competing accumulators, and combined with leakage, k > 0, we have the leaky competing accumulator model (Usher & McClelland, 2001). If u > 0, then weighted differences are accumulated by each mi (t). In the case of only two accumulators, one for a target and the other for a distractor, and assuming u = 1, both mi (t) accumulates the difference between evidence for a target versus evidence for a distractor, which is quite similar to a standard drift-diffusion model (see Bogacz et al., 2006; Ratcliff et al., 2007; Usher & McClelland, 2001), and when assuming positive leakage (k > 0) is quite similar to an Ornstein-Uhlenbeck process (Smith, 2010); this similarity can become mathematical identity with some added assumptions (Bogacz et al., 2006; Usher & McClelland, 2001). Finally, we also proposed a novel aspect to this general architecture, which we called a gated accumulator (Purcell et al., 2010, 2012). When g > 0 and the input is positive-rectified, as indicated by the + subscript in the equation, then only inputs that are sufficiently large can enter into the accumulation. For example, consider a gated accumulator assuming u > 0; this would mean that the differences in the evidence in favor of the target over the distractors must be sufficiently large before that differences will accumulate. Recall that we assumed that the inputs are defined by neurophysiology, which has no beginning or ending, apart from the birth or death of the organism. Intuitively, the gate forces the accumulators to accumulate signal, not merely noise, and noise is all that is present before
neurocognitive modeling of perceptual decision making
327
(a)
nonaccumulator
(b)
perfect accumulator
(c)
leaky accumulator
(d)
gated accumulator
1.0 P (RT < t)
easy 0.5 hard 0.0 100
200 300 400 Response Time (ms)
data model
Fig. 15.6 In Purcell et al. (2010), models (Figure 15.5) were tested on how well they could account for observed RT distributions of the onset of saccades in an easy visual search where the target and distractors were dissimilar or where the target and distractors were similar hard. Each panel shows observed cumulative RT distributions (symbols) for easy and hard search. Best-fitting model predictions for a subset of the models tested in Purcell et al. (2010) are shown for illustration, ranging left-to-right from a nonaccumulator model that does not integrate perceptual evidence over time, a perfect integrator model with no leakage, a leaky accumulator model, and a gated accumulator model. (Adapted from Purcell et al., 2010.)
perceptual processing has begun to discriminate targets from distractors. We evaluated the fits of competing model architectures to observed response probabilities and distributions of response times using standard model fitting techniques (e.g., Ratcliff & Tuerlinckx, 2002; Van Zandt, 2000). We systematically compared models assuming a horse race, a diffusion-like difference accumulation process, or competition via lateral inhibition, factorially combined with various leaky, nonleaky, or gated accumulators. For example, Figure 15.6 displays observed response time distributions for easy versus hard visual search along with a sample of predictions from some of the model architectures evaluated by Purcell et al. (2010); for these particular data (Bichot, Thompson, Rao, & Schall, 2001; Cohen et al., 2009), there were very few errors. As shown in the left two panels, models assuming nointegrationat all, meaning that the current value of mi (t) simply reflects the current inputs at time t, and models assuming perfect integration without leakage, provided a relatively poor fit to the observed behavioral data. Although these particular behavioral data were fairly limited, with only a response-time distribution for easy and hard visual search, we could rule out some potential model architectures. However, other competing models, including those with leakage or gate, assuming a competition or an accumulation of differences, all provided reasonable quantitative accounts of the behavioral data, a couple of examples of which are shown in the two right panels of Figure 15.6. Purcell et al. (2012) evaluated fits of these models to a more comprehensive dataset where set 328
new directions
size was systematically manipulated and where the search was difficult enough to produce significant errors (Cohen et al., 2009). Models were required to fit correct- and error-response probabilities as well as distributions of correct- and error-response times. These data are shown in Figure 15.7. Also shown are the predictions of the best fitting model, which was a gated accumulator model that assumed both significant leakage and competition via lateral inhibition. Likely because this dataset was larger, it also provided a greater challenge to other models, since many horse-race models and diffusion-like models failed to provide adequate fits to the observed data, whether they included leakage or gating (see Purcell et al., 2012). Just based on the quality of fits to observed data, models with leakage and competition via lateral inhibition provided comparable fits whether those models included gating or not in both Purcell et al. (2010) and Purcell et al. (2012). So based on parsimony, a nongated version, which is essentially a leaky competing accumulator model (Usher & McClelland, 2001), would win the theoretical competition. But our goal was also to test whether the accumulators in the competing models could provide a theoretical account of the movement-related neurons in FEF. To do that, we also tested whether the dynamics measured in the accumulators could predict the dynamics measured in movement-related neurons (see also Boucher et al., 2007a; Ratcliff et al., 2003, 2007).
Predicting Neural Dynamics Until now, the work we have described follows a long tradition of developing and testing computational and mathematical models of cognition.
(a)
(b)
100 Percent Correct
Mean RT (ms)
400 350 300 250 200
2
4
Error Data Model
80
60
8
Correct
2
4
8
Set Size
(c)
(d)
Correct
P(RT < t)
0.9
Set size 2 Set size 8
0.7
Error 0.9 0.7
Set size 4 0.5
0.5
0.3
0.3
0.1
0.1
100
300
500
100
300
500
Response time (ms) Fig. 15.7 In Purcell et al. (2012), models (Figure 15.5) were tested on how well they could account for correct- and error-response probabilities and correct- and error-response time distributions of saccades in a visual search task with three levels of set size: 2 (blue), 4 (green), or 8 (red) objects in the visual array. Predictions from the best-fitting gated accumulator model are shown. (a) Mean observed (symbols) and predicted (lines) correct- (solid) and error- (dashed) response times as a function of set size. (b) Mean observed (symbols) and predicted (lines) probability correct as a function of set size. (c) Observed (symbols) and predicted (lines) cumulative RT distributions of correct responses at each set size. (d) Observed (symbols) and predicted (lines) cumulative RT distributions of error responses at each set size. (Adapted from Purcell et al., 2012.)
Competing models are evaluated on their ability to predict behavioral data by optimizing parameters in order to maximize or minimize the fit of each model to the observed data, and then statistical tests are performed for nested or nonnested model comparison (e.g., see Busemeyer & Diederich, 2010; Lewandowsky & Farrell, 2010). We go beyond this approach to evaluate linking propositions (Schall, 2004; Teller, 1984) that aim to map particular cognitive model mechanisms onto observable neural dynamics. Specifically, we evaluate the linking proposition that movementrelated neurons in FEF instantiate an accumulation of evidence to a threshold. We do this by testing how well the simulated dynamics of accumulators in the various model architectures described in the previous section predict the observed dynamics in movement-related neurons. Although the qualitative relationship between accumulator dynamics and movement neuron dynamics has long been recognized (e.g., Hanes & Schall, 1996;
Ratcliff et al., 2003; Smith & Ratcliff, 2004), we go beyond noting qualitative relationships to test quantitative predictions. Following the approach used by Woodman et al. (2008), we evaluated how several key measures of neural dynamics varied according to the measured response time of a saccade. The top row of Figure 15.8 illustrates several hypotheses for how variability in response time is related to variability in the underlying neural dynamics. Fast responses could be associated with an early initial onset of the neural activity from baseline, whereas slow responses could be associated with a delayed onset. Alternatively, fast responses could be associated with high growth rate in spiking activity to threshold, whereas slow responses could be associated with low growth rate. Fast responses could be associated with an increased baseline firing rate or decreased threshold, whereas slow responses could be associated with a decreased baseline firing rate or increased threshold. To evaluate these proposals, the onset
neurocognitive modeling of perceptual decision making
329
time, growth rate, baseline, and threshold of neural activity were all measured within bins of trials defined by response times from fastest to slowest, both within conditions and across conditions (see Purcell et al., 2010, 2012, for details). The middle row shows the relationship between onset time, growth rate, baseline, and threshold of neural activity and mean response time for each bin of an RT distribution for a representative neuron in a representative condition.The bottom row shows the mean correlation of neural measures with RT as a function of set size from Purcell et al. (2012), with a significant relationship between onset time and response time observed in neural activity in movement-related neurons in FEF. Using analogous methods, we also measured the relationship between onset time, growth rate, baseline, and threshold of accumulator dynamics and response time predicted by each of the competing model architectures that we simulated. Shown in Figure 15.8 are the predictions of the gated accumulator model from Purcell et al. (2012), illustrating a good match between model and neurons. These are true model predictions, not model fits. After the model was fitted to behavioral data, the accumulator dynamics using the best-fitting model parameters were measured and compared directly with the observed neural dynamics. All other models failed to predict the observed neural dynamics. For example, models without gate typically predicted a significant negative correlation between baseline and response time that was completely absent in the observed data. Part of the reason for this is that, with nongated models, the accumulators are allowed to accumulate noise in the input defined by visually responsive neurons. Although a leakage term may be sufficient to keep a weak noise signal from leading to a premature accumulation to threshold, it cannot prevent significant differences in baseline activity from being correlated with differences in predicted response time when the accumulators reach threshold, at least without significantly compromising fits to the observed behavior.
Control over Perceptual Decisions We have also considered the neurophysiological basis of cognitive control over perceptual decisions. Mirroring our other research, we used cognitive models to better understand neural mechanisms and used neural data to constrain competing cognitive models. 330
new directions
Perhaps the most widely used task for studying normal and dysfunctional cognitive control is the stop-signal task (Lappin & Eriksen, 1966; Logan & Cowan, 1984). Saccade variants of this task have been used with monkeys, and neurophysiological activity has been recorded from neurons in FEF (Hanes et al., 1998). The basic stop-signal task with saccades is in certain ways a converse of the memory-guided saccade task illustrated in Figure 15.4. Monkeys initially fixate the center of the screen. After a variable amount of time, the fixation spot disappears and a peripheral target appears somewhere in the visual field, and the monkey must make a single saccade to the target in order to earn reward. This is the primary task, or go signal. On a fraction of trials, some time after the peripheral target appears, the fixation spot is reilluminated, and the monkey is rewarded for cancelling its saccade, maintaining fixation. This is the stop signal. The interval between the appearance of the go signal, the peripheral target, and the stop signal, the fixation point, is called stop signal delay (SSD). Monkeys’ ability to inhibit their saccade is probabilistic due to the stochastic variability of go and stop processes and depends on SSD. Figure 15.9 displays the key behavioral data observed in the saccade stop-signal paradigm (Hanes et al., 1998). Figure 15.9a displays the probability of responding to the go signal (y axis), despite the presence of a stop signal at a particular SSD (x axis). When the stop signal illuminates shortly after the appearance of the target, at a short SSD, the probability of responding to the go signal is quite small. Control over the saccade as a consequence of the stop signal has been successful. In contrast, for a long SSD, the probability of successfully inhibiting the saccade is rather small. Figure 15.9b displays distributions of response times for primary go trials with a stop signal (signal response trials), in which a saccade was erroneously made, shaded by gray according to SSD (see figure caption). These response times are significantly faster than response times without any stop signal (no-stop-signal trials) in black. Behavioral data in the stop-signal paradigm has long been accounted for by an independent race model (Logan & Cowan, 1984), which assumes that performance is the outcome of a race between a go process, responsible for initiating the movement, and a stop process, responsible for inhibiting the movement (see also Becker & Jürgens, 1979; Boucher, Stuphorn, Logan, Schall, & Palmeri, 2007b; Camalier et al., 2007;
Growth rate
200
Correlation with Response time
0
400 200 Respose time (ms)
1.0 **
**
***
0.0
–1.0
0.6
r = –0.58 p = 0.17
0.4 0.2 0.0
Threshold
40 r = –0.32 p = 0.48 Baseline (sp/s)
Growth rate (sp/s/ms)
Onset (ms)
400 r = 0.81 p < 0.05
Baseline
20
0
400 200 Respose time (ms)
1.0
1.0
0.0
0.0
r = 0.07 p = 0.87
40 Threshold (sp/s)
Onset
20
0
400 200 Respose time (ms)
400 200 Respose time (ms)
1.0 *
0.0
Data Model 2
4 Set size
8
–1.0
2
4 Set size
8
–1.0
2
4 Set size
8
–1.0
2
4 Set size
8
Fig. 15.8 Comparing observed neural dynamics and predicted model dynamics. Top row: Four possible hypotheses for how variability in RT is related to variability in neural or accumulator dynamics: from left to right, variability in RT could be correlated with variability in the onset time, growth rate, baseline, or threshold. Middle row: Following Woodman et al. (2008), correct RTs were binned in groups from fastest to slowest and within each bin the onset time, growth rate, baseline, and threshold of the spike density functions were calculated. The relationship between RT and neural measure (left to right: onset time, growth rate, baseline, and threshold) are shown for one representative neuron in set size 4 for one of the monkeys tested; the correlation between RT and neural measure and its associate p-value are also shown. Bottom row: Average correlation between RT and neural measure (left-to-right: onset time, growth rate, baseline, and threshold) as a function of set size observed in neural dynamics and predicted in model dynamics for the gated accumulator model. (Adapted from Purcell et al., 2012.)
(b)
Observed Model
0.5
0.0
SSD
0.5
0.0 50
100 150 200 Stop signal delay (ms)
0
250
100 200 Time from stimulus (ms)
(d)
1.0
Movement Neurons in: frontal eye field superior colliculus Model Simulation: Go Process
Normalized Proportion
P(RT < t)
0.4
0.5
0.0
SSRT
1.0
cancel time
(c)
1.0
Normalized Activation
Probability (signal-respond)
(a)
200
300 Reaction time (ms)
400
0.3 0.2 0.1 0.0
–50
0 Cancel time (ms)
50
Fig. 15.9 (a) Observed inhibition function(gray line) and simulated inhibition function from the interactive race model (black line). (b) Observed (thin lines) and simulated (thick lines) cumulative RT distributions from no stop signal (black line) and signal-response trials with progressively longer stop signal delays (progressively darker gray lines). (c) Illustration of simulated activity in the interactive race model of the go unit and stop unit activation on signal-inhibit (thick solid line) and latency-matched no-stop-signal trials (thin solid lines) with stop-signal delay (SSD) and stop-signal reaction time (SSRT) indicated. Cancel time is indicated by the downward arrow. (d) Histogram of cancel times of the go unit predicted by the interactive race model compared with the histogram of cancel times measured for movement-related neurons in FEF and SC.(Adapted from Boucher et al., 2007a.)
Logan, Van Zandt, Verbruggen, & Wagenmakers, 2014; Olman, 1973). Boucher et al. (2007a) addressed an apparent paradox of how seemingly interacting neurons in the brain could produce behavior that appears to be the outcome of independent processes. Mirroring the general model architectures described earlier and illustrated in the right half of Figure 15.5, they instantiated and tested models that assumed stochastic accumulators for the go process and for the stop process that were either an independent race or that assumed competitive, lateral interactions between stop and go. Outstanding fits to observed behavioral data for both the independent race model and the interactive race model were observed. Figures 9a and 9b show fits of the interactive race model, but fits of the independent race model were virtually identical. Parsimony would favor the independent race. But neural data favored the interactive race. In the absence of a stop signal, visually responsive neurons in FEF select the target, and movementrelated neurons in FEF increase their activity until 332
new directions
a threshold level is reached, shortly after which a saccade is made (Hanes & Schall, 1996), just as they do on memory-guided saccade tasks or visual search tasks. On trials with a stop signal, the dynamics of visually responsive neurons are unaffected (Hanes et al., 1998). For movementrelated neurons, we can distinguish between activity when a stop was successful, signal-inhibit trials, from activity when a stop was unsuccessful, that is signal-respond trials. On signal-respond trials, the activity of movement-related neurons is qualitatively the same as the activity on nosignal trials, with neurons reaching a threshold level before a saccade is made. Even more striking, the activity on signal-respond trials is quantitatively indistinguishable from activity on no-signal trials that are equated for response time (latency-matched trials). On signal-inhibit trials, the activity increases in a manner indistinguishable from latencymatched no-signal trials until some time after the SSD, at which point the activity of movementrelated neurons is reduced back to baseline without
reaching the threshold. The saccade has been inhibited. Figure 15.9c displays the predicted accumulator dynamics of the interactive race model (Boucher et al., 2007a). The dynamics of the go accumulator in the interactive race precisely mirrors the description of the dynamics of movement-related neurons provided earlier, with dynamics not observed in the independent race model. For signal-inhibit trials and latency-matched no-signal trials, activity increases for some time after SSD, after which activity on signal-inhibit trials returns to baseline while activity on latency-matched no-signal trials continues to threshold. The accumulator dynamics in the interactive race model qualitatively captures the neural dynamics of movement-related neurons. But we could go further than that. We also calculated a metric called cancel time (Hanes et al., 1998), which is a function of the time at which the dynamics statistically diverge between signal-inhibit trials and latency-matched no-signal trials. This time can be calculated from movementrelated neurons. It can also be calculated from accumulator dynamics. And as shown in Figure 15.9b, these measures from neurons and the model nicely converge. We emphasize that, as was the case for Purcell et al. (2010, 2012), these are true model predictions. Boucher et al. (2007a) fitted models to behavioral data, then calculated the cancel time predicted by the models, and compared that to the observed cancel time in neurons. Parameters were not adjusted to maximize the correspondence. The hypothesized locus of control in Boucher et al. (2007a) is inhibition of a stop process on the go process, with the stop process identified as activity of fixation-related neurons and the go process identified as activity of movement-related neurons. The gate in the gated accumulator model (Purcell et al., 2010, 2012) could be another hypothesized locus of control over perceptual decisions. In recent work, we have suggested that blocking the input to the go unit, rather than actively inhibiting it via a stop unit, could be an alternative mechanism for stopping. Indeed, a blocked input model predicted observed data and distributions of cancel times at least as well as the interactive race model (Logan, Schall, & Palmeri, 2015; Logan, Yamaguchi, Schall, & Palmeri, in press). One suggestion we made was that the stop process could raise a gate between visual neurons that select the target and movement neurons that generate a movement to it, blocking input to the
movement neurons and thereby preventing them from reaching threshold. As another example, in a stop-signal task, both humans and monkeys adapt their performance from trial to trial, for example, producing longer RTs after successfully inhibiting a planned movement (e.g., Bissett & Logan, 2011; Nelson, Boucher, Logan, Palmeri, & Schall, 2010; Verbruggen & Logan, 2008). For monkeys, within FEF, activity of visually responsive neurons are unaffected by these trial-to-trial adjustments, but the onset time of activity of movement-related neurons is significantly delayed (Pouget et al., 2011). Purcell et al. (2012) suggested that strategic adjustment in the level of the gate could explain the delayed onset of movement-related neurons in the absence of any modulation of visually responsive neurons. Moreover, they demonstrated that this strategic adjustment of gate could be couched in terms of optimality. It has been previously suggested that strategic modulation of accumulator threshold could maximize reward rate, which is defined as the proportion of correct responses per unit time (e.g., Gold & Shadlen, 2002; Lo & Wang, 2006). We observed that strategic modulation of the level of the gate could maximize reward rate in much the same way (Purcell et al., 2012).
Summary and Conclusions Here we reviewed some of our contributions to a growing synergy of mathematical psychology and systems neuroscience. Our starting point has been a class of successful cognitive models of perceptual decision-making that assume a stochastic accumulation of perceptual evidence to a threshold over time (Figure 15.2). Models of this sort have long provided excellent accounts of response probabilities and distributions of response times in a wide range of perceptual decision-making tasks and manipulations (e.g., see Nosofsky & Palmeri, 2015; Ratcliff & Smith, 2015). We have extended these models to account for response probabilities and distributions of response times for awake behaving monkeys to make saccades to target objects in their visual field (Boucher et al., 2007a; Pouget et al., 2011; Purcell et al., 2010, 2012). Applying techniques common to mathematical psychology, we instantiated different model architectures and ruled out models that provided poor fits to observed data. These models have free parameters that govern theoretical quantities like perceptual processing
neurocognitive modeling of perceptual decision making
333
time, the starting point of accumulation, the drift rate of accumulation, and the response threshold. We constrained many of these parameters using neurophysiology. Unlike some approaches that constrain parameters values based on neurophysiology, often based on neural findings with rather large confidence intervals, we replaced parameterized model assumptions directly with recorded neurophysiology. Specifically, we sampled from neural activity recorded from visually responsive neurons in FEF, feeding these spike trains directly into stochastic accumulator models, thereby creating a largely nonparametric neural theory of perceptual processing time and the drift rate of accumulation. Not only did this approach constrain computational modeling, it also provided a direct test of the hypothesis that the activity of visually responsive neurons in FEF encodes perceptual evidence: This neural code can be accumulated over time to predict where and when the monkey moves its eyes (Purcell et al., 2010, 2012). We also tested the hypothesis that movementrelated neurons in FEF instantiate a stochastic accumulation of evidence. Although it has long been acknowledged that these neurons behave in a way consistent with accumulator models (e.g., Hanes & Schall, 1996; Schall, 2001), we went beyond qualitative description to test whether movement neuron dynamics can be quantitatively predicted by accumulator model dynamics. We measured how the onset of activity, baseline activity, rate of growth, and threshold varies with behavioral response time in both movement-related neurons and model accumulators, and we found close correspondences for some models. Not only does this test an hypothesis about the theoretical role of FEF movement-related neurons in perceptual decision-making, it also provides a powerful means of contrasting models that otherwise make indistinguishable behavioral predictions. Our gated accumulator model, which enforces accumulation of discriminative neural signals from visually responsive neurons, not only accounted for the detailed saccade behavior of monkeys, but also predicted quantitatively the dynamics observed in movement-related neurons in FEF, whereas other models could not (Purcell et al., 2010, 2012; see also Boucher et al., 2007a). This gated accumulator model also suggests a potential locus of cognitive control over perceptual decisions. Increasing the gate may account for speed-accuracy tradeoffs (Purcell et al., 2012) as well as stopping behavior and trial history effects described by 334
new directions
Boucher et al. (2007a) and Pouget et al. (2011), respectively. Turning to more general issues, our work has confronted a common challenge in the development of mathematical and computational models of cognition where competing models reach a point where they make very similar predictions, examples of which are discussed in other chapters in this volume (Busemeyer, Wang, Townsend & Eidels, 2015). This could be a consequence of true mimicry, where models assuming vastly different mechanisms nonetheless produce mathematically identical predictions that cannot be distinguished behaviorally. Often, however, it is that the current corpus of experimental manipulations and measures are insufficient to discriminate between competing models. Cognitive modelers have long turned to predicting additional complexity in behavioral data to resolve mimicry, going from predicting accuracy alone to predicting response probabilities as well as response times, and from predicting mean response-times to predicting response, time distributions, including those for correct and error responses. Indeed, in our work reviewed here, predicting jointly response probabilities and response time distributions yielded considerable traction in discriminating between competing models. Unfortunately, outside the mathematical psychology community, it is not uncommon to hear researchers state with complete confidence that response time distributions yield no more useful information than response time means, sadly unknowledgeable about the state of reality (e.g., see Townsend, 1990). That said, recognition is emerging, for example, that response time distributions are key aspects of data that theories of visual cognition needs to account for (e.g., Palmer, Horowitz, Torralba, & Wolfe, 2011; Wolfe, Palmer, & Horowitz, 2010), that response time distributions provide challenging constraints for low-level spiking neural models (e.g., Lo, Boucher, Paré, Schall, & Wang, 2009), and more generally that considerations of behavioral variability can yield insights into neural processes (e.g., Churchland et al., 2011; Purcell, Heitz, Cohen, & Schall, 2012). But even joint modeling of response probabilities and response-time distributions may be insufficient to contrast competing models. Our work illustrates how neurophysiological data can also help distinguish between models. We have described cases in which two models fit behavioral data equally well (Boucher et al., 2007a;
Purcell et al., 2010, 2012) but one model is more complex than the other. With only behavioral data and an appeal to parsimony, we would have demanded the exclusion of the more complex model in favor of the simpler one. However, in order to successfully mapobserved neural dynamics onto predicted model dynamics, the assumptions of the more complex model were required. Key here is that we believe that it is the important to map between neural dynamics and model dynamics, not between neural dynamics and model parameters (see also e.g., Davis, Love, & Preston, 2012). Variation in model parameters need not uniquely map onto variation in neural dynamics, but predicted variation in model dynamics must. And while we have demonstrated the theoretical usefulness of neural data in adjudicating between competing models, we do not believe that neural data has any particular empirical primacy. Just as mimicry issues can emerge when examining behavioral measures like accuracy and response time, analogous mimicry issues may be found at the level of neurophysiology and neural dynamics. Neural data are not necessarily more intrinsically informative than behavioral data, but more data provides additional constraints for distinguishing between competing models. More generally, our work allies with a growing body of research supporting accumulator models of perceptual decision making (e.g., Nosofsky & Palmeri, 1997; Ratcliff & Rouder, 1998; Ratcliff & Smith, 2004; Usher & McClelland, 2001), not just as models that explain behavior but also as models that explain brain activity measured using neurophysiology (e.g., Boucher et al., 2007; Churchland & Ditterich, 2012; Purcell et al., 2010, 2012; Ratcliff et al., 2003; but see Heitz & Schall, 2012, 2013), EEG (e.g., Philiastides, Ratcliff, & Sajda, 2006), and fMRI (e.g., Turner et al., 2013; van Maanen et al., 2011; White, Mumford, & Poldrack, 2012). The relative simplicity of cognitive models like accumulator models is a virtue in that they are computationally tractable, making them easily applicable across a wide range of phenomena and levels of analysis. Making explicit links to brain mechanisms does expose complexities. Our focus here has been largely on FEF, but other brain areas have neurons with dynamics that are visually responsive or movementrelated, including SC (Hanes & Wurtz, 2001; Paré & Hanes, 2003) and LIP (Gold & Shadlen, 2007; Mazurek et al., 2003; Shadlen & Newsome, 2001). Compared to the relative simplicity of most
Box 1 Top-down versus Bottom-up Theoretical Approaches Computational cognitive neuroscience aims to understand the relationship between brain and behavior using computational and mathematical models of cognition. One approach is bottom up. Theorists begin with fairly detailed mathematical models of neurons based on current understanding of cellular and molecular neurobiology. A common approach is to develop and test a single model of a neural network built up from these detailed models of neurons along with hypotheses about their excitatory and inhibitory connectivity. Although these neural models provide excellent accounts of spiking and receptor dynamics of individual neurons and may also account well for emergent network activity, they may provide only fairly coarse accounts of observed behavior, have somewhat limited generalizability, and be impractical to rigorously simulate and evaluate quantitatively. Another approach is top down (e.g., Forstman et al., 2011; Palmeri, 2014). Cognitive models account for details of behavior across multiple conditions, have significant generalizability across tasks and subject populations, and are often relatively easy to simulate and evaluate. It is common to evaluate multiple competing models and to test the necessity and sufficiency of model assumption with nested model comparison techniques. Although these models do not provide the same level of detailed predictions of spiking and receptor dynamics, they can provide predictions about the temporal dynamics of neural activity at the same level of precision as commonly summarized in neurophysiological investigations, as we illustrated in our review. In fact, Carandini (2012) suggested that bridging between brain and behavior can only be done by considering intermediate-level theories, that the gap between low-level neural models and behavior is simply a “bridge too far.” Although he considered linear filtering and divisive normalization as example computations that may be carried out across cortex (Carandini & Heeger, 2011), we consider accumulation of evidence as a similar computation that may be carried out in various brain areas, included FEF. These computations can simultaneously explain behavioral and neural dynamics.
neurocognitive modeling of perceptual decision making
335
stochastic accumulator models, there isa network of brain areas involved in evidence accumulations for perceptual decision making (Gold & Shadlen, 2007; Heekeren et al., 2008; Schall, 2001; 2004). Such mechanisms involving accumulation of evidence for perceptual decision-making may be replicated across different sensory and effector systems in the brain, such as those for visually guided saccades, but there may also be domaingeneral mechanisms as well (e.g., Ho, Brown, & Serences, 2009). Although the dynamics of specific individual neurons within particular brain areas mirror the dynamics of accumulators in models, we also know that, within any given brain area, ensembles of tens of thousands of neurons are involved in the generation of any perceptual decision. We need to understand the scaling relations from simple accumulator models to complex ensembles of thousands of neural accumulators (Zandbelt, Purcell, Palmeri, Logan, & Schall, 2014) and how to map the relatively few parameters that define simple accumulator models onto the great number of parameters that define complex neural dynamics (Umakantha, Purcell, & Palmeri, 2014).
Acknowledgments This work was supported by NIH R01EY021833, NSF Temporal Dynamics of Learning Center SMA-1041755, NIH R01-MH55806, NIH R01-EY008890, NIH P30-EY08126, NIH P30-HD015052, and by Robin and Richard Patton through the E. Bronson Ingram Chair in Neuroscience. Address correspondences to Thomas J. Palmeri, Department of Psychology, Vanderbilt University, Nashville TN 37203. Electronic mail may be addressed to thomas.j.palmeri@ vanderbilt.edu.
Glossary drift rate: The mean rate of perceptual evidence accumulation in a stochastic accumulator model of perceptual decision-making. frontal eye field: An area of prefrontal cortex that governs whether, where, and when the eyes moves to a new location in the visual field. gated accumulator: A stochastic accumulator model that includes a gate that enforces accumulation of discriminative neural signals, a model which quantitatively accounts for both behavioral and neural dynamics of saccadic eye movement. leakage: A weighted self-inhibition on the accumulation of
336
new directions
perceptual evidence, turning a perfect integrator of perceptual evidence into a leaky integrator of perceptual evidence. movement-related neurons: Neurons in FEF that show little or no modulation to the appearance of the target in the visual field but pronounced growth of spike rate immediately preceding the production of a saccade. perceptual decision-making: Perceptual decision-making requires representing the world with respect to current task goals and using perceptual evidence to inform the selection of a particular action. saccade: A ballistic eye movement of some angle and velocity to a particular location in the visual field. stochastic accumulator model: A class of computational models that assume that noisy perceptual evidence is accumulated over time from a starting point to a threshold, allowing predictions of both response probabilities and distributions of response times. stop-signal task: A classic cognitive control paradigm in which a primary go task is occasionally interrupted with a stop signal. visually responsive neurons: Visually responsive neurons are neurons in FEF that respond to the appearance of an object in their receptive field relative to that object’s salience with respect to current task goals but show little or no change in activity prior to the onset of a saccade
References Ashby, F. G. (2000). A stochastic version of general recognition theory. Journal of Mathematical Psychology, 44, 310–329. Becker, W., & Jürgens, R. (1979). An analysis of the saccadic system by means of double step stimuli. Vision Research, 19, 976–983. Bichot, N. P., & Schall. J. D. (1999). Effects of similarity and history on neural mechanisms of visual selection. Nature Neuroscience, 2, 549–554. Bichot, N. P., Schall, J. D., & Thompson, K. G. (1996). Visual feature selectivity in frontal eye fields induced by experience in mature macaques. Nature, 381, 697–699. Bichot, N. P., Thompson, K. G., Rao, S. C., & Schall, J. D. (2001). Reliability of macaque frontal eye field neurons signaling saccade targets during visual search. Journal of Neuroscience, 21, 713–725. Bissett, P. G., & Logan, G. D. (2011). Balancing cognitive demands: Control adjustments in the stop-signal paradigm. Journal of Experimental Psychology: Learning, Memory and Cognition, 37, 392–404. Bogacz, R., Brown, E., Moehlis, J., Holmes, P., & Cohen, J. D. (2006). The physics of optimal decision making: A formal analysis of models of performance in two-alternative forcedchoice tasks. Psychological Review, 113, 700–765. Boucher, L., Palmeri, T. J., Logan, G. D., & Schall, J. D. (2007a). Inhibitory control in mind and brain: An interactive race model of countermanding saccades. Psychological Review, 114, 376–397.
Boucher, L., Stuphorn, V., Logan, G. D., Schall, J. D., & Palmeri, T. J. (2007b). Stopping eye and hand movements: Are the processes independent? Perception & Psychophysics, 69, 785–801. Brown, S. D., & Heathcote, A. (2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178. Bruce, C. J., & Goldberg, M. E. (1985). Primate frontal eye fields. I. Single neurons discharging before saccades. Journal of Neurophysiology, 53, 603–635. Bruce, C. J., Goldberg, M. E., Bushnell, M. C., & Stanton, G. B. (1985). Primate frontal eye fields: II. Physiological and anatomical correlates of electrically evoked eye movements. Journal of Neurophysiology, 54, 714–734. Busemeyer, J. R., & Diederich, A. (2010). Cognitive modeling. Thousand Oaks, CA: Sage Publications. Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review, 100(3), 432–459. Busemeyer, J. R., Wang, Z., Townsend, J. T., & Eidels A. (2015). Mathematical and computational models of cognition. Oxford, UK: Oxford University Press. Camalier, C. R., Gotler, A., Murthy, A., Thompson, K. G., Logan, G. D., Palmeri, T. J., & Schall, J. D. (2007). Dynamics of saccade target selection: Race model analyses of double step and search step saccade production in human and macaque. Vision Research, 47, 2187–2211. Carandini, M. (2012). From circuits to behavior: A bridge too far? Nature Neuroscience, 15, 507–509. Carandini, A. K., & Heeger, D. J. (2012). Normalization as a canonical neural computation. Nature Reviews Neuroscience, 13, 51–62. Churchland, A. K., & Ditterich, J. (2012). New advances in understanding decisions among multiple alternatives. Current Opinion in Neurobiology, 22(6), 920–926. Churchland, A. K., Kiani, R., Chaudhuri, R., Wang, X. J., Pouget, A., & Shadlen, M. N. (2011). Variance as a signature of neural computations during decision making. Neuron, 69(4), 818–831. Cisek, P., Puskas, G. A., & El-Murr, S. (2009). Decisions in changing conditions: The urgency-gating model. Journal of Neuroscience, 29(3), 11560–11571. Cohen, J. Y., Heitz, R. P., Woodman, G. F., Schall, J. D. (2009). Neural basis of the set-size effect in frontal eye field: Timing of attention during visual search. Journal of Neurophysiology, 101, 1699–1704. Dayan, P., & Daw, N. D. (2008). Decision theory, reinforcement learning, and the brain. Cognitive, Affective, & Behavioral Neuroscience, 8(4), 429–453. Davis, T., Love, B. C., & Preston, A. R. (2012). Learning the exception to the rule: Model-based fMRI reveals specialized representations for surprising category members. Cerebral Cortex, 22, 260–273. Ditterich, J. (2006). Stochastic models of decisions about motion direction: Behavior and physiology. Neural Networks, 19, 981–1012. Ditterich, J. (2010). A comparison between mechanisms of multi-alternative perceptual decision making: Ability to
explain human behavior, predictions for neurophysiology, and relationship with decision theory. Frontiers in Decision Neuroscience, 4. Ferrier, D. (1874). The localization of function in brain. Proceedings of the Royal Society of London, 22, 229–232. Forstmann, B. U., Wagenmakers, E. J., Eichele, T., Brown, S., & Serences, J. T. (2011). Reciprocal relations between cognitive neuroscience and formal cognitive models: opposites attract? Trends in Cognitive Sciences, 15(6), 272–279. Glimcher, P. W., & Rustichini, A. (2004). Neuroeconomics: The consilience of brain and decision. Science, 306 (5695), 447– 452. Gilchrist, I. D. (2011). Saccades. In S. P. Liversedge, I. P. Gilchrist, & S. Everling (Eds.), Oxford Handbook on Eye Movements, (pp. 85–94). Oxford, UK: Oxford University Press. Gold, J. I., & Shadlen, M. N. (2002). Banburismus and the brain: Decoding the relationship between sensory stimuli, decisions, and reward. Neuron, 36, 299–308. Gold, J. I., & Shadlen, M.N. (2007). The neural basis of decision making. Annual Review of Neuroscience, 30, 535– 560. Goldman-Rakic, P. S., & Porrino, L. J. (1985). The primate mediodorsal (MD) nucleus and its projection to the frontal lobe. Journal of Comparative Neurology, 242, 535–560. Grossberg, S. (1976). Adaptive pattern classification and universal recoding: II. Feedback, expectation, olfaction, illusions. Biological Cybernetics, 23, 187–202. Hanes, D. P., Patterson, W. F., II, & Schall, J. D. (1998). Role of frontal eye fields in countermanding saccades: Visual, movement, and fixation activity. Journal of Neurophysiology, 79, 817–834. Hanes, D. P., & Schall, J. D. (1996). Neural control of voluntary movement initiation. Science, 274, 427–430. Hanes, D. P., & Wurtz, R. H. (2001). Interaction of the frontal eye field and superior colliculus for saccade generation. Journal of Neurophysiology, 85(2), 804–815. Heekeren, H. R., Marrett, S., & Ungerleider, L. G. (2008). The neural systems that mediate human perceptual decision making. Nature Reviews Neuroscience, 9(6), 467–479. Heitz, R. P., & Schall, J. D. (2012) Neural mechanisms of speedaccuracy tradeoff. Neuron, 76, 616–628. Heitz, R. P., & Schall, J. D. (2013). Neural chronometry and coherency across speed-accuracy demands reveal lack of homomorphism between computational and neural mechanisms of evidence accumulation. Philosophical Transactions of the Royal Society of London B, 368, 20130071. Hikosaka, O., & Wurtz, R. H. (1983). Visual and oculomotor functions of monkey substantia nigra pars reticulata: IV. Relation of substantia nigra to superior colliculus. Journal of Neurophysiology, 49, 1285–1301. Ho, T. C., Brown, S., & Serences, J. T. (2009). Domain general mechanisms of perceptual decision making in human cortex. The Journal of Neuroscience, 29(27), 8675–8687. Jones, M., & Dzhafarov, E. N. (2014). Unfalsifiability of major modeling schemes for choice reaction time. Psychological Review, 121, 1–32. Kahneman, D., & Tversky, A. (1984). Choices, values, and frames. American Psychologist, 39(4), 341–350.
neurocognitive modeling of perceptual decision making
337
Laming, D. R. J. (1968). Information theory of choice-reaction times. New York, NY: Academic. Lappin, J. S., & Eriksen, C. W. (1966). Use of a delayed signal to stop a visual reaction–time response. Journal of Experimental Psychology, 72, 805–811. Lewandowsky, S., & Farrell, S. (2010). Computational modeling in cognition: principles and practice. Thousand Oaks, CA: Sage. Link, S. W. (1992). The wave theory of difference and similarity. Hillsdale, NJ: Erlbaum. Lo, C.-C., Boucher, L., Paré, M., Schall, J. D., & Wang, X.-J. (2009). Proactive inhibitory control and attractor dynamics in countermanding action: A spiking neural circuit model. Journal of Neuroscience, 29, 9059–9071. Lo, C.-C., & Wang, X. J. (2006). Cortico–basal ganglia circuit mechanism for a decision threshold in reaction time tasks. Nature Neuroscience, 9, 956–963. Logan, G. D. (1988). Toward an instance theory of automatization. Psychological Review, 95, 492–527. Logan, G. D. (2002). An instance theory of attention and memory. Psychological Review, 109, 376. Logan, G. D. & Cowan, W. B. (1984). On the ability to inhibit thought and action: A theory of an act of control. Psychological Review, 91, 295–327. Logan, G. D., & Gordon, R. D. (2001). Executive control of visual attention in dual-task situations. Psychological Review, 108, 393–434. Logan, G. D., Schall, J. D., & Palmeri, T. J. (2015). Neural models of stopping and going. Manuscript in preparation. To appear in B. Forstmann & E. J. Wagenmakers (Eds.), An introduction to model-based cognitive neuroscience. Springer Neuroscience. Logan, G. D., Van Zandt, T., Verbruggen, F., & Wagenmakers, E.-J. (2014). On the ability to inhibit thought and action: General and special theories of an act of control. Psychological Review, 121(1), 66–95. Logan, G. D., Yamaguchi, M., Schall, G. D., & Palmeri, T. J. (in press). Inhibitory control in mind and brain 2.0: A blocked-input model of saccadic countermanding, psychological review. Mack, M. L., & Palmeri, T. J. (2010). Modeling categorization of scenes containing consistent versus inconsistent objects. Journal of Vision, 10(3):11, 1–11. Mack, M. L., & Palmeri, T. J. (2011). The timing of visual object categorization. Frontiers in Perception Science. Mazurek, M. E., Roitman, J. D., Ditterich, J., & Shadlen, M. N. (2003). A role for neural integrators in perceptual decision making. Cerebral Cortex, 13, 1257–1269. Munoz, D. P., & Schall, J. D. (2004). Concurrent, distributed control of saccade initiation in the frontal eye field and superior colliculus. In W.C. Hall & A. Moschovakis, (Eds.), The superior colliculus: New approaches for studying sensorimotor integration (pp. 55–82). Boca Raton, FL: CRC Press. Murthy, A., Ray, S., Shorter, S. M., Schall, J. D., & Thompson, K. G. (2009). Neural control of visual search by frontal eye field: effects of unexpected target displacement on visual selection and saccade preparation. Journal of Neurophysiology, 101(5), 2485–2506.
338
new directions
Nelson, M. J., Boucher, L., Logan, G. D., Palmeri, T. J., Schall, J. D. (2010). Impact of nonstationary response time in stopping and stepping saccade tasks. Attention, Perception, & Performance, 72, 1913–1929. Nosofsky, R. M. (1986). Attention, similarity, and the identification-categorization relationship. Journal of Experimental Psychology: General, 115, 39–57. Nosofsky, R. M., & Palmeri, T. J. (1997). An exemplar-based random walk model of speeded classification. Psychological Review, 104, 266–299. Nosofsky, R. M., & Palmeri, T. J. (2015). Exemplar-based random walk model. In J. R. Busemeyer, Z. Wang, J. T. Townsend, & A. Eidels (Eds.), Mathematical and computational models of cognition. Oxford University Press. Palmer, E. M., Horowitz, T. S., Torralba, A., & Wolfe, J. M. (2011). What are the shapes of response time distributions in visual search? Journal of Experimental Psychology: Human Perception and Performance, 37, 58. Palmeri, T. J. (1997). Exemplar similarity and the development of automaticity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 324–354. Palmeri, T. J. (2014). An exemplar of model-based cognitive neuroscience. Trends in Cognitive Science, 18(2), 67–69. Palmeri, T. J., & Cottrell, G. (2009). Modeling perceptual expertise. In I. Gauthier, M. Tarr, & D. Bub (Eds.), Perceptual expertise: bridging brain and behavior. Oxford, UK: Oxford University Press. Palmeri, T. J., & Tarr, M. (2008). Visual object perception and long-term memory. In S. Luck & A. Hollingworth (Eds.), Visual Memory, (pp. 163–207). Oxford, UK: Oxford University Press. Palmeri, T. J., Wong, A. C.-N., & Gauthier, I. (2004). Computational approaches to the development of perceptual expertise. Trends in Cognitive Sciences, 8, 378–386. Paré, M., & Hanes, D. P. (2003). Controlled movement processing: superior colliculus activity associated with countermanded saccades. Journal of Neuroscience, 23(16), 6480– 6489. Philiastides, M. G., Ratcliff, R., & Sajda, P. (2006). Neural representation of task difficulty and decision making during perceptual categorization: a timing diagram. Journal of Neuroscience, 26 (35), 8965–8975. Pouget, P., Logan, G. D., Palmeri, T. J., Boucher, L., Paré, M., & Schall, J. D. (2011). Neural basis of adaptive response time adjustment during saccade countermanding. Journal of Neuroscience, 31(35), 12604–12612. Pouget, P., Stepniewska, I., Crowder, E. A., Leslie, M. W., Emeric, E. E., Nelson, M. J., & Schall, J. D. (2009). Visual and motor connectivity and the distribution of calciumbinding proteins in macaque frontal eye field: Implications for saccade target selection. Frontiers in Neuroanatomy, 3, 2. Purcell, B. A., Heitz, R. P., Cohen, J. Y., & Schall, J. D. (2012). Response variability of frontal eye field neurons modulates with sensory input and saccade preparation but not visual search salience. Journal of Neurophysiology. 108, 2737–2750 Purcell, B. A., Heitz, R. P., Cohen, J. Y., Schall, J. D., Logan, G. D., & Palmeri, T. J. (2010). Neurally constrained modeling of perceptual decision making. Psychological Review, 117, 1113–1143.
Purcell, B. A., Schall, J. D., Logan, G. D., & Palmeri, T. J. (2012). From salience to saccades: Multiple-alternative gated stochastic accumulator model of visual search. Journal of Neuroscience, 32, 3433–3446. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108. Ratcliff, R. (2013). Parameter variability and distributional assumptions in the diffusion model. Psychological Review, 120, 281–292. Ratcliff, R., Cherian, A., & Segraves, M. (2003). A comparison of macaque behavior and superior colliculus neuronal activity to predictions from models of two-choice decisions. Journal of Neurophysiology, 90, 1392–1407. Ratcliff, R., Hasegawa, Y. T., Hasegawa, R. P., Smith, P. L., & Segraves, M. A. (2007). Dual diffusion model for single-cell recording data from the superior colliculus in a brightness-discrimination task. Journal of Neurophysiology, 97, 1756–1774. Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9, 347–356. Ratcliff, R., & Smith, P. L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111, 333–367. Ratcliff, R., & Smith, P.L. (2015). Modeling simple decisions and applications using a diffusion model. In J. R. Busemeyer, Z. Wang, J. T. Townsend, & A. Eidels (Eds.), Mathematical and computational models of cognition, Oxford University Press. Ratcliff, R., & Tuerlinckx, F. (2002). Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability. Psychonomic Bulletin & Review, 9, 438–481. Rosenbaum, D. A. (2009). Human motor control. (2nd ed.). New York, NY: Academic. Sato, T., Murthy, A., Thompson, K. G., & Schall, J. D. (2001). Search efficiency but not response interference affects visual selection in frontal eye field. Neuron, 30, 583–591. Sato, T., & Schall, J. D. (2003). Effects of stimulus-response compatibility on neural selection in frontal eye field. Neuron, 38(4), 637–648. Schall, J. D. (2001). Neural basis of deciding, choosing and acting. Nature Reviews Neuroscience, 2, 33–42. Schall, J. D. (2004). On building a bridge between brain and behavior. Annual Review of Psychology, 55, 23–50. Schall, J. D., & Cohen, J. Y. (2011). The neural basis of saccade target selection. In S. P. Liversedge, I. P. Gilchrist, & S. Everling (Eds.). Oxford handbook on eye movements. Oxford, UK: Oxford University Press. Schall, J. D., Morel, A., King, D., & Bullier, J. (1995). Topography of visual cortex connections with frontal eye field in macaque: Convergence and segregation of processing streams. Journal of Neuroscience, 15, 4464–4487. Schneider, D. W., & Logan, G. D. (2005). Modeling task switching without switching tasks: A short-term priming account of explicitly cued performance. Journal of Experimental Psychology: General, 134, 343–367. Schneider, D. W., & Logan, G. D. (2009). Selecting a response in task switching: Testing a model of compound cue retrieval. Journal of Experimental Psychology: Learning, Memory and Cognition, 35, 122–136.
Scudder, C. A., Kaneko, C. S., & Fuchs, A. F. (2002). The brainstem burst generator for saccadic eye movements: A modern synthesis. Experimental Brain Research, 142, 439– 462. Shadlen, M. N., & Newsome, W. T. (2001). Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. Journal of Neurophysiology, 86 (4), 1916–1936. Smith, P. L. (2010). From Poisson shot noise to the integrated Ornstein-Uhlenbeck process: Neurally principled models of information accumulation in decision-making and response time. Journal of Mathematical Psychology, 54, 266–283. Smith, P. L., & Ratcliff, R. (2004). Psychology and neurobiology of simple decisions. Trends in Neuroscience, 27, 161–168. Smith, P. L., & Ratcliff, R. (2009). An integrated theory of attention and decision making in visual signal detection. Psychological Review, 116, 283. Smith, P. L., & Van Zandt, T. (2000). Time-dependent Poisson counter models of response latency in simple judgment. British Journal of Mathematical & Statistical Psychology, 53, 293–315. Sparks, D. L. (2002). The brainstem control of saccadic eye movements. Nature Reviews Neuroscience, 3, 952–964. Stanton, G. B., Bruce, C. J., & Goldberg, M. E. (1995). Topography of projections to posterior cortical areas from the macaque frontal eye fields. Journal of Comparative Neurology, 353, 291–305. Teller, D. Y. (1984). Linking propositions. Vision Research, 24, 1233–1246. Thompson, K. G., Biscoe, K. L., & Sato, T. R. (2005). Neuronal basis of covert spatial attention in the frontal eye field. Journal of Neuroscience, 25, 9479–9487. Thompson, K. G., Hanes, D. P., Bichot, N. P., & Schall, J. D. (1996). Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. Journal of Neurophysiology, 76, 4040–4055. Townsend, J. T. (1990). The truth and consequences of ordinal differences in statistical distributions: Toward a theory of hierarchical inference. Psychological Bulletin, 108, 551–567. Turner, B. M., Forstmann, B. U., Wagenmakers, E. J., Brown, S. D., Sederberg, P. B., and Steyvers, M. (2013). A Bayesian framework for simultaneously modeling neural and behavioral data. NeuroImage, 72, 193–206. Umakantha, A., Purcell, B. A., & Palmeri, T. J. (2014). Mapping between a spiking neural network model and the diffusion model of perceptual decision making. Usher, M., & McClelland, J. L. (2001). The time course of perceptual choice: The leaky, competing accumulator model. Psychological Review, 108, 550–592. van Maanen, L., Brown, S. D., Eichele, T., Wagenmakers, E. J., Ho, T., Serences, J., & Forstmann, B. U. (2011). Neural correlates of trial-to-trial fluctuations in response caution. Journal of Neuroscience, 31(48), 17488–17495. Van Zandt, T. (2000). How to fit a response time distribution. Psychonomic Bulletin & Review, 7, 424–465. Verbruggen, F., & Logan, G. D. (2008). Response inhibition in the stop-signal paradigm. Trends in Cognitive Sciences, 12, 418–424.
neurocognitive modeling of perceptual decision making
339
White, C. N., Mumford, J. A., & Poldrack, R. A. (2012). Perceptual criteria in the human brain. Journal of Neuroscience, 32(47), 16716–16724. Wolfe, J. M., Palmer, E. M., & Horowitz, T. S. (2010). Reaction time distributions constrain models of visual search. Vision Research, 50, 1304–1311. Wong, K. F., Huk, A. C., Shadlen, M. N., & Wang, X. J. (2007). Neural circuit dynamics underlying accumulation of time-varying evidence during perceptual decision making. Frontiers in Computational Neuroscience, 1, 1–11.
340
new directions
Wong, K. F., & Wang, X. J. (2006). A recurrent network mechanism of time integration in perceptual decisions. Journal of Neuroscience, 26, 1314–1328. Woodman, G. F., Kang, M. S., Thompson, K., & Schall, J. D. (2008). The effect of visual search efficiency on response preparation: Neurophysiological evidence for discrete flow. Psychological Science, 19, 128–136. Zandbelt, B. B., Purcell, B. A., Palmeri, T. J., Logan, G. D., Schall, J. D. (2014). Response times from ensembles of accumulators. Proceedings of the National Academy of Sciences, 111, 2848–2853.
CHAPTER
16
Mathematical and Computational Modeling in Clinical Psychology
Richard W. J. Neufeld
Abstract
This chapter begins with an introduction to the basic ideas behind clinical mathematical and computational modeling. In general, models of normal cognitive-behavioral functioning are titrated to accommodate performance deviations accompanying psychopathology; model features remaining intact indicate functions that are spared; those that are perturbed are triaged as signifying functions that are disorder affected. Distinctions and interrelations among forms of modeling in clinical science and assessment are stipulated, with an emphasis on analytical, mathematical modeling. Preliminary conceptual and methodological considerations are presented. Concrete examples illustrate the benefits of modeling as applied to specific disorders. Emphasis in each case is on clinically significant information uniquely yielded by the modeling enterprise. Implications for the functional side of clinical functional neuro-imaging are detailed. Challenges to modeling in the domain of clinical science and assessment are described, as are tendered solutions. The chapter ends with a description of continuing challenges and future opportunities. Key Words: clinical mathematical modeling, clinical cognitive modeling, analytical modeling,
psychological assessment, stochastic modeling, clinical bayesian modeling, clinical quantitative cognition, cognitive deficit, method of titration
Introduction “The important point for methodology of psychology is that just as in statistics one can have a reasonably precise theory of probable inference, being ‘quasi-exact about the inherently inexact,’ so psychologists should learn to be sophisticated and rigorous in their metathinking about open concepts at the substantive level. . . . In social and biological science, one should keep in mind that explicit definition of theoretical entities is seldom achieved in terms of initial observational variables of those sciences, but it becomes possible instead by theoretical reduction or fusion (Meehl, 1978; p. 815).” Mathematical and computational modeling of clinical-psychological phenomena can elucidate clinically significant constructs by translating them into variables of a quantitative system, and lending
them meaning according to their operation within that very system (e.g., Braithwaite, 1968). New explanatory properties are availed, as are options for clinical-science measurement, and tools for clinical-assessment technology. This chapter is designed to elaborate on these assets by providing examples where otherwise intractable or hidden clinical information has been educed. Issues of practicality and validity, indigenous to the clinical setting, are examined, as is the potentially unique contribution of clinical modeling to the broader modeling enterprise. Emphasis is on currently prominent domains of application, and exemplary instances within each. Background material for the current developments is available in several sources (e.g., Busemeyer & Diedrich, 2010; Neufeld, 1998; 2007a). 341
We begin by considering an overall epistemic strategy of clinical psychological modeling. Divisions of modeling in the clinical domain are then distinguished. Exemplary implementations are presented, as are certain challenges sui generis to this domain. Figure 16.1 summarizes the overall epistemic strategy of clinical psychological modeling. In its basic form, quantitative models of normal performance, typically on laboratory tasks, are titrated to accommodate performance deviations occurring with clinical disturbance. The requisite model tweaking, analogous to a reagent of chemical titration, in principle discloses the nature of change to the task-performance system taking place with clinical disturbance. Aspects of the model remaining intact are deemed as aligning with functions spared with the disturbance, and those that have been perturbed are triaged as pointing to functions that have been affected. Accommodation of performance deviations by the appropriated model infrastructure, in turn, speaks to validity of the latter. Successful accommodation of altered performance among clinical samples becomes a source of construct validity, over and against an appropriated model’s failure or strain in doing so. This aspect of model evaluation, an instance of “model-generalization testing” (Busemeyer & Wang, 2000), is one in which performance data from the clinical setting can play an important role. To illustrate the preceding strategy, consider a typical memory-search task (Sternberg, 1969). Such a task may be appropriated to tap ecologically significant processes: cognitively preparing and transforming (encoding) environmental stimulation into a format facilitating collateral cognitive operations; extracting and manipulating material in Mathematical and Computational Psychology
Mathematical and Computational Modeling of Experimental-Task Performance
Clinical Mathematical and Computational Psychology Expression of Performance Deviation according to Model Titration
Model-Generalization Testing
Fig. 16.1 Relations between clinical and nonclinical mathematical and computational psychology.
342
new directions
short-term or working memory, on which informed responding rests; and preparing and delivering the information-determined response. During each trial of the preceding task, a prememorized set of items (memory set), such as alphanumeric characters, are to be scanned, in order to ascertain “as quickly and accurately as possible” the presence or absence of a subsequently presented item (probe item). The subspan memory-set size varies from trial to trial, with the items within each set size also possibly varying over their presentations, or alternatively remaining constant within each set size (variable- versus fixed- set procedures). Other manipulations may be directed, for instance, to increasing probe-encoding demands (whereby, say, the font of the probe item mismatches, rather than matches, the font of the memory-set items). The principal response property tenably is latency from probe-item onset, accuracy being high, and not compromising the validity of latency-based inferences (e.g., no speed-accuracy trade-off ). Quantitatively informed convergent experimental evidence may point to an elongated probeencoding process as being responsible for delayed trial-wise response latencies in the clinical group. The encoding process may be represented by a portion of the trial-performance model. This portion may stipulate, for example, constituent encoding operations of the encoding process, there being k in number (to use model-parameter notation consistent with the clinical modeling literature). Such subprocesses may correspond with observable stimulus features, such as curves, lines, and intersections of the alphanumeric probe, extracted in the service of mental template matching to members of the trial’s memory set. The intensity of processing applicable to each of the respective k subprocesses (loosely, speed of completion, or number transacted per unit time; e.g., Rouder, Sun, Speckman, Lu, & Ahou, 2003; Townsend & Ashby, 1983) is denoted v. Decomposing clinical-sample performance through the model-based analysis potentially exonerates the parameter v, but indicts k as the source of encoding protraction. Specifically, modelpredicted configurations of performance latency, and intertrial variability, converge with the pattern of empirical deviations from control data, upon elevation in k , but not reduction in v. By way of parameter assignment in the modeling literature, strictly speaking, cognitive capacity (in the sense of processing speed) is intact, but efficiency of its deployment has suffered. The preceding example of mapping of formal theory onto empirical data
is known in clinical mathematical modeling, and elsewhere, as “abductive reasoning” (see Box 1). When it comes to potential “value added” beyond the immediate increment in information, the following come to the fore. Ecological significance is imbued in terms of assembling environmental information in the service of establishing necessary responses, including those germane to self-maintenance activities, and meeting environmental demands. On this note, basic deficit may ramify to more elaborate operations in which the affected process plays a key role (e.g., where judgments about complex multidimensional stimuli are built up from encoded constituent dimensions). Deficits in rudimentary processes moreover may parlay into florid symptom development or maintenance, as where thought-content disorder (delusions and thematic hallucinations) arise from insufficient encoding of cues that normally anchor the interpretation of other information. Additional implications pertain to memory, where heightened retrieval failure is risked owing to protraction of initial item encoding. Debility parameterization also may inform other domains of measurement, such as neuroimaging. A tenable model may demarcate selected intratrial epochs of cognitive tasks when a clinically significant constituent process is likely to figure prominently. In this way, times of measurement interest may complement brain-regions of interest, for a more informed navigation of space-time coordinates in functional neuroimaging. Symptom significance thus may be brokered to imaged neurocircuitry via formally modeled cognitive abnormalities.
Modeling Distinctions in Psychological Clinical Science There are several versions of “modeling” in psychological clinical science. Nonformal models, such as flow-diagrams and other organizational schemata, nevertheless, are ubiquitously labeled “models” (cf. McFall, Townsend & Viken, 1995). Our consideration here is restricted to formal models, where the progression of theoretical statements is governed by precisely stipulated rules of successive statement transitions. Most notable, and obviously dominating throughout the history of science are mathematical models. Formal languages for theory development other than mathematics include symbolic logic and computer syntax. Within the formal modeling enterprise, then is mathematical modeling, computational modeling [computer simulation, including “connectionist,” “(neural)
Box 1 Abductive Reasoning in Clinical Cognitive Science Scientific rigor does not demand that theoretical explanation for empirical findings be restricted to a specific account from a set of those bearing on the study that have been singled out before the study takes place. In fact, the value of certain formal developments to understanding obtained data configurations may become apparent only after the latter present themselves. It is epistemologically acceptable to explanatorily retrofit extant (formal) theory to empirical data (e.g., in the text, changing the clinical sample’s value of k versus v), a method known as abductive reasoning (Haig, 2008). Abductive reasoning not only has a bona-fide place in science, but it is economical in its applicability to already-published data (Novotney, 2009) and/or proffered conjectures about clinically significant phenomena. On the note of rigorous economy, the clinical scientist can play with model properties to explore the interactions of model-identified or other proposed sources, say, of pathocognition with various clinically significant variables, such as psychological stress. For example, it has been conjectured that some depressive disorders can be understood in terms of highly practiced, automatic negative thoughts supplanting otherwise viable competitors, and also that psychological stress enhances ascendancy of the former on the depressed individual’s cognitive landscape (Hartlage, Alloy, Vasquez & Dykman, 1993). Translating these contentions into terms established in mainstream quantitative cognitive science, however, discloses that psychological stress instead reduces the ascendancy of well-practiced negative thoughts, at least within this highly-defensible assumptive framework (Neufeld, 1996; Townsend & Neufeld, 2004). The quantitative translation begins with expressing the dominance of well-practiced (socalled automatic) negative versus less practiced (so-called effortful) non-negative thought content, as higher average completion rates for the former. With these rate properties in tow, the well-practiced and less-practiced thought content then enter a formally modeled “horse race,” where the faster rates for negative-thought generation evince higher winning probabilities, for all race durations. Note that although these
mathematical and computational
343
Box 1 Continued derivations result from basic computations in integral calculus, they nevertheless yield precise predictions, and lay bare their associated assumptions. Differentiating the above horse-race expression of the probability of negative-thought victory, with respect to a parameter conveying the effects of stress on processing capacity, leads to a negative derivative. Formalized in this way, then, the result shows psychological stress actually to handicap the negative-thought contender. It is conceivable that reduction in the ascendancy of well-practiced negative thoughts, in the face of stressing environmental demands and pressures, in favour of less-practiced but more adaptive cognitive processes conveys a certain protective function. In all events, this example illustrates the hazards of depending on unaided verbal reasoning in attempting to deal with complex intervariable relations (including stress effects on psychopathololgy), and exemplifies the disclosure through available formal modeling, of subtleties that are both plausible and clinically significant—if initially counterintuitive (Staddon, 1984; Townsend, 1984).
network,” “cellular automata,” and “computational informatics” modeling], and nonlinear dynamical systems modeling (“chaos-theoretic” modeling, in the popular vernacular). There is, of course, an arbitrary aspect to such divisions. Mathematical modeling can recruit computer computations (later), whereas nonlinear dynamical systems modeling entails differential equations, and so on. Like many systems of classification, the present one facilitates exposition, in this case of modeling activity within the domain of psychological clinical science. Points of contact and overlap among these divisions, as well as unique aspects, should become more apparent with the more detailed descriptions that follow. The respective types of formal modeling potentially provide psychopathology-significant information unique to their specific level of analysis (Marr, 1982). They also may inform each other, and provide across-level-analysis construct validity. For example, manipulation of connectionist-model algorithm parameters may be 344
new directions
guided by results from mathematical modeling. Connectionist-modeling results, in turn, may lend construct validity to mathematical model titration (earlier). Before delineating subsets of formal modeling, a word is in order about so-called statistical modeling (see, e.g., Rodgers, 2010). Statistical modeling, such as structural-equation modeling, including confirmatory factor analysis, hierarchical linear modeling, mixture growth modeling, and taxometric analysis (mixture-model testing for staggered, or quasi-staggered latent distributions of clinical and nonclinical groups) supply a platform for data organization, and inferences about its resulting structure. To be sure, parameters, such as path weights and factor loadings are estimated using methods shared with formal models, as demarcated here. Contra the present emphasis, however, the format of proposed model structure (typically one of multivariate covariance), and computational methods are transcontent, generic, and do not germinate within the staked out theoretical-content domain with its problem-specific depth of analysis. In the case of formal modeling, it might be said that measurement models and empiricaltesting methods are part and parcel of processmodels of observed responses and data production (see also Box 2). Extended treatment of formalmodel distinctives and assets in clinical science is available in alternate venues (e.g., Neufeld, 2007b; Shanahan, Townsend & Neufeld in press).
Forms of Modeling in Clinical Science mathematical modeling Clinical mathematical modeling is characterized by analytically derived accounts of cognitivebehavioral abnormalities of clinical disorders. In most instances, models are stochastic, meaning they provide for an intrinsic indeterminacy of the modeled phenomenon (not unlike Brownian motion being modeled by a Wiener process, in physics). Doob (1953) has described a stochastic model as a “. . . mathematical abstraction of an empirical process whose development is governed by probabilistic laws (p. v).” Roughly, built into the structure of stochastic models is a summary of nature’s perturbation of empirical values from one observation to the next. Predictions, therefore, by and large, are directed to properties of the distributions of observations, such as those of response latencies over cognitive-task trials. Model
expressions of clinical-sample deviations in distribution properties, therefore, come to the fore. Such properties may include summary statistics, such as distribution moments (notably means and intertrial variances), but can also include distribution features as detailed as the distribution’s probability density function (density function, for short; proportional to the relative frequency of process completion at a particular time since its commencement; see, e.g., Evans, Hastings & Peacock, 2000; Townsend & Ashby, 1983; Van Zandt, 2000). Grasping the results of mathematical modeling’s analytical developments can be challenging, but it can be aided by computer computations. Where the properties of derivations are not apparent from inspection of their structures themselves, the formulae may be explored numerically. Doing so often invokes three-dimensional response surfaces. The predicted response output expressed by the formula, plotted on the Y axis, is examined as the formula’s model parameters are varied on the X and Z axes. In the earlier example, for instance, the probability of finalizing a stimulus-encoding process within a specified time t may be examined as parameters k and v are varied. Note in passing that expression of laws of nature typically is the purview of analytical mathematics (e.g., Newton’s law of Gravity). computational modeling (computer simulation) Computational modeling expresses recurrent interactions (reciprocal influences) among activated, and activating units, that have been combined into a network architecture. The interactions are implemented through computer syntax (e.g., Farrell & Lewandowsky, 2010). Some of its proponents have advanced this form of modeling as a method uniquely addressing the computational capacity of the brain, and as such, have viewed the workings of a connectionist network as a brain metaphor. Accordingly, the network units can stand for neuronal entities (e.g., multineuron modules, or “neurodes”), whose strengths of connection vary over the course of the network’s ultimate generation of targeted output values. Variation in network architecture (essentially paths and densities of interneurode connections), and/or connection activation, present themselves as potential expressions of cognitive abnormalities. Exposition of neuro-connectionist modeling in clinical science has been relatively extensive (e.g., Bianchi, Klein, Caviness, & Cash, 2012; Carter
& Neufeld, 2007; Hoffman & McGlashan, 2007; Phillips & Silverstein, 2003; Siegle & Hasselmo, 2002; Stein & Young, 1992). Typically directed to cognitive functioning, the connectionist-network approach recently has been extended to the study of intersymptom relations, providing an unique perspective for example on the issue of co-morbidity (Boorsboom, & Cramer, 2013; Cramer, Waldorp, van der Maas & Boorsboom, 2010).
nonlinear dynamical system (chaos-theoretic) modeling This form of modeling again entails interconnected variables (“coupled system dimensions”), but in clinical science, by and large, these are drastically fewer in number than usually is the case with computational network modeling (hence, the common distinction between “high-dimensional networks” and “low-dimensional nonlinear networks”). Continuous interactions of the system variables over time are expressed in terms of differential equations. The latter stipulate the momentary change of each dimension at time t, as determined by the extant values of other system dimensions, potentially including that of the dimension whose momentary change is being specified (e.g., Fukano & Gunji, 2012). The nonlinear properties of the system can arise from the differential equations’ cross-product terms, conveying continuous interactions or nonlinear functions of individual system dimensions, such as raising extant momentary status to a power other than 1.0. The status of system variables at time t is available via solution to the set of differential equations. Because of the nonlinearity-endowed complexity, solutions virtually always are carried out numerically, meaning computed cumulative infinitesimal changes are added to starting values for the time interval of interest. System dimensions are endowed with their substantive significance according to the theoretical content being modeled (e.g., dimensions of subjective fear, and physical symptoms, in dynamical modeling of panic disorder; Fukano & Gunji, 2012). A variation on the preceding description comprises the progression of changes in system-variable states over discrete trials (e.g., successive husband-wife interchanges; Gottman, Murray, Swanson, Tyson & Swanson, 2002). Such sequential transitions are implemented through trial-wise difference-equations, which now take the place of differential equations.
mathematical and computational
345
Model-predicted trajectories can be tested against trajectories of empirical observations, as obtained, say, through diary methods involving on-line data-gathering sites (e.g., Qualtrics; Bolger, Davis & Rafaeli, 2003). In addition, empirical data can be evaluated for the presence of dynamical signatures (“numerical diagnostics”) generated by system-equation driven, computer-simulated time series. It is possible, moreover, to search empirical data for time-series numerical diagnostics that are general to system complexity of a nonlineardynamical-systems nature. This latter endeavour, however, has been cricized when undertaken without a precise tendered model of the responsible system, ideally buttressed with other forms of modeling (notably mathematical modeling, earlier)— such a more informed undertaking as historically exemplified in physics (Wagenmakers, van der Maas & Farrell, 2012). A branch of nonlinear dynamical systems theory, ‘catastrophe theory,’ has been implemented notably in the analysis of addiction relapse. For example, therapeutically significant dynamical aspects of aptitude-treatment intervention procedures (e.g., Dance & Neufeld, 1988) have been identified through a catastrophetheory-based reanalysis of data from a large-scale multisite treatment-evaluation project (Witkiewitz, van der Maas, Hufford & Marlatt, 2007). Catastrophe theory offers established sets of differential equations (“canonical forms”) depicting sudden jumps in nonlinear dynamical system output (e.g., relapse behaviour) occurring to gradual changes in input variables (e.g., incremental stress). Nonlinear dynamical systems modeling—whether originating in the content domain, or essentially imported, as in the case of catastrophe theory— often may be the method of choice when it comes to macroscopic, molar clinical phenomena, such as psychotherapeutic interactions (cf. Molenaar, 2010). Extended exposition of nonlinear dynamical systems model development, from a clinical-science perspective, including that of substantively significant patterns of dimension trajectories (“dynamical attractors”), has been presented in Gottman, et al. (2002), Levy et al., (2012), and Neufeld (1999).
Model Parameter Estimation in Psychological Clinical Science A parameter is “an arbitrary constant whose value affects the specific nature but not the properties of a mathematical expression” (Borowski & Borwein, 1989, p. 435). In modeling abnormalities in 346
new directions
clinical samples, typically it is the values of model parameters, rather than the model structure (the model’s mathematical organization) that are found to shift away from control values. The clinical significance of the shift depends on the meaning of the parameter(s) involved. Parameters are endowed with substantive significance according to their roles in the formal system in which they are positioned, and their mathematical properties displayed therein. For example, a parameter of a processing model may be deemed to express “task-performance competence.” Construct validity for this interpretation may be supported if the statistical moments of modeled performance-trial latencies (e.g., their mean or variance) are sent to the realm of infinity, barring a minimal value for the parameter. If this value is not met, a critical corpus of extremely long, or incomplete task trials ensues, incurring infinite response-latency moments in the formalized system. From a construct-validity standpoint, this type of effect on system behavior—severe performance impairment, or breakdown—is in keeping with the parameter’s ascribed meaning (Neufeld, 2007b; see also Pitt, Kim, Navarro & Myung, 2006). Added to this source of parameter information— analytical construct validity—is experimental construct validity. Here, estimated parameter values are selectively sensitive to experimental manipulations, diagnostic-group differences, or variation in psychometric measures on which they purportedly bear (e.g., additional constituent operations of a designated cognitive task, other demands being equal, resulting in elevation specifically in the parameter k’, earlier). In clinical science and assessment, values of model parameters to a large extent are estimated from empirical data. Such estimation differs from methods frequently used in the physical sciences, which more often have the luxury of direct measurement of parameter values (e.g., measuring parameters of liquid density and temperature, in modeling sedimentation in a petroleum-extraction tailings pond). Parameter estimation from the very empirical data being modeled—of course, with associated penalization in computations of empirical model fit—however, is only superficially suspect. As eloquently stated by Flanagan (1991), In physics there are many explanatory constructs, electrons for example, which cannot be measured independently of the observable situations in which they figure explanatorily. What vindicates the
explanatory use of such constructs is the fact that, given everything else we know about nature, electrons best explain the observable processes in a wide array of experimental tests, and lead to successful predictions (p. 380).
How, then might parameter values be estimated, with an eye to possible constraints imposed in the clinical arena? Multiple methods of parameter estimation in clinical science variously have been used, depending on desired statistical properties (e.g., maximum likelihood, unbiasedness, Bayes; see, e.g., Evans, et al., 2000) and data-acquisition constraints. Note that selection of parameterestimation methods is to be distinguished from methods of selecting from among competing models or competing model variants (for issues of model selection, see especially Wagenmakers & Vandekerckhove, this volume). moment matching One method of parameter estimation consists of moment matching. Moments of some stochastic distributions can be algebraically combined to provide a direct estimate. For example, the mean of the Erlang distribution of performance-trial latencies, expressed in terms of its parameter implementation k’ and v, earlier, is k’/v (e.g., Evans, et al., 2000). The intertrial variance is k’/v2 . From these terms, an estimate of v is available as the empirical mean divided by the empirical variance, and an estimate of k’ is available as the mean, squared, divided by the variance. maximum likelihood Maximum-likelihood parameter estimation means initially writing a function expressing the likelihood of obtained data, given the data-generation model. The maximum-likelihood estimate is the value of the model parameter that would make the observed data maximally likely, given that model. Maximum likelihood estimates can be obtained analytically, by differentiating the written likelihood function with respect to the parameter in question, setting it to 0, and solving (the second derivative being negative). For example, the maximumlikelihood estimate of v in the Erlang distribution, earlier, is Nk , N ti i=1
where the size of the empirical sample of latency values ti is N. With multiple parameters, such as v
and k’, the multiple derivatives are set to 0, followed by solving simultaneously. As model complexity increases, with respect to the number of parameters and possibly model structure, numerical solutions may be necessary. Solutions now are found by computationally searching for the likelihood-function maximum, while varying constituent parameters iteratively and reiteratively. Such search algorithms are available through R, through the MATLAB OPTIMIZATIOMN TOOLBOX, computer-algebra programs, such as Waterloo MAPLE, and elsewhere. As with other methods that rely exclusively on the immediate data, stability of estimates rests in good part on extensiveness of data acquisition. Extensive data acquisition, say, on a cognitive task, possibly amounting to hundreds or even thousands of trials, obtained over multiple sessions, may be prohibitive when it comes to distressed patients. For these reasons, Bayesian estimates may be preferred (later). moment fitting and related procedures Another straightforward method of parameter estimation is that of maximizing the conformity of model-predicted moments (typically means and intertrial standard deviations, considering instability of higher-order empirical moments; Ratcliff, 1979), across performance conditions and groups. An unweighted least-squares solution minimizes the sum of squared deviations of model predictions from the empirical moments. Although analytical solutions are available in principle (deriving minima using differential calculus, similar to deriving maxima in the case of maximum-likelihood solutions), use of a numerical search algorithm most often is the case. Elaborations on unweighted least-squares solutions include minimization of Pearson χ 2 , where data are response-category frequencies fri : . - p (fri,observed − fri,model−predicted )2 , min fri,model−predicted i=1 where there are p categories. Note that frequencies can include those of performance-trial latencies falling within successive time intervals (“bins”). Where the chief data are other than categorical frequencies, as in the case of moments, parameter values may be estimated by constructing and minimizing a pseudo- χ 2 function. Here, the theoretical and observed frequencies are replaced with the observed and theoretical moments (Townsend, 1984; Townsend & Ashby, 1983, chap. 13). As with the Pearson χ 2 , the squared differences between the
mathematical and computational
347
model-predicted and observed values are weighted by the inverse of the model-predicted values. It is important to take account of logistical constraints when it comes to parameter-estimation and other aspects of formal modeling in the clinical setting. In this setting, where available data can be sparse, the use of moments may be necessary for stability of estimation. Although coarse, as compared to other distribution properties, moments encompass the entire distribution of values, and can effectively attenuate estimate-destabilizing noise (cf. Neufeld & Gardner, 1990). Observe that, along with likelihood maximization, the present procedures are an exercise in function optimization. In the case of least squares and variants thereon, the exercise is one of function minimization. In fact, often an unweighted leastsquares solution also is the maximum likelihood. Similarly, where likelihood functions enter χ 2 or approximate χ 2 computations (likelihood-ratio G 2 ), the minimum χ 2 is also maximum likelihood. Evaluation of the adequacy of parameter estimation amid constraints that are intrinsic to psychological clinical science and assessment ultimately rests with empirical tests of model performance. Performance obviously will suffer with inaccuracy of parameter estimation. bayesian parameter estimation Recall that Bayes’s theorem states that the probability of an estimated entity A, given entityrelated evidence B (posterior probability of A) is the pre-evidence probability of A (its prior probability) times the conditional probability of B, given A (likelihood), divided by the unconditional probability of B (normalizing factor): Pr(A)Pr(B|A) . (1) Pr(B) As applied to parameter estimation, A becomes the candidate parameter value θ, and B becomes data D theoretically produced by the stochastic model in which θ participates. Recognizing θ as continuous, Eq. (1) becomes Pr(A|B) =
f (θ)Pr(D|θ) g(θ|D) = + ∞ − ∞ f (θ)Pr(D|θ)d θ
(2)
where g and f denote density functions, over and against discrete-value probabilities Pr. The data D may be frequencies of response categories, such as correct versus incorrect item recognition. Note that the data D as well may be continuous, as in the case of measured process latencies. If
348
new directions
so, the density functions again replace discretevalue probabilities. For an Erlang distribution, for instance, the density function for a latency datum ti is (vti )k −1 −vti ve . (k − 1) Allowing k to be fixed, for the present purposes of illustration (e.g., directly measured, or set to 1.0, as with the exponential distribution), and allowing θ to stand for v, then for a sample of N independent values, Pr(D|θ ) in Eq. (2) becomes the joint conditional density function of the N ti values, given v and k’, N (vti )k −1
i=1
(k − 1)
ve−vti .
The posterior density function of θ [e.g., Eq. (2)] can be computed for all candidate values of θ , the tendered estimate then being the mean of this distribution (the statistical property of this estimate is termed “Bayes”; see especially Kruschke & Vanpaemel, this volume). If an individual participant or client is the sole source of D, the Bayes estimate of θ de facto has been individualized accordingly. Bayesian parameter estimation potentially endows clinical science and practice with demonstrably important advantages. Allowance for individual differences in parameter values can be built into the Bayesian architecture, specifically in terms of the prior distribution of performancemodel parameters (Batchelder, 1998). In doing so, the architecture also handles the issue of overdispersion in performance-data, meaning greater variability than would occur to fixed parameter values for all participants (Batchelder & Riefer, 2007). Selecting a prior distribution of θ depends in part on the nature of θ . Included are strong priors, such as those from the Generalized Gamma family (Evans, et al., 2000) where 0 ≤ θ ; the Beta distribution, where 0 ≤ θ ≤ 1.0; and the normal or Gaussian distribution. Included as well are gentle (neutral) priors, notably the uniform distribution, whose positive height spans the range of possible non-0 values of θ (see also Berger, 1985, for Jeffreys’ uninformed, and other prior distributions). Grounds for priordistribution candidacy also includes “conjugation”
with the performance process model. The practical significance of distributions being conjugate essentially is that the resulting posterior distribution becomes more mathematically tractable, allowing a closed-form solution for its probability density function. Defensible strong priors (defensible on theoretical grounds, and those of empirical model fit) can add to the arsenal of clinically significant information, in and of themselves. To illustrate, the Bayesian prior distribution of θ values has its own governing parameters (hyperparameters). Hyperparameters, in turn, can be substantively significant within the addressed content domain (e.g., the taskwise competence parameter, described at the beginning of this section; or another expressing psychological-stress effects on processing speed). Because of the information provided by a strong Bayesian prior, the estimate of θ can be more precise and stable in the face of a smaller data set D than would be necessary when the estimate falls entirely to the data at hand (see, e.g., Batchelder, 1998). This variability-reducing influence on parameter estimation is known as Bayesian shrinkage (e.g., O’Hagan & Forster, 2004). Bayesian shrinkage can be especially valuable in the clinical setting, where it may be unreasonable to expect distressed participants or patients to offer up more than a modest reliable specimen of task performance. Integrating the performance sample with the prior-endowed information is analogous to referring a modest blood sample to the larger body of hematological knowledge in the biomedical assay setting. Diagnostic richness of the proffered specimen is exploited because its composition is subjected to the preexisting body of information that is brought to bear. Other potentially compelling Bayesian-endowed advantages to clinical science and assessment are described in the section, Special Considerations Applying to Mathematical and Computational Modeling in Psychological Clinical Science and Assessment, later. (See also Box 2 for an unique method of assembling posterior density functions of parameter values to ascertain the probability of parameter differences between task-performance conditions.)
Illustrative Examples of Contributions of Mathematical Psychology to Clinical Science and Assessment Information conveyed by rigorous quantitative models of clinically significant cognitive-behavioral
systems is illustrated in the following examples. Focus is on uniquely disclosed aspects of system functioning. Results from generic data-theory empirical analyses, such as group by performanceconditions statistical interactions, are elucidated in terms of their modeled performance-process underpinnings. In addition to illuminating results from conventional analyses, quantitative modeling can uncover group differences in otherwise conflated psychopathology-germane functions (e.g., Chechile, 2007; Riefer, Knapp, Batchelder, Bamber & Manifold, 2002). Theoretically unifying seemingly dissociated empirical findings through rigorous dissection of response configurations represents a further contribution of formal modeling to clinical science and assessment (White, Ratcliff, Vasey & McKoon, 2010a). Moreover, definitive assessment of prominent conjectures on pathocognition is availed through experimental paradigms exquisitely meshing with key conjecture elements (Johnson, Blaha, Houpt & Townsend, 2010). Such paradigms emanate from models addressing fundamentals of cognition, and carry the authority of theorem-proof continuity, and closed-form predictions (Townsend & Nozawa, 1995; see also Townsend & Wenger, 2004a). At the same time, measures in common clinical use have not been left behind (e.g., Fridberg, Queller, Ahn, Kim, Bishara & Busemeyer, 2010; Bishara, Kruschke, Stout, Bechara, McCabe & Busemeyer, 2010; Yechiam, Veinott, Busemeyer, & Stout, 2007). Mathematical modeling effectively has quantified cognitive processes at the root of performance on measures such the Wisconsin Card Sorting Test, the Iowa Gambling Task, and the Go– No-Go task (taken up under Cognitive Modeling of Routinely Used Measures in Clinical Science and Assessment, later). Further, formal models of clinical-group cognition can effectively inform findings from clinical neuroimaging. Events of focal interest in “event-related imaging” are neither the withinnor between-trial transitions of physical stimuli embedded in administered cognitive paradigms but, rather, the covert mental processes to which such transitions give rise. Modeled stochastic trajectories of the symptom-significant component processes that transact cognitive performance trials can stipulate intratrial epochs of special neuroimaging interest. The latter can complement brain regions of interest, together facilitating the calibration of space-time measurement coordinates in neuroimaging studies.
mathematical and computational
349
Multinomial Processing Tree Modeling of Memory and Related Processes; Unveiling and Elucidating Deviations Among Clinical Samples Possibly the most widely used mathematical modeling in clinical psychology is multinomial processing tree modeling (MPTM). Essentially, MPTM models the production of categorical responses, such as recall or recognition of previously studied items, or judgments of items about their earlier source of presentation (e.g., auditory versus visual, thereby potentially bearing on the nature of hallucinations; Batchelder & Riefer, 1990; 1999). Responses in such categories are modeled as having emanated from a sequence of stages, or processing operations. For example, successful recall of a pair of semantically linked items, such as “computer, Internet,” entails storage and retrieval of the duo, retrieval itself necessarily being dependent on initial storage. The tree in MPTM consists of branches emanating from nodes; processes branching from nodes proceed from other processes, on which the branching process is conditional (as in the case of retrieving a stored item). Each process in each successive branch has a probability of successful occurrence. The probability of a response in a specific category (e.g., accurate retrieval of an item pair) having taken place through a specific train of events, is the product of the probabilities of those events (e.g., probability of storage times the conditional probability of retrieval, given storage). Parameters conveying the event probabilities are viewed as “capacities of the associated processes.” Criteria for success of constituent processes implicitly are strong, in that execution of a process that took place earlier in the branching is a sufficient precondition for the process being fed; the probability of failure of the subsequent process falls to that process itself. In this way, MPTM isolates functioning of the individual response-producing operations, and ascertains deficits accordingly. The model structure of MPTM is conceptually tractable, thanks to its straightforward processing tree diagrams. It has, however, a strong analytical foundation, and indeed has spawned innovative methods of parameter estimation (Chechile, 1998), and notably rigorous integration with statistical science (labeled “cognitive psychometrics”; e.g., Batchelder, 1998; Riefer et al., 2002). Computer software advances have accompanied MPTM’s analytical developments (see Moshagen, 2010, for current renderings). Exposition of MPTM 350
new directions
measurement technology has been substantial (e.g., Batchelder & Riefer, 1990; Chechile, 2004; Riefer & Batchelder, 1988), including that tailored to clinical-science audiences (Batchelder, 1998; Batchelder & Riefer, 2007; Chechile, 2007; Riefer et al., 2002; Smith & Batchelder, 2010). Issues in clinical cognition potentially form natural connections with the parameters of MTM. Just as the categories of response addressable with MPTM are considerable, so are the parameterized constructs it accommodates. In addition to essential processes of memory, perception, and learning, estimates are available for the effects of guessing, and for degrees of participants’ confidence in responding. Moreover, MPTM has been extended to predictions of response latency (Hu, 2001; Schweickert, 1985; Schweickert & Han, 2012). Riefer et al. (2002) applied MPTM in two experiments to decipher the nature of recall performance among schizophrenia and brain-damaged alcoholic participants. In each experiment, the clinical group and controls (nonpsychotic patients, and non–organic-brain-syndrome alcoholics) received six trials of presentation and study of semantically related item pairs (earlier), each study period being followed by recall of items in any order. In both experiments, the research design comprised a “correlational experiment” (Maher, 1970). A correlational experiment consists of the diagnostic groups under study performing under multiple conditions of theoretical interest—a prominent layout in psychological clinical science (e.g., Yang, Tadin, Glasser Glasser, Hong & Park, 2013). Initial analyses of variance (ANOVA) were conducted on sheer proportions of items recalled. In each instance, significant main effects of groups and study-recall trials were obtained; a statistically significant trials-by-groups interaction, the test of particular interest, was reported only in the case of the brain-damaged alcoholic participants and their controls (despite liberal degrees of freedom for within-subjects effects). As noted by Riefer et al. (2002), this generic empirical analysis betrayed its own shortcomings for tapping potentially critical group differences in faculties subserving recall performance. Indeed, Riefer et al. simply but forcefully showed how reliance on typical statistical treatments of data from correlational experiments can generate demonstrably misleading inferences about group differences in experimentally addressed processes of clinical and other interest.
Riefer et al.’s theory-disciplined measures precisely teased apart storage and recall-retrieval processes, but went further in prescribing explicitly theory-driven significance tests on group differences (see also Link, 1982; Link & Day, 1992; Townsend, 1984). Pursuant to the superficially parallel group performance changes across trials, schizophrenia participants failed to match the controls in improvement of storage efficiency specifically over the last 3 trials of the experiment. Moreover, analysis of a model parameter distinguishing the rate of improvement in storage accuracy as set against its first-trial “baseline,” revealed greater improvement among the schizophrenia participants during trials 2 and 3, but a decline relative to controls during the last 3 trials. In other words, this aspect of recalltask performance arguably was decidedly spared by the disorder, notably during the initial portions of task engagement. Analysis of the model parameter distinguishing rate of improvement in retrieval, as set against its first-trial baseline now indicated a significantly slower rate among the schizophrenia participants throughout. The interplay of these component operations evidently was lost in the conventional analysis of proportion of items recalled. It goes without saying that precisely profiling disorder-spared and affected aspects of functioning, as exemplified here, can inform the navigation of therapeutic intervention strategies. It also can round out the “functional” picture brought to bear on possible functional neuroimaging measurement obtained during recall task performance. Applying the substantively derived measurement model in the study on alcoholics with organicity yielded potentially important processing specifics buried in the nevertheless now-significant groupsby-trials ANOVA interaction. As in the case of schizophrenia, the brain-damaged alcoholic participants failed to approximate the controls in improvement of storage operations, specifically over the last 3 trials of the experiment. Also, significant deficits in retrieval again were observed throughout. Further, the rate of improvement in retrieval, relative to the trial-1 baseline, more or less stalled over trials. In contrast to retrieval operations, the rate of improvement in storage among the sample of alcoholics with organicity kept up with that of controls—evidently a disorder-spared aspect of task execution. The mathematically derived performanceassessment MPTM methodology demonstrably evinced substantial informational added value, in terms of clinically significant measurement and
explanation. Multiple dissociation in spared and affected elements of performance was observed within diagnostic groups, moreover with further dissociation of these patterns across groups. Estimates of model parameters in these studies were accompanied by estimated variability therein (group and condition-wise standard deviations). Inferences additionally were strengthened with flanking studies supporting construct validity of parameter interpretation, according to selective sensitivity to parameter-targeted experimental manipulations. In addition, validity of parameterestimation methodology was attested to through large-scale simulations, which included provision for possible individual differences in parameter values (implemented according to “hierarchical mixture structures”). In like fashion, Chechile (2007) applied MPTM to expose disorder-affected memory processes associated with developmental dyslexia. Three groups were formed according to psychometrically identified poor, average, and above average reading performance. Presented items consisted of 16 sets of 6 words, some of which were phonologically similar (e.g., blue, zoo), semantically similar (bad, mean), orthographically similar (slap, pals), or dissimilar (e.g., stars, race). For each set of items, 6 pairs of cards formed a 2 × 6 array. The top row consisted of the words to be studied, and the second row was used for testing. A digit-repetition task intervened between study and test phase, controlling for rehearsal of the studied materials. Testing included that of word-item recall or word position, in the top row of the array. For recall trials, a card in the second row was turned over revealing a blank side, and the participant was asked what word was in the position just above. For recognition trials, the face-down side was exposed to reveal a word, with the participant questioned about whether that word was in the corresponding position in the top row. In some instances, the exposed word was in the corresponding position, and in others it was in a different position. A 6-parameter MPTM model was applied to response data arranged into 10 categories. The parameters included two storage parameters, one directed to identification of a previous presentation, and one reserved for the arguably deeper storage required to detect foils (words in a different position); a recall-retrieval parameter; two guessing parameters, and a response-confidence parameter. As in the case of Riefer et al. (2002), parameterized memory-process deviations proprietary to model implementation were identified.
mathematical and computational
351
Apropos of the present purposes, a highlight among other noteworthy group differences occurred as follows. Compared to above-average readers, performance data of poor readers produced lower values for the previous-presentation storage parameter, but higher values for recall retrieval. This dissociation of process strength and deficit was specific to the orthographically similar items. These inferences, moreover, were supported with a decidedly model-principled method of deriving the probability of two groups differing in their distributions of parameter values (see Box 2). The reciprocal effects of stronger retrieval and weaker storage on the poor readers’ performance with the orthographically similar items evinced a nonsignificant difference from the above-average readers on raw recall scores (p > .10). Without the problem-customized measurement model, not only would group differences in memory functions have gone undetected, but the nature of these differences would have remained hidden. Again, a pattern of strength and weakness, and the particular conditions to which the pattern applied (encountering orthographically similar memory items), were powerfully exposed. In this report as well, findings were fortified with model-validating collateral studies. Selective sensitivity to parameter-targeting experimental manipulations lent construct validity to parameter interpretation. In addition, the validity of the model structure hosting the respective parameters was supported with preliminary estimation of coherence between model properties and empirical data [Pr(coherence); Chechile, 2004]. Parameter recovery as well was ascertained through simulations for the adopted sample size. Furthermore, parameter estimation in this study employed an innovation developed by Chechile, called “Population Parameter Mapping” (detailed in Chechile, 1998; 2004; 2007).1 unification of disparate findings on threat-sensitivity among anxiety-prone individuals through a common-process model Random-walk models (Cox & Miller, 1965) are stochastic mathematical models that have a rich history in cognitive theory (e.g., Busemeyer & Townsend, 1993; Link & Heath, 1975; Ratcliff, 1978; see Ratcliff & Smith, this volume). Diffusion modeling (Ratcliff, 1978) presents itself as another form of modeling shown to be of extraordinary value to clinical science. This mathematical method 352
new directions
allows the dissection of decisional performance into component processes that act together to generate choice responses and their latencies (Ratcliff, 1978). Application of diffusion modeling has been used to advantage in simplifying explanation, and in unifying observations on sensitivity to threatvalenced stimulation among anxiety-prone individuals (White, Ratcliff, Vasey & McKoon, 2010a; for diffusion-model software developments, see Wagenmakers, van der Maas, Dolan & Grasman, 2008). Increased engagement of threatening stimulus content (e.g., words such as punishment or accident) among higher anxiety-prone (HA) as compared to lower anxiety-prone (LA) individuals has been demonstrated across multiple paradigms (e.g., the Stroop and dichotic listening tasks; and the dotprobe task, where for HA individuals, detection of the probe is disproportionately increased with its proximity to threatening versus neutral items in a visual array). Consistency of findings of significant HA-LA group differences, however, by and large has depended on presentation of the threat items in the company of nonthreat items. Such differences break down when items are presented singly. Specifically, the critical HA-LA by threat—nonthreat item interaction (group-item second-order difference, or two-way interaction) has tended to be statistically significant specifically when the two types of stimuli have occurred together. This pattern of findings has generated varyingly complex conjectures about responsible agents. The conjectures have emphasized processing competition between the two types of stimuli, and associated cognitivecontrol operations. Group differences have been attributed, for example, to differential tagging of the threat items, or to H-A participant threat-item disengagement deficit (reviewed in White, et al., 2010a). The difficulty in obtaining significant second order-differences with presentation of singletons has led some investigators to question the importance, or even existence, of heightened threatstimulus sensitivity as such among HA individuals. Others have developed neuro-connectionst computational models (e.g., Williams & Oaksford, 1992), and models comprising neuro-connectionist computational-analytical amalgams (Frewen, Dozois, Joanisse & Neufeld, 2008), expressly stipulating elevated threat sensitivity among HA individuals.2 If valid, such sensitivity stands to ramify into grosser clinical symptomatology (Neufeld & Broga, 1981).
Greater threat-stimulus sensitivity defensibly exists among HA individuals; but for reasons that are relatively straightforward, such sensitivity may be more apparent in the company of nonthreat items, as follows: The cognitive system brought to bear on the processing task stands to be one of limited capacity (see Houpt & Townsend, 2012; Townsend & Ashby, 1983; Wenger & Townsend, 2000). When items are presented together, a parallel processing structure arguably is in place for both HA and LA participants (Neufeld & McCarty, 1994; Neufeld, et al., 2007). With less prepotent salience of the threat item for the LA participants, more attentional capacity potentially is drawn off by the nonthreat item, attenuating their difference in processing latency between the two items. A larger interitem difference would occur for the HA participants, assuming their greater resistance to the erosion of processing capacity away from the threat item (see White, et al., 2010a, p. 674). This proposition lends itself to the following simple numerical illustration. We invoke an independent parallel, limited-capacity processing system (IPLC), and exponentially distributed itemcompletion times (Townsend & Ashby, 1983). Its technical specifics aside, the operation of this system makes for inferences about the present issue that are easy to appreciate. The resources of such a system illustratively are expressed as a value of 10 arbitrary units (essentially, the rate per unit time at which task elements are transacted) for both HA and LA participants. In the case of a solo presentation, a threat item fully engages the system resources of an HA participant, and 90% thereof in the case of an LA participant. The solo presentation of the nonthreat item engages 50% of the system’s resources for both participants. By the IPLC model, the second-order difference in mean latency then is (1/10 – 1/9) – 0 = −0.0111 (that is, latency varies inversely as capacity, expressed as a processing-rate parameter). Moving to the simultaneous-item condition, 80% of system-processing resources hypothetically are retained by the threat item in the case of the HA participant, but are evenly divided in the case of the LA participant. The second-order difference now is (1/8−1/5) − (1/2−1/5) = −0.375. Statistical power obviously will be greater for an increased statistical effect size accompanying such a larger difference.3 It should be possible, nevertheless, to detect the smaller effect size for the single-item condition, with a refined measure of processing. On that note, the larger second-order difference in raw
response times attending the simultaneous item condition (earlier) itself may be attenuated due to the conflation of processing time with collateral cognitive activities, such as item encoding and response organization and execution. On balance, a measure denuded of such collateral processes may elevate the statistical effect size of the solopresentation second-order difference at least to that of the paired-presentation raw reaction time. Such a refined measure of processing speed per se was endowed by the diffusion model as applied by White, et al. (2010a) to a lexical decision task (yes-no about whether presented letters form a word). Teased apart were speed of central decisional activities (diffusion-model drift rate), response style (covert accumulation of evidence pending a decision), bias in favor of stating the presence of an actual word, and encoding (initial preparation and transformation of raw stimulation). Analysis was directed to values for these parameters, as well as to those of raw latency and accuracy. In three independent studies, analysis of drift rates consistently yielded significant group,-itemtype second-order differences, whereas analysis of raw latency and accuracy rates consistently fell short. The significant second-order difference also was parameter-selective, being restricted to driftrate values, even when manipulations were conducive to drawing out possible response-propensity differences.4 Here too, findings were buttressed with supporting analyses. Included was construct-validity augmenting selective parameter sensitivity to parameter-directed experimental manipulations. A further asset accompanying use of this model is its demonstrable parametric economy, in the following way; Parameter values have been shown to be uncorrelated, attesting to their conveyance of independent information. Information also is fully salvaged inasmuch as both correct and incorrect response times are analyzed (see also Link, 1982). Parameter estimates were accompanied by calculations of their variability (standard deviations and ranges), for the current conditions of estimation. Diagnostic efficiency statistics (sensitivity, specificity, and positive and negative predictive power) were used to round out description of group separation on the drift-rate, as well as raw data values, employing optimal cut-off scores for predicted classification. In each instance, the driftrate parameter decidedly outperformed the latency mean and median, as well as raw accuracy. These results were endorsed according to signal-detection
mathematical and computational
353
analysis, where the “signal” was the presence of higher anxiety proneness. Altogether, the previously described developments make a strong case for model-delivered parsimony. Seemingly enigmatic and discordant findings are shown to cohere, as products of a common underlying process. measurement technology emanating from theoretical first principles: assessing fundamentals of cognition in autism spectrum disorders As averred by Meehl (1978; see quotation at the outset of this chapter), measurement technology emanating from formal theory of longerestablished disciplines has emerged from the formal theory itself [writ large in the currently prominent Higgs-boson-directed Large Hadron Collider; Close (2011); see McFall & Townsend (1998) for a still-current update of Meehl’s appraisal of measurement methods in clinical-science]. Systems Factorial Technology (SFT; Townsend & Nozawa, 1995; see also Townsend & Wenger, 2004a, and Chapter 4, this volume) comprises such a development in cognitive science, and has been used to notable advantage in clinical cognitive science (Johnson, et al., 2010; Neufeld, et al., 2007; Townsend, Fific, & Neufeld, 2007). Identifiability of fundamentals of cognition has been disclosed by a series of elegant theorem-proof continuities addressed to temporal properties of information processing (see Townsend & Nozawa, 1995 for details; see also Townsend & Altieri, 2012 for recent extensions incorporating the dual response properties of latency and accuracy). The axiomatic statements from which the proofs emanate, moreover, ensure that results are general, when it comes to candidate distributions of processing durations; continuity of underlying population distributions is assumed, but results transcend particular parametric expressions thereof (e.g., exponential, Weibull, etc.; see e.g., Evans, et al., 2000). The distributiongeneral feature is particularly important because it makes for robustness across various research settings, something especially to be welcomed in the field of clinical science. Elements of cognitive functioning exposed by SFT include: (a), the architecture, or structure of the information-processing system; (b), the system’s cognitive workload capacity; (c), selected characteristics of system control; and, (d), independence versus interdependence of constituent cognitive operations carried out by system components. 354
new directions
Architecture pertains to whether the system is designed to handle task constituents concurrently (in parallel channels) or successively, in a serial fashion (e.g., encoding curves, lines, and intersections of alphanumeric characters, simultaneously or sequentially). Within the parallel division, moreover, alternate versions can be entertained. The channels can function as segregated units, with the products of their processing remaining distinct from one another in task completion (regular parallel architecture). Alternately, the channels can act as tributaries to a common conduit that receives and conveys the sum of their contributions, dispatching the collective toward task finalization (co-active parallel architecture). Cognitive workload capacity is estimated in SFT through an index related to work and energy in physics (Townsend & Wenger, 2004b). The index registers the potential of the system to undertake cognitive transactions per unit time [analogous to the rate of dispatching ShannonWeaver (1949) bits of information]. An important aspect of system control entails cessation of processing upon sufficiency for informed responding, over and against extracriterial continuation (operative stopping rules). Furthermore, independence versus interdependence of system components refers to absence versus presence of either mutual facilitation or cross-impedance of system channels devoted to discrete task constituents (e.g., channels handling separate alphanumeric items or possibly item features). Significantly, SFT mathematically disentangles these key elements of cognition. For example, cognitive-workload capacity is isolated from stopping rules and system architecture. Such elements are conflated in macroscopic speed and/or accuracy, whose relative resistance to increased task load (e.g., added items of processing; or concomitantsecondary versus single-task requirements) typically is taken to indicate system capacity (see Neufeld, et al., 2007). Disproportionate change in such behavioral data may occur, however, for reasons other than limitation in system workload capacity. Uneconomical stopping rules may be at work, such as exhaustive processing (task constituents on all system channels are finalized), when selfterminating processing will suffice (informed responding requires completion of only one, or a subset, of task constituents). It also is possible that healthy participants’ seemingly greater workload capacity actually is attributable to a more efficient architecture (e.g., the presence of co-active parallel processing).
This quantitatively disciplined measurement infrastructure takes on increased significance for clinical cognitive science, when it is realized that certain highly prominent constructs therein align with cognitive elements measured by SFT. Especially noteworthy in the study of schizophrenia, for example, is the construct of cognitive capacity (see, e.g., Neufeld, et al., 2007). In addition, systemcontrol stopping rules impinge on so-called executive function, a construct cutting across the study of multiple disorders. Implicated by cognitive-control stopping rules are cognitive-resource conservation, and the robustness of selective inhibition. It should not go unnoticed that SFT fundamentals of cognition also are at the heart of the “automatic-controlled processing” construct.2 This construct arguably trumps all others in cognitiveclinical-science frequency of usage. In identifying variants of the cognitive elements enumerated earlier, the stringent mathematical developments of SFT are meshed with relatively straightforward experimental manipulations, illustrated as follows. In the study of processing mechanisms in autism spectrum disorder (ASD), Johnson et al. (2010) instantiated SFT as “double factorial technology” (Townsend & Nozawa, 1995). A designated visual target consisted of a figure of a right-pointing arrow in a visual display. Manipulations included the presence or absence of such a figure. The target figure could be present in the form of constituent items of the visual array being arranged into a pattern forming a rightpointing arrow (global target), the items themselves consisting of right-pointing arrows (local target) or both (double target). This manipulation is incorporated into quantitative indexes discerning the nature of system workload capacity. The specific target implementation appropriated by Johnson et al., is ideally suited to the assessment of processing deviation in ASD, because prominent hypotheses about ASD cognitive performance hold that more detailed (read “local”) processing is favored. An additional mathematical-theory-driven manipulation entails target salience in the doubletarget condition. The right-pointing item arrangement can be of high or low salience, as can the rightpointing items making up the arrangement, altogether resulting in four factorial combinations. The combinations, in lockstep with SFT’s mathematical treatment of associated processing-latency distributions, complement the capacity analysis given earlier by discerning competing system architectures,
stopping rules, and in(inter)dependence of processing channels handling the individual targets. A microanalysis of task-performance latency distributions (errors being homogeneously low for Johnson et al.’s both ASD and control participants) was undertaken via the lens of systems-factorial assessment technology.5 Mathematically authorized signatures of double target facilitation, over and against single-target facilitation of processing (“redundancy gain”), was in evidence for all ASD and control participants alike. This aspect of processing evidently was spared with the occurrence of ASD. Contra prominent hypotheses, which were described earlier, all ASD participants displayed a speed advantage for global target processing over local-target processing. In contrast, 4 of the controls exhibited a local-target advantage or approximate equality of target speed. On balance, the verdict from quantitatively disciplined diagnostics was that this property of performance was directly opposite to that predicted by major hypotheses about ASD cognitive functioning. At minimum, a global-target processing advantage was preserved within this ASD sample. Less prominent in the literature have been conjectures about cognitive control in ASD. However, exhaustive target processing was detected as potentially operative among 5, and definitively operative for 2 of the sample of 10 ASD participants (one case being inconclusive). In contrast, for a minority of the 11 controls—4 in number—exhaustive processing was either possibly or definitively operative. The analysis, therefore, revealed that postcriterial continuation of target processing (with possible implications for preservation of processing resources, and the processing apparatus’s inhibition mechanism) may be disorder affected. System workload capacity, chronometrically measured in its own right, nevertheless was at least that of controls—an additional component of evidently spared functioning. Observed violations of selective influence of target-salience manipulations, notably among the control participants, indicated the presence of cross target-processing-channel interactions. The violations impelled the construction of special performance-accommodating theoretical architectures. Certain candidate structures thus were mandated by the present clinical-science samples. The upshot is an example where clinical cognitive science reciprocates to nonclinical cognitive science, in this case by possibly hastening the uncovering
mathematical and computational
355
of potentially important structures in human cognition. cognitive modeling of routinely used measures in clinical science and assessment Measurement in clinical science and assessment frequently has been aimed at the important cognitive-behavioral domain of decision and choice. Examples include the assembling of physical objects based on a judged organizing principle, executing risky gambles, and withholding versus emitting a response to a presenting cue. These decisionchoice scenarios are instantiated in the Wisconsin Sorting Test (WCST; Berg, 1948), which targets frontal lobe “executive function”; the Iowa Gambling Task (Bechara, Damasio, Damasio & Anderson, 1994), which is directed to decisions potentially abetted by accompanying affect; and the Go/No-Go Discrimination Task (see, e.g., Hoaken, Shaughnessy & Pihl, 2003), which is thought to engage inhibitory aspects of cognitive control. Deficits in decisional operations are poised to be ecologically consequential, when it comes to social, occupational, and self-maintenance activities [see Neufeld & Broga, 1981, for a quantitative portrayal of “critical” (versus “differential”) deficit, a concept recently relabelled “functional deficit”, e.g., Green, Horan & Sugar, 2013]. The Expectancy Valence Learning Model (EVL; Busemeyer & Myung, 1992; Busemeyer & Stout, 2002; Yechiam, Veinott, Busemeyer, & Stout, 2007; see also Bishara et al., 2010, and Fridberg et al., 2010, for related sequential learning models) is a stochastic dynamic model (see Busemeyer & Townsend, 1993) that supplies a formal platform for interpreting performance on such measurement tasks. The model expresses the dynamics of decisional behaviors in terms of the progression of expected values accrued by task alternatives, as governed by the record of outcomes rendered by choice-responses to date. Dynamic changes in alternative-expectations are specified by the model structure, in which are embedded the psychological forces—model parameters—operative in generating selection likelihoods at the level of constituent selection trials. Parameters of the EVL model deliver notable psychometric “added value” when it comes to the interpretation and clinical-assessment utility of task-measure data.
356
new directions
Box 2 A Novel Statistical Test for Model-Parameter Differences As indicated in the text, mathematical modeling can prescribe its own measures, experimentation, and tests. Sometimes, proposed tests can transcend the specific domain from which they emerge. This is the case for a statistical test for inequality of model properties between groups, devised by Chechile (2007; 2010). Considerations begin with a “horse race” model of cognitive processes occurring in parallel (Townsend & Ashby, 1983, e.g., p. 249). At any time point since the start of the race t’, the probability density function of the first process of the pair winning is its density function f 1 (t’) times the probability of the second function remaining incomplete S 2 (t’), or f 1 (t’) S 2 (t’). Integrating this expression from t’ = 0 to t’= t gives the probability of the first completion being earlier than the second, as evaluated to time t. Integrating across the entire range of values (t’= 0 to t’= ∞) gives the unconditional probability of the first process having a shorter time than the second. Chechile has adapted this reasoning to the construction of a statistical test expressing the probability that a model-parameter value under one condition of cognitive-behavioral performance is less than its value under a comparison condition Pr(θ 1 <θ 2 ). The method uses Bayesian posterior distributions of θ 1 and θ 2 (with qualifications described and illustrated empirically in Chechile, 2007). Slicing the range of θ into small segments, the proportion of the distribution of θ 1 within a particular segment, times the proportion of the θ 2 distribution beyond that segment, gives the probability of θ 1 lying within the segment, and that of θ 2 lying beyond, analogous to f 1 (t’) and S 2 (t’). Summing the products over the entire range of θ (if θ 1 and θ 2 themselves are probabilities, then θ ranges from 0 to 1.0) directly gives the unconditional probability of θ 1 being lower than θ 2 . Such a probability of .95, for instance, corresponds to a 95% level of confidence that θ 1 < θ 2.
Model parameters tap motivational, learning and response, domains of decision and choice. The first ascertains relative sensitivity to positive versus negative outcomes to selected alternatives (e.g., payoff versus loss to a card-selection gamble). A second performance parameter encapsulates the comparative strength of more versus less recent choice outcomes in influencing current responses. And the third parameter conveys the degree to which responding is governed by accumulated information, as opposed to momentary, ephemeral influences (informational grounding, versus impulsivity). The model, therefore, contextualizes the dynamics of choice responding in terms of rigorously estimated, psychologically meaningful constructs. The EVL model, and its closely aligned sequential learning derivatives (e.g., Ahn, Busemeyer, Wagenmakers & Stout, 2008), have successfully deciphered sources of choice-selection abnormalities in several clinical contexts. Studied groups have included those with Huntington’s and Parkinson’s disease (Busermeyer & Stout, 2002), bipolar disorder (Yechiam, Hayden, Bodkins, O’Donnell & Hetrick, 2008), and various forms of substance abuse (Bishara, et al., 2010; Fridberg, et al., 2010; Yechiam, et al., 2007), and autism spectrum disorders (Yechiam, Arshavsky, ShamayTsoory, Yanov & Aharon, 2010). Attestation to the value of EVL analyses, among other forms, has been that of multiple dissociation of parameterized abnormalities across the studied groups. For example, among Huntington’s disease individuals, the influence of recent outcome experiences in IGT selections has ascended over that of more remote episodes; and responding has been less consistent with extant information as transduced into alternative-outcome modeled expectations (Busemeyer & Stout, 2002; Yechiam, Veinott, Busemeyer & Stout, 2007). Judgments of both stimulant- and alcohol-dependent individuals, on the other hand, have tracked negative feedback to WCST selections to a lesser degree than have those of controls. Although similarly less affected by negative feedback, stimulant-dependent individuals have been more sensitive than alcohol-dependent individuals to positive outcomes attending their selection responses (Bishara et al., 2010). Through the psychological significance of their constituent parameters, sequential learning models have endowed the routinely used measures, described earlier, with incremental content validity and construct representation (Embretson, 1983).
Conventional measures have been understood with regard to their dynamical-process underpinnings. For instance, WCST performance (e.g., perseverative errors, whereby card sorting has not transitioned to a newly imposed organizing principle) has been simulated by modeled sequential-learning (Bishara, et al., 2010). Errors of commission on the go–no-go task have been associated with elevated attention to reward outcomes, and errors of omission have been associated with greater attention to punishing outcomes (Yechiam et al., 2006). Model parameters also have lent incremental nomothetic span (Embretson, 1983) to routinely used measures. Specifically, in addition to producing theoretically consistent diagnosticgroup correlates (e.g., elevated sensitivity to IGTselection rewards, among cocaine users; Yechiam et al., 2007), parameters have been differentially linked to relevant multi-item psychometric measures (Yechiam, et al., 2008). Moreover model-based measures have displayed diagnostic efficacy incremental to conventional measures. Individual differences in model parameters have added to the prediction of bipolar disorder, over and above that of cognitive-functioning and personality/temperament inventories (Yechiam et al., 2008). In addition to its explanatory value for conventionally measured WCST performance, model parameters comprehensively have captured conventional measures’ diagnostic sensitivity to substance abuse, but conventional measures have failed to capture that of model parameters (Bishara et al., 2010). In addition to the credentials enumerated, performance of the EVL model and its affiliates have survived the crucible of competition against competing model architectures (e.g., Yechiam et al., 2007). Applied versions of EVL and closely related sequential learning models also have been vindicated against internal variations on the ultimately applied versions (e.g., attempts at either reduced or increased parameterization; Bishara, et al., 2010; Yechiam, et al., 2007). formal modeling of pathocognition, and functional neuroimaging (the case of stimulus-encoding elongation in schizophrenia) Mathematical and computational modeling of cognitive psychopathology can provide vital information when it comes to the functional component
mathematical and computational
357
of clinical functional neuroimaging (functional magnetic resonance imaging, magnetic resonance spectroscopy, electroencephalography, and electromagnetoencephalography). Cognitive-process unfolding, as mathematically mandated by a tenable process model, can speak to the why, where, when, and how of neurocircuitry estimation. In so doing, it can provide an authoritative antidote to the vexatious problem of reverse inference, whereby functions whose neurocircuitry is purported as being charted, are inferred from the monitored neuro-circuitry itself (e.g., Green, et al., 2013; see Poldrack, 2011). So called event-related neuroimaging seeks to track sites of neuro-activation, and intersite coactivation, aligned with transitions taking place in experimental cognitive-performance paradigms (e.g., occurrence of a probe item, for ascertainment of its presence among a set of previously memorized items, in a memory-search task; or presentation of a visual array, for ascertainment of target presence, in a visual-search task). Events of actual interest, however, are the covert cognitive operations activated by one’s experimental manipulations. Estimating the time course of events that are cognitive per se arguably necessitates a tenable mathematical model of their stochastic dynamics. Stipulation of such temporal properties seems all the more important when a covert process of principal interest defensibly is accompanied by collateral processes, within a cognitive-task trial (e.g., encoding the probe item for purposes of memory-set comparison, along with response operations,—earlier). Motivation for model construction is fueled further if the designated intratrial process has symptom and ecological significance. The upshot is that mathematical modeling of cognitive abnormality is uniquely poised to supply information as the why, where, and when of clinical and other neuroimaging. It arguably also speaks to the how of vascular- and electro-neurophysiological signal processing, in terms of selection from among the array of signal-analysis options. Clinical importance of neurocircuitry measures can be increased through their analytical interlacing with clinical symptomatology. Among individuals with schizophrenia, for example, elongated duration of encoding (cognitively preparing and transforming) presenting stimulation into a format that facilitaties collateral processes (e.g., those taking place in so-called working memory) can disproportionately jeopardize the intake of contextual cues that are vital to anchoring other 358
new directions
input in its objective significance (i.e., “context deficit”; e.g., Dobson & Neufeld, 1982; George & Neufeld, 1985). This combination potentially contributes to schizophrenia thought-content disorder (delusions and thematic hallucinations; Neufeld, 1991; 2007c). Formally prescribed cognitive psychopathology thus can help broker symptom significance to monitored neurocircuitry. Monitored neurocircuity likewise can be endowed with increased ecological significance. Intact encoding, for example, may be critical to negotiating environmental stressors and demands, especially when coping is cognition intensive (e.g., where threat minimization is contingent on predictive judgments surrounding coping alternatives; e.g., Neufeld, 2007b; Shanahan & Neufeld, 2010). Impairment of this faculty therefore may be part and parcel of stress susceptibility (Normal & Malla, 1994). Compromised stress resolution in turn may further impair encoding efficiency, and so on (Neufeld, et al., 2010). Turning to the where of neuro-imaging measurement, targeted cognitive functions in principle are used to narrow down neuro-anatomical regions of measurement interest, according to locations on which the functions are thought to supervene. Precision of function specification constrained by formal modeling in principle should reduce ambiguity about what cognitively is being charted neurophysiologically. Explicit quantitative stipulation of functions of interest should sharpen imaging’s functionally guided spatial zones of brain exploration. Formal modeling of cognitive-task performance seems virtually indispensable to carving out intratrial times of neuro-(co)activation measurement interest. Stochastic dynamic models specify probabilistic times of constituent-process occurrence. For instance, encoding a probe item for its comparison to memory-held items would have a certain time trajectory following the probe’s presentation. With encoding as the singled-out process, imaging signals identified with a trial’s model-estimated epoch, during which the ascendance of encoding was highly probable, would command special attention. A tenable stochastic model of trial performance can convey the stochastic dynamics of a designated symptom- and ecologically significant process, necessary for a reasonable estimate of its corresponding within-trial neuro-circuitry (additional details are available in Neufeld, 2007c, the mathematical and neuro-imaging specifics of which are illustrated in Neufeld, et al., 2010).
A delimited intratrial measurement epoch stipulated by a stochastic-model specified-time trajectory commands considerable temporal resolution of processed imaging signals. Among analysis options, covariance of activation between measurement sites, across time (“seed-voxel, time series covariance”) has been shown empirically and mathematically to have the temporal resolution necessary to monitor neurocircuitry during the designated epoch (Neufeld, 2012; see also Friston, Fletcher, Josephs, Holmes & Rugg, 1998). In this way, formal modeling of cognitive performance speaks to the how of imaging-signal processing by selecting out or creating neuro-imaging-signal analysis options with requisite temporal resolution. Note that success at ferreting out target-process neurocircuitry is supported by the selective occurrence of differential neuro-(co)activation to the modeled presence of the triaged process in the clinical sample, and conversely, its nonoccurrence otherwise (also known as “multiple dissociation”). Model-informed times of measurement interest stand to complement tendered regions of interest, in navigating space-time coordinates of neuroimaging measurement. Moreover, symptom and ecologically significant functions remain in the company of collateral functions involved in task-trial transaction (e.g., item encoding occurs alongside the memoryscanning and response processes, for which it exists). Keeping the assembly of trial-performance operations intact, while temporally throwing into relief the target process, augurs well for preserving the integrity of the target process, as it functions in situ. Doing so arguably increases the ecological validity of findings, over and against experimentally deconstructing the performance apparatus by experimentally separating out constituent processes, thereby risking distortion of their system-level operation.
Special Considerations Applying to Mathematical and Computational Modeling in Psychological Clinical Science and Assessment Clinical mathematical and computational modeling is a relatively recent development in the history of modeling activity (see, e.g., Townsend, 2008). It goes without saying that opportunities unlocked by clinical implementations are accompanied by intriguing challenges. These take the form of upholding rigor of application amid exigencies imposed by clinical constraints, along
with faithfully engaging major substantive issues found in clinical science and assessment.
Methodological Challenges One of the first challenges to clinical formal modeling consists of obtaining sufficient data for stability of model-property estimation. This challenge is part and parcel of the field’s longstanding concern with ensuring the applicability of inferences to a given individual or, at least, to a homogeneous class of individuals to which the one at hand belongs (Davisdon & Costello, 1969; Neufeld, 1977). Mainline mathematical modeling too has been concerned with this issue. The solution by and large has been to model performance based on a large repertoire of data obtained from a given participant, tested over the course of a substantial number of multitrial sessions. Clinical exigencies (e.g., participant availability or limited endurance), however, may proscribe such extensiveness of data accumulation; the trade-off between participants and trial number may have to be tipped toward more participants and fewer trials.6 The combination of fewer performance trials, but more participants requires built-in checks for relative homogeneity of individual data protocols, to ensure that the modeled aggregate is representative of its respective contributors. Collapsing data across participants within a testing session, then, risks folding systematic performance differences into their average, with a resultant centroid that is unrepresentative of any of its parts (see Estes, 1956). Note in passing that similar problems could attend participant-specific aggregation across experimental sessions, should systematic intersession performance drift or re-configuration occur. Methods have been put forth for detecting systematic heterogeneity (Batchelder & Riefer, 2007; Carter & Neufeld, 1999; Neufeld & McCarty, 1994), or ascertaining its possibility to be inconsequential to the integrity of modeling results (Riefer, Knapp, Batchelder, Bamber & Manifold, 2002). If present, methods of disentangling, and separately modeling the conflated clusters of homogenous data have been suggested (e.g., Carter, Neufeld, & Benn, 1998). Systematic individual differences in model properties also can be accommodated by building them into the model architecture itself. Mixture models implement heterogeneity in model operations across individual and/or performance trials. For example, model parameters may differ across
mathematical and computational
359
individual participants within diagnostic groups, the parameters in principle forming their own random distribution. As parameter values now are randomly distributed across individuals (hyperdistribution), a parameter value’s probability (discrete-value parameters) or probability density (continuous-value parameters) takes on the role of its Bayesian prior probability (density) for the current population of individuals. Combined with a sample of empirical task performance, on whose distribution the parameter bears (base distribution), an individualized estimate of the parameter is available through its Bayesian posterior parameter distribution (see Bayesian Parameter Estimation, earlier). A mixture-model approach can be especially appealing in cases of overdispersion, as indicated say by a relatively large coefficient of variation (standard deviation of individual mean performance values divided by their grand mean; see overdispersion under Bayesian Parameter Estimation, earlier).7 Also availed through mixture models are customized (again, Bayesian posterior) performance distributions. If the latter apply to process latencies, for example, they can be consulted to estimate strategic times of neuro-imaging measurement within subgroups of parametrically homogeneous participants (detailed in Neufeld, et al., 2010, Section 7). Apropos of the limitations indigenous to the clinical enterprise, mixture models potentially moderate demands on participants when it comes to their supplying a task- performance sample. A relatively modest sample tenably is all that is required, because of the stabilizing influence on participant-specific estimates, as bestowed by the mixing distribution’s prior-held information. This obviously is a boon in the case of clinical modeling, considering the possible practical constraints in obtaining a stable performance sample from already distressed individuals. Mixture models also open up related avenues of empirical model testing. For purposes of assessing model validity, predictions of individuals’ performance, mediated by modest performance samples, can be applied to separately obtained individual validation samples. In this way, model verisimilitude can be monitored, including the model’s functioning at the level of individual participants (for details exposited with empirical data, see Neufeld, et al., 2010). Because of the stabilizing influence of the Bayesian prior distribution of basedistribution properties, this strategy provides a 360
new directions
tack to the thorny issue of small-N model testing (see Bayesian shrinkage, under Bayesian Parameter Estimation, earlier). An additional asset conferred by a mixturemodel platform bridges methodological and clinical substantive issues, as follows. Emerging from this model design is a statistical—and cognitivebehavioral—science-principled measurement infrastructure for dynamically monitoring individual treatment response. The method exploits the distinct prior distributions of base-model properties obtained from varyingly symptomatic and healthy groups. Now Bayes’ theorem allows the profiling of an individual at hand, with respect to the latter’s current probability of belonging to group g, g = 1,2, . . . G (G being the number of referent groups), given obtained performance sample {*}: Pr(g|{*}). Such a profile can be updated with the progression of treatment. Moreover, closely related procedures can be applied to the assessment of treatment regimens. In this case, treatment efficacy is monitored by charting the degree to which the treated sample at large is being edged toward or possibly away from healthier functioning (computational specifics are illustrated with empirical data in Neufeld, 2007b, and Neufeld, et al., 2010).
Clinical substantive issues A substantive issue interlaced with measuretheoretic considerations is the so-called differentialdeficit, psychometric-artefact problem (Chapman & Chapman, 1973). Abnormalities that are more pronounced than others may have special etiological importance, in part because of associated neuro-physiological substrates. False inferences of differential deficit, however, are risked because the relative amounts by which examined faculties are disorder-affected, are conflated with the psychometric properties of the instruments used to measure them. This issue retains much currency in the contemporary clinical-science literature (e.g., Gold & Dickinson, 2013). Frailties of recommended solutions, consisting of intermeasure calibration toward equality on classical measurement properties (reliability and observedscore variance) have been noted almost since the recommendations’ inception (e.g., Neufeld, 1984a; Neufeld & Broga, 1981; with augmenting technical reports, Neufeld & Broga, 1977; Neufeld 1984b). Note that the arguments countering the original classical-measurement recommendations have been demonstrated to extend from diagnostic-group to
continuous-variable designs (as used, e.g., by Kang & MacDonald, 2010; see Neufeld & Gardner, 1990). It can be shown that transducing classical psychometric partitioning of variance into formally modeled sources not only lends a model-based substantive interpretation to the partitioned variance, but also renders the psychometric-artefact issue (which continues to plague reliance on offthe-shelf and subquantitative, often clinically, rather than contemporary cognitive-science, contrived measures) essentially obsolete (Neufeld, 2007b; see also McFall & Townsend, 1998; Neufeld, Vollick, Carter, Boksman, Jette, 2002; Silverstein, 2008). Partitioned sources of variance in task performance now are specified according to a governing formal performance model, which incorporates an analytical stipulation of its disorder-affected and spared features (see Figure 16.1, and related prose, in the Introduction of this chapter). These sources of variance include classical measurementerror variance (within-participant variance); withingroup interparticipant variance; and intergroup variance. Another major substantive issue encountered in clinical science concerns the integration of socalled cold and hot cognition. Cold cognition, in this context pertains more or less to the mechanisms of information processing, and hot cognition pertains to their informational product, notably with respect to its semantic and affective properties—roughly, How is the thought delivered (cold cognition), and what does the delivery consist of (hot cognition). Deviations in the processing apparatus leading to interpretations of one’s environment cum responses that bear directly on symptomatology hold special clinical significance. The synthesis of cold and hot cognition perhaps is epitomized in the work on risk of sexual coercion, and eating and other disorders, by Teresa Treat, Richard Viken, Rchard McFall, and their prominent cognitive-scientist collaborators. Examples include Treat et al. (2002), and Treat, Viken, Kruschke & McFall (2010; see also Treat & Viken, 2010). These investigators have shown how basic perceptual processes (perceptual organization) can ramify to other clinically cogent cognitivebehavioral operations, including those of classification in interpersonal perception, and memory. This program of research exemplifies integrative, translational, and collaborative psychological science. It has been rigorously eclectic in its recruitment
of modeling methodology. Included have been multidimensional scaling; classification paradigms; perceptual-independence—covariation paradigms; and memory paradigms–all work that brooks no compromise on state-of-the-art mathematical and computational developments. Deviations in cognitive mapping, classification, and memory retrieval, surrounding presented items bearing on clinically important problems comprising eating disorders, and risk of sexual aggression have been productively studied. A further point of contact between formal process modeling and substantive clinical issues involves relations between the former and multiitem psychometric inventories. Empirical associations between model properties, and multi-item measures, reciprocally contribute to each other’s nomothetic-span construct validity. Correlations with process-model properties furthermore lend construct-representation construct validity to psychometric measures. Psychometric measures, in turn, can provide useful estimates of model properties with which they sufficiently correlate, especially if their economy of administration exceeds that of direct individual-level estimates of the model properties (Carter, Neufeld & Benn, 1998; Wallsten, Pleskac & Lejuez, 2005). Furthermore, to liaison with multi-item psychometric inventories, responding to a test item can be viewed as an exercise in dynamical information processing. As such, modeling option selection via item-response theory can be augmented with modeling item-response time, the latter as the product of a formally portrayed dynamic stochastic process (e.g., Neufeld, 1998; van der Maas, Molenaar, Maris, Kievit & Boorsboom, 2011; certain direct parallels between multi-item inventories and stochastic process models have been developed in Neufeld, 2007b). Note that a vexing contaminant of scores on multi-item inventories is the influence of social desirability (SD), that is, the pervasive inclination to respond to item options in a socially desirable direction. Indeed, a prime mover of contemporary methods of test construction, Douglas N. Jackson, has stated that SD is the g factor of personality and clinical psychometric measures (e.g., Helmes, 2000). Interestingly, it has been shown that formal theory and its measurement models can circumvent the unwanted influence of SD, because of the socially neutral composition of the measures involved (Koritzky & Yechiam, 2010).
mathematical and computational
361
Conclusion Mathematical and computational modeling arguably is essential to accelerating progress in clinical science and assessment. Indeed, rather than its being relegated to the realms of an esoteric enterprise or quaint curiosity, a strong case can be made for the central role of quantitative modeling in lifting the cognitive side of clinical cognitive neuroscience out of current quagmires and avoiding future ones. Clinical formal modeling augurs for progress in the field, owing to the explicitness of theorizing to which the modeling endeavor is constrained. Returns on research investment are elevated because rigor of expression exposes empirical shortcomings of extant formulations; blind alleys are expressly tagged as such, thanks to the definiteness of derived formulas, so that wasteful retracing is avoided and needed future directions often are indicated, because of the forced exactness of one’s quantitative formulations. Otherwise unavailable or intractable information is released. Moreover, existing data can be exploited by freshly viewing and analyzing it through methods opened up by a valid mathematical, computational infrastructure (Novotney, 2009). Formal developments also stand to enjoy an elongated half-life of contribution to the discipline. Staying power of the value of rigorous and substantively important achievements has been seen in mathematical modeling generally. For instance, early fundamental work on memory dynamics (e.g., Atkinson, Belseford & Shiffrin, 1967) recently has been found useful in the study of memory in schizophrenia (Brown et al., 2007). A similar durability stands to occur for clinical mathematical psychology, within the field of clinical science itself, and beyond. Certain suspected agents of disturbance may resist study through strictly experimental manipulation of independent variables (e.g., non–modelinformed correlational experiments; see earlier section Mulitnomial Processing Tree Modeling of Memory and Related Processes; Unveiling and Elucidating Deviations in Clinical Samples earlier). Such may be the case on ethical grounds, or because the theoretically tendered agent may elude experimental induction (e.g., organismically endogenous stress activation suspected as generating cognitiondetracting intrusive associations and/or diminished cognitive-workload capacity). Tenability of such conjectures nevertheless may be tested by examining improvement in model fit to data from afflicted 362
new directions
individuals, through quantitatively inserting the otherwise inaccessible constructs into the model composition. Current challenges can be expected to spawn vigorous research activity in several future directions. Among others, included are: (a) capitalizing on modeling opportunities in multiple areas of clinical science, poised for modeling applications (e.g., stress, anxiety disorders, and depression); (b) surmounting constraints intrinsic to the clinical setting that pose special barriers to the valid extraction of modeling-based information (e.g., estimating and providing for potentially confounding effects of medication on cognitive-behavioral task performance); (c) developing sound methods for bridging mathematical and computational clinical science to clinical assessment and intervention; and, (d), creating methods of tapping modeldisciplined information from currently available published data—in the service of model-informed, substantively meaningful, meta analyses. It may be said that evidence-based practice is best practice only if it is based on best evidence. Best evidence goes hand in hand with maximizing methodological options, which compels the candidacy of those that stem from decidedly quantitative theorizing.
Acknowledgments Manuscript preparation, and the author’s own work reported in the manuscript, were supported by grants from the Social Science and Humanities Research Council of Canada (author, Principal Investigator), the Medical Research Council of Canada (author, Principal Investigator) and the Canadian Institutes of Health Research (author, co-investigator). I thank Mathew Shanahan for helpful comments on an earlier version, and Lorrie Lefebvre for her valuable assistance with manuscript preparation.
Notes 1. See also Chechile (2010) for the introduction of a novel procedure for individualizing MPTM parameter values, which is especially pertinent to clinical assessment technology. See also Smith & Batchelder (2010) on computational methods for dealing with the clinical-science issue of group-data variability, owing to individual differences in parameter values (the problem of “overdispersion”). 2. Still others (Ouimet, Gawronski & Dozois, 2009) have tendered extensive verbally conveyed schemata and flow diagrams [cf. McFall, Townsend & Viken (1995) for a poignant demarcation of models qua models] entailing dual-system cognition (extrapolation of “automatic” and “controlled” processing properties; Schneider & Shiffrin, 1977; Shiffrin & Schneider,
1977), and threat-stimulus disengagement deficit [possibly suggesting serial item processing; but cf. Neufeld & McCarty (1994); Neufeld, Townsend & Jette (2007); White, et al., 2010a, p. 674]. 3. An independent parallel, limited-capacity processing system, with exponentially distributed processing latencies, potentially characterizing the processing architectures of both HA and LA individuals at least during early processing (Neufeld & McCarty, 1994; Neufeld, Townsend & Jette, 2007): (a) parsimoniously can be shown to cohere with reaction-time findings on HA threat bias, including those taken to indicate threat stimulus disengagement deficit; and, (b) also can be shown to cohere with collateral oculomotor data (e.g., Mogg, Millar & Bradley, 2000; Neufeld, Mather, Merskey & Russell, 1995; see also, Townsend & Ashby, 1983, chapter 5). 4. Note that encoding and response processes typically have been labeled “base processes,” and often have been relegated to “wastebasket” status. They may embody important differences, however, between clinical samples and controls. By examining “boundary separation,” a response-criterion parameter of the Ratcliff diffusion model, for example, White, Ratcliff, Vasey and McKoon (2010b) have discovered that HA, but not LA participants, increase their level of caution in responding after making an error (an otherwise undetected “alarm reaction”). See also Neufeld, Boksman, Vollick, George and Carter (2010), for analyses of potentially symptom significant elongation of stimulus encoding—a process subserving collateral cognitive operations— among individuals with a diagnosis of schizophrenia. 5. Extensions to mathematically prescribed signatures of variants on the cognitive elements described earlier, recently have included statistical significance tests and charting of statistical properties (e.g., “unbiasedness” and “statistical consistency” of signature estimates; Houpt & Townsend, 2010; 2012) and measures of theoretic and methodological unification of concepts in the literature related to cognitive-workload capacity (known as “Grice” and “Miller inequalities”), all under qualitative properties of SFT’s quantitative diagnostics (Townsend & Eidels, 2011). Bayesian extensions also currently are in progress. A description of software for SFT, including a tutorial on its use, has been presented by Houpt, Blaha, McIntire, Havig and Townsend (2014). 6. The challenge of requisite data magnitude may be overblown. Meaningful results at the individual level of analysis, for example, can be obtained with as few as 160 trials per experimental condition–if the method of analysis is mathematical-theory mandated (see Johnson, et al., 2010). Furthermore, clinicians who may balk at the apparent demands of repeated-trial cognitive paradigms nevertheless may have no hesitation in asking patients to respond to 567 separate items of a multi-item inventory. 7. Of late, cognitive science has witnessed a burgeoning interest in mixture models, dubbed “hierarchical Bayesian analysis” (e.g., Lee, 2011). The use of such models in cognitive science nevertheless can be traced back at least two and one half decades (e.g., Morrison, 1979; Schweickert, 1982). Mixture models of various structures (infinite, and finite, continuous and discrete), along with their Bayesian assets, moreover have enjoyed a certain history of addressing prominent issues in clinical cognitive science (e.g., Batchelder, 1998; Carter, et al., 1998; Neufeld, 2007b; Neufeld, Vollick, Carter, Boksman, & Jette, 2002; Neufeld & Williamson, 1996), some of which are stated in the text.
Glossary analytical construct validity: Support for the interpretation of a model parameter that expresses an aspect of individual differences in cognitive performance, according to the parameter’s effects on model predictions. base distribution: Distribution of a cognitive-behavioral performance variable (e.g., speed or accuracy) specified by a task-performance model. construct-representation construct validity: Support for the interpretation of a measure in terms of the degree to which it comprises mechanisms that are theoretically meaningful in their own right. exponential distribution: A stochastic distribution of process latencies whose probability density function is νe−νt , where t is time (arbitrary units), and v is a parameter; latencies are loaded to the left (i.e., toward t = 0, their maximum probability density being at t = 0), and the distribution has a long tail. hyperdistribution: A stochastic distribution governing the probabilities or probability densities of properties (e.g., parameters) of a base distribution (see section Bayesian Parameter Estimation in the text). likelihood function: A mathematical expression conveying the probability or probability density of a set of observed data, as a function of the cognitive-behavioral task-performance model (see section Bayesian Parameter Estimation in the text). mixing distribution: See hyperdistribution. mixture model: A model of task performance, expressed in terms a base distribution of response values, whose base-distribution properties (e.g., parameters) are randomly distributed according to a mixing distribution (see section Bayesian Parameter Estimation in the text). multiple dissociation: Cognitive-behavioral performance patterns or estimated neuro-circuitry occurring for a clinical group under selected experimental conditions, with contrasting patterns occurring under other conditions. The profile obtained for the clinical group is absent or opposite for control or other clinical groups. nomothetic-span construct validity: Support for the interpretation of a measure according to its network of associations with variables to which it is theoretically related. normalizing factor: Overall probability or probability density of data or evidence, all model-prescribed conditions (e.g., parameter values) considered (see section Bayesian Parameter Estimation, in the text). overdispersion: Greater variability in task-performance data than would be expected if a fixed set of model parameters were operative across all participants. prior distribution: In the Bayesian framework, the probability distribution before data are collected (See hyperdistribution). posterior distribution: In the Bayesian framework, the distribution corresponding to the prior distribution after the data are collected. Bayes’ theorem is used to update the prior distribution, following data acquisition. The key agent of this updating is the likelihood function (see section Bayesian Parameter Estimation in the text).
mathematical and computational
363
References Ahn, W. Y., Busemeyer, J. R., Wagenmakers, E. J., & Stout, J. C.(2008). Comparison of decision learning models using the generalization criterion method. Cognitive Science, 32, 1376–1402. Atkinson, R. C., Brelsford, J. W., & Shiffrin, R. M. (1967). Multiprocess models for memory with applications to a continuous presentation task. Journal of Mathematical Psychology, 4, 277–300. Batchelder, W. H. (1998). Multinomial processing tree models and psychological assessment. Psychological Assessment, 10, 331–344. Batchelder, W. H., & Riefer, D. M. (1990). Multinomial processing models of source monitoring. Psychological Review, 97, 548–564. Batchelder W. H., & Riefer, D. M. (1999). Theoretical and empirical review of multinomial process tree modelling. Psychonomic Bulletin & Review, 6, 57–86. Batchelder, W. H., & Riefer, D. M. (2007). Using multinomial processing tree models to measure cognitive deficits in clinical populations. In R. W. J. Neufeld (Ed.), Advances in clinical cognitive science: Formal modeling of processes and symptoms (pp. 19–50). Washington, D.C: American Psychological Association. Bechara A., Damasio A. R., Damasio H., & Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex. Cognition 50, 7–15. Berg, E. A. (1948). A simple objective technique for measuring flexibility in thinking. Journal of General Psychology, 39, 15–22. Berger, J. O. (1985). Statistical decision theory and bayesian analysis, (2nd ed.). New York, NY: Springer. Bianchi, M. T., Klein, J. P., Caviness, V. S., & Cash, S. S. (2012). Synchronizing bench and bedside: A clinical overview of networks and oscillations. In M. T. Bianchi, V. S. Caviness, & S. S. Cash (Eds.), Network approaches to diseases of the brain: Clinical applications in neurology and psychiatry (pp. 3–12). Oak Park, IL: Bentham Science Publishers. Bishara, A., Kruschke, J., Stout, J., Bechara, A., McCabe, D., Busemeyer, J., (2010). Sequential learning models for the Wisconsin card sort task: Assessing processes in substance dependent individuals. Journal of Mathematical Psychology, 54, 5–13. Bolger, N., Davis, A., & Rafaeli, E. (2003) Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579–616. Borowski, E. J., & Borwein, J. M. (1989). The Harper Collins Dictionary of Mathematics (2nd ed.). New York, NY: Harper Collins. Borsboom, D., & Cramer, A. O. J. (2013). Network analysis: An integrative approach to the structure of psychopathology. Annual Review of Clinical Psychology, 9, 91–121. Braithwaite, R. B. (1968). Scientific explanation. London, England: Cambridge University Press. Brown, G. G., Lohr, J., Notestine, R., Turner, T., Gamst, A., & Eyler, L. T. (2007). Performance of schizophrenia and bipolar patients on verbal and figural working memory tasks. Journal of Abnormal Psychology, 116, 741–753. Busemeyer, J. R., & Diedrich, A. (2010) Cognitive modeling. Thousand Oaks, CA: Sage.
364
new directions
Busemeyer, J. R. & Myung, I. J. (1992). An adaptive approach to human decision making: Learning theory, decision theory, and human performance. Journal of Experimental Psychology: General, 121, 177–194. Busemeyer, J. R., & Stout, J. C. (2002). A contribution of cognitive decision models to clinical assessment: Decomposing performance on the Bechara gambling task. Psychological Assessment, 14, 253–262. Busemeyer, J. R., & Townsend, J. T. (1993). Decision field theory: A dynamic-cognitive approach to decision making in an uncertain environment. Psychological Review, 100, 432–459. Busemeyer, J. R. & Wang, Y. (2000). Model comparisons and model selections based on generalization test methodology. Journal of Mathematical Psychology, 44(1), 171–189. Carter, J. R., & Neufeld, R. W. J. (2007). Cognitive processing of facial affect: Neuro-connectionist modeling of deviations in schizophrenia. Journal of Abnormal Psychology, 166, 290–305. Carter, J. R., Neufeld, R. W. J., & Benn, K. D. (1998). Application of process models in assessment psychology: Potential assets and challenges. Psychological Assessment, 10, 379–395. Chapman, L. J., & Chapman, J. P. (1973). Problems in the measurement of cognitive deficit. Psychological Bulletin, 79, 380–385. Chechile, R. A. (1998). A new method for estimating model parameters for multinomial data. Journal of Mathematical Psychology, 42, 432–471. Chechile, R. A. (2004). New multinomial models for the Chechile–Meyer task. Journal of Mathematical Psychology, 48, 364–384. Chechile, R. A. (2007). A model-based storage-retrieval analysis of developmental dyslexia. In R. W. J. Neufeld (Ed.), Advances in clinical cognitive science: Formal modeling of processes and symptoms (pp. 51–79). Washington, DC: American Psychological Association. Chechile, R. A. (2010). Modeling storage and retrieval processes with clinical populations with applications examining alcohol-induced amnesia and Korsakoff amnesia. Journal of Mathematical Psychology, 54, 150–166. Close, F. (2011). The infinity puzzle: How the hunt to understand the universe led to extraordinary science, high politics, and the large hadron collider. Toronto, Canada: A. Knopff. Cox, D. R. & Miller, H. D. (1965). The theory of stochastic processes. London, England: Methuen. Cramer, A. O. J., Waldorp, L. J., van der Maas, H., & Borsboom, D. (2010). Comorbidity: A network perspective. Behavioral and Brain Sciences, 33, 137–193. Dance, K., & Neufeld, R. W. J. (1988). Aptitude-treatment interaction in the clinical setting: An attempt to dispel the patient-uniformity myth. Psychological Bulletin, 104, 192–213. Davidson, P. O., & Costello, G. G. (1969). N=1: Experimental studies of single cases. New York, NY: Van Nostrand Reinhold. Dobson, D., & Neufeld, R. W. J. (1982). Paranoid-nonparanoid schizophrenic distinctions in the implementation of external conceptual constraints. Journal of Nervous and Mental Disease, 170, 614–621. Doob, J. L. (1953). Stochastic processes. New York, NY: Wiley.
Embretson, W. S. (1983). Construct validity: Construct representation versus nomothetic span. Psychological Bulletin, 93, 179–197. Estes, W. K. (1956). The problem of inference from curves based on group data. Psychological Bulletin, 53, 134–140. Evans, M., Hastings, N., & Peacock, B. (2000). Statistical distributions (3rd Ed.). New York, NY: Wiley. Farrell, S., & Lewandowsky, S. (2010). Computational models as aids to better reasoning in psychology. Current Directions in Psychological Science, 19, 329–335. Flanagan, O. (1991). Science of the mind (2nd ed.). Cambridge, MA: MIT Press. Frewen, P. A., Dozois, D., Joanisse, M., & Neufeld, R. W. J. (2008). Selective attention to threat versus reward: Meta-analysis and neural-network modeling of the dot-probe task. Clinical Psychology Review, 28, 308–338. Fridberg, D. J., Queller, S., Ahn, W.-Y., Kim, W., Bishara, A. J., Busemeyer, J. R., Porrino, L., & Stout, J. C. (2010). Cognitive mechanisms underlying risky decision-making in chronic cannabis users. Journal of Mathematical Psychology, 54, 28–38. Friston, K. J., Fletcher, P., Josephs, O., Holmes, A., & Rugg, M. D. (1998). Event-related fMRI: Characterizing differential responses. Neuroimage, 7, 30–40. Fukano T, & Gunji Y. P.(2012). Mathematical models of panic disorder. Nonlinear Dynamics, Psychology and Life Sciences, 16, 457–470. George, L., & Neufeld, R. W. J. (1985). Cognition and symptomatology in schizophrenia. Schizophrenia Bulletin, 11, 264–285. Gold, J. M., & Dickinson, D. (2013). “Generalized Cognitive Deficit” in Schizophrenia: Overused or underappreciated? Schizophrenia Bulletin, 39, 263–265. Gottman, J. M., Murray, J. D., Swanson, C. C., Tyson, R., & Swanson, K. R. (2002). The mathematics of marriage: Dynamic nonlinear models. Cambridge, MA: MIT Press. Green, M. F., Horan, W. P., & Sugar, C. A. (2013). Has the generalized deficit become the generalized criticism? Schizophrenia Bulletin, 39, 257–262. Haig, B. D. (2008). Scientific method, abduction, and clinical reasoning. Journal of Clinical Psychology, 64, 1013–1127. Hartlage, S., Alloy, L. B., Vázquez, C. & Dykman, B.(1993). Automatic and effortful processing in depression. Psychological Bulletin, 113(2), 247–278. Helmes, E. (2000). The role of social desirability in the assessment of personality constructs. In R. D. Goffin & E. Helmes (Eds.). Problems and solutions in human assessment: Honoring Douglas N. Jackson at seventy. Norwell, MA: Kluwer. Hoaken, P. N. S., Shaughnessy, V. K., & Pihl, R. O. (2003). Executive cognitive functioning and aggression: Is it an issue of impulsivity? Aggressive Behavior, 29, 15–30. Hoffman, R. E., & McGlashan, T. H. (2007). Using a speech perception neural network simulation to study normal neurodevelopment and auditory hallucinations in schizophrenia. In R. W. J. Neufeld (Ed.), Advances in clinical cognitive science: Formal modeling of processes and symptoms (pp. 239–262). Washington, D.C.: American Psychological Association.
Houpt, J. W., & Townsend, J. T. (2010). The statistical properties of the survivor interaction contrast. Journal of Mathematical Psychology, 54, 446–453. Houpt, J. W. and Townsend, J. T. (2012). Statistical measures for workload capacity analysis. Journal of Mathematical Psychology, 56, 341–355. Houpt, J. W., Blaha, L. M., McIntire, J. P., Havig, P. R., & Townsend, J. T. (2014). Systems factorial technology with R. Behavior Research Methods, Instrumentation and Computing, 46, 307–330 (Available online from http://link.springer.com/article/10.3758%2Fs13428013-0377-3.) Hu, X. (2001). Extending general processing tree models to analyze reaction time experiments. Journal of Mathematical Psychology, 45, 603–634. Johnson, S. A., Blaha, L. M., Houpt, J. W., & Townsend, J. T. (2010). Systems Factorial Technology provides new insights on global-local information processing in autism spectrum disorders. Journal of Mathematical Psychology, 54, 53–72. Kang, S. S., & MacDonald, A. W. III (2010). Limitations of true score variance to measure discriminating power: Psychometric simulation study. Journal of Abnormal Psychology, 119, 300–306. Koritzky, G., & Yechiam, E. (2010). On the robustness of description and experience based decision tasks to social desirability. Journal of Behavioral Decision Making, 23, 83–99. Lee, M. D. (2011). How cognitive modeling can benefit from hierarchical Bayesian models. Journal of Mathematical Psychology, 55, 1–7. Levy, L. R., Yao, W., McGuire, G., Vollick, D. N., Jetté, J., Shanahan, M. J., & Neufeld, R. W. J. (2012). Nonlinear bifurcations of psychological stress negotiation: New properties of a formal dynamical model. Nonlinear Dynamics, Psychology and Life Sciences, 16, 429–456. Link, S. W. (1982), Correcting response measures for guessing and partial information. Psychological Bulletin, 92, 469–486. Link, S. W., & Day, R. B. (1992). A theory of cheating. Behavior Research Methods, Instruments, & Computers, 24, 311–316. Link, S. W., & Heath, R. A. (1975). A sequential theory of psychological discrimination. Psychometrika, 40, 77–105. Maher, B. (1970). Introduction to research in psychopathology. New York, NY: McGraw-Hill. Marr, D. (1982). Vision. San Francisco, OA: Freeman. McFall, R. M., & Townsend, J. T. (1998). Foundations of psychological assessment: Implications for cognitive assessment in clinical science. Psychological Assessment, 10, 316–330. McFall, R. M., Townsend, J. T. and & Viken, R. J. (1995). Diathesis stress model or "just so" story? Behavioral and Brain Sciences, 18(3), 565–566. Meehl, P. E. (1978). Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology. Journal of Consulting and Clinical Psychology, 46 (4), 806–843. Mogg, K., Millar, N., Bradley, B. P. (2000). Biases in eye movements to threatening facial expressions in generalized anxiety disorder and depressive disorder. Journal of Abnormal Psychology, 109, 695–704.
mathematical and computational
365
Molenaar, P. C. M. (2010). Note on optimization of psychotherapeutic processes. Journal of Mathematical Psychology, 54, 208–213. Morrison, D. G. (1979). An individual-difference pureextinction process. Journal of Mathematical Psychology, 19, 307–315. Moshagen, M. (2010). multiTree: A computer program for the analysis of multinomial processing tree models. Behavior Research Methods, 42(1), 42–54. Neufeld, R. W. J. (1977) Clinical quantitative methods, NY: Grone & Stratton. Neufeld, R. W. J. (1984a). Re: The incorrect application of traditional test discriminating power formulations to diagnostic-group studies. Journal of Nervous and Mental Disease, 172, 373–374. Neufeld, R. W. J. (1984b). Elaboration of incorrect application of traditional test discriminating power formulations to diagnosticgroup studies. Department of Psychology Research Bulletin Number 599. London, ON: University of Western Ontario. Neufeld, R. W. J. (1991). Memory in paranoid schizophrenia. In P. Magaro (Ed.), The cognitive bases of mental disorders: Annual review of psychopathology (Vol. 1, pp. 231–261). Newbury Park, CA: Sage. Neufeld, R. W. J. (1996). Stochastic models of information processing under stress. Research Bulletin No. 734, London, ON: Department of Psychology, University of Western Ontario. Neufeld, R. W. J. (1998). Intersections and disjunctions in process-model applications. Psychological Assessment, 10, 396–398. Neufeld, R. W. J. (1999). Dynamic differentials of stress and coping. Psychological Review, 106, 385–397. Neufeld, R. W. J. (2007a). Introduction. In R. W. J. Neufeld (Ed.), Advances in the clinical cognitive science: Formal modeling of processes and symptoms, Washington, DC: American Psychological Association (pp. 3–18). Neufeld, R. W. J. (2007b). Composition and uses of formal clinical cognitive science In B. Shuart, W. Spaulding & J. Poland (Eds.), Modeling Complex Systems: Nebraska Symposium on Motivation, 52, 1–83. Lincoln, Nebraska: University of Nebraska Press. Neufeld, R. W. J. (2007c). On the centrality and significance of encoding deficit in schizophrenia. Schizophrenia Bulletin, 33, 982–993. Neufeld, R. W. J. (2012). Quantitative clinical cognitive science, cognitive neuroimaging, and tacks to fMRI signal analysis: The case of encoding deficit in schizophrenia. Paper presented at the 45th Annual Meeting of the Society for Mathematical Psychology, Columbus, Ohio, July 21–24, 2012. Neufeld, R. W. J., Boksman, K., Vollick, D., George, L., & Carter, J. (2010). Stochastic dynamics of stimulus encoding in schizophrenia: Theory, testing, and application. Journal of Mathematical Psychology, 54, 90–108. Neufeld, R. W. J. & Broga, M. I. (1977). Fallacy of the reliability-discriminability principle in research on differential cognitive deficit. Department of Psychology Research Bulletin Number 360. London, ON: University of Western Ontario.
366
new directions
Neufeld, R. W. J., & Broga, M. I. (1981). Evaluation of information-sequential aspects of schizophrenic performance, II: Methodological considerations. Journal of Nervous and Mental Disease, 169, 569–579. Neufeld, R. W. J., Vollick, D. Carter, J. R., Boksman, K., & Jetté, J. (2002). Application of stochastic modelling to group and individual differences in cognitive functioning. Psychological Assessment, 14, 279–298. Neufeld, R. W. J., & Gardner, R. C. (1990). Data aggregation in evaluating psychological constructs: Multivariate and logicaldeductive considerations. Journal of Mathematical Psychology, 34, 276–296. Neufeld, R. W. J., & McCarty, T. (1994). A formal analysis of stressor and stress-proneness effects on basic information processing. British Journal of Mathematical and Statistical Psychology, 47, 193–226. Neufeld, R. W. J., & Williamson, P. (1996). Neuropsychological correlates of positive symptoms: Delusions and hallucinations. In C. Pantelis, H.E. Nelson, & T.R.E. Barnes (Eds.), Schizophrenia: A neuropsychological perspective (pp. 205–235). London, England: Wiley Neufeld, R. W. J., Townsend, J. T., & Jetté, J. (2007). Quantitative response time technology for measuring cognitive-processing capacity in clinical studies. In R.W.J. Neufeld (Ed.), Advances in clinical cognitive science: Formal modeling and assessment of processes and symptoms (pp. 207–238). Washington, D.C.: American Psychological Association. Neufeld, R. W. J., Mather, J. A., Merskey, H., & Russell, N. C. (1995). Multivariate structure of eye-movement dysfunction in schizophrenia. Multivariate Experimental Clinical Research, 11, 1–21. Norman, R. M. G. & Malla, A. K.(1994). A prospective study of daily stressors and symptomatology in schizophrenic patients. Social Psychiatry and Psychiatric Epidemiology, 29, 244–249. Novotney, A. (2009). Science on a shoestring. Monitor on Psychology, 40(1), 42–44. O’Hagan, A., & Forster, J. (2004). Kendall’s advanced theory of statistics: Vol. 2B. Bayesian inference (2nd ed.). London, England: Arnold. Ouimet, A. J., Gawronski, B., & Dozois, D. J. A. (2009). Cognitive vulnerability to anxiety: A review and an integrative model. Clinical Psychology Review, 29, 459–470. Phillips, W. A., & Silverstein, S. M. (2003) Convergence of biological and psychological perspectives on cognitive coordination in schizophrenia. Discussion. Behavioral and Brain Sciences, 26, 65–138. Pitt, M. A., Kim, W., Navarro, J., & Myung, J. I. (2006). Global model analysis by parameter space partitioning. Psychololgical Review, 113, 57–83. Poldrack, R. A. (2011). Inferring mental states from neuroimaging data: From reverse inference to large-scale decoding. Neuron, 72, 692–697. Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59–108. Ratcliff, R. (1979). Group reaction time distributions and an analysis of distribution statistics. Psychological Bulletin, 86, 446–461.
Riefer, D. M. & Batchelder, W. H. (1988). Multinomial modeling and the measurement of cognitive processes. Psychological Review, 95, 318–339. Riefer, D. M., Knapp, B. R., Batchelder, W. H., Bamber, D., & Manifold, V. (2002). Cognitive psychometrics: Assessing storage and retrieval deficits in special populations with multinomial processing tree models. Psychological Assessment, 14, 184–201. Rodgers, J. L. (2010) The epistemology of mathematical and statistical modeling: A quiet methodological revolution. American Psychologist, 65(1), 1–12. Rouder, J. N., Sun, D., Speckman, P. L., Lu, J. & Zhou, D. (2003). A hierarchical bayesian statistical framework for response time distributions. Psychometrika, 68(4), 589–606. Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information processing: I. Detection, search, and attention. Psychological Review, 84, 1–66. Schweickert, R. (1982). The bias of an estimate of coupled slack in stochastic PERT networks. Journal of Mathematical Psychology, 26, 1–12. Schweickert, R., (1985) Separable effects of factors on speed and accuracy: Memory scanning, lexical decisions and choice tasks. Psychological Bulletin, 7, 530–548. Schweickert, R., & Han, H. J. (2012). Reaction time predictions for factors selectively influencing processes in processing trees. Paper presented at Mathematical Psychology meeting, Columbus, OH, July 2012. Shanahan, M. J., & Neufeld, R. W. J. (2010). Coping with stress through decisional control: Quantification of negotiating the environment. British Journal of Mathematical and Statistical Psychology, 63, 575–601. Shanahan, M. J., Townsend, J. T. & Neufeld, R. W. J. (in press). Mathematical modeling in clinical psychology. In R. Cautlin & S. Liltienfield (Eds.); R. Zinbarg (Section Ed.) WileyBlackwell’s Encyclopedia of Clinical Psychology, London, Wiley-Blackwell. Shannon, C. E. (1949). The mathematical theory of communication. Urbana, IL: University of Illinois Press. Shiffrin, R. M., & Schneider, W. (1977). Controlled and automatic human information processing. II. Perceptual learning, automatic attending and a general theory. Psychological Review, 84, 127–190. Siegle G. J., Hasselmo M. E. (2002). Using connectionist models to guide assessment of psychological disorder. Psychological Assessment, 14, 263–278. Silverstein, S. S. (2008). Measuring specific, rather than generalized, cognitive deficits and maximizing betweengroup effect size in studies of cognition and cognitive change. Schizophrenia Bulletin, 34, 645–655. Smith, J. B., & Batchelder, W. H. (2010). Beta-MPT: Multinomial processing tree models for addressing individual differences. Journal of Mathematical Psychology, 54, 167–183. Staddon, J. E. R. (1984). Social learning theory and the dynamics of interaction. Psychological Review, 91(4), 502– 507. Stein, D. J., & Young, J. E. (1992). (Editors), Cognitive science and clinical disorders. New York, NY: Academic.
Sternberg, S(1969). The discovery of processing stages: Extensions of Donders’ method. In W. G. Koster (Ed.), Attention and performance II. Acta Psychologica, 30, 276–315. Townsend, J. T. (1984). “Uncovering mental processes with factorial experiments.” Journal of Mathematical Psychology, 28(4), 363–400. Townsend, J. T. (2008). Mathematical psychology: Prospects for the 21st century: A guest editorial. Journal of Mathematical Psychology, 52, 269–280. Townsend, J. T. and Ashby, F. G. (1983). Stochastic modelling of elementary psychological processes. Cambridge: Cambridge University Press. Towsend, J. T., & Altieri, N. (2012). An accuracy-response time capacity assessment function that measures performance against standard parallel predictions. Psychological Review, 119, 500–16. Townsend, J. T. & Eidels, A., (2011). Workload capacity spaces: A unified methodology for response time measures of efficiency as workload is varied. Psychonomic Bulletin & Review, 18, 659–681. Townsend, J. T., Fific, M., & Neufeld, R. W. J. (2007). Assessment of mental architecture in clinical/cognitive research. In T. A. Treat, R. R. Bootzin, T. B. Baker (Eds.), Psychological clinical science: Papers in Honor of Richard M. McFall (pp. 223–258). Mahwah, NJ: Erlbaum. Townsend, J. T. & Neufeld, R. W. J. (2004). Mathematical theory-driven methodology and experimentation in the emerging quantitative clinical cognitive science: Toward general laws of individual differences. Paper presented at the Association for Psychological Science-sponsored symposium on Translation Psychological Science in Honor of Richard M. McFall, Chicago, Illinois, May, 2004. Townsend, J. T., & Nozawa, G. (1995). Spatio-temporal properties of elementary perception: An investigation of parallel, serial, and coactive theories. Journal of Mathematical Psychology, 39, 321–359. Townsend, J. T., & Wenger, M. J. (2004a). The serial-parallel dilemma: A case study in a linkage of theory and method. Psychonomic Bulletin & Review, 11, 391–418. Townsend, J. T. & Wenger, M. J. (2004b). A theory of interactive parallel processing: New capacity measures and predictions for a response time inequality series. Psychological Review, 111, 1003–1035. Treat, T. A., & Viken, R. J. (2010). Cognitive processing of weight and emotional information in disordered eating. Current Directions in Psychological Science, 19, 81–85. Treat, T. A., McFall, R. M., Viken, R. J., Nosfosky, R. M., MacKay, D. B., & Kruschke, J. K. (2002). Assessing clinically relevant perceptual organization with multidimensional scaling techniques. Psychological Assessment, 14, 239–252. Treat, T. A., Viken, R. J., Kruschke, J. K., & McFall, R. M. (2010). Role of attention, memory, and covariationdetection processes in clinically significant eating-disorder symptoms. Journal of Mathematical Psychology, 54, 184–195. Van der Maas, H. L. J., Molenaar, D., Maris, G., Kievit, R. A., & Borsboom, D. (2011). Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118, 339–356.
mathematical and computational
367
Van Zandt, T. (2000). How to fit a response time distribution. Psychonomic Bulletin and Review, 7, 424–465. Wagenmakers, E.-J., van der Maas, H. L. J., Dolan, C., & Grasman, R. P. P. P. (2008). EZ does it! Extensions of the EZ-diffusion model. Psychonomic Bulletin & Review, 15, 1229–1235. Wagenmakers, E.-J., van der Maas, H. L. J., & Farrell, S. (2012). Abstract concepts require concrete models: Why cognitive scientists have not yet embraced nonlinearlycoupled, dynamical, self-organized critical, synergistic, scalefree, exquisitely context-sensitive, interaction-dominant, multifractal, interdependent brain-body-niche systems. TopiCS, 4, 87–93. Wallsten, T. S., Pleskac, T. J., Lejuez, C. W., (2005). Modeling a sequential risk-taking task. Psychological Review, 112, 862–880. Wenger, M. J., & Townsend, J. T. (2000). Basic tools for attention and general processing capacity in perception and cognition. Journal of General Psychology: Visual Attention, 127, 67–99. White, C. N., Ratcliff, R., Vasey, M. W., & McKoon, G. (2010a). Anxiety enhances threat processing without competition among multiple inputs: A diffusion model analysis. Emotion, 10, 662–677. White, C. N., Ratcliff, R., Vasey, M. W., McKoon, G. (2010b). Using diffusion models to understand clinical disorders. Journal of Mathematical Psychology, 54, 39–52 Williams, J. M. G., & Oaksford, M. (1992). Cognitive science, anxiety and depression: From experiments to connectionism.
368
new directions
In D. J. Stein & J. E. Young (Eds.) Cognitive science and clinical disorders (pp. 129–150). San Diego, CA: Academic. Witkiewitz, K., van der Maas, H. J., Hufford, M. R., & Marlatt, G. A. (2007). Non-normality and divergence in posttreatment alcohol use: re-examining the project MATCH data “another way.” Journal of Abnormal Psychology, 116, 378–394. Yang, E., Tadin, D., Glasser, D. M., Hong, S. W., Blake, R., & Park, S. (2013). Visual context processing in schizophrenia. Clinical Psychological Science, 1, 5–15. Yechiam, E., Arshavsky, O., Shamay-Tsoory, S. G., Yaniv, S., and Aharon, J. (2010). Adapted to explore: Reinforcement learning in Autistic Spectrum Conditions. Brain and Cognition, 72, 317–324. Yechiam, E., Goodnight, J., Bates, J. E., Busemeyer, J. R., Dodge, K. A., Pettit, G. S., & Newman, J. P. (2006). A formal cognitive model of the Go/No-Go discrimination task: Evaluation and implications. Psychological Assessment, 18, 239–249. Yechiam, E., Hayden, E. P., Bodkins, M., O’Donnell, B. F., and Hetrick, W. P. (2008). Decision making in bipolar disorder: A cognitive modeling approach. Psychiatry Research, 161, 142–152. Yechiam E., Veinott E. S., Busemeyer J. R., Stout J. C. (2007). Cognitive models for evaluating basic decision processes in clinical populations. In: Neufeld R. W. J. (Ed.), Advances in clinical cognitive science: Formal modeling and assessment of processes and symptoms (pp. 81–111). Washington, DC: APA Publications.
CHAPTER
17
Quantum Models of Cognition and Decision
Jerome R. Busemeyer, Zheng Wang, and Emmauel Pothos
Abstract
Quantum probability theory provides a new formalism for constructing probabilistic and dynamic systems of cognition and decision. The purpose of this chapter is to introduce psychologists to this fascinating theory. This chapter is organized into six sections. First, some of the basic psychological principles supporting a quantum approach to cognition and decision are summarized; second, some notations and definitions needed to understand quantum probability theory are presented; third, a comparison of quantum and classical probability theories is presented; fourth, quantum probability theory is used to account for some paradoxical findings in the field of human probability judgments; fifth, a comparison of quantum and Markov dynamic theories is presented; and finally, a quantum dynamic model is used to account for some puzzling findings of decision-making research. The chapter concludes with a summary of advantages and disadvantages of a quantum probability theoretical framework for modeling cognition and decision. Key Words: quantum probability, classical probability, Hilbert space, the law of total
probability, conjunction fallacy, disjunction fallacy, dynamic models of decision, Markov models, quantum interference, two-stage gambles, dynamic consistency
Reasons for a Quantum Approach to Cognition and Decision This chapter is not about quantum physics 1 per se. Instead, it explores the application of probabilistic dynamic systems derived from quantum theory to a new domain – cognition and decision making behavior. Applications of quantum theory have appeared in judgment (Aerts & Aerts, 1994; Busemeyer, Pothos, Franco, & Trueblood, 2011; Franco, 2009; Pothos, Busemeyer, & Trueblood, 2013; Wang & Busemeyer, 2013), decision making (Bordley & Kadane, 1999; Busemeyer, Wang, & Townsend, 2006; Khrennikov & Haven, 2009; Lambert-Mogiliansky, Zamir, & Zwirn, 2009; La Mura, 2009; Pothos & Busemeyer, 2009; Trueblood & Busemeyer, 2011; Yukalov & Sornette, 2011), conceptual combinations (Aerts, 2009; Aerts & Gabora, 2005; Blutner, 2009), memory (Brainerd, Wang, & Reyna, 2013; Bruza
et al., 2009), and perception (Atmanspacher, Filk, & Romer, 2004; Conte et al., 2009). Several review articles (Pothos & Busemeyer, 2013; Wang, Busemeyer, Atmanspacher, & Pothos, 2013) and books (Busemeyer & Bruza, 2012; Ivancevic & Ivancevic, 2010; Khrennikov, 2010) provide a summary of this new program of research. Before presenting the formal ideas, let us first examine why quantum theory should be applicable to human cognition and decision behavior.
Judgments Are Based upon Indefinite and Uncertain Cognitive States Models commonly used in psychology assume the cognitive system changes from moment to moment, but at any specific moment it is in a definite state with respect to some judgment to be made. For example, suppose a juror has just heard conflicting evidence from the prosecutor and 369
the defense and the juror has to consider two mutually exclusive and exhaustive hypotheses—guilty or not guilty. A Bayesian model would assign a probability distribution over the two hypotheses— a probability p(G|evidence) is assigned to guilt and a probability 1 − p(G|evidence) is assigned to not guilty. Therefore, the juror’s subjective probability with respect to the question of guilty or not boils down to a state represented by a point lying somewhere between zero and one on the probability scale at each moment. This probability may change from moment to moment to produce a definite trajectory of probability for guilt across time. However, at each moment, this subjective probability either favors guilt p(G|evidence) > .50, or it favors not guilty p(G|evidence) < .50, or it is exactly at p(G|evidence) = .50. At a single moment, the juror cannot be both favoring guilt p(G|evidence) > .50 and at the same time favoring not guilty p(G|evidence) < .50. In contrast, quantum theory assumes that during deliberation the juror is in an indefinite (superposition) state at each moment. While in an indefinite state, the juror does not necessarily favor guilty and at the same time the juror does not necessarily favor not guilty. Instead, the juror is in a superposition state that leaves the juror conflicted, ambiguous, confused, or uncertain about the guilty status. The potential for saying guilt may be greater than the potential for saying not guilty at one moment, and these potentials may change from one moment to the next, but either hypotheses could potentially be chosen at each moment. In quantum theory, there is no single trajectory or sample path across time before making a decision. When asked to make a decision, the juror would be forced to commit to either guilt or not.
Judgments Create Rather Than Record a Cognitive State Models commonly used in psychology assume that what we record at a particular moment reflects the state of the cognitive system as it existed immediately before we inquired about it. For example, if a person watches a scene of an exciting car chase and is asked “Are you afraid?” then the answer “Yes. I am afraid” reflects the person’s cognitive state with respect to that question just before we asked it. In contrast, quantum theory assumes that taking a measurement of a system creates rather than records a property of the system (Wang, Busemeyer, Atmanspacher, & Pothos, 2013). For example, the 370
new directions
person may be ambiguous about his or her feelings after watching the scene, but the answer “Yes. I am afraid” is constructed from the interaction of this indefinite state and the question, which results in a now definitely “afraid” state. This is, in fact, the basis for modern psychological theories of emotion (Schachter & Singer, 1962). Decision scientists also have shown evidence that beliefs and preferences are constructed online rather than simply being read straight out of memory (Payne, Bettman, & Johnson, 1992), and expressing choices and opinions can change preferences (Sharot, Velasquez, & Dolan, 2010).
Judgments Disturb Each Other Producing Order Effects According to quantum theory, the answer to a question can change a state from indefinite to definite state, and this change causes one to respond differently to subsequent questions. Intuitively, the answer to the first question sets up a context that changes the answer to the next question, and this produces order effects of the measurements. Order effects make it impossible to define a joint probability of answers to questions A and B (unless one conditionalizes the conjunction with an order parameter), and instead it is necessary to assign a probability to the sequence of answers to question A followed by question B. In quantum theory, if A and B are two measurements, and the probabilities of the outcomes depend on the order of the measurements, then the two measurements are noncommutative. Many of the mathematical properties of quantum theory, such as Heisenberg’s famous uncertainty principle (Heisenberg, 1958), arise from developing a probabilistic model for noncommutative measurements. Question order effects are a major concern for attitude researchers, who struggle for a theoretical understanding of these effects similar to that achieved in quantum theory (Feldman & Lynch, 1988). Of course quantum theory is not the only theory to explain order effects. Markov models, for example, also can produce order effects. Quantum theory, however, provides a more natural, elegant, and built in set of principles (as opposed to ad hoc assumptions) for explaining order effects (Wang, Solloway, Shiffrin, & Busemeyer, 2014).
Judgments Do Not Always Obey Classical Logic Probabilistic models commonly used in psychology are based on the Kolmogorov axioms (1933/1950), which define events as sets that
Preliminary Concepts, Definitions, and Notations Quantum theory is based on geometry and linear algebra defined on a Hilbert space. (Hilbert spaces are complex vector spaces with certain convergence properties.) Paul Dirac developed an elegant notation for expressing the abstract elements of the theory, which are used in this chapter. This chapter is restricted to finite spaces for simplicity, but note that the theory is also applicable to infinite dimensional spaces. In fact, to keep examples simple, this section introduces the ideas using only a three-dimensional space in order to visually present the ideas. Figure 17.1 shows a particular vector labeled S that lies within a three-dimensional space spanned by three basis vectors labeled A, B, and C. For example, a simple attitude model could interpret S as the state of opinion of a person with regard to the beauty of an artwork using three mutually exclusive evaluations “good,” “mediocre,” or “bad,” which are represented by the basis vectors A, B, and C, respectively. A finite Hilbert space is an N -dimensional vector space defined on a field of complex numbers and
0.25 S
0.2 0.15 C
obey the axioms of set theory and Boolean logic. One important axiom is the distributive axiom: If {G, T , F } are events then G ∩ (T ∪ F ) = (G ∩ T ) ∪ (G ∩ F ). Consider for example, the concept that a boy is good (G), and the pair of concepts that the boy told the truth (T ) versus the boy did not tell truth (F ). According to classical Boolean logic, the event G can only occur in one of two ways: either (G ∩ T ) occurs or (G ∩ F ) exclusively. From this distributive axiom, one can derive the law of total probability, p(G) = p(T )p(G|T ) + p(F )p(G|F ). Quantum probability theory is derived from the von Neumann axioms (1932/1955), which define events as subspaces that obey different axioms from those of set theory. In particular, the distributive axiom does not always hold (Hughes, 1989). For example, according to quantum logic, when you try to decide whether a boy is good without knowing if he is truthful or not, you are not forced to have only two thoughts: he is good and he is truthful, or he is good and he is not truthful. You can remain ambiguous or indeterminate over the truthful or not truthful attributes, which can be represented by a superposition state. The fact that quantum logic does not always obey the distributive axiom implies that the quantum model does not always obey the law of total probability (Khrennikov, 2010).
0.1 0.05 T
0 1 0.5 B
0.2
0 0
0.6
0.4
0.8
1
A
Fig. 17.1 Three-dimensional vector space spanned by basis vectors A, B, and C. 2
endowed with an inner product. The space has a basis, which is a set of N orthonormal basis vectors χ = {|X1 , ..., |XN } that span the space. The symbol |X represents an arbitrary vector in an N -dimensional vector space, which is called a “ket.” This vector can be expressed by its coordinates with respect to the basis χ as follows |X =
N
xi |Xi .
i=1
The coordinates, xi are complex numbers. The N coordinates representing the ket |X with respect to a basis χ forms an N × 1 column matrix ⎤ ⎡ x1 ⎥ ⎢ X = ⎣ ... ⎦ . xN Referring to Figure 17.1, the coordinates for the specific vector |S with respect to the {|A , |B , |C} basis equal ⎡ ⎤ 0.696 S = ⎣ 0.696 ⎦ . 0.1765 Referring back to our simple attitude model, the coordinates of S represents the potentials for each of the opinions. The symbol X | represents a linear functional in an N -dimensional (dual) vector space, which is called a “bra.” Each ket |X has a correspondX |. The conjugate transpose operation, ing bra |X † = X | changes ket into a bra. The N coordinates representing the bra X | with respect to a basis χ forms a 1 × N row matrix
∗ . X † = x1∗ · · · xN
quantum models of cognition and decision
371
The * symbol indicates complex conjugation. For example, the bra S| corresponding to the ket |S has the matrix representation given below as S† =
0.696
0.696
0.1765 .
(Here the numbers in the example are real, and so conjugation has no effect.) Hilbert spaces are endowed with an inner product. Psychologically, the inner product is a measure of similarity between two vectors. The inner product is a scalar formed by applying the bra to the ket to form a bra-ket Y |X =
N
yi∗ · xi .
i=1
For example
S|S = S † · S = 0.696 0.696 0.1765 ⎡ ⎤ 0.696 × ⎣ 0.696 ⎦ = 1. 0.1765
This shows that the ket |S is unit length. The outer product, denoted by |X Y |, is a linear operator, which is used to make transitions from one state to another. In particular, assuming that the kets are unit length, then the outer product |X Y | maps the ket |Y to the ket |X as follows: (|X Y |) · |Y = |X Y | Y = |X . Assuming |X is unit length, the outer product |X X | projects |X to itself, |X X | · |X = |X X |X = |X · 1, and |X X | projects any other ket |Y onto the ray spanned by |X as follows |X X |Y = X |Y · |X . For these reasons, |X X | is called the projector for the ray spanned by |X , which is also symbolized as MX = |X X |. Projectors correspond to subspaces that represent events in quantum theory. They are Hermitian and idempotent. Referring to Figure 17.1, the coordinates for the basis vector |A (with respect to the {|A , |B , |C} basis) simply equals ⎡ ⎤ 1 A=⎣ 0 ⎦ 0 and the matrix representation of the projector for this basis equals ⎡ ⎤ ⎡ ⎤ 1 1 0 0
A · A† = ⎣ 0 ⎦ · 1 0 0 = ⎣ 0 0 0 ⎦ . 0 0 0 0 The projector MA = A · A† corresponds to the subspace representing the event A. In our simple 372
new directions
attitude model, MA would be used to represent the event that the person decides the artwork to be “good” (which corresponds to event A). The matrix representation of the projection of the ket |S onto the ray spanned by the basis |A then equals ⎡
⎤ 0.696 A · A† · S = ⎣ 0 ⎦ . 0 In our simple attitude model, this projection is used to determine the probability that the person decides the artwork is “good.” According to quantum theory, the squared length of this projection, 0.6962 = 0.4844, equals the probability that the person will decide that the artwork is “good.” Similarly, the coordinates for the basis vector |B (with respect to the {|A , |B , |C} basis) simply equals ⎡
⎤ 0 B = ⎣ 1 ⎦. 0 So the matrix representation (with respect to the {|A , |B , |C} basis) for the projector for |B equals ⎡
⎤ 0 0 0 B · B† = ⎣ 0 1 0 ⎦ . 0 0 0 In our simple attitude model, this projector is used to represent the event that the person decides the artwork to be “mediocre” (which corresponds to event B). In addition, the horizontal plane shown in Figure 17.1 is spanned by the {|A , |B} basis vectors, and the projector that projects vectors onto this plane equals MA +MB = |A A|+|B B| which has the matrix representation ⎡
1 A · A† + B · B † = ⎣ 0 0
0 1 0
⎤ 0 0 ⎦. 0
In our simple attitude example, this corresponds to the event that the person thinks the artwork is “good” or “mediocre.” The vector labeled T in Figure 17.1 is the projection of the vector |S onto the plane spanned by the {|A , |B} basis vectors, which has the matrix representation
T = A · A† + B · B † · S
These three vectors form another orthogonal basis for spanning the three-dimensional space. The projector MV = |V V | projects vectors onto the ray spanned by the basis vector |V as follows: MV |X = |V V | X . Using the coordinates defined above for |V we obtain, ⎤ ⎡
⎡
⎤ ⎡ ⎤ 1 0 0 .696 = ⎣ 0 1 0 ⎦ · ⎣ .696 ⎦ 0 0 0 .1765 ⎡ ⎤ .696 = ⎣ .696 ⎦ . 0
V ·V† ·S = V ·
The squared length of this projection, T † T = 2 · 6962 = 0.969, equals the probability that the person decides the artwork to be “good” or “mediocre.” This is also the probability that the person thinks the artwork is not “bad” (1 − 0.17652 = 0.969). Referring back to our simple attitude model, suppose that instead of asking whether the artwork is beautiful, we ask what kind of moral message it conveys, and once again there are three answers such as “good,” “neutral,” or “bad.” Now the person needs to evaluate the same artwork with respect to a new point of view. In quantum theory, this new point of view is represented as a change in the basis. Figure 17.2 illustrates three new orthonormal vectors within the same three-dimensional space labeled U, V, and W in the figure. Now the basis vectors U, V, and W represent a “good,” “neutral,” or “bad” moral message, respectively. The state S now represents the person’s opinion with respect to this new moral message point of view. With respect to the {|A , |B , |C} basis, the coordinates for these three vectors are as follows ⎤ ⎤ ⎡ ⎡ √ 1/2 √1/2 −1/2 ⎦ , U = ⎣ 1/2 ⎦ , V = ⎣ √ 1/2 0 ⎡
⎤ −1/2 W = ⎣ √1/2 ⎦ 1/2
1 2
− 12
= (0.125) V ,
1 2
0.696 · ⎣ 0.696 ⎦ 0.1765
which indicates that 0.125 is the coordinate of the vector |S on the |V basis vector. This is the projection of S on the V basis, and the squared length of this projection, 0.1252 = 0.0156, equals the probability of this event, e.g., the probability that the person decides that the artwork is “neutral.” Repeating this procedure for |U and |W , we obtain the coordinates for the vector |S in Figure 17.2 with respect to the {|U , |V , |W } basis. ⎡ ⎤ 0.125 Y = ⎣0.9843⎦ 0.125 In sum, the same vector |S can be expressed by the coordinates X using the {|A , |B , |C} basis or by the coordinates Y using the {|U , |V , |W } basis. Note that the event “morally good” is represented by the vector U in Figure 17.2. This vector lies along the diagonal line of the A, B plane. Here we see an interesting feature of quantum theory. If a person is definite that the piece of artwork is “morally good” (represented by the vector U), then the person must be uncertain about whether it’s beauty is good versus mediocre (because U has a 45 degree angle with respect to each of the A, B vectors). However, if the person is certain that the artwork is “morally good” then the person is certain that it’s beauty is not “bad” (because U is contained in the A, B plane).
Quantum Compared to Classical Probabilities
0.8 0.7 W
0.6
V
0.5 0.4 0.3
S
0.2 0.1 0 1
U 0.5
0
−0.5
−1 −1
−0.5
0
0.5
1
Fig. 17.2 New basis U, V, W for representing the threedimensional vector space
This section presents the quantum probability axioms formulated by Paul Dirac (1958) and John von Neumann (1932/1955), and compares them systematically with the axioms of classical Kolmogorov probability theory (1933/1950) (see 3 Box 1 for a summary). For simplicity, we restrict this presentation to finite spaces in this chapter. Although the space is finite, the number of dimensions can be very large. The general theory is applicable to infinite dimensional spaces. See Chapter 2 in Busemeyer and Bruza (2012) for a more comprehensive introduction.
quantum models of cognition and decision
373
Box 1 A brief comparison of the classical Kolmogorov probability theory and the quantum probability theory • Kolmogorov Theory • • • • • • •
Sample space set of events Event A is represented as a subset State is a probability function p p(A) = probability assigned to event A if A ∩ B = , p(A ∪ B) = p(A) + p(B) p(A∩B) p(A|B) = p(B) , p(A ∩ B) = p(B ∩ A)
• Quantum Theory
•
Hilbert vector space of events Event A is represented as a subspace corresponding to a projector MA State is a vector |S in Hilbert space q(A) = MA |S2 if MA MB = 0, q(A ∨ B) = q(A) + q(B) 2 A MB |S q(A|B) = M√ ,
•
MA MB |S2 = MB MA |S2
• •
• • •
q(B)
Events Classical probability postulates a sample space χ , which we will assume contains a finite number of points, N (and N may be very large). The set of points in the sample space is defined as χ = {X1 , ..., XN }. An event A is a subset of this sample space A ⊆ χ. If A ⊆ χ is an event and B ⊆ χ is an event, then the intersection A ∩ B is an event; also the union A ∪ B is an event. Quantum theory postulates a Hilbert space χ, which we will assume has a finite dimension, N (and again N may be very large). The space is spanned by an orthonormal set of basis vectors 4 χ = {|X1 , ..., |XN } that form a basis for the space. An event A is a subspace spanned by a subset χA ⊆ χ of basis vectors. This event corresponds to a projector MA = i∈A |Xi Xi |. If A is an event spanned by χA ⊆ χ and B is an event spanned by χB ⊆ χ , then the meet (infimum)A ∧ B is an event spanned by χA ∩χB ; also the join (supremum) A∨B is an event spanned by χA ∪ χB . For example, in Figure 17.1, the event A is represented by the ray spanned by the vector |A, and “A or B” is represented by the horizontal 374
new directions
plane spanned by the two vectors {|A , |B} for the quantum model.
System State Classical probability postulates a probability function p that maps points in the sample space χ into positive real numbers which sum to unity. The empty set is mapped into zero, and the sample space is mapped into one, and all other events are mapped into the interval [0, 1]. If the pair of events {A ⊆ χ, B ⊆ χ} are mutually exclusive A ∩ B = Ø, then p(A ∪ B) = p(A) + p(B). The probability of the ¯ equals p(A) ¯ = 1 − P(A). event “not A,” denoted A, Quantum probability postulates a unit length state vector |X in the Hilbert space. The probability of an event A spanned by χA ⊆ χ is defined by q(A) = ||MA |X ||2 . For later work, it will be convenient to express ||M |X ||2 as the inner product ||M |X ||2 = X |M † M |X = X |M |X , where the last step made use of the idempotency of the projector M † = M = MM . If the pair of events {A, B}, both spanned by subsets of basis vectors from χ , are mutually exclusive, χA ∩ χB = Ø, then it follows from orthogonality that q(A∨B) = ||(MA +MB )|X ||2 = ||MA |X ||2 + ||MB |x||2 = q(A) + q(B). The event A¯ is the subspace that is the orthogonal complement to the subspace for the event A, and its probability ¯ = (I − MA ) |X 2 = 1 − q(A). equals q(A) For example, in Figure 17.1, the probability of the event A equals MA |S2 = ' ' 'A · A† · S '2 = |.696|2 , and the probability of the event “A or B” equals (MA + MB ) |S2 = ' ' ' A · A† + B · B† · S '2 = T 2 = 2 · |.696|2 .
State Revision
According to classical probability, if an event A ⊆ χ is observed, then a new conditional probability function is defined by the mapping p(Xi |A) = p(Xi ∩A) p(A) . The normalizing factor in the denominator is used to guarantee that the probability assigned to the entire sample space remains equal to one. According to quantum probability, if an event A is observed, then the new revised state is defined MA |X . The normalizing factor in the by |XA = ||M A |X || denominator is used to guarantee that the revised state remains unit length. The new revised state is then used (as described earlier) to compute probabilities for events. This is called Lüder’s rule.
For example, in Figure 17.1, if the event “A or B” is observed, then the matrix representation of the revised state equals ⎡ √ ⎤ √1/2 T = ⎣ 1/2 ⎦ = U . SAorB = T 0 The probability of event A given that “A or B” was '2 ' observed equals 'A · A† · SAorB ' = .50.
Commutativity Classical probability assumes that for any given experiment, there is only one sample space, χ, and all events are contained in this single sample space. Consequently, the intersection between two events and the union of two events is always well defined. A single probability function p is sufficient to assign probabilities to all events of the experiment. This is 5 called the principle of unicity (Griffiths, 2003). It follows from the commutative property of sets that joint probabilities are commutative, p(A) · p(B|A) = p(A ∩ B) = p(B ∩ A) = p(B) · p(A|B). Quantum probability assumes that there is only one Hilbert space and all events are contained in this single Hilbert space. For a single fixed basis, such as χ = {|Xi , i = 1, ..., N }, the meet and the join of two events spanned by a common set of basis vectors in χ are always well defined, and a probability function q can be used to assign probabilities to all the events defined with respect to the basis χ. When a common basis is used to define all the events, then the events are compatible. The beauty of a Hilbert space is that there are many choices for the basis that one can use to describe the space. For example, in Figure 17.2, a new basis using vectors {|U , |V , |W } was introduced to represent the state |S, which was obtained by rotating the original basis {|A , |B , |C} used in Figure 17.1. Suppose χ = {|Yi , i = 1, ..., N } is another orthonormal basis for the Hilbert space. If event A is spanned by χA ⊂ χ, and event B is spanned by χB ⊂ χ , then the meet for these two events is not defined; also the join for these two events is not defined either (Griffths, 2003). In this case, the events are not compatible. That is, the projectors for these two events do not commute, MA MB = MB MA , and the projectors for these two events do not share a common set of eigenvectors. In this case, it is not meaningful to assign a probability simultaneously to the pair of events {A, B} (Dirac, 1958). When the events are incompatible, the principle of unicity breaks
down and the events cannot all be described within a single sample space. The events spanned by χ, which are all compatible with each other, form one sample space; and the events spanned by χ , which are combatible with each other, form another sample space; but the events from χ are not compatible with the events from χ . In this case, there are two stochastically unrelated samples spaces (Dzhafarov & Kujala, 2012), and quantum theory provides a single state |S that can be used to assign probabilities to both sample spaces. For noncommutative events, probabilities are assigned to histories or sequences of events using Lüder’s rule. Suppose A is an event spanned by χA ⊆ χ, and event B is spanned by χB ⊆ χ . Consider the probability for the sequence of events: A followed by B. The probability of the first event A equals q(A) = ||MA |X ||2 ; the revised state, conditioned on obMA |X serving this event equals |XA = ||M ; the probA |X || ability of the second event, conditioned on the first event, equals q(B|A) = ||MB |XA ||2 ; therefore, the probability of event A followed by event B equals ' '2 ' MA |X ' ' M q(A) · q(B|A) = ||MA |X ||2 · ' ' B M |X ' A = MB MA |X 2 .
(1)
If the projectors do not commute, then order matters because q(A) · q(B|A) = MB MA |X 2 = MA MB |X 2 = q(B) · q(A|B). For example, referring to Figures 17.1 and 17.2, the probability of event “A or B” and then event V equals ' '2 ' ' MV (MA + MB ) |S2 = 'VV † · T ' = 0. The probability of the event V and then the event “A or B” equals '⎡ ⎤' 2 ' ' 0.123/2 ' ' ⎣ −0.123/2 ⎦' (MA + MB ) MV |S2 = ' ' ' ' ' 0 = 0.008. If all events are compatible, then quantum probability theory reduces to classical probability theory. In this sense, quantum probability is a generalization of classical probability theory (Gudder, 1988).
quantum models of cognition and decision
375
Violations of the Law of Total Probability The quantum axioms do not necessarily have to obey the classical law of total probability in the following manner. Consider an experiment with two different conditions. The first condition simply measures whether event B occurs. The second condition first measures whether A occurs, and then measures whether B occurs. For both conditions, we compute the probability of the event B. For the first condition, this is simply q(B) = ||MB |X ||2 . For the second condition, we could observe the sequence with event A and then event B with probability q(A) · q(B|A) = ||MB MA |X ||2 , or we could observe the sequence with event A¯ and then event B ¯ · q(B|A) = ||MB M ¯ |X ||2 , with probability q(A) A and so the total probability for event B in the second experiment equals the sum of these two ways: qT (B) = ||MB MA |X ||2 + ||MB MA¯ |X ||2 . The interference produced in this experiment is defined as the probability of event B observed in the first condition minus the total probability of event B observed in the second condition. According to quantum probability theory, the interference equals Int = q(B) − qT (B). To analyze this more closely, let us decompose the probability from the first condition as follows: q(B) = MB |X ' '2 = 'MB MA + MA¯ |X ' ' '2 = '(MB MA |X ) + MB MA¯ |X ' ' '2 = MB MA |X 2 + 'MB MA¯ |X ' + Int 2 1 2∗ 1 Int = X |MA MB MA¯ |X + X |MA MB MA¯ |X . (2) 2
An interference cross-product term, denoted Int, appears in the probability q (B) from the first condition, which produces deviations from the total probability qT (B) computed from the second condition. This interference term can be positive (i.e., constructive interference) or negative (i.e., destructive interference) or zero (i.e., no interference). If the two projectors commute so that MA MB = MB MA , then the interference is zero. There is also an interference term associated with the complementary probability ' ' ' ' ¯ = 'M ¯ MA |X '2 + 'M ¯ M ¯ |X '2 −Int. (3) q(B) B B A ¯ must The interference term associated with q(B) be the negative of the interference term associated 376
new directions
with q(B) because we must finally obtain ¯ 1 = q(B) + q(B)
' '2 1 = MB MA |X 2 + 'MB MA¯ |X ' ' '2 ' '2 + 'MB¯ MA |X ' + 'MB¯ MA¯ |X ' . A skeptic might argue that the preceding rules for assigning probabilities to events defined as subspaces are ad hoc, and maybe there are many other rules that one could use. In fact, a famous theorem by Gleason (1957) proves that these are the only rules one can use to assign probabilities to events defined as subspaces using an additive measure (at least for vector spaces of dimension greater than 2). Now let us turn to a couple of example applications of this theory. Quantum cognition and decision has been applied to a wide range of findings in psychology (see Box 2). In this chapter we only have space to show two illustrations— an application to probability judgment errors, and another application to violations of rational decision-making.
Application to Probability Judgment Errors Quantum theory provides a unified and coherent account for a broad range of puzzling findings in the area of human judgments. The theory has provided accounts for order effects on attitude judgments (Wang & Busemeyer, 2013; Wang et al., 2014), inference (Trueblood & Busemeyer, 2010), and causal reasoning (Trueblood & Busemeyer, 2011). The same theory has also been used to account for conjunction and disjunction errors found with probability judgments (Franco, 2009), as well as overextension and underextension errors found in conceptual combinations (Aerts, 2009). This section briefly describes how the theory accounts for conjunction and disjunction errors in probabilistic judgments (Busemeyer et al., 2011). Conjunction and disjunction probability judgment errors are very robust and they have been found with a wide variety of examples (Tversky & Kahneman, 1983). Here we consider an example, where a judge is provided with a brief story about a hypothetical woman named Linda (circa 1980s): Linda is 31 years old, single, outspoken, and very bright. She majored in philosophy. As a student, she was deeply concerned with issues of discrimination and social justice, and also participated in antinuclear demonstrations.
Box 2 Applications of quantum theory to cognition and decision 1. Choice and decision time (Busemeyer, Wang, & Townsend, 2006; Fuss & Navarro, 2013) 2. Violations of rational decision theory (Pothos & Busemeyer, 2009; Yukalov & Sornette, 2011) 3. Categorization and decision (Busemeyer, Wang, & Lambert-Mogiliansky, 2009) 4. Probability judgment errors (Busemeyer, Pothos, & Trueblood, 2012) 5. Similarity judgments (Pothos, Busemeyer, & Trueblood, 2013) 6. Causal reasoning (Trueblood & Busemeyer, 2011) 7. Bistable perception (Atmanspacher & Filk, 2010) 8. Conceptual combinations (Aerts, Gabora, & Sozzo, 2013) 9. Concept vagueness (Blutner, Pothos, & Bruza, 2013) 10. Associative memory (Bruza, Kitto, Nelson, & McEvoy, 2009) 11. Memory recognition (Brainerd, Wang, & Reyna, 2013) 12. Attitude question order effects (Wang & Busemeyer, 2013; Wang, Solloway, Shiffrin, & Busemeyer, 2014) 13. Order effects on inference (Trueblood & Busemeyer, 2010) 14. Game theory (Kvam, Lambert-Mogiliansky, & Busemeyer, 2013)
Then the judge is asked to rank the likelihood of the following events: Linda is (a) active in the feminist movement, (b) a bank teller, and (c) active in the feminist movement and a bank teller, (d) active in the feminist movement and not a bank teller, (e) active in the feminist movement or a bank teller. The conjunction fallacy occurs when option c is judged to be more likely than option b (even though it can be argued from the classical perspective that the latter contains the former), and the disjunction fallacy occurs when option a is judged to be more likely than option e (again, even though it can be argued that the latter contains the former). For example, in a study
(Morier & Borgida, 1984), the mean probability judgments were ordered as follows: using J (A) to denote the mean probability judgment for event A, J (feminist) = .83 > J (feminist or bank teller) = .60 > J (feminist and bank teller) = .36 > J (bank teller) = .26 (N = 64 observations per mean, and all pairwise differences are statistically significant). These results violate classical probability theory which is the reason that they are called fallacies. What follows is a simple yet general model for these types of findings. The first assumption is that after reading the story about Linda, a person forms an initial belief state |S that represents the person’s beliefs about features or properties that may or may not be true about Linda. Formally, this belief state is a vector within an N -dimensional vector space. This belief state is used to answer any possible question that might be asked about Linda. The second assumption is that a question such as “Is Linda a feminist?” is represented by a NF < N dimensional subspace of the N -dimensional vector space. This subspace corresponds to a projector MF that projects the state vector onto the subspace representing the feminist question. The question “Is Linda a bank teller?” is represented by another subspace of dimension NB < N with a corresponding projector MB . The third assumption is that the projectors MF , MB do not commute so that MF MB = MB MF , and thus, the order of their applications matters. The reason why these two projectors do not commute is the following. The two concepts (feminist, bank teller) are rarely experienced together, and so the person has not formed a compatible representation of beliefs about combinations of both concepts using a common basis. A person may have formed one basis representing features related to feminists, but this basis differs from the basis used to represent features related to bank tellers. The concepts do not share the same basis and so they are incompatible, and the person needs to change from one basis to the other sequentially in order to answer questions about each concept. The fourth assumption concerns the order that the concepts are processed when asked “Is Linda a feminist and a bank teller?”. Given that the events are incompatible, the person has to pick an order to process them. It is assumed that the more likely event is processed first. It is quite easy to judge the order of each individual question such as that J (feminist) > J (bank teller). But the question about
quantum models of cognition and decision
377
“feminist bank teller” is subtler, and this is not as easy as the previous two questions. The judgment for the conjunction requires forming an additional and subtler judgment about the conditional probability J (bank teller given feminist). These assumptions are now used to derive the quantum predictions for the probability of bank teller (using Eq. 2) : q(B) = MB |S2
'2 ' = MB MF |S2 + 'MB MF¯ |S' + Int 2 1 2∗ 1 Int = S|MF MB MF¯ |S + S|MF MB MF¯ |S
According to the quantum model, a conjunction error occurs when ' '2 q(B) = MB MF |S2 + 'MB MF¯ |S' + Int < ' '2 MB MF |S2 → Int < − 'MB MF¯ |S' .
BT
Formally, the negative interference term produces the conjunction error. Intuitively, the Linda story produces a belief state that is almost orthogonal to the subspace for the bank teller event. However, if this state is first projected onto the feminist subspace (eliminating some details about Linda that make it impossible for her to be a bank teller), then it becomes a bit more likely to think that this feminist can be a bank teller too. Figure 17.3 illustrates how this works using a simple two-dimensional example (though we stress that the specification of the model is general and not restricted to one-dimensional subspaces). The probability for the bank teller alone question is determined from the direct projection from the initial state Psi to the bank teller (BT) axis, which is shown as the shorter light grey vertical segment. The probability for the conjunction is represented
F
Ψ ~BT
~F Fig. 17.3 Example of conjunction fallacy for a special case with two dimensions.
378
new directions
by first projecting Psi onto feminist (F), and then projecting onto bank teller, which is shown as the longer dark grey vertical segment. Note that the projection for the conjunction (dark grey vertical segment) exceeds the projection for the bank teller alone (light grey vertical segment). The same theory can also account for the disjunction effect. The event that “Linda is a bank teller or a feminist” is the same as the denial of the event that “Linda is not a bank teller and not a feminist.” According to the quantum model, the probability of the event “not a bank teller and ¯ · q(F¯ |B), ¯ and so the not a feminist” equals q(B) ¯ · q(F¯ |B). ¯ probability of the denial equals 1 − q(B) The disjunction error is predicted to occur when the probability of “feminist” exceeds the disjunction, ¯ · q(F¯ |B) ¯ → q(F¯ ) < that is, when q(F ) > 1 − q(B) ¯ · q(F¯ |B). ¯ Therefore, according to the quantum q(B) model, a disjunction error occurs when ' '2 ' '2 q(F ) = 'MF¯ MB |S' + 'MF¯ MB¯ |S' ' '2 + Int < 'MF¯ MB¯ |S' ' '2 → Int < − 'MF¯ MB |S' . 1 2 1 2∗ Int = S|MB¯ MF¯ MB |S + S|MB¯ MF¯ MB |S To account for both of these fallacies using the same principles and parameters, the model must predict the following order effect (see Busemeyer et al., 2011, appendix) MB MF |S2 > MF MB |S2 . Intuitively, the probability obtained by first considering whether Linda is a feminist and then considering whether she is a bank teller must be greater than the probability obtained by the opposite order. Order effects in this direction have been reported—asking people to judge “is Linda a bank teller” before asking them to judge “is Linda a feminist and a bank teller” significantly reduces the size of the conjunction error as compared to the opposite order (Stolarz-Fantino, Fantino, Zizzo, & Wen, 2003). The fact that the quantum model can account for both conjunction and disjunction errors using the same principles and same parameters is a definite advantage over other accounts, such as an averaging model. As described in Busemeyer et al. (2011), there are many other qualitative predictions that can be derived from this model. In particular, because q(F ) > q(F )q(B|F ), this model cannot produce double conjunction errors, in which the conjunction is greater than both individual events.
Empirically, indeed single conjunction errors are much more common and double conjunction errors are infrequent (Yates & Carlson, 1986). Another prediction from this model is that assuming the conjunction error occurs so that q(F ) · q(B|F ) > q(B), then q(B|F ) ≥ q(B) because q(B|F ) ≥ q(F ) · q(B|F ) > q(B). The intuition here is that, given the detailed knowledge about Linda, it is almost impossible for Linda to be a bank teller; but given that she is viewed more generally as a feminist, it is more likely to think that a feminist also can be a bank teller. This is an important prediction that needs further empirical testing. (See Tentori & Crupi, 2013, for arguments against this prediction.) The model presented in this section can account for conjunction errors, disjunction errors, averaging effects, and order effects. It is, however, only one of many possible ways to build models of probability judgments using quantum principles. In particular, Aerts (Aerts, 2009) and his colleagues (Aerts & Gabora, 2005) have developed alternative quantum models that account for conjunction and disjunction errors in conceptual combinations. Importantly, their model can produce double conjunction errors; but unfortunately it must change parameters to account for differences between conjunction and disjunction errors. In summary, the quantum axioms provide a common set of general principles that can be implemented in different ways to construct more specific and competing quantum models of the same phenomena. Each of the specific quantum models can be compared with each other and with other classical models with respect to their ability account for empirical results.
Quantum Dynamics This section presents the quantum dynamical principles and compares them with Markov processes used in classical dynamical systems. Markov theory provides the basis for constructing a wide variety of classical probability models in cognitive science (e.g., random walk/diffusion models of decision-making). Once again, we restrict this presentation to finite dimensional systems in this chapter. Although finite, the number of dimensions can be very large, and both quantum and Markov processes can readily be extended to infinite dimensional systems. See Busemeyer et al. (2006) and Chapter 7 in Busemeyer and Bruza (2012) for a more comprehensive treatment.
State Space Both quantum and Markov models begin with a set of N states χ = {|X1 , ..., |XN }, where the number of states, N , can be very large. According to the Markov model, a state such as |Xi represents all the information required to characterize the system at some moment, and χ represents the set of all the possible characterizations of the system across time. At any moment in time, the Markov system is exactly located at some specific state in χ , and across time the state changes from one element to another in χ. In comparision, according to the quantum model, a state, such as |Xi , represents a basis vector used to describe the system, and the set χ is a set of basis vectors that span an N -dimensional vector space. At any moment in time, the system is in a superposition state, |ψ, which is a point within the vector space spanned by χ, and across time the point |ψ moves around in the vector space (until a measurement occurs, which reduces the state to the observed basis vector).
Initial State According to the Markov model, the system starts at some particular element of χ . However, this initial state may be unknown to the investigator, in which case a probability, denoted 0 ≤ φi (0) ≤ 1, is assigned to each state |Xi . The N initial probabilities form a N × 1 column matrix ⎤ ⎡ φ1 (0) ⎥ ⎢ .. φ (0) = ⎣ ⎦. . φN (0)
It will be convenient to define a 1 × N row matrix
as J = 1 · · · 1 , which is used for summation. More generally, φ (t) represents the probability distribution across states in χ at time t. The Markov model requires this probability distribution to sum to unity: J · φ (t) = 1. According to the quantum model, the system starts in a superposition state |ψ (0) = ψi (0) · |Xi where ψi is the coordinate (called amplitude) assigned to the basis vector |Xi . The N amplitudes for the initial state form N × 1 column matrix ⎡ ⎤ ψ1 (0) ⎢ ⎥ .. ψ (0) = ⎣ ⎦. . ψN (0)
More generally, ψ (t) represents the amplitude distribution across basis vectors in χ at time t. The quantum model requires the squared length of this amplitude distribution to equal unity: ψ (t)† ψ (t) = 1·
quantum models of cognition and decision
379
State Transitions According to the Markov model, the probability distribution across states evolves across time according to the linear transition law φ(t + τ ) = T (t + τ , t) · φ (t) , where T (t + τ , t) is a transition matrix with element Tij representing the probability of transiting to a state in row i from a state in column j. The transition matrix of a Markov model is called stochastic because the columns of T (t + h, t) must sum to one to guarantee that the resulting probability distribution continues to sum to one, that is J · φ(t + τ ) = 1, and recall that J = [1 1 1 ... 1]. (The rows, however, are not required to sum to one.) In many applications, it is assumed that the transition matrix is stationary so that T (t2 +τ , t2 ) = T (t1 + τ , t1 ) = T (τ ) for all t and τ . The transition matrix of a Markov model is called a stochastic matrix because the columns of the transition matrix must sum to unity. According to the quantum model, the amplitude distribution evolves across time according to the linear transition law ψ (t + τ ) = U (t + τ , t) · ψ (t) , where U (t + τ , t) is a unitary matrix with element Uij representing the amplitude for transiting to row i from column j. The unitary matrix must satisfy the unitary property U † · U = I (I is the identity matrix) in order to guarantee that ψ (t)† ψ (t) = 1. That is, the columns are unit length and each pair of columns is orthogonal. A transition matrix can be formed from the unitary matrix by taking the squared modulus of each of the cell entries of U (t + τ , t). The transition matrix formed in this manner is doubly stochastic: both the rows and columns of this transition matrix must sum to unity. This is a more restrictive constraint on the transition matrix as compared to the Markov model. In many applications, it is assumed that the unitary matrix is stationary so that U (t2 + τ , t2 ) = U (t1 + τ , t1 ) = U (τ ) for all t and τ . According to the Markov model, the stationary transition matrix obeys the Kolmogorov forward equation d T (t) = K · T (t), dt where K is the intensity matrix, with element Kij , and Kij ≥ 0 for i = j, and j Kij = 0 to guarantee that T (t) remains a transition matrix. 380
new directions
According to the quantum model, the stationary unitary matrix obeys the Schrödinger equation d U (t) = −i · H · U (t) , dt where H is the Hamiltonian matrix, which is a Hermitian matrix H † = H , to guarantee that U (t) is a unitary matrix. This is where complex numbers enter in a significant way into quantum models. For the Markov model, the solution to the Kolmogorov foward equation is the following matrix exponential function T (t) = exp (t · K ) . For the quantum model, the solution to the Schrödinger equation is the following complex matrix exponential function U (t) = exp (−i · t · H ) . In summary, the probability distribution across states for the Markov model at time t equals φ (t) = exp (t · K ) · φ (0) , and likewise the amplitude distribution across states for the quantum model at time t equals ψ (t) = exp (−i · t · H ) · ψ (0) . The most important step for building a dynamic model is specifying the intensity matrix for the Markov model or specifying the Hamiltonian matrix for the quantum model. Here psychological science enters by developing a mapping from the psychological factors onto the parameters that define these matrices. An example is provided following this section to illustrate this model development process.
Response Probabilities Consider the probability of observing the response Rk at time t, which is denoted p (R (t) = Rk ). In this section, we use the same choice probability notation for both the Markov and quantum models. Assume that φ (t) is the current probability distribution for the Markov model and ψ (t) is the current amplitude distribution for the quantum model at time t. Both the Markov and quantum models determine the probability of a response by evaluating the set of states that map onto that particular response. Suppose a subset of states, χk ⊂ χ, are mapped onto a response Rk . Define Mk as a N × N indicator matrix, which is a diagonal matrix with ones on the diagonal corresponding to
the states mapped onto the response Rk , and zeros everywhere else. Then according to the Markov model, the response probability equals (recall J = [1 1 1. . .1]) p (R(t) = Rk ) = J · Mk · φ (t) . If in fact, the response Rk is observed at time t, then the new probability distribution, conditioned on this observation equals φ (t|Rk ) =
MK · φ (t) . p (R (t) = Rk )
According to the quantum model, the response probability equals p (R (t) = Rk ) = Mk · ψ (t)2 . If in fact, the response Rk is observed at time t, then the new amplitude distribution, conditioned on this observation equals ψ (t|Rk ) = #
Mk · ψ (t) . p (R (t) = Rk )
The conditional states, φ (t|Rk ) for the Markov model and ψ (t|Rk ) for the quantum model, then become the “initial” states to be used for further evolution and future observations.
Application to Decision Making This section examines two puzzling findings from decision research. One is the violation of the “sure thing” principle (Tversky & Shafir, 1992). Savage introduced the “sure thing” principle as a rational axiom for the foundation of decision theory (1954). According to the sure thing principle, if under state of the world X, you prefer action A over B, and if under the complementary state of the world X , you also prefer action A over B, then you should prefer action A over B even when you do not know the state of the world. A violation of the sure thing principle occurs when A is preferred over B for each known state of the world, but the opposite preference occurs when the state of the world is unknown. The other puzzling finding is the violation of the principle of dynamic consistency, called dynamic inconsistency. Dynamic consistency is considered in standard theory to be a rational principle for making dynamic decisions involving a sequence of actions and events over time. According to the backward induction algorithm used to form optimal plans with dynamic decisions, a person works backward making plans at the end of the sequence
in order to decide actions at the beginning of the sequence. To be dynamically consistent, when reaching the decisions at the end of the sequence, one should follow through on the plan that was used to make the decision at the beginning of the sequence. Violations of dynamic decision-making occur when people change plans and fail to follow through on a plan once they arrive at the final decisions.
Two-Stage Gambling Paradigm Tversky and Shafir (1992) experimentally investigated the sure thing principle using a two-stage gamble. They presented 98 students with a target gamble that had an equal chance of winning $200 or losing $100 (they used hypothetical money). The students were asked to imagine that they already played the target gamble once, and now they were asked whether they wanted to play the same gamble a second time. Each person experienced three conditions that were separated by a week and mixed with other decision problems to produce independent decisions. They were asked if they wanted to play the gamble a second time, given that they won the first play (Condition 1: known win), given that they lost the first play (Condition 2: known loss), and when the outcome of the first play was unknown (Condition 3: unknown). If they thought they won the first gamble, the majority (69%) chose to play again; if they thought they lost the first gamble, then again the majority (59%) chose to play again; but if they didn’t know whether they won or lost, then the majority chose not to play (only 36% wanted to play again). Tversky and Shafir (1992) explained these findings by claiming that people fail to follow through on consequential reasoning. When a person knows she/he has won the first gamble, then a reason to play again arises from the fact that she/he has extra house money to play with. When the person knows she/he has lost the first gamble, then a reason to play again arises from the fact that she/he needs to recover for their losses. When the person does not know the outcome of the first play, these reasons fail to arise. However, why not? Pothos and Busemeyer (2009) explained these and other results found by Shafir and Tversky (1992) using the concept of quantum interference. Referring back to the section Violations of the Law of Total Probability, define the event B as deciding to play the gamble on the second stage, define event
quantum models of cognition and decision
381
A as winning the first play, and define event A¯ as losing the first play. Then Eq. 2 expresses the probability of playing the gamble on the second stage for the unknown condition in terms of the total probability, qT (B), of playing the second stage on either of the two known conditions, plus the interference term Int. Given that the probability of winning the first stage equals .50, then a violation of the sure thing principle is predicted whenever qT (B) > .50 and the interference term Int is sufficiently negative so that q(B) < .50. But what determines the interference term? To answer this question, Pothos and Busemeyer (2009) developed a dynamic quantum model to account for the violation of the sure thing principle. This model is described in detail later, but before presenting these modeling details, let us first examine the second puzzling finding regarding violations of dynamic consistency. The same model is used to explain both findings. Barkan and Busemeyer (1999, 2003) used the same two-stage gambling paradigm to study another phenomena called dynamic inconsistency, which occurs whenever a person changes plans during decision-making. Each study included a total of 100 people, and each person played a series of gambles twice. Each gamble had an equal chance of producing a win or a loss (e.g., equal chance to win 200 points or lose 100 points, where each point was worth $0.01). Different gambles were formed by changing the amounts to win or lose. For each gamble, the person was forced to play the first round, and then contingent on the outcome of the first round, they were given a choice whether to take the same gamble on the second round. Choices were made under two conditions: a planned versus a final choice. For the planned choice, contingent on winning the first round, the person had to select a plan about whether to take or reject the gamble on the second round; contingent on losing the first round, the person had to make another plan about whether to take or reject the gamble on the second round. Then the first-stage gamble was actually played out and the actual win or loss was revealed. For the final choice, after actually experiencing the win on the first round, the person made a final decision to take or reject the second round. Likewise, after actually experiencing a loss on the first round, the person had to decide whether to take or reject the gamble on the second round. The plan and the final decisions were made equally valuable because the experimenter randomly selected either the planned action or the final action to determine 382
new directions
the final payoff with real money at stake. The results showed that people violate the dynamic consistency principle: Following an actual win, they changed from planning to take to finally rejecting the second stage; following an actual loss, they changed from planning to reject to finally taking the second stage. For example, Table 17.1 shows the results from the four gambles used by Barkan and Busemeyer (1999). The first two columns show the amounts to win or lose, the next two colums show the probability of taking the gamble under the plan (conditioned a planned win or loss), and the last two columns show the probability to take the gamble for the final decision (conditioned on an experienced win or a loss). Similar results were found by Barkan and Busemeyer (2003) using 17 different gambles. For later reference, we will denote the amount of the win by xW and the amount of the loss by xL . So for example, in the first row, xW = 80 and xL = 100. It is worth mentioning that the results shown in Table 1 once again demonstate a violation of the classical law of total probability in the following way. If the law of total probability holds, then the probability of taking the gamble during the plan (denoted p(T |P) and shown in the columns labeled “Plan Win, Plan Lose”) equals the probability of winning the first play (denoted p(W ) which was stated to be equal to .50) times the probability to take the gambe after a win (denoted p(T |W ) and shown under the column “Final Win” in Table 17.1) plus the probability of losing the first play (denoted p(L) = 1 − p(W )) times the probability of taking the gamble following a loss (denoted p(T |L) and shown under the column “Final Loss” in Table 17.1) so that p(T |P) = p(W ) · p(T |W ) + p(L) · p(T |L). All the gambles have the same probability of winning, and p(W ) is fixed across gambles and is stated in the problem to be equal to .50. However, these assumptions fail to reproduce the findings shown in Table 17.1. For example, we require p(W ) = .64 to reproduce the data in the first row, but we require p(W ) = .43 to reproduce the third row, and we require p(W ) = .31 to reproduce the data in the fourth row, and even worse, no legitimate value of p(W ) can be found to reproduce the second row.
Markov Dynamic Model for Two-Stage Gambles First let us construct a general Markov model for this two-stage gambling task. The Markov model
Table 17.1. Barkan and Busemeyer (1999). Win
Lose
Plan Win
Plan Loss
Final Win
Final Loss
80
100
.25
.26
.20
.35
80
40
.76
.72
.69
.73
200
100
.68
.68
.60
.75
200
40
.84
.86
.76
.89
uses a four-dimensional state space with states {|BW AT , |BW AR , |BL AT , |BL AR } , where |BW AT represents the state “believe you win the first gamble and act to take the second gamble,” |BW AR represents the state “believe you win the first gamble and act to reject the second gamble,” |BL AT represents the state “believe you lose the first gamble and act to take the second gamble,” and |BL AR represents the state “believe you lose the first gamble and act to reject the second gamble.” The probability distribution over states is represented by a 4 × 1 column matrix (that sums to unity) composed of two parts ⎡ ⎡ ⎤ ⎤ φWT 0 ⎢ φWR ⎥ ⎢ 0 ⎥ ⎢ ⎥ ⎥ φ = φ W + φ L , φW = ⎢ ⎣ 0 ⎦ , φL = ⎣φLT ⎦ . φLR 0 Before evaluating the payoffs of the gamble, the decision maker has an initial state represented by φ (0). This initial state depends on information about the outcome of the first play. If the outcome of the first play is unknown (i.e., the planning stage), then the initial state is set equal to φ (0) = φU , which has coordinates φWT = φWR = φLT = φLR = 14 . If the first play is known to be a win, then the initial state is set equal to φ (0) = φW with coordinates φWT = φWR = 12 , φLT = φLR = 0. If the first play is known to be a loss, then the initial state is set equal to φ (0) = φL with coordinates φWT = φWR = 0, φLT = φLR = 12 . The probabilities of taking the gamble, depending on the win or lose first game belief states, are then determined by a transition matrix TW 0 , T (t) = 0 TL where TW is a 2 × 2 transition matrix conditioned on winning, and TL is a 2 × 2 transition matrix conditioned on losing.
The matrix that picks out the states corresponding to the action of “taking the gamble” is represented by MT 0 1 0 M= , MT = . 0 MT 0 0
Recall that J = 1 1 1 1 sums across states to obtain the probability of a response. Finally, the Markov model predicts: .50 , p (T |W ) = J · M · T (t) · φW = J · MT · TW · .50
.50 p (T |L) = J · M · T (t) · φL = J · MT · TL · , .50
0 MT and p (T |U ) = J · M · T (t) · φU = J · 0 MT ⎛⎡ ⎤ ⎡ ⎤⎞ .25 0 ⎢.25⎥ ⎢ 0 ⎥⎟ TW 0 ⎜ · ⎜⎢ ⎥ + ⎢ ⎥⎟ · 0 TL ⎝⎣ 0 ⎦ ⎣.25⎦⎠ 0 .25
.50 = (.50) · 1 1 · MT · TW · .50
.50 + (.50) · 1 1 · MT · TL · .50
= (.50) · p (T |W ) + (.50) · p (T |L) . The last line shows that the Markov model must satisfy the law of total probability. Note that the Markov model must always obey the law of total probability, and, thus, already qualitatively fails to account for the violation of the sure thing principle and the dynamic inconsistency results described earlier.
quantum models of cognition and decision
383
Quantum Dynamic Model for Two-Stage Gambles
H1 =
Pothos and Busemeyer (2009) developed a quantum dynamic model that has been applied to the two-stage gambling task. The quantum model also uses a four-dimensional vector space spanned by four basis vectors {|BW AT , |BW AR , |BL AT , |BL AR } , where |BW AT represents the event “believe you win the first gamble and act to take the second gamble,” |BW AR represents the event “believe you win the first gamble and act to reject the second gamble,” |BL AT represents the event “believe you lose the first gamble and act to take the second gamble,” and |BL AR represents “believe you lose the first gamble and act to reject the second gamble.” The decision-maker’s state is a superposition over these four basis states: |ψ = ψWT · |BW AT + ψWR · |BW AR + ψLT · |BL AT + ψLR · |BL AR The matrix representation of this superposition state is the 4 × 1 column matrix (length equal to one) composed of two parts ⎡
ψ = ψW + ψL , ψW
⎤
⎡
⎤
ψWT 0 ⎢ ψWR ⎥ ⎢ 0 ⎥ ⎢ ⎥ ⎥ =⎢ ⎣ 0 ⎦ , ψL = ⎣ψLT ⎦ . 0 ψLR
Before evaluating the payoffs of the gamble, the decision maker has an initial state represented by ψ (0). This initial state depends on information about the outcome of the first play. If the outcome of the first play is unknown (i.e., the planning stage), then the initial state is set equal to ψ (0) = ψU , which has coordinates .50. If the first play is known to be a win, then the initial state is set equal to√ψ (0) = ψW with coordinates ψWT = ψWR = .50, ψLT = ψLR = 0. If the first play is known to be a loss, then the initial state is set equal to ψ (0) = ψL with √ coordinates ψWT = ψWR = 0, ψLT = ψLR = .50. Evaluation of the gamble payoffs causes the initial state ψ (0) to evolve into a final state ψ (t) after a period of deliberation time t, and this final state is used to decide whether to take or reject the gamble at the second stage. The Hamiltonian H used for this evolution is H = H1 + H2 , where 384
new directions
HW 0
⎡ ⎢ HL = ⎣
√hW
0 1+h2 , HW = ⎣ 1 w HL √ 2
hL
1+hL2 1 1+hL2
⎡
⎡
⎤
1+hw
√1
⎤
2 1+hw ⎦
√−hW 2
1+hw
1 1+hL2 ⎥ ⎦, −h L 1+hL2
1 0 −c ⎢ 0 −1 H2 = √ ⎢ ⎣ 2 1 0 0 1
⎤ 1 0 0 1 ⎥ ⎥. −1 0 ⎦ 0 1
The matrix HW in the upper left corner of H1 rotates the state toward taking or rejecting the gamble depending on the final payoffs (xW + xW , xW − xL ), given an initial win of the amount of x W from the first play. The coefficients hW and hL in the Hamiltonian HW are supposed to range between −1 and +1. So we need to map the utility differences into this range. The hyperbolic tangent provides a smooth S-shaped mapping. We then define hW in terms of the utility difference following a win as follows: 2 a − 1, DW = uW − xW , 1 + e−DW ⎧1 · (xW + xW )a + 12 · (xW − xL )a ⎪ ⎪ ⎪2 ⎨ (xW > xL ) = 1 1 a a ⎪ · (x W + xW ) − 2 · b · (xL − xW ) ⎪ ⎪ ⎩2 (xL > xW ).
hW =
uW
The variable uW is the utility of playing the gamble after a win, which uses a risk-aversion parameter a and a loss-aversion parameter b. The variable DW is the difference between the utility of taking and rejecting the gamble after a win. The matrix HL in the bottom right corner of H1 rotates the state toward taking or rejecting the gamble depending on the final payoffs (xW − xL , −xL − xL ) given an initial loss of the amount x L from the first play. Once again, using the hyperbolic tangent, we map the utility differences following a loss into hL as follows: 2 − 1, DL = uL − xLa , hL = 1 + e −DL ⎧1 1 a a ⎪ ⎪ 2 · (xW − xL ) − 2 · b · (xL + xL ) ⎪ ⎨ (x > x ) W L uL = ⎪− 12 · b · (xL − xW )a − 12 · b · (xL + xL )a ⎪ ⎪ ⎩ (xL > xW ). The variable uL is the utility of playing the gamble after a loss, which uses the same risk aversion
,
parameter a and the same loss aversion parameter b. The variable DL is the difference between the utility of taking and rejecting the gamble after a loss. The matrix H2 is designed to align beliefs with actions. This produces a type of “hot hand” effect. The parameter c determines the extent that beliefs can change from their initial values during the evaluation process, and it is critical for producing interference effects. Critically, if the parameter c is set to zero, then the quantum model reduces to a special case of a Markov model, the law of total-probability holds, and there are no interference effects. According to the quantummodel hypothesis, a nonzero value of this parameter c is expected, which will reproduce the 17 different quantum interference terms for the 17 different gambles. The initial state evolves to the final state according to the unitary evolution ψ(t) = exp (−i · t · H ) · ψ (0) . Following Pothos and Busemeyer (2009), the deliberation time was set equal to t = π2 , because at this time point, the evolution of preference first reaches an extreme point. The projector for choosing to gamble is represented by the indicator matrix that picks the “take gamble” action 0 1 0 MT . M= , MT = 0 MT 0 0 The probability of taking the gamble for the known win, known loss, and unknown (plan) conditions then equals '2 ' p (T |W ) = 'M · exp( − i · t · H ) · ψW ' '2 ' p (T |L) = 'M · exp( − i · t · H ) · ψL ' ' '2 p (T |U ) = 'M · exp( − i · t · H ) · ψU ' . The parameter c is critical for producing violations of the law of total probability. If we set the quantum-model parameter c = 0, then the quantum model predicts ' ' MT 0 ' p (T |U ) = ' 0 MT 0 exp ( − i · t · HW ) · · 0 exp ( − i · t · HL ) ⎡
⎤ ⎡ .50 0 ⎢ .50 ⎥ ⎢ 0 ⎢ ⎥+⎢ ⎣ 0 ⎦ ⎣ .50 0 .50
⎤ '2 ' ' ⎥' ⎥' ⎦' ' '
' '2 √ ' .50 ' ' √ = (.50) · ' M · exp ( − i · t · H ) · W ' T .50 ' ' √ '2 ' .50 ' ' √ +(.50) · ' M · exp ( − i · t · H ) · L ' T .50 ' = (.50) · p(T |W ) + (.50) · p(T |L). Therefore, if c = 0, the quantum model violates the law of total probability; but if c = 0, the quantum model satisfies the law of total probability. In fact, when c = 0, the Markov model can reproduce the predictions of the quantum model by setting each element of the first row of the transition matrix TW equal to p (T |W ) predicted by the quantum model, and by setting each element of the first row of the transition matrix TL equal to p (T |L) predicted by the quantum model. In other words, we can obtain a Markov model from the quantum model by setting c = 0.
Model Comparisons Next, we compare the Markov (obtained by setting c = 0 in the quantum model) and the quantum model (allowing c = 0) with respect to their fits to the Barkan and Busemeyer (2003) results in three different ways. The first is to compare least-squares fits to the 17 (gambles with different payoff conditions) × 2 (plan versus final) = 34 mean choice proportions reported in Barkan and Busemeyer (2003). The second is to compare the models using maximum likelihood estimates at the individual level and using AIC and BIC methods. The third is to estimate the hierarchical Bayesian posterior distribution for the critical parameter c that distinguishes the two models. The models are first compared using R 2 = SSE SSE , and adjusted R 2 = 1 − TSS · 34−1 1 − TSS 34−n . The latter index includes a penalty term for extra parameters. These statistics were computed with 2 2 P i − pi Pi − P¯ , SSE = and TSS = where Pi is the observed mean proportion of trials to choose gamble i = 1, ..., 34, pi is the predicted mean proportion, P¯ is the grand mean, 34 = 17 (payoff conditions) ×2 (plan versus final stage choices) is the number of observed choice proportions being fit. The quantum model has n = 3 parameters, and the best-fitting parameters (minimizing sum of squared error) are a = 0.71 (risk aversion), b = 2.54 (loss aversion), and c = −4.40. The risk aversion parameter is a bit below one as expected, and the loss parameter b exceeds one, as it should be. The
quantum models of cognition and decision
385
model produced an R 2 = 0.8234 and an adjusted R 2 = 0.8120. The Markov model, obtained by setting c = 0 in the quantum model, has only two parameters and it produced an R 2 = 0.7854 and an adjusted R 2 = 0.7787, which are lower than those of the quantum model. Next, the models were compared using AIC and BIC methods based on maximum likehood fits to individuals. For person i on trial t we observe a data pattern Xi (t) = [xTT (t), xTR (t), xRT (t), xRR (t)] defined by xjk (t) = 1 if event (j, k) occurs and otherwise zero, where TT is the event “planned to take the gamble and finally took the gamble,” TR is the event “planned to take the gamble but finally rejected the gamble,” RT is the event “planned to reject the gamble but finally took the gamble” and RR is the event “planned to reject the gamble and finally rejected the gamble.” To allow for possible dependencies between a pair of choices within a single trial, an additional memory recall parameter, m, was included in each model. For both models, it was assumed that there is some probability 0 ≤ m ≤ 1 that the person simply recalls and repeats the planned choice during the final choice, and there is some probability 1−m that the person forgets or ignores the planned choice when making the final choice. After including this memory parameter, the prediction for each event becomes pTT = p(T |plan) · (m · 1 + (1 − m) · p(T |final)) pTR = p(T |plan) · (1 − m) · p(R|final) pRT = p(R|plan) · (1 − m) · p(T |final) pRR = p(R|plan) · (m · 1 + (1 − m) · p(R|final)) Using these definitions for each model, the log 6 likelihood function for the 33 trials (with a pair of plan and final choices on each trial) from a single person can be expressed as ln L (Xi (t))=
xjk (t) · ln pjk
j,k
ln L (Xi ) =
33
ln L (Xi (t)) .
t=1
The log likelihood from each person was converted into Gi2 = −2 · ln (Li ) which indexes the lack of fit, and the parameters that minimized 7 Gi2 were found for each person. The quantum model has one more parameter than the Markov model. In this case, the AIC badness of fit index is defined as Gi2 +2, where 2 is the penalty for 386
new directions
the one extra parameter. Using AIC, 48 out of the 100 participants produced AIC indices favoring the quantum model over the Markov model. The BIC penalty depends on the number of observations, which is 33 for each person, and so for one extra parameter, the penalty equals log(33) = 3.4965. Using the more conservative BIC index, 22 out of the 100 participants produced BIC indices favoring the quantum model over the Markov model. Thus a majority of participants were adequately fit by the Markov model, but a substantial percentage of participants were better fit by the quantum model. One final method used to compare models is to examine the posterior distribution of the parameter c when estimated by hiearchical Bayesian methods. The details for this analysis are described in Busemeyer, Wang, and Trueblood (2012) and the results are only briefly summarized here. The hierarchical Bayesian estimation method starts by assuming a prior distribution over the individuals for each of the four quantum model parameters. Then, the likelihoods from the individual fits are used to update the prior distribution into a posterior distribution over the indivduals for the four parameters. The posterior distribution of the critical quantum parameter c is shown in Figure 17.4. The entire distribution lies below zero, and the mean of the distribution equals to −2.67. This supports the hypothesis that the critical quantum parameter, c, is not zero, and the model does not reduce to the Markov model. It is worth noting that the same quantum model also accounts for the violations of the surething principle, whereas the Markov model cannot explain this violation. Furthermore, the same quantum model described here was used to explain two other puzzling findings (not reviewed here). One is concerned with order effects on inference (Trueblood & Busemeyer, 2010), and the other is the interference of categorization on decision making (Busemeyer et al., 2009). In sum, the same quantum model has been successfully applied to four distinct puzzling judgement and decision findings, which builds confidence in the broad applicability of the model.
Concluding Comments This chapter provides a brief introduction to the basic principles of quantum theory and a few major paradoxical judgement and decision findings that
0.2 0.18
Posterior Probability
0.16 0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 −5
−4
−3
−2
−1 0 1 Quantum Parameter c
2
3
4
5
Fig. 17.4 Posterior Distribution of quantum model parameter c across individuals.
the theory has been used to explain. The theory is new and needs further testing, but the initial successful applications demonstrate its viability and theoretical potential. Busemeyer and Bruza (2012) provide a more detailed presentation of the basic principles, and they also describe in detail a much larger number of empirical applications. Also Pothos and Busemeyer (2013) summarize applications of quantum theory to cognitive science. Finally, special issues on quantum cognition have recently appeared in Journal of Mathematical Psychology (Bruza, Busemeyer, & Gabora, 2009) and Topics in Cognitive Science (Wang, Busemeyer, Atmanspacher, & Pothos, 2013). What are the advantages and disadvantages of the quantum approach as compared to traditional cognitive theories? First, let us consider some of the disadvantages. One is that the concepts and mathematics are very new and unfamiliar to psychologists, and learning how to use them requires an investment of time and effort. Second, because of the unfamiliarity, it may seem difficult at first to intuitively connect these ideas to traditional concepts of cognitive psychology, such as memory, attention, and information processing. Finally, applications of quantum theory to cognition have to overcome skepticism that naturally arises when introducing a revolutionary scientific new idea. However, now consider the advantages. First, the mathematics is not as difficult as it seems, and it only requires knowledge of linear algebra and differential equations. Second, once one
does become familiar with the mathematics and concepts, it becomes apparent that quantum theory provides a conceptually elegant and innovative way to formalize and represent some of the major concepts from cognition. For example, the superposition principle provides a natural way to represent parallel processing of uncertain information and capture that deep ambiguous feelings. Moreover, the consideration of quantum models allows the (re)introduction of new and useful theoretical principles into psychology, such as incompatibility, interference, and entanglement. In all, the main advantage of quantum theory is that a small set of principles provide a coherent explanation for a wide variety of puzzling results that have never been connected before under a single theoretical framework (see Box 2).
Notes 1. In particular, this chapter does not rely on the quantum brain hypothesis (Hammeroff, 1998). 2. This section makes little use complex numbers, but the section on dynamics requires their use. 3. This chapter follows the Dirac representation of the state as a vector rather than the more general von Neumann representation of the state as a density matrix. 4. The basis χ is an arbitrary choice and there are many other choices for a basis. Initially, we restrict ourselves to one arbitrarily chosen basis. Later we discuss issues arising from using different choices for the basis. 5. Kolmogorov assigned a single sample space to the outcomes of an experiment. This allows one to use a different sample space
quantum models of cognition and decision
387
for each experiment. But then the problem is that these separate sample spaces are left stochastically unrelated. 6. 16 gambles were played twice, one other gamble was played only once. 7. A surprising feature was found with the log likelihood function of the quantum model as a function of the key quantum parameter c. The log likelihood function forms a damped oscillation that converges at a reasonably high log likelihood at the extremes, and this is true both for the average across participants as well as for individual participants.
References Aerts, D. (2009). Quantum structure in cognition. Journal of Mathematical Psychology, 53(5), 314–348. Aerts, D., & Aerts, S. (1994). Applications of quantum statistics in psychological studies of decision processes. Foundations of Science, 1, 85–97. Aerts, D., & Gabora, L. (2005). A theory of concepts and their combinations ii: A Hilbert space representation. Kybernetes, 34, 192–221. Aerts, D., Gabora, L., & Sozzo, S. (2013). Concepts and their dynamics: A quantum - theoretic modeling of human thought. Topics in Cognitive Science, 5, 737–773. Atmanspacher, H., & Filk, T. (2010). A proposed test of temporal nonlocality in bistable perception. Journal of Mathematical Psychology, 54, 314–321. Atmanspacher, H., Filk, T., & Romer, H. (2004). Quantum zero features of bistable perception. Biological Cybernetics, 90, 33–40. Barkan, R., & Busemeyer, J. R. (1999). Changing plans: dynamic inconsistency and the effect of experience on the reference point. Psychological Bulletin and Review, 10, 353– 359. Barkan, R., & Busemeyer, J. R. (2003). Modeling dynamic inconsistency with a changing reference point. Journal of Behavioral Decision Making, 16, 235–255. Blutner, R. (2009). Concepts and bounded rationality: An application of Niestegge’s approach to conditional quantum probabilities. In L. e. a. Acardi (Ed.), Foundations of probability and physics-5 (Vol. 1101, p. 302–310). Blutner, R., Pothos, E. M., & Bruza, P. (2013). A quantum probability perspective on borderline vagueness. Topics in Cognitive Science, 5(4), 711–736. Bordley, R. F., & Kadane, J. B. (1999). Experimentdependent priors in psychology. Theory and Decision, 47 (3), 213–227. Brainerd, C. J., Wang, Z., & Reyna, V. (2013). Superposition of episodic memories: Overdistribution and quantum models. Topics in Cognitive Science, 5(4), 773–799. Bruza, P., Kitto, K., Nelson, D., & McEvoy, C. (2009). Is there something quantum-like in the human mental lexicon? Journal of Mathematical Psychology, 53, 362–377. Bruza, P. D., Busemeyer, J., & Gabora, L. (Eds.). (2009). Special issue on quantum cognition (Vol. 53). Journal of Mathematical Psyvhology. Busemeyer, J. R., & Bruza, P. D. (2012). Quantum models of cognition and decision. Cambirdge University Press.
388
new directions
Busemeyer, J. R., Pothos, E. M., Franco, R., & Trueblood, J. S. (2011). A quantum theoretical explanation for probability judgment errors. Psychological Review, 118(2), 193–218. Busemeyer, J. R., Wang, Z., & Lambert-Mogiliansky, A. (2009). Empirical comparison of markov and quantum models of decision making. Journal of Mathematical Psychology, 53(5), 423–433. Busemeyer, J. R., Wang, Z., & Townsend, J. (2006). Quantum dynamics of human decision making. Journal of Mathematical Psychology, 50(3), 220–241. Busemeyer, J. R., Wang, Z., & Trueblood, J. S. (2012). Hierarchical bayesian estimation of quantum decision model parameters. In J. R. Busemeyer, F. DuBois, A. LambertMogiliansky, & M. Melucci (Eds.), Quantum interaction. lecture notes in computer science, vol. 7620 (pp. 80–89). Springer. Conte, E., Khrennikov, A. Y., Todarello, O., Federici, A., Mendolicchio, L., & Zbilut, J. P. (2009). Mental states follow quantum mechanics during perception and cognition of ambiguous figures. Open Systems and Information Dynamics, 16, 1–17. Dirac, P. A. M. (1958). The principles of quantum mechanics. Oxford University Press. Dzhafarov, E., & Kujala, J. V. (2012). Selectivity in probabilistic causality: Where psychology runs into quantum physics. Journal of Mathematical Psychology, 56, 54–63. Feldman, J. M., & Lynch, J. G. (1988). Self-generated validity and other effects of measurement on belief, attitude, intention, and behavior. Journal of Applied Psychology, 73(3), 421–435. Franco, R. (2009). Quantum amplitude amplification algorithm: An explanaton of availability bias. In P. Bruza, D. Sofge, W. Lawless, K. van Rijsbergen, & M. Klusch (Eds.), Quantum interaction (pp. 84–96). Springer. Fuss, I. G., & Navarro, D. J. (2013). Open parallel cooperative and competitive decsision processes: A potential provenance for quantum probability decision models. Topics in Cognitive Science, 5(4), 818–843. Gleason, A. M. (1957). Measures on the closed subspaces of a Hilbert space. Journal of Mathematical Mechanics, 6, 885– 893. Griffiths, R. B. (2003). Consistent quantum theory. Cambridge University Press. Gudder, S. P. (1988). Quantum probability. Academic Press. Hammeroff, S. R. (1998). Quantum computation in brain microtubles? the penrose - hameroff “orch or” model of consiousness. Philosophical Transactions Royal Society London (A), 356, 1869–1896. Heisenberg, W. (1958). Physics and philosophy. Harper and Row. Hughes, R. I. G. (1989). The structure and interpretation of quantum mechanics. Harvard University Press. Ivancevic, V. G., & Ivancevic, T. T. (2010). Quantum neural computation. Springer. Khrennikov, A. Y. (2010). Ubiquitous quantum structure: From psychology to finance. Springer. Kolmogorov, A. N. (1933/1950). Foundations of the theory of probability. N.Y.: Chelsea Publishing Co. Lambert-Mogiliansky, A., Zamir, S., & Zwirn, H. (2009). Type indeterminacy: A model of the “KT”
(Kahneman-Tversky)-man. Journal of Mathematical Psychology, 53(5), 349–361. La Mura, P. (2009). Projective expected utility. Journal of Mathematical Psychology, 53(5), 408–414. Morier, D. M., & Borgida, E. (1984). The conjuction fallacy: A task specific phenomena? Personality and Social Psychology Bulletin, 10, 243–252. Payne, J., Bettman, J. R., & Johnson, E. J. (1992). Behavioral decision research: A constructive processing perspective. Annual Review of Psychology, 43, 87–131. Peres, A. (1998). Quantum theory: Concepts and methods. Kluwer Academic. Pothos, E. M., & Busemeyer, J. R. (2012). Can quantum probability provide a new direction for cognitive modeling? Behavioral and Brain Sciences, 36, 255–274. Pothos, E. M., Busemeyer, J. R., & Trueblood, J. S. (2013). A quantum geometric model of similarity. Psychological Review, 120(3), 679–696. Savage, L. J. (1954). The foundations of statistics. John Wiley & Sons. Schachter, S., & Singer, J. E. (1962). Cognitive, social, and physiological determinants of emotional state. Psychological Review, 69(5), 379–399. Shafir, E., & Tversky, A. (1992). Thinking through uncertainty: nonconsequential reasoning and choice. Cognitive Psychology, 24, 449–474. Stolarz-Fantino, S., Fantino, E., Zizzo, D. J., & Wen, J. (2003). The conjunction effect: New evidence for robustness. American Journal of Psychology, 116 (1), 15–34. Tentori, K., & Crupi, V. (2013). Why quantum probability does not explan the conjunction fallacy. Behavioral and Brain Sciences, 36 (3), 308–310.
Trueblood, J. S., & Busemeyer, J. R. (2010). A quantum probability account for order effects on inference. Cognitive Science, 35, 1518–1552. Trueblood, J. S., & Busemeyer, J. R. (2011). A quantum probability model of causal reasoning. Frontiers in cognitive science, 3, 138. Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjuctive fallacy in probability judgment. Psychological Review, 90, 293–315. Tversky, A., & Shafir, E. (1992). The disjunction effect in choice under uncertainty. Psychological Science, 3, 305–309. Von Neumann, J. (1932/1955). Mathematical foundations of quantum theory. Princeton University Press. Wang, Z., & Busemeyer, J. R. (2013). A quantum question order model supported by empirical tests of an a priori and precise prediction. Topics in Cognitive Science, 5, 689–710. Wang, Z., Busemeyer, J. R., Atmanspacher, H., & Pothos, E. M. (2013). The potential of using quantum theory to build models of cognition. Topics in Cognitive Science, 5, 672–688. Wang, Z., Solloway, T., Shiffrin, R. M., & Busemeyer, J. (2014). Context effects produced by question orders reveal quantum nature of human judgments. Proceedings of the National Academy of Sciences, 111(26), 9431–9436. Yates, J. F., & Carlson, B. W. (1986). Conjunction errors: Evidence for multiple judgment procedures, including ’signed summation’. Organizational Behavior and Human Decision Processes, 37, 230–253. Yukalov, V. I., & Sornette, D. (2011). Decision theory with prospect interference and entanglement. Theory and Decision, 70, 283–328.
quantum models of cognition and decision
389
INDEX
Abductive reasoning in clinical cognitive science, 343–344 Absolute identification absolute and relative judgment, 129–130 intertrial interval and sequential effects, 136–138 learning, 130–133 perfect pitch versus, 133–135 response times, 135 theories of, 124–129 Absorbing barriers, 30 Accumulator models, 321–322, 327–328 Across-trial variability, 37–38, 46, 56–57 Actions, in Markov decision process (MDP), 102–103 ACT-R architectures, 126, 219, 301 Additive factors method, 69–70, 89 ADHD, 49–50 Affine transformation, 28 Aging studies, diffusion models in, 48 Akaike information criterion (AIC), 306–308 Alcohol consumption, 50 Aleatory uncertainty, 210 Algom, D., 63 Allais paradox, 219 ANCHOR-based exemplar model of absolute identification, 126 Anderson’s ACT-R model, 126, 219, 301 Anxiety, diffusion models of, 49 Anxiety-prone individuals, threat sensitivity modeling in, 352–354 Aphasia, 49–50 Ashby, F. G., 13 Assimilation and contrast, in absolute identification, 123–124, 128 Associative learning, 194–196 Associative recognition, 47 Attention allocation differences data, 291–292
descriptive model and parameters, 292–293 overview, 290–291 posterior distribution interpretation, 293–295 Attention-weight parameters, 144 Attraction, as context effect in DFT, 225–226 Austerweil, J. L., 187 Autism spectrum disorders, 354–356 Automaticity, 143, 148–150, 325 Autonomous search models, 178–179 Bandit tasks, 111 Basa ganglia model, 51 Baseball batting example, 282–290 data, 283 descriptive model and parameters, 283–285 overview, 282–283 posterior distribution interpretation, 285–290 shrinkage and multiple comparisons, 290 Basis functions, 201 Bayesian information criterion (BIC), 9, 21, 306–308 Bayesian models. See also Hierarchical models, Bayesian estimation in of cognition, 187–208 clustering observations, 192–196 conclusions, 203–204 continuous quantities, 200–203 features as perceptual units, 196–200 future directions, 204 mathematical background, 188–192 overview, 187–188 overview, 40, 169 parsimony principle in, 309–314 of shape perception, 258–260 Bayesian parameter estimation, 348–349
Bayes’ rule, 6, 281–282 BEAGLE (Bound Encoding of the Aggregate Language Environment) model, 243–244, 248 Bellman equation, 103 Benchmark model, 74–75 Benchmark phenomena, in perceptual judgment, 122–124 Berlin Institute of Physiology, 64 Bernoulli, Daniel, 210–211 Bessel, F. W., 65 Bias-variance trade-off, 190–191 BIC (Bayesian information criterion), 9, 21, 306–308 Blood sugar reduction, 50 Bootstrapping, 105 Boundary setting across tasks, 48 Bound Encoding of the Aggregate Language Environment (BEAGLE) model, 243–244, 248 Bow effects, in absolute identification, 123, 128 Brown, S. D., 121 Brown and Heathcote’s linear ballistic accumulator model, 301 BUGS modeling specification language, 282 Busemeyer, J. R., 1, 369 Calculus, 3–5 Candidate decision processes, 14 Capacity coefficient, 72–74 Capacity limitations, in absolute identification, 122–123 Capacity reallocation model, 69 Capacity theory, 90–91 Catastrophe theory, 346 Categorization, 29, 325. See also Bayesian models; Exemplar-based random walk (EBRW) model Category learning, 30–31, 189 Cattell, James McKeen, 66 Chaos-theoretic modeling, 345–346
391
Child development, diffusion models in, 48–49 Chinese restaurant process (CRP) metaphor, 193–195 Choice axiom testing, 211–214 Choice behavior, 199–200 Cholesky transformation, 20 “Chunking,” 66 Clinical psychology, mathematical and computational modeling in, 341–368 contributions of, 349–359 cognition in autism spectrum disorders, 354–356 cognitive modeling of routinely used measures, 356–357 multinomial processing tree modeling of memory, 350–352 in pathocognition and functional neuroimaging, 357–359 threat sensitivity modeling of anxiety-prone individuals, 352–354 distinctions in, 343–346 overview, 341–343 parameter estimation in, 346–349 special considerations, 359–361 Clustering observations, 192–196 Coactivation, 73 COALS (Correlated Occurrence Analogue to Lexical Semantics) model, 241, 248 Coexistence model (CXM), 303–304, 306, 308–309, 313 Cognition. See Bayesian models; Quantum models of cognition and decision Cognitive control of perceptual decisions, 330, 334 Cognitive modeling, 219–226 of clinical science measures, 356–357 context effects example, 225–226 decision field theory for multialternative choice problems, 222–225 multi-attribute, 221–222 overview, 220–221 “horse race,” 356
392
index
Cognitive-psychological complementarity, 87–90 Cognitive psychometrics, 290 Cohen’s PDP model, 301 Cold cognition, 361 Commutativity, 375 Competing accumulator models, 322 Complication experiment, 66–68 Component power laws model, 301 Compositional semantics, 249 Compromise, as context effect in DFT, 225–226 Computational reinforcement learning (CRL), 99–117 decision environment, 102 exploration and exploitation balance, 106 goal of, 101 good decision making, 103–104 historical perspective, 100–101 neural correlates of, 106–108 Q-learning, 105–106 research issues, 108–114 human exploration varieties, 110–113 model-based versus model-free learning, 108–109 reward varieties, 113–114 state representation influence, 109–110 temporal difference learning, 104–105 values for states and actions, 102–103 Conditioning, 101–103, 111, 195 Confidence judgments, 52–53 Conjunction probability judgment errors, 376–379 Connectionist models decision field theory as, 223, 225 of semantic memory, 234–239 Constancy, in shape perception, 256–257 Constructed semantics model (CSM), 247 Context, 175–178 Context-noise models, 172 Contingency table, 6f Continuous quantities, relationships of, 200–203 Contrast and assimilation, in absolute identification, 123–124, 128 Correlated Occurrence Analogue to Lexical Semantics (COALS) model, 241, 248
COVIS theory of category learning, 30–31 Credit assignment problem, in reinforcement learning, 100–101, 103 Criss, A. H., 165 CRL (Computational reinforcement learning). See Computational reinforcement learning (CRL) CrossCat model, 195 CRP (Chinese restaurant process) metaphor, 193–195 Crude two-part code, in MDL, 307–308 CSM (constructed semantics model), 247 Cued recall models of episodic memory, 173–174 Cumulative prospect theory, 217–219 CXM (coexistence model), 303–304, 306, 308–309, 313 Deadline tasks, 41–42 Decisional separability, 15–16, 22f, 23 Decision-boundary models, 30, 142 Decision field theory (DFT), 220–225 Decision-making models, 209–231. See also Computational reinforcement learning; Perceptual decision making, neurocognitive modeling of; Quantum models of cognition and decision choice axiom testing, 211–214 cognitive models context effects example, 225–226 decision field theory, 220–221 decision field theory for multialternative choice problems, 222–225 multi-attribute decision field theory, 221–222 overview, 219–220 historical development of, 210–211 overview, 209–210 rational choice models, 214–219 Decision rules for Bayesian posterior distribution, 287 Dennis, S., 232
Density estimation, in Bayesian models, 188–190 Depression, diffusion models of, 49 Derivatives and integrals, 3–5 Destructive updating model (DUM), 303–304, 306, 308–309, 313 Deterministic processes, 72 Diederich, A., 1, 209 Differential-deficit, psychometric-artifact problem, 360 Differential equations, 4–5 Diffusion models, 35–62 in aging studies, 48 in child development, 48–49 in clinical applications, 49–50, 352 competing two-choice models, 51–56 failure of, 50–51 in homeostatic state manipulations, 49–50 in individual differences studies, 48 in lexical decision, 46–47 optimality, 44–45 in perceptual tasks, 45–46 for practice effects, 301 for rapid decisions, 35–44 accuracy and RT distribution expressions, 38–41 drift rate, 36–38 overview, 35–36 standard two-choice task, 41–44 in recognition memory, 46 in semantic and recognition priming effects, 47 in value-based judgments, 47–48 Diffusion process, 30 Díríchlet-process mixture model, 192–194, 244 Disjunction probability judgment errors, 376–379 Dissociations, in categorization and recognition, 158–159 Distributional models of semantic memory, 239–247 latent semantic analysis, 239–240 moving window models, 240–241 probabilistic topic models, 243–246 random vector models, 241–243 retrieval-based semantics, 246–247
Domain of the function, 1 Donders, Franciscus, 65–66 Donkin, C., 121 Double factorial paradigm, 83 Drift rates accumulator model assumptions about, 321–322 across-trial variability in, 56–57 in perceptual decision making, 36–38, 45, 325–327 Dual process models of recognition, 166 DUM (destructive updating model), 303–304, 306, 308–309, 313 Dynamic attractor networks, 237–239 Dynamic decision models, 219–220. See also Decision-making models Dynamic programming, 104 Dyslexia, 50 Effective sample size (ESS) statistic, 282 EGCM (extended generalized context model) of absolute identification, 126, 143 Eidels, A., 1, 63 Emotional bias, 49 Episodic memory, 165–183 cued recall models of, 173–174 free recall models of, 174–179 future directions, 179 overview, 165–166 recognition memory models, 166–172 context-noise models, 172 global matching models, 167–168 retrieving effectively from memory (REM) model, 168–171 updating consequences, 171–172 Epistemic uncertainty, 210 Error signal, 4–5 ESS (effective sample size) statistic, 282 EUT (expected utility theory), 209, 211 EVL (Expectancy Valence Learning Model), 356–357 Exemplar-based random walk (EBRW) model of absolute identification, 125–126 of categorization and recognition, 142–164 automaticity and perceptual expertise, 148–150
old-new recognition RTs predicted by, 152–157 overview, 142–144 probabilistic feedback to contrast predictions, 150–152 research goals, 157–159 in response times, 144–146 similarity and practice effects, 146–148 in perceptual decision making, 325 Exemplar models of absolute identification, 125–126, 129 Exhaustive processing, 71–72 Expectancy Valence Learning Model (EVL), 356–357 Expectations, 7–8 Expected utility theory (EUT), 209, 211 Experience-based decision making, 215–216 Experimental Psychology (Woodworth), 83 Exploration/exploitation balance experiments in, 100 human varieties of, 110–113 in reinforcement learning, 106 Exponential functions, 2 Extended generalized context model (EGCM) of absolute identification, 126, 143 Eye movements, saccadic, 323–325 False alarm rates, 23 Feature inference, 196–199 Feature integration theory, 87 Feature-list models, 233–234 Fechnerian paradigm, 257–258 Fechner’s law of psychophysics, 307 Feed-forward networks, 235 FEF (frontal eye field), 321, 323 Fermat, Pierre de, 210 Flexibility-to-fit data, of models, 93 fMRI category-relevant dimensions shown by, 158 in clinical psychology, 357–359 context word approach and, 241 diffusion models and, 57 model-based analysis of, 107–108 Free recall models of episodic memory, 174–179 Frequentist methods, 281 Frontal eye field (FEF), 321, 323 Functions, mathematical, 1–3
index
393
Gabor patch orientation discrimination, 50 Galen, 64 Gate accumulator model, 327 Gaussian distribution, 189, 192 Generalizability, measures of, 303 Generalized context model (GCM), 126, 143, 152, 325 General recognition theory (GRT) application of, 14 applied to data, 17–21 empirical example, 24–28 multivariate normal distributions assumed by, 16 neural implementations of, 30–31 overview, 15–16 response accuracy and response time accounted for, 28–30 summary statistics approach, 22–24 GenSim software for semantic memory modeling, 246 Gershman, S. J., 187 Global matching models, 167–168 Go/No-Go Discrimination Task, 44, 349, 356 Goodness of fit evaluation, 20–21, 302 Grice inequality, 74–75, 92 Griffiths, T. L., 187 Grouping, power of, 66 GRT (general recognition theory). See General recognition theory (GRT) Guided search, 89 Gureckis, T. M., 99 HA-LA (higher anxiety-prone-lower anxiety-prone) group differences, 352–353 HAL (Hyperspace Analogue to Language) model, 240–241, 245, 248 Hamilton, Sir William, 66 Hawkins, R. X. D., 63 HDI (highest density interval), 285 Heathcote, A., 121 Hebbian learning, 235 HiDEx software for semantic memory modeling, 246 Hierarchical models, Bayesian estimation in, 279–299 attention allocation differences example, 290–295 data, 291–292
394
index
descriptive model and parameters, 292–293 overview, 290–291 posterior distribution interpretation, 293–295 baseball batting example, 282–290 data, 283 descriptive model and parameters, 283–285 overview, 282–283 posterior distribution interpretation, 285–290 shrinkage and multiple comparisons, 290 comparison of, 295–297 ideas in, 279–282 Higher anxiety-prone-lower anxiety-prone (HA-LA) group differences, 352–353 Highest density interval (HDI), 285 Hilbert space, in quantum theory, 371–372, 374–375 Histograms, 9 Homeostatic state manipulations, 49–50 “Horse race” model of cognitive processes, 356 Hot cognition, 361 Howard, M. W., 165 Human function learning, 202–203 Human information processing, 63–70 Donder’s complication experiment, 66–68 Sternberg’s work in, 68–70 von Helmholtz’s measurement of nerve impulse speed, 64–65 Wundt’s reaction time studies, 65–66 Human neuroscience, diffusion models for, 56–58 Hyperspace Analogue to Language (HAL) model, 240–241, 245, 248 IBP (Indian buffet process) metaphor, 194–195, 197–200, 203 Identification data, fitting GRT to, 18–21 Identification hit rate, 23 Importance sampling for Bayes factor, 312–313 Independence, axioms of, 211–212 Independent parallel, limited-capacity (IPLC) processing system, 353
Independent race model, 74–75 Indian buffet process (IBD) metaphor, 194–195, 197–200, 203 Individual differences studies, diffusion models in, 48 Infinite Relational Model (IRM), 195 Information criteria, in model comparison, 306–307 Instance theory, 301, 325 Institute for Collaborative Biotechnologies, 31 Instrumental conditioning, 101, 111 Integrals and derivatives, 3–5 Integrate-and-fire neurons, 39 Intercompletion time equivalence, 77–78 Intertrial interval, sequential effects and, 136–138 Inverse problem, shape perception as, 256–263 Iowa Gambling Task, 349, 356 IPLC (independent parallel, limited-capacity) processing system, 353 IRM (Infinite Relational Model), 195 James, William, 64 Jefferson, B., 63 Jeffreys weights, 313 Jones, M. N., 232 Kinnebrook, David, 65 Kolmogorov axioms, 307, 370, 373–374 Kruschke, J. K., 279 Kullback-Leibler divergence, 306 Languages, tonal, 134 Latent Díríchlet Allocation algorithms, 244 Latent semantic analysis (LSA), 239–240, 245, 248–249 Law of total probability, 376 LBA (Linear Ballistic Accumulator) model, 52, 301 Leaky competing accumulator (LCA) model, 36, 51–52, 128, 223–225, 327 Learning. See also Computational reinforcement learning absolute identification in, 130–133 associative, 194–196 Hebbian, 235 modeling human function, 202–203
procedural, 30–31 relationships in continuous quantities, 200–203 Lexical decisions, diffusion models in, 46–47 Lexicographic semi-order (LS) choice rule, 213 Li, Y., 255 Likelihood function, 18–19, 280, 296 Likelihood ratio test, 28 Limited capacity, 70, 73 Linear Ballistic Accumulator (LBA) model, 52, 301 Linear functions, 1–2 Linear regression, 8, 200–202 Logan, G. D., 320 Love, B. C., 99 LSA (latent semantic analysis), 239–240, 245, 248–249 LS (lexicographic semi-order) choice rule, 213 Lüder’s rule, 374 “Magical number seven,” 66 Mapping, functions for, 1 Marginal discriminabilities, 23 Marginal response invariance, 22 Markov Chain Monte Carlo (MCMC) algorithms, 244, 281–282, 293 Markov decision process (MDP), 102–104 Markov dynamic model for two-stage gambles, 382–383, 385–387 Maskelyn, Nevil, 65 Matched filter model, 167, 173 Mathematical concepts, review of, 1–10 derivatives and integrals, 3–5 expectations, 7–8 mathematical functions, 1–3 maximum likelihood estimation, 8–9 probability theory, 5–7 Matrix reasoning, 48 Matzke, D., 300 Maximum likelihood estimation (MLE), 8–9, 281, 347 MCMC (Markov Chain Monte Carlo) algorithms, 244, 281–282, 293 MDL (minimum description length), in model comparison, 307–309 MDP (Markov decision process), 102–104 MDS (multidimensional scaling), 143 Mean interaction contrast, 83–84
Measures of generalizability, 303 Memory, 350–352. See also Episodic memory; Semantic memory Memory interference models example, 303–306 Méré, Chevalier de, 210 Meyer, Irwin, Osman, and Kounios partial information paradigm, 42–44 Miller, George, 66 Minimum description length (MDL), in model comparison, 307–309 Minimum-time stopping rule, exhaustive processing versus, 71–72 Minkowski power model, 144 MLE (maximum likelihood estimation), 8–9, 281, 347 Model-based versus model-free learning, 108–109 Modeling. See Parsimony principle in model comparison; specifically named models Model mimicking degenerative, 80 ignoring parallel-serial, 87–90 prediction overlaps from, 75–78 in psychological science, 91–93 Moderate stochastic transitivity (MST), 212–213 Moment matching, in parameter estimation, 347–348 Monte-Carlo methods, 104, 311–313 Movement-related neurons, in FEF, 321, 323, 325–326, 328 Moving window models, 240–241 MPM (multiplicative prototype model), 290, 292 MPTs (multinomial processing tree models). See Multinomial processing tree models (MPTs) MST (moderate stochastic transitivity), 212–213 Müller, Johannes, 64 Multialternative choice problems, decision field theory for, 222–225 Multi-armed bandit tasks, 111 Multi-attribute decision field theory, 221–222 Multichoice-decision-making, 52–53 Multidimensional scaling (MDS), 143, 273
Multidimensional signal detection theory, 13–34 general recognition theory applied to data, 17–21 empirical example, 24–28 neural implementations of, 30–31 overview, 15–16 response accuracy and response time accounted for, 28–30 summary statistics approach, 22–24 multivariate normal model, 16–17 overview, 13–15 Multinomial processing tree models (MPTs), 301, 304–305, 307–311, 350–352 Multiple comparisons, shrinkage and, 290 Multiple linear regression, 8 Multiplicative prototype model (MPM), 290, 292 Multivariate normal model, 16–17 Myopic behavior, of agents, 103 National Institute of Neurological Disorders and Stroke, 31 Natural log functions, 2 NCM (no-conflict model), 303, 305–306, 308–309, 313 Nested models, comparing, 313–314 Neufeld, R. W. J., 341 Neural evidence of computational reinforcement learning, 106–108 of exemplar-based random walk, 158 of GRT, 30–31 in perceptual decision making, 325–330 Neurocognitive modeling of perceptual decision making. See Perceptual decision making, neurocognitive modeling of Neuro-connectionist modeling, 345 Neuroeconomics, 48 Neuroscience, decision making understanding from, 53–58 Newton-Raphson method, 19 Nietzsche, Friedrich, 64 No-conflict model (NCM), 303, 305–306, 308–309, 313 Noise, in perceptual systems, 15, 36
index
395
Nondecision time, 48–50, 57 Nonlinear dynamical system modeling, 345–346 Nonparametric models, 189–192, 194–195 Normal distribution, 7 Nosofsky, R. M., 142 Null list strength effects in (REM) model, 170–171 Numerosity discrimination task, 50 Observations, clustering, 192–196 Occam’s razor, 301–302 One-choice decisions, 53 Operant conditioning, 101 Optimality, 44–45 Optimal planning, 113 Ornstein-Uhlenbeck (OU) diffusion process, 50, 55 Overfitting, 190–191 Palmeri, T. J., 142, 320 Parallelism, 68 Parallel processing in benchmark model, 74–75 mathematics supported by, 77 parallel-serial mimicry ignored, 87–90 partial processing as basis of, 80–81 serial processing versus, 71 Parallel-Serial Tester (PST) paradigm, 82 Parametric models, 189–190 Parsimony principle in model comparison, 300–319 Bayes factors, 309–314 comparison of model comparisons, 314–315 information criteria, 306–307 memory interference models example, 303–306 minimum description length, 307–309 overview, 300–303 Partial information paradigm, 42–44 Pascal, Blaise, 210 Pathocognition, 357–359 Pavlovian conditioning, 195 PBRW (prototype-based random walk) model, 151–152 Perceptual decision making, neurocognitive modeling of, 320–340 architectures for, 327–328 conclusions, 333–336 control over, 330–333 neural dynamics, predictions of, 328–330
396
index
neural locus of drift rates, 325–327 overview, 320–323 saccadic eye movements and, 323–325 Perceptual expertise, automaticity and, 148–150 Perceptual independence, 15–16, 23 Perceptual judgment, 121–141 absolute identification issues, 129–139 absolute and relative judgment, 129–130 absolute identification versus perfect pitch, 133–135 intertrial interval and sequential effects, 136–138 learning, 130–133 response times, 135 absolute identification theories, 124–129 benchmark phenomena, 122–124 overview, 121–122 Perceptual separability, 15, 22f Perceptual tasks, diffusion models in, 36, 45–46 Perceptual units, features as, 196–200 Perfect pitch, absolute identification versus, 133–135 Perspective. See Shape perception Pizlo, Z., 255 Pleskac, T. J., 209 Poisson counter model, 36, 55 Poisson shot noise process, 55 Policies, in decision making, 103–104 Polynomial regression, 302–303 Posterior distribution in attention allocation differences, 293–295 in baseball batting example, 285–290 Monte Carlo sampling for, 311–313 in tests for model-parameter differences, 356 Pothos, E., 369 Power functions, 2 Power law, 300 Practice effects, 146–148, 300–301 Prediction error, 105, 107–108 Principles of Psychology (James), 64 Probabilistic topic models, 243–246, 248, 250 Probability density function, 72
Probability judgment error, 377–379 Probability mass function, 7 Probability theory, 5–7, 373–377. See also Bayesian models; Decision-making models Probability weighting function, 214–216 Problem of Points, 210 Procedural learning, 30–31 Prospect theory, 209, 214, 216–219 Prototype-based random walk (PBRW) model, 151–152 Prototype models, 142 PST (Parallel-Serial Tester) paradigm, 82 Psychology, mathematical and computational modeling in. See Clinical psychology, mathematical and computational modeling in Psychomotor vigilance task (PVT), 53 Q-learning, 104–106, 109, 111 Quadratic functions, 2 Quantile-probability plots, 40 Quantum models of cognition and decision, 369–389 classical probabilities versus, 373–377 concepts, definitions, and notation, 371–373 decision making applications, 381–387 Markov dynamic model for two-stage gambles, 382–383 model comparisons, 385–387 quantum dynamic model for two-stage gambles, 384–385 two-stage gambling paradigm, 381–382 dynamical principles, 379–381 probability judgment error applications, 377–379 reasons for, 369–371 Race model inequality, 74 Rae, B., 121 Random Permutations Model (RPM), 244 Random variables with continuous distribution, 7–8 Random vector models, 241–243 Random walk, 39. See also Exemplar-based random walk (EBRW) model
Range of the function, 1 Rank-dependent utility theory, 209 Rapid decisions. See Diffusion models Ratanalysis, 188 Ratcliff, R., 35 Ratcliff ’s diffusion model, 301 Rational choice models, 47, 214–219 Reaction time distributions, 83–87 Recognition and categorization. See Exemplar-based random walk (EBRW) model Recognition memory models, 46, 166–172 Region of practical equivalence (ROPE), in decision rules, 286 Regularization methods, in shape perception, 258–260 Reinforcement learning (RL). See Computational reinforcement learning Relative judgment models of absolute identification, 126–127, 129t, 136 Release-from-inhibition model, 50–51 REM (retrieving effectively from memory) model. See Retrieving effectively from memory (REM) model Rescorla-Wagner model, 111, 194 Response accuracy, 28–30, 90–91 Response signal tasks, 41–42 Response times (RT) absolute identification and, 128, 135 cognitive-psychological complementarity, 87–90 in diffusion models, 38–41 example of, 78–79 exemplar-based random walk model of, 144–146, 152–157 GRT to account for, 28–30 human information processing studied by, 63–70 Donder’s complication experiment, 66–68 Sternberg’s work in, 68–70 von Helmholtz’s measurement of nerve impulse speed, 64–65 Wundt’s reaction time studies, 65–66 metatheory expansion to encompass accuracy, 90–91 model mimicking, 75–78, 91–93
quantitative expressions of, 70–75 stopping rule distinctions based on set-size functions, 82–87 theoretical distinctions, 79–82 Restricted capacity models of absolute identification, 127–128, 129t, 136 Retrieval-based semantics, 246–247 Retrieved context models, 177–178 Retrieving effectively from memory (REM) model consequences of updating in, 171–172 overview, 168–170 word frequency and null list strength effects in, 170–171 Reward prediction error hypothesis, 107 Reward-rate optimality, 45 Reward varieties, in reinforcement learning, 113–114 Rickard’s component power laws model, 301 Risk in decision making. See Decision-making models RL (reinforcement learning). See Computational reinforcement learning ROPE (region of practical equivalence), in decision rules, 286 RPM (Random Permutations Model), 244 RT (response times). See Response times (RT) RT-distance hypothesis, 29, 150 Rule-plus-exception models, 142 Rumelhart networks, 235–237 Saccadic eye movements, 323–325 SAMBA (Selective Attention, Mapping, and Ballistic Accumulators) model of absolute identification, 128–129, 133, 135–138 Sampling independence test, 24 Savage-Dickey approximation to Bayes factor, 313–314 Sawada, T., 255 SBME (strength-based mirror effect), 171–172 Schall, J. D., 320 Schizophrenia, stimulus-encoding elongation in, 357–359 SCM (similarity-choice model), 21
SD (social desirability) contamination of scores, 361 Second-order conditioning, 102–103, 111 Selective Attention, Mapping, and Ballistic Accumulators (SAMBA) model of absolute identification, 128–129, 133, 135–138 Selective influence, 85 Semantic and recognition priming effects, 47 Semantic memory, 232–254 compositional semantics, 249 connectionist models of, 234–239 distributional models of, 239–247 latent semantic analysis, 239–240 moving window models, 240–241 probabilistic topic models, 243–246 random vector models, 241–243 retrieval-based semantics, 246–247 future directions, 249–250 grounding semantic models, 247–249 overview, 232–233 research models and themes, 233–234 Semantic networks, 233–234 SEMMOD software for semantic memory modeling, 246 Sensory preconditioning, 194–195 Sequential effects, 123–124, 136–138 Sequential-sampling models. See Diffusion models Serial processing mathematics supported by, 76–77 parallel processing versus, 71 parallel-serial mimicry ignored, 87–90 parallel-serial testing paradigm, 82 SEUT (subjective expected utility theory), 209, 211 SFT (Systems Factorial Technology), 354–355 Shape perception, 255–276 constancy, 256–257 constraints in regularization and Bayesian methods, 258–260
index
397
Shape perception (Cont.) eye and camera geometry, 260–263 Fechnerian paradigm inadequacy, 257–258 new definition of, 273–274 perspective and orthographic projection, 263–265 3D mirror-symmetrical shape recovery from 2D images, 269–273 3D symmetry and 2D orthographic and perspective projections, 265–269 uniqueness, 255–256 Shrinkage and multiple comparisons, 286–288, 290 Sichuan University, 134 Signal detection theory. See Multidimensional signal detection theory Sign-dependent utility theory, 209 Similarity as context effect in DFT, 225–226 kernels of, 201 practice effects and, 146–148 as search determinant, 89 similarity-choice model (SCM), 21 Single cell recording data, 54 Sleep deprivation, 50 Social desirability (SD) contamination of scores, 361 Soto, F. A., 13 Span of attention, 66 Spatial models, 233–234 Speed-accuracy tradeoff, 90–91 Speeded classification, 29, 146–148 Speeded visual search, 89 Sperling, George, 66 S-Space software for semantic memory modeling, 246 SST (strong stochastic transitivity), 212–213 State representation influence, in reinforcement learning, 109–110 States, in Markov decision process (MDP), 102–103 Sternberg, Saul, 68 Steven’s law of psychophysics, 307 Stimulus dimensionality, 133 Stimulus-response learning, 101 Stochastic difference equation, 5
398
index
Stochastic dominance, 217–218, 221–222 Stochastic independence, 72 Stochastic transitivity, 212–213 Stopping rule exhaustive processing versus, 71–72 set-size functions and, 82–87 Stop signal paradigm, 330, 332–333 St. Petersburg paradox, 210–211 Strength-based mirror effect (SBME), 171–172 Strong inference tactic, 92 Strong stochastic transitivity (SST), 212–213 Structural MRI, 57 Subjective expected utility theory (SEUT), 209, 211 Subtraction, method of, 66–69 Summary statistics approach to GRT, 22–24 Super capacity, 73 SuperMatrix software for semantic memory modeling, 246 Supertaskers, 75 Survivor interaction contrast, 84–85 Symmetry. See Shape perception Systematic exploration, 113 Systems Factorial Technology (SFT), 354–355 TAX (transfer of attention exchange) model, 219 Temporal Context Model (TCM), 244 Temporal difference learning, 104–105, 109, 111 Tenenbaum, J. B., 187 Test of English as a Foreign Language (TOEFL), 240 Theorizing process, 1 Threat sensitivity modeling in anxiety-prone individuals, 352–354 3D symmetry. See Shape perception Thurstonian models of absolute identification, 124–125, 129, 136 Time-varying processing, 44 TOEFL (Test of English as a Foreign Language), 240 Tolman, Edwin, 93 Tonal languages, 134 Topic models, 243–246, 248, 250 Total probability, law of, 375–376 Townsend, J. T., 1, 63 Townsend’s capacity reallocation, 69
Transfer of attention exchange (TAX) model, 219 Transformations, perspective projection as, 261 Transition probabilities, in Markov decision process (MDP), 102 Transitivity, axioms of, 211–214 Trial independent “random” exploration, 112–113 Trial-to-trial variability, 37 Trigonometric functions, 2–3 2D orthographic and perspective projections. See Shape perception Two-choice models, diffusion models versus, 51–56 Two-choice tasks, 41–44 Two-stage gambles Markov dynamic model for, 382–383 model comparisons for, 385–387 overview, 381–382 quantum dynamic model for, 384–385 Uncertainty, 281. See also Decision-making models Uniqueness, in shape perception, 255–256 Unlimited capacity and independent, parallel processing channels (UCIP), 74–75 U.S. Army Research Office, 31 Utility function, 211, 216–217 Value-based judgments, diffusion models in, 47–48 Vandekerckhove, J., 300 Vanpaemel, W., 279 Venn diagrams, 6 Veridical perception, 255, 258 Vickers accumulator model, 36 Visually responsive neurons, in FEF, 321, 323, 325–326 Visual search experiments, 69 Visual short-term memory (VSTM), 45 Vitalism, 64 von Helmholtz, Hermann, 64 von Neumann axioms, 371, 373 VSTM (visual short-term memory), 45 Wagenmakers, E.-J., 300 WAIS vocabulary, 48 Wallsten, T. S., 209 Wang, Z., 1, 369
Weak stochastic transitivity (WST), 212–213 Weighted additive utility model, 220 Wickens, T. D., 14, 19 William of Occam, 302
Willits, J., 232 Wisconsin Card Sorting Test, 349, 356–357 Woodworth, R. S., 63–64 Word frequency effects in (REM) model, 170–171
Word recognition, 47 Word-Similarity software for semantic memory modeling, 246 Workload capacity, 72–74, 85–87 Wundt, Wilhem, 65
index
399
