Learning Patterns from Visual Sequences

examines factors contributing to our learning of non-adjacent statistical patterns from visual sequences (click to read more)

Many events that humans and other organisms experience involve temporally ordered sequences. These include visual events, such as watching agents engaging in actions, and machines carrying out functions, as well as auditory events, such as hearing a sequence of words in a spoken sentence, or sounds within words, or even notes in a piece of music. In many cases, these events contain regularities in which certain elements within an event predict certain others. For example, in the action of hammering a nail, the agent first moves the hammer away from the nail, and then forcefully brings the hammer into contact with the nail. In the English present progressive, the copula, is, is followed by a verb with the inflection -ing, for example, …is baking…. Through experience, individuals learn about aspects of these regularities, and, once noticed, can use them to generate new knowledge, either explicit, such as the understanding of an artifact’s function, or implicit, such as the knowledge of the grammatical rules of one’s native language(s).

Substantial areas of cognitive development are devoted to understanding the processes by which experience leads to knowledge, and how these processes may be guided by more specialized or more general learning mechanisms. This project is part of an endeavor to understand the very first steps of these processes. It address the questions: what kinds of regularities do infants detect when they visually experience temporally sequenced events? How do they generalize those regularities and use those generalizations to make predictions about other events? The answers to these questions are important for constraining theories of cognitive development, as they provide evidence about the kinds of representations infants have available as the input to further learning.

Specifically, this project examines whether dynamic actions (seeing an action carried out in real time vs. seeing slices of the static endpoints of the action) and human agents (whether the action is performed by a human vs. an object) contribute to infants’ and adults’ visual statistical patterns. You can find related publications in my publications page.

TL;DR, Explain like I'm a baby!

You see a caregiver pull out the milk bottle, you know you are in for a delicious treat. Your caregiver might be doing any number of things in between---scratching their heads, looking on their phones, tapping on the milk bottle to check for temperature, etc., but that doesn't matter. You know too well the beginning and end of this sequence of actions. Congratulations! You've discovered your first nonadjacent dependency, items that frequently co-occur but are not next to each other sequentially. Congrats!

These patterns are typically harder to learn than if the two items are right next to each other (duh. duh.)

In this project, we looked at what may help the learning of such patterns. We found that seeing a human action played out in real time (as opposed to slices of images) aids nonadjacent dependency learning for both grownups and babies.