The Open Mind

Cogito Ergo Sum

Posts Tagged ‘Perception

The Brain as a Prediction Machine

leave a comment »

Over the last year, I’ve been reading a lot about Karl Friston and Andy Clark’s work on the concept of perception and action being mediated by a neurological schema centered on “predictive coding”, what Friston calls “active inference”, the “free energy principle”, and Bayesian inference in general as it applies to neuro-scientific models of perception, attention, and action.  Here’s a few links (Friston here; Clark here, here, and here) to some of their work as it is worth reading for those interested in neural modeling, information theory, and learning more about meta-theories pertaining to how the brain integrates and processes information.

I find it fascinating how this newer research and these concepts relate to and help to bring together some content from several of my previous blog posts, in particular, those that mention the concept of hierarchical neurological hardware and those that mention my own definition of knowledge “as recognized causal patterns that allow us to make successful predictions.”  For those that may be interested, here’s a list of posts I’ve made over the last few years that I think contain some relevant content (in chronological order).

The ideas formulated by Friston and expanded on by Clark center around the brain being (in large part) a prediction generating machine.  This fits in line with my own conclusions about what the brain seems to be doing when it’s acquiring knowledge over time (however limited my reading is on the subject).  Here’s an image of the basic predictive processing schema:

PPschema

The basic Predictive Processing schema (adapted from Lupyan and Clark (2014))

One key element in Friston and Clark’s work (among the work of some others) is the amalgamation of perception and action.  In this framework, perception itself is simply the result of the brain’s highest level predictions of incoming sensory data.  But also important in this framework is that prediction error minimization is accomplished through embodiment itself.  That is to say, their models posit that the brain not only tries to reduce prediction errors by updating its prediction models based on the actual incoming sensory information (with only the error feeding forward to update the models, similar to data compression schema), but the concept of active inference involves the minimization of prediction error through the use of motor outputs.  This could be taken to mean that motor outputs themselves are, in a sense, caused by the brain trying to reduce prediction errors pertaining to predicted sensory input — specifically sensory input that we would say stems from our desires and goals (e.g. desire to fulfill hunger, commuting to work, opening the car door, etc.).

To give a simple example of this model in action, let’s consider an apple resting on a table in front of me.  If I see the apple in front of me and I have a desire to grab it, my brain would not only predict what that apple looks like and how it is perceived over time (and how my arm looks while reaching for it), but it would also predict what it should feel like to reach for the apple.  So if I reach for it based on the somato-sensory prediction and there is some error in that prediction, corroborated by my visual cortex observing my arm moving in some wrong direction, the brain would respond by updating its models that predict what it should feel so that my arm starts moving in the proper direction.  This prediction error minimization is then fine-tuned as I get closer to the apple and can finally grab it.

This embodiment ingrained in the predictive processing models of Friston and Clark can also be well exemplified by the so-called “Outfielder’s Problem”.  In this problem, an outfielder is trying to catch a fly ball.  Now we know that outfielders are highly skilled at doing this rather effectively.  But if we ask the outfielder to merely stand still and watch a batted ball and predict where it will land, their accuracy is generally pretty bad.  So when we think about what strategy the brain takes to accomplish this when moving the body quickly, we begin to see the relevance of active inference and embodiment in the brain’s prediction schema.  The outfielder’s brain employs a brilliant strategy called “optical acceleration cancellation” (OAC).  Here, the well-trained outfielder sees the fly ball, and moves his or her body (while watching the ball) in order to cancel out any optical acceleration observed during the ball’s flight.  If they do this, then they will end up exactly where the ball was going to land, and then they’re able to catch it successfully.

We can imagine fine-grained examples of this active inference during everyday tasks, where I may simply be looking at a picture on my living room wall, and when my brain is predicting how it will look over the span of a few seconds, my head may slightly change its tilt, or direction, or my eyes may slowly move a fraction of a degree this way or that way, however imperceptible to me.  My brain in this case is predicting what the picture on the wall will look like over time and this prediction (according to my understanding of Clark) is identical to what we actually perceive.  One key thing to note here is that the prediction models are not simply updated based on the prediction error that is fed forward through the brain’s neurological hierarchies, but it is also getting some “help” from various motor movements to correct for the errors through action, rather than simply freezing all my muscles and updating the model itself (which may in fact be far less economical for the brain to do).

Another area of research that pertains to this framework, including ways of testing its validity, is that of evolutionary psychology and biology, where one would surmise (if these models are correct) that evolution likely provided our brains with certain hard-wired predictive models and our learning processes over time use these as starting points to produce innate reflexes (such as infant suckling to give a simple example) that allow us to survive long enough to update our models with actual new acquired information.  There are many different facets to this framework and I look forward to reading more about Friston and Clark’s work over the next few years.  I have a feeling that they have hit on something big, something that will help to answer a lot of questions about embodied cognition, perception, and even consciousness itself.

I encourage you to check out the links I provided pertaining to Friston and Clark’s work, to get a taste of the brilliant ideas they’ve been working on.

Advertisements

Conscious Realism & The Interface Theory of Perception

with 6 comments

A few months ago I was reading an interesting article in The Atlantic about Donald Hoffman’s Interface Theory of Perception.  As a person highly interested in consciousness studies, cognitive science, and the mind-body problem, I found the basic concepts of his theory quite fascinating.  What was most interesting to me was the counter-intuitive connection between evolution and perception that Hoffman has proposed.  Now it is certainly reasonable and intuitive to assume that evolutionary natural selection would favor perceptions that are closer to “the truth” or closer to the objective reality that exists independent of our minds, simply because of the idea that perceptions that are more accurate will be more likely to lead to survival than perceptions that are not accurate.  As an example, if I were to perceive lions as inert objects like trees, I would be more likely to be naturally selected against and eaten by a lion when compared to one who perceives lions as a mobile predator that could kill them.

While this is intuitive and reasonable to some degree, what Hoffman actually shows, using evolutionary game theory, is that with respect to organisms with comparable complexity, those with perceptions that are closer to reality are never going to be selected for nearly as much as those with perceptions that are tuned to fitness instead.  More so, truth in this case will be driven to extinction when it is up against perceptual models that are tuned to fitness.  That is to say, evolution will select for organisms that perceive the world in a way that is less accurate (in terms of the underlying reality) as long as the perception is tuned for survival benefits.  The bottom line is that given some specific level of complexity, it is more costly to process more information (costing more time and resources), and so if a “heuristic” method for perception can evolve instead, one that “hides” all the complex information underlying reality and instead provides us with a species-specific guide to adaptive behavior, that will always be the preferred choice.

To see this point more clearly, let’s consider an example.  Let’s imagine there’s an animal that regularly eats some kind of insect, such as a beetle, but it needs to eat a particular sized beetle or else it has a relatively high probability of eating the wrong kind of beetle (and we can assume that the “wrong” kind of beetle would be deadly to eat).  Now let’s imagine two possible types of evolved perception: it could have really accurate perceptions about the various sizes of beetles that it encounters so it can distinguish many different sizes from one another (and then choose the proper size range to eat), or it could evolve less accurate perceptions such that all beetles that are either too small or too large appear as indistinguishable from one another (maybe all the wrong-sized beetles whether too large or too small look like indistinguishable red-colored blobs) and perhaps all the beetles that are in the ideal size range for eating appear as green-colored blobs (that are again, indistinguishable from one another).  So the only discrimination in this latter case of perception is between red and green colored blobs.

Both types of perception would solve the problem of which beetles to eat or not eat, but the latter type (even if much less accurate) would bestow a fitness advantage over the former type, by allowing the animal to process much less information about the environment by not focusing on relatively useless information (like specific beetle size).  In this case, with beetle size as the only variable under consideration for survival, evolution would select for the organism that knows less total information about beetle size, as long as it knows what is most important about distinguishing the edible beetles from the poisonous beetles.  Now we can imagine that in some cases, the fitness function could align with the true structure of reality, but this is not what we ever expect to see generically in the world.  At best we may see some kind of overlap between the two but if there doesn’t have to be any then truth will go extinct.

Perception is Analogous to a Desktop Computer Interface

Hoffman analogizes this concept of a “perception interface” with the desktop interface of a personal computer.  When we see icons of folders on the desktop and drag one of those icons to the trash bin, we shouldn’t take that interface literally, because there isn’t literally a folder being moved to a literal trash bin but rather it is simply an interface that hides most if not all of what is really going on in the background — all those various diodes, resistors and transistors that are manipulated in order to modify stored information that is represented in binary code.

The desktop interface ultimately provides us with an easy and intuitive way of accomplishing these various information processing tasks because trying to do so in the most “truthful” way — by literally manually manipulating every diode, resistor, and transistor to accomplish the same task — would be far more cumbersome and less effective than using the interface.  Therefore the interface, by hiding this truth from us, allows us to “navigate” through that computational world with more fitness.  In this case, having more fitness simply means being able to accomplish information processing goals more easily, with less resources, etc.

Hoffman goes on to say that even though we shouldn’t take the desktop interface literally, obviously we should still take it seriously, because moving that folder to the trash bin can have direct implications on our lives, by potentially destroying months worth of valuable work on a manuscript that is contained in that folder.  Likewise we should take our perceptions seriously, even if we don’t take them literally.  We know that stepping in front of a moving train will likely end our conscious experience even if it is for causal reasons that we have no epistemic access to via our perception, given the species-specific “desktop interface” that evolution has endowed us with.

Relevance to the Mind-body Problem

The crucial point with this analogy is the fact that if our knowledge was confined to the desktop interface of the computer, we’d never be able to ascertain the underlying reality of the “computer”, because all that information that we don’t need to know about that underlying reality is hidden from us.  The same would apply to our perception, where it would be epistemically isolated from the underlying objective reality that exists.  I want to add to this point that even though it appears that we have found the underlying guts of our consciousness, i.e., the findings in neuroscience, it would be mistaken to think that this approach will conclusively answer the mind-body problem because the interface that we’ve used to discover our brains’ underlying neurobiology is still the “desktop” interface.

So while we may think we’ve found the underlying guts of “the computer”, this is far from certain, given the possibility of and support for this theory.  This may end up being the reason why many philosophers claim there is a “hard problem” of consciousness and one that can’t be solved.  It could be that we simply are stuck in the desktop interface and there’s no way to find out about the underlying reality that gives rise to that interface.  All we can do is maximize our knowledge of the interface itself and that would be our epistemic boundary.

Predictions of the Theory

Now if this was just a fancy idea put forward by Hoffman, that would be interesting in its own right, but the fact that it is supported by evolutionary game theory and genetic algorithm simulations shows that the theory is more than plausible.  Even better, the theory is actually a scientific theory (and not just a hypothesis), because it has made falsifiable predictions as well.  It predicts that “each species has its own interface (with some similarities between phylogenetically related species), almost surely no interface performs reconstructions (read the second link for more details on this), each interface is tailored to guide adaptive behavior in the relevant niche, much of the competition between and within species exploits strengths and limitations of interfaces, and such competition can lead to arms races between interfaces that critically influence their adaptive evolution.”  The theory predicts that interfaces are essential to understanding evolution and the competition between organisms, whereas the reconstruction theory makes such understanding impossible.  Thus, evidence of interfaces should be widespread throughout nature.

In his paper, he mentions the Jewel beetle as a case in point.  This beetle has a perceptual category, desirable females, which works well in its niche, and it uses it to choose larger females because they are the best mates.  According to the reconstructionist thesis, the male’s perception of desirable females should incorporate a statistical estimate of the true sizes of the most fertile females, but it doesn’t do this.  Instead, it has a category based on “bigger is better” and although this bestows a high fitness behavior for the male beetle in its evolutionary niche, if it comes into contact with a “stubbie” beer bottle, it falls into an infinite loop by being drawn to this supernormal stimuli since it is smooth, brown, and extremely large.  We can see that the “bigger is better” perceptual category relies on less information about the true nature of reality and instead chooses an “informational shortcut”.  The evidence of supernormal stimuli which have been found with many species further supports the theory and is evidence against the reconstructionist claim that perceptual categories estimate the statistical structure of the world.

More on Conscious Realism (Consciousness is all there is?)

This last link provided here shows the mathematical formalism of Hoffman’s conscious realist theory as proved by Chetan Prakash.  It contains a thorough explanation of the conscious realist theory (which goes above and beyond the interface theory of perception) and it also provides answers to common objections put forward by other scientists and philosophers on this theory.