 |
|
|
|
Computer Model Mimicks How Brain Recognizes Street ScenesScience Daily — At last, neuroscience is having an impact on computer science and artificial intelligence (AI). For the first time, scientists in Tomaso Poggio's laboratory at the McGovern Institute for Brain Research at MIT applied a computational model of how the brain processes visual information to a complex, real world task: recognizing the objects in a busy street scene. The researchers were pleasantly surprised at the power of this new approach.  The Poggio model for object recognition takes as input the unlabled images of digital photographs from the Street Scene Database (top) and generates automatic annotations of the type shown in the bottom row. The orange bounding boxes are for pedestrians ('ped') and cars ('car'). The system would have also detected bicycles if present. For sky, buildings, trees, and road, the system uses color coding (blue, brown, green, and grey). Note the false detection in the image on the right. A construction sign was mistaken for a pedestrian. (Graphic courtesy Stanley Bileschi, Ph.D., McGovern Institute for Brain Research at MIT) "People have been talking about computers imitating the brain for a long time," said Poggio, who is also the Eugene McDermott Professor in the Department of Brain and Cognitive Sciences and the co-director of the Center for Biological and Computational Learning at MIT. "That was Alan Turing's original motivation in the 1940s. But in the last 50 years, computer science and AI have developed independently of neuroscience. Our work is biologically inspired computer science." "We developed a model of the visual system that was meant to be useful for neuroscientists in designing and interpreting experiments, but that also could be used for computer science," said Thomas Serre, a former PhD student and now a post-doctoral researcher in Poggio's lab and lead author a paper about the street scene application in the 2007 IEEE Transactions on Pattern Analysis and Machine Intelligence. "We chose street scene recognition as an example because it has a restricted set of object categories, and it has practical social applications." Near-term applications include surveillance and automobile driver's assistance, and eventually visual search engines, biomedical imaging analysis, robots with realistic vision. On the neuroscience end, this research is essential for designing augmented sensory prostheses, such as one that could replicate the computations carried by damaged nerves from the retina. "And once you have a good model of how the human brain works," Serre explained, "you can break it to mimic a brain disorder." One brain disorder that involves distortions in visual perception is schizophrenia, but nobody understands the neurobiological basis for those distortions. "The versatility of the biological model turns computer vision from a trick into something really useful," said co-author Stanley Bileschi, a post-doctoral researcher in the Poggio lab. He and co-author Lior Wolf, a former post-doctoral associate who is now on the faculty of the Computer Science Department at Tel-Aviv University, are working with the MIT entrepreneur office, the Deshpande Center in the Sloan School. This center helps MIT students and professors bridge the gap between an intriguing idea or technology and a commercially viable concept. Recognizing Scenes The IEEE paper describes how the team "showed" the model randomly selected images so that it could "learn" to identify commonly occurring features in real-word objects, such as trees, cars, and people. In so-called supervised training sessions, the model used those features to label by category the varied examples of objects found in digital photographs of street scenes: buildings, cars, motorcycles, airplanes, faces, pedestrians, roads, skies, trees, and leaves. The photographs derive from a Street Scene Database compiled by Bileschi. Compared to traditional computer-vision systems, the biological model was surprisingly versatile. Traditional systems are engineered for specific object classes. For instance, systems engineered to detect faces or recognize textures are poor at detecting cars. In the biological model, the same algorithm can learn to detect widely different types of objects. To test the model, the team presented full street scenes consisting of previously unseen examples from the Street Scene Database. The model scanned the scene and, based on its supervised training, recognized the objects in the scene. The upshot is that the model learned from examples, which, according to Poggio, is a hallmark of artificial intelligence. Modeling Object Recognition Teaching a computer how to recognize objects has been exceedingly difficult because a computer model has two paradoxical goals. It needs to create a representation for a particular object that is very specific, such as a horse as opposed to a cow or a unicorn. At the same time the representation must be sufficiently "invariant" so as to discard meaningless changes in pose, illumination, size, position, and many other variations in appearances. Even a child's brain handles these contradictory tasks easily in rapid object recognition. Pixel-like information enters from the retina and passes in a fast feed-forward, bottom-up sweep through the hierarchical architecture of the visual cortex. What makes the Poggio lab's model so innovative and powerful is that, computationally speaking, it mimics the brain's own hierarchy. Specifically, the "layers" within the model replicate the way neurons process input and output stimuli -- according to neural recordings in physiological labs. Like the brain, the model alternates several times between computations that help build an object representation that is increasingly invariant to changes in appearances of an object in the visual field and computations that help build an object representation that is increasingly complex and specific to a given object. The model's success validates work in physiology labs that have measured the tuning properties of neurons throughout visual cortex. By necessity, most of those experiments are made with simplistic artificial stimuli, such as gratings, bars, and line drawings that bear little resemblance to real-world images. "We put together a system that mimics as closely as possible how cortical cells respond to simple stimuli like the ones that are used in the physiology lab," said Serre. "The fact that this system seems to work on realistic street scene images is a concept proof that the activity of neurons as measured in the lab is sufficient to explain how brains can perform complex recognition tasks." Making it More Useful The model used in the street scene application mimics only the computations the brain uses for rapid object recognition. The lab is now elaborating the model to include the brain's feedback loops from the cognitive centers. This slower form of object recognition provides time for context and reflection, such as: if I see a car, it must be on the road not in the sky. Giving the model the ability to recognize such semantic features will empower it for broader applications, including managing seemingly insurmountable amounts of data, work tasks, or even email. The team is also working on a model for recognizing motions and actions, such as walking or talking, which could be used to filter videos for anomalous behaviors -- or for smarter movie editing. The Street Scene database is freely available at Center for Biological and Computational Learning website (http://cbcl.mit.edu/). Co-author Maximilian Riesenbuber began the work on the model for visual recognition for his Ph.D. dissertation in Poggio's lab and continues this work as assistant professor of neuroscience at the Georgetown University Medical Center. This research was partially funded by the U.S. Defense Advanced Research Projects Agency (DARPA), U.S. Office of Naval Research, and the U.S. National Science Foundation-National Institutes of Health. Note: This story has been adapted from a news release issued by McGovern Institute for Brain Research. New! Search Science Daily or the entire web with Google: |
Health Videos & Features
Attention Deficit Hyperactivity Disorder: Myths and Facts
 ADHD information on Understanding ADHD, find information on attention deficit disorder, hyperactivity disorder, alternative medication and treatments for hyperactivity in children. Looking for concerta information? Click Here
Related News SectionsRelated News Topics> more ...
Related Science Stories> more ...
Related Encyclopedia Articles> more ...
Related Book ReviewsStorage Limits On Our Visual Hard Drive (April 15, 2004) -- The amount of information we can remember from a visual scene is extremely limited and the source of that limit may lie in the posterior parietal cortex, a region of the brain involved in visual ... > full story
Rutgers Researcher Finds Visual Memory Is Better Than Previously Thought (July 26, 2001) -- Why is it that you can park your car at a huge mall and find it a few hours later without much problem, or make your way through a store you have never been to before? The answer may lie in our ... > full story In The Mind's Eye: How The Brain Makes A Whole Out Of Parts (January 26, 2006) -- When a human looks at a number, letter or other shape, neurons in various areas of the brain's visual center respond to different components of that shape, almost instantaneously fitting them ... > full story
Computer Vision Study Links How Brain Recognizes Faces, Moods (July 3, 2003) -- The human brain combines motion and shape information to recognize faces and facial expressions, a new study suggests. That new finding, part of an engineer's quest to design computers that "see" ... > full story Discovery Shows How Brain "Fills In Blanks" To Help Us See (June 2, 2000) -- Researchers at the University of Toronto have discovered how the brain helps us see and interact with objects by filling in missing information, a discovery that could have implications for artifical ... > full story Working Memory Retains Visual Details Despite Distractions (January 20, 2006) -- The ability to retain memory about the details of a natural scene is unaffected by the distraction of another activity and this information is retained in "working memory" according to a study ... > full story 'Where Are My Glasses?' -- Study Reveals Clues To The Mechanism Of Short-term Memory (February 20, 2005) -- Understanding the biology of memory is a major goal of contemporary neuroscientists. Short-term or "working" memory is an important process that enables us to interact in meaningful ways with others ... > full story Functional neuroimaging -- Functional neuroimaging is the use of neuroimaging technology to measure an aspect of brain function, often with a view to understanding the relationship between activity in certain brain areas and ... > full article Motion perception -- Motion perception is the process of inferring the speed and direction of objects that move in a visual scene given some visual input. While this process appears straighforward to most observers, it ... > full article Computer vision -- Computer vision is the science and technology of machines that see. As a scientific discipline, computer vision is concerned with the theory and technology for building artificial systems that obtain ... > full article Computational neuroscience -- Computational Neuroscience is an interdisciplinary science that links the diverse fields of neuroscience, computer science, physics and applied mathematics together. It serves as the primary ... > full article Neuroscience -- Neuroscience is a field of study that deals with the structure, function, development, genetics, biochemistry, physiology, pharmacology, and pathology of the nervous system, divided into the central ... > full article Peripheral vision -- Peripheral vision is a part of vision that occurs outside the very center of gaze. There is in actuality a very broad set of non-central points in the field of view that is included in the notion of ... > full article Cognitive science -- Cognitive science is usually defined as the scientific study either of mind or of intelligence. Practically every formal introduction to cognitive science stresses that it is a highly ... > full article Neuropsychology -- Neuropsychology is a branch of psychology and neurology that aims to understand how the structure and function of the brain relate to specific psychological processes. It is scientific in its ... > full article Psycholinguistics -- Psycholinguistics or psychology of language is the study of the psychological and neurobiological factors that enable humans to acquire, use, and understand language. Initial forays into ... > full article Cognitive neuroscience -- The field of cognitive neuroscience concerns the scientific study of the neural mechanisms underlying cognition and is a branch of neuroscience. Cognitive neuroscience overlaps with cognitive ... > full article
|
||||||||||||||||||||
