MITs Computer Science and Artificial Intelligence Lab has devised a route to look inside neural network and shed some light on how theyre actually making decisions. The new process is a fully automated version of the organizations of the system the research squad behind it presented two years ago, which applied human reviewers to achieve the same ends.
Coming up with a method that can provide similar outcomes without human review could be a significant step towards helping us understand why neural networks that perform well are able do succeed as well as they do. Current deep learning techniques leave a lot of questions around how systems actually arrive at their results the networks hire successive layers of signal processing to categorize objects, translate text, or perform other functions, but we have very little means of gaining insight into how each layer of the network is doing its actual decision-making.
The MIT CSAIL teams system use doctored neural nets that report back the strength with which every individual node responds to a given input image, and those images that produce the strongest answer are then investigated. This analysis was originally performed by Mechanical Turk workers, who would catalogue each based on specific visual conceptions found in the images, but now that work has been automated, so that the category is machine-generated.
Already, the research is providing interesting insight into how neural nets operate, for example showing that a network trained to add color to black and white images aims up concentrating a significant portion of its nodes to identifying textures in the pictures. It also found that networks trained to identify objects in video dedicated many of their nodes to scene identification, while networks trained to identify scenes do exactly the opposite, committing many nodes to ID-ing objects.
Because we dont fully understand how humans suppose, categorize and distinguish information, either, and neural nets are based on hypothetical models of human thought, the research from this CSAIL team could eventually shed light on questions in neuroscience, too. The paper will be presented at this years Computer Vision and Pattern Recognition conference, and should provoke plenty of interest from the artificial intelligence research community.
Make sure to visit: CapGeneration.com