Reinforcement learning is a technique, common in computer science, in which a computer system learns how best to solve some problem through trial-and-error. Classic applications of reinforcement learning involve problems as diverse as robot navigation, network administration and automated surveillance.

At the Association for Uncertainty in Artificial Intelligence's annual conference this summer, researchers from MIT's Laboratory for Information and Decision Systems (LIDS) and Computer Science and Artificial Intelligence Laboratory will present a new reinforcement-learning algorithm that, for a wide range of problems, allows computer systems to find solutions much more efficiently than previous algorithms did.

The paper also represents the first application of a new programming framework that the researchers developed, which makes it much easier to set up and run reinforcement-learning experiments. Alborz Geramifard, a LIDS postdoc and first author of the new paper, hopes that the software, dubbed RLPy (for reinforcement learning and Python, the programming language it uses), will allow researchers to more efficiently test new algorithms and compare algorithms' performance on different tasks. It could also be a useful tool for teaching computer-science students about the principles of reinforcement learning.

Geramifard developed RLPy with Robert Klein, a master's student in MIT's Department of Aeronautics and Astronautics. RLPy and its source code were both released online in April.

Every reinforcement-learning experiment involves what's called an agent, which in artificial-intelligence research is often a computer system being trained to perform some task. The agent might be a robot learning to navigate its environment, or a software agent learning how to automatically manage a computer network. The agent has reliable information about the current state of some system: The robot might know where it is in a room, while the network administrator might know which computers in the network are operational and which have shut down. But there's some information the agent is missing—what obstacles the room contains, for instance, or how computational tasks are divided up among the computers.

Finally, the experiment involves a "reward function," a quantitative measure of the progress the agent is making on its task. That measure could be positive or negative: The network administrator, for instance, could be rewarded for every failed computer it gets up and running but penalized for every computer that goes down.

RLPy allowed the researchers to quickly test their new algorithm against a number of others. "Think of it as like a Lego set," Geramifard says. "You can snap one module out and snap another one in its place."

In particular, RLPy comes with a number of standard modules that represent different machine-learning algorithms; different problems (such as the network-administration problem, some standard control-theory problems that involve balancing pendulums, and some standard surveillance problems); different techniques for modeling the computer system's environment; and different types of agents.

It also allows anyone familiar with the Python programming language to build new modules. They just have to be able to hook up with existing modules in prescribed ways.

Geramifard and his colleagues found that in computer simulations, their new algorithm evaluated policies more efficiently than its predecessors, arriving at more reliable predictions in one-fifth the time.

RLPy can be used to set up experiments that involve computer simulations, such as those that the MIT researchers evaluated, but it can also be used to set up experiments that collect data from real-world interactions. In one ongoing project, for instance, Geramifard and his colleagues plan to use RLPy to run an experiment involving an autonomous vehicle learning to navigate its environment. In the project's initial stages, however, he's using simulations to begin building a battery of reasonably good policies. "While it's learning, you don't want to run it into a wall and wreck your equipment," he says.

Thursday June 06 2013 9:06AM

Umesh jhade

Replies -0

Views -1

 

Since ancient times, the pressures of excessive hunting have contributed to the gradual decline of wildlife populations and even the extinction of certain species in many areas.

Researchers from the Polytechnic University of Madrid (UPM) and the Centre for Forestry Research (INIA-CIFOR) have studied the developments in big game in Spain between 1972 and 2007 to gain an understanding of the hunting trends of the last decades.

"We have analysed the general trends in official statistics on hunters, hunting weapons, hunting grounds and captures. Our main conclusion is that the number of big game hunters has increased even though the total number of hunters has fallen," as María Martínez, researcher at the Centre for Forestry Research and co-author of the study, explains to SINC.

The paper also reveals that the number of open access game reserves has decreased in favour of a proliferation of territories in which captures are controlled.

"We know that this rise in big game captures has been accompanied by significant transformations in a series of factors related to supply and demand: families' higher purchasing power, a greater number of urban hunters away from the countryside who pursue big game and trophy hunting, abandonment of farming activity and livestock rearing, technological advances, restocking and releases.

All of this contributes to the creation of more and more fenced territories dedicated to the practice, often more intensively," notes Martínez.

Hunting rises in line with economic interests

Another of this study's conclusions is that the development of game as an economic good can be explained by the parameters that govern the market. As such, from 1972 to 1989, the most significant motivating factor for the rise in captures was the reduction in open access to game and its replacement by increased control over the use of these areas for commercial hunting. Subsequently, from 1989 to 2007, big game hunting grew mostly due to a rise in demand propelled by the greater interest of urban-dwelling tourists in these kinds of activities.

"However, the rise in big game at times reaches semi-domestication and can result in damages, for example to crops or difficulties with the natural regeneration of vegetation species. To obtain sustainable levels for the capture of big game and their populations, it is essential that those in charge are aware of these damages. At the same time, it is advisable to recognise the positive opportunities generated by hunting activity," the researcher concludes.

Editor's Note: This article was originally published by PHYS ORG, here, and is licenced as Public Domain under Creative Commons. See Creative Commons - Attribution Licence.