![[LOGO]](/workbook/skins/workbook/create_small2.png)
Learning
Contents |
Purpose
These exercises will introduce the topic a machine learning, with a focus on reinforcement learning. Please refer to chapter 21 of "The Robotics Primer".
Overview
As you have seen by this point, the difficulty of creating robot control policies quickly increases in proportion to the difficulty of the task, environment, and complexity of the robot. In many cases it makes more sense to have the robot learn it's control policy, or learn to recognize patterns in the sensory information. This is especially true when numerous parameters interact with each other, and when some state information is hidden.
Machine learning is a diverse field of research composed of numerous topics, including supervised and unsupervised learning, pattern recognition and classification, and statistical and symbolic learning. The following exercises will focus on one specific type of learning that is fairly intuitive called reinforcement learning. This type of learning is based on the notion of receiving positive and negative rewards in accordance to the robot's performance. For example, a common situation might involve learning to navigate a maze. The robot would receive small negative rewards for each time step, encouraging the robot to finish quickly, and a larger positive reward when the robot finishes. The types of rewards and when they are applied help inform the robot about which actions were good and which were bad. Going into all the ins and outs of reinforcement learning is beyond the scope of this workbook, but it is well covered in Chapter 21 of the "Robotics Primer".
Hardware
| Exercise | Track 1 | Track 2 | Misc |
|---|---|---|---|
| Exercise 1 | No | Yes | N/A |
| Exercise 2 | No | Yes | N/A |
Exercise1: Learn To Follow a Line
|
This first exercise will introduce you to reinforcement learning through a simple task. The focus of this exercise will be on how and when to assign rewards, and some of the problems with reinforcement learning, including convergence times, exploration vs exploitation, and delayed rewards. |
 |
Exercise2: Learn to avoid obstacles
|
This exercise builds off the previous by adding a sensor and defining a slightly more difficult control problem. A sonar sensor attached to the front of the Create robot will detect distance to obstacles. The robot must use this information to slow down and stop before hitting an object in its path. This exercise will make use of sonar sensor to detect obstacles. The robot's goal is to learn a policy that prevents it from hitting a obstacle. |
|
