As part of the Deep reinforcement learning nano degree with Udacity, this is an implementation of the DDPG algorithm to train an agent how to move a double jointed arm within moving target.
For any reinforcement learning implementation, the agent has to interact with their environment. In this case, the environment space is represented by 33 different variables that incorporate position, rotation, velocity, and the arm's angular velocity.
The agent's goal is to maintain the arm in the target area for as long as possible (as measured by time steps). As such the agent recieves a reward of +.1 for each time step that the arm is in the target area.
The state space has to total of 37 dimensions and keeps information on the velocity of the agent, as well a ray based perception of objects in front of the agent.
Within the environment, there is a discrete action space of 4. Per the udacity program the agent able to take action represented by four numbers that correspond to the torque of the arm's two joints.
In order to successfully complete the episodic task, the agent is required to obtain an average score of 30 or above over the course 100 episodes.
-
To get up and running, you'll want to download the defined Udacity dependencies here
-
You'll need to grab the following:
-
Next you'll need the navigation simulation. Download the environment that corresponds to your operating system.
-
Version 1: One (1) Agent - Linux: click here - Mac OSX: click here - Windows (32-bit): click here - Windows (64-bit): click here
(For Windows users) Check out this link if you need help with determining if your computer is running a 32-bit version or 64-bit version of the Windows operating system.
(For AWS) If you'd like to train the agent on AWS (and have not enabled a virtual screen), then please use this link (version 1) or this link (version 2) to obtain the "headless" version of the environment. You will not be able to watch the agent without enabling a virtual screen, but you will be able to train the agent. (To watch the agent, you should follow the instructions to enable a virtual screen, and then download the environment for the Linux operating system above.)
-
Move the downloaded file to the
p2_continuous-control/folder in the DRLND repository. Decompress the file and you'll be ready to go. -
Navigate to the
Continuous_Control.ipynband load the file using the drlnd kernel.
Credit to Udacity for collating the above resources