[SOLVED] CS489 - Assignment 3

Reinforcement Learning

1 Introduction

The goal of this assignment is to do experiment with model-free control, includ­ing on-policy learning (Sarsa) and off-policy learning (Q-learning). For deep understanding of the principles of these two iterative approaches and the differ­ences between them, you will implement Sarsa and Q-learning at the application of the Cliff Walking Example, respectively.

2 Cliff Walking

Figure 1: Cliff Walking

Consider the gridworld shown in the Figure 1. This is a standard undis­counted, episodic task, with start state (S), goal state (G), and the usual actions causing movement up, down, right, and left. Reward is -1 on all transitions ex­cept those into the region marked “The Cliff”. Stepping into this region incurs a reward of -100 and sends the agent instantly back to the start.

3 Experiment Requirments

• Programming language: python3
• You should build the Cliff Walking environment and search the optimal travel path by Sara and Q-learning, respectively.
• Different settings for f can bring different exploration on policy update. Try several f (e.g. f = 0.1 and f = 0) to investigate their impacts on

4 Report and Submission

• Your reports and source code should be compressed and named after ”stu­dentID+name”.
• The files should be submitted on Canvas before Apr. 16, 2021.