\documentclass[11pt]{article}

\usepackage{fullpage}

\newcommand{\comment}[1]{}

\begin{document}

\thispagestyle{empty}

\begin{center}
\Huge\bf
Homework \#\,\,4\\
\Large\bf
15-486/782: Artificial Neural Networks\\
Dave Touretzky, Fall 2006\\[.2in]
\end{center}

\begin{itemize}

\item Due November 1, 2006

\item Recommended reading: \emph{Forward models: Supervised learning with a
distal teacher} by Michael I. Jordan and David E. Rumelhart.

\item Software you need is in
\verb+/afs/cs/academic/class/15782-f06/matlab/distal+

\item Answers must be typed.  Handwritten answers will not be accepted.

\end{itemize}

In this howework we will explore the use of neural networks as controllers.
Following the material presented in the lecture on this subject, you
are asked to build two controllers for the three link kinematic
arm.  One controller is a direct inverse model controller, while the
other is a controller trained using the distal teacher discussed in the
paper by Jordan and Rumelhart.

\section*{Preliminaries}

\begin{enumerate}

{}\item The dataset used in this homework is in the file
{}\emph{distal\_data.mat}, type: {\bf load distal\_data} to load the
data.  The training dataset is contained in the arrays
{}\emph{train\_x} {}\emph{train\_q} and {}\emph{train\_qnet}.
{}\emph{train\_x} contains a set of end effector (free end of the arm)
positions in $(x,y)$. {}\emph{train\_q} contains the corresponding
joint angles in radians, $(q_1,q_2,q_3)$, and {}\emph{train\_qnet}
contains a transformation of the joint angles in {}\emph{train\_q} to
values between -1 and 1.  {}\emph{train\_qnet} is used in place of
{}\emph{train\_q} for input and target output of the neural networks.
The transformation is $(\cos(q_1 + \pi/2), \cos(q_2),
\cos(q_3))$.

\item The matlab script \emph{run} initializes the graphical
display and sets up some important variables.  Run this script before
all other scripts and functions in the directory.  You can play with
the graphical arm by moving the sliders corresponding to the joint
angles.  The script \emph{run} and a function called
\emph{test\_controller} use a function \emph{forward} that computes
the forward kinematics of the three link arm.  That is, it takes the
joint angles as inputs and computes the end effector position in
$(x,y)$ as well as the positions of all the joints.

\item The script \emph{train\_fwd\_model} was used to train the
forward model weights, \emph{fwd\_Weight1} and {}\emph{fwd\_Weight2}.
It uses the delta-bar-delta method (see Bishop, chapter 7) to adapt
the learning rate and momentum.  Feel free to use this code as the
basis for the code you will be generating to train the controllers.

\end{enumerate}

\section*{Problems}

\begin{enumerate}

\item Write a program called \emph{train\_inv\_model} to train a neural
network, using backpropagation, as a direct inverse model to control
the three link robotic arm.  The network will take as input the
desired position of the end effector and give as output the joint
angles required to achieve the desired end effector position.  For
training, use \emph{train\_x} as input and {}\emph{train\_qnet} as
output.  Note: you must use a much lower learning rate to train the
inverse model than was used to train the forward model.  Stop training
when the error has changed by less than 1\% of its value over 50
epochs.  Hand in this code.

{}\item Test the performance of the inverse model controller.  The
function {}\emph{test\_controller(Weights1,Weights2)} produces a
quiver plot of the deviation between the desired and actual position
of the end effector.  The arrows point from the desired position to
the actual position.  The larger the arrow, the larger the error at
that point.  Hand in this plot.  Your results should be
comparable to the corresponding results shown in the Jordan and
Rumelhart paper.

\item The arrays \emph{fwd\_Weight1} and \emph{fwd\_Weight2} 
provided in {}\emph{distal\_data} are the weight values of a previously
trained forward model.  These weights were trained with the joint
angles {}\emph{train\_qnet} as input and end effector positions
{}\emph{train\_x} as output.  Using these weights in a model of the
forward kinematics, write a program that trains a neural network using
the distal teacher approach.  Initialize the weights as the final
weight values found while training the direct inverse model.  As
before, stop training when the error has changed by less than 1\% of
its value over 50 epochs (this will take a while).  Hand in this code.

\item Use \emph{test\_controller} to test the performance of the
distal teacher controller.  Hand in the quiver plot of the performance
of the controller trained using a distal teacher.  Again, your results
should resemble the corresponding results shown in the Jordan
and Rumelhart paper.

\item Compare the two approaches.  (a) How does the performance of
the distal teacher trained controller compare to the controller
trained as a direct inverse model, based on the plots?  (b) How many
epochs were required to train each controller?  (c) What was the final
sum-squared error for each controllre?

\comment {
\item Jordan and Rumelhart claim that the controller trained with a
distal teacher achieves better performance than the inverse model
controller because of the nonconvexity of the three link arm domain.
Explain what this means and why it would adversely effect the direct
inverse controller and not the controller trained with a distal
teacher.

\item Give an example of a convex control problem domain and explain why
it is convex.  For this example, would you recommend the direct
inverse controller or a controller trained using a distal teacher
(give your reasoning)?
}

\end{enumerate}

\end{document}