[Ml-stat-talks] Fwd: Course on ADP and RL-please respond!

David Blei blei at CS.Princeton.EDU
Thu Sep 16 15:04:05 EDT 2010

hi ml-stat-talks

see below for information about warren powell's course on approximate
dynamic programming and reinforcement learning.  it looks to be a very
interesting course.


---------- Forwarded message ----------
From: Warren Powell <powell at princeton.edu>
Date: Wed, Sep 15, 2010 at 7:51 PM
Subject: Course on ADP and RL-please respond!

Last year, I sent out an email to gauge the interest in a graduate seminar
on approximate dynamic programming and reinforcement learning.  Given the
strong response, the course is now on the books as ORF 569.  The first
meeting is tentatively set for Wednesday at 3pm (I will email the classroom

If you are a student interested in taking the course (this includes my own
students), please respond to this email right away and let me know.  Also,
if this time causes a conflict, let me know this as well.  I can move the
first meeting to an evening time after which I will do my best to find a
time slot that works for everyone.

To the faculty receiving this: please forward this to any students who may
be interested.

As a reminder, the initial summary is listed below.

Prof. Powell

I am thinking of organizing a graduate research seminar that would
investigate in depth the state of convergence proofs for various algorithms
in approximate dynamic programming/reinforcement learning.  I am looking to
draw students from different fields and discuss algorithms geared to
different classes of applications.

Convergence theory for algorithms in ADP/RL introduces a rich set of issues
that arise in the different algorithmic strategies within this broad field.
 Issues include:

o How to approximate value functions (machine learning), and how to handle
biased observations
o The role of a sampling policy (off-line vs. on-line)
o The exploration vs. exploitation problem
o Value function updating strategies
o Stepsizes and rate of convergence
o The challenge of high-dimensional actions/decisions/controls

My plan is to have each student prepare one or two papers drawn from the
literature.  Depending on the participation, I would talk to other faculty
advisers to identify theory papers relevant to their area of research.  I am
willing to give a tutorial lecture on ADP/RL, and provide additional
instruction as necessary, so prior knowledge of these algorithms is not
necessary.  But the course would focus on different proof techniques, and
identifying the types of strategies people use to prove convergence.  I have
found that a deep understanding of convergence theory helps in the
development of practical algorithms.

I would enjoy having students from computer science, operations research,
and control theory (the primary methodological disciplines) with broad
application interests.

The course would take place in a single weekly meeting in the fall semester
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cs.princeton.edu/pipermail/ml-stat-talks/attachments/20100916/7d395300/attachment.html>

More information about the Ml-stat-talks mailing list