Understanding how world knowledge
can be grounded in sensori-motor
experience has been a long-standing goal of philosophy, psychology, and
artificial intelligence. So far this goal has remained distant, but
recent progress in machine learning, cognitive science, neuroscience,
engineering, and other fields seems to bring nearer the possibility of
addressing it productively.
The objective of this workshop is
to provide cross-fertilization of
ideas between diverse research communities interested in this subject. This
workshop will serve as a meeting point for researchers from these
various disciplines to share their perspectives and insights on the
issue of representing knowledge in terms of sensori-motor experience.
Dr.
Richard Sutton
Richard S. Sutton is professor and iCORE chair of computing science at
the University of Alberta. He is a fellow of the American
Association for Artificial Intelligence and co-author of the textbook
Reinforcement Learning: An Introduction from MIT Press. Before joining
the University of Alberta in 2003, he worked in industry at AT&T
and GTE Labs, and in academia at the University of Massachusetts. He
received a PhD in computer science from the University of Massachusetts
in 1984 and a BA in psychology from Stanford University in 1978.
Rich's research interests center on the learning problems facing a
decision-maker interacting with its environment, which he sees as
central to artificial intelligence. He is also interested in
animal learning psychology, in connectionist networks, and generally in
systems that continually improve their representations and models of
the world.
Dr. Deb Roy
Deb Roy is an associate professor of media arts and sciences at the
Massachusetts Institute of Technology, where he heads the Media Lab's
Cognitive Machines research group. In 2003 he was appointed AT&T
Career Development Professor. Roy has published over 50 peer-reviewed
papers in the areas of artificial intelligence, cognitive modeling,
data mining, robotics, and human-machine interface design. He has
served as guest editor for the journal Artificial Intelligence, and as
an associate of the journal Behavioral and Brain Sciences. Roy
collaborates closely with industry in the areas of data visualization,
data mining, and the design of human-machine collaborative systems. He
holds a BASc in computer engineering from University of Waterloo,
Canada, and MS and PhD degrees in media arts and sciences from MIT.
Dr. Mark H.
Bickhard
Mark H. Bickhard received his B.S. in Mathematics, M.S. in Statistics,
and Ph. D. in Human Development, all from the University of
Chicago. He taught at the University of Texas at Austin for
eighteen years before joining Lehigh University in 1990 as Henry R.
Luce Professor in Cognitive Robotics and the Philosophy of
Knowledge. He is affiliated with the Departments of Psychology,
Philosophy, Biology, Counseling, and Computer Science, and is Director
of the Institute for Interactivist Studies and of the Complex Systems
Research Group. He was Director of Cognitive Science from 1992
thru 2003. His work focuses on the nature and development of
persons, as biological, psychological, and social beings. This
work has generated an integrated organization of models encompassing
the whole person, ranging from the nature of biological function
through perception, cognition, processes of and constraints on
development, rationality, emotions, reflexive consciousness, language,
psychopathology, and the relationships between the emergence of social
reality and the social ontology of persons.
Dr. Rajesh Rao
Rajesh Rao is an associate professor in the Computer Science and
Engineering department at the University of Washington, where he heads
the Laboratory for Neural Systems. He received his PhD from the
University of Rochester and was a Sloan Postdoctoral Fellow at the Salk
Institute for Biological Studies before joining the University of
Washington. His research spans the areas of computational neuroscience,
humanoid robotics, and brain-computer interfaces. He is the recipient
of a David and Lucile Packard Fellowship, an Alfred P. Sloan Fellowship
for junior faculty, an ONR Young Investigator Award, and an NSF Career
award. He is the co-editor of two books: Probabilistic Models of the
Brain (2002) and Bayesian Brain (2007).
Dr.
Bernard Balleine
Bernard Balleine is Professor in the Department of Psychology and
Associate Director of the Brain Research Institute,
UCLA.
He received his BA from the University of Sydney, Australia and his PhD
from the University of Cambridge, UK where he was subsequently elected
a Research Fellow of Jesus College. His research focuses on the
motivational, cognitive and neural determinants of goal-directed action
as a part of the larger goal of establishing the fundamental
distinctions between reflexive, volitional and habitual actions.
Confirmed Posters (not all are listed
yet)
POMDP Homomorphisms
Alicia Peregrin Wolfe
The problem of finding hidden state in a POMDP and the problem of
finding abstractions for MDPs are closely related. This work analyzes
the connection between existing Predictive State Representation methods
and homomorphic reductions of Markov Processes. We formally define a
POMDP homomorphism, then extend PSR reduction methods to find POMDP
homomorphisms when the original POMDP is known. The resulting methods
find more compact abstract models in situations where different
observations have the same meaning.
Neighborhood Components Analysis for Reward-Based
Dimensionality Reduction
Nathan Sprague
There has been a great deal of research that attempts to explain the
structure of biological receptive fields in terms of various methods
for adapting basis vectors based on the statistical structure of visual
input. These include principal components analysis (Hancock et
al., 1992), independent components analysis (Bell & Sejnowski,
1997), non-negative matrix factorization (Lee & Seung, 1999), and
predictive coding (Rao & Ballard, 1999), among others. Typically,
such approaches are based purely on the structure of the visual input;
there is no consideration of the role that visual information plays in
the goal directed behavior of an organism. The motivation for the
current work is to explore mechanisms of basis vector adaptation that
are explicitly driven by the behavioral demands of a situated agent.
Alexander Stoytchev
This paper formulates five basic principles of developmental robotics.
These principles are formulated based on some of the recurring themes
in the developmental learning literature and in the author's own
research. The five principles follow logically from the
verification principle (postulated by Richard Sutton) which is assumed
to be self-evident. This paper also gives an example of how these
principles can be applied to the problem of autonomous tool use in
robots.
State Similarity Based Approach for Improving Performance in
RL
Sertan Girgin, Faruk Polat and
Reda Alhajj
In most of the realistic and complex domains, the task that a
reinforcement learning agent tries to solve contains various subtasks,
each of which repeats many times at different regions of the state
space. Although the solutions of the instances of the same or similar
subtasks are almost identical, without any guidance, an agent has to
learn related sub-policies independent of each other; this would cause
the agent to pass through similar learning stages again and again, and
as a result it will be harder to converge to optimal behavior in
reasonable time. The main reason of the problem is the lack of
connections that would allow solution sharing. Based on the fact that
states with similar patterns of behavior constitute the regions of
state space corresponding to different instances of similar subtasks,
the notion of state equivalence can be used as a mechanism to
facilitate the sharing of solutions. By reflecting experience acquired
on one state to all similar states, connections between similar
subtasks can be established implicitly; this would reduce the
repetition in learning, and consequently improve the performance.
In this paper, based on the (partial) MDP homomorphism notion of
Ravindran and Barto, we propose a method to identify states with
similar sub-policies without requiring a model of the MDP or
equivalence relations, and show how they can be integrated into the
reinforcement learning framework. As the learning progresses, using the
observed history of events, potentially useful policy fragments are
generated and stored in a labeled tree. A metric, which is based on the
number of common action-reward sequences, is devised to measure the
similarity between two states, and the tree is utilized to apply the
metric and determine states with similar sub-policy behavior.
Eligibility requirements are imposed on the structure of the tree in
order to keep its size manageable and focus on recent and frequently
used sub-policies. Updates on the value function of a state are then
reflected to all similar states, expanding the influence of new
experiences. The proposed method can be treated as a meta-heuristic and
it is possible to apply it to existing reinforcement learning
algorithms. We demonstrate the effectiveness of the approach by
reporting test results on sample problems.
Human State Estimation Through Learning Over Common Sense Data
William Pentney & Matthai
Philipose & Jeff Bilmes & Henry Kautz
We seek to tackle the problem of human state recognition, in which
sensor-based observations are used to reason about the state of the
general
human environment. Recent work [Pentney et al., 2006] has shown promise
in using large publicly available hand-contributed commonsense
databases as joint models that can be used to interpret day-to-day ob
ject-use data. We discuss the development of a graphical model for
reasoning over large amounts of commonsense information about human
activity, and the use of Web-based information retrieval techniques to
evaluate and enhance such information for more effective use. The large
scale of this commonsense data creates issues of scale in inference
over our graphical model; we present some means of efficiently
performing inference over such a model. Additionally, we discuss how to
improve the performance of our model through the use of learning
techniques which can scale to the very large networks induced by this
commonsense data. Finally, we present experiments to show how these
techniques can be used to provide improved results in the prediction of
everyday human state.
Social interaction through movement: concepts from
perception-action interplay
Emilia I. Barakova,
The human brain has evolved for governing motor activity by
transforming sensory patterns to patterns of motor coordination.
Movement and action (action is understood as purposeful movement) are
the primary expressions of behaviour. Tracing the evolution of species,
movement takes incrasingly more complex and abstract forms. By humans,
movement is grounding cognition, language, and even social interaction.
Interaction through movement and its implications for creation of
social agents are discussed in an attempt to outline a new design
framework. After reviewing the main frameworks on perception and action
and the design concepts they suggest, we choose the common coding
paradigm as a basis for design of social interactions. The common
coding is supported by the latest discoveries in neuroscience and
experimental psychology. In particular, the discovery of the mirror
neuron system in humans have given new dimension of understanding the
sensorimotor system and its interaction to a complex environment,
including the interactions with another agents. This design paradigm is
illustrated by an experiment.
Learning Subjective Representations Through Dimensionality
Reduction
Dana Wilkinson and Michael
Bowling and Ali Ghodsi
There are a variety of domains where one wishes to learn a
representation of an environment defined by a stream of sensori-motor
experience. In many cases, such a representation is necessary as
the observational data is too plentiful to be stored in a
computationally feasible way. In other words, the primary feature
of a learned representation is that it must be compact---summarizing
information in a way that alleviates storage demands.
This admits a new way of phrasing the problem: as a variation of
dimensionality reduction. There are a variety of well-studied
algorithms for the dimensionality reduction problem, we argue that any
of these can be useful for learning compact representations as long as
additional constraints to the problem are respected---namely that the
result is useful for reasoning and planning.
Here, we formalize the problem of learning a subjective representation,
clearly articulating solution features that are necessary for a learned
representation to be ``useful''. Further, we briefly present a
possible solution to the newly defined problem (Action Respecting
Embedding) and demonstrate it's effectiveness.
An Information Theoretic Approach for Building Approximate
Predictive Models
Susanna Still, Monica
Dinculescu, Doina Precup
Participation in the form of a
poster will be by invitation from the
program committee based on a small written submission, either a short
paper or extended abstract on your relevant work (this may be work that
has been previously
published elsewhere).
We encourage submissions from all
disciplines that are related to the topic of the workshop. The
poster session is expected to reflect that wide variety of interesting
ideas surrounding our topic.
