A Theory of Cheap Control in Embodied Systems

A Theory of Cheap Control in Embodied Systems, Guido Montúfar, Keyan Ghazi-Zahedi, Nihat Ay, PLoS Computational Biology, 2015

 

Abstract

We present a framework for designing cheap control architectures of embodied agents. Our derivation is guided by the classical problem of universal approximation, whereby we explore the possibility of exploiting the agent’s embodiment for a new and more efficient universal approximation of behaviors generated by sensorimotor control. This embodied universal approximation is compared with the classical non-embodied universal approximation. To exemplify our approach, we present a detailed quantitative case study for policy models defined in terms of conditional restricted Boltzmann machines. In contrast to non-embodied universal approximation, which requires an exponential number of parameters, in the embodied setting we are able to generate all possible behaviors with a drastically smaller model, thus obtaining cheap universal approximation. We test and corroborate the theory experimentally with a six-legged walking machine. The experiments indicate that the controller complexity predicted by our theory is close to the minimal sufficient value, which means that the theory has direct practical implications.

http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004427

In a nutshell

We model the behaviour of system by the conditional probability over it’s world states

     \begin{align*} \mathbb{P}^\pi(w_1, w_2, \ldots, w_T|w_0) := \sum_\mathcal{S}\sum_\mathcal{A}\prod_{t=0}^{T-1} \beta(s_t|w_t) \pi(a_t|s_t) \alpha(w_{t+1}|w_t,a_t) \end{align*}

We can now define a policy-behaviour map, which assigns a behaviour to each policy

     \begin{align*} \psi_\infty\colon \Delta_\mathcal{A}^\mathcal{S}\longrightarrow \Delta_{\mathcal{W}^\infty}^\mathcal{W},\qquad \pi \mapsto \mathbb{P}^\pi(w_1, w_2, \ldots|w_0) \end{align*}

Given a morphology, it is clear that there are policies, which generate the same behaviour

     \begin{align*} \psi_\infty(\pi_1) = \psi_\infty(\pi_2) \end{align*}

So we can ask the question, what is the minimal set of policies that generate all possible behaviours. We want to now find the minimal model  \overline{\mathcal{M}} that parameterises this set:

     \begin{align*} \psi_\infty(\overline{\mathcal{M}}) = \psi_\infty(\Delta^\mathcal{S}_\mathcal{A}) \end{align*}

Finally, we can define the embodiment behaviour dimension in the following way

     \begin{align*} d=\dim(\Delta^\mathcal{S}_\mathcal{A}) \end{align*}

We apply this theory to estimate the number of hidden unit  m of a CRBM that are required to reproduce a target behaviour:

     \begin{align*} m \, \geq \, |{\mathcal S}_\beta| + d_{\alpha,\beta} - 1, \end{align*}

where |{\mathcal S}_\beta| is the approx. the number of different sensor values that the agent receives, and  d_{\alpha,\beta} is the corresponding embodiment behaviour dimension.

We used this to estimate that a tripod walking behaviour required 65 hidden units. The details are presented in the paper. The videos can be found below.

Videos

This video shows the behaviour of the hexapod with a limited brain of only 35 hidden units. The hexapod is able to move but is far from reproducing the target behaviour (shown as transparent hexapod). The left hand of the video shows how the sensor values are translated into binary input units and how the output of the CRBM is transformed to control the robot.

As predicted, the CRBM is able to reproduce the target behaviour with 65 hidden units.

The full list of videos can be found here.

Leave a Reply

Your email address will not be published. Required fields are marked *