Python library for representation, design and visualization of Explauto's sensorimotor models


(Sébastien Forestier) #1

I’ve played with Explauto’s sensorimotor models and written some code to handle basic hierarchies of sensorimotor models (see the doc of my internship here for more details).

After discussing with @Pierre, it seems that a module (being part of Explauto or standalone) to properly handle the different sensorimotor models could be beneficial for us.

The main features we see are the following:

  • A way to easily configure the sensorimotor models and how they are connected,
  • Mechanisms to properly handle the flow of information in the graph of the different models,

Another interesting feature would be some sort of GUI to

  • Configure the different models and their connections,
  • Visualize the flow of information between the models (e.g. in a step by step debug, or replay mode),
  • Allow interaction possibilities:
    • click on a sensori space to set a sensori goal,
    • click to give human social feedback,
    • visualize the points explored in a given space (1D, 2D, 2D projections of more dimensional spaces),
    • choose a point to be replayed on the corresponding simulator or robot.

The graph of models (or hierarchy of tasks) should be a Directed Acyclic Graph, and it should be possible to dynamically add a node (for the agent, and eventually for the human to impose or advise new tasks).

The interface could be web-based.

What do you think about the idea ?
Do you know existing piece of software that might be related to what we want ?

@Pierre @oudeyer


(Pierre Rouanet) #2

I’ll just start adding a few library that may be related to what we have in mind:

  • ETE: A Python Environment for Tree Exploration
  • Nebri: Event Based Framework

(Pierre-Yves Oudeyer) #3

I think the idea of having such a tool for facilitating experimentation is a good idea. However, each part should be developed just for the specific needs of a specific experiment/paper,
otherwise trying to develop something “general” potentially useful for the future is bound to end in too much work with low probability of reuse in actual scientific experiments.
So the good way to proceed: think of an interesting scientific experiment to make, then develop the tools just needed for this.