From IPRE Wiki
Revision as of 22:32, 20 September 2011 by Cclabaugh (Talk | contribs) (Quick Start)

Jump to: navigation, search

Luar stands for Lua Universal Architecture for Human Robotics (the H is silent, and, in fact, invisible!)


If you would like to work on this on your own computer, you'll need the following:

  1. Download Webots - comes with RobotStadium simulator
  2. Download Luar - humanoid control system


  • sudo apt-get install liblua5.1-0-dev
  • May need to edit WebotsController/Makefile:
    • -I/usr/include/lua5.1/
    • OR do this:
    • sudo cp /usr/include/lua5.1/* /usr/include/
  • May need to edit WebotsController/Makefile:
    • change WEBOTS_HOME
  • Fix Makefiles in UPennalizers/Lib, or just copy headers:
    • sudo cp /usr/include/lua5.1/* /usr/include/

To setup the system, please follow these instructions (click on view raw):

Quick Start

  1. git clone
  1. cd UPennalizers/WebotsController
  2. make
  3. cd ../Lib
  4. make setup_webots
  5. Edit UPennalizers/Player/World/gcm.lua
  6. Change last lines to be:
    function in_penalty()
    return false; -- get_game_penalty()[get_team_player_id()] > 0;

Every once in a while, you may want to:

  1. cd UPennalizers
  2. git pull

That will take the items from the git repository and merge them with your own code.

Assign the teams

Use the maketeam command to set the teams:

  1. maketeam /usr/local/webots/projects/contests/robotstadium/controllers/nao_team_0.bak nao_team_0
  2. maketeam /home/dblank/UPennalizers/WebotsController nao_team_1

Here, team_0 is the original, and team_1 is your new Luar team.

Play soccer

  1. webots
  2. Select Robotstadium Contest
  3. close 4 error dialogs
  4. Soccer will automatically start


Shared Memory Interface

Luar is a multi-process architecture in that there are multiple processes running simultaneously and at different frequencies. Luar is composed of two main processes: a Motion process and a Cognitive process. The Motion process runs at about 100 updates a second (~100Hz) and the Cognitive process runs at about 30 updates a second (~30Hz).

Of course, there are data that need to be shared between these parallel processes. To accomplish this, Luar uses shared memory. In Luar all shared memory interfaces are handled by special Lua modules called Communication Managers. In general a communication manager will end with *cm.lua (e.g. vcm.lua is the Vision Communication Manager and handles all of the shared memory vision information). The Communication Manager is designed to standardize and abstract all shared memory interfacing and to make adding additional shared memory variables easily.

Every Communication Manager has the following special variables.

A table indicating the shared memory variables to define including the type and size of the variable.
Each member of the shared table indicates a new shared memory segment.
The variables in each segment are indicated by the members of each segment table.
A table indicating the size of the shared memory segment to create.
You can indicated the total size of the segment by adding it as a key to this table with size of the segment as the value.
The key should be the same used in the shared table.
If no entry exists in the shsize table then the default size (2^16 bytes) is used.

The second half of the communication managers in the creation of the shared memory segments and accessors for the data. This is handled by the util.init_shm_segment function. This functions will only create/destroy a segment or variable if it is needed. If the segment already exists and is the correct size then the current segment is used. Similarly for variables, new instances are only created if they do not already exist or the existing one is the incorrect size.

The accessors and setter functions are automatically created from the shared table structure. The same basic format is followed for all the segments/variables.

Simply put, the following entry:

 shared.[segment name].[variable name]

will result in the following function definitions to access and set the data respectively:

 get_[segment name]_[variable name]()
 set_[segment name]_[variable name](val)

The following data types are supported:

single numbers are indicated by setting the value to a vector of length 1.
NOTE: the accessor will return the number not a vector of length 1.
fixed length arrays by setting the value to a vector of length n > 1
the length of the array is specified by the size of the vector
vector.zeros(4) indicates to the program to create an array of length 4 in the shared memory
variable length strings indicated by setting the value to any string
the actual value set in the shared table is ignored, just the fact that it is a string is noted.
c array
fixed size c arrays indicated by setting the value to a number indicating the size in bytes of the array
The accessor/setter for a c array accept and return userdata

NOTE: When using the setters you must give it a variable of the correct size. (unless it is for the string)


The following is how to setup an example Communication Manager we will call Test Communication Manager (tcm) with the lua module tcm.lua.


 module(..., package.seeall);
 -- shared properties
 shared = {};
 shsize = {};
 shared.segment1 = {};
 shared.segment1.number_a = vector.zeros(1); -- vector of size 1 indicates a number
 shared.segment1.vector_b = vector.zeros(3); -- vector of size > 1 indicates an array
 shared.segment1.string_c = ' '; -- any string indicates a string (the actual value here is ignored when creating the block)
 shared.segment2 = {};
 shared.segment2.number_a = vector.zeros(1); -- vector of size 1 indicates a number
 -- indicate segment size for segment2
 shsize.segment2 = 2^12; -- bytes
 -- initialize shared memory segments/variables and create accessors/setters
 util.init_shm_segment(getfenv(), _NAME, shared, shsize);

This Communication Manager will have two shared memory segments indicated by the 'segment1' and 'segment2' fields in the table shared. The actual segment names for these will be 'tcmSegment1' and 'tcmSegment2' respectively and 'tcmSegment2' will have a total size of 2^12 bytes. The variables in each segment are:

number_a - which has a size of 1
vector_b - which has a size of 3
string_c - which is a variable length string
number_a - which has a size of 1

The initialization also automatically creates accessors and setters for the shared memory variables.


Testing the Test Comminication Manager:

We can test the new tcm.lua module interactively Lua. First, make the tcm.lua file as shown above and open two terminals with an instance of Lua. Each line is a step in time.

Terminal 1 Terminal 2
> dofile('init.lua');
> require('tcm')
> tcm.set_segment1_number_a(4)
> print(tcm.get_segment1_number_a())
> tcm.set_segment1_vector_b({1,2,3})
> tcm.set_segment1_string_c('hello')
> print(tcm.get_segment1_string_c())
> print(tcm.get_segment2_number_a())
> dofile('init.lua')
> require('tcm')
> print(tcm.get_segment1_number_a())
> tcm.set_segment1_number_a(1)
> print(tcm.get_segment1_number_a())
> print(tcm.get_segment1_vector_b())
{1, 2, 3}
> print(tcm.get_segment1_string_c())
> tcm.set_segment1_string_c('bye')
WARNING: Input size 3 != current block size 5. Resizing string_c block.
> tcm.set_segment2_number_a(2)
Independent Shared Memories

Memory segment names changed to make them personal to each robot on each team in each user account. Previously: [segment name][variable name] Now: [segment name][variable name][teamID][playerID][user]

One can check the segment names by using "cd /dev/shm/" and then "ls" in terminal.

NOTE "Permission Denied" error: This directory may cause problems as the naming scheme is not multi-user friendly. If running webots and getting a "Permission Denied" error in the webots console, try restarting the computer. This is because the "/dev/shm/" is owned by the person who first creates it. So if someone else created this file previously on the computer you are using, you will not be able to access it. This will need to be changed in the future for multi-user capabilities.

Now when testing *cm files, we must define team and player IDs so that we are watching one memory of one robot. Example:

 > = 1;
 > = 2;


When controlling the robot joints we consider the robot's body as an array of joints. To control a joint you must use the index value for that joint. Below is the index to joint mapping that we use:

(# is the index, followed by joint name)

  1. Head Yaw
  2. Head Pitch
  3. Left Shoulder Pitch
  4. Left Shoulder Roll
  5. Left Elbow Yaw
  6. Left Elbow Roll
  7. Left Hip Yaw Pitch
  8. Left Hip Roll
  9. Left Hip Pitch
  10. Left Knee Pitch
  11. Left Ankle Pitch
  12. Left Ankle Roll
  13. Right Hip Pitch Yaw
  14. Right Hip Roll
  15. Right Hip Pitch
  16. Right Knee Pitch
  17. Right Ankle Pitch
  18. Right Ankle Roll
  19. Right Shoulder Pitch
  20. Right Shoulder Roll
  21. Right Elbow Yaw
  22. Right Elbow Roll

Code Structure

The code is divided into two main components: high and low level processing. The low level code is mainly written in C/C++ and compiled into libraries that have Lua interfaces. These libraries are used mainly for device drivers and anything designed to execute quickly (e.g. image processing and forward/inverse kinematics calculations). The high level code is mainly scripted Lua. The high level code includes the robots behavioral state machines which use the low level libraries.

The following is a brief description of the code following the provided directory structure. The low level code is rooted at the Lib directory and the high level code is rooted at the Player directory.

Low Level Interface

The low-level interfaces are found in the ./Lib directory.

  • Platform -- humanoid robot platforms (Nao, OP, Webots, Webots_OP)
    There are several directories contained here, one corresponding to each of the robot platforms supported. The code contained in these directories is everything that is platform dependent. This includes the robots forward/inverse kinematics and device drivers for controlling the robots sensors and actuators. All of the libraries have the same interface to allow you to just drop in the needed libraries/Lua files without changing the high level behavioral code. The directory trees for each platform (Webots/Nao) are the same:
    • Body -- Body contains the device interface for controlling the robot's sensors and actuators. This includes controlling joint angles, reading IMU data, etc.
    • Camera -- Camera contains the device interface for controlling the robots camera.
    • Kinematics -- Kinematics contains library for computing the forward and inverse kinematics of the robot.
  • ImageProc -- This directory contains all of the image processing libraries written in C/C++: Segmentation and finding connected components.
  • Util -- These are all of the C/C++ utility function libraries.
    • CArray -- CArray allows access to C arrays in Lua.
    • CUtil -- CUtil contains functions that can be performed on C arrays (such as array-to-string conversion) that can then be called by Lua.
    • Shm -- Shm is the Lua interface to Boost shared memory objects (only used for the Nao).
    • Unix -- This library provides a Lua interface to a number of important Unix functions; including time, sleep, and chdir to name a few.
    • Matlab -- Contains the Matlab utility used to create the colortables necessary for vision
    • PyQT -- An incomplete attempt at writing a Python colortable utility, to replace the existing Matlab one.
  • NaoQi -- This contains the custom NaoQi module allowing access to the Nao device communication manager in Lua.
    • actuator_process.cpp --This script seems to handle command priority and execution in the shared memory space for actuators.
    • dcmprocess.cpp -- This deals with accessing DCM (the Device Communication Manager), which is access from shared memory and allows code to control the actuators and sensors on the robot itself.
    • luadcm.cpp -- This allows Lua to send commands to the DCM.
    • sensor_process.cpp -- This script seems to handle command priority and execution in the shared memory space for sensors.
    • shm_util.cpp -- This is the script that actually creates and manages the SHM (shared memory space) itself.
    • shmmap.cpp -- This code allows the sensor and actuator to access the SHM (though a good deal of it is commented out).

High Level Interface

player.lua is the main entry point for the code and contains the robot's main loop.

  • BodyFSM -- The state machine definition and states for the robot body are found here. These robot states include spinning to look for the ball, walking toward the ball and kicking the ball when positioned.
  • Config -- This directory contains the only high level platform dependent code, in the form of configuration files. Config.lua links to other environment-dependent configuration files, where the walk parameters, camera parameters and the names of the device interface libraries to use are all defined.
  • Dev -- This directory contains the Lua modules for controlling the devices (actuators/sensors) on the robot.
  • Data -- This directory contains any logging information produced. Currently this is only in the form of saved images.
  • HeadFSM -- The head state machine definition and states are located here. The head is controlled separately from the rest of the body and transitions between searching for the goals, searching for the ball, and tracking the ball once found.
  • Lib -- Lib contains all of the compiled, low level C libraries and Lua files that were created in ./Lib.
  • Motion -- Here is where all of the robot's motions are defined. It contains the walk and kick engines along with keyframe motions used for the get-up routines.
  • Util -- Utility functions are located here. The base finite state machine description and a Lua vector class are defined here.
  • Vision -- The main image processing pipeline is located here. It uses the output from the low level image processing to detect objects of interest (ball, goals, lines, spot, and landmarks).
  • World -- This is the code relating to the robots world model.
Team Positions

The robot with Player 1 ID is automatically the goalie (see Config_Nao.lua) and never changes positions; however, other members of the team change positions dynamically (see Team.lua). Based on the distant to goal, the team member is either attack, defence, or support.

Lua Functions

This section documents the high-level interface in Lua.


  • set_actuator_command(cmd, index)
    set the position of the given joint. Starting at index, set the joints to the cmd values (based on the size of the cmd array)
  • set_actuator_velocity(vel, index)
    set the velocity of the given joint. Starting at index, set the joint velocities to the vel values (based on the size of the vel array)
  • set_actuator_hardness(h, index)
    set the hardness of the given joint. Starting at index, set the joint hardnesses to the h values (based on the size of the h array)
  • get_sensor_position(index)
    return the current position of the given joint. If index is not specified then return an array of all joint positions.
  • get_sensor_imuAngle(index)
    return the state of the 3-axis imu. if index is specified then return only that axis' value, otherwise return all 3.
  • get_sensor_button()
    return the state of the button (1 - pressed/0 - not pressed)
  • get_head_position()
    convience function to return an array of all the head joint positions
  • get_larm_position()
    convience function to return an array of all the left arm joint positions
  • get_rarm_position()
    convience function to return an array of all the right arm joint positions
  • get_lleg_position()
    convience function to return an array of all the left leg joint positions
  • get_rleg_position()
    convience function to return an array of all the right leg joint positions
  • set_body_hardness(h)
    convience function to set the hardness of all the joints in the body to the same value
  • set_head_hardness(h)
    convience function to set the hardness of all the head joints to the same value
  • set_larm_hardness(h)
    convience function to set the hardness of all the left arm joints to the same value
  • set_rarm_hardness(h)
    convience function to set the hardness of all the right arm joints to the same value
  • set_lleg_hardness(h)
    convience function to set the hardness of all the left leg joints to the same value
  • set_rleg_hardness(h)
    convience function to set the hardness of all the right leg joints to the same value
  • set_head_command(cmd)
    convience function to set the head joint positions
  • set_larm_command(cmd)
    convience function to set the left arm joint positions
  • set_rarm_command(cmd)
    convience function to set the right arm joint positions
  • set_lleg_command(cmd)
    convience function to set the left leg joint positions
  • set_rleg_command(cmd)
    convience function to set the right leg joint positions
  • update()
    this function is run at every time step on the robot. it actually updates the robot joint positions based on the target velocity and desired position.


  • set_param(param, val)
    set the given camera parameter
  • get_param(param)
    return the current value of the camera parameter
  • get_height()
    return the height of the images the camera takes
  • get_width()
    return the width of the images the camera takes
  • get_image()
    return the actual image from the camera. This image should be in YUV422 format.
  • get_camera_position()
    returns the position of the camera being used. This is needed for the Naos because they have to cameras.
  • select(bottom)
    select a camera (used for Nao). If bottom != 0 then select the bottom camera otherwise select the top camera.
  • get_select()
    return the current selected camera.


  • os.getenv('USER')
    get the current user name; useful in avoiding bad/not multi-user friendly naming schemes

Keyframe Motions

Luar has the ability for play back sequences of moves, or keyframe motions. For examples, see Player/Motion/keyframes/*.lua

Keyframe motions are scripted, open loop motions that the robots can run. These are mainly used for the get-up routine and kicks. All keframe motions we provide have the following naming convention: km_<Name>.lua (e.g. km_StandupFromFront.lua is the keyframe motion file for standing up from the robots front/stomach).

Keyframe motions are lua tables with the following members:

  • mot -- tables include:
    • mot.servos -- list of motors/servos you want to control.
    • mot.keyframes -- list of frames that the robot will go through
  • stiffness -- the motor stiffness/hardness for this frame
  • angles -- the target motor positions for this frame
  • duration -- the amount of time that it should take the robot to reach the target motor positions

Finite State Machine

Assuming that the UPEnnalizers code is saved to the home directory, FSM lua files are found in the directory:

  • ~/UPennalizers/Player/BodyFSM/NaoPlayer



The main code that defines the Body FSM's states and transitions is called BodyFSM.lua.

All the other files in the directory are files that define the entry, update, and exit methods for the different states:

  • bodyIdle.lua
    The initial state of the Game FSM, it moves robot from sitting position to stance position, using states in the Motion directory.
  • bodyReady.lua
    In gameReady state in the game fsm, bodyReady is set. Move robots into position game positions.
  • bodyStop.lua
    The robot stops walking.
  • bodyStart.lua
    Called in the Game FSM state gamePlaying, the Robot begins walking. State transitions to bodyPosition.
  • bodyPosition.lua
    Depending on the role assigned to the robot:
    • 2: Based on the position of the ball, position body to defend the goal
    • 3: Support, move towards attacking goal and face the ball
    • All other roles: Move towards the ball
  • bodyOrbit.lua
    It seems that the Robot 'orbits' the ball trying to align it's body to the ball and the goal.
  • bodySearch.lua
    Attempts to locate the ball by spinning in a circle. If it cannot locate the ball, the search will timeout.
  • bodyGotoCenter.lua
    If the robot is the Goalie, move to the center of the goal. Otherwise move to either the attack or defensive position.
  • bodyApproach.lua
    Using the known location of the ball, move towards the ball.
  • bodyKick.lua
    Determines kind of kick based on ball position, and attempts to kick the ball. After a kick, the head state is changed to 'headTrack'
  • bodyObstacle.lua
    Stop walking, check to see if there is an obstacle. Then resume walking.
  • bodyObstacleAvoid.lua
    Attempts to avoid the obstacle by walking away from detected object. If not completed successfully in appropriate amount of time, function timesout.
  • bodyChase.lua
    It seems that the Robot chases after the ball, and checks if there is an obstacle.
  • bodyPause.lua
    The robot stops walking for a certain amount of time, and says "Defending."


The main code that defines the Head FSM's states and transitions is called HeadFSM.lua.

All the other files in the directory are files that define the entry, update, and exit methods for the different states:

  • headIdle.lua
    This is the initial state which is called in the game FSM state gameInitial. The default head position is defined by 'yaw' and 'pitch' constants in 'set_head_command.' This also continuously switches cameras. Transitions to headStart.
  • headStart.lua
    Called in the game FSM state gamePlaying. This state seems to be a placeholder as no changes are made to the head. Transitions to headTrack.
  • headReady.lua
    Called in the game FSM state gameReady. It finds the head angle and based on this value, sets the head into a new position. A timer is set, and if this takes more than 5 seconds, transition to headReadyLookGoal. Again it continuously switches cameras.
  • headReadyLookGoal.lua
    Determines the closest attack between attackAngle and defendAngle. This state only uses the top camera. The 'yaw' which is used to set the head position is determined based on the closest attack. If the time takes more than 1.5 seconds, then it either times out or determines it is lost, and both of these transition back to headReady.
  • headTrack.lua
    Transitions from headStart. This state uses only the bottom camera. The robot locates the ball and updates its head based on this location. The timer either times out and transitions to headLookGoal or determines that the ball is lost and transitions to headScan.
  • headScan.lua
    Continuously switches between the two cameras. It updates the head position by continiously scanning left-right and up-down. The scan is started in the ball's last known direction. If the ball it found, it transitions back to headTrack, and if it times out, it stays in the same state and starts over.
  • headLookGoal.lua
    Uses only the top camera. This also determines the closest attack between attackAngle and defendAngle and updates the head position based on the closest attack. If it times out it transitions to headTrack, and if the ball is lost, it transitions to headSweep.
  • headSweep.lua
    Only uses the top camera. It finds the head angle and determines the new head position based on this value. If the time has been more than 2 seconds, it transitions to headTrack.