Torsten Kroeger

Stanford Artificial Intelligence Laboratory

Control Architectures

This research has been conducted at the Institut für Robotik und Prozessinformatik at the Technische Universität Braunschweig, Germany.

Motivation: Easy Multi-Sensor Integration and Sensor-Guided Motion Control

The integration of sensors belongs to one of the most important future domains for achieving future advancements in robot motion control systems. As even nowadays commercial control units are rarely open for sensor integration, sensor signals are often (even in research institutions) used for sensor-based trajectory and path adaptations. This is a very implicit way of sensor-based control (e.g., force/torque control or visual servo control), and it—of course—already leads to great benefits compared to simple point to point operations, but in such systems, sensors are not part of the feedback control loop, and it is not possible at all to instantaneously react to unforeseen (sensor) events. On the other hand, we can find plenty of approaches using sensors in the feedback loops. Force/torque control has had a dedicated community since the beginning of the 1980s as well as the field of visual servo control has since the beginning of the 1990s; we can find plenty of publications in both areas, but the feature of instantaneous switchings is of fundamental relevance for many real-world problems and applications.

A 'Hacked' Stäubli System

In order to perform real-world experiments using new and highly advanced robot motion control architectures, special control hardware is required. The original controller of a six-joint Stäubli RX60 industrial manipulator was replaced, and the frequency inverters were directly interfaced. Three PCs running with QNX as real-time operating system perform a control rate of 10 kHz for the joint controllers; a hybrid switched-system controller is used for Cartesian space control and runs at a frequency of 1 kHz (in some experiments up to 4 kHz). An RX60 manipulator executing a sensor guided assembly task is shown on the picture below. Its control cabinet stands in the background. The human machine interface (HMI) is mounted with a pivot arm to the control cabinet and provides excellent possibilities for manual control and teach-in programming. Besides the most important operational controls a touch screen panel and a Space Mouse is available.

  • Sample set-up for a force-guided assembly task with a Stäubli RX60 robot manipulator (click to enlarge). Sample set-up for a force-guided assembly task with a Stäubli RX60 robot manipulator (click to enlarge).

  • Control terminal with Space Mouse and Stäubli RX60 robot manipulator (click to enlarge). Control terminal with Space Mouse and Stäubli RX60 robot manipulator (click to enlarge).

However, this work does not focus on a convenient HMI, but moreover on open interfaces, extensibility, scalability, robustness for external disturbances, and especially on the requirement to keep the usage of sensors entirely open. The above-mentioned possibilities for manual control a teach-in programming are only the necessary base.

Unless parts of the original control system match the new requirements, the parts are reused in the new system. Essentially, the power electronics, i.e. the frequency inverters and their accessories, have to be mentioned here. For manipulator operation, at least one control PC is required. A motion control board within this PC constitutes the interface to an electronic adaptation assembly, which delivers velocity set points to each frequency inverter on the one hand side, and which provides the joint positions to the control computer. In addition, I/O ports for further periphery (brakes, inverter control, valves, etc.) are provided. One major requirement is to make the absolute manipulator position permanently available to the controller. Therefore, an electronic assembly group for position tracking was developed. This one transmits the current position with an accuracy of 13 Bits via the serial port to the control PC. To guarantee the board's functionality at AC power failures and during transports, the position-tracking unit supplied by accumulators if required. The communication vie the serial port occurs only at during the start-up procedure, afterwards, die control computer receives the position information from the motion control board.

  • Control cabinet containing three PCs and the power electronics (click to enlarge). Control cabinet containing three PCs and the power electronics (click to enlarge).

Modular Hard- and Software

To assure an excellent dynamic behavior, signal quality loss and signal latency have to be kept small, such that the necessary bandwidth of the components for signal adaptation between PC and drive electronic was determined preliminary to the development. Interfaces to the control computer are such open, that the user can freely decide how to design his software architecture. The signals for basic manipulator control are only the joint velocity set points for the drive controllers and the position inputs. The proposed architecture uses a joint position controller, which is provided as interface to higher software layers. Besides the possibility to run all processes (joint position control, hybrid control, trajectory planning, user application, etc.) on one machine, is it useful to swap further processes out to other PCs and to take a real time communication system between them.

To follow the introductory mentioned aim of multi-sensor integration, the following figure depicts a suggestion for a possible architecture:

  • Exemplary control architecture with three PCs (click to enlarge). Exemplary control architecture with three PCs (click to enlarge).

As a result, the requirement of a modular software environment is preeminent, besides a modular and scalable hardware environment. A real-time capable middleware running on a distributed system of several computer nodes for all intercommuncation issues promises to be a good approach. MiRPA (Middleware for Robotics and Process Control) is an in-house solution fitting these demands. Further information can be found in the middleware section.

Please refer to the following publications:

  • JediBot - Experiments in Human-Robot Sword-Fighting.
    In Proc. of the International Symposium on Experimental Robotics, , June 2012. BibTeX

    @inproceedings{Kroeger:12b,
    	author = {{T.~Kr{\"o}ger} and K.~Oslund and T.~Jenkins and D.~Torczynski and N.~Hippenmeyer and R.~B.~Rusu and O.~Khatib},
    	title = {JediBot\,---\,Experiments in Human-Robot Sword-Fighting},
    	booktitle = {Proc. of the International Symposium on Experimental Robotics},
    	month = {June},
    	address = {Qu\'{e}bec City, Canada},
    	year = {2012}
    }
    			
  • Simple and Robust Visual Servo Control of Robot Arms Using an On-Line Trajectory Generator.
    In Proc. of the IEEE International Conference on Robotics and Automation, pp. 4862-4869, Saint Paul, MN, USA, May 2012. BibTeX

    @inproceedings{Kroeger:12a,
    	author = {T.~Kr{\"o}ger and J.~Padial},
    	title = {Simple and Robust Visual Servo Control of Robot Arms Using an On-Line Trajectory Generator},
    	booktitle = {Proc. of the IEEE International Conference on Robotics and Automation},
    	month = {May},
    	pages = {4862--4869},
    	address = {Saint Paul, MN, USA},
    	year = {2012}
    }
    			
  • A Depth Space Approach to Human-Robot Collision Avoidance.
    In Proc. of the IEEE International Conference on Robotics and Automation, pp. 338-345, Saint Paul, MN, USA, May 2012. BibTeX

    @inproceedings{Flacco:12,
    	author = {F.~Flacco and T.~Kr{\"o}ger and A.~De~Luca and O.~Khatib},
    	title = {A Depth Space Approach to Human-Robot Collision Avoidance},
    	booktitle = {Proc. of the IEEE International Conference on Robotics and Automation},
    	month = {May},
    	pages = {338--345},
    	address = {Saint Paul, MN, USA},
    	year = {2012}
    }
    			
  • Robot Motion Control During Abrupt Switchings Between Manipulation Primitives.
    In Workshop on Mobile Manipulation at the IEEE International Conference on Robotics and Automation, Shanghai, China, May 2011. BibTeX

    @inproceedings{Kroeger:11a,
    	author = {T.~Kr{\"o}ger and B.~Finkemeyer},
    	title = {Robot Motion Control During Abrupt Switchings Between Manipulation Primitives},
    	month = {May},
    	booktitle = {Workshop on Mobile Manipulation at the {I}{E}{E}{E} International Conference on Robotics and Automation},
    	address = {Shanghai, China},
    	year = {2011}
    }
    			
  • Manipulation Primitives - A Universal Interface Between Sensor-Based Motion Control and Robot Programming.
    In Robot Systems for Handling and Assembly. Springer Tracts in Advanced Robotics, Vol. 67, Springer, pp. 293-313, 2010. BibTeX

    @incollection{Kroeger:10b,
    	author = {T.~Kr{\"o}ger and B.~Finkemeyer and F.~M.~Wahl},
    	title = {Manipulation Primitives\,---\,{A} Universal Interface Between Sensor-Based Motion Control and Robot Programming},
    	booktitle = {Robot Systems for Handling and Assembly},
    	editor = {D.~Sch{\"u}tz and F.~M.~Wahl},
    	address = {Berlin, Heidelberg, Germany},
    	series = {Springer Tracts in Advanced Robotics},
    	publisher = {Springer},
    	volume = {67},
    	pages = {293--313},
    	edition = {first},
    	year = {2010}
    }
    			
  • Stabilizing Hybrid Switched Motion Control Systems with an On-Line Trajectory Generator.
    In Proc. of the IEEE International Conference on Robotics and Automation, pp. 4009-4015, Anchorage, AK, USA, May 2010. BibTeX

    @inproceedings{Kroeger:10c,
    	author = {T.~Kr{\"o}ger and F.~M.~Wahl},
    	title = {Stabilizing Hybrid Switched Motion Control Systems with an On-Line Trajectory Generator},
    	booktitle = {Proc. of the IEEE International Conference on Robotics and Automation},
    	month = {May},
    	pages = {4009--4015},
    	address = {Anchorage, AK, USA},
    	year = {2010}
    }
    			
  • .
    In Workshop on Innovative Robot Control Architectures for Demanding (Research) Applications - How to Modify and Enhance Commercial Controllers at the IEEE International Conference on Robotics and Automation, pp. 38-45, Anchorage, AK, USA, May 2010. BibTeX

    @inproceedings{Kubus:10e,
    	author = {D.~Kubus and A.~Sommerkorn and T.~Kr{\"o}ger and J.~Maa{\ss} and F.~M.~Wahl},
    	title = {Low-Level Control of Robot Manipulators: Distributed Open Real-Time Control Architectures for St{\"a}ubli RX and TX Manipulators},
    	month = {May},
    	booktitle = {Workshop on Innovative Robot Control Architectures for Demanding (Research) Applications - How to Modify and Enhance Commercial Controllers at the {I}{E}{E}{E} International Conference on Robotics and Automation},
    	address = {Anchorage, AK, USA},
    	pages = {38--45},
    	year = {2010}
    }
    			
  • The Adaptive Selection Matrix - A Key Component for Sensor-Based Control of Robotic Manipulators.
    In Proc. of the IEEE International Conference on Robotics and Automation, pp. 3855-3862, Anchorage, AK, USA, May 2010. BibTeX

    @inproceedings{Finkemeyer:10a,
    	author = {B.~Finkemeyer and T.~Kr{\"o}ger and F.~M.~Wahl},
    	title = {The Adaptive Selection Matrix\,---\,{A} Key Component for Sensor-Based Control of Robotic Manipulators},
    	booktitle = {Proc. of the IEEE International Conference on Robotics and Automation},
    	month = {May},
    	pages = {3855--3862},
    	address = {Anchorage, AK, USA},
    	year = {2010}
    }
    			
  • A Manipulator Plays Jenga.
    IEEE Robotics and Automation Magazine. Vol. 15, No. 3, pp. 79-84, September 2008. BibTeX

    @article{Kroeger:08b,
    	author = {T.~Kr{\"o}ger and B.~Finkemeyer and S.~Winkelbach and S.~Molkenstruck and L.-O.~Eble and F.~M.~Wahl},
    	title = {A Manipulator Plays {J}enga},
    	journal = {IEEE Robotics and Automation Magazine},
    	month = {September},
    	volume = {15},
    	number = {3},
    	pages = {79--84},
    	year = {2008}
    }
    			
  • Demonstration of Multi-Sensor Integration in Industrial Manipulation (Poster).
    In Proc. of the IEEE International Conference on Robotics and Automation, pp. 4282-4284, Orlando, FL, USA, May 2006. BibTeX

    @inproceedings{Kroeger:06,
    	author = {T.~Kr{\"o}ger and B.~Finkemeyer and S.~Winkelbach and S.~Molkenstruck and L.-O.~Eble and F.~M.~Wahl},
    	title = {Demonstration of Multi-Sensor Integration in Industrial Manipulation (Poster)},
    	booktitle = {Proc. of the IEEE International Conference on Robotics and Automation},
    	address = {Orlando, FL, USA},
    	month = {May},
    	pages = {4282--4284},
    	year = {2006}
    }
    			
  • Demonstration of Multi-Sensor Integration in Industrial Manipulation (Video).
    In Proc. of the IEEE International Conference on Robotics and Automation, Orlando, FL, USA, May 2006. BibTeX

    @inproceedings{Kroeger:06a,
    	author = {T.~Kr{\"o}ger and B.~Finkemeyer and S.~Winkelbach and S.~Molkenstruck and L.-O.~Eble and F.~M.~Wahl},
    	title = {Demonstration of Multi-Sensor Integration in Industrial Manipulation (Video)},
    	booktitle = {Proc. of the IEEE International Conference on Robotics and Automation},
    	address = {Orlando, FL, USA},
    	month = {May},
    	year = {2006}
    }
    			
  • A Task Frame Formalism for Practical Implementations.
    In Proc. of the IEEE International Conference on Robotics and Automation, pp. 5218-5223, New Orleans, LA, USA, April 2004. BibTeX

    @inproceedings{Kroeger:04,
    	author = {T.~Kr{\"o}ger and B.~Finkemeyer and F.~M.~Wahl},
    	title = {A Task Frame Formalism for Practical Implementations},
    	booktitle = {Proc. of the IEEE International Conference on Robotics and Automation},
    	address = {New Orleans, LA, USA},
    	month = {April},
    	pages = {5218--5223},
    	year = {2004}
    }
    			
  • Compliant Motion Programming: The Task Frame Formalism Revisited.
    In Mechatronics and Robotics, pp. 1029-1034, Aachen, Germany, September 2004. BibTeX

    @inproceedings{Kroeger:04a,
    	author = {T.~Kr{\"o}ger and B.~Finkemeyer and U.~Thomas and F.~M.~Wahl},
    	title = {Compliant Motion Programming: {T}he Task Frame Formalism Revisited},
    	booktitle = {Mechatronics and Robotics},
    	address = {Aachen, Germany},
    	month = {September},
    	pages = {1029--1034},
    	year = {2004}
    }
    			
  • Manipulation Primitives as Interface between Task Programming and Execution.
    In Workshop on Issues and Approaches to Task Level Control at the IEEE/RSJ International Conference on Intellegent Robots and Systems, Sendai, Japan, September 2004. BibTeX

    @inproceedings{Kroeger:04c,
    	author = {T.~Kr{\"o}ger and B.~Finkemeyer and F.~M.~Wahl},
    	title = {Manipulation Primitives as Interface between Task Programming and Execution},
    	booktitle = {Workshop on Issues and Approaches to Task Level Control at the {I}{E}{E}{E}/{R}{S}{J} International Conference on Intellegent Robots and Systems},
    	address = {Sendai, Japan},
    	month = {September},
    	year = {2004}
    }
    			
  • Error-Tolerant Execution of Complex Robot Tasks Based on Skill Primitives.
    In Proc. of the IEEE International Conference on Robotics and Automation, Vol. 3, pp. 3069-3075, Taipei, Taiwan, September 2003. BibTeX

    @inproceedings{Thomas:03,
    	author = {U.~Thomas and B.~Finkemeyer and T.~Kr{\"o}ger and F.~M.~Wahl},
    	title = {Error-Tolerant Execution of Complex Robot Tasks Based on Skill Primitives},
    	booktitle = {Proc. of the IEEE International Conference on Robotics and Automation},
    	volume = {3},
    	address = {Taipei, Taiwan},
    	month = {September},
    	pages = {3069--3075},
    	year = {2003}
    }