Heterogeneous Real-Time and Visual Servoing Systems

Interface Boards, Relays and Cabling
Polymorphic Runtime Environment
Levitating Ball and Camera

Context

Today's embedded systems (ranging from small mobile phones to large-scale distributed designs) rarely feature only one type of processing resource (e.g., a processor or an FPGA) and only locally attached connections to the outside world (e.g., to sensors and actuators). Instead, multiple - potentially heterogeneous - processing units are interconnected by a communication network to satisfy the requirements of the job in hand. Most importantly, these are reliability, predictability, performance and overall system cost.

Interface and Communication Issues at hand

Besides these computational aspects both design and verification of the communication subsystems (managing the data flow not only on one local processing unit, but also between several distributed nodes) become more and more challenging. Locally, this is caused by continuously increasing data rates and the need for flexible data routing during runtime due to reconfiguration of the processing resources. On system-level, data routing has to be synchronized to global computation planning to enable smooth fail-over in case computation node or communication link outages.

Experimental Setups

To evaluate both functional and temporal aspects of our proposed architectures, we rely on a Magnetic Levitating Sphere as a Representative Use Case. Continuous levitation is achieved via Visual Servoing, i.e., optically detecting the sphere’s current position followed by computing the required actuation signal within the order of milliseconds. The Visual Servoing System (consisting of actuator, sensor and heterogeneous Processing Platform) relies on Gigabit Ethernet as a Communication Backbone. With Ethernet now being widely used in industrial automation and many Visual Servoing Applications requiring similar low-latency Image Acquisition and Processing, our Experimental Setup thus serves as a representative use case for such scenarios

Experiments with a nearly-zero latency sensor (i.e., a light barrier) have shown that stable levitation is possible with sampling rates as low as 125 Hz. With Visual Servoing, however, the increased sensing delay (due to image acquisition and processing) requires compensation via higher rates.

The Magnetic Levitating Sphere in action!

Theses & Internships

Currently, the following thesis topics are available - please get in touch for more details:

  • Development of an HDMI interface on Zynq FPGAs in VHDL

The following theses are already assigned or completed:

Please refer to the RCS theses page for additional topics.

Publications

  • Geier, Martin; Pitzl, Florian; Chakraborty, Samarjit: GigE Vision Data Acquisition for Visual Servoing Using SG/DMA Proxying. ACM/IEEE Symposium on Embedded Systems for Real-Time Multimedia (ESTIMedia) (ESTIMedia'16), ACM, 2016, 19--28 more… BibTeX Full text ( DOI ) Full text (mediaTUM)

Team

The following individuals are currently working on this project:

  • Martin Geier focuses on the design of network and communication sub-systems, both between two and inside one processing unit(s).