Learning Systems and Robotics Lab, Munich & Toronto

SwarmGPT meets the new Crazyflie Colour LED Deck 🚁🌈

We are excited to share our recent collaboration with Bitcraze, showcasing how Bitcraze's new Crazyflie Colour LED Deck takes drone swarm choreographies to the next level.

The programmable LED Deck adds a powerful visual layer to our swarm performance: synchronization, grouping, role assignment, and timing become immediately visible in the air. This not only makes swarm behaviour more expressive and engaging, but also easier to inspect and understand in real time.

In parallel, our recent work, SwarmGPT (https://ieeexplore.ieee.org/document/11197931), explores how large language models can be used as a high-level interface for designing such choreographies. Instead of manually specifying trajectories, SwarmGPT translates natural-language intent and music-derived cues into structured swarm behaviours that are executed safely using model-based planning and control.

Together with Bitcraze, we demonstrate SwarmGPT on the Crazyflie brushless platform equipped with the new Colour LED Deck, highlighting how hardware innovation and swarm research can seamlessly reinforce each other. Hope you enjoy the video and happy holidays!

Special thanks to Fredrik Ehrenstråle and Marcel Rath for creating the soundtrack and for making the video.

-collaborations

Looking for an easy-to-use way to deploy your VLAs and diffusion policies on real robots (maybe even before the ICRA video deadline next week 👀)?

🚀 Excited to share our latest work: 𝗖𝗥𝗜𝗦𝗣 - 𝗖𝗼𝗺𝗽𝗹𝗶𝗮𝗻𝘁 𝗥𝗢𝗦2 𝗖𝗼𝗻𝘁𝗿𝗼𝗹𝗹𝗲𝗿𝘀 𝗳𝗼𝗿 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴-𝗕𝗮𝘀𝗲𝗱 𝗠𝗮𝗻𝗶𝗽𝘂𝗹𝗮𝘁𝗶𝗼𝗻 𝗣𝗼𝗹𝗶𝗰𝗶𝗲𝘀 𝗮𝗻𝗱 𝗧𝗲𝗹𝗲𝗼𝗽𝗲𝗿𝗮𝘁𝗶𝗼𝗻!

Why CRISP?
Most learning-based policies (diffusion, VLAs, etc.) output low-frequency or discontinuous commands, which don’t play nicely with real hardware. CRISP bridges that gap with lightweight, compliant, torque-based controllers built on hashtag control.

Features:
🐍 𝗣𝘆𝘁𝗵𝗼𝗻 𝗶𝗻𝘁𝗲𝗿𝗳𝗮𝗰𝗲 — control your robot around without worrying about topics, spinning, etc., but still keep full ROS2 flexibility.
🔁 𝗚𝘆𝗺𝗻𝗮𝘀𝗶𝘂𝗺 𝗲𝗻𝘃𝗶𝗿𝗼𝗻𝗺𝗲𝗻𝘁 — use teleoperation to record robot and visual data in hashtag format & seamlessly deploy learning-based policies.
🎥 𝗗𝗲𝗺𝗼-𝗿𝗲𝗮𝗱𝘆 — examples for single-arm & bimanual manipulation with the Franka FR3.
🤖 𝗖𝗮𝗿𝘁𝗲𝘀𝗶𝗮𝗻 & 𝗷𝗼𝗶𝗻𝘁-𝘀𝗽𝗮𝗰𝗲 𝗰𝗼𝗻𝘁𝗿𝗼𝗹𝗹𝗲𝗿𝘀 – built for torque control and smooth, compliant interaction.
🚫 𝗡𝗼 𝗠𝗼𝘃𝗲𝗜𝘁 𝗼𝗿 𝗵𝗲𝗮𝘃𝘆 𝗽𝗹𝗮𝗻𝗻𝗶𝗻𝗴 𝘀𝘁𝗮𝗰𝗸𝘀 required, ready to use.

Our goal: Make deploying learning-based methods on real robots as frictionless as possible, reducing the gap between data collection, simulation, and deployment.

📄 Paper: https://arxiv.org/abs/2509.06819
💻 Code: https://github.com/utiasDSL/crisp_controllers
🌐 Website: https://utiasdsl.github.io/crisp_controllers/

This work is the result of a great team effort by Daniel San José Pro, Oliver Hausdörfer, Ralf Römer, Maximilian Dösch, and Martin Schuck!

Some summer entertainment needed? Prof. Schoellig is giving a virtual seminar tomorrow Tuesday, Aug 26th, 13:00-14:30 (CEST) / 7:00-8:30 (EST) on "Aerial Swarms: From Safe Motion Planning to Language-Based Interaction." Teaser video below.

Join live (or watch the recording): https://youtu.be/fY5CAQrRJRI?si=Nl4UuvNyuST53KfI

This is a pre-event of the IROS workshop on Multi-Agent Cooperative Systems and Swarm Robotics in the Era of Generative AI workshop with amazing fellow speakers: https://lnkd.in/dzZtb_PF