Software Engineer in Robotics and Computer Vision
I am a skilled C++ software engineer with 8⁺ years of experience in Humanoid Robotics and Computer Vision. I have worked as a research engineer for both CNRS (France) and AIST (Japan) on control of state-of-the-art full size Humanoid Robots. Our work has lead to the development of the mc_rtc robotics framework.
Projects
I have worked on the following 21 projects, and more!
2016-12-31
Present
I am one of the lead developpers of the mc_rtc robotics framework. It has been used extensively at both AIST and LIRMM research labs. Notable achievements with the framework include:
- DARPA Robotic Challenge Finals
- COMANOID project: large scale aircraft manufacturing in partnership with Airbus
- ANA Avatar XPrize Finals
- Supports 20+ Robots (Humanoids, Manipulators and Quadrupeds)
- Walking, including stair climbing (see https://github.com/jrl-umi3218/lipm_walking_controller)
- Widely used within the research community
- Fully open-source under BSD-2
Roles
- Co-Lead Developper and Maintainer
- Implementation of industrial demonstrators
- Support for PhD and PostDocs (Paper experiments, framework development)
- Integration with robots and sensors
C++
MC_RTC
ROS
ROS2
REAL-TIME CONTROL
HUMANOID ROBOTS
HRP-2
HRP-4
HRP-5P
PEPPER
NAO
SAWYER
PANDA
KUKKA LWR
2023
Centre des Arts, Enghien-les-Bains
This Robotic Dance Performance was produced as part of the Co-Évolution, Co-Création et Improvisation Homme-Machine (CECCI-H2M) project of the EUR d'ArTeC, led by Pr. Chu-Yin Chen, the Image Numérique et Réalité Virtuelle (INREV) research team of the labo. AIAC of Université Paris 8, in collaboration with CDA d'Enghien-les-Bains and LIRMM Interactive Digital Human (IDH) team.
C++
MC_RTC
XSENS MVN
TELEOPERATION
ON-STAGE
OPEN-SOURCE
2023
AIST, Japan - LIRMM, France - Los Angeles, USA
This video presents experimental results demonstrating the performance of the avatar system developed by Team JANUS to compete at the ANA Avatar XPRIZE - Finals global competition.
FINALIST
INTERNATIONAL COMPETITION
MC_RTC
HUMANOID ROBOTICS
C++
AUGEMENTED REALITY
TELEOPERATION
UNITY
ANA AVATAR XPRIZE
2020-10-01
AIST - Tsukuba - Japan
Closed-loop MPC with Dense Visual SLAM-Stability through Reactive Stepping - Arnaud Tanguy, Daniele de Simone, Andrew Comport, Giuseppe Oriolo, Abderrahmane Kheddar
Roles
- Implementation of the whole demo
- Integration of the HRP-5P robot with mc_rtc
C++
MC_RTC
SLAM
MODEL PREDICTIVE CONTROL
2015-01-31
2018-12-31
Airbus - Saint Nazaire
COMANOID (Multi-Contact Collaborative Humanoids in Aircraft Manufacturing) is a RIA four-year European research project that started in January 2015 as part of the Horizon H2020 program. The project focuses on developping abilities of humanoid robots to evolve in complex enviroment -namely an aircraft fuselage- through whole body multi-contact planning motion with embeded 6D dense SLAM localization.
Roles
- Integration of Dense Visual SLAM for Localization and Mapping within the Airplane
- Accurate Control of the Humanoid robot based on Visual SLAM
BEST PAPER AWARD
C++
MC_RTC
SLAM
MANUFACTURING
2015-01-31
2018-12-31
Confidential - Industrial Factory
To efficiently achieve complex humanoid loco-manipulation tasks in industrial contexts, we propose a combined vision-based tracker-localization interplay integrated as part of a task-space whole-body optimization control. To achieve good perception complementarity between manipulation and localization, a new fast dense 3D model-based tracking using wide-angle depth image is developed and used in conjunction with a simultaneous localization and mapping software. Our approach allows humanoid robots, targeted for industrial manufacturing, to manipulate and assemble large-scale objects while walking. It is assessed with experiments consisting in rolling and assembling in an unwinder a heavy and wide bobbin using bimanual grasping and bipedal locomotion at a time. This experimental use-case is found in some large-scale manufacturing where bobbins are enrolled with various materials (cables, papers, rubbers, etc.). The same experiments are made using two different humanoid robots of the same family.
Roles
- Software Engineering for the demonstrator's implementation
- Consulting for the implementation of the Dense Visual Tracker for the Bobbin
C++
MC_RTC
SLAM
MANUFACTURING
2014-10
2018-11
I3S Sophia-Antipolis - LIRMM Montpellier - AIST Tskububa, Japan
This thesis deals with the problem of localizing and controlling humanoid robots with respect to its environment, as observed by its on-board sensors. Dense visual SLAM, consisting in the simultaneous estimation of a 3D map of the environment and of the robot localization within that maps is exploited to extend and robustify multi-contact planning and control. Establishing and exploiting robot-environment contacts allows the accomplishment of both locomotion and manipulation tasks. Uncertainties in the initial robot posture, and perturbations arising from improper contact-modelling and external causes are accounted for by observing the state of the robot and its environment. A whole-body calibration method is also proposed, so that robust knowledge of the robot's kinematic structure is known, a prerequisite to all robot-environment interaction tasks. Finally, a walking method based on model predictive control is robustified by taking into account large perturbations, and adjusting the footstep and center-of-mass trajectories accordingly to guarantee stability while accomplishing desired objectives.Several experiments on an HRP-2Kai and an HRP-4 humanoid robots are presented and discussed to illustrate and validate each of the proposed methods.
THESIS
SLAM
HUMANOID ROBOTICS
C++
MC_RTC
2015
Las Vegas - USA
- Took part in major worldwide competition lead by US military research agency
- Developed robot-control strategies for disaster response scenarios, designed specifically to emulate the conditions of Fukushima nuclear plant
- Ranked 10 out of 23 leading labs
Roles
- Localization and Mapping
- Object Registration (ICP)
FINALIST
INTERNATIONAL COMPETITION
TEAM AIST-NEDO
MC_RTC
SLAM
ITERATIVE CLOSEST POINT
2023
2024-04
Developpement of an mc_rtc controller for evaluation of novel knee orthosis devices.
C++
MC_RTC
ORTHOSIS
MEDICAL ROBOTICS
PANDA
FORCE CONTROL
2023
2024-04
BoneTag is developping a novel sensor to track the motion of knee prosthesis in patients in real-time. I developped a controller using Franka Emika's Panda robot to evaluate the accuracy of the sensor and help improve their deep-learning based tracker.
C++
MC_RTC
MEDICAL ROBOTICS
PANDA
SENSOR INTEGRATION
2022
LIRMM - Montpellier - France
Robots usually establish contacts at rigid surfaces with near-zero relative velocities. Otherwise, impact-induced energy propagates in the robot’s linkage and may cause irreversible damage to the hardware. Moreover, abrupt changes in task-space contact velocity and peak impact forces also result in abrupt changes in robot joint velocities and torques;which can compromise controllers’ stability, especially for those based on smooth models. In reality, several tasks would require establishing contact with moderately high velocity. We propose to enhance task-space multi-objective controllers formulated as a quadratic program (QP) to be resilient to frictional impacts in three dimensions. We devise new constraints and reformulate the usual ones to be robust to the abrupt joint state changes mentioned earlier. The impact event becomes a controlled process once the optimal control search space is aware of: (1) the hardware-affordable impact bounds and (2) the analytically-computed feasible set (polyhedra) that constrain post-impact critical states. Prior to and nearby the targeted contact spot, we assume, at each control cycle, that the impact will occurat the next iteration. This somewhat one-step preview makes our controller robust to impact time and location. To assess our approach, we experimented its resilience to moderate impacts with the Panda manipulator and achieved swift grabbing tasks with the HRP-4 humanoid robot.
PAPER
C++
MC_RTC
HRP-4
LOGISTICS
IMPACT
2021-03
AIST - Tsukuba - Japan
The main video of Online Object Searching by a Humanoid Robot in an Unknown Environment, ICRA-RAL2021. M.Tsuru, A.Escande, A.Tanguy, K.Chappellet, and K.Harada, AIST, Japan.
PAPER
C++
MC_RTC
HRP-2KAI
SLAM
MAPPING
LOCALIZATION
EXPLORATION
OBJECT MANIPULATION
2021
LIRMM - Montpellier - France
In this video, the HRP-4 humanoid robot demonstrates contact interchanges from fully-fixed to multi-sliding and also shuffling of the foot. This scenarios illustrate the performance of the proposed whole-body control strategy for humanoid robots in multi-contact settings that enables switching between fixed and sliding contacts under active balance. We compute, in real-time, a safe center-of-mass position and wrench distribution of the contact points based on the Chebyshev center. Our solution is formulated as a quadratic programming problem without a priori computation of balance regions.
C++
MC_RTC
FORCE CONTROL
STABILITY
2020-10-01
AIST - Tsukuba - Japan
C++
MC_RTC
VISUAL SERVOING
MANUFACTURING
2020
LIRMM - Montpellier - France
In this video, we present an open-source software interface, called mc_naoqi, that allows to perform whole-body task-space Quadratic Programming based control, implemented in mc_rtc framework, on the SoftBank Robotics Europe humanoid robots. We showcase the use of the developed open-source tools for running the human-robot close contact interaction experiments with real human subjects inspired from assistance scenarios. Finally, we describe in detail the entire system including the control interface, associated robot description packages and modules and how the developed tools connect to the mc_rtc controller.
Roles
- Integration of the robot Pepper with mc_rtc
- Help with controller implementation
C++
MC_RTC
PEPPER
INTEGRATION
2019
LIRMM - Montpellier - France
HRP-4 keeps balance in multi-contact while sliding its hand on a wall. To maintain balance, the robot holds its center of mass (CoM) inside a CoM support area (CSA) that takes all contacts and their contact modes into consideration. When we want to apply more force on the hand contact, the controller moves its CoM towards the wall accordingly. This video compares the performance of this control strategy with a baseline where the CoM is simply kept between the feet. Pushing and sliding operations are demonstrated.
Roles
- Implemantation of the demonstrator
C++
MC_RTC
FORCE CONTROL
STABILITY
2016
AIST - Tsukuba - Japan
Closed-loop RGB-D SLAM multi-contact control for humanoid robots A Tanguy, P Gergondet, AI Comport, A Kheddar System Integration (SII), 2016 IEEE/SICE International Symposium on, 51-57
BEST PAPER FINALIST AWARD
C++
MC_RTC
SLAM
MODEL PREDICTIVE CONTROL
STAIR CLIMBING
OBJECT MANIPULATION
2014-10-31
2015-02-28
Robohow is a four-year European research project that started in February 2012. It aims at enabling robots to competently perform everyday human-scale manipulation activities - both in human working and living environments. In order to achieve this goal, Robohow pursues a knowledge-enabled and plan-based approach to robot programming and control. The vision of the project is that of a cognitive robot that autonomously performs complex everyday manipulation tasks and extends its repertoire of such by acquiring new skills using web-enabled and experience-based learning as well as by observing humans.
2014-03-31
2014-09-30
Visual odometery consists of tracking the position and orientation of camera-like sensors as they are moving. As the camera moves, drift in the estimated camera pose can accumulate. Loop-closure detection is a technique to minimise the pose drift, by exploiting knowledge about previously visited locations. The goal of this project was to explore the feasibility of using convolutional neural networks to perform place-recognition.
Roles
- Integrated Siamese Neural Networks into the Caffe deep learning framework
- Trained and evaluated the use of convolutional neural networks to perform place recognition
- Implemented a loop-closure detection algorithm in C++ and Python
C++
PYTHON
DEEP LEARNING
CONVOLUTIONAL NEURAL NETWORKS
CAFFE FRAMEWORK
CUDA
SLAM
LOOP-CLOSURE DETECTION
2008-06-30
2012-03-31
Fotowall is an opensource tool that creates patchworks with photos, text, live videos from your webcam. The development of this application helped me to build up my C++ experience along with my knowledge of the framwork Qt. It also developed my capacities of teamwork, and I learned to use project managment tools like git.
C++
QT
SELF-TAUGHT
GIT
COLLABORATIVE PROJECT
OPEN SOURCE
2012-01-31
2012-03-31
3D Game implemented specifically for visually impaired people, that can be played based on listening to simulated engine sounds.
JAVA
OPENGL
GAME DEVELOPMENT
COMPUTER GRAPHICS
COMPUTER SCIENCE
UNIVERSITY
Robots
I have worked with the following 16 robots, and more!
HRP-2Kai
HRP-4
HRP-4CR
HRP-5P
Talos
NAO
Pepper
HOAP-3
Sawyer
UR5e
Panda
Bazar
Fetch
Aliengo
Yaskawa HC10DT
FANUC R-1000iA
HRP-2Kai
HRP-4
HRP-4CR
HRP-5P
Talos
NAO
Pepper
HOAP-3
Sawyer
UR5e
Panda
Bazar
Fetch
Aliengo
Yaskawa HC10DT
FANUC R-1000iA
Work Experience
Robotics Engineer - Humanoid Robotics @ Centre national de la recherche scientifique
2021-10-312024-04-31Montpellier, Occitanie, France- Live robot dance performance on stage at Centre des Arts Enghain-les-Bains
- ANA Avatar XPrize Finalist
- Large-scale loco-manipulation demonstration on-sight for an industrial partner (confidential)
- Industrial project (confidential) - Using a large 7 DoF robot Fanuc R1000IA to unload a containter full of randomly stacked objects
C++HUMANOID ROBOTICSUNITYAUGMENTED REALITYANA AVATAR XPRIZEMC_RTC FRAMEWORKResearch Engineer - Humanoid Robotics @ National Institute of Advanced Industrial Science and Technology (AIST)
2019-11-302021-04-30Tsukuba, Japan- Responsible for the unification of two control frameworks: mc_rtc and HMC
- Assisting students and researchers
- Responsible for experiments and demonstrations on the robots (HRP-5P, HRP-2Kai, Sawyer, Panda, etc)
C++HUMANOID ROBOTICSUNITYAUGMENTED REALITYANA AVATAR XPRIZEMC_RTC FRAMEWORKResearch Engineer in Humanoid Robotics @ Centre national de la recherche scientifique
2018-10-312019-11-30Montpellier, France- Responsible for the integration of visual SLAM for robust localisation of the HRP4 Humanoid Robot, notably withing an Airbus aircraft for the final demonstration of the H2020 COMANOID project
- Responsible for the development of all experiments performed on the HR4 at LIRMM
- Integration of new methods and tools within our control framework
C++HUMANOID ROBOTICSSLAMCOMANOID H2020MC_RTC FRAMEWORKPhD in Humanoid Robotics and Computer Vision @ CNRS - Centre national de la recherche scientifique
2014-10-312018-11-30Sophia-Antipolis, France | Tsukuba, Japan | Montpellier, France- Improved robot-environment interactions and reliability with state-of-the-art real-time visual localization and mapping from color and depth images (SLAM)
- Under CNRS contract funded by COMANOID European research project
- Worked in three major international labs: Joint Robotics Laboratory - Tsukuba, Japan ; LIRMM - Montpellier, France ; I3S-CNRS - Sophia-Antipolis, France
- Obtained best paper finalist award at IEEE International Symposium on System Integration for the paper on "Closed-loop RGB-D SLAM Multi-contact Control for Humanoid Robots"
HUMANOID ROBOTICSVISUAL SLAMC++ROSMC_RTC FRAMEWORKHUMANOID ROBOTSOPTIMIZATIONQUADRATIC PROGRAMMINGMODEL PREVIEW CONTROLDARPA Robotics Challenge Finalist @ Defense Advanced Research Projects Agency (DARPA)
2015-06-302015-06-30United States- Took part in major worldwide competition lead by US military research agency
- Developed robot-control strategies for disaster response scenarios, designed specifically to emulate the conditions of Fukushima nuclear plant
- Ranked 10 out of 23 leading labs
INTERNATIONAL COMPETITIONC++ROSSLAMICPHUMANOID ROBOTICSHRP-2Reasearch Internship - Loop-closure detection for SLAM @ Technische Universität München
Place recognition using Convolutional Neural Networks (deep-learning) applied to SLAM Loop-closure detection algorithms.2014-03-312014-09-30MunichC++PYTHONDEEP LEARNINGCONVOLUTIONAL NEURAL NETWORKSCAFFE FRAMEWORKCUDASLAMLOOP-CLOSURE DETECTIONUniversity Projects@ Polytech'Nice-Sophia, Trinity College Dublin
2011-12-312014-12-31Sophia-Antipolis, France- 3D Game Programming (physics and rendering engine)
- Interactive curve-fitting for Scanning Tunnelling Spectroscopy (Atomic Microscope)
- Distributed computing grid exploiting unused smartphone resources
- 3D-Game for visually-disabled players (Project DEVINT)
C++JAVAOPENGL3DGAME DEVELOPMENTCOMPUTER GRAPHICSCOMPUTER SCIENCEResearch Internship - Scanning Tunneling Spectroscopy @ CRANN
Interactive curve-fitting for Scanning Tunneling Spectroscopy (Atomic Microscope)2013-06-302013-08-31Dublin, IrelandC++QTCOMPUTER SCIENCEDevelopement of Fotowall @ High school
2008-12-312011-12-31France- Developed a popular open-source interactive image manipulation program Fotowall with an Italian partner
- Over 470, 000 downloads (as of December 2011)
C++QTSELF-TAUGHTGITCOLLABORATIVE PROJECTOPEN SOURCE
Education
PhD in Humanoid Robotics and Computer Vision @ Laboratoire d'Informatique et de Robotique de l'Université de Montpellier - I3S Sophia-Antipolis - Joint Robotics Laboratory
2016-10-312019-10-31HUMANOID ROBOTICSVISUAL SLAMC++ROSMC_RTC FRAMEWORKHUMANOID ROBOTSOPTIMIZATIONQUADRATIC PROGRAMMINGMODEL PREVIEW CONTROLSoftware Engineer (Master's degree) @ Polytech'Nice-Sophia
2011-12-312014-12-31COMPUTER VISIONGAME DEVELOPMENTJAVAC++COMPUTER SCIENCEALGORITHMICMATHSLANGUAGESMaster's Degree in Interactive Entertainment Technologies @ Trinity College
2012-12-312013-12-31INTERACTIVE ENTERTAINMENT TECHNOLOGIESClasses Préparatoires aux Grandes Écoles (CPGE MPSI) @ Lycée de Kérichen
2009-12-312011-12-31MATHSPHYSICSENGINEERINGCOMPUTER SCIENCE (FUNCTIONAL PROGRAMMING, ALGORITHMS, COMPLEXITY)LANGUAGESBaccalauréat @ Lycée de l’Harteloire
2002-12-312009-12-31MATHSPHYSICSEARTH AND LIFE SCIENCESLITTERATUREPHYLOSOPHYENGLISHGERMAN