Publications | Movement Generation and Control - Max Planck Institute for Intelligent Systems

115 results (View BibTeX file of all listed publications)

2023

On the Use of Torque Measurement in Centroidal State Estimation

Khorshidi, S., Gazar, A., Rotella, N., Naveau, M., Righetti, L., Bennewitz, M., Khadiv, M.

IEEE International Conference on Robotics and Automation (ICRA), pages: 9931-9937, May 2023 (conference)

2023

Khorshidi, S., Gazar, A., Rotella, N., Naveau, M., Righetti, L., Bennewitz, M., Khadiv, M. On the Use of Torque Measurement in Centroidal State Estimation IEEE International Conference on Robotics and Automation (ICRA), pages: 9931-9937, May 2023 (conference)

DOI [BibTex]

Visual-Inertial and Leg Odometry Fusion for Dynamic Locomotion

Dhédin, V., Li, H., Khorshidi, S., Mack, L., Ravi, A. K. C., Meduri, A., Shah, P., Grimminger, F., Righetti, L., Khadiv, M., Stueckler, J.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023 (inproceedings)

Abstract

Implementing dynamic locomotion behaviors on legged robots requires a high-quality state estimation module. Especially when the motion includes flight phases, state-of-the-art approaches fail to produce reliable estimation of the robot posture, in particular base height. In this paper, we propose a novel approach for combining visual-inertial odometry (VIO) with leg odometry in an extended Kalman filter (EKF) based state estimator. The VIO module uses a stereo camera and IMU to yield low-drift 3D position and yaw orientation and drift-free pitch and roll orientation of the robot base link in the inertial frame. However, these values have a considerable amount of latency due to image processing and optimization, while the rate of update is quite low which is not suitable for low-level control. To reduce the latency, we predict the VIO state estimate at the rate of the IMU measurements of the VIO sensor. The EKF module uses the base pose and linear velocity predicted by VIO, fuses them further with a second high-rate IMU and leg odometry measurements, and produces robot state estimates with a high frequency and small latency suitable for control. We integrate this lightweight estimation framework with a nonlinear model predictive controller and show successful implementation of a set of agile locomotion behaviors, including trotting and jumping at varying horizontal speeds, on a torque-controlled quadruped robot.

preprint video link (url) DOI [BibTex]

Dhédin, V., Li, H., Khorshidi, S., Mack, L., Ravi, A. K. C., Meduri, A., Shah, P., Grimminger, F., Righetti, L., Khadiv, M., Stueckler, J. Visual-Inertial and Leg Odometry Fusion for Dynamic Locomotion In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023 (inproceedings)

preprint video link (url) DOI [BibTex]

BiConMP: A Nonlinear Model Predictive Control Framework for Whole Body Motion Planning

Meduri, A., Shah, P., Viereck, J., Khadiv, M., Havoutis, I., Righetti, L.

IEEE Transactions on Robotics, 39(2):905-922, IEEE, 2023 (article)

link (url) DOI [BibTex]

Meduri, A., Shah, P., Viereck, J., Khadiv, M., Havoutis, I., Righetti, L. BiConMP: A Nonlinear Model Predictive Control Framework for Whole Body Motion Planning IEEE Transactions on Robotics, 39(2):905-922, IEEE, 2023 (article)

link (url) DOI [BibTex]

2022

Introducing Force Feedback in Model Predictive Control

Kleff, E. D. E. S. G. M. N. R. L.

Proceedings of the 2022 International Conference on Intelligent Robots and Systems (IROS), pages: 13379-13385, IEEE, International Conference on Intelligent Robots and Systems (IROS), October 2022 (conference)

DOI [BibTex]

2022

Kleff, E. D. E. S. G. M. N. R. L. Introducing Force Feedback in Model Predictive Control Proceedings of the 2022 International Conference on Intelligent Robots and Systems (IROS), pages: 13379-13385, IEEE, International Conference on Intelligent Robots and Systems (IROS), October 2022 (conference)

DOI [BibTex]

Nonlinear Stochastic Trajectory Optimization for Centroidal Momentum Motion Generation of Legged Robots

Gazar, A., Khadiv, M., Kleff, S., DelPrete, A., Righetti, L.

In Robotics Research, pages: 420-435, Springer Proceedings in Advanced Robotics, 27, (Editors: Billard, Aude and Asfour, Tamim and Khatib, Oussama), Springer, Cham, 20th International Symposium on Robotics Research (ISRR 2022), September 2022 (inproceedings)

Abstract

Generation of robust trajectories for legged robots remains a challenging task due to the underlying nonlinear, hybrid and intrinsically unstable dynamics which needs to be stabilized through limited contact forces. Furthermore, disturbances arising from unmodelled contact interactions with the environment and model mismatches can hinder the quality of the planned trajectories leading to unsafe motions. In this work, we propose to use stochastic trajectory optimization for generating robust centroidal momentum trajectories to account for additive uncertainties on the model dynamics and parametric uncertainties on contact locations. Through an alternation between the robust centroidal and whole-body trajectory optimizations, we generate robust momentum trajectories while being consistent with the whole-body dynamics. We perform an extensive set of simulations subject to different uncertainties on a quadruped robot showing that our stochastic trajectory optimization problem reduces the amount of foot slippage for different gaits while achieving better performance over deterministic planning.

link (url) DOI [BibTex]

Gazar, A., Khadiv, M., Kleff, S., DelPrete, A., Righetti, L. Nonlinear Stochastic Trajectory Optimization for Centroidal Momentum Motion Generation of Legged Robots In Robotics Research, pages: 420-435, Springer Proceedings in Advanced Robotics, 27, (Editors: Billard, Aude and Asfour, Tamim and Khatib, Oussama), Springer, Cham, 20th International Symposium on Robotics Research (ISRR 2022), September 2022 (inproceedings)

link (url) DOI [BibTex]

Model-free reinforcement learning for robust locomotion using demonstrations from trajectory optimization

Bogdanovic, M., Khadiv, M., Righetti, L.

Frontiers in Robotics and AI, 9, August 2022 (article)

DOI [BibTex]

Bogdanovic, M., Khadiv, M., Righetti, L. Model-free reinforcement learning for robust locomotion using demonstrations from trajectory optimization Frontiers in Robotics and AI, 9, August 2022 (article)

DOI [BibTex]

iRiSC: Iterative Risk Sensitive Control for Nonlinear Systems with Imperfect Observations

Hammoud, B., Jordana, A., Righetti, L.

In 2022 American Control Conference (ACC 2022), pages: 3550-3557, IEEE, Piscataway, NJ, American Control Conference (ACC 2022), June 2022 (inproceedings)

DOI [BibTex]

Hammoud, B., Jordana, A., Righetti, L. iRiSC: Iterative Risk Sensitive Control for Nonlinear Systems with Imperfect Observations In 2022 American Control Conference (ACC 2022), pages: 3550-3557, IEEE, Piscataway, NJ, American Control Conference (ACC 2022), June 2022 (inproceedings)

DOI [BibTex]

Stagewise Newton Method for Dynamic Game Control With Imperfect State Observation

Jordana, A., Hammoud, B., Carpentier, J., Righetti, L.

IEEE Control Systems Letters, 6, pages: 3241-3246, IEEE, Piscataway, NJ, USA, 2022 (article)

DOI [BibTex]

Jordana, A., Hammoud, B., Carpentier, J., Righetti, L. Stagewise Newton Method for Dynamic Game Control With Imperfect State Observation IEEE Control Systems Letters, 6, pages: 3241-3246, IEEE, Piscataway, NJ, USA, 2022 (article)

DOI [BibTex]

2021

A unified framework for walking and running of bipedal robots

Boroujeni, M. G., Daneshmand, E., Righetti, L., Khadiv, M.

20th International Conference on Advanced Robotics (ICAR), December 2021 (conference) Accepted

Abstract

In this paper, we propose a novel framework capable of generating various walking and running gaits for bipedal robots. The main goal is to relax the fixed center of mass (CoM) height assumption of the linear inverted pendulum model (LIPM) and generate a wider range of walking and running motions, without a considerable increase in complexity. To do so, we use the concept of virtual constraints in the centroidal space which enables generating motions beyond walking while keeping the complexity at a minimum. By a proper choice of these virtual constraints, we show that we can generate different types of walking and running motions. More importantly, enforcing the virtual constraints through feedback renders the dynamics linear and enables us to design a feedback control mechanism which adapts the next step location and timing in face of disturbances, through a simple quadratic program (QP). To show the effectiveness of this framework, we showcase different walking and running simulations of the biped robot Bolt in the presence of both environmental uncertainties and external disturbances.

link (url) [BibTex]

2021

Boroujeni, M. G., Daneshmand, E., Righetti, L., Khadiv, M. A unified framework for walking and running of bipedal robots 20th International Conference on Advanced Robotics (ICAR), December 2021 (conference) Accepted

link (url) [BibTex]

Robust walking based on MPC with viability guarantees

Yeganegi, M. H., Khadiv, M., Prete, A. D., Moosavian, S. A. A., Righetti, L.

IEEE Transactions on Robotics, October 2021 (article) Accepted

Abstract

Model predictive control (MPC) has shown great success for controlling complex systems such as legged robots. However, when closing the loop, the performance and feasibility of the finite horizon optimal control problem (OCP) solved at each control cycle is not guaranteed anymore. This is due to model discrepancies, the effect of low-level controllers, uncertainties and sensor noise. To address these issues, we propose a modified version of a standard MPC approach used in legged locomotion with viability (weak forward invariance) guarantees. In this approach, instead of adding a (conservative) terminal constraint to the problem, we propose to use the measured state projected to the viability kernel in the OCP solved at each control cycle. Moreover, we use past experimental data to find the best cost weights, which measure a combination of performance, constraint satisfaction robustness, or stability (invariance). These interpretable costs measure the trade off between robustness and performance. For this purpose, we use Bayesian optimization (BO) to systematically design experiments that help efficiently collect data to learn a cost function leading to robust performance. Our simulation results with different realistic disturbances (i.e. external pushes, unmodeled actuator dynamics and computational delay) show the effectiveness of our approach to create robust controllers for humanoid robots.

link (url) [BibTex]

Yeganegi, M. H., Khadiv, M., Prete, A. D., Moosavian, S. A. A., Righetti, L. Robust walking based on MPC with viability guarantees IEEE Transactions on Robotics, October 2021 (article) Accepted

link (url) [BibTex]

The o80 C++ templated toolbox: Designing customized Python APIs for synchronizing realtime processes

Berenz, V., Naveau, M., Widmaier, F., Wüthrich, M., Passy, J., Guist, S., Büchler, D.

Journal of Open Source Software, 6(66), October 2021 (article)

Abstract

o80 (pronounced "oh-eighty") is software for synchronizing and organizing message exchange between (realtime) processes via simple customized Python APIs. Its target domain is robotics and machine learning. Our motivation for developing o80 is to ease the setup of robotics experiments (i.e., integration of various hardware and software) by machine learning scientists. Such setup typically requires time and technical effort, especially when realtime processes are involved. Ideally, scientists should have access to a simple Python API that hides the lower level communication details and simply allows the sending of actions and receiving of observations. o80 is a tool box for creating such API.

link (url) DOI [BibTex]

Berenz, V., Naveau, M., Widmaier, F., Wüthrich, M., Passy, J., Guist, S., Büchler, D. The o80 C++ templated toolbox: Designing customized Python APIs for synchronizing realtime processes Journal of Open Source Software, 6(66), October 2021 (article)

link (url) DOI [BibTex]

Rapid Convex Optimization of Centroidal Dynamics using Block Coordinate Descent

Shah, P., Meduri, A., Merkt, W., Khadiv, M., Havoutis, I., Righetti, L.

In Proceedings of the International Conference on Intelligent Robots and Systems (IROS 2021) , IEEE, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021) in Prague, Czech Republic., September 2021 (inproceedings) Accepted

Abstract

In this paper we explore the use of block coordinate descent (BCD) to optimize the centroidal momentum dynamics for dynamically consistent multi-contact behaviors. The centroidal dynamics have recently received a large amount of attention in order to create physically realizable motions for robots with hands and feet while being computationally more tractable than full rigid body dynamics models. Our contribution lies in exploiting the structure of the dynamics in order to simplify the original non-convex problem into two convex subproblems. We iterate between these two subproblems for a set number of iterations or until a consensus is reached. We explore the properties of the proposed optimization method for the centroidal dynamics and verify in simulation that motions generated by our approach can be tracked by the quadruped Solo12. In addition, we compare our method to a recently proposed convexification using a sequence of convex relaxations as well as a more standard interior point method used in the off- the-shelf solver IPOPT to show that our approach finds similar, if not better, trajectories (in terms of cost), and is more than four times faster than both approaches. Finally, compared to previous approaches, we note its practicality due to the convex nature of each subproblem which allows our method to be used with any off-the-shelf quadratic programming solver.

link (url) [BibTex]

Shah, P., Meduri, A., Merkt, W., Khadiv, M., Havoutis, I., Righetti, L. Rapid Convex Optimization of Centroidal Dynamics using Block Coordinate Descent In Proceedings of the International Conference on Intelligent Robots and Systems (IROS 2021) , IEEE, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021) in Prague, Czech Republic., September 2021 (inproceedings) Accepted

link (url) [BibTex]

Stochastic and robust mpc for bipedal locomotion: A comparative study on robustness and performance

Gazar, A., Khadiv, M., DelPrete, A., Righetti, L.

pages: 1-8, IEEE, IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids), July 2021 (conference)

Abstract

Linear Model Predictive Control (MPC) has been successfully used for generating feasible walking motions for humanoid robots. However, the effect of uncertainties on constraints satisfaction has only been studied using Robust MPC (RMPC) approaches, which account for the worst-case realization of bounded disturbances at each time instant. In this letter, we propose for the first time to use linear stochastic MPC (SMPC) to account for uncertainties in bipedal walking. We show that SMPC offers more flexibility to the user (or a high level decision maker) by tolerating small (user-defined) probabilities of constraint violation. Therefore, SMPC can be tuned to achieve a constraint satisfaction probability that is arbitrarily close to 100%, but without sacrificing performance as much as tube-based RMPC. We compare SMPC against RMPC in terms of robustness (constraint satisfaction) and performance (optimality). Our results highlight the benefits of SMPC and its interest for the robotics community as a powerful mathematical tool for dealing with uncertainties.

DOI [BibTex]

Gazar, A., Khadiv, M., DelPrete, A., Righetti, L. Stochastic and robust mpc for bipedal locomotion: A comparative study on robustness and performance pages: 1-8, IEEE, IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids), July 2021 (conference)

DOI [BibTex]

Impedance Optimization for Uncertain Contact Interactions Through Risk Sensitive Optimal Control

Hammoud, B., Khadiv, M., Righetti, L.

IEEE Robotics and Automation Letters, Robotics and Automation Letters, 6(3):4766-4773, IEEE, July 2021 (article)

Abstract

This paper addresses the problem of computing optimal impedance schedules for legged locomotion tasks involving complex contact interactions. We formulate the problem of impedance regulation as a trade-off between disturbance rejection and measurement uncertainty. We extend a stochastic optimal control algorithm known as Risk Sensitive Control to take into account measurement uncertainty and propose a formal way to include such uncertainty for unknown contact locations. The approach can efficiently generate optimal state and control trajectories along with local feedback control gains, i.e. impedance schedules. Extensive simulations demonstrate the capabilities of the approach in generating meaningful stiffness and damping modulation patterns before and after contact interaction. For example, contact forces are reduced during early contacts, damping increases to anticipate a high impact event and tracking is automatically traded-off for increased stability. In particular, we show a significant improvement in performance during jumping and trotting tasks with a simulated quadruped robot.

link (url) DOI [BibTex]

Hammoud, B., Khadiv, M., Righetti, L. Impedance Optimization for Uncertain Contact Interactions Through Risk Sensitive Optimal Control IEEE Robotics and Automation Letters, Robotics and Automation Letters, 6(3):4766-4773, IEEE, July 2021 (article)

link (url) DOI [BibTex]

High-frequency nonlinear model predictive control of a manipulator

Kleff, S., Meduri, A., Budhiraja, R., Mansard, N., Righetti, L.

In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages: 7330-7336 , The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (inproceedings)

Abstract

Model Predictive Control (MPC) promises to endow robots with enough reactivity to perform complex tasks in dynamic environments by frequently updating their motion plan based on measurements. Despite its appeal, it has seldom been deployed on real machines because of scaling constraints. This paper presents the first hardware implementation of closed-loop nonlinear MPC on a 7-DoF torque-controlled robot. Our controller leverages a state-of-the art optimal control solver, namely Differential Dynamic Programming (DDP), in order to replan state and control trajectories at real-time rates (1kHz). In addition to this experimental proof of concept, we present exhaustive performance analysis on the iconic pick-and-place task and show that our controller outperforms open-loop MPC. We also exhibit the importance of a sufficient preview horizon and full robot dynamics in the controller performance through comparisons with inverse dynamics and kinematic optimization.

DOI [BibTex]

Kleff, S., Meduri, A., Budhiraja, R., Mansard, N., Righetti, L. High-frequency nonlinear model predictive control of a manipulator In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages: 7330-7336 , The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (inproceedings)

DOI [BibTex]

Leveraging Forward Model Prediction Error for Learning Control

Bechtle, S., Hammoud, B., Rai, A., Meier, F., Righetti, L.

2021 IEEE International Conference on Robotics and Automation (ICRA) , pages: 4445-4451 , IEEE International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

Abstract

Learning for model based control can be sample-efficient and generalize well, however successfully learning models and controllers that represent the problem at hand can be challenging for complex tasks. Using inaccurate models for learning can lead to sub-optimal solutions, that are unlikely to perform well in practice. In this work, we present a learning approach which iterates between model learning and data collection and leverages forward model prediction error for learning control. We show how using the controller's prediction as input to a forward model can create a differentiable connection between the controller and the model, allowing us to formulate a loss in the state space. This lets us include forward model prediction error during controller learning and we show that this creates a loss objective that significantly improves learning on different motor control tasks. We provide empirical and theoretical results that show the benefits of our method and present evaluations in simulation for learning control on a 7 DoF manipulator and an underactuated 12 DoF quadruped. We show that our approach successfully learns controllers for challenging motor control tasks involving contact switching.

link (url) [BibTex]

Bechtle, S., Hammoud, B., Rai, A., Meier, F., Righetti, L. Leveraging Forward Model Prediction Error for Learning Control 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages: 4445-4451 , IEEE International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

link (url) [BibTex]

Learning a Centroidal Motion Planner for Legged Locomotion

Viereck, J., Righetti, L.

2021 IEEE International Conference on Robotics and Automation (ICRA), pages: 4905-4911 , IEEE, The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

Abstract

Whole-body optimizers have been successful at automatically computing complex dynamic locomotion behaviors. However they are often limited to offline planning as they are computationally too expensive to replan with a high frequency. Simpler models are then typically used for online replanning. In this paper we present a method to generate whole body movements in real-time for locomotion tasks. Our approach consists in learning a centroidal neural network that predicts the desired centroidal motion given the current state of the robot and a desired contact plan. The network is trained using an existing whole body motion optimizer. Our approach enables to learn with few training samples dynamic motions that can be used in a complete whole-body control framework at high frequency, which is usually not attainable with typical full-body optimizers. We demonstrate our method to generate a rich set of walking and jumping motions on a real quadruped robot.

DOI [BibTex]

Viereck, J., Righetti, L. Learning a Centroidal Motion Planner for Legged Locomotion 2021 IEEE International Conference on Robotics and Automation (ICRA), pages: 4905-4911 , IEEE, The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

DOI [BibTex]

DeepQ Stepper: A framework for reactive dynamic walking on uneven terrain

Meduri, A., Khadiv, M., Righetti, L.

The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

Abstract

Reactive stepping and push recovery for biped robots is often restricted to flat terrains because of the difficulty in computing capture regions for nonlinear dynamic models. In this paper, we address this limitation by using reinforcement learning to approximately learn the 3D capture region for such systems. We propose a novel 3D reactive stepper, The DeepQ stepper, that computes optimal step locations for walking at different velocities using the 3D capture regions approximated by the action-value function. We demonstrate the ability of the approach to learn stepping with a simplified 3D pendulum model and a full robot dynamics. Further, the stepper achieves a higher performance when it learns approximate capture regions while taking into account the entire dynamics of the robot that are often ignored in existing reactive steppers based on simplified models. The DeepQ stepper can handle non convex terrain with obstacles, walk on restricted surfaces like stepping stones and recover from external disturbances for a constant computational cost.

link (url) [BibTex]

Meduri, A., Khadiv, M., Righetti, L. DeepQ Stepper: A framework for reactive dynamic walking on uneven terrain The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

link (url) [BibTex]

Variable Horizon MPC with Swing Foot Dynamics for Bipedal Walking Control

Daneshmand, E., Khadiv, M., Grimminger, F., Righetti, L.

IEEE Robotics and Automation Letters, 6(2):2349-2356, April 2021 (article)

Abstract

In this paper, we present a novel two-level variable Horizon Model Predictive Control (VH-MPC) framework for bipedal locomotion. In this framework, the higher level computes the landing location and timing (horizon length) of the swing foot to stabilize the unstable part of the center of mass (CoM) dynamics, using feedback from the CoM states. The lower level takes into account the swing foot dynamics and generates dynamically consistent trajectories for landing at the desired time as close as possible to the desired location. To do that, we use a simplified model of the robot dynamics projected in swing foot space that takes into account joint torque constraints as well as the friction cone constraints of the stance foot. We show the effectiveness of our proposed control framework by implementing robust walking patterns on our torque-controlled and open-source biped robot, Bolt. We report extensive simulations and real robot experiments in the presence of various disturbances and uncertainties.

link (url) DOI [BibTex]

Daneshmand, E., Khadiv, M., Grimminger, F., Righetti, L. Variable Horizon MPC with Swing Foot Dynamics for Bipedal Walking Control IEEE Robotics and Automation Letters, 6(2):2349-2356, April 2021 (article)

link (url) DOI [BibTex]

Efficient Multi-Contact Pattern Generation with Sequential Convex Approximations of the Centroidal Dynamics

Ponton, B., Khadiv, M., Meduri, A., Righetti, L.

IEEE Transactions on Robotics, 37(5):1661-1679, IEEE, February 2021 (article)

Abstract

This article investigates the problem of efficient computation of physically consistent multicontact behaviors. Recent work showed that under mild assumptions, the problem could be decomposed into simpler kinematic and centroidal dynamic optimization problems. Based on this approach, we propose a general convex relaxation of the centroidal dynamics leading to two computationally efficient algorithms based on iterative resolutions of second-order cone programs. They optimize centroidal trajectories, contact forces, and importantly the timing of the motions. We include the approach in a kinodynamic optimization method to generate full-body movements. Finally, the approach is embedded in a mixed-integer solver to further find dynamically consistent contact sequences. Extensive numerical experiments demonstrate the computational efficiency of the approach, suggesting that it could be used in a fast receding horizon control loop. Executions of the planned motions on simulated humanoids and quadrupeds and on a real quadruped robot further show the quality of the optimized motions.

link (url) DOI [BibTex]

Ponton, B., Khadiv, M., Meduri, A., Righetti, L. Efficient Multi-Contact Pattern Generation with Sequential Convex Approximations of the Centroidal Dynamics IEEE Transactions on Robotics, 37(5):1661-1679, IEEE, February 2021 (article)

link (url) DOI [BibTex]

Robot Learning with Crash Constraints

Marco, A., Baumann, D., Khadiv, M., Hennig, P., Righetti, L., Trimpe, S.

IEEE Robotics and Automation Letters, 6(2):1439-1446, IEEE, February 2021 (article)

Abstract

In the past decade, numerous machine learning algorithms have been shown to successfully learn optimal policies to control real robotic systems. However, it is common to encounter failing behaviors as the learning loop progresses. Specifically, in robot applications where failing is undesired but not catastrophic, many algorithms struggle with leveraging data obtained from failures. This is usually caused by (i) the failed experiment ending prematurely, or (ii) the acquired data being scarce or corrupted. Both complicate the design of proper reward functions to penalize failures. In this paper, we propose a framework that addresses those issues. We consider failing behaviors as those that violate a constraint and address the problem of learning with crash constraints, where no data is obtained upon constraint violation. The no-data case is addressed by a novel GP model (GPCR) for the constraint that combines discrete events (failure/success) with continuous observations (only obtained upon success). We demonstrate the effectiveness of our framework on simulated benchmarks and on a real jumping quadruped, where the constraint threshold is unknown a priori. Experimental data is collected, by means of constrained Bayesian optimization, directly on the real robot. Our results outperform manual tuning and GPCR proves useful on estimating the constraint threshold.

link (url) DOI [BibTex]

Marco, A., Baumann, D., Khadiv, M., Hennig, P., Righetti, L., Trimpe, S. Robot Learning with Crash Constraints IEEE Robotics and Automation Letters, 6(2):1439-1446, IEEE, February 2021 (article)

link (url) DOI [BibTex]

Reactive Balance Control for Legged Robots under Visco-Elastic Contacts

Flayols, T., Prete, A. D., Khadiv, M., Mansard, N., Righetti, L.

Applied Sciences, 11(1), MDPI, 2021 (article)

Abstract

Contacts between robots and environment are often assumed to be rigid for control purposes. This assumption can lead to poor performance when contacts are soft and/or underdamped. However, the problem of balancing on soft contacts has not received much attention in the literature. This paper presents two novel approaches to control a legged robot balancing on visco-elastic contacts, and compares them to other two state-of-the-art methods. Our simulation results show that performance heavily depends on the contact stiffness and the noises/uncertainties introduced in the simulation. Briefly, the two novel controllers performed best for soft/medium contacts, whereas “inverse-dynamics control under rigid-contact assumptions” was the best one for stiff contacts. Admittance control was instead the most robust, but suffered in terms of performance. These results shed light on this challenging problem, while pointing out interesting directions for future investigation.

link (url) DOI [BibTex]

Flayols, T., Prete, A. D., Khadiv, M., Mansard, N., Righetti, L. Reactive Balance Control for Legged Robots under Visco-Elastic Contacts Applied Sciences, 11(1), MDPI, 2021 (article)

link (url) DOI [BibTex]

Meta-Learning via Learned Loss

Bechtle, S., Molchanov, A., Chebotar, Y., Grefenstette, E., Righetti, L., Sukhatme, G., Meier, F.

In 2020 25th International Conference on Pattern Recognition (ICPR), IEEE, January 2021 (inproceedings)

Abstract

Typically, loss functions, regularization mechanisms and other important aspects of training parametric models are chosen heuristically from a limited set of options. In this paper, we take the first step towards automating this process, with the view of producing models which train faster and more robustly. Concretely, we present a meta-learning method for learning parametric loss functions that can generalize across different tasks and model architectures. We develop a pipeline for “meta-training” such loss functions, targeted at maximizing the performance of the model trained under them. The loss landscape produced by our learned losses significantly improves upon the original task-specific losses in both supervised and reinforcement learning tasks. Furthermore, we show that our meta-learning framework is flexible enough to incorporate additional informa- tion at meta-train time. This information shapes the learned loss function such that the environment does not need to provide this information during meta-test time. We make our code available at https://sites.google.com/view/mlthree

[BibTex]

Bechtle, S., Molchanov, A., Chebotar, Y., Grefenstette, E., Righetti, L., Sukhatme, G., Meier, F. Meta-Learning via Learned Loss In 2020 25th International Conference on Pattern Recognition (ICPR), IEEE, January 2021 (inproceedings)

[BibTex]

Jerk Control of Floating Base Systems With Contact-Stable Parameterized Force Feedback

Gazar, A., Nava, G., Chavez, F. J. A., Pucci, D.

IEEE Transactions on Robotics, 37(1):1-15, IEEE, 2021 (article)

DOI [BibTex]

Gazar, A., Nava, G., Chavez, F. J. A., Pucci, D. Jerk Control of Floating Base Systems With Contact-Stable Parameterized Force Feedback IEEE Transactions on Robotics, 37(1):1-15, IEEE, 2021 (article)

DOI [BibTex]

2020

On the use of simulation in robotics: Opportunities challenges, and suggestions for moving forward

Choi, H., Crump, C., Duriez, C., Elmquist, A., Hager, G., Han, D., Hearl, F., Hodgins, J., Jain, A., Leve, F., Li, C., Meier, F., Negrut, D., Righetti, L., Rodriguez, A., Tan, J., Trinkle, J.

PNAS, 118(1), December 2020 (article)

DOI [BibTex]

2020

Choi, H., Crump, C., Duriez, C., Elmquist, A., Hager, G., Han, D., Hearl, F., Hodgins, J., Jain, A., Leve, F., Li, C., Meier, F., Negrut, D., Righetti, L., Rodriguez, A., Tan, J., Trinkle, J. On the use of simulation in robotics: Opportunities challenges, and suggestions for moving forward PNAS, 118(1), December 2020 (article)

DOI [BibTex]

TriFinger: An Open-Source Robot for Learning Dexterity

Wüthrich, M., Widmaier, F., Grimminger, F., Akpo, J., Joshi, S., Agrawal, V., Hammoud, B., Khadiv, M., Bogdanovic, M., Berenz, V., Viereck, J., Naveau, M., Righetti, L., Schölkopf, B., Bauer, S.

Proceedings of the 4th Conference on Robot Learning (CoRL), 155, pages: 1871-1882, Proceedings of Machine Learning Research, (Editors: Jens Kober and Fabio Ramos and Claire J. Tomlin), PMLR, November 2020 (conference)

PDF link (url) [BibTex]

Wüthrich, M., Widmaier, F., Grimminger, F., Akpo, J., Joshi, S., Agrawal, V., Hammoud, B., Khadiv, M., Bogdanovic, M., Berenz, V., Viereck, J., Naveau, M., Righetti, L., Schölkopf, B., Bauer, S. TriFinger: An Open-Source Robot for Learning Dexterity Proceedings of the 4th Conference on Robot Learning (CoRL), 155, pages: 1871-1882, Proceedings of Machine Learning Research, (Editors: Jens Kober and Fabio Ramos and Claire J. Tomlin), PMLR, November 2020 (conference)

PDF link (url) [BibTex]

Enabling Remote Whole-Body Control with 5G Edge Computing

Zhu, H., Sharma, M., Pfeiffer, K., Mezzavilla, M., Shen, J., Rangan, S., Righetti, L.

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 3553-3560, IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2020 (conference) Accepted

Abstract

Real-world applications require light-weight, energy-efficient, fully autonomous robots. Yet, increasing auton- omy is oftentimes synonymous with escalating computational requirements. It might thus be desirable to offload intensive computation—not only sensing and planning, but also low- level whole-body control—to remote servers in order to reduce on-board computational needs. Fifth Generation (5G) wireless cellular technology, with its low latency and high bandwidth capabilities, has the potential to unlock cloud-based high per- formance control of complex robots. However, state-of-the-art control algorithms for legged robots can only tolerate very low control delays, which even ultra-low latency 5G edge computing can sometimes fail to achieve. In this work, we investigate the problem of cloud-based whole-body control of legged robots over a 5G link. We propose a novel approach that consists of a standard optimization-based controller on the network edge and a local linear, approximately optimal controller that significantly reduces on-board computational needs while increasing robustness to delay and possible loss of commu- nication. Simulation experiments on humanoid balancing and walking tasks that includes a realistic 5G communication model demonstrate significant improvement of the reliability of robot locomotion under jitter and delays likely to be experienced in 5G wireless links.

link (url) DOI [BibTex]

Zhu, H., Sharma, M., Pfeiffer, K., Mezzavilla, M., Shen, J., Rangan, S., Righetti, L. Enabling Remote Whole-Body Control with 5G Edge Computing IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 3553-3560, IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2020 (conference) Accepted

link (url) DOI [BibTex]

Learning Variable Impedance Control for Contact Sensitive Tasks

Bogdanovic, M., Khadiv, M., Righetti, L.

IEEE Robotics and Automation Letters, 5(4), IEEE, October 2020 (article)

Abstract

Reinforcement learning algorithms have shown great success in solving different problems ranging from playing video games to robotics. However, they struggle to solve delicate robotic problems, especially those involving contact interactions. Though in principle a policy outputting joint torques should be able to learn these tasks, in practice we see that they have difficulty to robustly solve the problem without any structure in the action space. In this paper, we investigate how the choice of action space can give robust performance in presence of contact uncertainties. We propose to learn a policy that outputs impedance and desired position in joint space as a function of system states without imposing any other structure to the problem. We compare the performance of this approach to torque and position control policies under different contact uncertainties. Extensive simulation results on two different systems, a hopper (floating-base) with intermittent contacts and a manipulator (fixed-base) wiping a table, show that our proposed approach outperforms policies outputting torque or position in terms of both learning rate and robustness to environment uncertainty.

DOI [BibTex]

Bogdanovic, M., Khadiv, M., Righetti, L. Learning Variable Impedance Control for Contact Sensitive Tasks IEEE Robotics and Automation Letters, 5(4), IEEE, October 2020 (article)

DOI [BibTex]

Walking Control Based on Step Timing Adaptation

Khadiv, M., Herzog, A., Moosavian, S. A. A., Righetti, L.

IEEE Transactions on Robotics, 36(3):629-643, IEEE, June 2020 (article)

Abstract

Step adjustment can improve the gait robustness of biped robots; however, the adaptation of step timing is often neglected as it gives rise to nonconvex problems when optimized over several footsteps. In this article, we argue that it is not necessary to optimize walking over several steps to ensure gait viability and show that it is sufficient to merely select the next step timing and location. Using this insight, we propose a novel walking pattern generator that optimally selects step location and timing at every control cycle. Our approach is computationally simple compared to standard approaches in the literature, yet guarantees that any viable state will remain viable in the future. We propose a swing foot adaptation strategy and integrate the pattern generator with an inverse dynamics controller that does not explicitly control the center of mass nor the foot center of pressure. This is particularly useful for biped robots with limited control authority over their foot center of pressure, such as robots with point feet or passive ankles. Extensive simulations on a humanoid robot with passive ankles demonstrate the capabilities of the approach in various walking situations, including external pushes and foot slippage, and emphasize the importance of step timing adaptation to stabilize walking.

link (url) DOI [BibTex]

Khadiv, M., Herzog, A., Moosavian, S. A. A., Righetti, L. Walking Control Based on Step Timing Adaptation IEEE Transactions on Robotics, 36(3):629-643, IEEE, June 2020 (article)

link (url) DOI [BibTex]

Crocoddyl: An efficient and versatile framework for multi-contact optimal control

Mastalli, C., Budhiraja, R., Merkt, W., Saurel, G., Hammoud, B., Naveau, M., Carpentier, J., Righetti, L., Vijayakumar, S., Mansard, N.

In Proceedings of the IEEE International Conference on Robotics and Automation, IEEE, International Conference on Robotics and Automation, May 2020 (inproceedings)

Abstract

We introduce Crocoddyl (Contact RObot COntrol by Differential DYnamic Library), an open-source framework tailored for efficient multi-contact optimal control. Crocoddyl efficiently computes the state trajectory and the control policy for a given predefined sequence of contacts. Its efficiency is due to the use of sparse analytical derivatives, exploitation of the problem structure, and data sharing. It employs differential geometry to properly describe the state of any geometrical system, e.g. floating-base systems. Additionally, we propose a novel optimal control algorithm called Feasibility-driven Differential Dynamic Programming (FDDP). Our method does not add extra decision variables which often increases the computation time per iteration due to factorization. FDDP shows a greater globalization strategy compared to classical Differential Dynamic Programming (DDP) algorithms. Con- cretely, we propose two modifications to the classical DDP algo- rithm. First, the backward pass accepts infeasible state-control trajectories. Second, the rollout keeps the gaps open during the early “exploratory” iterations (as expected in multiple- shooting methods with only equality constraints). We showcase the performance of our framework using different tasks. With our method, we can compute highly-dynamic maneuvers (e.g. jumping, front-flip) within few milliseconds.

[BibTex]

Mastalli, C., Budhiraja, R., Merkt, W., Saurel, G., Hammoud, B., Naveau, M., Carpentier, J., Righetti, L., Vijayakumar, S., Mansard, N. Crocoddyl: An efficient and versatile framework for multi-contact optimal control In Proceedings of the IEEE International Conference on Robotics and Automation, IEEE, International Conference on Robotics and Automation, May 2020 (inproceedings)

[BibTex]

Curious ilqr: Resolving uncertainty in model-based rl

Bechtle, S., Lin, Y., Rai, A., Righetti, L., Meier, F.

Conference on Robot Learning, May 2020 (conference)

Abstract

Curiosity as a means to explore during reinforcement learning problems has recently become very popular. However, very little progress has been made in utilizing curiosity for learning control. In this work, we propose a model-based reinforcement learning (MBRL) framework that combines Bayesian modeling of the system dynamics with curious iLQR, an iterative LQR approach that considers model uncertainty. During trajectory optimization the curious iLQR attempts to minimize both the task-dependent cost and the uncertainty in the dynamics model. We demonstrate the approach on reaching tasks with 7-DoF manipulators in simulation and on a real robot. Our experiments show that MBRL with curious iLQR reaches desired end-effector targets more reliably and with less system rollouts when learning a new task from scratch, and that the learned model generalizes better to new reaching tasks.

[BibTex]

Bechtle, S., Lin, Y., Rai, A., Righetti, L., Meier, F. Curious ilqr: Resolving uncertainty in model-based rl Conference on Robot Learning, May 2020 (conference)

[BibTex]

An Open Torque-Controlled Modular Robot Architecture for Legged Locomotion Research

Grimminger, F., Meduri, A., Khadiv, M., Viereck, J., Wüthrich, M., Naveau, M., Berenz, V., Heim, S., Widmaier, F., Flayols, T., Fiene, J., Badri-Spröwitz, A., Righetti, L.

IEEE Robotics and Automation Letters, 5(2):3650-3657, IEEE, April 2020 (article)

Youtube Open Dynamic Robot Initiative link (url) DOI Project Page [BibTex]

Grimminger, F., Meduri, A., Khadiv, M., Viereck, J., Wüthrich, M., Naveau, M., Berenz, V., Heim, S., Widmaier, F., Flayols, T., Fiene, J., Badri-Spröwitz, A., Righetti, L. An Open Torque-Controlled Modular Robot Architecture for Legged Locomotion Research IEEE Robotics and Automation Letters, 5(2):3650-3657, IEEE, April 2020 (article)

Youtube Open Dynamic Robot Initiative link (url) DOI Project Page [BibTex]

Robust Humanoid Contact Planning with Learned Zero-and One-Step Capturability Prediction

Lin, Y., Righetti, L., Berenson, D.

IEEE Robotics and Automation Letters, Robotics and Automation Letters, 5, pages: 2451-2458, IEEE, February 2020 (article)

Abstract

Humanoid robots maintain balance and navigate by controlling the contact wrenches applied to the environ- ment. While it is possible to plan dynamically-feasible motion that applies appropriate wrenches using existing methods, a humanoid may also be affected by external disturbances. Existing systems typically rely on controllers to reactively recover from disturbances. However, such controllers may fail when the robot cannot reach contacts capable of rejecting a given disturbance. In this paper, we propose a search-based footstep planner which aims to maximize the probability of the robot successfully reaching the goal without falling as a result of a disturbance. The planner considers not only the poses of the planned contact sequence, but also alternative contacts near the planned contact sequence that can be used to recover from external disturbances. Although this additional consideration significantly increases the computation load, we train neural networks to efficiently predict multi-contact zero- step and one-step capturability, which allows the planner to generate robust contact sequences efficiently. Our results show that our approach generates footstep sequences that are more robust to external disturbances than a conventional footstep planner in four challenging scenarios.

[BibTex]

Lin, Y., Righetti, L., Berenson, D. Robust Humanoid Contact Planning with Learned Zero-and One-Step Capturability Prediction IEEE Robotics and Automation Letters, Robotics and Automation Letters, 5, pages: 2451-2458, IEEE, February 2020 (article)

[BibTex]

A Real-Robot Dataset for Assessing Transferability of Learned Dynamics Models

Agudelo-España, D., Zadaianchuk, A., Wenk, P., Garg, A., Akpo, J., Grimminger, F., Viereck, J., Naveau, M., Righetti, L., Martius, G., Krause, A., Schölkopf, B., Bauer, S., Wüthrich, M.

IEEE International Conference on Robotics and Automation (ICRA), pages: 8151-8157, IEEE, 2020 (conference)

Project Page PDF DOI Project Page [BibTex]

Agudelo-España, D., Zadaianchuk, A., Wenk, P., Garg, A., Akpo, J., Grimminger, F., Viereck, J., Naveau, M., Righetti, L., Martius, G., Krause, A., Schölkopf, B., Bauer, S., Wüthrich, M. A Real-Robot Dataset for Assessing Transferability of Learned Dynamics Models IEEE International Conference on Robotics and Automation (ICRA), pages: 8151-8157, IEEE, 2020 (conference)

Project Page PDF DOI Project Page [BibTex]

EXPLORING BY EXPLOITING BAD MODELS IN MODEL-BASED REINFORCEMENT LEARNING

Lin, Y., Bechtle, S., Righetti, L., Rai, A., Meier, F.

International Conference on Learning Representations, 2020 (conference)

Abstract

Exploration for reinforcement learning (RL) is well-studied for model-free methods but a relatively unexplored topic for model-based methods. In this work, we investigate several exploration techniques injected into the two stages of model-based RL:(1) during optimization: adding transition-space and action-space noise when optimizing a policy using learned dynamics, and (2) after optimization: injecting action-space noise when executing an optimized policy on the real environment. When given a good deterministic dynamics model, like the ground-truth simulation, exploration can significantly improve performance. However, using randomly initialized neural networks to model environment dynamics can _implicitly_ induce exploration in model-based RL, reducing the need for explicit exploratory techniques. Surprisingly, we show that in the case of a local optimizer, using a learned model with this implicit exploration can actually _outperform_ using the ground-truth model without exploration, while adding exploration to the ground-truth model reduces the performance gap. However, the learned models are highly local, in that they perform well _only_ for the task for which it is optimized, and fail to generalize to new targets.

[BibTex]

Lin, Y., Bechtle, S., Righetti, L., Rai, A., Meier, F. EXPLORING BY EXPLOITING BAD MODELS IN MODEL-BASED REINFORCEMENT LEARNING International Conference on Learning Representations, 2020 (conference)

[BibTex]

2019

Epstein, Project Maven, and Some Reasons to Think About Where We Get Our Funding [Ethical, Legal, and Societal Issues]

Bretl, T., Righetti, L., Madhavan, R.

IEEE Robotics & Automation Magazine, 26, December 2019 (article)

[BibTex]

2019

Bretl, T., Righetti, L., Madhavan, R. Epstein, Project Maven, and Some Reasons to Think About Where We Get Our Funding [Ethical, Legal, and Societal Issues] IEEE Robotics & Automation Magazine, 26, December 2019 (article)

[BibTex]

Learning to Explore in Motion and Interaction Tasks

Bogdanovic, M., Righetti, L.

Proceedings 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 2686-2692, IEEE, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019, ISSN: 2153-0866 (conference)

Abstract

Model free reinforcement learning suffers from the high sampling complexity inherent to robotic manipulation or locomotion tasks. Most successful approaches typically use random sampling strategies which leads to slow policy convergence. In this paper we present a novel approach for efficient exploration that leverages previously learned tasks. We exploit the fact that the same system is used across many tasks and build a generative model for exploration based on data from previously solved tasks to improve learning new tasks. The approach also enables continuous learning of improved exploration strategies as novel tasks are learned. Extensive simulations on a robot manipulator performing a variety of motion and contact interaction tasks demonstrate the capabilities of the approach. In particular, our experiments suggest that the exploration strategy can more than double learning speed, especially when rewards are sparse. Moreover, the algorithm is robust to task variations and parameter tuning, making it beneficial for complex robotic problems.

DOI [BibTex]

Bogdanovic, M., Righetti, L. Learning to Explore in Motion and Interaction Tasks Proceedings 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 2686-2692, IEEE, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019, ISSN: 2153-0866 (conference)

DOI [BibTex]

Robust Humanoid Locomotion Using Trajectory Optimization and Sample-Efficient Learning

Yeganegi, M. H., Khadiv, M., Moosavian, S. A. A., Zhu, J., Prete, A. D., Righetti, L.

Proceedings International Conference on Humanoid Robots, IEEE, 2019 IEEE-RAS International Conference on Humanoid Robots, October 2019 (conference)

Abstract

Trajectory optimization (TO) is one of the most powerful tools for generating feasible motions for humanoid robots. However, including uncertainties and stochasticity in the TO problem to generate robust motions can easily lead to intractable problems. Furthermore, since the models used in TO have always some level of abstraction, it can be hard to find a realistic set of uncertainties in the model space. In this paper we leverage a sample-efficient learning technique (Bayesian optimization) to robustify TO for humanoid locomotion. The main idea is to use data from full-body simulations to make the TO stage robust by tuning the cost weights. To this end, we split the TO problem into two phases. The first phase solves a convex optimization problem for generating center of mass (CoM) trajectories based on simplified linear dynamics. The second stage employs iterative Linear-Quadratic Gaussian (iLQG) as a whole-body controller to generate full body control inputs. Then we use Bayesian optimization to find the cost weights to use in the first stage that yields robust performance in the simulation/experiment, in the presence of different disturbance/uncertainties. The results show that the proposed approach is able to generate robust motions for different sets of disturbances and uncertainties.

https://arxiv.org/abs/1907.04616 link (url) [BibTex]

Yeganegi, M. H., Khadiv, M., Moosavian, S. A. A., Zhu, J., Prete, A. D., Righetti, L. Robust Humanoid Locomotion Using Trajectory Optimization and Sample-Efficient Learning Proceedings International Conference on Humanoid Robots, IEEE, 2019 IEEE-RAS International Conference on Humanoid Robots, October 2019 (conference)

https://arxiv.org/abs/1907.04616 link (url) [BibTex]

Unintended Consequences of Biased Robotic and Artificial Intelligence Systems [Ethical, Legal, and Societal Issues]

Righetti, L., Madhavan, R., Chatila, R.

IEEE Robotics & Automation Magazine, 26, September 2019 (article)

[BibTex]

Righetti, L., Madhavan, R., Chatila, R. Unintended Consequences of Biased Robotic and Artificial Intelligence Systems [Ethical, Legal, and Societal Issues] IEEE Robotics & Automation Magazine, 26, September 2019 (article)

[BibTex]

Efficient Humanoid Contact Planning using Learned Centroidal Dynamics Prediction

Lin, Y., Ponton, B., Righetti, L., Berenson, D.

International Conference on Robotics and Automation (ICRA), pages: 5280-5286, IEEE, May 2019 (conference)

DOI [BibTex]

Lin, Y., Ponton, B., Righetti, L., Berenson, D. Efficient Humanoid Contact Planning using Learned Centroidal Dynamics Prediction International Conference on Robotics and Automation (ICRA), pages: 5280-5286, IEEE, May 2019 (conference)

DOI [BibTex]

Leveraging Contact Forces for Learning to Grasp

Merzic, H., Bogdanovic, M., Kappler, D., Righetti, L., Bohg, J.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) 2019, IEEE, International Conference on Robotics and Automation, May 2019 (inproceedings)

Abstract

Grasping objects under uncertainty remains an open problem in robotics research. This uncertainty is often due to noisy or partial observations of the object pose or shape. To enable a robot to react appropriately to unforeseen effects, it is crucial that it continuously takes sensor feedback into account. While visual feedback is important for inferring a grasp pose and reaching for an object, contact feedback offers valuable information during manipulation and grasp acquisition. In this paper, we use model-free deep reinforcement learning to synthesize control policies that exploit contact sensing to generate robust grasping under uncertainty. We demonstrate our approach on a multi-fingered hand that exhibits more complex finger coordination than the commonly used two- fingered grippers. We conduct extensive experiments in order to assess the performance of the learned policies, with and without contact sensing. While it is possible to learn grasping policies without contact sensing, our results suggest that contact feedback allows for a significant improvement of grasping robustness under object pose uncertainty and for objects with a complex shape.

video arXiv [BibTex]

Merzic, H., Bogdanovic, M., Kappler, D., Righetti, L., Bohg, J. Leveraging Contact Forces for Learning to Grasp In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) 2019, IEEE, International Conference on Robotics and Automation, May 2019 (inproceedings)

video arXiv [BibTex]

Optimal Stair Climbing Pattern Generation for Humanoids Using Virtual Slope and Distributed Mass Model

Shahrokhshahi, A., Yousefi-Koma, A., Khadiv, M., Mansouri, S., Mohtasebi, S. S.

Journal of Intelligent and Robotics Systems, 94:1, pages: 43-59, April 2019 (article)

DOI [BibTex]

Shahrokhshahi, A., Yousefi-Koma, A., Khadiv, M., Mansouri, S., Mohtasebi, S. S. Optimal Stair Climbing Pattern Generation for Humanoids Using Virtual Slope and Distributed Mass Model Journal of Intelligent and Robotics Systems, 94:1, pages: 43-59, April 2019 (article)

DOI [BibTex]

A Robustness Analysis of Inverse Optimal Control of Bipedal Walking

Rebula, J. R., Schaal, S., Finley, J., Righetti, L.

IEEE Robotics and Automation Letters, 4(4):4531-4538, 2019 (article)

DOI [BibTex]

Rebula, J. R., Schaal, S., Finley, J., Righetti, L. A Robustness Analysis of Inverse Optimal Control of Bipedal Walking IEEE Robotics and Automation Letters, 4(4):4531-4538, 2019 (article)

DOI [BibTex]

Rigid vs compliant contact: an experimental study on biped walking

Khadiv, M., Moosavian, S. A. A., Yousefi-Koma, A., Sadedel, M., Ehsani-Seresht, A., Mansouri, S.

Multibody System Dynamics, 45(4):379-401, 2019 (article)

DOI [BibTex]

Khadiv, M., Moosavian, S. A. A., Yousefi-Koma, A., Sadedel, M., Ehsani-Seresht, A., Mansouri, S. Rigid vs compliant contact: an experimental study on biped walking Multibody System Dynamics, 45(4):379-401, 2019 (article)

DOI [BibTex]

Birch tar production does not prove Neanderthal behavioral complexity

Schmidt, P., Blessing, M., Rageot, M., Iovita, R., Pfleging, J., Nickel, K. G., Righetti, L., Tennie, C.

Proceedings of the National Academy of Sciences (PNAS), 116(36):17707-17711, 2019 (article)

DOI [BibTex]

Schmidt, P., Blessing, M., Rageot, M., Iovita, R., Pfleging, J., Nickel, K. G., Righetti, L., Tennie, C. Birch tar production does not prove Neanderthal behavioral complexity Proceedings of the National Academy of Sciences (PNAS), 116(36):17707-17711, 2019 (article)

DOI [BibTex]

2018

A Whole-Body Model Predictive Control Scheme Including External Contact Forces and CoM Height Variations

Mirjalili, R., Yousefi-koma, A., Shirazi, F. A., Nikkhah, A., Nazemi, F., Khadiv, M.

Proceedings International Conference on Humanoid Robots, IEEE, Beijing, China, 2018 IEEE-RAS International Conference on Humanoid Robots, November 2018 (conference)

Abstract

In this paper, we present an approach for generating a variety of whole-body motions for a humanoid robot. We extend the available Model Predictive Control (MPC) approaches for walking on flat terrain to plan for both vertical motion of the Center of Mass (CoM) and external contact forces consistent with a given task. The optimization problem is comprised of three stages, i. e. the CoM vertical motion, joint angles and contact forces planning. The choice of external contact (e. g. hand contact with the object or environment) among all available locations and the appropriate time to reach and maintain a contact are all computed automatically within the algorithm. The presented algorithm benefits from the simplicity of the Linear Inverted Pendulum Model (LIPM), while it overcomes the common limitations of this model and enables us to generate a variety of whole body motions through external contacts. Simulation and experimental implementation of several whole body actions in multi-contact scenarios on a humanoid robot show the capability of the proposed algorithm.

link (url) DOI [BibTex]

2018

Mirjalili, R., Yousefi-koma, A., Shirazi, F. A., Nikkhah, A., Nazemi, F., Khadiv, M. A Whole-Body Model Predictive Control Scheme Including External Contact Forces and CoM Height Variations Proceedings International Conference on Humanoid Robots, IEEE, Beijing, China, 2018 IEEE-RAS International Conference on Humanoid Robots, November 2018 (conference)

link (url) DOI [BibTex]

Robust Physics-based Motion Retargeting with Realistic Body Shapes

Borno, M. A., Righetti, L., Black, M. J., Delp, S. L., Fiume, E., Romero, J.

Computer Graphics Forum, 37, pages: 6:1-12, July 2018 (article)

Abstract

Motion capture is often retargeted to new, and sometimes drastically different, characters. When the characters take on realistic human shapes, however, we become more sensitive to the motion looking right. This means adapting it to be consistent with the physical constraints imposed by different body shapes. We show how to take realistic 3D human shapes, approximate them using a simplified representation, and animate them so that they move realistically using physically-based retargeting. We develop a novel spacetime optimization approach that learns and robustly adapts physical controllers to new bodies and constraints. The approach automatically adapts the motion of the mocap subject to the body shape of a target subject. This motion respects the physical properties of the new body and every body shape results in a different and appropriate movement. This makes it easy to create a varied set of motions from a single mocap sequence by simply varying the characters. In an interactive environment, successful retargeting requires adapting the motion to unexpected external forces. We achieve robustness to such forces using a novel LQR-tree formulation. We show that the simulated motions look appropriate to each character’s anatomy and their actions are robust to perturbations.

pdf video Project Page [BibTex]

Borno, M. A., Righetti, L., Black, M. J., Delp, S. L., Fiume, E., Romero, J. Robust Physics-based Motion Retargeting with Realistic Body Shapes Computer Graphics Forum, 37, pages: 6:1-12, July 2018 (article)

pdf video Project Page [BibTex]

On Time Optimization of Centroidal Momentum Dynamics

Ponton, B., Herzog, A., Del Prete, A., Schaal, S., Righetti, L.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 5776-5782, IEEE, Brisbane, Australia, 2018 (inproceedings)

Abstract

Recently, the centroidal momentum dynamics has received substantial attention to plan dynamically consistent motions for robots with arms and legs in multi-contact scenarios. However, it is also non convex which renders any optimization approach difficult and timing is usually kept fixed in most trajectory optimization techniques to not introduce additional non convexities to the problem. But this can limit the versatility of the algorithms. In our previous work, we proposed a convex relaxation of the problem that allowed to efficiently compute momentum trajectories and contact forces. However, our approach could not minimize a desired angular momentum objective which seriously limited its applicability. Noticing that the non-convexity introduced by the time variables is of similar nature as the centroidal dynamics one, we propose two convex relaxations to the problem based on trust regions and soft constraints. The resulting approaches can compute time-optimized dynamically consistent trajectories sufficiently fast to make the approach realtime capable. The performance of the algorithm is demonstrated in several multi-contact scenarios for a humanoid robot. In particular, we show that the proposed convex relaxation of the original problem finds solutions that are consistent with the original non-convex problem and illustrate how timing optimization allows to find motion plans that would be difficult to plan with fixed timing † †Implementation details and demos can be found in the source code available at https://git-amd.tuebingen.mpg.de/bponton/timeoptimization.

link (url) DOI [BibTex]