Intelligent Systems
Note: This research group has relocated. Discover the updated page here

75 results (View BibTeX file of all listed publications)

2023

no image
On the Use of Torque Measurement in Centroidal State Estimation

Khorshidi, S., Gazar, A., Rotella, N., Naveau, M., Righetti, L., Bennewitz, M., Khadiv, M.

IEEE International Conference on Robotics and Automation (ICRA), pages: 9931-9937, May 2023 (conference)

DOI [BibTex]

2023

Khorshidi, S., Gazar, A., Rotella, N., Naveau, M., Righetti, L., Bennewitz, M., Khadiv, M. On the Use of Torque Measurement in Centroidal State Estimation IEEE International Conference on Robotics and Automation (ICRA), pages: 9931-9937, May 2023 (conference)

DOI [BibTex]

Visual-Inertial and Leg Odometry Fusion for Dynamic Locomotion
Visual-Inertial and Leg Odometry Fusion for Dynamic Locomotion

Dhédin, V., Li, H., Khorshidi, S., Mack, L., Ravi, A. K. C., Meduri, A., Shah, P., Grimminger, F., Righetti, L., Khadiv, M., Stueckler, J.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023 (inproceedings)

Abstract
Implementing dynamic locomotion behaviors on legged robots requires a high-quality state estimation module. Especially when the motion includes flight phases, state-of-the-art approaches fail to produce reliable estimation of the robot posture, in particular base height. In this paper, we propose a novel approach for combining visual-inertial odometry (VIO) with leg odometry in an extended Kalman filter (EKF) based state estimator. The VIO module uses a stereo camera and IMU to yield low-drift 3D position and yaw orientation and drift-free pitch and roll orientation of the robot base link in the inertial frame. However, these values have a considerable amount of latency due to image processing and optimization, while the rate of update is quite low which is not suitable for low-level control. To reduce the latency, we predict the VIO state estimate at the rate of the IMU measurements of the VIO sensor. The EKF module uses the base pose and linear velocity predicted by VIO, fuses them further with a second high-rate IMU and leg odometry measurements, and produces robot state estimates with a high frequency and small latency suitable for control. We integrate this lightweight estimation framework with a nonlinear model predictive controller and show successful implementation of a set of agile locomotion behaviors, including trotting and jumping at varying horizontal speeds, on a torque-controlled quadruped robot.

preprint video link (url) DOI [BibTex]

Dhédin, V., Li, H., Khorshidi, S., Mack, L., Ravi, A. K. C., Meduri, A., Shah, P., Grimminger, F., Righetti, L., Khadiv, M., Stueckler, J. Visual-Inertial and Leg Odometry Fusion for Dynamic Locomotion In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023 (inproceedings)

preprint video link (url) DOI [BibTex]

2022

no image
Introducing Force Feedback in Model Predictive Control

Kleff, E. D. E. S. G. M. N. R. L.

Proceedings of the 2022 International Conference on Intelligent Robots and Systems (IROS), pages: 13379-13385, IEEE, International Conference on Intelligent Robots and Systems (IROS), October 2022 (conference)

DOI [BibTex]

2022

Kleff, E. D. E. S. G. M. N. R. L. Introducing Force Feedback in Model Predictive Control Proceedings of the 2022 International Conference on Intelligent Robots and Systems (IROS), pages: 13379-13385, IEEE, International Conference on Intelligent Robots and Systems (IROS), October 2022 (conference)

DOI [BibTex]

no image
Nonlinear Stochastic Trajectory Optimization for Centroidal Momentum Motion Generation of Legged Robots

Gazar, A., Khadiv, M., Kleff, S., DelPrete, A., Righetti, L.

In Robotics Research, pages: 420-435, Springer Proceedings in Advanced Robotics, 27, (Editors: Billard, Aude and Asfour, Tamim and Khatib, Oussama), Springer, Cham, 20th International Symposium on Robotics Research (ISRR 2022), September 2022 (inproceedings)

Abstract
Generation of robust trajectories for legged robots remains a challenging task due to the underlying nonlinear, hybrid and intrinsically unstable dynamics which needs to be stabilized through limited contact forces. Furthermore, disturbances arising from unmodelled contact interactions with the environment and model mismatches can hinder the quality of the planned trajectories leading to unsafe motions. In this work, we propose to use stochastic trajectory optimization for generating robust centroidal momentum trajectories to account for additive uncertainties on the model dynamics and parametric uncertainties on contact locations. Through an alternation between the robust centroidal and whole-body trajectory optimizations, we generate robust momentum trajectories while being consistent with the whole-body dynamics. We perform an extensive set of simulations subject to different uncertainties on a quadruped robot showing that our stochastic trajectory optimization problem reduces the amount of foot slippage for different gaits while achieving better performance over deterministic planning.

link (url) DOI [BibTex]

Gazar, A., Khadiv, M., Kleff, S., DelPrete, A., Righetti, L. Nonlinear Stochastic Trajectory Optimization for Centroidal Momentum Motion Generation of Legged Robots In Robotics Research, pages: 420-435, Springer Proceedings in Advanced Robotics, 27, (Editors: Billard, Aude and Asfour, Tamim and Khatib, Oussama), Springer, Cham, 20th International Symposium on Robotics Research (ISRR 2022), September 2022 (inproceedings)

link (url) DOI [BibTex]

no image
iRiSC: Iterative Risk Sensitive Control for Nonlinear Systems with Imperfect Observations

Hammoud, B., Jordana, A., Righetti, L.

In 2022 American Control Conference (ACC 2022), pages: 3550-3557, IEEE, Piscataway, NJ, American Control Conference (ACC 2022), June 2022 (inproceedings)

DOI [BibTex]

Hammoud, B., Jordana, A., Righetti, L. iRiSC: Iterative Risk Sensitive Control for Nonlinear Systems with Imperfect Observations In 2022 American Control Conference (ACC 2022), pages: 3550-3557, IEEE, Piscataway, NJ, American Control Conference (ACC 2022), June 2022 (inproceedings)

DOI [BibTex]

2021

no image
A unified framework for walking and running of bipedal robots

Boroujeni, M. G., Daneshmand, E., Righetti, L., Khadiv, M.

20th International Conference on Advanced Robotics (ICAR), December 2021 (conference) Accepted

Abstract
In this paper, we propose a novel framework capable of generating various walking and running gaits for bipedal robots. The main goal is to relax the fixed center of mass (CoM) height assumption of the linear inverted pendulum model (LIPM) and generate a wider range of walking and running motions, without a considerable increase in complexity. To do so, we use the concept of virtual constraints in the centroidal space which enables generating motions beyond walking while keeping the complexity at a minimum. By a proper choice of these virtual constraints, we show that we can generate different types of walking and running motions. More importantly, enforcing the virtual constraints through feedback renders the dynamics linear and enables us to design a feedback control mechanism which adapts the next step location and timing in face of disturbances, through a simple quadratic program (QP). To show the effectiveness of this framework, we showcase different walking and running simulations of the biped robot Bolt in the presence of both environmental uncertainties and external disturbances.

link (url) [BibTex]

2021

Boroujeni, M. G., Daneshmand, E., Righetti, L., Khadiv, M. A unified framework for walking and running of bipedal robots 20th International Conference on Advanced Robotics (ICAR), December 2021 (conference) Accepted

link (url) [BibTex]

no image
Rapid Convex Optimization of Centroidal Dynamics using Block Coordinate Descent

Shah, P., Meduri, A., Merkt, W., Khadiv, M., Havoutis, I., Righetti, L.

In Proceedings of the International Conference on Intelligent Robots and Systems (IROS 2021) , IEEE, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021) in Prague, Czech Republic., September 2021 (inproceedings) Accepted

Abstract
In this paper we explore the use of block coordinate descent (BCD) to optimize the centroidal momentum dynamics for dynamically consistent multi-contact behaviors. The centroidal dynamics have recently received a large amount of attention in order to create physically realizable motions for robots with hands and feet while being computationally more tractable than full rigid body dynamics models. Our contribution lies in exploiting the structure of the dynamics in order to simplify the original non-convex problem into two convex subproblems. We iterate between these two subproblems for a set number of iterations or until a consensus is reached. We explore the properties of the proposed optimization method for the centroidal dynamics and verify in simulation that motions generated by our approach can be tracked by the quadruped Solo12. In addition, we compare our method to a recently proposed convexification using a sequence of convex relaxations as well as a more standard interior point method used in the off- the-shelf solver IPOPT to show that our approach finds similar, if not better, trajectories (in terms of cost), and is more than four times faster than both approaches. Finally, compared to previous approaches, we note its practicality due to the convex nature of each subproblem which allows our method to be used with any off-the-shelf quadratic programming solver.

link (url) [BibTex]

Shah, P., Meduri, A., Merkt, W., Khadiv, M., Havoutis, I., Righetti, L. Rapid Convex Optimization of Centroidal Dynamics using Block Coordinate Descent In Proceedings of the International Conference on Intelligent Robots and Systems (IROS 2021) , IEEE, 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2021) in Prague, Czech Republic., September 2021 (inproceedings) Accepted

link (url) [BibTex]

no image
Stochastic and robust mpc for bipedal locomotion: A comparative study on robustness and performance

Gazar, A., Khadiv, M., DelPrete, A., Righetti, L.

pages: 1-8, IEEE, IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids), July 2021 (conference)

Abstract
Linear Model Predictive Control (MPC) has been successfully used for generating feasible walking motions for humanoid robots. However, the effect of uncertainties on constraints satisfaction has only been studied using Robust MPC (RMPC) approaches, which account for the worst-case realization of bounded disturbances at each time instant. In this letter, we propose for the first time to use linear stochastic MPC (SMPC) to account for uncertainties in bipedal walking. We show that SMPC offers more flexibility to the user (or a high level decision maker) by tolerating small (user-defined) probabilities of constraint violation. Therefore, SMPC can be tuned to achieve a constraint satisfaction probability that is arbitrarily close to 100%, but without sacrificing performance as much as tube-based RMPC. We compare SMPC against RMPC in terms of robustness (constraint satisfaction) and performance (optimality). Our results highlight the benefits of SMPC and its interest for the robotics community as a powerful mathematical tool for dealing with uncertainties.

DOI [BibTex]

Gazar, A., Khadiv, M., DelPrete, A., Righetti, L. Stochastic and robust mpc for bipedal locomotion: A comparative study on robustness and performance pages: 1-8, IEEE, IEEE-RAS 20th International Conference on Humanoid Robots (Humanoids), July 2021 (conference)

DOI [BibTex]

no image
High-frequency nonlinear model predictive control of a manipulator

Kleff, S., Meduri, A., Budhiraja, R., Mansard, N., Righetti, L.

In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages: 7330-7336 , The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (inproceedings)

Abstract
Model Predictive Control (MPC) promises to endow robots with enough reactivity to perform complex tasks in dynamic environments by frequently updating their motion plan based on measurements. Despite its appeal, it has seldom been deployed on real machines because of scaling constraints. This paper presents the first hardware implementation of closed-loop nonlinear MPC on a 7-DoF torque-controlled robot. Our controller leverages a state-of-the art optimal control solver, namely Differential Dynamic Programming (DDP), in order to replan state and control trajectories at real-time rates (1kHz). In addition to this experimental proof of concept, we present exhaustive performance analysis on the iconic pick-and-place task and show that our controller outperforms open-loop MPC. We also exhibit the importance of a sufficient preview horizon and full robot dynamics in the controller performance through comparisons with inverse dynamics and kinematic optimization.

DOI [BibTex]

Kleff, S., Meduri, A., Budhiraja, R., Mansard, N., Righetti, L. High-frequency nonlinear model predictive control of a manipulator In 2021 IEEE International Conference on Robotics and Automation (ICRA), pages: 7330-7336 , The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (inproceedings)

DOI [BibTex]

no image
Leveraging Forward Model Prediction Error for Learning Control

Bechtle, S., Hammoud, B., Rai, A., Meier, F., Righetti, L.

2021 IEEE International Conference on Robotics and Automation (ICRA) , pages: 4445-4451 , IEEE International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

Abstract
Learning for model based control can be sample-efficient and generalize well, however successfully learning models and controllers that represent the problem at hand can be challenging for complex tasks. Using inaccurate models for learning can lead to sub-optimal solutions, that are unlikely to perform well in practice. In this work, we present a learning approach which iterates between model learning and data collection and leverages forward model prediction error for learning control. We show how using the controller's prediction as input to a forward model can create a differentiable connection between the controller and the model, allowing us to formulate a loss in the state space. This lets us include forward model prediction error during controller learning and we show that this creates a loss objective that significantly improves learning on different motor control tasks. We provide empirical and theoretical results that show the benefits of our method and present evaluations in simulation for learning control on a 7 DoF manipulator and an underactuated 12 DoF quadruped. We show that our approach successfully learns controllers for challenging motor control tasks involving contact switching.

link (url) [BibTex]

Bechtle, S., Hammoud, B., Rai, A., Meier, F., Righetti, L. Leveraging Forward Model Prediction Error for Learning Control 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages: 4445-4451 , IEEE International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

link (url) [BibTex]

no image
Learning a Centroidal Motion Planner for Legged Locomotion

Viereck, J., Righetti, L.

2021 IEEE International Conference on Robotics and Automation (ICRA), pages: 4905-4911 , IEEE, The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

Abstract
Whole-body optimizers have been successful at automatically computing complex dynamic locomotion behaviors. However they are often limited to offline planning as they are computationally too expensive to replan with a high frequency. Simpler models are then typically used for online replanning. In this paper we present a method to generate whole body movements in real-time for locomotion tasks. Our approach consists in learning a centroidal neural network that predicts the desired centroidal motion given the current state of the robot and a desired contact plan. The network is trained using an existing whole body motion optimizer. Our approach enables to learn with few training samples dynamic motions that can be used in a complete whole-body control framework at high frequency, which is usually not attainable with typical full-body optimizers. We demonstrate our method to generate a rich set of walking and jumping motions on a real quadruped robot.

DOI [BibTex]

Viereck, J., Righetti, L. Learning a Centroidal Motion Planner for Legged Locomotion 2021 IEEE International Conference on Robotics and Automation (ICRA), pages: 4905-4911 , IEEE, The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

DOI [BibTex]

no image
DeepQ Stepper: A framework for reactive dynamic walking on uneven terrain

Meduri, A., Khadiv, M., Righetti, L.

The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

Abstract
Reactive stepping and push recovery for biped robots is often restricted to flat terrains because of the difficulty in computing capture regions for nonlinear dynamic models. In this paper, we address this limitation by using reinforcement learning to approximately learn the 3D capture region for such systems. We propose a novel 3D reactive stepper, The DeepQ stepper, that computes optimal step locations for walking at different velocities using the 3D capture regions approximated by the action-value function. We demonstrate the ability of the approach to learn stepping with a simplified 3D pendulum model and a full robot dynamics. Further, the stepper achieves a higher performance when it learns approximate capture regions while taking into account the entire dynamics of the robot that are often ignored in existing reactive steppers based on simplified models. The DeepQ stepper can handle non convex terrain with obstacles, walk on restricted surfaces like stepping stones and recover from external disturbances for a constant computational cost.

link (url) [BibTex]

Meduri, A., Khadiv, M., Righetti, L. DeepQ Stepper: A framework for reactive dynamic walking on uneven terrain The 2021 International Conference on Robotics and Automation (ICRA 2021), June 2021 (conference)

link (url) [BibTex]

no image
Meta-Learning via Learned Loss

Bechtle, S., Molchanov, A., Chebotar, Y., Grefenstette, E., Righetti, L., Sukhatme, G., Meier, F.

In 2020 25th International Conference on Pattern Recognition (ICPR), IEEE, January 2021 (inproceedings)

Abstract
Typically, loss functions, regularization mechanisms and other important aspects of training parametric models are chosen heuristically from a limited set of options. In this paper, we take the first step towards automating this process, with the view of producing models which train faster and more robustly. Concretely, we present a meta-learning method for learning parametric loss functions that can generalize across different tasks and model architectures. We develop a pipeline for “meta-training” such loss functions, targeted at maximizing the performance of the model trained under them. The loss landscape produced by our learned losses significantly improves upon the original task-specific losses in both supervised and reinforcement learning tasks. Furthermore, we show that our meta-learning framework is flexible enough to incorporate additional informa- tion at meta-train time. This information shapes the learned loss function such that the environment does not need to provide this information during meta-test time. We make our code available at https://sites.google.com/view/mlthree

[BibTex]

Bechtle, S., Molchanov, A., Chebotar, Y., Grefenstette, E., Righetti, L., Sukhatme, G., Meier, F. Meta-Learning via Learned Loss In 2020 25th International Conference on Pattern Recognition (ICPR), IEEE, January 2021 (inproceedings)

[BibTex]

2020

TriFinger: An Open-Source Robot for Learning Dexterity
TriFinger: An Open-Source Robot for Learning Dexterity

Wüthrich, M., Widmaier, F., Grimminger, F., Akpo, J., Joshi, S., Agrawal, V., Hammoud, B., Khadiv, M., Bogdanovic, M., Berenz, V., Viereck, J., Naveau, M., Righetti, L., Schölkopf, B., Bauer, S.

Proceedings of the 4th Conference on Robot Learning (CoRL), 155, pages: 1871-1882, Proceedings of Machine Learning Research, (Editors: Jens Kober and Fabio Ramos and Claire J. Tomlin), PMLR, November 2020 (conference)

PDF link (url) [BibTex]

2020

Wüthrich, M., Widmaier, F., Grimminger, F., Akpo, J., Joshi, S., Agrawal, V., Hammoud, B., Khadiv, M., Bogdanovic, M., Berenz, V., Viereck, J., Naveau, M., Righetti, L., Schölkopf, B., Bauer, S. TriFinger: An Open-Source Robot for Learning Dexterity Proceedings of the 4th Conference on Robot Learning (CoRL), 155, pages: 1871-1882, Proceedings of Machine Learning Research, (Editors: Jens Kober and Fabio Ramos and Claire J. Tomlin), PMLR, November 2020 (conference)

PDF link (url) [BibTex]

no image
Enabling Remote Whole-Body Control with 5G Edge Computing

Zhu, H., Sharma, M., Pfeiffer, K., Mezzavilla, M., Shen, J., Rangan, S., Righetti, L.

IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 3553-3560, IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2020 (conference) Accepted

Abstract
Real-world applications require light-weight, energy-efficient, fully autonomous robots. Yet, increasing auton- omy is oftentimes synonymous with escalating computational requirements. It might thus be desirable to offload intensive computation—not only sensing and planning, but also low- level whole-body control—to remote servers in order to reduce on-board computational needs. Fifth Generation (5G) wireless cellular technology, with its low latency and high bandwidth capabilities, has the potential to unlock cloud-based high per- formance control of complex robots. However, state-of-the-art control algorithms for legged robots can only tolerate very low control delays, which even ultra-low latency 5G edge computing can sometimes fail to achieve. In this work, we investigate the problem of cloud-based whole-body control of legged robots over a 5G link. We propose a novel approach that consists of a standard optimization-based controller on the network edge and a local linear, approximately optimal controller that significantly reduces on-board computational needs while increasing robustness to delay and possible loss of commu- nication. Simulation experiments on humanoid balancing and walking tasks that includes a realistic 5G communication model demonstrate significant improvement of the reliability of robot locomotion under jitter and delays likely to be experienced in 5G wireless links.

link (url) DOI [BibTex]

Zhu, H., Sharma, M., Pfeiffer, K., Mezzavilla, M., Shen, J., Rangan, S., Righetti, L. Enabling Remote Whole-Body Control with 5G Edge Computing IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 3553-3560, IEEE, Piscataway, NJ, USA, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2020 (conference) Accepted

link (url) DOI [BibTex]

no image
Crocoddyl: An efficient and versatile framework for multi-contact optimal control

Mastalli, C., Budhiraja, R., Merkt, W., Saurel, G., Hammoud, B., Naveau, M., Carpentier, J., Righetti, L., Vijayakumar, S., Mansard, N.

In Proceedings of the IEEE International Conference on Robotics and Automation, IEEE, International Conference on Robotics and Automation, May 2020 (inproceedings)

Abstract
We introduce Crocoddyl (Contact RObot COntrol by Differential DYnamic Library), an open-source framework tailored for efficient multi-contact optimal control. Crocoddyl efficiently computes the state trajectory and the control policy for a given predefined sequence of contacts. Its efficiency is due to the use of sparse analytical derivatives, exploitation of the problem structure, and data sharing. It employs differential geometry to properly describe the state of any geometrical system, e.g. floating-base systems. Additionally, we propose a novel optimal control algorithm called Feasibility-driven Differential Dynamic Programming (FDDP). Our method does not add extra decision variables which often increases the computation time per iteration due to factorization. FDDP shows a greater globalization strategy compared to classical Differential Dynamic Programming (DDP) algorithms. Con- cretely, we propose two modifications to the classical DDP algo- rithm. First, the backward pass accepts infeasible state-control trajectories. Second, the rollout keeps the gaps open during the early “exploratory” iterations (as expected in multiple- shooting methods with only equality constraints). We showcase the performance of our framework using different tasks. With our method, we can compute highly-dynamic maneuvers (e.g. jumping, front-flip) within few milliseconds.

[BibTex]

Mastalli, C., Budhiraja, R., Merkt, W., Saurel, G., Hammoud, B., Naveau, M., Carpentier, J., Righetti, L., Vijayakumar, S., Mansard, N. Crocoddyl: An efficient and versatile framework for multi-contact optimal control In Proceedings of the IEEE International Conference on Robotics and Automation, IEEE, International Conference on Robotics and Automation, May 2020 (inproceedings)

[BibTex]

no image
Curious ilqr: Resolving uncertainty in model-based rl

Bechtle, S., Lin, Y., Rai, A., Righetti, L., Meier, F.

Conference on Robot Learning, May 2020 (conference)

Abstract
Curiosity as a means to explore during reinforcement learning problems has recently become very popular. However, very little progress has been made in utilizing curiosity for learning control. In this work, we propose a model-based reinforcement learning (MBRL) framework that combines Bayesian modeling of the system dynamics with curious iLQR, an iterative LQR approach that considers model uncertainty. During trajectory optimization the curious iLQR attempts to minimize both the task-dependent cost and the uncertainty in the dynamics model. We demonstrate the approach on reaching tasks with 7-DoF manipulators in simulation and on a real robot. Our experiments show that MBRL with curious iLQR reaches desired end-effector targets more reliably and with less system rollouts when learning a new task from scratch, and that the learned model generalizes better to new reaching tasks.

[BibTex]

Bechtle, S., Lin, Y., Rai, A., Righetti, L., Meier, F. Curious ilqr: Resolving uncertainty in model-based rl Conference on Robot Learning, May 2020 (conference)

[BibTex]

no image
A Real-Robot Dataset for Assessing Transferability of Learned Dynamics Models

Agudelo-España, D., Zadaianchuk, A., Wenk, P., Garg, A., Akpo, J., Grimminger, F., Viereck, J., Naveau, M., Righetti, L., Martius, G., Krause, A., Schölkopf, B., Bauer, S., Wüthrich, M.

IEEE International Conference on Robotics and Automation (ICRA), pages: 8151-8157, IEEE, 2020 (conference)

Project Page PDF DOI Project Page [BibTex]

Agudelo-España, D., Zadaianchuk, A., Wenk, P., Garg, A., Akpo, J., Grimminger, F., Viereck, J., Naveau, M., Righetti, L., Martius, G., Krause, A., Schölkopf, B., Bauer, S., Wüthrich, M. A Real-Robot Dataset for Assessing Transferability of Learned Dynamics Models IEEE International Conference on Robotics and Automation (ICRA), pages: 8151-8157, IEEE, 2020 (conference)

Project Page PDF DOI Project Page [BibTex]

no image
EXPLORING BY EXPLOITING BAD MODELS IN MODEL-BASED REINFORCEMENT LEARNING

Lin, Y., Bechtle, S., Righetti, L., Rai, A., Meier, F.

International Conference on Learning Representations, 2020 (conference)

Abstract
Exploration for reinforcement learning (RL) is well-studied for model-free methods but a relatively unexplored topic for model-based methods. In this work, we investigate several exploration techniques injected into the two stages of model-based RL:(1) during optimization: adding transition-space and action-space noise when optimizing a policy using learned dynamics, and (2) after optimization: injecting action-space noise when executing an optimized policy on the real environment. When given a good deterministic dynamics model, like the ground-truth simulation, exploration can significantly improve performance. However, using randomly initialized neural networks to model environment dynamics can _implicitly_ induce exploration in model-based RL, reducing the need for explicit exploratory techniques. Surprisingly, we show that in the case of a local optimizer, using a learned model with this implicit exploration can actually _outperform_ using the ground-truth model without exploration, while adding exploration to the ground-truth model reduces the performance gap. However, the learned models are highly local, in that they perform well _only_ for the task for which it is optimized, and fail to generalize to new targets.

[BibTex]

Lin, Y., Bechtle, S., Righetti, L., Rai, A., Meier, F. EXPLORING BY EXPLOITING BAD MODELS IN MODEL-BASED REINFORCEMENT LEARNING International Conference on Learning Representations, 2020 (conference)

[BibTex]

2019

Learning to Explore in Motion and Interaction Tasks
Learning to Explore in Motion and Interaction Tasks

Bogdanovic, M., Righetti, L.

Proceedings 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 2686-2692, IEEE, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019, ISSN: 2153-0866 (conference)

Abstract
Model free reinforcement learning suffers from the high sampling complexity inherent to robotic manipulation or locomotion tasks. Most successful approaches typically use random sampling strategies which leads to slow policy convergence. In this paper we present a novel approach for efficient exploration that leverages previously learned tasks. We exploit the fact that the same system is used across many tasks and build a generative model for exploration based on data from previously solved tasks to improve learning new tasks. The approach also enables continuous learning of improved exploration strategies as novel tasks are learned. Extensive simulations on a robot manipulator performing a variety of motion and contact interaction tasks demonstrate the capabilities of the approach. In particular, our experiments suggest that the exploration strategy can more than double learning speed, especially when rewards are sparse. Moreover, the algorithm is robust to task variations and parameter tuning, making it beneficial for complex robotic problems.

DOI [BibTex]

2019

Bogdanovic, M., Righetti, L. Learning to Explore in Motion and Interaction Tasks Proceedings 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 2686-2692, IEEE, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), November 2019, ISSN: 2153-0866 (conference)

DOI [BibTex]

no image
Robust Humanoid Locomotion Using Trajectory Optimization and Sample-Efficient Learning

Yeganegi, M. H., Khadiv, M., Moosavian, S. A. A., Zhu, J., Prete, A. D., Righetti, L.

Proceedings International Conference on Humanoid Robots, IEEE, 2019 IEEE-RAS International Conference on Humanoid Robots, October 2019 (conference)

Abstract
Trajectory optimization (TO) is one of the most powerful tools for generating feasible motions for humanoid robots. However, including uncertainties and stochasticity in the TO problem to generate robust motions can easily lead to intractable problems. Furthermore, since the models used in TO have always some level of abstraction, it can be hard to find a realistic set of uncertainties in the model space. In this paper we leverage a sample-efficient learning technique (Bayesian optimization) to robustify TO for humanoid locomotion. The main idea is to use data from full-body simulations to make the TO stage robust by tuning the cost weights. To this end, we split the TO problem into two phases. The first phase solves a convex optimization problem for generating center of mass (CoM) trajectories based on simplified linear dynamics. The second stage employs iterative Linear-Quadratic Gaussian (iLQG) as a whole-body controller to generate full body control inputs. Then we use Bayesian optimization to find the cost weights to use in the first stage that yields robust performance in the simulation/experiment, in the presence of different disturbance/uncertainties. The results show that the proposed approach is able to generate robust motions for different sets of disturbances and uncertainties.

https://arxiv.org/abs/1907.04616 link (url) [BibTex]

Yeganegi, M. H., Khadiv, M., Moosavian, S. A. A., Zhu, J., Prete, A. D., Righetti, L. Robust Humanoid Locomotion Using Trajectory Optimization and Sample-Efficient Learning Proceedings International Conference on Humanoid Robots, IEEE, 2019 IEEE-RAS International Conference on Humanoid Robots, October 2019 (conference)

https://arxiv.org/abs/1907.04616 link (url) [BibTex]

no image
Efficient Humanoid Contact Planning using Learned Centroidal Dynamics Prediction

Lin, Y., Ponton, B., Righetti, L., Berenson, D.

International Conference on Robotics and Automation (ICRA), pages: 5280-5286, IEEE, May 2019 (conference)

DOI [BibTex]

Lin, Y., Ponton, B., Righetti, L., Berenson, D. Efficient Humanoid Contact Planning using Learned Centroidal Dynamics Prediction International Conference on Robotics and Automation (ICRA), pages: 5280-5286, IEEE, May 2019 (conference)

DOI [BibTex]

Leveraging Contact Forces for Learning to Grasp
Leveraging Contact Forces for Learning to Grasp

Merzic, H., Bogdanovic, M., Kappler, D., Righetti, L., Bohg, J.

In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) 2019, IEEE, International Conference on Robotics and Automation, May 2019 (inproceedings)

Abstract
Grasping objects under uncertainty remains an open problem in robotics research. This uncertainty is often due to noisy or partial observations of the object pose or shape. To enable a robot to react appropriately to unforeseen effects, it is crucial that it continuously takes sensor feedback into account. While visual feedback is important for inferring a grasp pose and reaching for an object, contact feedback offers valuable information during manipulation and grasp acquisition. In this paper, we use model-free deep reinforcement learning to synthesize control policies that exploit contact sensing to generate robust grasping under uncertainty. We demonstrate our approach on a multi-fingered hand that exhibits more complex finger coordination than the commonly used two- fingered grippers. We conduct extensive experiments in order to assess the performance of the learned policies, with and without contact sensing. While it is possible to learn grasping policies without contact sensing, our results suggest that contact feedback allows for a significant improvement of grasping robustness under object pose uncertainty and for objects with a complex shape.

video arXiv [BibTex]

Merzic, H., Bogdanovic, M., Kappler, D., Righetti, L., Bohg, J. Leveraging Contact Forces for Learning to Grasp In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) 2019, IEEE, International Conference on Robotics and Automation, May 2019 (inproceedings)

video arXiv [BibTex]

2018

no image
A Whole-Body Model Predictive Control Scheme Including External Contact Forces and CoM Height Variations

Mirjalili, R., Yousefi-koma, A., Shirazi, F. A., Nikkhah, A., Nazemi, F., Khadiv, M.

Proceedings International Conference on Humanoid Robots, IEEE, Beijing, China, 2018 IEEE-RAS International Conference on Humanoid Robots, November 2018 (conference)

Abstract
In this paper, we present an approach for generating a variety of whole-body motions for a humanoid robot. We extend the available Model Predictive Control (MPC) approaches for walking on flat terrain to plan for both vertical motion of the Center of Mass (CoM) and external contact forces consistent with a given task. The optimization problem is comprised of three stages, i. e. the CoM vertical motion, joint angles and contact forces planning. The choice of external contact (e. g. hand contact with the object or environment) among all available locations and the appropriate time to reach and maintain a contact are all computed automatically within the algorithm. The presented algorithm benefits from the simplicity of the Linear Inverted Pendulum Model (LIPM), while it overcomes the common limitations of this model and enables us to generate a variety of whole body motions through external contacts. Simulation and experimental implementation of several whole body actions in multi-contact scenarios on a humanoid robot show the capability of the proposed algorithm.

link (url) DOI [BibTex]

2018

Mirjalili, R., Yousefi-koma, A., Shirazi, F. A., Nikkhah, A., Nazemi, F., Khadiv, M. A Whole-Body Model Predictive Control Scheme Including External Contact Forces and CoM Height Variations Proceedings International Conference on Humanoid Robots, IEEE, Beijing, China, 2018 IEEE-RAS International Conference on Humanoid Robots, November 2018 (conference)

link (url) DOI [BibTex]

no image
On Time Optimization of Centroidal Momentum Dynamics

Ponton, B., Herzog, A., Del Prete, A., Schaal, S., Righetti, L.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 5776-5782, IEEE, Brisbane, Australia, 2018 (inproceedings)

Abstract
Recently, the centroidal momentum dynamics has received substantial attention to plan dynamically consistent motions for robots with arms and legs in multi-contact scenarios. However, it is also non convex which renders any optimization approach difficult and timing is usually kept fixed in most trajectory optimization techniques to not introduce additional non convexities to the problem. But this can limit the versatility of the algorithms. In our previous work, we proposed a convex relaxation of the problem that allowed to efficiently compute momentum trajectories and contact forces. However, our approach could not minimize a desired angular momentum objective which seriously limited its applicability. Noticing that the non-convexity introduced by the time variables is of similar nature as the centroidal dynamics one, we propose two convex relaxations to the problem based on trust regions and soft constraints. The resulting approaches can compute time-optimized dynamically consistent trajectories sufficiently fast to make the approach realtime capable. The performance of the algorithm is demonstrated in several multi-contact scenarios for a humanoid robot. In particular, we show that the proposed convex relaxation of the original problem finds solutions that are consistent with the original non-convex problem and illustrate how timing optimization allows to find motion plans that would be difficult to plan with fixed timing † †Implementation details and demos can be found in the source code available at https://git-amd.tuebingen.mpg.de/bponton/timeoptimization.

link (url) DOI [BibTex]

Ponton, B., Herzog, A., Del Prete, A., Schaal, S., Righetti, L. On Time Optimization of Centroidal Momentum Dynamics In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 5776-5782, IEEE, Brisbane, Australia, 2018 (inproceedings)

link (url) DOI [BibTex]

no image
Unsupervised Contact Learning for Humanoid Estimation and Control

Rotella, N., Schaal, S., Righetti, L.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 411-417, IEEE, Brisbane, Australia, 2018 (inproceedings)

Abstract
This work presents a method for contact state estimation using fuzzy clustering to learn contact probability for full, six-dimensional humanoid contacts. The data required for training is solely from proprioceptive sensors - endeffector contact wrench sensors and inertial measurement units (IMUs) - and the method is completely unsupervised. The resulting cluster means are used to efficiently compute the probability of contact in each of the six endeffector degrees of freedom (DoFs) independently. This clustering-based contact probability estimator is validated in a kinematics-based base state estimator in a simulation environment with realistic added sensor noise for locomotion over rough, low-friction terrain on which the robot is subject to foot slip and rotation. The proposed base state estimator which utilizes these six DoF contact probability estimates is shown to perform considerably better than that which determines kinematic contact constraints purely based on measured normal force.

link (url) DOI [BibTex]

Rotella, N., Schaal, S., Righetti, L. Unsupervised Contact Learning for Humanoid Estimation and Control In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 411-417, IEEE, Brisbane, Australia, 2018 (inproceedings)

link (url) DOI [BibTex]

no image
Learning Task-Specific Dynamics to Improve Whole-Body Control

Gams, A., Mason, S., Ude, A., Schaal, S., Righetti, L.

In Hua, IEEE, Beijing, China, November 2018 (inproceedings)

Abstract
In task-based inverse dynamics control, reference accelerations used to follow a desired plan can be broken down into feedforward and feedback trajectories. The feedback term accounts for tracking errors that are caused from inaccurate dynamic models or external disturbances. On underactuated, free-floating robots, such as humanoids, high feedback terms can be used to improve tracking accuracy; however, this can lead to very stiff behavior or poor tracking accuracy due to limited control bandwidth. In this paper, we show how to reduce the required contribution of the feedback controller by incorporating learned task-space reference accelerations. Thus, we i) improve the execution of the given specific task, and ii) offer the means to reduce feedback gains, providing for greater compliance of the system. With a systematic approach we also reduce heuristic tuning of the model parameters and feedback gains, often present in real-world experiments. In contrast to learning task-specific joint-torques, which might produce a similar effect but can lead to poor generalization, our approach directly learns the task-space dynamics of the center of mass of a humanoid robot. Simulated and real-world results on the lower part of the Sarcos Hermes humanoid robot demonstrate the applicability of the approach.

link (url) [BibTex]

Gams, A., Mason, S., Ude, A., Schaal, S., Righetti, L. Learning Task-Specific Dynamics to Improve Whole-Body Control In Hua, IEEE, Beijing, China, November 2018 (inproceedings)

link (url) [BibTex]

no image
An MPC Walking Framework With External Contact Forces

Mason, S., Rotella, N., Schaal, S., Righetti, L.

In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 1785-1790, IEEE, Brisbane, Australia, May 2018 (inproceedings)

Abstract
In this work, we present an extension to a linear Model Predictive Control (MPC) scheme that plans external contact forces for the robot when given multiple contact locations and their corresponding friction cone. To this end, we set up a two-step optimization problem. In the first optimization, we compute the Center of Mass (CoM) trajectory, foot step locations, and introduce slack variables to account for violating the imposed constraints on the Zero Moment Point (ZMP). We then use the slack variables to trigger the second optimization, in which we calculate the optimal external force that compensates for the ZMP tracking error. This optimization considers multiple contacts positions within the environment by formulating the problem as a Mixed Integer Quadratic Program (MIQP) that can be solved at a speed between 100-300 Hz. Once contact is created, the MIQP reduces to a single Quadratic Program (QP) that can be solved in real-time ({\textless}; 1kHz). Simulations show that the presented walking control scheme can withstand disturbances 2-3× larger with the additional force provided by a hand contact.

link (url) DOI [BibTex]

Mason, S., Rotella, N., Schaal, S., Righetti, L. An MPC Walking Framework With External Contact Forces In 2018 IEEE International Conference on Robotics and Automation (ICRA), pages: 1785-1790, IEEE, Brisbane, Australia, May 2018 (inproceedings)

link (url) DOI [BibTex]

2017

no image
Pattern Generation for Walking on Slippery Terrains

Khadiv, M., Moosavian, S. A. A., Herzog, A., Righetti, L.

In 2017 5th International Conference on Robotics and Mechatronics (ICROM), Iran, August 2017 (inproceedings)

Abstract
In this paper, we extend state of the art Model Predictive Control (MPC) approaches to generate safe bipedal walking on slippery surfaces. In this setting, we formulate walking as a trade off between realizing a desired walking velocity and preserving robust foot-ground contact. Exploiting this for- mulation inside MPC, we show that safe walking on various flat terrains can be achieved by compromising three main attributes, i. e. walking velocity tracking, the Zero Moment Point (ZMP) modulation, and the Required Coefficient of Friction (RCoF) regulation. Simulation results show that increasing the walking velocity increases the possibility of slippage, while reducing the slippage possibility conflicts with reducing the tip-over possibility of the contact and vice versa.

link (url) [BibTex]

2017

Khadiv, M., Moosavian, S. A. A., Herzog, A., Righetti, L. Pattern Generation for Walking on Slippery Terrains In 2017 5th International Conference on Robotics and Mechatronics (ICROM), Iran, August 2017 (inproceedings)

link (url) [BibTex]

2016

no image
On the Effects of Measurement Uncertainty in Optimal Control of Contact Interactions

Ponton, B., Schaal, S., Righetti, L.

In The 12th International Workshop on the Algorithmic Foundations of Robotics WAFR, Berkeley, USA, 2016 (inproceedings)

Abstract
Stochastic Optimal Control (SOC) typically considers noise only in the process model, i.e. unknown disturbances. However, in many robotic applications involving interaction with the environment, such as locomotion and manipulation, uncertainty also comes from lack of precise knowledge of the world, which is not an actual disturbance. We analyze the effects of also considering noise in the measurement model, by devel- oping a SOC algorithm based on risk-sensitive control, that includes the dynamics of an observer in such a way that the control law explicitly de- pends on the current measurement uncertainty. In simulation results on a simple 2D manipulator, we have observed that measurement uncertainty leads to low impedance behaviors, a result in contrast with the effects of process noise that creates stiff behaviors. This suggests that taking into account measurement uncertainty could be a potentially very interesting way to approach problems involving uncertain contact interactions.

link (url) [BibTex]

2016

Ponton, B., Schaal, S., Righetti, L. On the Effects of Measurement Uncertainty in Optimal Control of Contact Interactions In The 12th International Workshop on the Algorithmic Foundations of Robotics WAFR, Berkeley, USA, 2016 (inproceedings)

link (url) [BibTex]

no image
A Convex Model of Momentum Dynamics for Multi-Contact Motion Generation

Ponton, B., Herzog, A., Schaal, S., Righetti, L.

In 2016 IEEE-RAS 16th International Conference on Humanoid Robots Humanoids, pages: 842-849, IEEE, Cancun, Mexico, 2016 (inproceedings)

Abstract
Linear models for control and motion generation of humanoid robots have received significant attention in the past years, not only due to their well known theoretical guarantees, but also because of practical computational advantages. However, to tackle more challenging tasks and scenarios such as locomotion on uneven terrain, a more expressive model is required. In this paper, we are interested in contact interaction-centered motion optimization based on the momentum dynamics model. This model is non-linear and non-convex; however, we find a relaxation of the problem that allows us to formulate it as a single convex quadratically-constrained quadratic program (QCQP) that can be very efficiently optimized and is useful for multi-contact planning. This convex model is then coupled to the optimization of end-effector contact locations using a mixed integer program, which can also be efficiently solved. This becomes relevant e.g. to recover from external pushes, where a predefined stepping plan is likely to fail and an online adaptation of the contact location is needed. The performance of our algorithm is demonstrated in several multi-contact scenarios for a humanoid robot.

link (url) DOI [BibTex]

Ponton, B., Herzog, A., Schaal, S., Righetti, L. A Convex Model of Momentum Dynamics for Multi-Contact Motion Generation In 2016 IEEE-RAS 16th International Conference on Humanoid Robots Humanoids, pages: 842-849, IEEE, Cancun, Mexico, 2016 (inproceedings)

link (url) DOI [BibTex]

no image
Inertial Sensor-Based Humanoid Joint State Estimation

Rotella, N., Mason, S., Schaal, S., Righetti, L.

In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages: 1825-1831, IEEE, Stockholm, Sweden, 2016 (inproceedings)

Abstract
This work presents methods for the determination of a humanoid robot's joint velocities and accelerations directly from link-mounted Inertial Measurement Units (IMUs) each containing a three-axis gyroscope and a three-axis accelerometer. No information about the global pose of the floating base or its links is required and precise knowledge of the link IMU poses is not necessary due to presented calibration routines. Additionally, a filter is introduced to fuse gyroscope angular velocities with joint position measurements and compensate the computed joint velocities for time-varying gyroscope biases. The resulting joint velocities are subject to less noise and delay than filtered velocities computed from numerical differentiation of joint potentiometer signals, leading to superior performance in joint feedback control as demonstrated in experiments performed on a SARCOS hydraulic humanoid.

link (url) DOI [BibTex]

Rotella, N., Mason, S., Schaal, S., Righetti, L. Inertial Sensor-Based Humanoid Joint State Estimation In 2016 IEEE International Conference on Robotics and Automation (ICRA), pages: 1825-1831, IEEE, Stockholm, Sweden, 2016 (inproceedings)

link (url) DOI [BibTex]

no image
Stepping Stabilization Using a Combination of DCM Tracking and Step Adjustment

Khadiv, M., Kleff, S., Herzog, A., Moosavian, S. A. A., Schaal, S., Righetti, L.

In 2016 4th International Conference on Robotics and Mechatronics (ICROM), pages: 130-135, IEEE, Teheran, Iran, 2016 (inproceedings)

Abstract
In this paper, a method for stabilizing biped robots stepping by a combination of Divergent Component of Motion (DCM) tracking and step adjustment is proposed. In this method, the DCM trajectory is generated, consistent with the predefined footprints. Furthermore, a swing foot trajectory modification strategy is proposed to adapt the landing point, using DCM measurement. In order to apply the generated trajectories to the full robot, a Hierarchical Inverse Dynamics (HID) is employed. The HID enables us to use different combinations of the DCM tracking and step adjustment for stabilizing different biped robots. Simulation experiments on two scenarios for two different simulated robots, one with active ankles and the other with passive ankles, are carried out. Simulation results demonstrate the effectiveness of the proposed method for robots with both active and passive ankles.

link (url) DOI [BibTex]

Khadiv, M., Kleff, S., Herzog, A., Moosavian, S. A. A., Schaal, S., Righetti, L. Stepping Stabilization Using a Combination of DCM Tracking and Step Adjustment In 2016 4th International Conference on Robotics and Mechatronics (ICROM), pages: 130-135, IEEE, Teheran, Iran, 2016 (inproceedings)

link (url) DOI [BibTex]

no image
Structured contact force optimization for kino-dynamic motion generation

Herzog, A., Schaal, S., Righetti, L.

In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 2703-2710, IEEE, Daejeon, South Korea, 2016 (inproceedings)

Abstract
Optimal control approaches in combination with trajectory optimization have recently proven to be a promising control strategy for legged robots. Computationally efficient and robust algorithms were derived using simplified models of the contact interaction between robot and environment such as the linear inverted pendulum model (LIPM). However, as humanoid robots enter more complex environments, less restrictive models become increasingly important. As we leave the regime of linear models, we need to build dedicated solvers that can compute interaction forces together with consistent kinematic plans for the whole-body. In this paper, we address the problem of planning robot motion and interaction forces for legged robots given predefined contact surfaces. The motion generation process is decomposed into two alternating parts computing force and motion plans in coherence. We focus on the properties of the momentum computation leading to sparse optimal control formulations to be exploited by a dedicated solver. In our experiments, we demonstrate that our motion generation algorithm computes consistent contact forces and joint trajectories for our humanoid robot. We also demonstrate the favorable time complexity due to our formulation and composition of the momentum equations.

link (url) DOI [BibTex]

Herzog, A., Schaal, S., Righetti, L. Structured contact force optimization for kino-dynamic motion generation In 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages: 2703-2710, IEEE, Daejeon, South Korea, 2016 (inproceedings)

link (url) DOI [BibTex]

no image
Balancing and Walking Using Full Dynamics LQR Control With Contact Constraints

Mason, S., Rotella, N., Schaal, S., Righetti, L.

In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pages: 63-68, IEEE, Cancun, Mexico, 2016 (inproceedings)

Abstract
Torque control algorithms which consider robot dynamics and contact constraints are important for creating dynamic behaviors for humanoids. As computational power increases, algorithms tend to also increase in complexity. However, it is not clear how much complexity is really required to create controllers which exhibit good performance. In this paper, we study the capabilities of a simple approach based on contact consistent LQR controllers designed around key poses to control various tasks on a humanoid robot. We present extensive experimental results on a hydraulic, torque controlled humanoid performing balancing and stepping tasks. This feedback control approach captures the necessary synergies between the DoFs of the robot to guarantee good control performance. We show that for the considered tasks, it is only necessary to re-linearize the dynamics of the robot at different contact configurations and that increasing the number of LQR controllers along desired trajectories does not improve performance. Our result suggest that very simple controllers can yield good performance competitive with current state of the art, but more complex, optimization-based whole-body controllers. A video of the experiments can be found at https://youtu.be/5T08CNKV1hw.

link (url) DOI [BibTex]

Mason, S., Rotella, N., Schaal, S., Righetti, L. Balancing and Walking Using Full Dynamics LQR Control With Contact Constraints In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pages: 63-68, IEEE, Cancun, Mexico, 2016 (inproceedings)

link (url) DOI [BibTex]

no image
Step Timing Adjustement: a Step toward Generating Robust Gaits

Khadiv, M., Herzog, A., Moosavian, S. A. A., Righetti, L.

In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pages: 35-42, IEEE, Cancun, Mexico, 2016 (inproceedings)

Abstract
Step adjustment for humanoid robots has been shown to improve robustness in gaits. However, step duration adaptation is often neglected in control strategies. In this paper, we propose an approach that combines both step location and timing adjustment for generating robust gaits. In this approach, step location and step timing are decided, based on feedback from the current state of the robot. The proposed approach is comprised of two stages. In the first stage, the nominal step location and step duration for the next step or a previewed number of steps are specified. In this stage which is done at the start of each step, the main goal is to specify the best step length and step duration for a desired walking speed. The second stage deals with finding the best landing point and landing time of the swing foot at each control cycle. In this stage, stability of the gaits is preserved by specifying a desired offset between the swing foot landing point and the Divergent Component of Motion (DCM) at the end of current step. After specifying the landing point of the swing foot at a desired time, the swing foot trajectory is regenerated at each control cycle to realize desired landing properties. Simulation on different scenarios shows the robustness of the generated gaits from our proposed approach compared to the case where no timing adjustment is employed.

link (url) DOI [BibTex]

Khadiv, M., Herzog, A., Moosavian, S. A. A., Righetti, L. Step Timing Adjustement: a Step toward Generating Robust Gaits In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pages: 35-42, IEEE, Cancun, Mexico, 2016 (inproceedings)

link (url) DOI [BibTex]

2015

no image
Trajectory generation for multi-contact momentum control

Herzog, A., Rotella, N., Schaal, S., Righetti, L.

In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pages: 874-880, IEEE, Seoul, South Korea, 2015 (inproceedings)

Abstract
Simplified models of the dynamics such as the linear inverted pendulum model (LIPM) have proven to perform well for biped walking on flat ground. However, for more complex tasks the assumptions of these models can become limiting. For example, the LIPM does not allow for the control of contact forces independently, is limited to co-planar contacts and assumes that the angular momentum is zero. In this paper, we propose to use the full momentum equations of a humanoid robot in a trajectory optimization framework to plan its center of mass, linear and angular momentum trajectories. The model also allows for planning desired contact forces for each end-effector in arbitrary contact locations. We extend our previous results on linear quadratic regulator (LQR) design for momentum control by computing the (linearized) optimal momentum feedback law in a receding horizon fashion. The resulting desired momentum and the associated feedback law are then used in a hierarchical whole body control approach. Simulation experiments show that the approach is computationally fast and is able to generate plans for locomotion on complex terrains while demonstrating good tracking performance for the full humanoid control.

link (url) DOI [BibTex]

2015

Herzog, A., Rotella, N., Schaal, S., Righetti, L. Trajectory generation for multi-contact momentum control In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pages: 874-880, IEEE, Seoul, South Korea, 2015 (inproceedings)

link (url) DOI [BibTex]

no image
Humanoid Momentum Estimation Using Sensed Contact Wrenches

Rotella, N., Herzog, A., Schaal, S., Righetti, L.

In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pages: 556-563, IEEE, Seoul, South Korea, 2015 (inproceedings)

Abstract
This work presents approaches for the estimation of quantities important for the control of the momentum of a humanoid robot. In contrast to previous approaches which use simplified models such as the Linear Inverted Pendulum Model, we present estimators based on the momentum dynamics of the robot. By using this simple yet dynamically-consistent model, we avoid the issues of using simplified models for estimation. We develop an estimator for the center of mass and full momentum which can be reformulated to estimate center of mass offsets as well as external wrenches applied to the robot. The observability of these estimators is investigated and their performance is evaluated in comparison to previous approaches.

link (url) DOI [BibTex]

Rotella, N., Herzog, A., Schaal, S., Righetti, L. Humanoid Momentum Estimation Using Sensed Contact Wrenches In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), pages: 556-563, IEEE, Seoul, South Korea, 2015 (inproceedings)

link (url) DOI [BibTex]

2014

no image
Dual Execution of Optimized Contact Interaction Trajectories

Toussaint, M., Ratliff, N., Bohg, J., Righetti, L., Englert, P., Schaal, S.

In 2014 IEEE/RSJ Conference on Intelligent Robots and Systems, pages: 47-54, IEEE, Chicago, USA, 2014 (inproceedings)

Abstract
Efficient manipulation requires contact to reduce uncertainty. The manipulation literature refers to this as funneling: a methodology for increasing reliability and robustness by leveraging haptic feedback and control of environmental interaction. However, there is a fundamental gap between traditional approaches to trajectory optimization and this concept of robustness by funneling: traditional trajectory optimizers do not discover force feedback strategies. From a POMDP perspective, these behaviors could be regarded as explicit observation actions planned to sufficiently reduce uncertainty thereby enabling a task. While we are sympathetic to the full POMDP view, solving full continuous-space POMDPs in high-dimensions is hard. In this paper, we propose an alternative approach in which trajectory optimization objectives are augmented with new terms that reward uncertainty reduction through contacts, explicitly promoting funneling. This augmentation shifts the responsibility of robustness toward the actual execution of the optimized trajectories. Directly tracing trajectories through configuration space would lose all robustness-dual execution achieves robustness by devising force controllers to reproduce the temporal interaction profile encoded in the dual solution of the optimization problem. This work introduces dual execution in depth and analyze its performance through robustness experiments in both simulation and on a real-world robotic platform.

link (url) DOI [BibTex]

2014

Toussaint, M., Ratliff, N., Bohg, J., Righetti, L., Englert, P., Schaal, S. Dual Execution of Optimized Contact Interaction Trajectories In 2014 IEEE/RSJ Conference on Intelligent Robots and Systems, pages: 47-54, IEEE, Chicago, USA, 2014 (inproceedings)

link (url) DOI [BibTex]

no image
Balancing experiments on a torque-controlled humanoid with hierarchical inverse dynamics

Herzog, A., Righetti, L., Grimminger, F., Pastor, P., Schaal, S.

In 2014 IEEE/RSJ Conference on Intelligent Robots and Systems, pages: 981-988, IEEE, Chicago, USA, 2014 (inproceedings)

Abstract
Recently several hierarchical inverse dynamics controllers based on cascades of quadratic programs have been proposed for application on torque controlled robots. They have important theoretical benefits but have never been implemented on a torque controlled robot where model inaccuracies and real-time computation requirements can be problematic. In this contribution we present an experimental evaluation of these algorithms in the context of balance control for a humanoid robot. The presented experiments demonstrate the applicability of the approach under real robot conditions (i.e. model uncertainty, estimation errors, etc). We propose a simplification of the optimization problem that allows us to decrease computation time enough to implement it in a fast torque control loop. We implement a momentum-based balance controller which shows robust performance in face of unknown disturbances, even when the robot is standing on only one foot. In a second experiment, a tracking task is evaluated to demonstrate the performance of the controller with more complicated hierarchies. Our results show that hierarchical inverse dynamics controllers can be used for feedback control of humanoid robots and that momentum-based balance control can be efficiently implemented on a real robot.

link (url) DOI [BibTex]

Herzog, A., Righetti, L., Grimminger, F., Pastor, P., Schaal, S. Balancing experiments on a torque-controlled humanoid with hierarchical inverse dynamics In 2014 IEEE/RSJ Conference on Intelligent Robots and Systems, pages: 981-988, IEEE, Chicago, USA, 2014 (inproceedings)

link (url) DOI [BibTex]

no image
Full Dynamics LQR Control of a Humanoid Robot: An Experimental Study on Balancing and Squatting

Mason, S., Righetti, L., Schaal, S.

In 2014 IEEE-RAS International Conference on Humanoid Robots, pages: 374-379, IEEE, Madrid, Spain, 2014 (inproceedings)

Abstract
Humanoid robots operating in human environments require whole-body controllers that can offer precise tracking and well-defined disturbance rejection behavior. In this contribution, we propose an experimental evaluation of a linear quadratic regulator (LQR) using a linearization of the full robot dynamics together with the contact constraints. The advantage of the controller is that it explicitly takes into account the coupling between the different joints to create optimal feedback controllers for whole-body control. We also propose a method to explicitly regulate other tasks of interest, such as the regulation of the center of mass of the robot or its angular momentum. In order to evaluate the performance of linear optimal control designs in a real-world scenario (model uncertainty, sensor noise, imperfect state estimation, etc), we test the controllers in a variety of tracking and balancing experiments on a torque controlled humanoid (e.g. balancing, split plane balancing, squatting, pushes while squatting, and balancing on a wheeled platform). The proposed control framework shows a reliable push recovery behavior competitive with more sophisticated balance controllers, rejecting impulses up to 11.7 Ns with peak forces of 650 N, with the added advantage of great computational simplicity. Furthermore, the controller is able to track squatting trajectories up to 1 Hz without relinearization, suggesting that the linearized dynamics is sufficient for significant ranges of motion.

link (url) DOI [BibTex]

Mason, S., Righetti, L., Schaal, S. Full Dynamics LQR Control of a Humanoid Robot: An Experimental Study on Balancing and Squatting In 2014 IEEE-RAS International Conference on Humanoid Robots, pages: 374-379, IEEE, Madrid, Spain, 2014 (inproceedings)

link (url) DOI [BibTex]

no image
State Estimation for a Humanoid Robot

Rotella, N., Bloesch, M., Righetti, L., Schaal, S.

In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages: 952-958, IEEE, Chicago, USA, 2014 (inproceedings)

Abstract
This paper introduces a framework for state estimation on a humanoid robot platform using only common proprioceptive sensors and knowledge of leg kinematics. The presented approach extends that detailed in prior work on a point-foot quadruped platform by adding the rotational constraints imposed by the humanoid's flat feet. As in previous work, the proposed Extended Kalman Filter accommodates contact switching and makes no assumptions about gait or terrain, making it applicable on any humanoid platform for use in any task. A nonlinear observability analysis is performed on both the point-foot and flat-foot filters and it is concluded that the addition of rotational constraints significantly simplifies singular cases and improves the observability characteristics of the system. Results on a simulated walking dataset demonstrate the performance gain of the flat-foot filter as well as confirm the results of the presented observability analysis.

link (url) DOI [BibTex]

Rotella, N., Bloesch, M., Righetti, L., Schaal, S. State Estimation for a Humanoid Robot In 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages: 952-958, IEEE, Chicago, USA, 2014 (inproceedings)

link (url) DOI [BibTex]

2013

no image
AGILITY – Dynamic Full Body Locomotion and Manipulation with Autonomous Legged Robots

Hutter, M., Bloesch, M., Buchli, J., Semini, C., Bazeille, S., Righetti, L., Bohg, J.

In 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pages: 1-4, IEEE, Linköping, Sweden, 2013 (inproceedings)

link (url) DOI [BibTex]

2013

Hutter, M., Bloesch, M., Buchli, J., Semini, C., Bazeille, S., Righetti, L., Bohg, J. AGILITY – Dynamic Full Body Locomotion and Manipulation with Autonomous Legged Robots In 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR), pages: 1-4, IEEE, Linköping, Sweden, 2013 (inproceedings)

link (url) DOI [BibTex]

no image
Learning Objective Functions for Manipulation

Kalakrishnan, M., Pastor, P., Righetti, L., Schaal, S.

In 2013 IEEE International Conference on Robotics and Automation, IEEE, Karlsruhe, Germany, 2013 (inproceedings)

Abstract
We present an approach to learning objective functions for robotic manipulation based on inverse reinforcement learning. Our path integral inverse reinforcement learning algorithm can deal with high-dimensional continuous state-action spaces, and only requires local optimality of demonstrated trajectories. We use L 1 regularization in order to achieve feature selection, and propose an efficient algorithm to minimize the resulting convex objective function. We demonstrate our approach by applying it to two core problems in robotic manipulation. First, we learn a cost function for redundancy resolution in inverse kinematics. Second, we use our method to learn a cost function over trajectories, which is then used in optimization-based motion planning for grasping and manipulation tasks. Experimental results show that our method outperforms previous algorithms in high-dimensional settings.

link (url) DOI [BibTex]

Kalakrishnan, M., Pastor, P., Righetti, L., Schaal, S. Learning Objective Functions for Manipulation In 2013 IEEE International Conference on Robotics and Automation, IEEE, Karlsruhe, Germany, 2013 (inproceedings)

link (url) DOI [BibTex]

no image
Learning Task Error Models for Manipulation

Pastor, P., Kalakrishnan, M., Binney, J., Kelly, J., Righetti, L., Sukhatme, G. S., Schaal, S.

In 2013 IEEE Conference on Robotics and Automation, IEEE, Karlsruhe, Germany, 2013 (inproceedings)

Abstract
Precise kinematic forward models are important for robots to successfully perform dexterous grasping and manipulation tasks, especially when visual servoing is rendered infeasible due to occlusions. A lot of research has been conducted to estimate geometric and non-geometric parameters of kinematic chains to minimize reconstruction errors. However, kinematic chains can include non-linearities, e.g. due to cable stretch and motor-side encoders, that result in significantly different errors for different parts of the state space. Previous work either does not consider such non-linearities or proposes to estimate non-geometric parameters of carefully engineered models that are robot specific. We propose a data-driven approach that learns task error models that account for such unmodeled non-linearities. We argue that in the context of grasping and manipulation, it is sufficient to achieve high accuracy in the task relevant state space. We identify this relevant state space using previously executed joint configurations and learn error corrections for those. Therefore, our system is developed to generate subsequent executions that are similar to previous ones. The experiments show that our method successfully captures the non-linearities in the head kinematic chain (due to a counterbalancing spring) and the arm kinematic chains (due to cable stretch) of the considered experimental platform, see Fig. 1. The feasibility of the presented error learning approach has also been evaluated in independent DARPA ARM-S testing contributing to successfully complete 67 out of 72 grasping and manipulation tasks.

link (url) DOI [BibTex]

Pastor, P., Kalakrishnan, M., Binney, J., Kelly, J., Righetti, L., Sukhatme, G. S., Schaal, S. Learning Task Error Models for Manipulation In 2013 IEEE Conference on Robotics and Automation, IEEE, Karlsruhe, Germany, 2013 (inproceedings)

link (url) DOI [BibTex]

2012

no image
Encoding of Periodic and their Transient Motions by a Single Dynamic Movement Primitive

Ernesti, J., Righetti, L., Do, M., Asfour, T., Schaal, S.

In 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pages: 57-64, IEEE, Osaka, Japan, November 2012 (inproceedings)

link (url) DOI [BibTex]

2012

Ernesti, J., Righetti, L., Do, M., Asfour, T., Schaal, S. Encoding of Periodic and their Transient Motions by a Single Dynamic Movement Primitive In 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pages: 57-64, IEEE, Osaka, Japan, November 2012 (inproceedings)

link (url) DOI [BibTex]

no image
Learning Force Control Policies for Compliant Robotic Manipulation

Kalakrishnan, M., Righetti, L., Pastor, P., Schaal, S.

In ICML’12 Proceedings of the 29th International Coference on International Conference on Machine Learning, pages: 49-50, Edinburgh, Scotland, 2012 (inproceedings)

[BibTex]

Kalakrishnan, M., Righetti, L., Pastor, P., Schaal, S. Learning Force Control Policies for Compliant Robotic Manipulation In ICML’12 Proceedings of the 29th International Coference on International Conference on Machine Learning, pages: 49-50, Edinburgh, Scotland, 2012 (inproceedings)

[BibTex]

no image
Quadratic programming for inverse dynamics with optimal distribution of contact forces

Righetti, L., Schaal, S.

In 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pages: 538-543, IEEE, Osaka, Japan, November 2012 (inproceedings)

Abstract
In this contribution we propose an inverse dynamics controller for a humanoid robot that exploits torque redundancy to minimize any combination of linear and quadratic costs in the contact forces and the commands. In addition the controller satisfies linear equality and inequality constraints in the contact forces and the commands such as torque limits, unilateral contacts or friction cones limits. The originality of our approach resides in the formulation of the problem as a quadratic program where we only need to solve for the control commands and where the contact forces are optimized implicitly. Furthermore, we do not need a structured representation of the dynamics of the robot (i.e. an explicit computation of the inertia matrix). It is in contrast with existing methods based on quadratic programs. The controller is then robust to uncertainty in the estimation of the dynamics model and the optimization is fast enough to be implemented in high bandwidth torque control loops that are increasingly available on humanoid platforms. We demonstrate properties of our controller with simulations of a human size humanoid robot.

link (url) DOI [BibTex]

Righetti, L., Schaal, S. Quadratic programming for inverse dynamics with optimal distribution of contact forces In 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pages: 538-543, IEEE, Osaka, Japan, November 2012 (inproceedings)

link (url) DOI [BibTex]

no image
Towards Associative Skill Memories

Pastor, P., Kalakrishnan, M., Righetti, L., Schaal, S.

In 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pages: 309-315, IEEE, Osaka, Japan, November 2012 (inproceedings)

Abstract
Movement primitives as basis of movement planning and control have become a popular topic in recent years. The key idea of movement primitives is that a rather small set of stereotypical movements should suffice to create a large set of complex manipulation skills. An interesting side effect of stereotypical movement is that it also creates stereotypical sensory events, e.g., in terms of kinesthetic variables, haptic variables, or, if processed appropriately, visual variables. Thus, a movement primitive executed towards a particular object in the environment will associate a large number of sensory variables that are typical for this manipulation skill. These association can be used to increase robustness towards perturbations, and they also allow failure detection and switching towards other behaviors. We call such movement primitives augmented with sensory associations Associative Skill Memories (ASM). This paper addresses how ASMs can be acquired by imitation learning and how they can create robust manipulation skill by determining subsequent ASMs online to achieve a particular manipulation goal. Evaluation for grasping and manipulation with a Barrett WAM/Hand illustrate our approach.

link (url) DOI [BibTex]

Pastor, P., Kalakrishnan, M., Righetti, L., Schaal, S. Towards Associative Skill Memories In 2012 12th IEEE-RAS International Conference on Humanoid Robots (Humanoids 2012), pages: 309-315, IEEE, Osaka, Japan, November 2012 (inproceedings)

link (url) DOI [BibTex]

no image
Template-based learning of grasp selection

Herzog, A., Pastor, P., Kalakrishnan, M., Righetti, L., Asfour, T., Schaal, S.

In 2012 IEEE International Conference on Robotics and Automation, pages: 2379-2384, IEEE, Saint Paul, USA, 2012 (inproceedings)

Abstract
The ability to grasp unknown objects is an important skill for personal robots, which has been addressed by many present and past research projects, but still remains an open problem. A crucial aspect of grasping is choosing an appropriate grasp configuration, i.e. the 6d pose of the hand relative to the object and its finger configuration. Finding feasible grasp configurations for novel objects, however, is challenging because of the huge variety in shape and size of these objects. Moreover, possible configurations also depend on the specific kinematics of the robotic arm and hand in use. In this paper, we introduce a new grasp selection algorithm able to find object grasp poses based on previously demonstrated grasps. Assuming that objects with similar shapes can be grasped in a similar way, we associate to each demonstrated grasp a grasp template. The template is a local shape descriptor for a possible grasp pose and is constructed using 3d information from depth sensors. For each new object to grasp, the algorithm then finds the best grasp candidate in the library of templates. The grasp selection is also able to improve over time using the information of previous grasp attempts to adapt the ranking of the templates. We tested the algorithm on two different platforms, the Willow Garage PR2 and the Barrett WAM arm which have very different hands. Our results show that the algorithm is able to find good grasp configurations for a large set of objects from a relatively small set of demonstrations, and does indeed improve its performance over time.

link (url) DOI [BibTex]

Herzog, A., Pastor, P., Kalakrishnan, M., Righetti, L., Asfour, T., Schaal, S. Template-based learning of grasp selection In 2012 IEEE International Conference on Robotics and Automation, pages: 2379-2384, IEEE, Saint Paul, USA, 2012 (inproceedings)

link (url) DOI [BibTex]