The vision of intelligent and fully autonomous robots, which are part of our daily lives and automatically learn from mistakes and adapt to new situations, has been around for many decades. However, this vision has been elusive so far. Although reinforcement learning is a principled framework for learning from trial and error and has led to success stories in the context of games, we need to address a practical challenge when it comes to learning with mechanical systems: data efficiency, i.e., the ability to learn from scarce data in complex domains. In this talk, I will outline three approaches, based on probabilistic modeling and inference, that explicitly address the data-efficiency challenge in reinforcement learning and robotics. First, I will give a brief overview of a model-based RL algorithm that can learn from small datasets. Second, I will describe an idea based on model predictive control that allows us to learn even faster while taking care of state or control constraints, which is important for safe exploration. Finally, I will introduce latent-variable approach to meta learning (in the context of model-based RL) for transferring knowledge from known tasks to tasks that have never been encountered.