This thesis comprises five works, organized into three parts: the first focuses on average-reward Reinforcement Learning (RL), the second on distributed learning under adversaries in heterogeneous and asynchronous setups, ...