This thesis presents six novel works involving several research domains, such as reinforcement learning (RL)– both with or without function approximators including deep neural networks, multi-agent RL, stochastic optimization, ...