Personalized Recommendation Using Reinforcement Learning

Abstract

The massive volume of information available on the web leads to the problem of information overload, which makes it difficult for a decision maker to make right decisions. Recommender systems (RSs) are software tools and algorithms that have been developed with the idea of helping users find their items of interest through predicting their preferences or ratings on items. It has been shown that the problem of recommending items to the user could be considered as a sequential decision problem and be formulated as a Markov decision process, so reinforcement learning (RL) algorithms can be used to solve this problem. The primary aim of this dissertation is to investigate this topic and to propose new recommendation approaches using RL. The first part of this thesis, namely chapters 2 and 3, presents a healthcare use case of intelligent agents and RSs. In particular, chapter 2 presents a high-level design, called ALAN, to play the role of a patient decision aid for prostate cancer patients. ALAN is a multilayered, multi-agent system in which each agent is responsible to provide a specific service in order to facilitate shared decision making for these patients. Moreover, an article RS with learning ability is proposed in chapter 3 to represent the Learning agent in ALAN, which combines multi-armed bandits with knowledge-based RSs for the provision of information for cancer patients. Motivated by the first part, the second part of this thesis (chapters 4 and 5) deeply explores the topic of recommendation using RL algorithms. More precisely, chapter 4 provides a thorough literature review on reinforcement learning based recommender systems (RLRSs). The main goal of this chapter is to provide a deep analysis of almost all important RLRSs proposed and to depict a vista toward the field since the beginning. This chapter illustrates the importance of deep RL (DRL) in reviving the use of RL for RSs. Chapter 5 is an outcome of this chapter, which tries to address an important problem of DRL when applied to real applications like RSs, i.e., sample inefficiency. In this chapter, RL is combined with imitation learning in order to accelerate RL agent’s learning and to make it sample efficient. Finally, chapter 6 proposes a new recommendation approach from a totally new perspective. This chapter borrows ideas from Computer Networks field, clustering in wireless sensor networks in particular, and presents a probabilistic recommendation approach that can balance the similarity-diversity trade-off. The proposed approach is simple, scalable, and completely explainable.