Personalised recommendation systems

Definition: - Systems that suggest items (products, content, services) tailored to an individual user’s preferences, behavior, and context. Main approaches: - Collaborative Filtering: recommends based on behavior of similar users (user-based or item-based). Strength: captures complex tastes without item metadata. Weakness: cold-start, sparsity. (See: Resnick et al., 1994; Sarwar et al., 2001.) - Content-Based Filtering: recommends items similar to those the user liked, using item features. Strength: handles new items; interpretable. Weakness: limited serendipity, needs good features. (See: Pazzani & Billsus, 2007.) - Hybrid Methods: combine collaborative and content approaches to offset weaknesses (e.g., matrix factorization + side information). (See: Burke, 2002.) - Context-Aware and Session-Based Models: incorporate time, location, device, or short-session signals; useful for dynamic preferences. - Deep Learning & Representation Learning: use neural networks (embeddings, sequence models, transformers) to model complex patterns and sequential behavior. Key components: - Data sources: explicit feedback (ratings), implicit feedback (clicks, views, purchases), contextual signals, user/item metadata. - Algorithms: nearest neighbors, matrix factorization, factorization machines, neural recommenders, bandits for exploration. - Evaluation metrics: precision, recall, MAP, NDCG, AUC, click-through rate, and business KPIs; offline vs. online (A/B) testing. Challenges and trade-offs: - Cold-start for new users/items. - Scalability to many users and items. - Diversity vs. accuracy (filter bubbles). - Privacy and fairness concerns; need for transparency and user control. - Long-term engagement vs. short-term clicks; optimizing for long-term utility may require causal or reinforcement learning methods. Ethical and philosophical considerations: - Autonomy and manipulation: recommendations can shape preferences and behavior. - Bias amplification and fairness: training data can encode social biases. - Privacy: collecting and profiling users raises consent and surveillance issues. (See: O’Neil, 2016; Barocas & Selbst, 2016; Ekstrand et al., 2018.) Concise further reading: - Ricci, Rokach, Shapira (eds.), Recommender Systems Handbook (2015). - Burke, R., Hybrid Recommender Systems, 2002. - Ekstrand, T., et al., 2018, “All the cool kids” (fairness in recommender systems).

Selection in recommendation systems means choosing which items to present to a user from a large candidate pool. Here are concise, concrete examples showing how different approaches perform that selection, and what trade-offs they reveal: - Collaborative filtering (user-based): “Show movies that users with similar watch histories liked.” Example selection: because similar users watched Inception and Interstellar, recommend Tenet. Trade-off: good at finding niche tastes but fails for a new movie with no watchers (cold-start). - Collaborative filtering (item-based): “Show items similar to what the user consumed.” Example selection: because you streamed several indie-folk tracks, recommend another artist whose listeners also played those tracks. Trade-off: captures co-consumption patterns but can overfit popular co-occurrences. - Content-based filtering: “Recommend items with similar features to those you liked.” Example selection: because you read articles tagged ‘climate policy’ and ‘economics’, surface a new article with those tags and similar keywords. Trade-off: handles new items and explains why, but may produce narrow, predictable lists. - Hybrid selection: “Combine signals to pick candidates.” Example selection: rank a mix of collaborative candidates (popular among similar users) and content-based ones (matching item attributes), then blend with a diversity constraint. Trade-off: reduces cold-start and improves diversity but adds complexity. - Context-aware/session-based selection: “Select items based on current context.” Example selection: during a morning commute, recommend short news summaries or podcasts popular on mobile at that hour. Trade-off: increases relevance for current needs but requires timely context signals. - Deep-learning/sequence models: “Predict next items from a learned user/item embedding and session sequence.” Example selection: use a transformer trained on browsing sessions to propose the most likely next clicks (e.g., product pages leading to purchase). Trade-off: captures complex sequential patterns but needs lots of data and is less interpretable. - Bandit-driven selection (exploration/exploitation): “Occasionally show exploratory items to learn preferences.” Example selection: 80% of recommendations are high-confidence top picks; 20% are exploratory new releases to test interest. Trade-off: improves long-term learning at cost of some short-term relevance. - Fairness- or privacy-aware selection: “Filter or re-rank candidates to satisfy ethical constraints.” Example selection: down-weight candidates that would amplify demographic stereotyping or avoid using sensitive attributes when ranking. Trade-off: may reduce short-term engagement but improves equity and trust. Each selection step typically involves candidate generation (broad set) and ranking (final ordered list), with business KPIs and policy constraints guiding the final mix. For technical grounding, see Resnick et al., 1994; Burke, 2002; Ricci et al., 2015; and Ekstrand et al., 2018.Title: Examples of Selection in Personalised Recommendation Systems Selection in recommender systems means choosing which items to present to a specific user from a large set. Below are concise examples showing how different approaches perform selection and what trade-offs arise. 1) Collaborative filtering (user-based) - Example: “Users like you also bought” list on an e-commerce site. - How selection works: find users with similar purchase histories and recommend items they bought that the target user hasn’t. - Trade-offs: captures nuanced tastes but can miss new items (cold-start) and amplify popular trends. 2) Content-based filtering - Example: news app recommending articles similar to those you’ve read (same topics, authors, keywords). - How selection works: score candidate items by similarity to the user’s profile of item features and rank top matches. - Trade-offs: handles new items well and is interpretable, but may limit novelty (echo chamber). 3) Matrix factorization / latent-factor models - Example: movie platform ranks films by predicted rating from user and item embeddings (Netflix-style). - How selection works: compute dot product between user and item vectors to score and select highest. - Trade-offs: captures latent tastes across sparse data but needs retraining for new users/items or uses side information. 4) Session-based / sequence models - Example: streaming service recommending next song based on the last few tracks played. - How selection works: use a sequence model (RNN/transformer) to predict likely next items given recent session context and present top-k predictions. - Trade-offs: adapts to short-term intent but may ignore long-term preferences unless combined. 5) Context-aware and bandit selection - Example: mobile app shows location-relevant coupons and tests variants to maximize long-term engagement. - How selection works: contextual bandits score items using context (time, location, device) and explore less-certain options to learn user responses. - Trade-offs: balances exploration/exploitation and can improve personalization over time but requires careful reward design and safety constraints. 6) Hybrid selection with fairness/privacy constraints - Example: job platform blending collaborative signals with content filters and enforcing demographic balance in candidate exposure. - How selection works: combine multiple scoring components (e.g., CF score + content score + fairness adjustment) to produce a final ranked list that meets utility and fairness constraints. - Trade-offs: improves robustness and addresses bias/privacy, but increases system complexity and may reduce raw accuracy. Practical selection considerations (short): - Candidate generation then ranking: large catalogs require a two-stage pipeline (cheap recall then expensive ranking). - Evaluation: choose metrics (NDCG, CTR, long-term retention) aligned with goals; A/B test changes. - Ethics: monitor for manipulation, bias amplification, and privacy intrusions; provide transparency and user control. Further reading: Ricci et al., Recommender Systems Handbook (2015); Burke, 2002; Ekstrand et al., 2018.