0cfdd655100055f234fd23ebecd915504b8e00e3
Cross-Domain Preference Distillation: Leveraging Large Language Models for Explainable and Interactive Recommendations
Current cross-domain recommender systems struggle to effectively transfer user preferences across diverse domains, especially when dealing with cold-start users or items with limited interactions. This leads to suboptimal recommendations and poor user experiences when users explore new domains.
Existing methods often rely on shared user embeddings or domain-specific feature extraction, which can lead to suboptimal transfer and limited explainability. Large language models have demonstrated impressive zero-shot generalization capabilities across diverse tasks. By leveraging LLMs to distill domain-agnostic preference knowledge, we can potentially enable more effective and explainable cross-domain recommendations.
We propose Cross-Domain Preference Distillation (CDPD), a novel framework that uses LLMs to extract and transfer domain-agnostic user preferences. The method consists of three main steps: 1) Fine-tuning an LLM on user reviews and item descriptions from multiple domains to learn a unified preference representation space. 2) For each user, prompting the LLM to generate abstract preference statements that capture their tastes independent of specific domains. These statements form a preference knowledge graph. 3) For recommendations, using a graph neural network to propagate information through this knowledge graph, combining it with domain-specific item embeddings. The LLM is also used to generate natural language explanations for recommendations by traversing paths in the preference graph.
Step 1: Data Preparation
Use the Amazon review dataset across multiple product categories (e.g., Books, Electronics, Movies & TV, Clothing). Preprocess the data to extract user reviews, item descriptions, and user-item interactions.
Step 2: LLM Fine-tuning
Fine-tune GPT-3.5 (text-davinci-003) on a subset of user reviews and item descriptions using prompt templates like 'Review: {review_text}\nItem: {item_description}\nPreference: {preference_statement}'. Use few-shot learning with manually crafted examples to guide the model.
Step 3: Preference Statement Generation
For each user, prompt the fine-tuned LLM with their reviews and generate abstract preference statements. Example prompt: 'Based on the following reviews, generate 5 abstract preference statements that capture the user's tastes across domains:\n{user_reviews}\nPreference statements:'
Step 4: Preference Knowledge Graph Construction
Create a graph where nodes are users, items, and preference statements. Connect users to their generated preference statements and items to relevant preference statements using the LLM's zero-shot classification capabilities.
Step 5: Graph Neural Network Implementation
Implement a Graph Attention Network (GAT) to propagate information through the preference knowledge graph. Use PyTorch Geometric library for efficient graph operations.
Step 6: Recommendation Generation
For a given user and target domain, use the GAT to generate embeddings for the user and candidate items. Rank items based on the cosine similarity between these embeddings.
Step 7: Explanation Generation
For each recommended item, prompt the LLM to generate an explanation by traversing the preference graph. Example prompt: 'Explain why {item_name} is recommended to the user based on the following preference path:\n{preference_path}'
Step 8: Evaluation
Evaluate the CDPD method on standard metrics like NDCG@k and Precision@k. Compare against baselines including domain-specific matrix factorization, cross-domain collaborative filtering, and LLM-based zero-shot recommenders.
Step 9: Human Evaluation
Conduct a small-scale human evaluation (e.g., 50 participants) to assess the quality of preference transfer and explanation relevance. Use a 5-point Likert scale for ratings.
Step 10: Ablation Studies
Perform ablation studies by removing components like the preference knowledge graph or explanation generation to analyze their impact on performance.
Baseline Prompt Input (Domain-specific Matrix Factorization)
User ID: 12345, Target Domain: Electronics
Baseline Prompt Expected Output (Domain-specific Matrix Factorization)
Recommended Items: [1. Smartphone X, 2. Laptop Y, 3. Headphones Z]
Proposed Prompt Input (CDPD; Step 1: Preference Statement Generation)
Generate abstract preference statements for User 12345 based on their reviews across Books and Movies domains:\n[Review 1: 'I love sci-fi novels with complex world-building.', Review 2: 'This action movie had great special effects but a weak plot.']
Proposed Prompt Expected Output (CDPD; Step 1: Preference Statement Generation)
Proposed Prompt Input (CDPD; Step 2: Recommendation Generation)
Recommend Electronics items for User 12345 based on their preference statements and the preference knowledge graph
Proposed Prompt Expected Output (CDPD; Step 2: Recommendation Generation)
Recommended Items: [1. VR Headset A, 2. 4K HDR TV B, 3. High-fidelity Audio System C]
Proposed Prompt Input (CDPD; Step 3: Explanation Generation)
Explain why VR Headset A is recommended to User 12345
Proposed Prompt Expected Output (CDPD; Step 3: Explanation Generation)
VR Headset A is recommended because it aligns with your preference for immersive and detailed fictional universes. This device offers a highly immersive experience, allowing you to explore complex virtual worlds, which resonates with your enjoyment of sci-fi novels with intricate world-building. Additionally, the VR headset's advanced graphics capabilities cater to your appreciation for visual spectacle in storytelling, similar to your interest in movies with great special effects.
Explanation
The CDPD method generates more personalized and explainable recommendations by leveraging abstract preference statements that transfer across domains. Unlike the baseline, which relies solely on past interactions in the Electronics domain, CDPD can recommend items (like the VR headset) that align with the user's preferences inferred from other domains (Books and Movies).
If the proposed CDPD method doesn't significantly outperform baselines, we can pivot the project to focus on analyzing the generated preference statements and explanations. We could investigate how well LLMs capture domain-agnostic preferences and identify patterns in successful vs. unsuccessful transfers. This analysis could provide insights into the challenges of cross-domain preference modeling and inform future research directions. Additionally, we could explore alternative graph structures or neural architectures for combining the preference knowledge with domain-specific information. Another approach would be to incorporate a self-supervised learning component, where the model tries to predict masked preference statements or item attributes, potentially improving the quality of the learned representations.