Federated Adaptation for Foundation Model-based Recommendations

Code for ijcai-24 paper. Federated Adaptation for Foundation Model-based Recommendations.

Abatract

With the recent success of large language models, particularly foundation models with generalization abilities, applying foundation models for recommendations becomes a new paradigm to improve existing recommendation systems. It becomes a new open challenge to enable the foundation model to capture user preference changes in a timely manner with reasonable communication and computation costs while preserving privacy. This paper proposes a novel federated adaptation mechanism to enhance the foundation model-based recommendation system in a privacy-preserving manner. Specifically, each client will learn a lightweight personalized adapter using its private data. The adapter then collaborates with pre-trained foundation models to provide recommendation service efficiently with fine-grained manners. Importantly, users’ private behavioral data remains secure as it is not shared with the server. This data localization-based privacy preservation is embodied via the federated learning framework. The model can ensure that shared knowledge is incorporated into all adapters while simultaneously preserving each user’s personal preferences. Experimental results on four benchmark datasets demonstrate our method’s superior performance.

Figure: The model architecture of proposed FedPA.

Prepare Dataset

Please download the dataset from the official link and then preprocess the dataset following 'data_preprocess.py'.

Running the Code

Please run the code following files in 'Open_Train' first and then files in 'Fed_Train'.

Citation

If you find this project helpful, please consider to cite the following paper:

@inproceedings{zhang2024federated,
  title={Federated Adaptation for Foundation Model-based Recommendations},
  author={Zhang, Chunxu and Long, Guodong and Guo, Hongkuan and Fang, Xiao and Song, Yang and Liu, Zhaojie and Zhou, Guorui and Zhang, Zijian and Liu, Yang and Yang, Bo,
  booktitle={Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence},
  pages={5453--5461},
  year={2024}
}