Federated learning-driven collaborative recommendation system for multi-modal art analysis and enhanced recommendations

Ida Puteri Mahsan

QR Code Link :
Type :	Article
Subject :	L Education (General)
ISBN :	2376-5992
Main Author :	Ida Puteri Mahsan
Title :	Federated learning-driven collaborative recommendation system for multi-modal art analysis and enhanced recommendations
Hits :	68

Place of Production :	Tanjung Malim
Publisher :	Fakulti Seni, Kelestarian & Industri Kreatif
Year of Publication :	2024
Notes :	PeerJ Computer Science
Corporate Name :	Universiti Pendidikan Sultan Idris
HTTP Link :	Click to view web link
PDF Full Text :	You have no permission to view this item.

Abstract : Universiti Pendidikan Sultan Idris

With the rapid development of artificial intelligence technology, recommendation systems have been widely applied in various fields. However, in the art field, art similarity search and recommendation systems face unique challenges, namely data privacy and copyright protection issues. To address these problems, this article proposes a cross-institutional artwork similarity search and recommendation system (AI-based Collaborative Recommendation System (AICRS) framework) that combines multimodal data fusion and federated learning. This system uses pre-trained convolutional neural networks (CNN) and Bidirectional Encoder Representation from Transformers (BERT) models to extract features from image and text data. It then uses a federated learning framework to train models locally at each participating institution and aggregate parameters to optimize the global model. Experimental results show that the AICRS framework achieves a final accuracy of 92.02% on the SemArt dataset, compared to 81.52% and 83.44% for traditional CNN and Long Short-Term Memory (LSTM) models, respectively. The final loss value of the AICRS framework is 0.1284, which is better than the 0.248 and 0.188 of CNN and LSTM models. The research results of this article not only provide an effective technical solution but also offer strong support for the recommendation and protection of artworks in practice. © 2024 Gong et al.

References

Appel G, Neelbauer J, Schweidel DA. 2023. Generative AI Has an Intellectual Property Problem. Available at https://hbr.org/2023/04/generative-ai-has-an-intellectual-property-problem%7D.

Ajmal S, Awais M, Khurshid KS, Shoaib M, Abdelrahman A. 2023. Data mining-based recommendation system using social networks—an analytical study. PeerJ Computer Science 9(1):e1202 DOI 10.7717/peerj-cs.1202.

Arthur JK, Zhou C, Mantey EA, Osei-Kwakye J, Chen Y. 2022. A discriminative-based geometric deep learning model for cross domain recommender systems. Applied Sciences 12(10):5202 DOI 10.3390/app12105202.

BakerHostetler. 2024. Getty Images v. Stability AI. Available at https://www.bakerlaw.com/gettyimages-v-stability-ai/. Chen X, Yao L, McAuley J, Zhou G, Wang X. 2023. Deep reinforcement learning in recommender systems: a survey and new perspectives. Knowledge-Based Systems 264:110335 DOI 10.1016/j.knosys.2023.110335.

Daneshvar H, Ravanmehr R. 2022. A social hybrid recommendation system using lstm and cnn. Concurrency and Computation-Practice & Experience 34(18):e7015 DOI 10.1002/cpe.7015.

Dong Q, Liu B, Zhang X, Qin J, Wang B. 2023. Sequential POI recommend based on personalized federated learning. Neural Processing Letters 55(6):7351–7368 DOI 10.1007/s11063-023-11264-7.

Dong M, Yuan F, Yao L, Wang X, Xu X, Zhu L. 2022. A survey for trust-aware recommender systems: a deep learning perspective. Knowledge-Based Systems 249(7):108954 DOI 10.1016/j.knosys.2022.108954.

Elayan H, Aloqaily M, Guizani M. 2021. Deep federated learning for iot-based decentralized healthcare systems. In: IWCMC 2021: 2021 17th International Wireless Communications and Mobile Computing Conference. Piscataway: IEEE, 105–109.

Fachrela J, Pravitasaria AA, Yulitab IN, Ardhisasmitac MN, Indrayatnaa F. 2023. A comparison between cnn and combined cnn-lstm for chest x-ray based covid-19 detection. Decision Science Letters 12(2):199–210 DOI 10.5267/j.dsl.2023.2.004.

Jeong SY, Kim YK. 2022. Deep learning-based context-aware recommender system considering contextual features. Applied Sciences 12(1):45 DOI 10.3390/app12010045.

Jeon I, Hong M, Yun J, Kim G. 2023. Federated learning via meta-variational dropout. In: Oh A, Naumann T, Globerson A, Saenko K, Hardt M, Levine S, eds. Advances in Neural Information Processing Systems. Vol. 36. Curran Associates, Inc. 11168–11193. Available at https://proceedings.neurips.cc/paper_files/paper/2023/file/24a8d40f6656e542f3fd43bac678e71b-PaperConference.pdf.

Karayel E. 2023. An embarrassingly parallel optimal-space cardinality estimation algorithm. Arxiv preprint DOI 10.48550/arXiv.2307.00985.

Kim M, Kang D, Lee N. 2019. Feature extraction from oriental painting for wellness contents recommendation services. IEEE Access 7:59263–59270 DOI 10.1109/ACCESS.2019.2910135.

Lee S, Kim D. 2022. Deep learning based recommender system using cross convolutional filters. Information Sciences 592:112–122 DOI 10.1016/j.ins.2022.01.033.

Liu H, Yu Y. 2022. A non-heuristic approach to time-space tradeoffs and optimizations for BKW. In: Agrawal S, Lin D, eds. Advances in Cryptology–ASIACRYPT 2022. ASIACRYPT 2022. Lecture Notes in Computer Science. Vol. 13793. Cham: Springer, 741–770 DOI 10.1007/978-3-031-22969-5_25.

Malandrino F, Chiasserini C, Molner N, de la Oliva A. 2021. Network support for highperformance distributed machine learning. IEEE/ACM Transactions on Networking 31(1):264–278 DOI 10.1109/TNET.2022.3189077.

Messina P, Dominguez V, Parra D, Trattner C, Soto A. 2019. Content-based artwork recommendation: integrating painting metadata with neural and manually-engineered visual features. User Modeling and User-Adapted Interaction 29(2):251–290 DOI 10.1007/s11257-018-9206-9.

Mintie K. 2023. A means of protection or destruction? copyright notifications on paintings in the united states, 1870-1911. American Art 37(3):9–12 DOI 10.1086/727541.

Musto C, Narducci F, Lops P, de Gemmis M, Semeraro G. 2010. Integrating a content-based recommender system into digital libraries for cultural heritage. In: Agosti M, Esposito F, Thanos C, eds. Digital Libraries. Volume 91 of Communications in Computer and Information Science, DELOS Assoc; Inst Inform Sci & Technol Italian, Natl Res Council; Univ Padua, Dept Informat Engn. 6th Italian Research Conference on Digital Libraries, Univ Padua, Dept Informat Engn, Padua, Italy. 27–38.

This material may be protected under Copyright Act which governs the making of photocopies or reproductions of copyrighted materials.
You may use the digitized material for private study, scholarship, or research.

Back to search page