DEVELOPMENT OF A BIG DATA-BASED MARKET SEGMENTATION SYSTEM TO IMPROVE THE EFFECTIVENESS OF PERSONALIZED MARKETING

Abstract

The article presents the design and initial implementation of a reproducible prototype for a Big Data-driven customer segmentation system aimed at personalized marketing. Building upon a prior review of system development challenges, this work focuses on the practical engineering aspects. It justifies the selection of the Big Data stack (Apache Spark, Kafka, Parquet, MLflow) and proposes a hybrid lambda architecture supporting both batch and stream processing. Feature engineering incorporates RFM metrics, behavioral, and temporal patterns. Experiments on a synthetic dataset simulating user behavior on a streaming platform compare K-Means and DBSCAN clustering algorithms, with K-Means yielding more interpretable segments based on silhouette and Calinski-Harabasz scores. The prototype provides a complete pipeline from data ingestion and storage to modeling and segment serving. The practical significance lies in demonstrating a scalable approach to reduce customer acquisition cost and improve targeting precision, leading to potential increases in CTR and CVR. The code and methodology are fully disclosed to ensure reproducibility.