Zhejiang Mobile Big Data Foundational Platform — Tackling Big Data and High Concurrency
Project Background
The telecommunications industry is facing immense challenges driven by the rise of mobile internet and smart devices:
Shifting market demand: Customer needs are becoming increasingly diverse and niche, product portfolios are exploding, and touchpoints are rapidly multiplying. Traditional mass-marketing models can no longer keep pace with the current market dynamics.
Intensifying competition: The mobile internet era has escalated market rivalry. Operators now face competition not only from peer telecom companies but also from cross-sector players. As internet companies surge forward, the telco's historical dominance across the value chain is being systematically diluted. This evolving ecosystem has made competition more fierce than ever.
Technological hurdles: Rapid mobile internet growth has caused a data explosion in business support systems, with data types expanding to encompass massive volumes of network, service, user, and location data. Confronted with vast internet-scale data, legacy processing technologies have become a critical bottleneck to system evolution.
To overcome these challenges, Zhejiang Mobile must seize the opportunities of the big data era. It needs to adapt to trends such as diverse demands, fragmented behaviors, and a proliferation of applications by transforming its traditional marketing mindset. Centered on “Big Data, Hyper-Segmentation, Micro-Marketing,” the goal is to drive the transformation of pricing, channel, marketing, and communication capabilities, building a service system fit for the mobile internet age.
Requirements Analysis
The enterprise-level big data foundational platform deployment for a Zhejiang-based company needs to support the launch of 5 to 10 internal applications and 5 to 10 external applications, enabling data monetization and revenue generation;
Build an MPP resource pool cluster, primarily comprising a core data warehouse and data marts;
The core data warehouse MPP cluster mainly handles data modeling and foundational data storage and computing for the B (Business Support) and O (Operations Support) domains;
The data mart MPP cluster primarily supports internal applications and external monetization applications. Traditional data mart thematic applications such as VGOP, ESOP, and the Innovation Application Incubation Platform will also be gradually migrated to the data mart MPP cluster.
Solutions
Overall Architecture of the Big Data Foundational Platform (Current Phase):
All data exchange between the MPP cluster and external systems is handled through the cloud-based ETL platform. Application development for the MPP cluster is carried out on DACP (Data Management Subsystem). The Cloud Management Execution Center collects MPP cluster metrics and provides them to the Cloud Resource Management Platform;
The data warehouse MPP cluster primarily handles data modeling and foundational data storage. After business launch, the original data warehouse operations will be decommissioned. The GBase 8a MPP Cluster database platform adopts a 3+12×3 model, consisting of 3 data loading nodes and 12 data shard groups, each with 3 redundant copies. The current data volume is 168 TB. A 10 Gigabit Ethernet network provides high-speed connectivity within the cluster;
The data mart MPP cluster mainly supports internal applications and external monetization applications. The GBase 8a MPP Cluster database platform adopts a 3+24×2 model, consisting of 3 data loading nodes and 24 data shard groups, each with 2 redundant copies.
Value Highlights
Cost-Efficiency: GBase 8a MPP Cluster runs on low‑cost x86 PC servers, dramatically reducing hardware investment and cutting total system costs to about one‑tenth of the original.
Dynamic Scalability: GBase 8a MPP Cluster efficiently handles petabyte‑scale data, meeting the storage demands of both foundational and application data. It replaces traditional vertical scaling with horizontal scaling based on data volume, enabling dynamic expansion without service interruption to ensure service continuity.
High Availability: Through proper configuration, it achieves load balancing, fully leverages the computing power of each node, and enhances overall system collaboration efficiency. A redundant backup strategy ensures that node failures do not disrupt service continuity.
High Performance: It boosts data service capabilities to monetize data and generate revenue. Efficient data processing and query performance meet the needs of various thematic analyses and innovative applications.