Construction of a telecom operator’s base business analysis system — Efficient massive data processing and fast query response

● Rapid Deployment: GBase 8a MPP supports the SQL92 standard, and its syntax is highly compatible with traditional DB2-based data warehouses, significantly shortening application development cycles and reducing project labor costs.

● Massive Data Support: GBase 8a MPP serves as the core data warehouse for the entire base's business analytics system, handling multi-layered data processing. Leveraging efficient complex relational operations and data association capabilities, it ensures robust integration and timely processing of massive and diverse business data.

● Fast Query Response: As a data warehouse platform, GBase 8a MPP dramatically improves data retrieval efficiency and processing parallelism, enabling both efficient offline batch processing and interactive data workflows such as self-service data extraction on a single platform.

The platform architecture in this phase includes the following components:

● Data Interfaces: The platform aggregates existing business data, including data from big data platforms, core platforms, and related business systems. Interface data encompasses various types collected from business systems, such as weather, lifestyle, work, agricultural meteorology, government affairs, core platform transactions, marketing support, base websites, and hotline services.

● Data Computing & Storage: The data warehouse employs an MPP architecture to deliver high-concurrency, high-performance data processing. It consists of ODS, DWD, DW, and DM layers, forming an application-oriented data warehouse model.

● Data Applications: The system provides external indicator and labeling services.

● Data Sharing: Service delivery supports three modes of data sharing: API, file, and messaging.

● Data Asset Management: Provides unified scheduling, monitoring, and distribution of data.

As the data storage and management layer, the data warehouse platform uses GBase 8a MPP Cluster to store and manage ODS data collected and cleansed from various systems. Leveraging the MPP database’s powerful complex relationship processing and data association capabilities, it incrementally processes data on top of ODS to form the in-database DWD and DWA warehouse layers. Data mart (DM) data is then exported from the warehouse layer to support upper-level applications for indicator computation, labeling algorithms, and self-service data extraction. For 94,156 tables in the ODS layer and 7,219 tables in the DW layer, the system completes all data loading processes at 2:00 AM every night and ensures all indicator calculations and label generation are finished before business hours start at 8:30 AM the next day, significantly boosting data processing throughput.

This facility is a national public welfare information service platform built and operated by a major telecom operator to serve people’s livelihoods nationwide. As its business scale and diversity grew, data volumes expanded at an unprecedented rate and data types became increasingly varied. Legacy data management capabilities could no longer meet evolving business needs, making centralized data storage and robust data asset governance imperative. Meanwhile, business demands for data development and utilization also diversified, including data labeling, data mining model creation, and KPI monitoring and visualization. These requirements drove an urgent need to build a user data analytics platform with a modern architecture. At the time, the business analytics system running on minicomputers and traditional relational databases suffered from poor scalability, performance degradation as data grew, inflexible data models, and high database optimization costs—especially for big data applications like data analysis. The goal of the user data center analytics platform project is to effectively aggregate the facility’s data and enable efficient processing of massive datasets. This phase adopts an advanced data warehouse architecture, models, and software capabilities to standardize data asset management and seamlessly integrate internal data.

The site’s user data center analytics platform achieves effective consolidation of internal data, exposing tag and metric capabilities through various open interfaces such as self-service data extraction and APIs. The platform provides market-oriented marketing support for role-based users and business operations:

● Enable centralized storage of data and centralized management of data assets;

● Build upstream and downstream data interfaces with business platforms and supporting platforms;

● Integrate with the Chongqing business big data platform.

The analytical business objectives achieved in this project include:

● Indicator visualization (KPIs, reports), data indicators: 6 business areas, totaling 135 indicators;

● Data tags: 6 business areas, totaling 734 tags;

● Data mining: 6 business areas, one model per area, totaling 6 models;

● External tag services: provide external tag services through various methods such as self-service data extraction and APIs.

As the total data scale hosted by the system's data warehouse, it handles a data volume of 60TB, with a daily data intake of 200GB. The entire analytical business involves nearly 100,000 tables, and daily data processing must be completed before the next day's business hours (8:30 AM).