State Grid Corporation of China “Online State Grid” Project

Project Background

"Online State Grid" is a mobile interactive service platform developed by the State Grid Corporation of China. It primarily enables customers to pay bills, purchase electricity, check consumption, subscribe to information, and access online customer service via mobile devices. Its features include bill payment and electricity purchase, consumption inquiry, information subscription, online customer service, service outlet navigation, and power outage announcements. The platform serves 26 provinces (including autonomous regions and municipalities), covering more than 88% of the country's land area and a population of over 1.1 billion.

Business Needs

The State Grid required the Online State Grid platform to deliver millisecond-level query performance against single tables containing more than 20 billion rows. Neither Oracle, the Hadoop ecosystem, nor the State Grid's own big data platform could achieve the required sub-millisecond response times.

Implementation Requirements

Support reporting and statistical analysis for the Online State Grid: extract and aggregate data from over 300 business tables daily, each exceeding 100 million records (approximately 50 GB); synchronize an average of more than 10 million profile change records daily from provincial subsidiaries; and aggregate over 20 million user behavioral log records every day.

For metric computation and extraction: perform batch processing on hundreds of millions of records each day to provide data support for more than 300 operational metrics.

For base data distribution: distribute the aggregated data from the Online State Grid to 27 organizational units across the network daily, dispatching over 20 million records (approximately 15 GB) every day.

The system is built on GBase 8a MPP Cluster with 20 nodes. Data on each node is compressed, with each node storing over 10 TB of data.

Business scale: Over 3 billion rows of data are ingested daily (incremental data exceeds 300 GB per day), while supporting real-time queries of app bills and consumption data for 20 million users.

Report and analytics: Over 300 business data tables (each with more than 100 million records, approximately 50 GB) are extracted and aggregated daily. On average, over 10 million profile change records are synchronized from provincial networks each day, and over 20 million user behavior log records are aggregated.

Metric computation: Daily batch processing of hundreds of millions of records is performed to provide over 300 operational metrics.

Data distribution: Aggregated data from State Grid Online is distributed to 26 provinces (autonomous regions and municipalities) daily, with over 40 million records (approximately 30 GB) sent out each day.

Figure 12-1 Architecture of the State Grid Online Report and Analytics System

Implementation

The system is built on GBase 8a MPP Cluster with 20 nodes (20 data nodes and 3 management nodes). Currently, it holds approximately 200TB of data, with a daily data increment of around 400GB and typical concurrency between 100 and 500.

Results and Value

Massive Data Processing: Provides a cost-effective, massively parallel complex data processing platform, empowering the customer to deliver services such as electricity bill inquiry, consumption and charge analysis, and energy usage analysis across 26 provinces (autonomous regions and municipalities) serving 1.1 billion electricity consumers.

High Performance: The system architecture is highly scalable, with performance scaling as nodes are added, ensuring that customers can integrate more comprehensive business data to meet their business needs.

High Compression Ratio: Delivers comprehensive compressed-state storage for managing massive data, further reducing data warehouse construction costs and boosting system performance.