Sichuan Mobile Integrated Operations Platform (IOP) Data Application and Service System Project

1. Project Overview

   • Ability to load data collected by the integrated business management platform into the database in real time;

1. Background

The source data in the Integrated Operations Platform (IOP) is prepared for big data analysis. Given the characteristics of concurrent queries and data loading of IOP source data, and with the development of telecom services, more source data will need to be loaded into the IOP staging area with higher concurrency. GBase 8s excels at handling such ultra-large-scale concurrent processing, so this workload is separated from the existing IOP to form a new IOP staging system that provides better data support services for IOP. To improve the high availability of the IOP staging system and facilitate horizontal scaling as business grows, GBase8s database clustering technology supports the IOP staging system. The separated IOP staging system has a data volume of approximately 10TB and handles around 500 concurrent operations.

2. Key Concerns

Real-time processing capability for ultra-large-scale concurrent queries and data loading with data volumes exceeding 10TB.

3. Requirements

• High stability to support 7×24 operations of the integrated business management platform;

• A high-availability cluster based on shared storage cluster architecture;

• Support over 500 concurrent operations during peak hours, with a data volume exceeding 10TB;

• Provide high-availability automatic node failover without manual intervention;

• Provide bidirectional replication between database servers for specific tables;

• Millisecond-level response time for tables with hundreds of millions of rows, meeting the platform's peak processing requirements;

• The database provides load balancing to accommodate significant future data growth;

Provide a solution based entirely on China-developed technologies, spanning servers, storage, and database, requiring the database to fully adapt to China-developed

hardware platforms.

1. Solution

In the Sichuan Mobile data application and service system, indicator source data is primarily loaded in batch mode, and business queries mainly retrieve and display relevant indicator data, with a concurrency of around 500. The total data volume is approximately 10TB. The GBase8s shared-nothing cluster is used to support operations. Based on the current concurrency and data volume, a 2-node database cluster is initially set up; additional nodes can be added later when performance no longer meets user requirements due to increased concurrency or data growth. In the 2-node cluster, one node acts as the primary to handle batch data loading, while concurrent query operations can leverage both nodes to balance the overall system load.

Applications connect to the database cluster through the Connection Manager (CM). When the primary node fails (e.g., power outage, network disconnection, CPU/memory failure, OS or database software failure), applications that were connected to the primary switch automatically to the secondary node, which then becomes the primary. The switchover time depends on the primary node's transaction load at the time, with a minimum of under 10 seconds. When the secondary node fails, applications connected to it switch automatically to the primary node, which continues supporting all business applications. For batch data loading, the GBase8s external table feature can rapidly load data into the database. To better support concurrent queries, all tables should use row-level locking. Large tables should be fragmented to improve parallel operations, efficiency, and data management. For related tables, indexes should be created on query conditions or join fields to speed up queries. Large table indexes can also be fragmented to leverage parallel processing for faster index data retrieval. When loading large data volumes or creating indexes on large tables, PDQ (Parallel Data Query) can be enabled to accelerate data loading or index creation.

1. Primary and secondary nodes do not share storage and have independent database storage space, preventing single point of failure in the cluster. In addition to being connected via fiber optic network or within a LAN, a heartbeat network is configured to monitor each other's status and prevent node failures.

2. All applications connect to the database nodes through the Connection Manager (CM) rather than directly. CM provides read/write splitting, load balancing, and failover when the primary node fails, with a minimum switchover time of less than 10 seconds depending on the workload. To ensure comprehensive high availability, it is recommended to deploy CM on multiple application servers separate from the database nodes.

3. In addition to the database cluster, various utilities are provided for database management, application development, security management, and database migration for user convenience.

• Application Outcomes

  Independence and Security Strategy: This project has set a positive example for the financial industry in advancing the 'de-IOE' strategy, accelerating the implementation of independently developed and secure IT regulations, and promoting the evolution of industry technology architecture;

  Fully Compatible, Fully Adaptive China-Developed OLTP Database: Delivers a financial business platform solution built entirely on China-developed technologies, from servers and storage to the database, ensuring robust data security for customer business systems;

  Meeting Customer Business Requirements: Satisfies the need for a high-concurrency data processing system, enhances system high availability, and supports future system expansion as business grows.