Independently Developed Full-Stack Replacement Solution for Bank of Jilin's Data Warehouse

1. Project Background

In Bank of Jilin's financial business scenarios, the data warehouse system involves a variety of data sources and types collected from multiple channels, requiring customized processing and computation for different data. The existing system could no longer meet the demands of continuous business development, prompting the bank to build a new, fully homegrown distributed data warehouse.

2. Construction Goals

The core purpose of this solution is to replace the foreign databases heavily used in current data warehouse operations, based on an all-indigenous technology infrastructure. The goals for the new platform include:

1. Build the data warehouse platform using Chinese-made chips, domestic operating systems, and domestic databases, creating a fully indigenous data warehouse platform.

2. Process massive amounts of structured data to enhance overall system performance and processing efficiency.

3. Sort out, standardize, and govern key financial business elements and processes; build foundational service components to achieve streamlined and standardized business workflows, standardized and centralized data, and componentized, service-oriented applications.

4. Provide standard interfaces, migration tools, and methods to help users quickly and easily complete the overall data warehouse migration.

5. Deliver high availability and high reliability to ensure uninterrupted business services and sustainable scalability to meet growing business demands.

6. Ensure consistent performance levels across all business systems.

7. Provide visual management and monitoring capabilities to help users understand real-time operational status and quickly locate and resolve issues.

3. Implementation Plan

The Bank of Jilin data warehouse indigenous replacement project began on July 15, 2020, using GBase 8a MPP Cluster— a large-scale distributed parallel database cluster—to replace the original Oracle database and build a new data platform that meets business needs and delivers greater value. The system went live on September 22, 2020. The project adopted a fully indigenous environment: “Hygon C86 7280 processor + NeoKylin V10 operating system + General Data Technology GBase 8a MPP Cluster V9 database.”

The project has been running for over 700 days. Compared with the previous system, batch data processing performance improved by 30%; storage space utilization was reduced by more than 50%; hardware and software investment was cut by 50% to 90%, and power consumption was reduced by 30% to 50%.

The platform consists of 20 nodes, with a total data volume nearing 1,000 TB, a daily increment of approximately 100 GB, over 15,000 tables, and the largest table exceeding 10 billion rows. Over 10,000 in-database processing jobs run daily. It provides 24/7 services to over 60,000 bank employees.

In terms of data loading, processing the main table of the data warehouse previously took 1 hour and 35 minutes on Oracle, while GBase 8a MPP Cluster completes it in just 15 minutes.

Daily batch processing performance improvement: Leveraging the fast multi-table join query capabilities of the MPP database, average model batch runtime was reduced by a factor of 5 to 10. Previously, the process took over 8 hours on Oracle; with GBase 8a MPP Cluster, it finishes within 3 hours—cutting batch processing time by more than half.

Data synchronization timeliness improvement: Using GBase data synchronization tools, efficient synchronization with the business system transactional databases was achieved, reducing sync time from 1 hour to 12 minutes and significantly boosting data analysis efficiency.

4. Project Value

The project built a complete indigenous data warehouse system, significantly reducing construction costs and offering an indigenous replacement solution for data warehouse construction in the financial industry.

The new system comprehensively supports applications in external regulatory compliance, asset-liability management, business management, and risk management. It provides efficient, timely, and accurate massive data services across the bank's six major business systems, serving as an innovation engine for business operations and development.