Replace foreign integrated hardware-software MPP products: GBase drives data warehouse localization, accelerating cost reduction and efficiency gains.

According to IDC's "Digital Universe" research report, by 2020, China's total data volume is expected to reach 8,000 EB, accounting for 22% of the global total. Meanwhile, according to the "2019 China Big Data Market Annual Report", China's big data market reached RMB 10.005 billion in 2019, and its annual growth rate is expected to exceed 30% over the next 3–4 years, making it a leading data powerhouse and a global data center hub.

With the rapid growth of big data in China and the continuous expansion of massive data, when building and scaling systems, users require database products to ensure high security, high performance, maturity and stability, while also significantly reducing data processing costs to achieve cost reduction and efficiency gains.

Foreign Integrated Hardware-Software MPP Products

Foreign integrated hardware-software MPP products enjoyed an early-entry advantage in China, capturing significant market share. However, with the implementation of the domestic substitution strategy, their peak in the Chinese market has passed, and they have attracted almost no new customers.

Foreign integrated hardware-software MPP products are built on closed architectures with stable performance, suitable for medium-sized data warehouses. However, they face difficulties and poor efficiency when scaling to petabyte (PB) levels. They are expensive, costing several times more than domestic alternatives; difficult to scale due to tight hardware-software integration; incur high expansion, maintenance, and service costs. Their closed architecture locks customers into a single vendor for hardware, software, and services, and their limited technical support staff in China further constrains service capabilities.

Therefore, based on considerations such as reducing construction costs, decoupling from integrated proprietary systems, improving system expansion flexibility, saving operational expenses, achieving independent control, and the maturity of domestic products, there are two approaches to replace foreign integrated hardware-software MPP products: one is to achieve compatibility and provide replacement and services; the other is to collaborate with third-party partners to offer a broader range of product replacements and services.

GBase 8a MPP vs. Foreign Integrated Hardware-Software MPP Products

GBase 8a MPP is an independently developed product by GBASE (General Data Technology Co., Ltd.), offering full control and independence. It is built on an open x86 and Linux architecture, delivering high performance, maturity, stability, easy scalability, low cost, and seamless integration with upstream and downstream software. It is suitable for platforms ranging from terabytes to 10 petabytes, including data warehouses and data marts. With a technical team of 200 professionals in China, it provides high-quality localized services to customers.

Similarities between GBase 8a MPP and Foreign Integrated Hardware-Software MPP Products

Differences between Foreign Integrated Hardware-Software MPP Products & GBase Product and Services

Advantages of GBase & Foreign Integrated Hardware-Software MPP Products

GBase 8a MPP offers advantages such as unified management of massive data, high levels of data sharing, efficient and deep multi-dimensional association analysis, low-cost storage and management of large data volumes, high availability, near-linear online scalability, and an independently developed domestic database that ensures full control. It also achieves compatibility with foreign integrated hardware-software MPP products, providing analytical database replacement and services.

Feasibility of Replacing Foreign Integrated Hardware-Software MPP Products

For a big data system expansion of 200 TB, foreign integrated hardware-software MPP products would cause a complete service interruption of 10 hours during expansion, disrupting all user operations. With GBase 8a MPP, data volume expansion only affects data modification and deletion operations, while query and incremental data services remain uninterrupted.

If 90 data zipper jobs run in parallel, based on results, using 24 GBase nodes maintains full performance and allows applications to be decoupled from the foreign integrated hardware-software MPP system. In the original environment: 32 nodes supported 140 applications, with deployment costs in the tens of millions (RMB). After switching to GBase 8a MPP: 24 nodes support 140 applications, with costs reduced to the millions.

GBase 8a MPP Replacement and Expansion Solution

For building a new big data platform with GBase 8a MPP, a domestic distributed database, or replacing/expanding an existing foreign integrated hardware-software MPP system, the solutions are as follows:

For customers who have not yet built a data platform: foreign integrated hardware-software MPP products offer stable performance but are prohibitively expensive, with unsustainable later-stage expansion costs. GBase 8a MPP can be used to build the entire data platform. For customers already using foreign integrated hardware-software MPP products who face expansion needs or cost pressure: GBase 8a MPP is the ideal choice for offloading or gradually migrating to an MPP architecture. The foreign MPP product continues to serve as the primary data warehouse for high-concurrency, high-intensity operational workloads, while GBase 8a MPP can handle warm data analytics. For customers who have built their system on a foreign integrated MPP product but can no longer afford the costs: they can adopt a coexistence approach, gradually migrating workloads to GBase 8a MPP, ultimately achieving a perfect replacement of the foreign product with a cost-effective domestic alternative.

Example of Analytical Database Feature Compatibility Migration

GBase 8a MPP Migration Plan

GBase 8a MPP now has a comprehensive and well-established migration plan, covering the migration process and risk mitigation strategies, to make migration: tool-automated, process-standardized, task-simplified, and risk-minimized.

GBase 8a MPP Migration Plan

During migration, engineers deploy the migration tool on a Windows or Linux server. They launch the tool and configure migration tasks using its configuration wizard, setting JDBC connection parameters for both the source and target databases. They then configure the content to be migrated and field mapping relationships. The tool provides default field mappings that meet the needs of most migration scenarios.

Once the migration tasks are configured and started, the tool supports concurrent execution of multiple table and data migration tasks. After launching, migration progress can be monitored via the interface. The migration tool retrieves table structures and data from the source database through JDBC according to user settings, transforms the source content based on mapping rules, and then writes the converted structures and data to the target database via JDBC, completing each table’s migration until all tables are migrated.

GBaseMigrationToolKit System Architecture

For DSQL script migration, engineers prepare the DSQL scripts to be migrated. The migration tool is deployed on a Windows or Linux server, and the input directories, files, and related configuration files are set up. After starting the configured migration tasks, the tool reads the script files from the input directory, parses and converts the DSQL statements, and then directly outputs and executes them on GBase 8a MPP. The executed statements are logged, completing the migration.

DSQL Script Migration Tool System Architecture

Replacement Case Study

Take a data warehouse migration project at an insurance company as an example. In the original environment: performance pressure was high, the foreign integrated hardware-software MPP data warehouse handled many workloads with high load, daily reports could at best be updated only on T+2, and monthly reports were often not released until the 15th. The architecture was complex, data flows were convoluted, and data synchronization between heterogeneous upstream and downstream databases relied on complex scheduling programs. Costs were high due to the tightly coupled hardware-software of the foreign MPP product, resulting in high operational and maintenance costs. Scaling was extremely difficult: the warehouse’s storage capacity and processing performance had reached a bottleneck, preventing vertical scaling, and scaling required service interruption and could not reuse existing hardware.

After migrating to GBase 8a MPP, incremental processing time was reduced from 18 hours to 7 hours, with daily incremental base data available before 7:00 AM, a 2.5x improvement. Monthly report execution improved from 48 hours to 4.5 hours, allowing data release before the 3rd of each month, a 10x improvement. Full processing time was cut from 7 days to 16 hours, a 10x improvement, delivering an overall 2–10x performance uplift.

GBase 8a MPP is comparable to foreign integrated hardware-software MPP products in functionality, performance, stability, and availability. The case study demonstrates a 100% migration success rate with manageable effort, proving that GBase 8a MPP can replace foreign MPP products across various application scenarios.

Meanwhile, with the continuously improving compatibility of GBASE products, the adoption of cloud applications, the advancement of Chinese big data technology, and the development of domestic hardware, data middle platforms, and open-source data analysis tools, various components of data platforms can now be decoupled — especially big data analytics capabilities. This allows foreign integrated hardware-software solutions to be replaced through partnerships. As a result, in the era of massive data, cost reduction and efficiency gains can be achieved for the growing data demands of data-intensive industries.