GBase 8c Distributed Database Core Technology Revealed: Online Scaling

With the evolution of the internet era, online services often face sudden high-concurrency traffic during peak hours. Rapidly growing business demands place immense pressure on database computing and storage, causing load spikes that can severely disrupt normal system access. To address these challenges, GBase 8c, a multi-model multi-state distributed database, has enhanced its online scaling (expansion and contraction) capabilities.

Online Scaling

Thanks to its robust online scaling capabilities, GBase 8c comprehensively covers business scenarios with fluctuating traffic. Users can set scaling rules based on business requirements and strategies. When demand grows, the system automatically adds database nodes to ensure sufficient computing and storage capacity; when demand decreases, it automatically removes nodes to reduce costs.

Scaling Data Flow Diagram

Based on the scaling method, online scaling in GBase 8c can be classified into two types: regular tables and Hashbucket tables. Regular tables are created by default and are suitable for scenarios with small data volumes and ordered data files. Hashbucket tables excel in scenarios with large data volumes, complex data files, and high concurrency, where scaling and query performance are superior.

The principles of online scaling for regular tables and Hashbucket tables are outlined below.

Online Scaling Principles for Regular Tables

In GBase 8c, regular table scaling experiences only a brief (sub-second or sub-millisecond) offline period during the table switching operation in the data redistribution phase. Throughout the rest of the scaling process, online business operations remain fully supported. Storage node scaling consists of two phases: adding nodes to the cluster and redistributing data.

1. Node Addition Phase. This phase involves installing new nodes, synchronizing metadata, and starting the nodes.

First, new nodes are initialized and added to the cluster. The metadata from existing nodes is then synchronized to the new nodes to prepare for the subsequent data redistribution phase. After metadata synchronization, the new nodes are started, and the Installation Group is switched to the newly created Node Group.

2. Data Redistribution Phase. This phase handles the redistribution of data after storage node expansion and metadata switching.

Before redistribution, preparatory steps such as checking the old and new Node Groups, creating temporary tables, and setting up delete_delta tables are performed. Using a proprietary Hash data distribution algorithm, baseline data is redistributed, and incremental business data generated during the scaling period is caught up. After the final round of data catch-up, the catch-up mode is disabled, and finishing operations such as table switching, temporary column deletion, and Node Group updates are completed.

Online Scaling Principles for Hashbucket Tables

Additionally, GBase 8c supports creating Hashbucket tables. The creation command is as follows:

CREATE TABLE tab_name(rel_name rel_type) with(segment=on,bashbucket=on);

Hashbucket tables use a segment-page storage approach to address the problem of numerous data files.

Each bucketid is stored in a contiguous segment, and the same bucketid is stored in the same file. Therefore, during online scaling, only the block files corresponding to the changed bucketids need to be moved. During table switching and physical file exchange, only modifications to the storage structure are required. This significantly reduces the amount of data to be moved.

Hashbucket Table Expansion Data Flow Diagram

The data flow diagram for Hashbucket table expansion is shown above. For example, before expansion, data with bucketid 1, 2, and 3 reside on node DN1, while data with bucketid 4, 5, and 6 reside on node DN2.

During expansion, if the new node is not yet in the original cluster, the newly added DN3 must first be incorporated into the cluster (as shown in step ①). Based on the new map generated by the data distribution algorithm, only the data for bucketid 3 and 6 are moved from the existing nodes to the new DN3 node (as shown in step ②); data for other bucketid values do not need to be relocated. Thus, instead of redistributing data across all nodes, only the data corresponding to changed bucketids is moved, reducing the amount of data transferred during scaling and effectively improving the expansion speed.

About GBase 8c

GBase 8c is a multi-model multi-state distributed database built on openGauss 3.0. It supports multiple storage modes—row-based, column-based, and in-memory—as well as various deployment topologies such as standalone, primary-standby, and distributed. With features including high performance, high availability, elastic scalability, and robust security, GBase 8c can be deployed on physical servers, virtual machines, containers, private clouds, and public clouds. It delivers secure, stable, and reliable data storage and management services for core systems in critical industries, internet business systems, and government/enterprise applications.