Fun Talk on GBase 8a Database Cluster (Part 2)
Hello GBase enthusiasts and those interested in our technology! The GBase 8a Database Cluster is a core analytical database product independently developed by GBase. Starting R&D in 2009, it has undergone 13 years of evolution and development. It has achieved large-scale deployment in financial and telecom sectors, and covers all other industries beyond finance and telecom, with over 30,000 nodes deployed and managing 250+ PB of data.
In the previous 'Fun Talk', we learned that the evolution of the GBase 8a MPP Cluster product is divided into three stages: column-store database stage, MPP + column-store database stage, and logical data warehouse (LDW) stage. The earlier stage was the R&D foundation for the later one. This 'Fun Talk (Part 2)' continues from the first article to discuss the product architecture of GBase 8a, which can also be roughly divided into three parts:
(1) Architecture of the Column-Store Database Stage
The column-store database stage is the nascent phase of GBase 8a MPP Cluster's core features. It is a single-node architecture product, similar to traditional relational database architectures. The difference is that traditional relational databases are usually row-store databases, whereas GBase 8a single-node database is a column-store database, meaning its data storage organization differs from traditional relational databases. This is the most core part of GBase 8a single-node database. Additionally, around column-store, it introduces data compression storage, intelligent indexing, and parallel execution capabilities. What do these features bring? In one word: 'Fast'. They deliver exponential performance improvements for analytical queries. The principle behind this 'Fast' is shown below:
In most applications, the performance bottleneck is disk I/O. Therefore, the design of GBase 8a single-node database primarily aims at reducing disk I/O. Techniques employed include column storage, compression, and intelligent indexing (coarse-grained). Each of these can potentially improve performance by 10 times. The theoretical maximum is 1000x speed improvement, and in practice, 10-100x is achieved. As illustrated above, for a table occupying 1TB of storage, during statistical queries, column storage technology reduces disk I/O by reading only the queried columns for pruning; data compression reduces disk I/O; intelligent indexing filters out irrelevant rows to reduce disk I/O, thus enhancing query performance. You might ask, does 'Select *' also see performance improvement? My answer is: if the principles above are clear, you will obtain the answer yourself, which is far more accurate than my simply saying 'Yes' or 'No'.
(2) Architecture of the MPP + Column-Store Database Stage
The MPP + column-store database stage is the growth and maturation phase of the GBase 8a MPP Cluster product. It went through three major version iterations, scaling from dozens of nodes to hundreds, and then to over three hundred nodes. Step by step, GBase 8a grew with the round-the-clock dedication of its R&D team. In this MPP + column-store database stage, there are three architectures: symmetric deployment architecture based on security groups, asymmetric deployment architecture based on multiple replicas, and the federated architecture.
Enough talk, let's get to the diagram. A picture is worth a thousand words—those who understand will get it at a glance, and for those who don't, more words won't help.
Symmetric Deployment Architecture Based on Security Groups
In this cluster architecture, security groups are the building blocks. Each security group can consist of up to three nodes to achieve high availability. Multiple security groups enable data distribution and parallel computing to boost performance. On each cluster node, GCluster, GNode, and Gcware services are deployed and run, responsible for cluster management, storage and computing, and cluster consistency services, respectively.
Asymmetric Deployment Architecture with Multiple Replicas
Compared to the previous version, the asymmetric deployment architecture with multiple replicas no longer relies on security groups but uses multiple replicas for high availability. GCluster, GNode, and Gcware services are more independent. GCluster and GNode services can be deployed on separate cluster nodes (in the previous version, each cluster node had to run all services). That is, the GCluster cluster as the service layer can be deployed independently, and the GNode cluster as the compute and storage layer can also be deployed independently.
Federated Architecture
Compared to the previous version, the compute and storage layer can deploy multiple GNode clusters, each called a VC (Virtual Cluster), which share the same GCluster cluster service layer.
(3) Architecture of the Logical Data Warehouse (LDW) Stage
The architecture of the logical data warehouse (LDW) stage incorporates cloud-native features, that is, the separation of compute and storage characteristic of cloud computing. As follows:
The overall architecture consists of three layers: the cloud service layer, the virtual warehouse layer, and the data storage layer. The cloud service layer provides management and operational capabilities, including order management, billing management, cloud data warehouse instance account management, cloud service monitoring and O&M management, cloud data warehouse instance management, and upgrade management for cloud services and cloud data warehouse. It also provides the cloud data warehouse instance access entry (URL), authentication, access control, SQL parsing, execution plan generation, optimization, metadata management and storage, transaction management and execution scheduling, user and permission management, warehouse management, database management, and functional services for cloud data warehouse. The virtual warehouse layer provides computing resources of different specifications to execute compute tasks dispatched by the cloud service layer. The data storage layer offers distributed file storage or object storage capabilities to persist cloud data warehouse data files, and provides caching for temporary table data and result set cache persistence during queries.