GBase 8a MPP Cluster V9 Virtual Cluster Technology
During the actual use of MPP clusters, there are management challenges such as limited cluster scale, inability to address hardware generational differences within a single cluster, and the inability to achieve real-time data synchronization and rapid application switching across multiple clusters for active-active configurations. To address these issues, GBase 8a MPP Cluster V9 adopts virtual cluster technology.
The virtual cluster technology adopted by GBase 8a MPP Cluster V9 is a means of resource isolation. It physically partitions a large cluster vertically into multiple logical sub-clusters. Each logical sub-cluster can independently plan and scale cluster size and computing resources based on the storage and computing requirements of different services.
The virtual cluster provides a unified access entry, a unified metadata view, unified resource management, unified execution scheduling, and unified authentication and authorization management for all logical sub-clusters. It enables data interaction, data migration, and data association across clusters. It supports cluster mirroring, with table-level real-time synchronization between mirrored clusters, realizing real-time active-active data and T+0 high availability.
Application Scenarios
Virtual clusters are suitable for planning and building multiple clusters within a system, where the services of each cluster are independent and can be managed independently. Virtual clusters encompass data management, user management, and cluster version management. Transparent data migration, data association, and data sharing can be achieved among logical sub-clusters within a virtual cluster.
User Value
Virtual clusters are ideal for big data platforms, comprehensive BI systems, data warehouses, and data mart systems that contain relatively independent business domains or different types of analysis. Different application scenarios run in independent logical sub-clusters, with unified management of all logical sub-clusters. This not only addresses the high cost of managing, monitoring, and maintaining multiple physical clusters but also meets the differentiated characteristics of different business scenarios, maximizes resource utilization, and enhances cluster scalability and maintainability.
Virtual Cluster Features
Virtual clusters unify the entry and management of multiple MPP clusters, enabling data integration across multiple MPP clusters and functioning as a single logical cluster overall. The cluster is managed in layers, divided into the scheduling management cluster and virtual clusters. The scheduling management cluster serves as the unified entry and management for the entire cluster, while virtual clusters handle data storage and computing, and can be scaled indefinitely. Data from any virtual cluster in the large cluster can be accessed through the scheduling management cluster.
Unified Management
By partitioning virtual clusters within the cluster, expansion capability is enhanced while providing a unified management view, allowing the cluster to scale to thousands of nodes.
Unified Entry
Users select any coordinator node in the coordinator cluster as the cluster entry point. The coordinator node determines the default VC based on the user.
Note: Coordinator nodes store metadata for all databases and tables of all VCs.
Service Isolation
Virtual clusters achieve vertical resource isolation within the cluster, enabling physical isolation of resources for different services in actual business scenarios.
Transparent Access
Virtual clusters provide a unified access entry (enabling a unified access point for warehouses and data marts). Applications access multiple physical clusters transparently, as if accessing a single unified cluster.
Virtual Clusters vs. Multi-Cluster + Large Cluster
Compared with other solutions, virtual clusters offer the following value from a user perspective:
Mirror Virtual Clusters for Active-Active
Real-time data synchronization between mirrored virtual clusters enables active-active and stronger high availability on the database side:
Data in a mirror table is a real-time backup of the primary table; modifications to the primary table are synchronized to the mirror table in real time.
High Availability: With the mirroring relationship established for the primary table, the SELECT module gains additional shard backups for the primary table. The SELECT module can leverage these additional backups to enhance SELECT high availability.
Supports creating and deleting mirroring relationships at the database level. All tables under the database will be mirrored. All functions, stored procedures, and views under the database will be synchronously created in the target database.
Mirror clusters can be used in scenarios such as disaster recovery and read-write separation.
Disaster Recovery: Supports deploying clusters in the same city or different locations.
Read-Write Separation: Data warehouses are used for data analysis and computation, and result data is synchronized to data marts via mirror cluster technology for real-time query and analysis by business systems.