GBase 8a Database Data Loading Features and Tuning (Part 3)

Published on 2025-02-21

This article, in three chapters (Chapter 3), explains the loading function module of the GBase 8a database cluster from both principles and architecture, provides loading performance parameter tuning solutions and Kafka data source loading and exporting solutions, offering a reference for users to understand and use the GBase 8a loading features.

Kafka Data Integration

The GBase 8a cluster supports using a Kafka cluster as a data source for data loading and exporting. Information about Kafka data source loading tasks can be viewed in the table topicname_dbname_tbname within the gclusterdb system database.

The SQL for loading text data from a Kafka cluster is as follows:

The SQL for exporting text data to a Kafka cluster is as follows:

Kafka Broker High Availability

GBase 8a supports configuring Kafka broker high availability and supports a Kafka cluster with Kerberos authentication as a loading data source:

1. Add the parameter in the gcluster configuration file:

gcluster_kafka_brokers 192.168.146.20:9092,192.168.146.21:9092’

2. Restart the gcluster service to make it take effect:

gcluster_services all start

Kafka Streaming Loading

GBase 8a implements Kafka streaming loading through scripts, for example: