“G” Moment: A Complete Guide to Migrating from HBase to GBase 8a Without Getting Lost

Published on 2025-06-30

In today’s data-driven business landscape, data migration is like moving your “digital assets” to a new home—you need to ensure all “furniture” arrives intact and the new layout is optimized. Recently, many community members have been asking how to smoothly migrate Hadoop business data to a GBase database cluster. Below we provide a detailed migration implementation process for your reference, so your data relocation is no longer a maze.

 

Migration Implementation Plan

Prerequisites from Hadoop Cluster

To allow GBase 8a MPP Cluster to access HDFS data, the Hadoop cluster must provide the following:

  • Hostname-to-IP mapping for Hadoop cluster NameNodes and DataNodes

  • Kerberos client configuration file krb5.conf

  • Kerberos authentication keytab file (.kt) and principal name

  • Ports to open (RPC protocol):

a) Port from all Hadoop cluster nodes to all GBase cluster management nodes (default 5258)

b) Port from all Hadoop cluster nodes to all GBase cluster data nodes (default 5050)

c) Port from all GBase cluster nodes to all Hadoop cluster nodes (e.g., 25000)

d) Port range from all GBase cluster nodes to the KDC server, such as 27230–27232 (can be found in krb5.conf)

  • Table schemas for all tables to be migrated

 

GBase Cluster Implementation Plan

Creating Table Structures

Based on the provided schemas, create the target tables in the appropriate database; we recommend creating hash-distributed tables.

Configuring /etc/hosts

Since Hadoop and Kerberos are highly dependent on DNS (both forward and reverse lookups), it is recommended to use hostnames rather than IP addresses in the URLs of load and export statements within a Kerberos-enabled environment.

Therefore, configure the hostname-to-IP mappings for all Hadoop cluster nodes in /etc/hosts on every GBase cluster node.

Installing the Kerberos Client

Install the Kerberos client on all GBase cluster nodes as follows:

# Run as the root user

yum install libkadm5 krb5-libs krb5-devel krb5-workstation

Distributing Keytab Information

Modify the ownership of the keytab file yitong_guest.kt and distribute it to the config directory of the cluster nodes.

Change the file owner to gbase and set permissions to 644:

# Run as the root user

chown gbase: gbase biao_guest.kt
chmod 644 biao_guest.kt

Distribute to the cluster config path:

# Run as the gbase user

cpush coor:biao_guest.kt /gbase/gcluster/config
cpush data:biao_guest.kt /gbase/gnode/config

Distributing the KDC Configuration File

Distribute the KDC configuration file to /etc/ on all cluster nodes:

# Run as the root user

cpush all: krb5.cnf /etc/

Setting Cluster Parameters

Log into the database and execute the following commands:

# Run as the gbase user

gccli -uxxx -pxxx
gbase>
set global gbase_hdfs_auth_mode=kerberos ;
set globl gbase_hdfs_protocol=RPC’;
set globl gbase_hdfs_port=25000’;
Set global gbase_hdfs_principal =’biao_guest@HADOOP.COM’;
Set global gbase_hdfs_namenodes=’namenode1,namenode2’;

# Management node:

Set globalgbase_hdfs_kt =’/gbase/gcluster/config/biao_guest.kt’;

# Data node:

gncli -uxxx -pxxx
Set global gbase_hdfs_kt =’/gbase/gnode/config/biao_guest.kt’;
Note: After successful verification, configure the above parameters in the config file to avoid losing them after cluster node service restarts.

Kerberos Ticket Authentication

On the management node:

kinit -kt /gbase/gcluster/config/biao_guest.kt biao_guest@HADOOP.COM

On data nodes:

kinit -kt /gbase/gnode/config/biao_guest.kt biao_guest@HADOOP.COM

Test Data Loading

Create a test table in the database and then execute the following test load statement:

# Run as the gbase user

gccli -uxxx -px
gbase>load data infile ‘hdp://biao_guest@namenode1:25000//user/biao_guest/gbasetest20250313.txt’ into table all_judicial_inquiry_new_2020 data_format 3 fields terminated by ‘|@|;

 

Impact of Installing Kerberos Authentication on the GBase Cluster

Impact on Cluster Expansion

After cluster expansion, administrators must manually install and configure the Kerberos client on the new nodes.

Impact on Cluster Upgrades

When upgrading from a version that does not support Kerberos authentication to one that does, administrators must manually install and configure the Kerberos client on all nodes.

Node Replacement Tool

The node replacement tool in Kerberos-enabled cluster versions supports this feature. During file synchronization, the tool must also copy files with the .kt extension under $GCLUSTER_BASE/config or $GBASE_BASE/config to the replacement node.