生产实践---信创dass配置kafka consumer(kerberos认证)

要求配置2个管理节点 、vc_dass：
[gbase@GBASE-arm-coor04 ~]$ ip a |grep 32.1
[gbase@GBASE-arm-coor05 ~]$ ip a |grep 32.1

root用户改/etc/hosts，把星环的机器对应ip填进去
所有节点都要配
### TDH HH Cluster Start
#### Rack /C03
32.12.224.101 p-tdhhh1-mg1
32.12.224.1 p-tdhhh2-dn1
32.12.224.2 p-tdhhh3-dn2
32.12.224.3 p-tdhhh4-dn3
32.12.224.18 p-tdhhh21-dn18

root：安装kerberos客户端
[gbase@GBASE-arm-coor04 ~]$ rpm -qa|grep krb5
krb5-1.17-9.ky10.aarch64
krb5-libs-1.17-9.ky10.aarch64
krb5-client-1.17-9.ky10.aarch64
-------------

root：
krb5.conf 文件放到/etc目录，只配置管理节点，属主是root，权限是644
[gbase@GBASE-arm-coor04 ~]$ ll /etc/krb5.conf
-rw-r--r-- 1 root root 639 11月  8 10:01 /etc/krb5.conf

krb5.conf内容：
[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log

[libdefaults]
default_ccache_name= FILE:/tmp/krb5cc_%{uid}
default_realm = TDH
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
renew_lifetime = 7d
forwardable = true
allow_weak_crypto = true
default_tkt_enctypes = des-cbc-md5 des-cbc-crc des3-cbc-sha1
default_tgs_enctypes = des-cbc-md5 des-cbc-crc des3-cbc-sha1
permitted_enctypes = des-cbc-md5 des-cbc-crc des3-cbc-sha1
udp_preference_limit = 1000000

[realms]
TDH = {
kdc = p-tdhhh2-dn1:1088
kdc = p-tdhhh6-dn4:1088
kdc = p-tdhhh10-dn7:1088
}
==============================================

编辑 jaas.conf 
jaas.conf 内容：
KafkaServer {
 com.sun.security.auth.module.Krb5LoginModule required
 useKeyTab=true
 keyTab="/home/gbase/gbase_tdh/rds-gbase.keytab"
 storeKey=true
 useTicketCache=false
 principal="rds-gbase@TDH";
};

KafkaClient {
 com.sun.security.auth.module.Krb5LoginModule required
 useKeyTab=true
 keyTab="/home/gbase/gbase_tdh/rds-gbase.keytab"
 storeKey=true
 useTicketCache=false
 principal="rds-gbase@TDH";
};

// Zookeeper client authentication
Client {
 com.sun.security.auth.module.Krb5LoginModule required
 useKeyTab=true
 storeKey=true
 useTicketCache=false
 keyTab="/home/gbase/gbase_tdh/rds-gbase.keytab"
 principal="rds-gbase@TDH";
};
=============================================================

配置gbase用户环境变量文件.bash_profile
export KAFKA_OPTS="-Djava.security.auth.login.config=/home/gbase/gbase_tdh/jaas.conf -Djava.security.krb5.conf=/etc/krb5.conf -Dsun.security.krb5.debug=true"

rds-gbase.keytab 二进制文件
星环给过来的是keytab，改成rds-gbase.keytab（rds-gbase是星环创建的用户）

gbase用户下初始化：成功执行，无任何输出
cd /home/gbase/gbase_tdh
kinit -kt rds-gbase.keytab rds-gbase@TDH
查看信息
klist
查看安装kerberos的客户端版本
klist -V
查看信息
klist -ket  rds-gbase.keytab

如果要删除票据：执行   kdestroy
验证 ： klist

gc层参数：
gbase_kafka_keytab=/home/gbase/gbase_tdh/rds-gbase.keytab --要改
gbase_kafka_principal=rds-gbase@TDH   --要改
gcluster_lock_level = 10
gcluster_assign_kafka_topic_period=20
gcluster_kafka_max_message_size=1000000000
_gcluster_insert_cache_buffer_flag=1
gcluster_kafka_consume_batch=100
gcluster_kafka_consumer_enable=1
gcluster_kafka_batch_commit_dml_count=1000000
gcluster_kafka_local_queue_size=2001000
gcluster_kafka_user_allowed_max_latency=20000

gn层参数：
_gbase_transaction_disable=1
gbase_tx_log_mode=NO_USE
gbase_buffer_insert=1024M
gbase_tx_log_flush_time=5
_gbase_kafka_transaction_mode=ON

创建consumer：consumer和topic名，要用下划线，中划线会报错
use vc vc_dass;
create kafka consumer zhss_s_cbod_cboddbpd_saacntxn_131  
transaction topic 'gbase-zhss-rds-s_cbod_cboddbpd_saacntxn_131'  
brokers '32.12.224.20:9092,32.12.224.21:9092,32.12.224.22:9092,32.12.224.40:9092,32.12.224.41:9092';

启动kafka
start kafka consumer zhss_s_cbod_cboddbpd_saacntxn_131;

查询日志以及表的数据
select consumer,max_offset ,commit_offset ,max_offset-commit_offset  as offset_diff,exception from information_schema.kafka_consumer_status;

kafka状态
show kafka  consumer  vc_dass.zhss_s_cbod_cboddbpd_saacntxn_131;

消费脚本测试：
[gbase@GBASE-arm-coor04 gbase_tdh]$ 
/home/gbase/gbase_tdh/TDH-Client/kafka/bin/kafka-console-consumer.sh --bootstrap-server p-tdhhh23-dn20:9092 --topic tdcpro-xinghua-inceptor2-rds-s_cbod_cboddbpd_saacntxn --consumer-property security.protocol=SASL_PLAINTEXT --consumer-property sasl.kerberos.service.name=kafka