sql优化_函数试用错误,参数不合理使用导致触发查询超时
部分点dataworks同步数据从gbase到odps,原下发sql为
SELECT MIN(gbloadtime),MAX(gbloadtime) FROM sigmon WHERE (to_char(gbloadtime,'yyyymmdd') = '20220421' AND gbloadtime IS NOT NULL);
该sql在gbase执行效率较低,并有概率发生执行超时,在express日志上报The query is killed by user!!!;

经分析初步定为达到网络连接超时限制之后,dataworks断开连接报错。
对dataworks下发sql进行优化,修改to_char转换为直接值比较,优化后sql如下:
SELECT MIN(guid),MAX(guid),count(*) FROM cdbpridb.sigmon WHERE gbloadtime >= '202204210000000000' and gbloadtime < '202204220000000000' AND guid IS NOT NULL;
或者
SELECT MIN(guid),MAX(guid),count(*) FROM cdbpridb.sigmon WHERE gbloadtime >= '20220421' and gbloadtime < '20220422' AND guid IS NOT NULL;
在dataworks任务配置中,原配置如下:

修改条件为 gbloadtime > '${bdp.system.bizdate}' and gbloadtime <= ('${bdp.system.bizdate}' + 1 )
修改后,min、max计算这一步,时间缩减至秒级响应。实际下发至gbase的sql为
SELECT MIN(guid),MAX(guid) FROM sigmon WHERE (gbloadtime > '20220421' and gbloadtime <= ('20220421' + 1 ) AND guid IS NOT NULL);
热门帖子
- 12025-12-01浏览数:182759
- 22023-05-09浏览数:25044
- 42023-09-25浏览数:18519
- 52020-05-11浏览数:17526