概述

本文是對[1]的整理和復(fù)現(xiàn)。

環(huán)境

組件	版本
Hadoop	3.1.2
Zookeeper	3.6.0
Mysql	8.0.22-0ubuntu0.20.04.2
Hive	3.1.2

Hive準(zhǔn)備工作

use db1;
create table address?(addressid int, province string, city string) row format delimited fields terminated by ',';
insert into address values(1,'guangdong','guangzhou');
insert into address values(1,'guangdong','guangzhou');
insert into address values(2,'guangdong','shenzhen');
insert into address values(3,'shanxi','xian');
insert into address values(4,'shanxi','hanzhong');
insert into address values(6,'jiangshu','nanjing');

?

注冊GenericUDAF流程

注冊命令(hive/beeline中進(jìn)行)	備注
add jar /home/appleyuchi/桌面/Flink_Code/FLINK讀寫各種數(shù)據(jù)源/Java/target/table_api-1.0-SNAPSHOT.jar;	指明自定義依賴包
create temporary function udf_fieldlength as 'FieldLength';	依賴包中的FieldLength注冊為udf_fieldlength

?

使用GenericUDAF流程

hive命令	實驗結(jié)果	備注
use db1; select * from address;	1?? ?guangdong?? ?guangzhou 2?? ?guangdong?? ?shenzhen 3?? ?shanxi?? ?xian	檢查hive功能是否ok
select province, count(city), udf_fieldlength(city) from address group by province;	guangdong?? ?15?? ?133 shanxi?? ?1?? ?4	使用寫好的udaf

?

完整代碼:

https://gitee.com/appleyuchi/Flink_Code/blob/master/FLINK讀寫各種數(shù)據(jù)源/Java/src/main/java/FieldLength.java

https://gitee.com/appleyuchi/Flink_Code/blob/master/FLINK讀寫各種數(shù)據(jù)源/Java/src/main/java/FieldLengthUDAFEvaluator.java

https://gitee.com/appleyuchi/Flink_Code/blob/master/FLINK讀寫各種數(shù)據(jù)源/Java/src/main/java/FieldLengthAggregationBuffer.java

?

Reference:

[1]hive學(xué)習(xí)筆記之十：用戶自定義聚合函數(shù)(UDAF)

?

總結(jié)

以上是生活随笔為你收集整理的GenericUDAF使用流程记载(转载+自己整理)的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。