2020.12.08號作業題

1.啟動redis

redis-server /usr/local/redis/redis.conf

2.寫代碼

問題1.計算出總的成交量總額（結果保存到redis中）
問題2.計算每個商品分類的成交量（結果保存到redis中）
問題3.計算每個省份的成交總額（結果保存到redis）

object Work {def main(args: Array[String]): Unit = {Logger.getLogger("org").setLevel(Level.WARN)//1.連接redisval pool: JedisPool = new JedisPool(new GenericObjectPoolConfig, "qianfeng01", 6379)val jedis: Jedis = pool.getResourcejedis.auth("123456")//2.拿到數據val spark: SparkSession = SparkSession.builder().appName("work").master("local").getOrCreate()import spark.implicits._val df: DataFrame = spark.read.format("csv").load("data/producer.csv")val df1: DataFrame = df.toDF("id", "ip", "producer", "type", "price","province")df1.printSchema()df1.createTempView("t1")//問題1.計算出總的成交量總額（結果保存到redis中）val sql="""|select sum(cast(price as int)) as sum|from t1|""".stripMargin//spark.sql(sql).show()val frame: DataFrame = spark.sql(sql)val rows: Array[Row] = frame.collect()for(a<-rows){//保存到redisjedis.set("總的成交量總額",a.get(0).toString)}//問題2.計算每個商品分類的成交量（結果保存到redis中）val sql1="""|select producer,sum(cast(price as int)) as sum|from t1|group by producer|""".stripMarginval frame1: DataFrame = spark.sql(sql1)frame1.show()val rows1: Array[Row] = frame1.collect()for(b<-rows1){//保存到redisjedis.set(b.get(0).toString,b.get(1).toString)}//問題3.計算每個省份的成交總額（結果保存到redis）val sql2="""|select province,sum(cast(price as int)) as sum|from t1|group by province|""".stripMarginval frame2: DataFrame = spark.sql(sql2)frame2.show()val rows2: Array[Row] = frame2.collect()for(row<-rows2){//保存到redisjedis.set(row.get(0).toString,row.get(1).toString)}}}

3.參考數據

A0001,202.106.196.115,手機,iphone8,8000,海南省
A0002,202.106.196.116,服裝,Tshirt,450,湖南省
A0003,202.106.196.117,藥品,阿莫西林,40,廣東省
A0004,202.106.196.118,藥品,板藍根,23,湖北省
A0005,202.106.196.119,手機,iphone9,8000,海南省
A0006,202.106.196.120,服裝,Tshirt,320,海南省
A0007,202.106.196.121,藥品,阿莫西林,40,湖北省
A0008,202.106.196.122,藥品,板藍根,23,湖北省
A0009,202.106.196.123,手機,iphone10,8000,湖北省
A0010,202.106.196.124,服裝,Tshirt,450,湖北省
A0011,202.106.196.125,藥品,阿莫西林,40,湖北省
A0012,202.106.196.126,藥品,板藍根,23,廣東省
A0013,202.106.196.127,手機,iphone11,8000,湖南省
A0014,202.106.196.128,服裝,Tshirt,450,湖南省
A0015,202.106.196.129,藥品,阿莫西林,40,湖南省
A0016,202.106.196.130,藥品,板藍根,23,湖南省
A0017,202.106.196.131,手機,iphone12,9999,廣東省
A0018,202.106.196.132,服裝,Tshirt,340,湖南省

4.pom坐標

<dependencies><dependency><groupId>org.apache.spark</groupId><artifactId>spark-streaming_2.11</artifactId><version>2.2.3</version></dependency><dependency><groupId>org.apache.spark</groupId><artifactId>spark-streaming-kafka-0-10_2.11</artifactId><version>2.2.3</version></dependency><dependency><groupId>redis.clients</groupId><artifactId>jedis</artifactId><version>3.0.0</version></dependency><dependency><groupId>org.apache.spark</groupId><artifactId>spark-core_2.11</artifactId><version>2.2.3</version></dependency><dependency><groupId>org.apache.spark</groupId><artifactId>spark-sql_2.11</artifactId><version>2.2.3</version></dependency><dependency><groupId>org.apache.spark</groupId><artifactId>spark-hive_2.11</artifactId><version>2.2.3</version></dependency><dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId><version>5.1.28</version></dependency></dependencies>

總結

以上是生活随笔為你收集整理的SparkSql-redis：将查询到的结果保存到redis中的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。