由于之前在往上沒有找到比較好的關于IDEA配置spark環境，現在分享一下

1、打開IDEA

create New Project

2、創建項目，如下圖所示

?

3、配置maven文件，即使你本地沒有spark、scala、hadoop環境也沒有關系，maven會自動下載及加載到maven里面，只需要你指定相應版本就好，指定好maven之后，自動下載過程需要等待一段時間，如下圖所示：

4、New scala class編寫代碼測試，如下圖所示：

注：附maven文件，你可以和我的maven文件一樣，下次再創建項目工程的時候，還可以配置相同版本的環境maven就不會重新下載啦，它會自動加載已經下載好的環境！

<project?xmlns="http://maven.apache.org/POM/4.0.0"?xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"?xsi:schemaLocation="http://maven.apache.org/POM/4.0.0?http://maven.apache.org/maven-v4_0_0.xsd">??

??<modelVersion>4.0.0</modelVersion>??

??<groupId>com_bupt_ai</groupId>??

??<artifactId>bupt_at</artifactId>??

??<version>1.0-SNAPSHOT</version>??

??<inceptionYear>2008</inceptionYear>??

??

??<!--?聲明子項目公用的配置屬性?-->??

??<properties>??

????<scala.version>2.11.8</scala.version>??

????<hadoop.version>2.6.0</hadoop.version>??

????<spark.version>2.0.1</spark.version>??

????<hbase.version>1.2.0</hbase.version>??

????<kafka.version>0.8.2.2</kafka.version>??

????<jedis.version>2.9.0</jedis.version>??

????<mysql.version>5.1.6</mysql.version>??

????<log4j.version>1.2.17</log4j.version>??

????<slf4j.version>1.7.22</slf4j.version>??

????<pool.version>2.4.2</pool.version>??

????<jackson.version>2.8.6</jackson.version>??

????<net.sf.json.version>2.4</net.sf.json.version>??

????<joda.version>2.9.9</joda.version>??

????<postgresql.version>42.1.4</postgresql.version>??

????<javax.mail>1.6.2</javax.mail>??

????<qconf.version>1.2.2</qconf.version>??

????<xgboost4j.version>0.72</xgboost4j.version>??

????<elasticsearch.version>6.0.1</elasticsearch.version>??

????<druid.version>1.1.10</druid.version>??

??</properties>??

??

??<!--?聲明并引入子項目共有的依賴?-->??

??<dependencies>??

????<!--?引入Spark相關的Jar包?-->??

????<dependency>??

??????<groupId>org.apache.spark</groupId>??

??????<artifactId>spark-core_2.11</artifactId>??

??????<version>${spark.version}</version>??

??????<!--<scope>provided</scope>-->??

????</dependency>??

????<dependency>??

??????<groupId>org.apache.spark</groupId>??

??????<artifactId>spark-sql_2.11</artifactId>??

??????<version>${spark.version}</version>??

??????<!--<scope>provided</scope>-->??

????</dependency>??

????<dependency>??

??????<groupId>org.apache.spark</groupId>??

??????<artifactId>spark-streaming_2.11</artifactId>??

??????<version>${spark.version}</version>??

??????<!--<scope>provided</scope>-->??

????</dependency>??

????<dependency>??

??????<groupId>org.scala-lang</groupId>??

??????<artifactId>scala-library</artifactId>??

??????<version>${scala.version}</version>??

??????<!--<scope>provided</scope>-->??

????</dependency>??

????<dependency>??

??????<groupId>org.apache.spark</groupId>??

??????<artifactId>spark-hive_2.11</artifactId>??

??????<version>${spark.version}</version>??

??????<!--<scope>provided</scope>-->??

????</dependency>??

????<!--?https://mvnrepository.com/artifact/org.apache.hbase/hbase-client?-->??

????<dependency>??

??????<groupId>org.apache.hbase</groupId>??

??????<artifactId>hbase-client</artifactId>??

??????<version>${hbase.version}</version>??

??????<!--<scope>provided</scope>-->??

????</dependency>??

????<!--?https://mvnrepository.com/artifact/org.apache.hbase/hbase-server?-->??

????<dependency>??

??????<groupId>org.apache.hbase</groupId>??

??????<artifactId>hbase-server</artifactId>??

??????<version>${hbase.version}</version>??

??????<!--<scope>provided</scope>-->??

????</dependency>??

????<!--?用來連接Kafka的工具類?-->??

????<dependency>??

??????<groupId>org.apache.kafka</groupId>??

??????<artifactId>kafka-clients</artifactId>??

??????<version>${kafka.version}</version>??

??????<!--<scope>provided</scope>-->??

????</dependency>??

????<dependency>??

??????<groupId>org.apache.spark</groupId>??

??????<artifactId>spark-streaming-kafka-0-8_2.11</artifactId>??

??????<version>${spark.version}</version>??

??????<!--<scope>provided</scope>-->??

????</dependency>??

????<dependency>??

??????<groupId>redis.clients</groupId>??

??????<artifactId>jedis</artifactId>??

??????<version>${jedis.version}</version>??

????</dependency>??

????<dependency>??

??????<groupId>mysql</groupId>??

??????<artifactId>mysql-connector-java</artifactId>??

??????<version>${mysql.version}</version>??

????</dependency>??

????<!--?https://mvnrepository.com/artifact/org.postgresql/postgresql?-->??

????<dependency>??

??????<groupId>org.postgresql</groupId>??

??????<artifactId>postgresql</artifactId>??

??????<version>${postgresql.version}</version>??

????</dependency>??

????<dependency>??

??????<groupId>com.sun.mail</groupId>??

??????<artifactId>javax.mail</artifactId>??

??????<version>${javax.mail}</version>??

????</dependency>??

??

????<!--?https://mvnrepository.com/artifact/ml.dmlc/xgboost4j?-->??

????<dependency>??

??????<groupId>ml.dmlc</groupId>??

??????<artifactId>xgboost4j-spark</artifactId>??

??????<version>${xgboost4j.version}</version>??

????</dependency>??

??

????<!--?https://mvnrepository.com/artifact/org.apache.spark/spark-graphx?-->??

????<dependency>??

??????<groupId>org.apache.spark</groupId>??

??????<artifactId>spark-graphx_2.11</artifactId>??

??????<version>${spark.version}</version>??

????</dependency>??

??

????<!--?https://mvnrepository.com/artifact/org.apache.spark/spark-mllib?-->??

????<dependency>??

??????<groupId>org.apache.spark</groupId>??

??????<artifactId>spark-mllib_2.11</artifactId>??

??????<version>${spark.version}</version>??

????</dependency>??

??

????<!--?https://mvnrepository.com/artifact/org.scalanlp/breeze-viz?-->??

????<dependency>??

??????<groupId>org.scalanlp</groupId>??

??????<artifactId>breeze-viz_2.10</artifactId>??

??????<version>0.12</version>??

????</dependency>??

??

????<!--?https://mvnrepository.com/artifact/org.elasticsearch/elasticsearch-spark-20?-->??

????<dependency>??

??????<groupId>org.elasticsearch</groupId>??

??????<artifactId>elasticsearch-spark-20_2.11</artifactId>??

??????<version>${elasticsearch.version}</version>??

????</dependency>??

??

????<!--?https://mvnrepository.com/artifact/jfree/jfreechart?-->??

????<dependency>??

??????<groupId>jfree</groupId>??

??????<artifactId>jfreechart</artifactId>??

??????<version>1.0.13</version>??

????</dependency>??

??

????<!--?https://mvnrepository.com/artifact/com.alibaba/druid?-->??

????<dependency>??

??????<groupId>com.alibaba</groupId>??

??????<artifactId>druid</artifactId>??

??????<version>${druid.version}</version>??

????</dependency>??

??

????<dependency>??

??????<groupId>com.zaxxer</groupId>??

??????<artifactId>HikariCP</artifactId>??

??????<version>2.7.8</version>??

????</dependency>??

??

??</dependencies>??

??

??<build>??

????<plugins>??

??????<plugin>??

????????<groupId>net.alchim31.maven</groupId>??

????????<artifactId>scala-maven-plugin</artifactId>??

????????<version>3.2.2</version>??

????????<executions>??

??????????<execution>??

????????????<goals>??

??????????????<goal>compile</goal>??

??????????????<goal>testCompile</goal>??

????????????</goals>??

??????????</execution>??

????????</executions>??

??????</plugin>??

????</plugins>??

??

????<pluginManagement>??

??????<plugins>??

????????<plugin>??

??????????<groupId>org.apache.maven.plugins</groupId>??

??????????<artifactId>maven-assembly-plugin</artifactId>??

??????????<version>3.0.0</version>??

??????????<executions>??

????????????<execution>??

??????????????<id>make-assembly</id>??

??????????????<phase>package</phase>??

??????????????<goals>??

????????????????<goal>single</goal>??

??????????????</goals>??

????????????</execution>??

??????????</executions>??

????????</plugin>??

??????</plugins>??

????</pluginManagement>??

??</build>??

??

</project>??

?

總結

以上是生活随笔為你收集整理的Idea配置spark环境及编写scala代码的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。