Spark SQL and DataFrame Guide

啟動Start Shell

[jifeng@feng03 spark-1.3.0-bin-hadoop2.4]$ ./bin/spark-shell Spark assembly has been built with Hive, including Datanucleus jars on classpath log4j:WARN No appenders could be found for logger (org.apache.hadoop.metrics2.lib.MutableMetricsFactory). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 15/03/14 10:54:39 INFO SecurityManager: Changing view acls to: jifeng 15/03/14 10:54:39 INFO SecurityManager: Changing modify acls to: jifeng 15/03/14 10:54:39 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jifeng); users with modify permissions: Set(jifeng) 15/03/14 10:54:39 INFO HttpServer: Starting HTTP Server 15/03/14 10:54:39 INFO Server: jetty-8.y.z-SNAPSHOT 15/03/14 10:54:39 INFO AbstractConnector: Started SocketConnector@0.0.0.0:35435 15/03/14 10:54:39 INFO Utils: Successfully started service 'HTTP class server' on port 35435. Welcome to____ __/ __/__ ___ _____/ /___\ \/ _ \/ _ `/ __/ '_//___/ .__/\_,_/_/ /_/\_\ version 1.3.0/_/Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java 1.7.0_45) Type in expressions to have them evaluated. Type :help for more information. 15/03/14 10:54:42 INFO SparkContext: Running Spark version 1.3.0 15/03/14 10:54:43 INFO SecurityManager: Changing view acls to: jifeng 15/03/14 10:54:43 INFO SecurityManager: Changing modify acls to: jifeng 15/03/14 10:54:43 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(jifeng); users with modify permissions: Set(jifeng) 15/03/14 10:54:43 INFO Slf4jLogger: Slf4jLogger started 15/03/14 10:54:43 INFO Remoting: Starting remoting 15/03/14 10:54:43 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@feng03:56764] 15/03/14 10:54:43 INFO Utils: Successfully started service 'sparkDriver' on port 56764. 15/03/14 10:54:43 INFO SparkEnv: Registering MapOutputTracker 15/03/14 10:54:43 INFO SparkEnv: Registering BlockManagerMaster 15/03/14 10:54:43 INFO DiskBlockManager: Created local directory at /tmp/spark-25749b16-5dcf-4728-8c4d-7ade07ff9d97/blockmgr-db96ec22-6fcc-4ccc-b179-79363fe9928c 15/03/14 10:54:43 INFO MemoryStore: MemoryStore started with capacity 265.4 MB 15/03/14 10:54:43 INFO HttpFileServer: HTTP File server directory is /tmp/spark-625015d6-d9ee-4c68-a33e-40cf8caed924/httpd-87a3be78-c156-41d3-8620-285f3cea0221 15/03/14 10:54:43 INFO HttpServer: Starting HTTP Server 15/03/14 10:54:43 INFO Server: jetty-8.y.z-SNAPSHOT 15/03/14 10:54:43 INFO AbstractConnector: Started SocketConnector@0.0.0.0:39473 15/03/14 10:54:43 INFO Utils: Successfully started service 'HTTP file server' on port 39473. 15/03/14 10:54:43 INFO SparkEnv: Registering OutputCommitCoordinator 15/03/14 10:54:43 INFO Server: jetty-8.y.z-SNAPSHOT 15/03/14 10:54:43 INFO AbstractConnector: Started SelectChannelConnector@0.0.0.0:4040 15/03/14 10:54:43 INFO Utils: Successfully started service 'SparkUI' on port 4040. 15/03/14 10:54:43 INFO SparkUI: Started SparkUI at http://feng03:4040 15/03/14 10:54:44 INFO Executor: Starting executor ID <driver> on host localhost 15/03/14 10:54:44 INFO Executor: Using REPL class URI: http://10.6.3.202:35435 15/03/14 10:54:44 INFO AkkaUtils: Connecting to HeartbeatReceiver: akka.tcp://sparkDriver@feng03:56764/user/HeartbeatReceiver 15/03/14 10:54:44 INFO NettyBlockTransferService: Server created on 42855 15/03/14 10:54:44 INFO BlockManagerMaster: Trying to register BlockManager 15/03/14 10:54:44 INFO BlockManagerMasterActor: Registering block manager localhost:42855 with 265.4 MB RAM, BlockManagerId(<driver>, localhost, 42855) 15/03/14 10:54:44 INFO BlockManagerMaster: Registered BlockManager 15/03/14 10:54:44 INFO SparkILoop: Created spark context.. Spark context available as sc. 15/03/14 10:54:44 INFO SparkILoop: Created sql context (with Hive support).. SQL context available as sqlContext.scala>

Creating DataFrames

val sc: SparkContext // An existing SparkContext. val sqlContext = new org.apache.spark.sql.SQLContext(sc)val df = sqlContext.jsonFile("examples/src/main/resources/people.json")// Displays the content of the DataFrame to stdout df.show() scala> val sqlContext = new org.apache.spark.sql.SQLContext(sc) sqlContext: org.apache.spark.sql.SQLContext = org.apache.spark.sql.SQLContext@692d5cf2scala> val df = sqlContext.jsonFile("examples/src/main/resources/people.json") 15/03/14 11:04:37 INFO MemoryStore: ensureFreeSpace(163705) called with curMem=0, maxMem=278302556 15/03/14 11:04:37 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 159.9 KB, free 265.3 MB) 15/03/14 11:04:37 INFO MemoryStore: ensureFreeSpace(22692) called with curMem=163705, maxMem=278302556 15/03/14 11:04:37 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 22.2 KB, free 265.2 MB) 15/03/14 11:04:37 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on localhost:59982 (size: 22.2 KB, free: 265.4 MB) 15/03/14 11:04:37 INFO BlockManagerMaster: Updated info of block broadcast_0_piece0 15/03/14 11:04:37 INFO SparkContext: Created broadcast 0 from textFile at JSONRelation.scala:98 15/03/14 11:04:37 INFO FileInputFormat: Total input paths to process : 1 15/03/14 11:04:37 INFO SparkContext: Starting job: reduce at JsonRDD.scala:51 15/03/14 11:04:37 INFO DAGScheduler: Got job 0 (reduce at JsonRDD.scala:51) with 2 output partitions (allowLocal=false) 15/03/14 11:04:37 INFO DAGScheduler: Final stage: Stage 0(reduce at JsonRDD.scala:51) 15/03/14 11:04:37 INFO DAGScheduler: Parents of final stage: List() 15/03/14 11:04:37 INFO DAGScheduler: Missing parents: List() 15/03/14 11:04:37 INFO DAGScheduler: Submitting Stage 0 (MapPartitionsRDD[3] at map at JsonRDD.scala:51), which has no missing parents 15/03/14 11:04:37 INFO MemoryStore: ensureFreeSpace(3200) called with curMem=186397, maxMem=278302556 15/03/14 11:04:37 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 3.1 KB, free 265.2 MB) 15/03/14 11:04:37 INFO MemoryStore: ensureFreeSpace(2279) called with curMem=189597, maxMem=278302556 15/03/14 11:04:37 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 2.2 KB, free 265.2 MB) 15/03/14 11:04:37 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on localhost:59982 (size: 2.2 KB, free: 265.4 MB) 15/03/14 11:04:37 INFO BlockManagerMaster: Updated info of block broadcast_1_piece0 15/03/14 11:04:37 INFO SparkContext: Created broadcast 1 from broadcast at DAGScheduler.scala:839 15/03/14 11:04:37 INFO DAGScheduler: Submitting 2 missing tasks from Stage 0 (MapPartitionsRDD[3] at map at JsonRDD.scala:51) 15/03/14 11:04:37 INFO TaskSchedulerImpl: Adding task set 0.0 with 2 tasks 15/03/14 11:04:37 INFO TaskSetManager: Starting task 0.0 in stage 0.0 (TID 0, localhost, PROCESS_LOCAL, 1354 bytes) 15/03/14 11:04:37 INFO TaskSetManager: Starting task 1.0 in stage 0.0 (TID 1, localhost, PROCESS_LOCAL, 1354 bytes) 15/03/14 11:04:37 INFO Executor: Running task 0.0 in stage 0.0 (TID 0) 15/03/14 11:04:37 INFO Executor: Running task 1.0 in stage 0.0 (TID 1) 15/03/14 11:04:37 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37 15/03/14 11:04:37 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36 15/03/14 11:04:37 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id 15/03/14 11:04:37 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id 15/03/14 11:04:37 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap 15/03/14 11:04:37 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition 15/03/14 11:04:37 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id 15/03/14 11:04:37 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 2092 bytes result sent to driver 15/03/14 11:04:37 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 2092 bytes result sent to driver 15/03/14 11:04:37 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 221 ms on localhost (1/2) 15/03/14 11:04:37 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 238 ms on localhost (2/2) 15/03/14 11:04:37 INFO DAGScheduler: Stage 0 (reduce at JsonRDD.scala:51) finished in 0.252 s 15/03/14 11:04:37 INFO TaskSchedulerImpl: Removed TaskSet 0.0, whose tasks have all completed, from pool 15/03/14 11:04:37 INFO DAGScheduler: Job 0 finished: reduce at JsonRDD.scala:51, took 0.361336 s df: org.apache.spark.sql.DataFrame = [age: bigint, name: string]scala> df.show() 15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(81443) called with curMem=191876, maxMem=278302556 15/03/14 11:04:45 INFO MemoryStore: Block broadcast_2 stored as values in memory (estimated size 79.5 KB, free 265.1 MB) 15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(31262) called with curMem=273319, maxMem=278302556 15/03/14 11:04:45 INFO MemoryStore: Block broadcast_2_piece0 stored as bytes in memory (estimated size 30.5 KB, free 265.1 MB) 15/03/14 11:04:45 INFO BlockManagerInfo: Added broadcast_2_piece0 in memory on localhost:59982 (size: 30.5 KB, free: 265.4 MB) 15/03/14 11:04:45 INFO BlockManagerMaster: Updated info of block broadcast_2_piece0 15/03/14 11:04:45 INFO SparkContext: Created broadcast 2 from textFile at JSONRelation.scala:98 15/03/14 11:04:45 INFO FileInputFormat: Total input paths to process : 1 15/03/14 11:04:45 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121 15/03/14 11:04:45 INFO DAGScheduler: Got job 1 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false) 15/03/14 11:04:45 INFO DAGScheduler: Final stage: Stage 1(runJob at SparkPlan.scala:121) 15/03/14 11:04:45 INFO DAGScheduler: Parents of final stage: List() 15/03/14 11:04:45 INFO DAGScheduler: Missing parents: List() 15/03/14 11:04:45 INFO DAGScheduler: Submitting Stage 1 (MapPartitionsRDD[8] at map at SparkPlan.scala:96), which has no missing parents 15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(4048) called with curMem=304581, maxMem=278302556 15/03/14 11:04:45 INFO MemoryStore: Block broadcast_3 stored as values in memory (estimated size 4.0 KB, free 265.1 MB) 15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(2764) called with curMem=308629, maxMem=278302556 15/03/14 11:04:45 INFO MemoryStore: Block broadcast_3_piece0 stored as bytes in memory (estimated size 2.7 KB, free 265.1 MB) 15/03/14 11:04:45 INFO BlockManagerInfo: Added broadcast_3_piece0 in memory on localhost:59982 (size: 2.7 KB, free: 265.4 MB) 15/03/14 11:04:45 INFO BlockManagerMaster: Updated info of block broadcast_3_piece0 15/03/14 11:04:45 INFO SparkContext: Created broadcast 3 from broadcast at DAGScheduler.scala:839 15/03/14 11:04:45 INFO DAGScheduler: Submitting 1 missing tasks from Stage 1 (MapPartitionsRDD[8] at map at SparkPlan.scala:96) 15/03/14 11:04:45 INFO TaskSchedulerImpl: Adding task set 1.0 with 1 tasks 15/03/14 11:04:45 INFO TaskSetManager: Starting task 0.0 in stage 1.0 (TID 2, localhost, PROCESS_LOCAL, 1354 bytes) 15/03/14 11:04:45 INFO Executor: Running task 0.0 in stage 1.0 (TID 2) 15/03/14 11:04:45 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36 15/03/14 11:04:45 INFO Executor: Finished task 0.0 in stage 1.0 (TID 2). 2055 bytes result sent to driver 15/03/14 11:04:45 INFO DAGScheduler: Stage 1 (runJob at SparkPlan.scala:121) finished in 0.026 s 15/03/14 11:04:45 INFO TaskSetManager: Finished task 0.0 in stage 1.0 (TID 2) in 25 ms on localhost (1/1) 15/03/14 11:04:45 INFO TaskSchedulerImpl: Removed TaskSet 1.0, whose tasks have all completed, from pool 15/03/14 11:04:45 INFO DAGScheduler: Job 1 finished: runJob at SparkPlan.scala:121, took 0.038548 s 15/03/14 11:04:45 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121 15/03/14 11:04:45 INFO DAGScheduler: Got job 2 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false) 15/03/14 11:04:45 INFO DAGScheduler: Final stage: Stage 2(runJob at SparkPlan.scala:121) 15/03/14 11:04:45 INFO DAGScheduler: Parents of final stage: List() 15/03/14 11:04:45 INFO DAGScheduler: Missing parents: List() 15/03/14 11:04:45 INFO DAGScheduler: Submitting Stage 2 (MapPartitionsRDD[8] at map at SparkPlan.scala:96), which has no missing parents 15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(4048) called with curMem=311393, maxMem=278302556 15/03/14 11:04:45 INFO MemoryStore: Block broadcast_4 stored as values in memory (estimated size 4.0 KB, free 265.1 MB) 15/03/14 11:04:45 INFO MemoryStore: ensureFreeSpace(2764) called with curMem=315441, maxMem=278302556 15/03/14 11:04:45 INFO MemoryStore: Block broadcast_4_piece0 stored as bytes in memory (estimated size 2.7 KB, free 265.1 MB) 15/03/14 11:04:45 INFO BlockManagerInfo: Added broadcast_4_piece0 in memory on localhost:59982 (size: 2.7 KB, free: 265.4 MB) 15/03/14 11:04:45 INFO BlockManagerMaster: Updated info of block broadcast_4_piece0 15/03/14 11:04:45 INFO SparkContext: Created broadcast 4 from broadcast at DAGScheduler.scala:839 15/03/14 11:04:45 INFO DAGScheduler: Submitting 1 missing tasks from Stage 2 (MapPartitionsRDD[8] at map at SparkPlan.scala:96) 15/03/14 11:04:45 INFO TaskSchedulerImpl: Adding task set 2.0 with 1 tasks 15/03/14 11:04:45 INFO TaskSetManager: Starting task 0.0 in stage 2.0 (TID 3, localhost, PROCESS_LOCAL, 1354 bytes) 15/03/14 11:04:45 INFO Executor: Running task 0.0 in stage 2.0 (TID 3) 15/03/14 11:04:45 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37 15/03/14 11:04:45 INFO Executor: Finished task 0.0 in stage 2.0 (TID 3). 2030 bytes result sent to driver 15/03/14 11:04:45 INFO TaskSetManager: Finished task 0.0 in stage 2.0 (TID 3) in 15 ms on localhost (1/1) 15/03/14 11:04:45 INFO DAGScheduler: Stage 2 (runJob at SparkPlan.scala:121) finished in 0.015 s 15/03/14 11:04:45 INFO TaskSchedulerImpl: Removed TaskSet 2.0, whose tasks have all completed, from pool 15/03/14 11:04:45 INFO DAGScheduler: Job 2 finished: runJob at SparkPlan.scala:121, took 0.025898 s age name null Michael 30 Andy 19 Justin scala>

DataFrame Operations

val sc: SparkContext // An existing SparkContext. val sqlContext = new org.apache.spark.sql.SQLContext(sc)// Create the DataFrame val df = sqlContext.jsonFile("examples/src/main/resources/people.json")// Show the content of the DataFrame df.show() // age name // null Michael // 30 Andy // 19 Justin // Print the schema in a tree format df.printSchema() // root // |-- age: long (nullable = true) // |-- name: string (nullable = true)// Select only the "name" column df.select("name").show() // name // Michael // Andy // Justin // Select everybody, but increment the age by 1 df.select("name", df("age") + 1).show() // name (age + 1) // Michael null // Andy 31 // Justin 20 // Select people older than 21 df.filter(df("name") > 21).show() // age name // 30 Andy// Count people by age df.groupBy("age").count().show() // age count // null 1 // 19 1 // 30 1 scala> df.printSchema() root|-- age: long (nullable = true)|-- name: string (nullable = true)scala> df.select("name").show() 15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(215230) called with curMem=318205, maxMem=278302556 15/03/14 11:06:25 INFO MemoryStore: Block broadcast_5 stored as values in memory (estimated size 210.2 KB, free 264.9 MB) 15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(31262) called with curMem=533435, maxMem=278302556 15/03/14 11:06:25 INFO MemoryStore: Block broadcast_5_piece0 stored as bytes in memory (estimated size 30.5 KB, free 264.9 MB) 15/03/14 11:06:25 INFO BlockManagerInfo: Added broadcast_5_piece0 in memory on localhost:59982 (size: 30.5 KB, free: 265.3 MB) 15/03/14 11:06:25 INFO BlockManagerMaster: Updated info of block broadcast_5_piece0 15/03/14 11:06:25 INFO SparkContext: Created broadcast 5 from textFile at JSONRelation.scala:98 15/03/14 11:06:25 INFO FileInputFormat: Total input paths to process : 1 15/03/14 11:06:25 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121 15/03/14 11:06:25 INFO DAGScheduler: Got job 3 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false) 15/03/14 11:06:25 INFO DAGScheduler: Final stage: Stage 3(runJob at SparkPlan.scala:121) 15/03/14 11:06:25 INFO DAGScheduler: Parents of final stage: List() 15/03/14 11:06:25 INFO DAGScheduler: Missing parents: List() 15/03/14 11:06:25 INFO DAGScheduler: Submitting Stage 3 (MapPartitionsRDD[14] at map at SparkPlan.scala:96), which has no missing parents 15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(5048) called with curMem=564697, maxMem=278302556 15/03/14 11:06:25 INFO MemoryStore: Block broadcast_6 stored as values in memory (estimated size 4.9 KB, free 264.9 MB) 15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(3461) called with curMem=569745, maxMem=278302556 15/03/14 11:06:25 INFO MemoryStore: Block broadcast_6_piece0 stored as bytes in memory (estimated size 3.4 KB, free 264.9 MB) 15/03/14 11:06:25 INFO BlockManagerInfo: Added broadcast_6_piece0 in memory on localhost:59982 (size: 3.4 KB, free: 265.3 MB) 15/03/14 11:06:25 INFO BlockManagerMaster: Updated info of block broadcast_6_piece0 15/03/14 11:06:25 INFO SparkContext: Created broadcast 6 from broadcast at DAGScheduler.scala:839 15/03/14 11:06:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 3 (MapPartitionsRDD[14] at map at SparkPlan.scala:96) 15/03/14 11:06:25 INFO TaskSchedulerImpl: Adding task set 3.0 with 1 tasks 15/03/14 11:06:25 INFO TaskSetManager: Starting task 0.0 in stage 3.0 (TID 4, localhost, PROCESS_LOCAL, 1354 bytes) 15/03/14 11:06:25 INFO Executor: Running task 0.0 in stage 3.0 (TID 4) 15/03/14 11:06:25 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36 15/03/14 11:06:25 INFO Executor: Finished task 0.0 in stage 3.0 (TID 4). 1973 bytes result sent to driver 15/03/14 11:06:25 INFO DAGScheduler: Stage 3 (runJob at SparkPlan.scala:121) finished in 0.021 s 15/03/14 11:06:25 INFO TaskSetManager: Finished task 0.0 in stage 3.0 (TID 4) in 20 ms on localhost (1/1) 15/03/14 11:06:25 INFO TaskSchedulerImpl: Removed TaskSet 3.0, whose tasks have all completed, from pool 15/03/14 11:06:25 INFO DAGScheduler: Job 3 finished: runJob at SparkPlan.scala:121, took 0.032207 s 15/03/14 11:06:25 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121 15/03/14 11:06:25 INFO DAGScheduler: Got job 4 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false) 15/03/14 11:06:25 INFO DAGScheduler: Final stage: Stage 4(runJob at SparkPlan.scala:121) 15/03/14 11:06:25 INFO DAGScheduler: Parents of final stage: List() 15/03/14 11:06:25 INFO DAGScheduler: Missing parents: List() 15/03/14 11:06:25 INFO DAGScheduler: Submitting Stage 4 (MapPartitionsRDD[14] at map at SparkPlan.scala:96), which has no missing parents 15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(5048) called with curMem=573206, maxMem=278302556 15/03/14 11:06:25 INFO MemoryStore: Block broadcast_7 stored as values in memory (estimated size 4.9 KB, free 264.9 MB) 15/03/14 11:06:25 INFO MemoryStore: ensureFreeSpace(3461) called with curMem=578254, maxMem=278302556 15/03/14 11:06:25 INFO MemoryStore: Block broadcast_7_piece0 stored as bytes in memory (estimated size 3.4 KB, free 264.9 MB) 15/03/14 11:06:25 INFO BlockManagerInfo: Added broadcast_7_piece0 in memory on localhost:59982 (size: 3.4 KB, free: 265.3 MB) 15/03/14 11:06:25 INFO BlockManagerMaster: Updated info of block broadcast_7_piece0 15/03/14 11:06:25 INFO SparkContext: Created broadcast 7 from broadcast at DAGScheduler.scala:839 15/03/14 11:06:25 INFO DAGScheduler: Submitting 1 missing tasks from Stage 4 (MapPartitionsRDD[14] at map at SparkPlan.scala:96) 15/03/14 11:06:25 INFO TaskSchedulerImpl: Adding task set 4.0 with 1 tasks 15/03/14 11:06:25 INFO TaskSetManager: Starting task 0.0 in stage 4.0 (TID 5, localhost, PROCESS_LOCAL, 1354 bytes) 15/03/14 11:06:25 INFO Executor: Running task 0.0 in stage 4.0 (TID 5) 15/03/14 11:06:25 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37 15/03/14 11:06:25 INFO Executor: Finished task 0.0 in stage 4.0 (TID 5). 1949 bytes result sent to driver 15/03/14 11:06:25 INFO TaskSetManager: Finished task 0.0 in stage 4.0 (TID 5) in 15 ms on localhost (1/1) 15/03/14 11:06:25 INFO DAGScheduler: Stage 4 (runJob at SparkPlan.scala:121) finished in 0.015 s 15/03/14 11:06:25 INFO TaskSchedulerImpl: Removed TaskSet 4.0, whose tasks have all completed, from pool 15/03/14 11:06:25 INFO DAGScheduler: Job 4 finished: runJob at SparkPlan.scala:121, took 0.025134 s name Michael Andy Justin scala> df.select("name", df("age") + 1).show() <console>:26: error: overloaded method value select with alternatives:(col: String,cols: String*)org.apache.spark.sql.DataFrame <and>(cols: org.apache.spark.sql.Column*)org.apache.spark.sql.DataFramecannot be applied to (String, org.apache.spark.sql.Column)df.select("name", df("age") + 1).show()^scala> df.select("name",df("age") + 1).show() <console>:26: error: overloaded method value select with alternatives:(col: String,cols: String*)org.apache.spark.sql.DataFrame <and>(cols: org.apache.spark.sql.Column*)org.apache.spark.sql.DataFramecannot be applied to (String, org.apache.spark.sql.Column)df.select("name",df("age") + 1).show()^scala> df.filter(df("name") > 21).show() 15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(215230) called with curMem=581715, maxMem=278302556 15/03/14 11:08:51 INFO MemoryStore: Block broadcast_8 stored as values in memory (estimated size 210.2 KB, free 264.6 MB) 15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(31262) called with curMem=796945, maxMem=278302556 15/03/14 11:08:51 INFO MemoryStore: Block broadcast_8_piece0 stored as bytes in memory (estimated size 30.5 KB, free 264.6 MB) 15/03/14 11:08:51 INFO BlockManagerInfo: Added broadcast_8_piece0 in memory on localhost:59982 (size: 30.5 KB, free: 265.3 MB) 15/03/14 11:08:51 INFO BlockManagerMaster: Updated info of block broadcast_8_piece0 15/03/14 11:08:51 INFO SparkContext: Created broadcast 8 from textFile at JSONRelation.scala:98 15/03/14 11:08:51 INFO FileInputFormat: Total input paths to process : 1 15/03/14 11:08:51 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121 15/03/14 11:08:51 INFO DAGScheduler: Got job 5 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false) 15/03/14 11:08:51 INFO DAGScheduler: Final stage: Stage 5(runJob at SparkPlan.scala:121) 15/03/14 11:08:51 INFO DAGScheduler: Parents of final stage: List() 15/03/14 11:08:51 INFO DAGScheduler: Missing parents: List() 15/03/14 11:08:51 INFO DAGScheduler: Submitting Stage 5 (MapPartitionsRDD[20] at map at SparkPlan.scala:96), which has no missing parents 15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(5456) called with curMem=828207, maxMem=278302556 15/03/14 11:08:51 INFO MemoryStore: Block broadcast_9 stored as values in memory (estimated size 5.3 KB, free 264.6 MB) 15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(3728) called with curMem=833663, maxMem=278302556 15/03/14 11:08:51 INFO MemoryStore: Block broadcast_9_piece0 stored as bytes in memory (estimated size 3.6 KB, free 264.6 MB) 15/03/14 11:08:51 INFO BlockManagerInfo: Added broadcast_9_piece0 in memory on localhost:59982 (size: 3.6 KB, free: 265.3 MB) 15/03/14 11:08:51 INFO BlockManagerMaster: Updated info of block broadcast_9_piece0 15/03/14 11:08:51 INFO SparkContext: Created broadcast 9 from broadcast at DAGScheduler.scala:839 15/03/14 11:08:51 INFO DAGScheduler: Submitting 1 missing tasks from Stage 5 (MapPartitionsRDD[20] at map at SparkPlan.scala:96) 15/03/14 11:08:51 INFO TaskSchedulerImpl: Adding task set 5.0 with 1 tasks 15/03/14 11:08:51 INFO TaskSetManager: Starting task 0.0 in stage 5.0 (TID 6, localhost, PROCESS_LOCAL, 1354 bytes) 15/03/14 11:08:51 INFO Executor: Running task 0.0 in stage 5.0 (TID 6) 15/03/14 11:08:51 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36 15/03/14 11:08:51 INFO Executor: Finished task 0.0 in stage 5.0 (TID 6). 1800 bytes result sent to driver 15/03/14 11:08:51 INFO TaskSetManager: Finished task 0.0 in stage 5.0 (TID 6) in 17 ms on localhost (1/1) 15/03/14 11:08:51 INFO DAGScheduler: Stage 5 (runJob at SparkPlan.scala:121) finished in 0.018 s 15/03/14 11:08:51 INFO TaskSchedulerImpl: Removed TaskSet 5.0, whose tasks have all completed, from pool 15/03/14 11:08:51 INFO DAGScheduler: Job 5 finished: runJob at SparkPlan.scala:121, took 0.029776 s 15/03/14 11:08:51 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121 15/03/14 11:08:51 INFO DAGScheduler: Got job 6 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false) 15/03/14 11:08:51 INFO DAGScheduler: Final stage: Stage 6(runJob at SparkPlan.scala:121) 15/03/14 11:08:51 INFO DAGScheduler: Parents of final stage: List() 15/03/14 11:08:51 INFO DAGScheduler: Missing parents: List() 15/03/14 11:08:51 INFO DAGScheduler: Submitting Stage 6 (MapPartitionsRDD[20] at map at SparkPlan.scala:96), which has no missing parents 15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(5456) called with curMem=837391, maxMem=278302556 15/03/14 11:08:51 INFO MemoryStore: Block broadcast_10 stored as values in memory (estimated size 5.3 KB, free 264.6 MB) 15/03/14 11:08:51 INFO MemoryStore: ensureFreeSpace(3728) called with curMem=842847, maxMem=278302556 15/03/14 11:08:51 INFO MemoryStore: Block broadcast_10_piece0 stored as bytes in memory (estimated size 3.6 KB, free 264.6 MB) 15/03/14 11:08:51 INFO BlockManagerInfo: Added broadcast_10_piece0 in memory on localhost:59982 (size: 3.6 KB, free: 265.3 MB) 15/03/14 11:08:51 INFO BlockManagerMaster: Updated info of block broadcast_10_piece0 15/03/14 11:08:51 INFO SparkContext: Created broadcast 10 from broadcast at DAGScheduler.scala:839 15/03/14 11:08:51 INFO DAGScheduler: Submitting 1 missing tasks from Stage 6 (MapPartitionsRDD[20] at map at SparkPlan.scala:96) 15/03/14 11:08:51 INFO TaskSchedulerImpl: Adding task set 6.0 with 1 tasks 15/03/14 11:08:51 INFO TaskSetManager: Starting task 0.0 in stage 6.0 (TID 7, localhost, PROCESS_LOCAL, 1354 bytes) 15/03/14 11:08:51 INFO Executor: Running task 0.0 in stage 6.0 (TID 7) 15/03/14 11:08:51 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37 15/03/14 11:08:51 INFO Executor: Finished task 0.0 in stage 6.0 (TID 7). 1800 bytes result sent to driver 15/03/14 11:08:51 INFO TaskSetManager: Finished task 0.0 in stage 6.0 (TID 7) in 15 ms on localhost (1/1) 15/03/14 11:08:51 INFO TaskSchedulerImpl: Removed TaskSet 6.0, whose tasks have all completed, from pool 15/03/14 11:08:51 INFO DAGScheduler: Stage 6 (runJob at SparkPlan.scala:121) finished in 0.015 s 15/03/14 11:08:51 INFO DAGScheduler: Job 6 finished: runJob at SparkPlan.scala:121, took 0.026000 s age namescala> df.groupBy("age").count().show() 15/03/14 11:15:49 INFO MemoryStore: ensureFreeSpace(215230) called with curMem=846575, maxMem=278302556 15/03/14 11:15:49 INFO MemoryStore: Block broadcast_11 stored as values in memory (estimated size 210.2 KB, free 264.4 MB) 15/03/14 11:15:49 INFO MemoryStore: ensureFreeSpace(31262) called with curMem=1061805, maxMem=278302556 15/03/14 11:15:49 INFO MemoryStore: Block broadcast_11_piece0 stored as bytes in memory (estimated size 30.5 KB, free 264.4 MB) 15/03/14 11:15:49 INFO BlockManagerInfo: Added broadcast_11_piece0 in memory on localhost:59982 (size: 30.5 KB, free: 265.2 MB) 15/03/14 11:15:49 INFO BlockManagerMaster: Updated info of block broadcast_11_piece0 15/03/14 11:15:49 INFO SparkContext: Created broadcast 11 from textFile at JSONRelation.scala:98 15/03/14 11:15:49 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121 15/03/14 11:15:49 INFO FileInputFormat: Total input paths to process : 1 15/03/14 11:15:49 INFO DAGScheduler: Registering RDD 27 (mapPartitions at Exchange.scala:64) 15/03/14 11:15:49 INFO DAGScheduler: Got job 7 (runJob at SparkPlan.scala:121) with 1 output partitions (allowLocal=false) 15/03/14 11:15:49 INFO DAGScheduler: Final stage: Stage 8(runJob at SparkPlan.scala:121) 15/03/14 11:15:49 INFO DAGScheduler: Parents of final stage: List(Stage 7) 15/03/14 11:15:49 INFO DAGScheduler: Missing parents: List(Stage 7) 15/03/14 11:15:49 INFO DAGScheduler: Submitting Stage 7 (MapPartitionsRDD[27] at mapPartitions at Exchange.scala:64), which has no missing parents 15/03/14 11:15:49 INFO MemoryStore: ensureFreeSpace(8952) called with curMem=1093067, maxMem=278302556 15/03/14 11:15:49 INFO MemoryStore: Block broadcast_12 stored as values in memory (estimated size 8.7 KB, free 264.4 MB) 15/03/14 11:15:49 INFO MemoryStore: ensureFreeSpace(5822) called with curMem=1102019, maxMem=278302556 15/03/14 11:15:49 INFO MemoryStore: Block broadcast_12_piece0 stored as bytes in memory (estimated size 5.7 KB, free 264.4 MB) 15/03/14 11:15:49 INFO BlockManagerInfo: Added broadcast_12_piece0 in memory on localhost:59982 (size: 5.7 KB, free: 265.2 MB) 15/03/14 11:15:49 INFO BlockManagerMaster: Updated info of block broadcast_12_piece0 15/03/14 11:15:49 INFO SparkContext: Created broadcast 12 from broadcast at DAGScheduler.scala:839 15/03/14 11:15:49 INFO DAGScheduler: Submitting 2 missing tasks from Stage 7 (MapPartitionsRDD[27] at mapPartitions at Exchange.scala:64) 15/03/14 11:15:49 INFO TaskSchedulerImpl: Adding task set 7.0 with 2 tasks 15/03/14 11:15:49 INFO TaskSetManager: Starting task 0.0 in stage 7.0 (TID 8, localhost, PROCESS_LOCAL, 1343 bytes) 15/03/14 11:15:49 INFO TaskSetManager: Starting task 1.0 in stage 7.0 (TID 9, localhost, PROCESS_LOCAL, 1343 bytes) 15/03/14 11:15:49 INFO Executor: Running task 0.0 in stage 7.0 (TID 8) 15/03/14 11:15:49 INFO Executor: Running task 1.0 in stage 7.0 (TID 9) 15/03/14 11:15:49 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:0+36 15/03/14 11:15:49 INFO HadoopRDD: Input split: file:/home/jifeng/hadoop/spark-1.3.0-bin-hadoop2.4/examples/src/main/resources/people.json:36+37 15/03/14 11:15:50 INFO Executor: Finished task 0.0 in stage 7.0 (TID 8). 2205 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Finished task 1.0 in stage 7.0 (TID 9). 2205 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Finished task 0.0 in stage 7.0 (TID 8) in 851 ms on localhost (1/2) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 1.0 in stage 7.0 (TID 9) in 850 ms on localhost (2/2) 15/03/14 11:15:50 INFO TaskSchedulerImpl: Removed TaskSet 7.0, whose tasks have all completed, from pool 15/03/14 11:15:50 INFO DAGScheduler: Stage 7 (mapPartitions at Exchange.scala:64) finished in 0.852 s 15/03/14 11:15:50 INFO DAGScheduler: looking for newly runnable stages 15/03/14 11:15:50 INFO DAGScheduler: running: Set() 15/03/14 11:15:50 INFO DAGScheduler: waiting: Set(Stage 8) 15/03/14 11:15:50 INFO DAGScheduler: failed: Set() 15/03/14 11:15:50 INFO DAGScheduler: Missing parents for Stage 8: List() 15/03/14 11:15:50 INFO DAGScheduler: Submitting Stage 8 (MapPartitionsRDD[31] at map at SparkPlan.scala:96), which is now runnable 15/03/14 11:15:50 INFO MemoryStore: ensureFreeSpace(9736) called with curMem=1107841, maxMem=278302556 15/03/14 11:15:50 INFO MemoryStore: Block broadcast_13 stored as values in memory (estimated size 9.5 KB, free 264.3 MB) 15/03/14 11:15:50 INFO MemoryStore: ensureFreeSpace(6476) called with curMem=1117577, maxMem=278302556 15/03/14 11:15:50 INFO MemoryStore: Block broadcast_13_piece0 stored as bytes in memory (estimated size 6.3 KB, free 264.3 MB) 15/03/14 11:15:50 INFO BlockManagerInfo: Added broadcast_13_piece0 in memory on localhost:59982 (size: 6.3 KB, free: 265.2 MB) 15/03/14 11:15:50 INFO BlockManagerMaster: Updated info of block broadcast_13_piece0 15/03/14 11:15:50 INFO SparkContext: Created broadcast 13 from broadcast at DAGScheduler.scala:839 15/03/14 11:15:50 INFO DAGScheduler: Submitting 1 missing tasks from Stage 8 (MapPartitionsRDD[31] at map at SparkPlan.scala:96) 15/03/14 11:15:50 INFO TaskSchedulerImpl: Adding task set 8.0 with 1 tasks 15/03/14 11:15:50 INFO TaskSetManager: Starting task 0.0 in stage 8.0 (TID 10, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 0.0 in stage 8.0 (TID 10) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 5 ms 15/03/14 11:15:50 INFO Executor: Finished task 0.0 in stage 8.0 (TID 10). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Finished task 0.0 in stage 8.0 (TID 10) in 43 ms on localhost (1/1) 15/03/14 11:15:50 INFO DAGScheduler: Stage 8 (runJob at SparkPlan.scala:121) finished in 0.043 s 15/03/14 11:15:50 INFO TaskSchedulerImpl: Removed TaskSet 8.0, whose tasks have all completed, from pool 15/03/14 11:15:50 INFO DAGScheduler: Job 7 finished: runJob at SparkPlan.scala:121, took 1.001202 s 15/03/14 11:15:50 INFO SparkContext: Starting job: runJob at SparkPlan.scala:121 15/03/14 11:15:50 INFO MapOutputTrackerMaster: Size of output statuses for shuffle 0 is 167 bytes 15/03/14 11:15:50 INFO DAGScheduler: Got job 8 (runJob at SparkPlan.scala:121) with 199 output partitions (allowLocal=false) 15/03/14 11:15:50 INFO DAGScheduler: Final stage: Stage 10(runJob at SparkPlan.scala:121) 15/03/14 11:15:50 INFO DAGScheduler: Parents of final stage: List(Stage 9) 15/03/14 11:15:50 INFO DAGScheduler: Missing parents: List() 15/03/14 11:15:50 INFO DAGScheduler: Submitting Stage 10 (MapPartitionsRDD[31] at map at SparkPlan.scala:96), which has no missing parents 15/03/14 11:15:50 INFO MemoryStore: ensureFreeSpace(9736) called with curMem=1124053, maxMem=278302556 15/03/14 11:15:50 INFO MemoryStore: Block broadcast_14 stored as values in memory (estimated size 9.5 KB, free 264.3 MB) 15/03/14 11:15:50 INFO MemoryStore: ensureFreeSpace(6476) called with curMem=1133789, maxMem=278302556 15/03/14 11:15:50 INFO MemoryStore: Block broadcast_14_piece0 stored as bytes in memory (estimated size 6.3 KB, free 264.3 MB) 15/03/14 11:15:50 INFO BlockManagerInfo: Added broadcast_14_piece0 in memory on localhost:59982 (size: 6.3 KB, free: 265.2 MB) 15/03/14 11:15:50 INFO BlockManagerMaster: Updated info of block broadcast_14_piece0 15/03/14 11:15:50 INFO SparkContext: Created broadcast 14 from broadcast at DAGScheduler.scala:839 15/03/14 11:15:50 INFO DAGScheduler: Submitting 199 missing tasks from Stage 10 (MapPartitionsRDD[31] at map at SparkPlan.scala:96) 15/03/14 11:15:50 INFO TaskSchedulerImpl: Adding task set 10.0 with 199 tasks 15/03/14 11:15:50 INFO TaskSetManager: Starting task 0.0 in stage 10.0 (TID 11, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 1.0 in stage 10.0 (TID 12, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 2.0 in stage 10.0 (TID 13, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 3.0 in stage 10.0 (TID 14, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 1.0 in stage 10.0 (TID 12) 15/03/14 11:15:50 INFO Executor: Running task 0.0 in stage 10.0 (TID 11) 15/03/14 11:15:50 INFO Executor: Running task 2.0 in stage 10.0 (TID 13) 15/03/14 11:15:50 INFO Executor: Running task 3.0 in stage 10.0 (TID 14) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 1.0 in stage 10.0 (TID 12). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 4.0 in stage 10.0 (TID 15, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms 15/03/14 11:15:50 INFO Executor: Running task 4.0 in stage 10.0 (TID 15) 15/03/14 11:15:50 INFO Executor: Finished task 3.0 in stage 10.0 (TID 14). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 5.0 in stage 10.0 (TID 16, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 5.0 in stage 10.0 (TID 16) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 1.0 in stage 10.0 (TID 12) in 30 ms on localhost (1/199) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 3.0 in stage 10.0 (TID 14) in 30 ms on localhost (2/199) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 4.0 in stage 10.0 (TID 15). 894 bytes result sent to driver 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 22 ms 15/03/14 11:15:50 INFO TaskSetManager: Starting task 6.0 in stage 10.0 (TID 17, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 6.0 in stage 10.0 (TID 17) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 5.0 in stage 10.0 (TID 16). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 7.0 in stage 10.0 (TID 18, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 7.0 in stage 10.0 (TID 18) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 6.0 in stage 10.0 (TID 17). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Finished task 2.0 in stage 10.0 (TID 13). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 8.0 in stage 10.0 (TID 19, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 8.0 in stage 10.0 (TID 19) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 9.0 in stage 10.0 (TID 20, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 9.0 in stage 10.0 (TID 20) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:50 INFO Executor: Finished task 7.0 in stage 10.0 (TID 18). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Finished task 9.0 in stage 10.0 (TID 20). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 10.0 in stage 10.0 (TID 21, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 11.0 in stage 10.0 (TID 22, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 10.0 in stage 10.0 (TID 21) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 6.0 in stage 10.0 (TID 17) in 23 ms on localhost (3/199) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 2.0 in stage 10.0 (TID 13) in 60 ms on localhost (4/199) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 7.0 in stage 10.0 (TID 18) in 19 ms on localhost (5/199) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 9.0 in stage 10.0 (TID 20) in 13 ms on localhost (6/199) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 5.0 in stage 10.0 (TID 16) in 36 ms on localhost (7/199) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 4.0 in stage 10.0 (TID 15) in 38 ms on localhost (8/199) 15/03/14 11:15:50 INFO Executor: Running task 11.0 in stage 10.0 (TID 22) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 10.0 in stage 10.0 (TID 21). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 12.0 in stage 10.0 (TID 23, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 10.0 in stage 10.0 (TID 21) in 13 ms on localhost (9/199) 15/03/14 11:15:50 INFO Executor: Running task 12.0 in stage 10.0 (TID 23) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:50 INFO Executor: Finished task 8.0 in stage 10.0 (TID 19). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 13.0 in stage 10.0 (TID 24, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 8.0 in stage 10.0 (TID 19) in 32 ms on localhost (10/199) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Running task 13.0 in stage 10.0 (TID 24) 15/03/14 11:15:50 INFO Executor: Finished task 12.0 in stage 10.0 (TID 23). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Finished task 11.0 in stage 10.0 (TID 22). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 14.0 in stage 10.0 (TID 25, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 12.0 in stage 10.0 (TID 23) in 12 ms on localhost (11/199) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 15.0 in stage 10.0 (TID 26, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 15.0 in stage 10.0 (TID 26) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 11.0 in stage 10.0 (TID 22) in 27 ms on localhost (12/199) 15/03/14 11:15:50 INFO Executor: Running task 14.0 in stage 10.0 (TID 25) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 13.0 in stage 10.0 (TID 24). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Finished task 15.0 in stage 10.0 (TID 26). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 16.0 in stage 10.0 (TID 27, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 13.0 in stage 10.0 (TID 24) in 16 ms on localhost (13/199) 15/03/14 11:15:50 INFO Executor: Running task 16.0 in stage 10.0 (TID 27) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 17.0 in stage 10.0 (TID 28, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 15.0 in stage 10.0 (TID 26) in 11 ms on localhost (14/199) 15/03/14 11:15:50 INFO Executor: Running task 17.0 in stage 10.0 (TID 28) 15/03/14 11:15:50 INFO Executor: Finished task 14.0 in stage 10.0 (TID 25). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 18.0 in stage 10.0 (TID 29, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 18.0 in stage 10.0 (TID 29) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 14.0 in stage 10.0 (TID 25) in 15 ms on localhost (15/199) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 16.0 in stage 10.0 (TID 27). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Finished task 0.0 in stage 10.0 (TID 11). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 19.0 in stage 10.0 (TID 30, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 20.0 in stage 10.0 (TID 31, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Finished task 17.0 in stage 10.0 (TID 28). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Finished task 0.0 in stage 10.0 (TID 11) in 107 ms on localhost (16/199) 15/03/14 11:15:50 INFO Executor: Running task 20.0 in stage 10.0 (TID 31) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 21.0 in stage 10.0 (TID 32, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 17.0 in stage 10.0 (TID 28) in 14 ms on localhost (17/199) 15/03/14 11:15:50 INFO Executor: Running task 21.0 in stage 10.0 (TID 32) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 20.0 in stage 10.0 (TID 31). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 22.0 in stage 10.0 (TID 33, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 19.0 in stage 10.0 (TID 30) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 16.0 in stage 10.0 (TID 27) in 25 ms on localhost (18/199) 15/03/14 11:15:50 INFO Executor: Finished task 21.0 in stage 10.0 (TID 32). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Finished task 20.0 in stage 10.0 (TID 31) in 13 ms on localhost (19/199) 15/03/14 11:15:50 INFO Executor: Running task 22.0 in stage 10.0 (TID 33) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 23.0 in stage 10.0 (TID 34, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 23.0 in stage 10.0 (TID 34) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 21.0 in stage 10.0 (TID 32) in 13 ms on localhost (20/199) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 23.0 in stage 10.0 (TID 34). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Finished task 22.0 in stage 10.0 (TID 33). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 24.0 in stage 10.0 (TID 35, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 24.0 in stage 10.0 (TID 35) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 25.0 in stage 10.0 (TID 36, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 23.0 in stage 10.0 (TID 34) in 12 ms on localhost (21/199) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 22.0 in stage 10.0 (TID 33) in 13 ms on localhost (22/199) 15/03/14 11:15:50 INFO Executor: Running task 25.0 in stage 10.0 (TID 36) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 24.0 in stage 10.0 (TID 35). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 26.0 in stage 10.0 (TID 37, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 24.0 in stage 10.0 (TID 35) in 12 ms on localhost (23/199) 15/03/14 11:15:50 INFO Executor: Running task 26.0 in stage 10.0 (TID 37) 15/03/14 11:15:50 INFO Executor: Finished task 25.0 in stage 10.0 (TID 36). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 27.0 in stage 10.0 (TID 38, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 27.0 in stage 10.0 (TID 38) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 25.0 in stage 10.0 (TID 36) in 23 ms on localhost (24/199) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 26.0 in stage 10.0 (TID 37). 894 bytes result sent to driver 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO TaskSetManager: Starting task 28.0 in stage 10.0 (TID 39, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO TaskSetManager: Finished task 26.0 in stage 10.0 (TID 37) in 21 ms on localhost (25/199) 15/03/14 11:15:50 INFO Executor: Running task 28.0 in stage 10.0 (TID 39) 15/03/14 11:15:50 INFO Executor: Finished task 27.0 in stage 10.0 (TID 38). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 29.0 in stage 10.0 (TID 40, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 29.0 in stage 10.0 (TID 40) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 27.0 in stage 10.0 (TID 38) in 21 ms on localhost (26/199) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 28.0 in stage 10.0 (TID 39). 894 bytes result sent to driver 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO TaskSetManager: Starting task 30.0 in stage 10.0 (TID 41, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 28.0 in stage 10.0 (TID 39) in 12 ms on localhost (27/199) 15/03/14 11:15:50 INFO Executor: Running task 30.0 in stage 10.0 (TID 41) 15/03/14 11:15:50 INFO Executor: Finished task 29.0 in stage 10.0 (TID 40). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 31.0 in stage 10.0 (TID 42, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 31.0 in stage 10.0 (TID 42) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 29.0 in stage 10.0 (TID 40) in 11 ms on localhost (28/199) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 18.0 in stage 10.0 (TID 29). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 32.0 in stage 10.0 (TID 43, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 32.0 in stage 10.0 (TID 43) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 18.0 in stage 10.0 (TID 29) in 87 ms on localhost (29/199) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:50 INFO Executor: Finished task 32.0 in stage 10.0 (TID 43). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 33.0 in stage 10.0 (TID 44, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 32.0 in stage 10.0 (TID 43) in 12 ms on localhost (30/199) 15/03/14 11:15:50 INFO Executor: Finished task 31.0 in stage 10.0 (TID 42). 894 bytes result sent to driver 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO TaskSetManager: Starting task 34.0 in stage 10.0 (TID 45, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 31.0 in stage 10.0 (TID 42) in 23 ms on localhost (31/199) 15/03/14 11:15:50 INFO Executor: Finished task 30.0 in stage 10.0 (TID 41). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Running task 33.0 in stage 10.0 (TID 44) 15/03/14 11:15:50 INFO Executor: Finished task 19.0 in stage 10.0 (TID 30). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Running task 34.0 in stage 10.0 (TID 45) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:50 INFO Executor: Finished task 33.0 in stage 10.0 (TID 44). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Finished task 34.0 in stage 10.0 (TID 45). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 35.0 in stage 10.0 (TID 46, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 30.0 in stage 10.0 (TID 41) in 41 ms on localhost (32/199) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 36.0 in stage 10.0 (TID 47, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 19.0 in stage 10.0 (TID 30) in 106 ms on localhost (33/199) 15/03/14 11:15:50 INFO Executor: Running task 36.0 in stage 10.0 (TID 47) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 37.0 in stage 10.0 (TID 48, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 38.0 in stage 10.0 (TID 49, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 35.0 in stage 10.0 (TID 46) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO TaskSetManager: Finished task 33.0 in stage 10.0 (TID 44) in 30 ms on localhost (34/199) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 34.0 in stage 10.0 (TID 45) in 29 ms on localhost (35/199) 15/03/14 11:15:50 INFO Executor: Finished task 36.0 in stage 10.0 (TID 47). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 39.0 in stage 10.0 (TID 50, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO Executor: Running task 38.0 in stage 10.0 (TID 49) 15/03/14 11:15:50 INFO Executor: Finished task 35.0 in stage 10.0 (TID 46). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Finished task 36.0 in stage 10.0 (TID 47) in 15 ms on localhost (36/199) 15/03/14 11:15:50 INFO Executor: Running task 39.0 in stage 10.0 (TID 50) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 40.0 in stage 10.0 (TID 51, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 35.0 in stage 10.0 (TID 46) in 29 ms on localhost (37/199) 15/03/14 11:15:50 INFO Executor: Running task 40.0 in stage 10.0 (TID 51) 15/03/14 11:15:50 INFO Executor: Running task 37.0 in stage 10.0 (TID 48) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Finished task 37.0 in stage 10.0 (TID 48). 894 bytes result sent to driver 15/03/14 11:15:50 INFO Executor: Finished task 39.0 in stage 10.0 (TID 50). 894 bytes result sent to driver 15/03/14 11:15:50 INFO TaskSetManager: Starting task 41.0 in stage 10.0 (TID 52, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Starting task 42.0 in stage 10.0 (TID 53, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 37.0 in stage 10.0 (TID 48) in 29 ms on localhost (38/199) 15/03/14 11:15:50 INFO TaskSetManager: Finished task 39.0 in stage 10.0 (TID 50) in 16 ms on localhost (39/199) 15/03/14 11:15:50 INFO Executor: Running task 41.0 in stage 10.0 (TID 52) 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:50 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:50 INFO Executor: Running task 42.0 in stage 10.0 (TID 53) 15/03/14 11:15:50 INFO Executor: Finished task 38.0 in stage 10.0 (TID 49). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 40.0 in stage 10.0 (TID 51). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 43.0 in stage 10.0 (TID 54, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 44.0 in stage 10.0 (TID 55, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 38.0 in stage 10.0 (TID 49) in 793 ms on localhost (40/199) 15/03/14 11:15:51 INFO Executor: Running task 44.0 in stage 10.0 (TID 55) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 40.0 in stage 10.0 (TID 51) in 778 ms on localhost (41/199) 15/03/14 11:15:51 INFO Executor: Running task 43.0 in stage 10.0 (TID 54) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 41.0 in stage 10.0 (TID 52). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 45.0 in stage 10.0 (TID 56, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 41.0 in stage 10.0 (TID 52) in 778 ms on localhost (42/199) 15/03/14 11:15:51 INFO Executor: Running task 45.0 in stage 10.0 (TID 56) 15/03/14 11:15:51 INFO Executor: Finished task 43.0 in stage 10.0 (TID 54). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 42.0 in stage 10.0 (TID 53). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 46.0 in stage 10.0 (TID 57, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 46.0 in stage 10.0 (TID 57) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 43.0 in stage 10.0 (TID 54) in 16 ms on localhost (43/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 47.0 in stage 10.0 (TID 58, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 42.0 in stage 10.0 (TID 53) in 780 ms on localhost (44/199) 15/03/14 11:15:51 INFO Executor: Running task 47.0 in stage 10.0 (TID 58) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 45.0 in stage 10.0 (TID 56). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 46.0 in stage 10.0 (TID 57). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 48.0 in stage 10.0 (TID 59, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 49.0 in stage 10.0 (TID 60, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 45.0 in stage 10.0 (TID 56) in 18 ms on localhost (45/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 46.0 in stage 10.0 (TID 57) in 16 ms on localhost (46/199) 15/03/14 11:15:51 INFO Executor: Running task 49.0 in stage 10.0 (TID 60) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 44.0 in stage 10.0 (TID 55). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Running task 48.0 in stage 10.0 (TID 59) 15/03/14 11:15:51 INFO Executor: Finished task 47.0 in stage 10.0 (TID 58). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 50.0 in stage 10.0 (TID 61, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 44.0 in stage 10.0 (TID 55) in 37 ms on localhost (47/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO Executor: Running task 50.0 in stage 10.0 (TID 61) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 51.0 in stage 10.0 (TID 62, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 51.0 in stage 10.0 (TID 62) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 49.0 in stage 10.0 (TID 60). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 52.0 in stage 10.0 (TID 63, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 52.0 in stage 10.0 (TID 63) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO Executor: Finished task 51.0 in stage 10.0 (TID 62). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO Executor: Finished task 50.0 in stage 10.0 (TID 61). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO TaskSetManager: Finished task 47.0 in stage 10.0 (TID 58) in 40 ms on localhost (48/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 49.0 in stage 10.0 (TID 60) in 28 ms on localhost (49/199) 15/03/14 11:15:51 INFO Executor: Finished task 52.0 in stage 10.0 (TID 63). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 53.0 in stage 10.0 (TID 64, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 54.0 in stage 10.0 (TID 65, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Finished task 48.0 in stage 10.0 (TID 59). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Running task 54.0 in stage 10.0 (TID 65) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 55.0 in stage 10.0 (TID 66, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 53.0 in stage 10.0 (TID 64) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Finished task 52.0 in stage 10.0 (TID 63) in 36 ms on localhost (50/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 50.0 in stage 10.0 (TID 61) in 44 ms on localhost (51/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 51.0 in stage 10.0 (TID 62) in 44 ms on localhost (52/199) 15/03/14 11:15:51 INFO Executor: Running task 55.0 in stage 10.0 (TID 66) 15/03/14 11:15:51 INFO Executor: Finished task 53.0 in stage 10.0 (TID 64). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 54.0 in stage 10.0 (TID 65). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 56.0 in stage 10.0 (TID 67, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 56.0 in stage 10.0 (TID 67) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 48.0 in stage 10.0 (TID 59) in 58 ms on localhost (53/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 57.0 in stage 10.0 (TID 68, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 58.0 in stage 10.0 (TID 69, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 53.0 in stage 10.0 (TID 64) in 33 ms on localhost (54/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 54.0 in stage 10.0 (TID 65) in 35 ms on localhost (55/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO Executor: Running task 58.0 in stage 10.0 (TID 69) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO Executor: Finished task 55.0 in stage 10.0 (TID 66). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Running task 57.0 in stage 10.0 (TID 68) 15/03/14 11:15:51 INFO Executor: Finished task 56.0 in stage 10.0 (TID 67). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO TaskSetManager: Starting task 59.0 in stage 10.0 (TID 70, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Finished task 55.0 in stage 10.0 (TID 66) in 43 ms on localhost (56/199) 15/03/14 11:15:51 INFO Executor: Running task 59.0 in stage 10.0 (TID 70) 15/03/14 11:15:51 INFO Executor: Finished task 58.0 in stage 10.0 (TID 69). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 60.0 in stage 10.0 (TID 71, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 60.0 in stage 10.0 (TID 71) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 61.0 in stage 10.0 (TID 72, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 61.0 in stage 10.0 (TID 72) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO TaskSetManager: Finished task 58.0 in stage 10.0 (TID 69) in 145 ms on localhost (57/199) 15/03/14 11:15:51 INFO Executor: Finished task 59.0 in stage 10.0 (TID 70). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Finished task 56.0 in stage 10.0 (TID 67) in 150 ms on localhost (58/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 62.0 in stage 10.0 (TID 73, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 62.0 in stage 10.0 (TID 73) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 59.0 in stage 10.0 (TID 70) in 138 ms on localhost (59/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms 15/03/14 11:15:51 INFO Executor: Finished task 60.0 in stage 10.0 (TID 71). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 63.0 in stage 10.0 (TID 74, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 63.0 in stage 10.0 (TID 74) 15/03/14 11:15:51 INFO Executor: Finished task 61.0 in stage 10.0 (TID 72). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 64.0 in stage 10.0 (TID 75, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Running task 64.0 in stage 10.0 (TID 75) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 61.0 in stage 10.0 (TID 72) in 35 ms on localhost (60/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 60.0 in stage 10.0 (TID 71) in 37 ms on localhost (61/199) 15/03/14 11:15:51 INFO Executor: Finished task 57.0 in stage 10.0 (TID 68). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 62.0 in stage 10.0 (TID 73). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 65.0 in stage 10.0 (TID 76, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 65.0 in stage 10.0 (TID 76) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 66.0 in stage 10.0 (TID 77, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 57.0 in stage 10.0 (TID 68) in 164 ms on localhost (62/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 62.0 in stage 10.0 (TID 73) in 16 ms on localhost (63/199) 15/03/14 11:15:51 INFO Executor: Running task 66.0 in stage 10.0 (TID 77) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 65.0 in stage 10.0 (TID 76). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 63.0 in stage 10.0 (TID 74). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 64.0 in stage 10.0 (TID 75). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 67.0 in stage 10.0 (TID 78, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO Executor: Running task 67.0 in stage 10.0 (TID 78) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 65.0 in stage 10.0 (TID 76) in 11 ms on localhost (64/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 68.0 in stage 10.0 (TID 79, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 68.0 in stage 10.0 (TID 79) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 63.0 in stage 10.0 (TID 74) in 19 ms on localhost (65/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 69.0 in stage 10.0 (TID 80, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 69.0 in stage 10.0 (TID 80) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 64.0 in stage 10.0 (TID 75) in 17 ms on localhost (66/199) 15/03/14 11:15:51 INFO Executor: Finished task 66.0 in stage 10.0 (TID 77). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 70.0 in stage 10.0 (TID 81, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 66.0 in stage 10.0 (TID 77) in 15 ms on localhost (67/199) 15/03/14 11:15:51 INFO Executor: Running task 70.0 in stage 10.0 (TID 81) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 69.0 in stage 10.0 (TID 80). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 71.0 in stage 10.0 (TID 82, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 71.0 in stage 10.0 (TID 82) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Finished task 69.0 in stage 10.0 (TID 80) in 11 ms on localhost (68/199) 15/03/14 11:15:51 INFO Executor: Finished task 67.0 in stage 10.0 (TID 78). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 72.0 in stage 10.0 (TID 83, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 72.0 in stage 10.0 (TID 83) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 67.0 in stage 10.0 (TID 78) in 17 ms on localhost (69/199) 15/03/14 11:15:51 INFO Executor: Finished task 70.0 in stage 10.0 (TID 81). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 73.0 in stage 10.0 (TID 84, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 73.0 in stage 10.0 (TID 84) 15/03/14 11:15:51 INFO Executor: Finished task 68.0 in stage 10.0 (TID 79). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 74.0 in stage 10.0 (TID 85, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 68.0 in stage 10.0 (TID 79) in 18 ms on localhost (70/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 70.0 in stage 10.0 (TID 81) in 15 ms on localhost (71/199) 15/03/14 11:15:51 INFO Executor: Running task 74.0 in stage 10.0 (TID 85) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO Executor: Finished task 71.0 in stage 10.0 (TID 82). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 75.0 in stage 10.0 (TID 86, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 71.0 in stage 10.0 (TID 82) in 13 ms on localhost (72/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Running task 75.0 in stage 10.0 (TID 86) 15/03/14 11:15:51 INFO Executor: Finished task 72.0 in stage 10.0 (TID 83). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 73.0 in stage 10.0 (TID 84). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 74.0 in stage 10.0 (TID 85). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 76.0 in stage 10.0 (TID 87, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 76.0 in stage 10.0 (TID 87) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 77.0 in stage 10.0 (TID 88, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 74.0 in stage 10.0 (TID 85) in 121 ms on localhost (73/199) 15/03/14 11:15:51 INFO Executor: Running task 77.0 in stage 10.0 (TID 88) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 73.0 in stage 10.0 (TID 84) in 123 ms on localhost (74/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO Executor: Finished task 75.0 in stage 10.0 (TID 86). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 78.0 in stage 10.0 (TID 89, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 79.0 in stage 10.0 (TID 90, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 79.0 in stage 10.0 (TID 90) 15/03/14 11:15:51 INFO Executor: Running task 78.0 in stage 10.0 (TID 89) 15/03/14 11:15:51 INFO Executor: Finished task 76.0 in stage 10.0 (TID 87). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 77.0 in stage 10.0 (TID 88). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Finished task 72.0 in stage 10.0 (TID 83) in 143 ms on localhost (75/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 75.0 in stage 10.0 (TID 86) in 133 ms on localhost (76/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 80.0 in stage 10.0 (TID 91, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 81.0 in stage 10.0 (TID 92, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 80.0 in stage 10.0 (TID 91) 15/03/14 11:15:51 INFO Executor: Running task 81.0 in stage 10.0 (TID 92) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 76.0 in stage 10.0 (TID 87) in 27 ms on localhost (77/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 77.0 in stage 10.0 (TID 88) in 26 ms on localhost (78/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 78.0 in stage 10.0 (TID 89). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 82.0 in stage 10.0 (TID 93, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Finished task 79.0 in stage 10.0 (TID 90). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Running task 82.0 in stage 10.0 (TID 93) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 78.0 in stage 10.0 (TID 89) in 16 ms on localhost (79/199) 15/03/14 11:15:51 INFO Executor: Finished task 80.0 in stage 10.0 (TID 91). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 83.0 in stage 10.0 (TID 94, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 83.0 in stage 10.0 (TID 94) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 84.0 in stage 10.0 (TID 95, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 79.0 in stage 10.0 (TID 90) in 19 ms on localhost (80/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 80.0 in stage 10.0 (TID 91) in 13 ms on localhost (81/199) 15/03/14 11:15:51 INFO Executor: Running task 84.0 in stage 10.0 (TID 95) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO Executor: Finished task 82.0 in stage 10.0 (TID 93). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 85.0 in stage 10.0 (TID 96, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 85.0 in stage 10.0 (TID 96) 15/03/14 11:15:51 INFO Executor: Finished task 83.0 in stage 10.0 (TID 94). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 86.0 in stage 10.0 (TID 97, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Finished task 84.0 in stage 10.0 (TID 95). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 87.0 in stage 10.0 (TID 98, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 86.0 in stage 10.0 (TID 97) 15/03/14 11:15:51 INFO Executor: Running task 87.0 in stage 10.0 (TID 98) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 82.0 in stage 10.0 (TID 93) in 17 ms on localhost (82/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 84.0 in stage 10.0 (TID 95) in 16 ms on localhost (83/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 83.0 in stage 10.0 (TID 94) in 16 ms on localhost (84/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 81.0 in stage 10.0 (TID 92). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 87.0 in stage 10.0 (TID 98). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 88.0 in stage 10.0 (TID 99, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 89.0 in stage 10.0 (TID 100, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 81.0 in stage 10.0 (TID 92) in 39 ms on localhost (85/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 87.0 in stage 10.0 (TID 98) in 16 ms on localhost (86/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 85.0 in stage 10.0 (TID 96). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 86.0 in stage 10.0 (TID 97). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Running task 89.0 in stage 10.0 (TID 100) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 90.0 in stage 10.0 (TID 101, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 91.0 in stage 10.0 (TID 102, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 91.0 in stage 10.0 (TID 102) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 85.0 in stage 10.0 (TID 96) in 27 ms on localhost (87/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 86.0 in stage 10.0 (TID 97) in 26 ms on localhost (88/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 89.0 in stage 10.0 (TID 100). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Running task 90.0 in stage 10.0 (TID 101) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 92.0 in stage 10.0 (TID 103, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 88.0 in stage 10.0 (TID 99) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 89.0 in stage 10.0 (TID 100) in 18 ms on localhost (89/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Running task 92.0 in stage 10.0 (TID 103) 15/03/14 11:15:51 INFO Executor: Finished task 88.0 in stage 10.0 (TID 99). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 93.0 in stage 10.0 (TID 104, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 88.0 in stage 10.0 (TID 99) in 29 ms on localhost (90/199) 15/03/14 11:15:51 INFO Executor: Running task 93.0 in stage 10.0 (TID 104) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 91.0 in stage 10.0 (TID 102). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 92.0 in stage 10.0 (TID 103). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 11 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 94.0 in stage 10.0 (TID 105, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 95.0 in stage 10.0 (TID 106, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 91.0 in stage 10.0 (TID 102) in 30 ms on localhost (91/199) 15/03/14 11:15:51 INFO Executor: Running task 95.0 in stage 10.0 (TID 106) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 92.0 in stage 10.0 (TID 103) in 20 ms on localhost (92/199) 15/03/14 11:15:51 INFO Executor: Finished task 90.0 in stage 10.0 (TID 101). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 93.0 in stage 10.0 (TID 104). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Running task 94.0 in stage 10.0 (TID 105) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 96.0 in stage 10.0 (TID 107, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 97.0 in stage 10.0 (TID 108, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 90.0 in stage 10.0 (TID 101) in 35 ms on localhost (93/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 93.0 in stage 10.0 (TID 104) in 15 ms on localhost (94/199) 15/03/14 11:15:51 INFO Executor: Running task 96.0 in stage 10.0 (TID 107) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 95.0 in stage 10.0 (TID 106). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 94.0 in stage 10.0 (TID 105). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Running task 97.0 in stage 10.0 (TID 108) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 98.0 in stage 10.0 (TID 109, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 99.0 in stage 10.0 (TID 110, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 99.0 in stage 10.0 (TID 110) 15/03/14 11:15:51 INFO Executor: Finished task 96.0 in stage 10.0 (TID 107). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Finished task 95.0 in stage 10.0 (TID 106) in 18 ms on localhost (95/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 94.0 in stage 10.0 (TID 105) in 19 ms on localhost (96/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 100.0 in stage 10.0 (TID 111, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 98.0 in stage 10.0 (TID 109) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO Executor: Running task 100.0 in stage 10.0 (TID 111) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 8 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 97.0 in stage 10.0 (TID 108). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 100.0 in stage 10.0 (TID 111). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Finished task 96.0 in stage 10.0 (TID 107) in 18 ms on localhost (97/199) 15/03/14 11:15:51 INFO Executor: Finished task 98.0 in stage 10.0 (TID 109). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 101.0 in stage 10.0 (TID 112, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 101.0 in stage 10.0 (TID 112) 15/03/14 11:15:51 INFO Executor: Finished task 99.0 in stage 10.0 (TID 110). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Finished task 97.0 in stage 10.0 (TID 108) in 30 ms on localhost (98/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 102.0 in stage 10.0 (TID 113, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 100.0 in stage 10.0 (TID 111) in 14 ms on localhost (99/199) 15/03/14 11:15:51 INFO Executor: Running task 102.0 in stage 10.0 (TID 113) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 103.0 in stage 10.0 (TID 114, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 98.0 in stage 10.0 (TID 109) in 22 ms on localhost (100/199) 15/03/14 11:15:51 INFO Executor: Running task 103.0 in stage 10.0 (TID 114) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 104.0 in stage 10.0 (TID 115, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 104.0 in stage 10.0 (TID 115) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 101.0 in stage 10.0 (TID 112). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 105.0 in stage 10.0 (TID 116, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 105.0 in stage 10.0 (TID 116) 15/03/14 11:15:51 INFO Executor: Finished task 102.0 in stage 10.0 (TID 113). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 106.0 in stage 10.0 (TID 117, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Finished task 103.0 in stage 10.0 (TID 114). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Finished task 102.0 in stage 10.0 (TID 113) in 12 ms on localhost (101/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 101.0 in stage 10.0 (TID 112) in 13 ms on localhost (102/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 99.0 in stage 10.0 (TID 110) in 33 ms on localhost (103/199) 15/03/14 11:15:51 INFO Executor: Running task 106.0 in stage 10.0 (TID 117) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 107.0 in stage 10.0 (TID 118, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 103.0 in stage 10.0 (TID 114) in 13 ms on localhost (104/199) 15/03/14 11:15:51 INFO Executor: Running task 107.0 in stage 10.0 (TID 118) 15/03/14 11:15:51 INFO Executor: Finished task 104.0 in stage 10.0 (TID 115). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 108.0 in stage 10.0 (TID 119, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 108.0 in stage 10.0 (TID 119) 15/03/14 11:15:51 INFO Executor: Finished task 105.0 in stage 10.0 (TID 116). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 109.0 in stage 10.0 (TID 120, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 109.0 in stage 10.0 (TID 120) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO Executor: Finished task 106.0 in stage 10.0 (TID 117). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Finished task 104.0 in stage 10.0 (TID 115) in 18 ms on localhost (105/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 39 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 110.0 in stage 10.0 (TID 121, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 35 ms 15/03/14 11:15:51 INFO Executor: Running task 110.0 in stage 10.0 (TID 121) 15/03/14 11:15:51 INFO Executor: Finished task 107.0 in stage 10.0 (TID 118). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 108.0 in stage 10.0 (TID 119). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Finished task 106.0 in stage 10.0 (TID 117) in 51 ms on localhost (106/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 105.0 in stage 10.0 (TID 116) in 55 ms on localhost (107/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 111.0 in stage 10.0 (TID 122, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 107.0 in stage 10.0 (TID 118) in 52 ms on localhost (108/199) 15/03/14 11:15:51 INFO Executor: Running task 111.0 in stage 10.0 (TID 122) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 109.0 in stage 10.0 (TID 120). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 112.0 in stage 10.0 (TID 123, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 112.0 in stage 10.0 (TID 123) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 108.0 in stage 10.0 (TID 119) in 55 ms on localhost (109/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 113.0 in stage 10.0 (TID 124, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Running task 113.0 in stage 10.0 (TID 124) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 109.0 in stage 10.0 (TID 120) in 54 ms on localhost (110/199) 15/03/14 11:15:51 INFO Executor: Finished task 111.0 in stage 10.0 (TID 122). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 114.0 in stage 10.0 (TID 125, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 111.0 in stage 10.0 (TID 122) in 12 ms on localhost (111/199) 15/03/14 11:15:51 INFO Executor: Running task 114.0 in stage 10.0 (TID 125) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 112.0 in stage 10.0 (TID 123). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 113.0 in stage 10.0 (TID 124). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO TaskSetManager: Starting task 115.0 in stage 10.0 (TID 126, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 115.0 in stage 10.0 (TID 126) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 116.0 in stage 10.0 (TID 127, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 112.0 in stage 10.0 (TID 123) in 15 ms on localhost (112/199) 15/03/14 11:15:51 INFO Executor: Running task 116.0 in stage 10.0 (TID 127) 15/03/14 11:15:51 INFO Executor: Finished task 114.0 in stage 10.0 (TID 125). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 117.0 in stage 10.0 (TID 128, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 114.0 in stage 10.0 (TID 125) in 12 ms on localhost (113/199) 15/03/14 11:15:51 INFO Executor: Running task 117.0 in stage 10.0 (TID 128) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 115.0 in stage 10.0 (TID 126). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 116.0 in stage 10.0 (TID 127). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 118.0 in stage 10.0 (TID 129, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 113.0 in stage 10.0 (TID 124) in 24 ms on localhost (114/199) 15/03/14 11:15:51 INFO Executor: Running task 118.0 in stage 10.0 (TID 129) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 115.0 in stage 10.0 (TID 126) in 14 ms on localhost (115/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 119.0 in stage 10.0 (TID 130, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 119.0 in stage 10.0 (TID 130) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 116.0 in stage 10.0 (TID 127) in 15 ms on localhost (116/199) 15/03/14 11:15:51 INFO Executor: Finished task 117.0 in stage 10.0 (TID 128). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 120.0 in stage 10.0 (TID 131, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 120.0 in stage 10.0 (TID 131) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 117.0 in stage 10.0 (TID 128) in 15 ms on localhost (117/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO Executor: Finished task 118.0 in stage 10.0 (TID 129). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 119.0 in stage 10.0 (TID 130). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 121.0 in stage 10.0 (TID 132, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 121.0 in stage 10.0 (TID 132) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 118.0 in stage 10.0 (TID 129) in 13 ms on localhost (118/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 122.0 in stage 10.0 (TID 133, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 122.0 in stage 10.0 (TID 133) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 119.0 in stage 10.0 (TID 130) in 12 ms on localhost (119/199) 15/03/14 11:15:51 INFO Executor: Finished task 120.0 in stage 10.0 (TID 131). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 123.0 in stage 10.0 (TID 134, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 123.0 in stage 10.0 (TID 134) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 120.0 in stage 10.0 (TID 131) in 10 ms on localhost (120/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 122.0 in stage 10.0 (TID 133). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 124.0 in stage 10.0 (TID 135, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 124.0 in stage 10.0 (TID 135) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 122.0 in stage 10.0 (TID 133) in 10 ms on localhost (121/199) 15/03/14 11:15:51 INFO Executor: Finished task 121.0 in stage 10.0 (TID 132). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 125.0 in stage 10.0 (TID 136, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 125.0 in stage 10.0 (TID 136) 15/03/14 11:15:51 INFO Executor: Finished task 123.0 in stage 10.0 (TID 134). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 126.0 in stage 10.0 (TID 137, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 121.0 in stage 10.0 (TID 132) in 14 ms on localhost (122/199) 15/03/14 11:15:51 INFO Executor: Running task 126.0 in stage 10.0 (TID 137) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 123.0 in stage 10.0 (TID 134) in 12 ms on localhost (123/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 124.0 in stage 10.0 (TID 135). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 110.0 in stage 10.0 (TID 121). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 125.0 in stage 10.0 (TID 136). 894 bytes result sent to driver 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO TaskSetManager: Starting task 127.0 in stage 10.0 (TID 138, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 110 ms 15/03/14 11:15:51 INFO Executor: Running task 127.0 in stage 10.0 (TID 138) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 128.0 in stage 10.0 (TID 139, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 128.0 in stage 10.0 (TID 139) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 129.0 in stage 10.0 (TID 140, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 129.0 in stage 10.0 (TID 140) 15/03/14 11:15:51 INFO Executor: Finished task 126.0 in stage 10.0 (TID 137). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 130.0 in stage 10.0 (TID 141, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 110.0 in stage 10.0 (TID 121) in 186 ms on localhost (124/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 124.0 in stage 10.0 (TID 135) in 127 ms on localhost (125/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 125.0 in stage 10.0 (TID 136) in 126 ms on localhost (126/199) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 126.0 in stage 10.0 (TID 137) in 127 ms on localhost (127/199) 15/03/14 11:15:51 INFO Executor: Running task 130.0 in stage 10.0 (TID 141) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 129.0 in stage 10.0 (TID 140). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 131.0 in stage 10.0 (TID 142, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 129.0 in stage 10.0 (TID 140) in 14 ms on localhost (128/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO Executor: Finished task 130.0 in stage 10.0 (TID 141). 894 bytes result sent to driver 15/03/14 11:15:51 INFO Executor: Finished task 128.0 in stage 10.0 (TID 139). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 132.0 in stage 10.0 (TID 143, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 130.0 in stage 10.0 (TID 141) in 14 ms on localhost (129/199) 15/03/14 11:15:51 INFO TaskSetManager: Starting task 133.0 in stage 10.0 (TID 144, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO Executor: Running task 133.0 in stage 10.0 (TID 144) 15/03/14 11:15:51 INFO Executor: Running task 131.0 in stage 10.0 (TID 142) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 128.0 in stage 10.0 (TID 139) in 21 ms on localhost (130/199) 15/03/14 11:15:51 INFO Executor: Running task 132.0 in stage 10.0 (TID 143) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO Executor: Finished task 131.0 in stage 10.0 (TID 142). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 134.0 in stage 10.0 (TID 145, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 134.0 in stage 10.0 (TID 145) 15/03/14 11:15:51 INFO Executor: Finished task 132.0 in stage 10.0 (TID 143). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 135.0 in stage 10.0 (TID 146, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 131.0 in stage 10.0 (TID 142) in 22 ms on localhost (131/199) 15/03/14 11:15:51 INFO Executor: Running task 135.0 in stage 10.0 (TID 146) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 132.0 in stage 10.0 (TID 143) in 18 ms on localhost (132/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO Executor: Finished task 134.0 in stage 10.0 (TID 145). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 136.0 in stage 10.0 (TID 147, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 136.0 in stage 10.0 (TID 147) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 134.0 in stage 10.0 (TID 145) in 13 ms on localhost (133/199) 15/03/14 11:15:51 INFO Executor: Finished task 135.0 in stage 10.0 (TID 146). 894 bytes result sent to driver 15/03/14 11:15:51 INFO TaskSetManager: Starting task 137.0 in stage 10.0 (TID 148, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:51 INFO Executor: Running task 137.0 in stage 10.0 (TID 148) 15/03/14 11:15:51 INFO TaskSetManager: Finished task 135.0 in stage 10.0 (TID 146) in 15 ms on localhost (134/199) 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:51 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Finished task 137.0 in stage 10.0 (TID 148). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Finished task 133.0 in stage 10.0 (TID 144). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Finished task 136.0 in stage 10.0 (TID 147). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Finished task 127.0 in stage 10.0 (TID 138). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 138.0 in stage 10.0 (TID 149, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 138.0 in stage 10.0 (TID 149) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 139.0 in stage 10.0 (TID 150, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 139.0 in stage 10.0 (TID 150) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 140.0 in stage 10.0 (TID 151, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 140.0 in stage 10.0 (TID 151) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 141.0 in stage 10.0 (TID 152, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 137.0 in stage 10.0 (TID 148) in 220 ms on localhost (135/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 133.0 in stage 10.0 (TID 144) in 250 ms on localhost (136/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 136.0 in stage 10.0 (TID 147) in 228 ms on localhost (137/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 127.0 in stage 10.0 (TID 138) in 278 ms on localhost (138/199) 15/03/14 11:15:52 INFO Executor: Running task 141.0 in stage 10.0 (TID 152) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Finished task 140.0 in stage 10.0 (TID 151). 894 bytes result sent to driver 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO Executor: Finished task 138.0 in stage 10.0 (TID 149). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 142.0 in stage 10.0 (TID 153, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO TaskSetManager: Starting task 143.0 in stage 10.0 (TID 154, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 143.0 in stage 10.0 (TID 154) 15/03/14 11:15:52 INFO Executor: Running task 142.0 in stage 10.0 (TID 153) 15/03/14 11:15:52 INFO Executor: Finished task 141.0 in stage 10.0 (TID 152). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Finished task 140.0 in stage 10.0 (TID 151) in 22 ms on localhost (139/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 138.0 in stage 10.0 (TID 149) in 27 ms on localhost (140/199) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 144.0 in stage 10.0 (TID 155, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO TaskSetManager: Finished task 141.0 in stage 10.0 (TID 152) in 25 ms on localhost (141/199) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Finished task 143.0 in stage 10.0 (TID 154). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Finished task 142.0 in stage 10.0 (TID 153). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Finished task 139.0 in stage 10.0 (TID 150). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 145.0 in stage 10.0 (TID 156, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 144.0 in stage 10.0 (TID 155) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 146.0 in stage 10.0 (TID 157, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 147.0 in stage 10.0 (TID 158, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 147.0 in stage 10.0 (TID 158) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 143.0 in stage 10.0 (TID 154) in 14 ms on localhost (142/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 142.0 in stage 10.0 (TID 153) in 16 ms on localhost (143/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 139.0 in stage 10.0 (TID 150) in 36 ms on localhost (144/199) 15/03/14 11:15:52 INFO Executor: Running task 146.0 in stage 10.0 (TID 157) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:52 INFO Executor: Finished task 144.0 in stage 10.0 (TID 155). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Running task 145.0 in stage 10.0 (TID 156) 15/03/14 11:15:52 INFO Executor: Finished task 146.0 in stage 10.0 (TID 157). 894 bytes result sent to driver 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Finished task 147.0 in stage 10.0 (TID 158). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Finished task 145.0 in stage 10.0 (TID 156). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 148.0 in stage 10.0 (TID 159, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 148.0 in stage 10.0 (TID 159) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 149.0 in stage 10.0 (TID 160, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 149.0 in stage 10.0 (TID 160) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 150.0 in stage 10.0 (TID 161, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 150.0 in stage 10.0 (TID 161) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 151.0 in stage 10.0 (TID 162, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 151.0 in stage 10.0 (TID 162) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 144.0 in stage 10.0 (TID 155) in 33 ms on localhost (145/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 146.0 in stage 10.0 (TID 157) in 28 ms on localhost (146/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 147.0 in stage 10.0 (TID 158) in 28 ms on localhost (147/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 145.0 in stage 10.0 (TID 156) in 30 ms on localhost (148/199) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Finished task 150.0 in stage 10.0 (TID 161). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 152.0 in stage 10.0 (TID 163, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Finished task 151.0 in stage 10.0 (TID 162). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Finished task 150.0 in stage 10.0 (TID 161) in 11 ms on localhost (149/199) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO Executor: Running task 152.0 in stage 10.0 (TID 163) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 153.0 in stage 10.0 (TID 164, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 151.0 in stage 10.0 (TID 162) in 18 ms on localhost (150/199) 15/03/14 11:15:52 INFO Executor: Running task 153.0 in stage 10.0 (TID 164) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 4 ms 15/03/14 11:15:52 INFO Executor: Finished task 148.0 in stage 10.0 (TID 159). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 154.0 in stage 10.0 (TID 165, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:52 INFO TaskSetManager: Finished task 148.0 in stage 10.0 (TID 159) in 28 ms on localhost (151/199) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Running task 154.0 in stage 10.0 (TID 165) 15/03/14 11:15:52 INFO Executor: Finished task 152.0 in stage 10.0 (TID 163). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Finished task 153.0 in stage 10.0 (TID 164). 894 bytes result sent to driver 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:52 INFO TaskSetManager: Starting task 155.0 in stage 10.0 (TID 166, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 156.0 in stage 10.0 (TID 167, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 153.0 in stage 10.0 (TID 164) in 12 ms on localhost (152/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 152.0 in stage 10.0 (TID 163) in 22 ms on localhost (153/199) 15/03/14 11:15:52 INFO Executor: Finished task 149.0 in stage 10.0 (TID 160). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Running task 156.0 in stage 10.0 (TID 167) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 157.0 in stage 10.0 (TID 168, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 149.0 in stage 10.0 (TID 160) in 35 ms on localhost (154/199) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Running task 157.0 in stage 10.0 (TID 168) 15/03/14 11:15:52 INFO Executor: Finished task 154.0 in stage 10.0 (TID 165). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 158.0 in stage 10.0 (TID 169, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO TaskSetManager: Finished task 154.0 in stage 10.0 (TID 165) in 22 ms on localhost (155/199) 15/03/14 11:15:52 INFO Executor: Running task 158.0 in stage 10.0 (TID 169) 15/03/14 11:15:52 INFO Executor: Finished task 157.0 in stage 10.0 (TID 168). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 159.0 in stage 10.0 (TID 170, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 157.0 in stage 10.0 (TID 168) in 18 ms on localhost (156/199) 15/03/14 11:15:52 INFO Executor: Running task 159.0 in stage 10.0 (TID 170) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Running task 155.0 in stage 10.0 (TID 166) 15/03/14 11:15:52 INFO Executor: Finished task 158.0 in stage 10.0 (TID 169). 894 bytes result sent to driver 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 20 ms 15/03/14 11:15:52 INFO Executor: Finished task 159.0 in stage 10.0 (TID 170). 894 bytes result sent to driver 15/03/14 11:15:52 INFO Executor: Finished task 156.0 in stage 10.0 (TID 167). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 160.0 in stage 10.0 (TID 171, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 160.0 in stage 10.0 (TID 171) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 161.0 in stage 10.0 (TID 172, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO TaskSetManager: Starting task 162.0 in stage 10.0 (TID 173, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Running task 161.0 in stage 10.0 (TID 172) 15/03/14 11:15:52 INFO Executor: Finished task 155.0 in stage 10.0 (TID 166). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 163.0 in stage 10.0 (TID 174, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 163.0 in stage 10.0 (TID 174) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:52 INFO TaskSetManager: Finished task 156.0 in stage 10.0 (TID 167) in 40 ms on localhost (157/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 155.0 in stage 10.0 (TID 166) in 42 ms on localhost (158/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 158.0 in stage 10.0 (TID 169) in 26 ms on localhost (159/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 159.0 in stage 10.0 (TID 170) in 22 ms on localhost (160/199) 15/03/14 11:15:52 INFO Executor: Running task 162.0 in stage 10.0 (TID 173) 15/03/14 11:15:52 INFO Executor: Finished task 161.0 in stage 10.0 (TID 172). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 164.0 in stage 10.0 (TID 175, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO TaskSetManager: Finished task 161.0 in stage 10.0 (TID 172) in 16 ms on localhost (161/199) 15/03/14 11:15:52 INFO Executor: Running task 164.0 in stage 10.0 (TID 175) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO Executor: Finished task 163.0 in stage 10.0 (TID 174). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Starting task 165.0 in stage 10.0 (TID 176, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Finished task 162.0 in stage 10.0 (TID 173). 894 bytes result sent to driver 15/03/14 11:15:52 INFO TaskSetManager: Finished task 163.0 in stage 10.0 (TID 174) in 15 ms on localhost (162/199) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:52 INFO TaskSetManager: Starting task 166.0 in stage 10.0 (TID 177, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Finished task 160.0 in stage 10.0 (TID 171). 894 bytes result sent to driver 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:52 INFO TaskSetManager: Starting task 167.0 in stage 10.0 (TID 178, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:52 INFO Executor: Running task 167.0 in stage 10.0 (TID 178) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 160.0 in stage 10.0 (TID 171) in 25 ms on localhost (163/199) 15/03/14 11:15:52 INFO TaskSetManager: Finished task 162.0 in stage 10.0 (TID 173) in 24 ms on localhost (164/199) 15/03/14 11:15:52 INFO Executor: Running task 166.0 in stage 10.0 (TID 177) 15/03/14 11:15:52 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO Executor: Finished task 164.0 in stage 10.0 (TID 175). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Starting task 168.0 in stage 10.0 (TID 179, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 165.0 in stage 10.0 (TID 176) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO Executor: Finished task 167.0 in stage 10.0 (TID 178). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Running task 168.0 in stage 10.0 (TID 179) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 169.0 in stage 10.0 (TID 180, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 167.0 in stage 10.0 (TID 178) in 685 ms on localhost (165/199) 15/03/14 11:15:53 INFO Executor: Running task 169.0 in stage 10.0 (TID 180) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO Executor: Finished task 166.0 in stage 10.0 (TID 177). 894 bytes result sent to driver 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO TaskSetManager: Starting task 170.0 in stage 10.0 (TID 181, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 166.0 in stage 10.0 (TID 177) in 696 ms on localhost (166/199) 15/03/14 11:15:53 INFO Executor: Running task 170.0 in stage 10.0 (TID 181) 15/03/14 11:15:53 INFO Executor: Finished task 169.0 in stage 10.0 (TID 180). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 168.0 in stage 10.0 (TID 179). 1116 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Finished task 164.0 in stage 10.0 (TID 175) in 708 ms on localhost (167/199) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:53 INFO Executor: Finished task 165.0 in stage 10.0 (TID 176). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 170.0 in stage 10.0 (TID 181). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Starting task 171.0 in stage 10.0 (TID 182, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 171.0 in stage 10.0 (TID 182) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 172.0 in stage 10.0 (TID 183, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 172.0 in stage 10.0 (TID 183) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 173.0 in stage 10.0 (TID 184, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 174.0 in stage 10.0 (TID 185, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 174.0 in stage 10.0 (TID 185) 15/03/14 11:15:53 INFO Executor: Running task 173.0 in stage 10.0 (TID 184) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 169.0 in stage 10.0 (TID 180) in 26 ms on localhost (168/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 168.0 in stage 10.0 (TID 179) in 37 ms on localhost (169/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 165.0 in stage 10.0 (TID 176) in 720 ms on localhost (170/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 170.0 in stage 10.0 (TID 181) in 27 ms on localhost (171/199) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO Executor: Finished task 171.0 in stage 10.0 (TID 182). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 172.0 in stage 10.0 (TID 183). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 174.0 in stage 10.0 (TID 185). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Starting task 175.0 in stage 10.0 (TID 186, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 175.0 in stage 10.0 (TID 186) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 176.0 in stage 10.0 (TID 187, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 177.0 in stage 10.0 (TID 188, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 177.0 in stage 10.0 (TID 188) 15/03/14 11:15:53 INFO Executor: Running task 176.0 in stage 10.0 (TID 187) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 172.0 in stage 10.0 (TID 183) in 25 ms on localhost (172/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 171.0 in stage 10.0 (TID 182) in 28 ms on localhost (173/199) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO Executor: Finished task 173.0 in stage 10.0 (TID 184). 894 bytes result sent to driver 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO TaskSetManager: Starting task 178.0 in stage 10.0 (TID 189, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 173.0 in stage 10.0 (TID 184) in 38 ms on localhost (174/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 174.0 in stage 10.0 (TID 185) in 38 ms on localhost (175/199) 15/03/14 11:15:53 INFO Executor: Running task 178.0 in stage 10.0 (TID 189) 15/03/14 11:15:53 INFO Executor: Finished task 176.0 in stage 10.0 (TID 187). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 177.0 in stage 10.0 (TID 188). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Starting task 179.0 in stage 10.0 (TID 190, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 176.0 in stage 10.0 (TID 187) in 23 ms on localhost (176/199) 15/03/14 11:15:53 INFO Executor: Running task 179.0 in stage 10.0 (TID 190) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO TaskSetManager: Starting task 180.0 in stage 10.0 (TID 191, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 180.0 in stage 10.0 (TID 191) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 3 ms 15/03/14 11:15:53 INFO Executor: Finished task 175.0 in stage 10.0 (TID 186). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Starting task 181.0 in stage 10.0 (TID 192, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO TaskSetManager: Finished task 177.0 in stage 10.0 (TID 188) in 32 ms on localhost (177/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 175.0 in stage 10.0 (TID 186) in 36 ms on localhost (178/199) 15/03/14 11:15:53 INFO Executor: Running task 181.0 in stage 10.0 (TID 192) 15/03/14 11:15:53 INFO Executor: Finished task 179.0 in stage 10.0 (TID 190). 894 bytes result sent to driver 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO TaskSetManager: Starting task 182.0 in stage 10.0 (TID 193, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 182.0 in stage 10.0 (TID 193) 15/03/14 11:15:53 INFO Executor: Finished task 178.0 in stage 10.0 (TID 189). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 180.0 in stage 10.0 (TID 191). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Finished task 179.0 in stage 10.0 (TID 190) in 19 ms on localhost (179/199) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 183.0 in stage 10.0 (TID 194, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 183.0 in stage 10.0 (TID 194) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 184.0 in stage 10.0 (TID 195, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO TaskSetManager: Finished task 178.0 in stage 10.0 (TID 189) in 39 ms on localhost (180/199) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:53 INFO TaskSetManager: Finished task 180.0 in stage 10.0 (TID 191) in 23 ms on localhost (181/199) 15/03/14 11:15:53 INFO Executor: Running task 184.0 in stage 10.0 (TID 195) 15/03/14 11:15:53 INFO Executor: Finished task 182.0 in stage 10.0 (TID 193). 894 bytes result sent to driver 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO Executor: Finished task 183.0 in stage 10.0 (TID 194). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 181.0 in stage 10.0 (TID 192). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Starting task 185.0 in stage 10.0 (TID 196, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Finished task 184.0 in stage 10.0 (TID 195). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Starting task 186.0 in stage 10.0 (TID 197, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 187.0 in stage 10.0 (TID 198, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 186.0 in stage 10.0 (TID 197) 15/03/14 11:15:53 INFO Executor: Running task 187.0 in stage 10.0 (TID 198) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 188.0 in stage 10.0 (TID 199, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 182.0 in stage 10.0 (TID 193) in 25 ms on localhost (182/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 181.0 in stage 10.0 (TID 192) in 31 ms on localhost (183/199) 15/03/14 11:15:53 INFO Executor: Running task 185.0 in stage 10.0 (TID 196) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 183.0 in stage 10.0 (TID 194) in 15 ms on localhost (184/199) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO Executor: Finished task 187.0 in stage 10.0 (TID 198). 1129 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 185.0 in stage 10.0 (TID 196). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 186.0 in stage 10.0 (TID 197). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Running task 188.0 in stage 10.0 (TID 199) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 189.0 in stage 10.0 (TID 200, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 190.0 in stage 10.0 (TID 201, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 191.0 in stage 10.0 (TID 202, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 191.0 in stage 10.0 (TID 202) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 187.0 in stage 10.0 (TID 198) in 15 ms on localhost (185/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 184.0 in stage 10.0 (TID 195) in 27 ms on localhost (186/199) 15/03/14 11:15:53 INFO Executor: Running task 189.0 in stage 10.0 (TID 200) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 186.0 in stage 10.0 (TID 197) in 16 ms on localhost (187/199) 15/03/14 11:15:53 INFO Executor: Running task 190.0 in stage 10.0 (TID 201) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 1 ms 15/03/14 11:15:53 INFO Executor: Finished task 191.0 in stage 10.0 (TID 202). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 189.0 in stage 10.0 (TID 200). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 190.0 in stage 10.0 (TID 201). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Starting task 192.0 in stage 10.0 (TID 203, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 193.0 in stage 10.0 (TID 204, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 194.0 in stage 10.0 (TID 205, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 194.0 in stage 10.0 (TID 205) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO Executor: Running task 193.0 in stage 10.0 (TID 204) 15/03/14 11:15:53 INFO Executor: Finished task 188.0 in stage 10.0 (TID 199). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Running task 192.0 in stage 10.0 (TID 203) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO TaskSetManager: Finished task 191.0 in stage 10.0 (TID 202) in 15 ms on localhost (188/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 190.0 in stage 10.0 (TID 201) in 23 ms on localhost (189/199) 15/03/14 11:15:53 INFO Executor: Finished task 194.0 in stage 10.0 (TID 205). 894 bytes result sent to driver 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO TaskSetManager: Finished task 185.0 in stage 10.0 (TID 196) in 40 ms on localhost (190/199) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 2 ms 15/03/14 11:15:53 INFO TaskSetManager: Finished task 189.0 in stage 10.0 (TID 200) in 27 ms on localhost (191/199) 15/03/14 11:15:53 INFO Executor: Finished task 193.0 in stage 10.0 (TID 204). 894 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 192.0 in stage 10.0 (TID 203). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Starting task 195.0 in stage 10.0 (TID 206, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 195.0 in stage 10.0 (TID 206) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 196.0 in stage 10.0 (TID 207, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 197.0 in stage 10.0 (TID 208, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 197.0 in stage 10.0 (TID 208) 15/03/14 11:15:53 INFO TaskSetManager: Starting task 198.0 in stage 10.0 (TID 209, localhost, PROCESS_LOCAL, 1056 bytes) 15/03/14 11:15:53 INFO Executor: Running task 198.0 in stage 10.0 (TID 209) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 188.0 in stage 10.0 (TID 199) in 45 ms on localhost (192/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 194.0 in stage 10.0 (TID 205) in 19 ms on localhost (193/199) 15/03/14 11:15:53 INFO Executor: Running task 196.0 in stage 10.0 (TID 207) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 192.0 in stage 10.0 (TID 203) in 20 ms on localhost (194/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 193.0 in stage 10.0 (TID 204) in 20 ms on localhost (195/199) 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO Executor: Finished task 195.0 in stage 10.0 (TID 206). 894 bytes result sent to driver 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 0 ms 15/03/14 11:15:53 INFO TaskSetManager: Finished task 195.0 in stage 10.0 (TID 206) in 16 ms on localhost (196/199) 15/03/14 11:15:53 INFO Executor: Finished task 197.0 in stage 10.0 (TID 208). 894 bytes result sent to driver 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Getting 2 non-empty blocks out of 2 blocks 15/03/14 11:15:53 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in 9 ms 15/03/14 11:15:53 INFO TaskSetManager: Finished task 197.0 in stage 10.0 (TID 208) in 19 ms on localhost (197/199) 15/03/14 11:15:53 INFO Executor: Finished task 198.0 in stage 10.0 (TID 209). 1129 bytes result sent to driver 15/03/14 11:15:53 INFO Executor: Finished task 196.0 in stage 10.0 (TID 207). 894 bytes result sent to driver 15/03/14 11:15:53 INFO TaskSetManager: Finished task 198.0 in stage 10.0 (TID 209) in 23 ms on localhost (198/199) 15/03/14 11:15:53 INFO TaskSetManager: Finished task 196.0 in stage 10.0 (TID 207) in 28 ms on localhost (199/199) 15/03/14 11:15:53 INFO TaskSchedulerImpl: Removed TaskSet 10.0, whose tasks have all completed, from pool 15/03/14 11:15:53 INFO DAGScheduler: Stage 10 (runJob at SparkPlan.scala:121) finished in 3.044 s 15/03/14 11:15:53 INFO DAGScheduler: Job 8 finished: runJob at SparkPlan.scala:121, took 3.095694 s age count null 1 19 1 30 1

總結(jié)

以上是生活随笔為你收集整理的Spark SQL and DataFrame for Spark 1.3的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。