'What's the difference between spark-shell and spark-sql? Any performance differences?
Spark-shell: which basically opens the scala> prompt. Where query needs to write in below manner
val sqlContext = new org.apache.spark.sql.hive.HiveContext(sc)
// Queries are expressed in HiveQL
sqlContext.sql("FROM src SELECT key, value").collect().foreach(println)
spark-sql: Which seems to connect directly to hive metastore and we can write a query in a similar way to hive. And query over existing data in hive
I want to know the difference between these two.. And does processing any query in spark-sql is same as in spark-shell? I mean can we leverage the performance benefits of spark in spark-sql?
Spark 1.5.2's here.
Sources
This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.
Source: Stack Overflow
| Solution | Source |
|---|