Apache Spark: Difference between revisions
(Created page with "==Apache Spark== ===Purpose=== * Getting up and running with * Getting experience with non-trivial installation * Using IntelliJ IDEA. * Writing and running your own first Sp...") |
No edit summary |
||
| Line 6: | Line 6: | ||
* Writing and running your own first Spark program | * Writing and running your own first Spark program | ||
For a general introduction, see the slides to [[:File: | For a general introduction, see the slides to [[:File:S2-Spark-intro.pdf | Session 2 on Apache Spark]]. Here is a useful tutorial: https://www.tutorialspoint.com/spark_sql/spark_introduction.htm . Configuring Spark dependency in InjelliJ IDEA http://spark.apache.org/docs/latest/rdd-programming-guide.html | ||
===Preparations=== | ===Preparations=== | ||
Revision as of 20:53, 30 August 2018
Apache Spark
Purpose
- Getting up and running with
- Getting experience with non-trivial installation
- Using IntelliJ IDEA.
- Writing and running your own first Spark program
For a general introduction, see the slides to Session 2 on Apache Spark. Here is a useful tutorial: https://www.tutorialspoint.com/spark_sql/spark_introduction.htm . Configuring Spark dependency in InjelliJ IDEA http://spark.apache.org/docs/latest/rdd-programming-guide.html
Preparations
As for Hadoop, you will run Spark standalone on your computers (and independently of your previous Hadoop installation to keep things simple). Running Spark on a cluster of many computers is harder to set up (and you will need a cluster of computers), but after that, the coding and running of code is the same. Installing Spark Standalone to a Cluster http://spark.apache.org/docs/latest/spark-standalone.html
Follow these preparations to install Spark on your Linux or Windows-machine. If you are on MacOS, it runs BSD Unix under the hood, so most Linux-commands should work in a Terminal window on your Mac too.