Search notes:

Apache Spark

Spark is a fast and general cluster computing system for Big Data.

High-level APIs in Scala, Java, Python, and R

In contrast to Hadoop MapReduce, which is disk based, Spark uses memory. Apparently, this is useful for machine learning algorithms.

Spark runs on

Spark accesses various data sources.

Originally developed in 2009 in UC Berkeley's AMP lab.

Fully open sourced in 2010 - now a top level project at the Apache Software Foundation

Documentation

Transformations

Actions

MLlib: machine learning

GraphX: graph processing

Spark Streaming