Builder for SparkSession

The entry point to programming Spark with the Dataset and DataFrame API

User-facing configuration API, accessible through SparkSession.conf

A distributed collection of data grouped into named columns

A set of methods for aggregations on a DataFrame, created by DataFrame.groupBy()

A column in a DataFrame

A row in DataFrame

Struct type, consisting of a list of StructField

A field in StructType

Utility functions for defining window in DataFrames

A window specification that defines the partitioning, ordering, and frame boundaries

Interface used to load a DataFrame from external storage systems

Interface used to write a DataFrame to external storage systems

Interface used to load a streaming DataFrame from external storage systems

Interface used to write a streaming DataFrame to external

A handle to a query that is executing continuously in the background as new data arrives

@chainable T

Adds dot chaining syntax to the type, i.e. automatically translate:

foo.bar(a)

into

bar(foo, a)

For single-argument functions also support implicit calls, e.g:

foo.bar.baz(a, b)

is treated the same as:

foo.bar().baz(a, b)

Note that @chainable works by overloading Base.getproperty(), making it impossible to customize it for T. To have more control, one may use the underlying wrapper type - DotCaller.

DotChainer{O, Fn}

See @chainable for details.