org.apache.spark.ml.util

Class MLWriter

Object

org.apache.spark.ml.util.MLWriter

All Implemented Interfaces:

Logging

Direct Known Subclasses:

CrossValidatorModel.CrossValidatorModelWriter, GeneralMLWriter, TrainValidationSplitModel.TrainValidationSplitModelWriter

public abstract class MLWriter
extends Object
implements Logging

Abstract class for utility classes that can save ML instances in Spark's internal format.

Constructor Summary

Constructors

Constructor and Description
MLWriter()

Method Summary

Modifier and Type	Method and Description
MLWriter	context(SQLContext sqlContext) Sets the Spark SQLContext to use for saving/loading.
MLWriter	option(String key, String value) Adds an option to the underlying MLWriter.
scala.Option<SparkSession>	optionSparkSession()
MLWriter	overwrite() Overwrites if the output path already exists.
void	save(String path) Saves the ML instances to the input path.
SparkContext	sc() Returns the underlying `SparkContext`.
MLWriter	session(SparkSession sparkSession) Sets the Spark Session to use for saving/loading.
SparkSession	sparkSession() Returns the user-specified Spark Session or the default.
SQLContext	sqlContext() Returns the user-specified SQL context or the default.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.spark.internal.Logging

initializeLogging, initializeLogIfNecessary, initializeLogIfNecessary, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

MLWriter

public MLWriter()

Method Detail

context

public MLWriter context(SQLContext sqlContext)

Sets the Spark SQLContext to use for saving/loading.

Parameters:

sqlContext - (undocumented)

Returns:

(undocumented)

option

public MLWriter option(String key,
                       String value)

Adds an option to the underlying MLWriter. See the documentation for the specific model's writer for possible options. The option name (key) is case-insensitive.

Parameters:

key - (undocumented)

value - (undocumented)

Returns:

(undocumented)

overwrite

public MLWriter overwrite()

Overwrites if the output path already exists.

Returns:

(undocumented)

save

public void save(String path)
          throws java.io.IOException

Saves the ML instances to the input path.

Parameters:

path - (undocumented)

Throws:

java.io.IOException

session

public MLWriter session(SparkSession sparkSession)

Sets the Spark Session to use for saving/loading.

Parameters:

sparkSession - (undocumented)

Returns:

(undocumented)

optionSparkSession

public scala.Option<SparkSession> optionSparkSession()

sc

public SparkContext sc()

Returns the underlying `SparkContext`.

sparkSession

public SparkSession sparkSession()

Returns the user-specified Spark Session or the default.

Returns:

(undocumented)

sqlContext

public SQLContext sqlContext()

Returns the user-specified SQL context or the default.

Returns:

(undocumented)