org.apache.spark.rpc

Interface RpcEnvFileServer

public interface RpcEnvFileServer

A server used by the RpcEnv to server files to other processes owned by the application.

The file server can return URIs handled by common libraries (such as "http" or "hdfs"), or it can return "spark" URIs which will be handled by RpcEnv#fetchFile.

Method Summary

Modifier and Type	Method and Description
String	addDirectory(String baseUri, java.io.File path) Adds a local directory to be served via this file server.
String	addFile(java.io.File file) Adds a file to be served by this RpcEnv.
String	addJar(java.io.File file) Adds a jar to be served by this RpcEnv.
String	validateDirectoryUri(String baseUri) Validates and normalizes the base URI for directories.

Method Detail

addDirectory

String addDirectory(String baseUri,
                    java.io.File path)

Adds a local directory to be served via this file server.

Parameters:

baseUri - Leading URI path (files can be retrieved by appending their relative path to this base URI). This cannot be "files" nor "jars".

path - Path to the local directory.

Returns:

URI for the root of the directory in the file server.

addFile

String addFile(java.io.File file)

Adds a file to be served by this RpcEnv. This is used to serve files from the driver to executors when they're stored on the driver's local file system.

Parameters:

file - Local file to serve.

Returns:

A URI for the location of the file.

addJar

String addJar(java.io.File file)

Adds a jar to be served by this RpcEnv. Similar to addFile but for jars added using SparkContext.addJar.

Parameters:

file - Local file to serve.

Returns:

A URI for the location of the file.

validateDirectoryUri

String validateDirectoryUri(String baseUri)

Validates and normalizes the base URI for directories.