org.apache.spark.sql.parquet

Class ParquetTableScan

Object

org.apache.spark.sql.catalyst.trees.TreeNode<PlanType>

org.apache.spark.sql.catalyst.plans.QueryPlan<org.apache.spark.sql.execution.SparkPlan>

org.apache.spark.sql.execution.SparkPlan

org.apache.spark.sql.parquet.ParquetTableScan

All Implemented Interfaces:

java.io.Serializable, Logging, org.apache.spark.sql.catalyst.trees.LeafNode<org.apache.spark.sql.execution.SparkPlan>, scala.Equals, scala.Product

public class ParquetTableScan
extends org.apache.spark.sql.execution.SparkPlan
implements org.apache.spark.sql.execution.LeafNode, scala.Product, scala.Serializable

:: DeveloperApi :: Parquet table scan operator. Imports the file that backs the given ParquetRelation as a RDD[Row].

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
ParquetTableScan(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> attributes, ParquetRelation relation, scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> columnPruningPred)

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute>	attributes()
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression>	columnPruningPred()
RDD<org.apache.spark.sql.Row>	execute()
scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute>	output()
ParquetTableScan	pruneColumns(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> prunedAttributes) Applies a (candidate) projection.
ParquetRelation	relation()
scala.Tuple2<Object,Object>[]	requestedPartitionOrdinals()

Methods inherited from class org.apache.spark.sql.execution.SparkPlan

codegenEnabled, executeCollect, executeTake, isTraceEnabled, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning, makeCopy, newMutableProjection, newOrdering, newPredicate, newProjection, org$apache$spark$Logging$$log__$eq, org$apache$spark$Logging$$log_, outputPartitioning, requiredChildDistribution, sparkContext, sqlContext

Methods inherited from class org.apache.spark.sql.catalyst.plans.QueryPlan

expressions, inputSet, missingInput, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionDown$1, org$apache$spark$sql$catalyst$plans$QueryPlan$$transformExpressionUp$1, outputSet, printSchema, references, schema, schemaString, simpleString, statePrefix, transformAllExpressions, transformExpressions, transformExpressionsDown, transformExpressionsUp

Methods inherited from class org.apache.spark.sql.catalyst.trees.TreeNode

apply, argString, asCode, children, collect, fastEquals, flatMap, foreach, foreachUp, generateTreeString, getNodeNumbered, map, mapChildren, nodeName, numberedTreeString, origin, otherCopyArgs, stringArgs, toString, transform, transformChildrenDown, transformChildrenUp, transformDown, transformUp, treeString, withNewChildren

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.sql.catalyst.trees.LeafNode

children

Methods inherited from interface scala.Product

productArity, productElement, productIterator, productPrefix

Methods inherited from interface scala.Equals

canEqual, equals

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, log_

Constructor Detail

ParquetTableScan

public ParquetTableScan(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> attributes,
                ParquetRelation relation,
                scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> columnPruningPred)

Method Detail

attributes

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> attributes()

relation

public ParquetRelation relation()

columnPruningPred

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Expression> columnPruningPred()

output

public scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> output()

Specified by:

output in class org.apache.spark.sql.catalyst.plans.QueryPlan<org.apache.spark.sql.execution.SparkPlan>

requestedPartitionOrdinals

public scala.Tuple2<Object,Object>[] requestedPartitionOrdinals()

execute

public RDD<org.apache.spark.sql.Row> execute()

Specified by:

execute in class org.apache.spark.sql.execution.SparkPlan

pruneColumns

public ParquetTableScan pruneColumns(scala.collection.Seq<org.apache.spark.sql.catalyst.expressions.Attribute> prunedAttributes)

Applies a (candidate) projection.

Parameters:

prunedAttributes - The list of attributes to be used in the projection.

Returns:

Pruned TableScan.