org.apache.spark.streaming.kafka

Class DirectKafkaInputDStream.DirectKafkaInputDStreamCheckpointData

Object

org.apache.spark.streaming.dstream.DStreamCheckpointData<R>

org.apache.spark.streaming.kafka.DirectKafkaInputDStream.DirectKafkaInputDStreamCheckpointData

All Implemented Interfaces:

java.io.Serializable, Logging

Enclosing class:

DirectKafkaInputDStream<K,V,U extends kafka.serializer.Decoder<K>,T extends kafka.serializer.Decoder<V>,R>

public class DirectKafkaInputDStream.DirectKafkaInputDStreamCheckpointData
extends DStreamCheckpointData<R>

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
DirectKafkaInputDStream.DirectKafkaInputDStreamCheckpointData()

Method Summary

Methods

Modifier and Type	Method and Description
scala.collection.mutable.HashMap<Time,scala.Tuple4<String,Object,Object,Object>[]>	batchForTime()
void	cleanup(Time time) Cleanup old checkpoint data.
void	restore() Restore the checkpoint data.
void	update(Time time) Updates the checkpoint data of the DStream.

Methods inherited from class org.apache.spark.streaming.dstream.DStreamCheckpointData

toString

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.spark.Logging

initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning

Constructor Detail

DirectKafkaInputDStream.DirectKafkaInputDStreamCheckpointData

public DirectKafkaInputDStream.DirectKafkaInputDStreamCheckpointData()

Method Detail

batchForTime

public scala.collection.mutable.HashMap<Time,scala.Tuple4<String,Object,Object,Object>[]> batchForTime()

update

public void update(Time time)

Description copied from class: DStreamCheckpointData

Updates the checkpoint data of the DStream. This gets called every time the graph checkpoint is initiated. Default implementation records the checkpoint files to which the generate RDDs of the DStream has been saved.

Overrides:

update in class DStreamCheckpointData<R>

cleanup

public void cleanup(Time time)

Description copied from class: DStreamCheckpointData

Cleanup old checkpoint data. This gets called after a checkpoint of time has been written to the checkpoint directory.

Overrides:

cleanup in class DStreamCheckpointData<R>

restore

public void restore()

Description copied from class: DStreamCheckpointData

Restore the checkpoint data. This gets called once when the DStream graph (along with its DStreams) are being restored from a graph checkpoint file. Default implementation restores the RDDs from their checkpoint files.

Overrides:

restore in class DStreamCheckpointData<R>