Class Stream
- All Implemented Interfaces:
- ResourceDeclarer<Stream>,- IAggregatableStream
There are five types of operations that can be performed on streams in Trident
1. **Partiton-Local Operations** - Operations that are applied locally to each partition and do not involve network transfer 2. **Repartitioning Operations** - Operations that change how tuples are partitioned across tasks(thus causing network transfer), but do not change the content of the stream. 3. **Aggregation Operations** - Operations that *may* repartition a stream (thus causing network transfer) 4. **Grouping Operations** - Operations that may repartition a stream on specific fields and group together tuples whose fields values are equal. 5. **Merge and Join Operations** - Operations that combine different streams together.
- 
Constructor SummaryConstructors
- 
Method SummaryModifier and TypeMethodDescriptionaddSharedMemory(SharedMemory request) Add in request for shared memory that this component will use.aggregate(Aggregator agg, Fields functionFields) aggregate(CombinerAggregator agg, Fields functionFields) aggregate(ReducerAggregator agg, Fields functionFields) aggregate(Fields inputFields, Aggregator agg, Fields functionFields) aggregate(Fields inputFields, CombinerAggregator agg, Fields functionFields) aggregate(Fields inputFields, ReducerAggregator agg, Fields functionFields) applyAssembly(Assembly assembly) Applies an `Assembly` to this `Stream`.## Repartitioning Operation.## Repartitioning Operation.Returns a stream consisting of the elements of this stream that match the given filter.Returns a stream consisting of the elements of this stream that match the given filter.flatMap(FlatMapFunction function) Returns a stream consisting of the results of replacing each value of this stream with the contents produced by applying the provided mapping function to each value.flatMap(FlatMapFunction function, Fields outputFields) Returns a stream consisting of the results of replacing each value of this stream with the contents produced by applying the provided mapping function to each value.getName()Returns the label applied to the stream.global()## Repartitioning Operation.## Grouping Operation.## Repartitioning Operation.## Repartitioning Operation.map(MapFunction function) Returns a stream consisting of the result of applying the given mapping function to the values of this stream.map(MapFunction function, Fields outputFields) Returns a stream consisting of the result of applying the given mapping function to the values of this stream.max(Comparator<TridentTuple> comparator) This aggregator operation computes the maximum of tuples in a stream by using the givencomparatorwithTridentTuples.This aggregator operation computes the maximum of tuples by the giveninputFieldNameand it is assumed that its value is an instance ofComparable.<T> StreammaxBy(String inputFieldName, Comparator<T> comparator) This aggregator operation computes the maximum of tuples by the giveninputFieldNamein a stream by using the givencomparator.min(Comparator<TridentTuple> comparator) This aggregator operation computes the minimum of tuples in a stream by using the givencomparatorwithTridentTuples.This aggregator operation computes the minimum of tuples by the giveninputFieldNameand it is assumed that its value is an instance ofComparable.<T> StreamminBy(String inputFieldName, Comparator<T> comparator) This aggregator operation computes the minimum of tuples by the giveninputFieldNamein a stream by using the givencomparator.Applies a label to the stream.parallelismHint(int hint) Applies a parallelism hint to a stream.## Repartitioning Operation.partition(CustomStreamGrouping partitioner) ## Repartitioning Operation.partitionAggregate(Aggregator agg, Fields functionFields) partitionAggregate(CombinerAggregator agg, Fields functionFields) partitionAggregate(ReducerAggregator agg, Fields functionFields) partitionAggregate(Fields inputFields, Aggregator agg, Fields functionFields) partitionAggregate(Fields inputFields, CombinerAggregator agg, Fields functionFields) partitionAggregate(Fields inputFields, ReducerAggregator agg, Fields functionFields) partitionBy(Fields fields) ## Repartitioning Operation.partitionPersist(StateFactory stateFactory, StateUpdater updater) partitionPersist(StateFactory stateFactory, StateUpdater updater, Fields functionFields) partitionPersist(StateFactory stateFactory, Fields inputFields, StateUpdater updater) partitionPersist(StateFactory stateFactory, Fields inputFields, StateUpdater updater, Fields functionFields) partitionPersist(StateSpec stateSpec, StateUpdater updater) partitionPersist(StateSpec stateSpec, StateUpdater updater, Fields functionFields) partitionPersist(StateSpec stateSpec, Fields inputFields, StateUpdater updater) partitionPersist(StateSpec stateSpec, Fields inputFields, StateUpdater updater, Fields functionFields) Returns a stream consisting of the trident tuples of this stream, additionally performing the provided action on each trident tuple as they are consumed from the resulting stream.persistentAggregate(StateFactory stateFactory, CombinerAggregator agg, Fields functionFields) persistentAggregate(StateFactory stateFactory, ReducerAggregator agg, Fields functionFields) persistentAggregate(StateFactory stateFactory, Fields inputFields, CombinerAggregator agg, Fields functionFields) persistentAggregate(StateFactory stateFactory, Fields inputFields, ReducerAggregator agg, Fields functionFields) persistentAggregate(StateSpec spec, CombinerAggregator agg, Fields functionFields) persistentAggregate(StateSpec spec, ReducerAggregator agg, Fields functionFields) persistentAggregate(StateSpec spec, Fields inputFields, CombinerAggregator agg, Fields functionFields) persistentAggregate(StateSpec spec, Fields inputFields, ReducerAggregator agg, Fields functionFields) Filters out fields from a stream, resulting in a Stream containing only the fields specified by `keepFields`.setCPULoad(Number load) Sets the CPU Load resource for the current operation.setMemoryLoad(Number onHeap) Sets the Memory Load resources for the current operation.setMemoryLoad(Number onHeap, Number offHeap) Sets the Memory Load resources for the current operation.shuffle()## Repartitioning Operation.slidingWindow(int windowCount, int slideCount, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of tuples which are aggregated results of a sliding window with everywindowCountof tuples and slides the window afterslideCount.slidingWindow(BaseWindowedBolt.Duration windowDuration, BaseWindowedBolt.Duration slidingInterval, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of tuples which are aggregated results of a window which slides at duration ofslidingIntervaland completes a window atwindowDuration.stateQuery(TridentState state, QueryFunction function, Fields functionFields) stateQuery(TridentState state, Fields inputFields, QueryFunction function, Fields functionFields) toStream()tumblingWindow(int windowCount, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of tuples which are aggregated results of a tumbling window with everywindowCountof tuples.tumblingWindow(BaseWindowedBolt.Duration windowDuration, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of tuples which are aggregated results of a window that tumbles at duration ofwindowDuration.window(WindowConfig windowConfig, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns stream of aggregated results based on the given window configuration.window(WindowConfig windowConfig, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of aggregated results based on the given window configuration which uses inmemory windowing tuple store.
- 
Constructor Details- 
Stream
 
- 
- 
Method Details- 
nameApplies a label to the stream. Naming a stream will append the label to the name of the bolt(s) created by Trident and will be visible in the Storm UI.- Parameters:
- name- - The label to apply to the stream
 
- 
parallelismHintApplies a parallelism hint to a stream.
- 
setCPULoadSets the CPU Load resource for the current operation.- Specified by:
- setCPULoadin interface- ResourceDeclarer<Stream>
- Parameters:
- load- the amount of CPU
- Returns:
- this for chaining
 
- 
setMemoryLoadSets the Memory Load resources for the current operation. offHeap becomes default.- Specified by:
- setMemoryLoadin interface- ResourceDeclarer<Stream>
- Parameters:
- onHeap- the amount of on heap memory
- Returns:
- this for chaining
 
- 
setMemoryLoadSets the Memory Load resources for the current operation.- Specified by:
- setMemoryLoadin interface- ResourceDeclarer<Stream>
- Parameters:
- onHeap- the amount of on heap memory
- offHeap- the amount of off heap memory
- Returns:
- this for chaining
 
- 
projectFilters out fields from a stream, resulting in a Stream containing only the fields specified by `keepFields`.For example, if you had a Stream `mystream` containing the fields `["a", "b", "c","d"]`, calling" ```java mystream.project(new Fields("b", "d")) ``` would produce a stream containing only the fields `["b", "d"]`. - Parameters:
- keepFields- The fields in the Stream to keep
 
- 
groupBy## Grouping Operation.
- 
partitionBy## Repartitioning Operation.
- 
partition## Repartitioning Operation.
- 
partition## Repartitioning Operation.This method takes in a custom partitioning function that implements CustomStreamGrouping
- 
shuffle## Repartitioning Operation.Use random round robin algorithm to evenly redistribute tuples across all target partitions. 
- 
localOrShuffle## Repartitioning Operation.Use random round robin algorithm to evenly redistribute tuples across all target partitions, with a preference for local tasks. 
- 
global## Repartitioning Operation.All tuples are sent to the same partition. The same partition is chosen for all batches in the stream. 
- 
batchGlobal## Repartitioning Operation.All tuples in the batch are sent to the same partition. Different batches in the stream may go to different partitions. 
- 
broadcast## Repartitioning Operation.Every tuple is replicated to all target partitions. This can useful during DRPC – for example, if you need to do a stateQuery on every partition of data. 
- 
identityPartition## Repartitioning Operation.
- 
applyAssemblyApplies an `Assembly` to this `Stream`.- See Also:
 
- 
each- Specified by:
- eachin interface- IAggregatableStream
 
- 
each
- 
each
- 
partitionAggregate- Specified by:
- partitionAggregatein interface- IAggregatableStream
 
- 
partitionAggregate
- 
partitionAggregate
- 
partitionAggregate
- 
partitionAggregate
- 
partitionAggregate
- 
stateQuerypublic Stream stateQuery(TridentState state, Fields inputFields, QueryFunction function, Fields functionFields) 
- 
stateQuery
- 
partitionPersistpublic TridentState partitionPersist(StateFactory stateFactory, Fields inputFields, StateUpdater updater, Fields functionFields) 
- 
partitionPersistpublic TridentState partitionPersist(StateSpec stateSpec, Fields inputFields, StateUpdater updater, Fields functionFields) 
- 
partitionPersistpublic TridentState partitionPersist(StateFactory stateFactory, Fields inputFields, StateUpdater updater) 
- 
partitionPersist
- 
partitionPersistpublic TridentState partitionPersist(StateFactory stateFactory, StateUpdater updater, Fields functionFields) 
- 
partitionPersistpublic TridentState partitionPersist(StateSpec stateSpec, StateUpdater updater, Fields functionFields) 
- 
partitionPersist
- 
partitionPersist
- 
filterReturns a stream consisting of the elements of this stream that match the given filter.- Parameters:
- filter- the filter to apply to each trident tuple to determine if it should be included.
- Returns:
- the new stream
 
- 
filterReturns a stream consisting of the elements of this stream that match the given filter.- Parameters:
- inputFields- the fields of the input trident tuple to be selected.
- filter- the filter to apply to each trident tuple to determine if it should be included.
- Returns:
- the new stream
 
- 
mapReturns a stream consisting of the result of applying the given mapping function to the values of this stream.- Parameters:
- function- a mapping function to be applied to each value in this stream.
- Returns:
- the new stream
 
- 
mapReturns a stream consisting of the result of applying the given mapping function to the values of this stream. This method replaces old output fields with new output fields, achieving T -> V conversion.- Parameters:
- function- a mapping function to be applied to each value in this stream.
- outputFields- new output fields
- Returns:
- the new stream
 
- 
flatMapReturns a stream consisting of the results of replacing each value of this stream with the contents produced by applying the provided mapping function to each value. This has the effect of applying a one-to-many transformation to the values of the stream, and then flattening the resulting elements into a new stream.- Parameters:
- function- a mapping function to be applied to each value in this stream which produces new values.
- Returns:
- the new stream
 
- 
flatMapReturns a stream consisting of the results of replacing each value of this stream with the contents produced by applying the provided mapping function to each value. This has the effect of applying a one-to-many transformation to the values of the stream, and then flattening the resulting elements into a new stream. This method replaces old output fields with new output fields, achieving T -> V conversion.- Parameters:
- function- a mapping function to be applied to each value in this stream which produces new values.
- outputFields- new output fields
- Returns:
- the new stream
 
- 
peekReturns a stream consisting of the trident tuples of this stream, additionally performing the provided action on each trident tuple as they are consumed from the resulting stream. This is mostly useful for debugging to see the tuples as they flow past a certain point in a pipeline.- Parameters:
- action- the action to perform on the trident tuple as they are consumed from the stream
- Returns:
- the new stream
 
- 
chainedAgg
- 
minByThis aggregator operation computes the minimum of tuples by the giveninputFieldNameand it is assumed that its value is an instance ofComparable. If the value of tuple with fieldinputFieldNameis not an instance ofComparablethen it throwsClassCastException- Parameters:
- inputFieldName- input field name
- Returns:
- the new stream with this operation.
 
- 
minByThis aggregator operation computes the minimum of tuples by the giveninputFieldNamein a stream by using the givencomparator. If the value of tuple with fieldinputFieldNameis not an instance ofTthen it throwsClassCastException- Parameters:
- inputFieldName- input field name
- comparator- comparator used in for finding minimum of two tuple values of- inputFieldName.
- Returns:
- the new stream with this operation.
 
- 
minThis aggregator operation computes the minimum of tuples in a stream by using the givencomparatorwithTridentTuples.- Parameters:
- comparator- comparator used in for finding minimum of two tuple values.
- Returns:
- the new stream with this operation.
 
- 
maxByThis aggregator operation computes the maximum of tuples by the giveninputFieldNameand it is assumed that its value is an instance ofComparable. If the value of tuple with fieldinputFieldNameis not an instance ofComparablethen it throwsClassCastException- Parameters:
- inputFieldName- input field name
- Returns:
- the new stream with this operation.
 
- 
maxByThis aggregator operation computes the maximum of tuples by the giveninputFieldNamein a stream by using the givencomparator. If the value of tuple with fieldinputFieldNameis not an instance ofTthen it throwsClassCastException- Parameters:
- inputFieldName- input field name
- comparator- comparator used in for finding maximum of two tuple values of- inputFieldName.
- Returns:
- the new stream with this operation.
 
- 
maxThis aggregator operation computes the maximum of tuples in a stream by using the givencomparatorwithTridentTuples.- Parameters:
- comparator- comparator used in for finding maximum of two tuple values.
- Returns:
- the new stream with this operation.
 
- 
aggregate
- 
aggregate
- 
aggregate
- 
aggregate
- 
aggregate
- 
aggregate
- 
tumblingWindowpublic Stream tumblingWindow(int windowCount, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of tuples which are aggregated results of a tumbling window with everywindowCountof tuples.- Parameters:
- windowCount- represents number of tuples in the window
- windowStoreFactory- intermediary tuple store for storing windowing tuples
- inputFields- projected fields for aggregator
- aggregator- aggregator to run on the window of tuples to compute the result and emit to the stream.
- functionFields- fields of values to emit with aggregation.
- Returns:
- the new stream with this operation.
 
- 
tumblingWindowpublic Stream tumblingWindow(BaseWindowedBolt.Duration windowDuration, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of tuples which are aggregated results of a window that tumbles at duration ofwindowDuration.- Parameters:
- windowDuration- represents tumbling window duration configuration
- windowStoreFactory- intermediary tuple store for storing windowing tuples
- inputFields- projected fields for aggregator
- aggregator- aggregator to run on the window of tuples to compute the result and emit to the stream.
- functionFields- fields of values to emit with aggregation.
- Returns:
- the new stream with this operation.
 
- 
slidingWindowpublic Stream slidingWindow(int windowCount, int slideCount, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of tuples which are aggregated results of a sliding window with everywindowCountof tuples and slides the window afterslideCount.- Parameters:
- windowCount- represents tuples count of a window
- slideCount- the number of tuples after which the window slides
- windowStoreFactory- intermediary tuple store for storing windowing tuples
- inputFields- projected fields for aggregator
- aggregator- aggregator to run on the window of tuples to compute the result and emit to the stream.
- functionFields- fields of values to emit with aggregation.
- Returns:
- the new stream with this operation.
 
- 
slidingWindowpublic Stream slidingWindow(BaseWindowedBolt.Duration windowDuration, BaseWindowedBolt.Duration slidingInterval, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of tuples which are aggregated results of a window which slides at duration ofslidingIntervaland completes a window atwindowDuration.- Parameters:
- windowDuration- represents window duration configuration
- slidingInterval- the time duration after which the window slides
- windowStoreFactory- intermediary tuple store for storing windowing tuples
- inputFields- projected fields for aggregator
- aggregator- aggregator to run on the window of tuples to compute the result and emit to the stream.
- functionFields- fields of values to emit with aggregation.
- Returns:
- the new stream with this operation.
 
- 
windowpublic Stream window(WindowConfig windowConfig, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns a stream of aggregated results based on the given window configuration which uses inmemory windowing tuple store.- Parameters:
- windowConfig- window configuration like window length and slide length.
- inputFields- input fields
- aggregator- aggregator to run on the window of tuples to compute the result and emit to the stream.
- functionFields- fields of values to emit with aggregation.
- Returns:
- the new stream with this operation.
 
- 
windowpublic Stream window(WindowConfig windowConfig, WindowsStoreFactory windowStoreFactory, Fields inputFields, Aggregator aggregator, Fields functionFields) Returns stream of aggregated results based on the given window configuration.- Parameters:
- windowConfig- window configuration like window length and slide length.
- windowStoreFactory- intermediary tuple store for storing tuples for windowing
- inputFields- input fields
- aggregator- aggregator to run on the window of tuples to compute the result and emit to the stream.
- functionFields- fields of values to emit with aggregation.
- Returns:
- the new stream with this operation.
 
- 
persistentAggregatepublic TridentState persistentAggregate(StateFactory stateFactory, CombinerAggregator agg, Fields functionFields) 
- 
persistentAggregatepublic TridentState persistentAggregate(StateSpec spec, CombinerAggregator agg, Fields functionFields) 
- 
persistentAggregatepublic TridentState persistentAggregate(StateFactory stateFactory, Fields inputFields, CombinerAggregator agg, Fields functionFields) 
- 
persistentAggregatepublic TridentState persistentAggregate(StateSpec spec, Fields inputFields, CombinerAggregator agg, Fields functionFields) 
- 
persistentAggregatepublic TridentState persistentAggregate(StateFactory stateFactory, ReducerAggregator agg, Fields functionFields) 
- 
persistentAggregatepublic TridentState persistentAggregate(StateSpec spec, ReducerAggregator agg, Fields functionFields) 
- 
persistentAggregatepublic TridentState persistentAggregate(StateFactory stateFactory, Fields inputFields, ReducerAggregator agg, Fields functionFields) 
- 
persistentAggregatepublic TridentState persistentAggregate(StateSpec spec, Fields inputFields, ReducerAggregator agg, Fields functionFields) 
- 
toStream- Specified by:
- toStreamin interface- IAggregatableStream
 
- 
getNameReturns the label applied to the stream.- Returns:
- the label applied to the stream.
 
- 
getOutputFields- Specified by:
- getOutputFieldsin interface- IAggregatableStream
 
 
-