Package org.apache.storm.hdfs.blobstore
Class HdfsClientBlobStore
java.lang.Object
org.apache.storm.blobstore.ClientBlobStore
org.apache.storm.hdfs.blobstore.HdfsClientBlobStore
- All Implemented Interfaces:
- AutoCloseable,- Shutdownable
Client to access the HDFS blobStore. At this point, this is meant to only be used by the
 supervisor.  Don't trust who the client says they are so pass null for all Subjects.
 
The HdfsBlobStore implementation takes care of the null Subjects. It assigns Subjects based on what hadoop says who the users are. These users must be configured accordingly in the SUPERVISOR_ADMINS for ACL validation and for the supervisors to download the blobs. This API is only used by the supervisor in order to talk directly to HDFS.
- 
Nested Class SummaryNested classes/interfaces inherited from class org.apache.storm.blobstore.ClientBlobStoreClientBlobStore.WithBlobstore
- 
Constructor SummaryConstructors
- 
Method SummaryModifier and TypeMethodDescriptionvoidclose()createBlobToExtend(String key, SettableBlobMeta meta) Client facing API to create a blob.voidCreates state inside a zookeeper.voiddeleteBlob(String key) Client facing API to delete a blob.Client facing API to read a blob.getBlobMeta(String key) Client facing API to read the metadata information.intgetBlobReplication(String key) Client facing API to read the replication of a blob.longClient facing API to get the last update time of existing blobs in a blobstore.booleanisRemoteBlobExists(String blobKey) Decide if the blob is deleted from cluster.listKeys()List keys.voidSets up the client API by parsing the configs.voidsetBlobMetaToExtend(String key, SettableBlobMeta meta) Client facing API to set the metadata for a blob.booleansetClient(Map<String, Object> conf, NimbusClient client) Client facing API to set a nimbus client.voidshutdown()updateBlob(String key) Client facing API to update a blob.intupdateBlobReplication(String key, int replication) Client facing API to update the replication of a blob.Methods inherited from class org.apache.storm.blobstore.ClientBlobStorecreateBlob, setBlobMeta, withConfiguredClient
- 
Constructor Details- 
HdfsClientBlobStorepublic HdfsClientBlobStore()
 
- 
- 
Method Details- 
prepareDescription copied from class:ClientBlobStoreSets up the client API by parsing the configs.- Specified by:
- preparein class- ClientBlobStore
- Parameters:
- conf- The storm conf containing the config details
 
- 
createBlobToExtendpublic AtomicOutputStream createBlobToExtend(String key, SettableBlobMeta meta) throws AuthorizationException, KeyAlreadyExistsException Description copied from class:ClientBlobStoreClient facing API to create a blob.- Specified by:
- createBlobToExtendin class- ClientBlobStore
- Parameters:
- key- blob key name
- meta- contains ACL information
- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
- AuthorizationException
- KeyAlreadyExistsException
 
- 
updateBlobpublic AtomicOutputStream updateBlob(String key) throws AuthorizationException, KeyNotFoundException Description copied from class:ClientBlobStoreClient facing API to update a blob.- Specified by:
- updateBlobin class- ClientBlobStore
- Parameters:
- key- blob key name
- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
- AuthorizationException
- KeyNotFoundException
 
- 
getBlobMetaDescription copied from class:ClientBlobStoreClient facing API to read the metadata information.- Specified by:
- getBlobMetain class- ClientBlobStore
- Parameters:
- key- blob key name
- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
- AuthorizationException
- KeyNotFoundException
 
- 
isRemoteBlobExistsDescription copied from class:ClientBlobStoreDecide if the blob is deleted from cluster.- Specified by:
- isRemoteBlobExistsin class- ClientBlobStore
- Parameters:
- blobKey- blob key
- Throws:
- AuthorizationException
 
- 
setBlobMetaToExtendpublic void setBlobMetaToExtend(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException Description copied from class:ClientBlobStoreClient facing API to set the metadata for a blob.- Specified by:
- setBlobMetaToExtendin class- ClientBlobStore
- Parameters:
- key- blob key name
- meta- contains ACL information
- Throws:
- AuthorizationException
- KeyNotFoundException
 
- 
deleteBlobDescription copied from class:ClientBlobStoreClient facing API to delete a blob.- Specified by:
- deleteBlobin class- ClientBlobStore
- Parameters:
- key- blob key name
- Throws:
- AuthorizationException
- KeyNotFoundException
 
- 
getBlobDescription copied from class:ClientBlobStoreClient facing API to read a blob.- Specified by:
- getBlobin class- ClientBlobStore
- Parameters:
- key- blob key name
- Returns:
- an InputStream to read the metadata for a blob
- Throws:
- AuthorizationException
- KeyNotFoundException
 
- 
listKeysDescription copied from class:ClientBlobStoreList keys.- Specified by:
- listKeysin class- ClientBlobStore
- Returns:
- Iterator for a list of keys currently present in the blob store.
 
- 
getBlobReplicationDescription copied from class:ClientBlobStoreClient facing API to read the replication of a blob.- Specified by:
- getBlobReplicationin class- ClientBlobStore
- Parameters:
- key- blob key name
- Returns:
- int indicates the replication factor of a blob
- Throws:
- AuthorizationException
- KeyNotFoundException
 
- 
updateBlobReplicationpublic int updateBlobReplication(String key, int replication) throws AuthorizationException, KeyNotFoundException Description copied from class:ClientBlobStoreClient facing API to update the replication of a blob.- Specified by:
- updateBlobReplicationin class- ClientBlobStore
- Parameters:
- key- blob key name
- replication- int indicates the replication factor a blob has to be set
- Returns:
- int indicates the replication factor of a blob
- Throws:
- AuthorizationException
- KeyNotFoundException
 
- 
setClientDescription copied from class:ClientBlobStoreClient facing API to set a nimbus client.- Specified by:
- setClientin class- ClientBlobStore
- Parameters:
- conf- storm conf
- client- NimbusClient
- Returns:
- indicates where the client connection has been setup.
 
- 
createStateInZookeeperDescription copied from class:ClientBlobStoreCreates state inside a zookeeper. Required for blobstore to write to zookeeper when Nimbus HA is turned on in order to maintain state consistency.- Specified by:
- createStateInZookeeperin class- ClientBlobStore
 
- 
shutdownpublic void shutdown()
- 
closepublic void close()- Specified by:
- closein interface- AutoCloseable
- Specified by:
- closein class- ClientBlobStore
 
- 
getRemoteBlobstoreUpdateTimeDescription copied from class:ClientBlobStoreClient facing API to get the last update time of existing blobs in a blobstore. This is only required for use on supervisors.- Specified by:
- getRemoteBlobstoreUpdateTimein class- ClientBlobStore
- Returns:
- the timestamp of when the blobstore was last updated. -1L if the blobstore does not support this.
- Throws:
- IOException
 
 
-