public class HadoopJobHistoryLoader extends LoadFunc
| Modifier and Type | Class and Description |
|---|---|
static class |
HadoopJobHistoryLoader.HadoopJobHistoryInputFormat |
static class |
HadoopJobHistoryLoader.HadoopJobHistoryReader |
static class |
HadoopJobHistoryLoader.JobHistoryPathFilter |
static class |
HadoopJobHistoryLoader.JobKeys
Job Keys
|
static class |
HadoopJobHistoryLoader.MRJobInfo |
| Modifier and Type | Field and Description |
|---|---|
static String |
MAP_INPUT_RECORDS |
static String |
REDUCE_INPUT_RECORDS |
static String |
TASK_COUNTER_GROUP |
| Constructor and Description |
|---|
HadoopJobHistoryLoader() |
| Modifier and Type | Method and Description |
|---|---|
org.apache.hadoop.mapreduce.InputFormat |
getInputFormat()
This will be called during planning on the front end.
|
Tuple |
getNext()
Retrieves the next tuple to be processed.
|
static void |
parseJobHistory(org.apache.hadoop.conf.Configuration jobConf,
JobInfo jobInfo,
HadoopJobHistoryLoader.MRJobInfo value) |
static Map<String,String> |
parseJobXML(InputStream in) |
void |
prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader,
PigSplit split)
Initializes LoadFunc for reading data.
|
void |
setLocation(String location,
org.apache.hadoop.mapreduce.Job job)
Communicate to the loader the location of the object(s) being loaded.
|
getAbsolutePath, getCacheFiles, getLoadCaster, getPathStrings, getShipFiles, join, relativeToAbsolutePath, setUDFContextSignature, warnpublic static final String TASK_COUNTER_GROUP
public static final String MAP_INPUT_RECORDS
public static final String REDUCE_INPUT_RECORDS
public org.apache.hadoop.mapreduce.InputFormat getInputFormat()
throws IOException
LoadFuncgetInputFormat in class LoadFuncIOException - if there is an exception during InputFormat
constructionpublic Tuple getNext() throws IOException
LoadFuncgetNext in class LoadFuncIOException - if there is an exception while retrieving the next
tuplepublic void prepareToRead(org.apache.hadoop.mapreduce.RecordReader reader,
PigSplit split)
throws IOException
LoadFuncprepareToRead in class LoadFuncreader - RecordReader to be used by this instance of the LoadFuncsplit - The input PigSplit to processIOException - if there is an exception during initializationpublic void setLocation(String location, org.apache.hadoop.mapreduce.Job job) throws IOException
LoadFuncLoadFunc.relativeToAbsolutePath(String, Path). Implementations
should use this method to communicate the location (and any other information)
to its underlying InputFormat through the Job object.
This method will be called in the frontend and backend multiple times. Implementations
should bear in mind that this method is called multiple times and should
ensure there are no inconsistent side effects due to the multiple calls.setLocation in class LoadFunclocation - Location as returned by
LoadFunc.relativeToAbsolutePath(String, Path)job - the Job object
store or retrieve earlier stored information from the UDFContextIOException - if the location is not valid.public static void parseJobHistory(org.apache.hadoop.conf.Configuration jobConf,
JobInfo jobInfo,
HadoopJobHistoryLoader.MRJobInfo value)
public static Map<String,String> parseJobXML(InputStream in)
Copyright © 2007-2012 The Apache Software Foundation