Class PigConverter


  • public class PigConverter
    extends org.apache.pig.PigServer
    Extension from PigServer to convert Pig scripts into logical relational algebra plans and SQL statements.
    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.pig.PigServer

        org.apache.pig.PigServer.Graph
    • Constructor Summary

      Constructors 
      Modifier Constructor Description
      private PigConverter​(FrameworkConfig config, org.apache.pig.ExecType execType)  
    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      static PigConverter create​(FrameworkConfig config)  
      PigRelBuilder getBuilder()  
      java.lang.String getPigScript​(java.io.InputStream in, java.util.Map<java.lang.String,​java.lang.String> params)
      Gets a Pig script string from a file after doing param substitution.
      private java.util.List<RelNode> optimizePlans​(java.util.List<RelNode> originalRels, java.util.List<RelOptRule> rules)  
      private java.util.List<RelNode> pigPlan2Rel​(org.apache.pig.newplan.logical.relational.LogicalPlan pigPlan, boolean planRewrite, boolean usePigRules)  
      java.util.List<RelNode> pigQuery2Rel​(java.lang.String pigQuery)
      Parses a Pig script and converts it into relational algebra plans, optimizing the result.
      java.util.List<RelNode> pigQuery2Rel​(java.lang.String pigQuery, boolean planRewrite, boolean validate, boolean usePigRules)
      Parses a Pig script and converts it into relational algebra plans.
      java.util.List<RelNode> pigScript2Rel​(java.lang.String fileName, java.util.Map<java.lang.String,​java.lang.String> params, boolean planRewrite)
      Parses a Pig script and converts it into relational algebra plans.
      java.util.List<java.lang.String> pigToSql​(java.lang.String pigQuery, SqlDialect sqlDialect)
      Converts a Pig script to a list of SQL statements.
      private java.util.List<java.lang.String> pigToSql​(java.lang.String pigQuery, SqlWriter writer)
      Converts a Pig script to a list of SQL statements.
      private void resetPlannerRules​(RelOptPlanner planner, java.util.List<RelOptRule> rulesToSet)  
      • Methods inherited from class org.apache.pig.PigServer

        addPathToSkip, capacity, debugOff, debugOn, deleteFile, discardBatch, dumpSchema, dumpSchemaNested, executeBatch, executeBatch, existsFile, explain, explain, fileSize, getAliases, getAliasKeySet, getClonedGraph, getCurrentDAG, getExamples, getJobs, getLastRel, getLogicalPlanData, getPigContext, isBatchEmpty, isBatchOn, launchPlan, listPaths, mkdirs, openIterator, paramMapToList, parseAndBuild, printAliases, printHistory, registerCode, registerFunction, registerJar, registerQuery, registerQuery, registerScript, registerScript, registerScript, registerScript, registerScript, registerScript, registerScript, registerScript, registerStreamingCommand, renameFile, resetScope, setBatchOn, setDefaultParallel, setJobName, setJobPriority, setSkipParseInRegisterForBatch, setValidateEachStatement, shutdown, store, store
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
    • Field Detail

      • PIG_RULES

        private static final java.util.List<RelOptRule> PIG_RULES
      • TRANSFORM_RULES

        private static final java.util.List<RelOptRule> TRANSFORM_RULES
    • Constructor Detail

      • PigConverter

        private PigConverter​(FrameworkConfig config,
                             org.apache.pig.ExecType execType)
                      throws java.lang.Exception
        Throws:
        java.lang.Exception
    • Method Detail

      • pigQuery2Rel

        public java.util.List<RelNode> pigQuery2Rel​(java.lang.String pigQuery)
                                             throws java.io.IOException
        Parses a Pig script and converts it into relational algebra plans, optimizing the result.

        Equivalent to pigQuery2Rel(pigQuery, true, true, true).

        Parameters:
        pigQuery - Pig script
        Returns:
        A list of root nodes of the translated relational plans. Each of these root corresponds to a sink operator (normally a STORE command) in the Pig plan
        Throws:
        java.io.IOException - Exception during parsing or translating Pig
      • pigQuery2Rel

        public java.util.List<RelNode> pigQuery2Rel​(java.lang.String pigQuery,
                                                    boolean planRewrite,
                                                    boolean validate,
                                                    boolean usePigRules)
                                             throws java.io.IOException
        Parses a Pig script and converts it into relational algebra plans.
        Parameters:
        pigQuery - Pig script
        planRewrite - Whether to rewrite the translated plan
        validate - Whether to validate the Pig logical plan before doing translation
        usePigRules - Whether to use Pig Rules (see PigRelPlanner} to rewrite translated rel plan
        Returns:
        A list of root nodes of the translated relational plans. Each of these root corresponds to a sink operator (normally a STORE command) in the Pig plan
        Throws:
        java.io.IOException - Exception during parsing or translating Pig
      • getPigScript

        public java.lang.String getPigScript​(java.io.InputStream in,
                                             java.util.Map<java.lang.String,​java.lang.String> params)
                                      throws java.io.IOException
        Gets a Pig script string from a file after doing param substitution.
        Parameters:
        in - Pig script file
        params - Param sub map
        Throws:
        java.io.IOException
      • pigScript2Rel

        public java.util.List<RelNode> pigScript2Rel​(java.lang.String fileName,
                                                     java.util.Map<java.lang.String,​java.lang.String> params,
                                                     boolean planRewrite)
                                              throws java.io.IOException
        Parses a Pig script and converts it into relational algebra plans.
        Parameters:
        fileName - File name
        params - Param substitution map
        planRewrite - Whether to rewrite the translated plan
        Returns:
        A list of root nodes of the translated relational plans. Each of these root corresponds to a sink operator (normally a STORE command) in the Pig plan
        Throws:
        java.io.IOException - Exception during parsing or translating Pig
      • pigPlan2Rel

        private java.util.List<RelNode> pigPlan2Rel​(org.apache.pig.newplan.logical.relational.LogicalPlan pigPlan,
                                                    boolean planRewrite,
                                                    boolean usePigRules)
                                             throws org.apache.pig.impl.logicalLayer.FrontendException
        Throws:
        org.apache.pig.impl.logicalLayer.FrontendException
      • pigToSql

        public java.util.List<java.lang.String> pigToSql​(java.lang.String pigQuery,
                                                         SqlDialect sqlDialect)
                                                  throws java.io.IOException
        Converts a Pig script to a list of SQL statements.
        Parameters:
        pigQuery - Pig script
        sqlDialect - Dialect of SQL language
        Throws:
        java.io.IOException - Exception during parsing or translating Pig
      • pigToSql

        private java.util.List<java.lang.String> pigToSql​(java.lang.String pigQuery,
                                                          SqlWriter writer)
                                                   throws java.io.IOException
        Converts a Pig script to a list of SQL statements.
        Parameters:
        pigQuery - Pig script
        writer - The SQL writer to decide dialect and format of SQL statements
        Throws:
        java.io.IOException - Exception during parsing or translating Pig
      • optimizePlans

        private java.util.List<RelNode> optimizePlans​(java.util.List<RelNode> originalRels,
                                                      java.util.List<RelOptRule> rules)
      • resetPlannerRules

        private void resetPlannerRules​(RelOptPlanner planner,
                                       java.util.List<RelOptRule> rulesToSet)