Package org.rumbledb.runtime.flwor.clauses

Class LetClauseIterator

java.lang.Object
org.rumbledb.runtime.plan.RuntimePlan<FlworTuple>
org.rumbledb.runtime.TupleRuntimePlan
org.rumbledb.runtime.flwor.clauses.LetClauseIterator
All Implemented Interfaces:
Serializable, DataFrameRuntimePlan<FlworTuple>, LocalRuntimePlan<FlworTuple>, NativeQueryRuntimePlan
public class LetClauseIterator extends TupleRuntimePlan implements DataFrameRuntimePlan<FlworTuple>
See Also:

  • Constructor Details

  • Method Details

    • createNativeCursor

      public Cursor<FlworTuple> createNativeCursor(DynamicContext context)
      Description copied from interface: LocalRuntimePlan
      Creates an independent cursor for one native local evaluation.
      Specified by:
      createNativeCursor in interface LocalRuntimePlan<FlworTuple>
      Specified by:
      createNativeCursor in class TupleRuntimePlan
      Parameters:
      context - the dynamic context for that evaluation
      Returns:
      an unopened cursor

    • createNativeDataFrame

      public FlworDataFrame createNativeDataFrame(DynamicContext context)
      Description copied from interface: DataFrameRuntimePlan
      Builds the DataFrame for one evaluation.
      Specified by:
      createNativeDataFrame in interface DataFrameRuntimePlan<FlworTuple>
      Parameters:
      context - the dynamic context for that evaluation
      Returns:
      the resulting DataFrame

    • getDataFrameAsJoin

      public FlworDataFrame getDataFrameAsJoin(DynamicContext context, Map<Name,DynamicContext.VariableDependency> parentProjection, FlworDataFrame childDF)

    • isExpressionIndependentFromInputTuple

      public static boolean isExpressionIndependentFromInputTuple(ItemRuntimePlan sequenceIterator, TupleRuntimePlan tupleIterator)

    • getDynamicContextVariableDependencies

      public Map<Name,DynamicContext.VariableDependency> getDynamicContextVariableDependencies()
      Description copied from class: TupleRuntimePlan
      Variable dependencies are variables that MUST be provided by the parent clause in the dynamic context for successful execution of this clause. These variables are: 1. All variables that the expression of the clause depends on (recursive call of getVariableDependencies on the expression) 2. Except those variables bound in the current FLWOR (obtained from the auxiliary method getVariablesBoundInCurrentFLWORExpression), because those are provided in the Tuples 3. Plus (recursively calling getVariableDependencies) all the Variable Dependencies of the child clause if it exists.
      Overrides:
      getDynamicContextVariableDependencies in class TupleRuntimePlan
      Returns:
      a map of variable names to dependencies (FULL, COUNT, ...) that this clause needs to obtain from the dynamic context.

    • getOutputTupleVariableNames

      public Set<Name> getOutputTupleVariableNames()
      Description copied from class: TupleRuntimePlan
      Returns the output tuple variable names. These variables can be removed from the dependencies of expressions in ascendent (subsequent) clauses, because their values are provided in the tuples rather than the dynamic context object.
      Overrides:
      getOutputTupleVariableNames in class TupleRuntimePlan
      Returns:
      the set of variable names that are bound by descendant clauses.

    • print

      public void print(StringBuilder buffer, int indent)
      Overrides:
      print in class TupleRuntimePlan

    • getInputTupleVariableDependencies

      public Map<Name,DynamicContext.VariableDependency> getInputTupleVariableDependencies(Map<Name,DynamicContext.VariableDependency> parentProjection)
      Description copied from class: TupleRuntimePlan
      Builds the DataFrame projection that this clause needs to receive from its child clause. The intent is that the result of this method is forwarded to the child clause in getDataFrame() so it can optimize some values away. Invariant: all keys in getInputTupleVariableDependencies(...) MUST be output tuple variables, i.e., appear in this.child.getOutputTupleVariableNames()
      Specified by:
      getInputTupleVariableDependencies in class TupleRuntimePlan
      Parameters:
      parentProjection - the projection needed by the parent clause.
      Returns:
      the projection needed by this clause.

    • bindLetVariableInDataFrame

      public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> bindLetVariableInDataFrame(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, Name newVariableName, SequenceType sequenceType, ItemRuntimePlan newVariableExpression, DynamicContext context, List<Name> variablesInInputTuple, Map<Name,DynamicContext.VariableDependency> outputTupleVariableDependencies, boolean hash, RumbleConfiguration conf)
      Extends a DataFrame with a new column obtained from the evaluation of an expression for each tuple.
      Parameters:
      dataFrame - the DataFrame to extend
      newVariableName - the name of the new column (variable)
      sequenceType - the sequence type of the new bound item, not used in case of hash
      newVariableExpression - the expression to evaluate
      context - the context (in addition to each tuple) in which to evaluation the expression
      variablesInInputTuple - the name of the variables that can be found in the input tuple (as opposed to those in the context)
      outputTupleVariableDependencies - the dependencies to project to (possibly null to keep everything).
      hash - whether or not to compute single-item hashes rather than the actual serialized sequences of items.
      Returns:
      the DataFrame with the new column

    • registerLetClauseUDF

      public static boolean registerLetClauseUDF(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, ItemRuntimePlan newVariableExpression, DynamicContext context, org.apache.spark.sql.types.StructType inputSchema, List<FlworDataFrameColumn> UDFcolumns, SequenceType sequenceType)

    • tryNativeQuery

      public static org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> tryNativeQuery(org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> dataFrame, Name newVariableName, ItemRuntimePlan iterator, List<FlworDataFrameColumn> allColumns, org.apache.spark.sql.types.StructType inputSchema, DynamicContext context)
      Try to generate the native query for the let clause and run it, if successful return the resulting dataframe, otherwise it returns null
      Parameters:
      dataFrame - input dataframe for the query
      newVariableName - name of the new bound variable
      iterator - let variable assignment expression iterator
      allColumns - other columns required in following clauses
      inputSchema - input schema of the dataframe
      context - current dynamic context of the dataframe
      Returns:
      resulting dataframe of the let clause if successful, null otherwise

    • containsClause

      public boolean containsClause(FLWOR_CLAUSES kind)
      Description copied from class: TupleRuntimePlan
      Says whether or not the clause and its descendants include a clause of the specified kind.
      Specified by:
      containsClause in class TupleRuntimePlan
      Parameters:
      kind - the kind of clause to test for.
      Returns:
      true if there is one. False otherwise.

    • isSparkJobNeeded

      public boolean isSparkJobNeeded()
      Says whether this expression evaluation triggers a Spark job.
      Specified by:
      isSparkJobNeeded in class TupleRuntimePlan
      Returns:
      true if the execution triggers a Spark, false otherwise, null if undetermined yet.

    • generateNativeQuery

      public NativeClauseContext generateNativeQuery(NativeClauseContext nativeClauseContext)
      Description copied from class: TupleRuntimePlan
      This function generate (if possible) a native spark-sql query that maps the inner working of the iterator
      Specified by:
      generateNativeQuery in interface NativeQueryRuntimePlan
      Overrides:
      generateNativeQuery in class TupleRuntimePlan
      Parameters:
      nativeClauseContext - context information to generate the native query
      Returns:
      a native clause context with the spark-sql native query to get an equivalent result of the iterator, or [NativeClauseContext.NoNativeQuery] if it is not possible