Package org.rumbledb.runtime.dataframe

Interface RuntimeDataFrame<T>

Type Parameters:
T - the logical runtime value represented by each row
All Superinterfaces:
Serializable
All Known Implementing Classes:
FlworDataFrame, HomogeneousItemDataFrame
public interface RuntimeDataFrame<T> extends Serializable
A Spark DataFrame whose rows represent runtime values of type T.

Spark stores the physical representation as Row; implementations define how rows are mapped back to the logical runtime type.

  • Method Summary

    Modifier and Type
    Method
    Description
    org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>
    getDataFrame()
    Returns the underlying physical Spark DataFrame.
    default List<T>
    toList(ExceptionMetadata metadata)
    Materializes this DataFrame as logical runtime values.
    org.apache.spark.api.java.JavaRDD<T>
    toRDD(ExceptionMetadata metadata)
    Converts this DataFrame to its logical runtime representation.

  • Method Details

    • getDataFrame

      org.apache.spark.sql.Dataset<org.apache.spark.sql.Row> getDataFrame()
      Returns the underlying physical Spark DataFrame.

    • toRDD

      org.apache.spark.api.java.JavaRDD<T> toRDD(ExceptionMetadata metadata)
      Converts this DataFrame to its logical runtime representation.
      Parameters:
      metadata - query metadata used if a row cannot be decoded
      Returns:
      an RDD of logical runtime values

    • toList

      default List<T> toList(ExceptionMetadata metadata)
      Materializes this DataFrame as logical runtime values.
      Parameters:
      metadata - query metadata used if a row cannot be decoded
      Returns:
      a list of logical runtime values