Package org.rumbledb.api

Class SequenceWriter

java.lang.Object

org.rumbledb.api.SequenceWriter

public class SequenceWriter
extends Object

Helper class to configure and materialize the output of a SequenceOfItems. This class is effectively immutable: all configuration methods such as mode(), format(), option(), etc. return a new SequenceWriter instance instead of mutating the current one. There are two mutually exclusive internal modes:

• DataFrame mode: dataFrameWriter != null and mode == null. In this case, the sequence can be represented as a Spark Dataset / DataFrameWriter and is written using Spark's native writers (json/csv/parquet/...).
• RDD mode: dataFrameWriter == null and mode != null. In this case, the sequence is serialized item-by-item via Serializer and saved as text files.

Output formats (json/csv/parquet/...) select a file writer, independently of the W3C serialization method. The special output format serialize writes the entire sequence as one string using its serialization parameters. serialize-each-item serializes each item independently and separates the resulting strings with newlines.

Method Summary

Modifier and Type	Method	Description
SequenceWriter	bucketBy(int numBuckets, String colName, String... colNames)
void	csv(String path)
SequenceWriter	format(String source)
Serializer	getSerializer()
void	insertInto(String tableName)
void	json(String path)
SequenceWriter	mode(String saveMode)
SequenceWriter	mode(org.apache.spark.sql.SaveMode saveMode)
SequenceWriter	option(String key, boolean value)
SequenceWriter	option(String key, double value)
SequenceWriter	option(String key, long value)
SequenceWriter	option(String key, String value)
SequenceWriter	options(Map<String,String> options)
SequenceWriter	options(org.apache.spark.sql.util.CaseInsensitiveStringMap options)
void	orc(String path)
void	parquet(String path)
SequenceWriter	partitionBy(String... colNames)
void	save()
void	save(String path)
void	saveAsTable(String tableName)
SequenceWriter	sortBy(String colName, String... colNames)
void	text(String path)
void	tyson(String path)
void	yaml(String path)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

mode

public SequenceWriter mode(String saveMode)

mode

public SequenceWriter mode(org.apache.spark.sql.SaveMode saveMode)

format

public SequenceWriter format(String source)

option

public SequenceWriter option(String key,
 String value)

option

public SequenceWriter option(String key,
 boolean value)

option

public SequenceWriter option(String key,
 long value)

option

public SequenceWriter option(String key,
 double value)

options

public SequenceWriter options(Map<String,String> options)

options

public SequenceWriter options(org.apache.spark.sql.util.CaseInsensitiveStringMap options)

partitionBy

public SequenceWriter partitionBy(String... colNames)

bucketBy

public SequenceWriter bucketBy(int numBuckets,
 String colName,
 String... colNames)

sortBy

public SequenceWriter sortBy(String colName,
 String... colNames)

save

public void save(String path)

getSerializer

public Serializer getSerializer()

save

public void save()

insertInto

public void insertInto(String tableName)

saveAsTable

public void saveAsTable(String tableName)

tyson

public void tyson(String path)

yaml

public void yaml(String path)

json

public void json(String path)

parquet

public void parquet(String path)

text

public void text(String path)

orc

public void orc(String path)

csv

public void csv(String path)