distributed_computing:data_processing:formats

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
distributed_computing:data_processing:formats [2019/10/28 21:45] – phreazerdistributed_computing:data_processing:formats [2019/10/28 21:48] (current) – [Parquet] phreazer
Line 13: Line 13:
   * First metadata should be read to find column chunks   * First metadata should be read to find column chunks
   * Non-nested schema: Nulls encoded with run-length encoding (0, 1000 times)   * Non-nested schema: Nulls encoded with run-length encoding (0, 1000 times)
 +
 +==== Encodings ====
 +
 +  * Plain = 0
 +  * Dictionary encoding (PLAIN_DICTIONARY = 2 and RLE_DICTIONARY = 8)
 +  * Run Lenght Encoding / Bit-Packing Hybrid (RLE = 3)
 +  * ...
 +
 +https://github.com/apache/parquet-format/blob/master/Encodings.md
 +
  
 ==== Implementations ==== ==== Implementations ====
  • distributed_computing/data_processing/formats.1572299129.txt.gz
  • Last modified: 2019/10/28 21:45
  • by phreazer