Tech News
← Home  ·  All topics

Parquet

4 GoKawiil briefs on this topic · RSS for parquet

DuckDB adds DuckLake extension for SQL-based lakehouse storage

DuckDB released a DuckLake extension, enabling DuckDB to read and write an open lakehouse table format that stores metadata in a catalog database and data as Parquet files. Users can install it via INSTALL ducklake, attach a DuckLake database with standard ATTACH syntax, and then create, query and modify tables using regular SQL, including updates, time travel queries, and schema evolution.

Analysis argues Amazon S3's disk-era design is becoming outdated as SSD prices fall

A technical essay examines how Amazon S3 and similar cloud object stores, built around spinning-disk economics from two decades ago, now underpin most modern data architectures including data lakes, Kafka alternatives, and vector databases. The author notes S3's per-request throughput and latency limits stem from disk physics, forcing developers to add caching layers, batch writes, and separate metadata stores to work around them. The piece highlights that SSD prices have fallen steadily since S3's 2005 launch, narrowing the cost gap with hard disks to about 3x.

DataHaskell/Dataframe adds native Parquet writer

The DataHaskell/Dataframe library now includes a writeParquet function that lets users export a dataframe directly to a Parquet file using default row group and page sizes. A writeParquetWithOptions variant is also provided for developers who want finer control over those settings.

Developer shows Parquet cube + range requests can replace a database for dashboards

A blogger built a customer analytics drilldown dashboard using only a 40MB pre-aggregated Parquet 'data cube' stored in R2, an 18KB browser-based Hyparquet reader, and HTTP range requests, avoiding any database or query engine. The demo rolled up NYC's 34-million-row 311 service request dataset into a compact cube that supports fast filtering by reading only small slices of the file at a time.