analytics

Lance: 100x Faster Data Access

Name: Lance: 100x Faster Data Access
Author: lance-format

Faster data analysis for founders, compatible with Pandas and PyTorch, with 6.8k+ GitHub stars

6,782 stars753 forksRustQuality 8/10Updated 7/13/2026100% free · open source

What it does

Lance enables founders to accelerate data analysis by converting Parquet files to its Open Lakehouse Format, allowing for 100x faster random access, vector index, and data versioning.

Install / run

cargo install lance

When to use it

•When working with large datasets and needing faster data access
•When integrating with popular data analysis libraries like Pandas, PyTorch, or DuckDB
•When requiring data versioning and efficient random access for machine learning model training

Quick start

1Convert a Parquet file to Lance format using the command: `lance convert -i input.parquet -o output.lance`
2Verify the conversion by checking the output file size and format
3Use the Lance format with Pandas by reading the converted file: `import pandas as pd; df = pd.read_lance('output.lance')`
4Utilize Lance's vector index by creating an index on a specific column: `lance index -i output.lance -c column_name`
5Query the indexed data using a library like PyTorch or DuckDB

Ready-to-paste prompt

Convert a sample Parquet file to Lance format and read it with Pandas: `lance convert -i sample.parquet -o sample.lance; import pandas as pd; df = pd.read_lance('sample.lance'); print(df.head())`

Heads up: Ensure you have Rust and Cargo installed on your system, as Lance is built with Rust and relies on the Cargo package manager for installation and dependency management.

Saves to your device