libdplyr

A Rust-based transpiler that converts R dplyr syntax to SQL queries.

Overview

libdplyr enables R users to write database queries using familiar dplyr syntax and converts them to efficient SQL for execution. It supports multiple SQL dialects (PostgreSQL, MySQL, SQLite, DuckDB) for use across various database environments.

✨ Key Features

• dplyr Syntax Support: Core support for select(), distinct(), filter(), mutate(), arrange(), group_by(), summarise(), and limited count()/tally()
• Pipeline Operations: Chain operations using the %>% pipe operator
• Multiple Dialects: PostgreSQL, MySQL, SQLite, DuckDB
• Performance: High-performance Rust implementation
• Dual Mode: Use as a Rust library or standalone CLI tool

🚀 Quick Start

Linux/macOS:

curl -sSL https://raw.githubusercontent.com/mrchypark/libdplyr/main/install.sh | bash

Windows (PowerShell):

Irm https://raw.githubusercontent.com/mrchypark/libdplyr/main/install.ps1 | iex

Try it out:

echo "select(name, age) %>% filter(age > 18)" | libdplyr --pretty

Note: For detailed installation options, troubleshooting, and platform support, see the Installation Guide.

Usage

As a CLI Tool

The most efficient way to use libdplyr is through stdin/stdout pipelines:

# Basic usage
echo "select(name, age) %>% filter(age > 18)" | libdplyr

# Specify dialect (postgres, mysql, sqlite, duckdb)
echo "select(name)" | libdplyr --dialect mysql

# Output formatting
echo "select(name)" | libdplyr --pretty
echo "select(name)" | libdplyr --json
echo "select(name)" | libdplyr --compact

As a Rust Library

use libdplyr::{Transpiler, PostgreSqlDialect};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let transpiler = Transpiler::new(Box::new(PostgreSqlDialect::new()));
    let dplyr_code = "select(name, age) %>% filter(age > 18)";
    
    let sql = transpiler.transpile(dplyr_code)?;
    println!("{}", sql);
    Ok(())
}

DuckDB Parser Override

DuckDB 1.5.x에서는 SET allow_parser_override_extension = 'fallback';로 dplyr pipeline을 네이티브 AST로 변환할 수 있습니다. parser override는 호출 세션의 임시 테이블과 트랜잭션을 그대로 사용합니다. 암시적 pipeline의 dplyr_pipe_syntax는 parser override API에 ClientContext가 없는 제약 때문에 DB-global 값만 읽으며, SET GLOBAL dplyr_pipe_syntax = 'native';처럼 지정합니다. session 값은 암시적 pipeline에 영향을 주지 않으며, 연결별 문법이 필요하면 dplyr(query, mode)를 사용합니다. 설정이 없으면 DPLYR_PIPE_SYNTAX 환경 변수(기본값 magrittr)를 사용합니다. 자세한 호환성 및 설정 규칙은 submodule compatibility 문서를 참고하세요.

📋 Supported Functions

libdplyr supports a wide range of dplyr verbs and R functions. Exact partial-support and rejection boundaries are documented in the dplyr syntax support matrix.

Core Verbs

Function	Description	Example
select()	Select/rename columns	select(id, name)
distinct()	Keep distinct rows or identifier columns	distinct(dept, role)
filter()	Filter rows	filter(age > 18)
mutate()	Create/modify columns	mutate(total = price * qty)
rename()	Rename columns	rename(new = old)
arrange()	Sort rows	arrange(desc(date))
group_by()	Group rows	group_by(dept)
summarise()	Aggregate data	summarise(avg = mean(val))
count() / tally()	Count rows by identifier-only keys or current groups	count(dept)
*_join()	Equality joins (inner, left, etc.)	left_join(other, by=c("id", "left"="right"))
Set Ops	union, intersect, setdiff	union(other)

Helper Functions

• Aggregation: mean, sum, min, max, n, n_distinct, count, median, mode
• Window: row_number, rank, lead, lag, ntile
• Math: abs, sqrt, round, floor, log, exp
• String: tolower, toupper, substr, trimws
• Logic: ifelse, case_when, between, is.na, coalesce

Examples

PostgreSQL

let transpiler = Transpiler::new(Box::new(PostgreSqlDialect::new()));
let sql = transpiler.transpile("select(name) %>% filter(age > 18)")?;
// SELECT "name" FROM "data" WHERE "age" > 18

Complex Pipeline

let code = r#"
    select(dept, salary) %>%
    filter(salary > 50000) %>%
    group_by(dept) %>%
    summarise(avg_sal = mean(salary)) %>%
    arrange(desc(avg_sal))
"#;

Error Handling & Troubleshooting

libdplyr provides detailed error codes:

• Exit 0: Success
• Exit 4: Validation Error (syntax issues)
• Exit 5: Transpilation Error (generation failed)

Debug Mode:

libdplyr --verbose --debug

For more troubleshooting details, see INSTALL.md.

Performance

libdplyr is optimized for speed using Rust's zero-cost abstractions. Run benchmarks:

cargo bench

Development

# Setup
git clone https://github.com/mrchypark/libdplyr.git
cd libdplyr
cargo build

# Test
cargo test

Support

• Documentation: docs.rs/libdplyr
• Installation: INSTALL.md
• Issues: GitHub Issues

License

MIT License. See LICENSE.