Welcome to Opaque SQL’s documentation!

Opaque SQL is a package for Apache Spark SQL that enables encryption for DataFrames using the OpenEnclave framework. The aim is to enable analytics on sensitive data in an untrusted cloud. Once the contents of a DataFrame are encrypted, subsequent operations will run within hardware enclaves (such as Intel SGX).

This project is based on our NSDI 2017 paper [1]. The oblivious execution mode is currently not included in this release.

This is an alpha preview of Opaque SQL, and the software is still in active development. It currently has the following limitations:

• Not all Spark SQL operations are supported (see the list of supported functionalities). UDFs must be implemented in C++

• Computation integrity verification (section 4.2 of the NSDI paper) is currently work in progress.

[1] Wenting Zheng, Ankur Dave, Jethro Beekman, Raluca Ada Popa, Joseph Gonzalez, and Ion Stoica. Opaque: An Oblivious and Encrypted Distributed Analytics Platform. NSDI 2017, March 2017.

Table of contents

• Installation
  • Using the install script
  • Installing dependencies manually
  • Running tests
  • Additional configurations for running on a Spark cluster
• Query submission via the MC² Client
  • Starting the gRPC Listeners
  • Using the MC² Client
• Query submission via the Spark Driver
  • Starting Opaque SQL
  • Encrypting a DataFrame with the Driver
• Usage
  • Saving a DataFrame
  • Using the DataFrame interface
  • Using the SQL interface
• Supported functionalities
  • SQL interface
  • DataFrame interface
  • User-Defined Functions (UDFs)
• Benchmarking
  • Running on your own cluster
  • Our TPC-H results
• Contributing
  • Submitting a pull request
  • Debugging your changes