Installation#

Using the install script#

Opaque SQL has an install script (opaque-sql/scripts/install.sh) to take care of installing/downloading everything for Opaque SQL to work immediately. To use this, first clone/fork the repository, then run

cd opaque-sql
source ./scripts/install.sh

Note that the source is necessary to properly set environment variables Opaque SQL needs. If alternatively you don’t want to use the install script, see the section on installing dependencies manually below. For testing, skip to running tests.

Installing dependencies manually#

After downloading the Opaque codebase, install necessary dependencies as follows.

  1. Install dependencies and the OpenEnclave SDK. We currently support OE version 0.17.1 (so please install with open-enclave=0.17.1) and Ubuntu 18.04.

    # For Ubuntu 18.04:
    sudo apt -y install wget build-essential openjdk-8-jdk python libssl-dev libmbedtls-dev
    pip3 install grpcio grpcio-tools # Needed for PySpark listener
    
    # Install a newer version of CMake (3.15)
    wget https://github.com/Kitware/CMake/releases/download/v3.15.6/cmake-3.15.6-Linux-x86_64.sh
    sudo bash cmake-3.15.6-Linux-x86_64.sh --skip-license --prefix=/usr/local
    rm cmake-3.15.6-Linux-x86_64.sh
    
    # Install Spark 3.1.1 (if not already done)
    wget https://archive.apache.org/dist/spark/spark-3.1.1/spark-3.1.1-bin-hadoop2.7.tgz
    tar xvf spark-3.1.1*
    sudo mkdir /opt/spark
    sudo mv spark-3.1.1*/* /opt/spark
    rm -rf spark-3.1.1*
    sudo mkdir /opt/spark/work
    sudo chmod -R a+wx /opt/spark/work
    
  2. Change into the Opaque root directory and edit Opaque’s environment variables in opaqueenv (including Spark configurations) if desired. Export Opaque SQL and Open Enclave environment variables via

    source opaqueenv
    source /opt/openenclave/share/openenclave/openenclaverc
    

    By default, Opaque runs in hardware mode (environment variable MODE=HARDWARE). If you do not have a machine with a hardware enclave but still wish to test out Opaque’s functionality locally, then set export MODE=SIMULATE.

Running tests#

  1. To run the Opaque tests:

    build/sbt test
    
  2. Alternatively, to run tests and generate coverage reports:

    build/sbt clean coverage test
    build/sbt coverageReport
    

Additional configurations for running on a Spark cluster#

Opaque SQL needs three Spark properties to be set:

  • spark.executor.instances=n (n is usually the number of machines in the cluster)

  • spark.task.maxFailures=10 (attestation uses Sparks fault tolerance property)

  • spark.driver.defaultJavaOptions="-Dscala.color" (if querying with MC2 Client)

These properties can be be set in a custom configuration file, the default being located at ${SPARK_HOME}/conf/spark-defaults.conf, or as a spark-submit or spark-shell argument: --conf <key>=<value>. For more details on running a Spark cluster, see the Spark documentation