Skip to content

Latest commit

 

History

History
51 lines (41 loc) · 2.23 KB

EC2.md

File metadata and controls

51 lines (41 loc) · 2.23 KB

Instructions on running ml-matrix on EC2

  1. Launch a cluster using Spark's EC2 scripts

  2. After the cluster launches, ssh to the master and clone the matrix-bench repository.

  3. Run cd matrix-bench; bash setup-ec2.sh and wait for OpenBLAS to build.

  4. After OpenBLAS build completes, copy /root/openblas-install to all the machines with /root/spark-ec2/copy-dir /root/openblas-install

  5. Clone and build ml-matrix. git clone https://github.com/amplab/ml-matrix.git; cd ml-matrix; sbt/sbt clean assembly After the build finishes, copy ml-matrix to all the slaves with /root/spark-ec2/copy-dir /root/ml-matrix. NOTE: This is required for the extraClassPath setup we have below

  6. Configure the BLAS using instructions below

Native BLAS / LAPACK for netlib, using spark-submit

Since ATLAS is pre-installed in the Spark AMI, do the following so we can link to OpenBlas:

~/spark/sbin/slaves.sh rm /etc/ld.so.conf.d/atlas-x86_64.conf
~/spark/sbin/slaves.sh ldconfig

After each executor receives a built OpenBlas install, do these:

~/spark/sbin/slaves.sh ln -sf /root/openblas-install/lib/libblas.so.3 /usr/lib64/liblapack.so.3
~/spark/sbin/slaves.sh ln -sf /root/openblas-install/lib/libblas.so.3 /usr/lib64/libblas.so.3

Also we need to update the BLAS, LAPACK on the master node. To do that run

rm /etc/ld.so.conf.d/atlas-x86_64.conf
ldconfig
ln -sf /root/openblas-install/lib/libblas.so.3 /usr/lib64/liblapack.so.3
ln -sf /root/openblas-install/lib/libblas.so.3 /usr/lib64/libblas.so.3

After you are done running this, you can verify if the right BLAS is linked by running something like

~/spark/sbin/slaves.sh readlink -e /usr/lib64/libblas.so.3

Which should return something like /root/openblas-install/lib/libopenblas_sandybridgep-r0.2.14.so

Before running an application, copy-dir the application jar, and make sure the application jar is prepended to spark.executor.extraClassPath in conf/spark-defaults.conf:

spark.executor.extraClassPath
/root/ml-matrix/target/scala-2.10/mlmatrix-assembly-0.1.1.jar:/root/ephemeral-hdfs/conf

Lastly, make sure export OMP_NUM_THREADS=1 is in spark-env.sh.