IBS similarity matrix (--matrix, --distance-matrix) using MIBS / MDIST formats

[davidonlaptop]

Description

This feature adds the --matrix and --distance-matrix option(s) based on the input file described in #3. For more info, check: http://pngu.mgh.harvard.edu/~purcell/plink/strat.shtml#matrix

Analysis

Add a comment to this issue with:

• plink version used as reference (1.07 or 1.90 beta 3)
• relevant C++ function name(s) and the file name(s) where they appear
• Document the structure of the MIBS and MDIST file formats generated by plink.
• Document the algorithm and/or mathematical formula to compute:
  • --matrix option
  • --distance-matrix option

Design

Add a comment to this issue describing how this will be implemented in Spark, and how it differs from plink.

Also update the class diagram on the wiki page describing PLink formats (when incomplete) and add a class diagram describing the models implemented in Scala for this feature on the wiki page on the MGL804 formats.

Implementation

The implementation should use:

• Scala
• Spark RDD
• Spark MLlib / GraphX (if appropriate)

Important note: The model can be only in memory for now, but you'll need to integrate into the ADAM format later on. You'll probably need to create a new record type.

[Aboubakary]

• plink version used as reference: 1.07
• relevant C++ function name(s) and the file name(s) where they appear; cluster