Skip to content

Latest commit

 

History

History
163 lines (131 loc) · 8.52 KB

benchmark.org

File metadata and controls

163 lines (131 loc) · 8.52 KB

Benchmarks of various cryptographic functions

The objective of this work is to measure the time different cryptograhic primitives take with text files of various sizes.

The operations we’re going to test are:

  • Symmetric encryption and decryption using the AES algorithm.
  • Asymmetric encryption and decryption using the RSA algorithm.
  • Hashing using the SHA2-256 algorithm.

This program will be done using Literate Programming, as to make the program itself and the report one and the same, removing the need to switch back and forth while reading the report to see the implementation.

Expanding on the above point, we will be specifically be using Org-Babel, which comes included in Emacs by default.

How this program works (in steps):

  1. Generates all files that we will run the tests on.
  2. Runs multiple simulations of encrypting, decrypting and hashing on a given set of files
  3. Records various performance metrics.
  4. Computes the average of those metrics.
  5. Displays them visually using plots.

How to run these simulation yourself:

To run this program you will need the following:

  • sbcl, altough it will probably work with other implementations
  • emacs, along with org-babel configured for lisp
  • either slime or sly, I personally use sly due to its advantages (ascii cat art being the most notable), but it shouldn’t make a difference
  • quicklisp

For this program we will be using the ironclad cryptography system.

(ql:quickload "ironclad")
(ql:quickload "alexadria")

Constants

First we need to declare our constants:

  1. The number of simulations we will run.
  2. The list of file sizes we will use to generate the files used for the benchmarks.
  3. The list holding all alphabet characters in lowercase.
  4. The RSA key size.
  5. The AES key size.

Obviously the more simulations we run the more precise the results will be, altough a high number ofsimulations will significantly slow down our program. 100 simulations should be a good starting point.

Its also important to note that the first simulation will be significantly slower than the subsequent ones due to cold-cache.

On a side note, while using lists to store our values is quite a bit more ineficient than simply using arrays, but considering how few elements each list holds optimization isn’t too important anyway. I also won’t be really opimizing the rest of the code, mainly adding type annotation, for now at least.

(defconstant +simulations+ 100 "The number of simulations to run.")
(defconstant +aes-file-sizes+ '(8 64 512 32768 262144 2097152) "The set of file sizes to test aes encryption/decryption times on.")
(defconstant +rsa-file-sizes+ '(2 4 8 16 32 64 128) "The set of file sizes to test rsa encryption/decryption on.")
(defconstant +sha-file-sizes+ '(8 64 512 32768 262144 2097152) "The set of file sizes to test sha hashing on.")
(defconstant +alphabet+ '(#\a #\b #\c #\d #\e #\f #\g #\h #\i #\j #\k #\l #\m #\n #\o #\p #\q #\r #\s #\t #\u #\v #\w #\x #\y #\z) "A list containing all english alphabet characters.")
(defconstant +rsa-key-size+ 2048 "The size of the rsa key to be used (in bits).")
(defconstant +aes-key-size+ 256 "The size of the aes key to be used (in bits).")

Before running the tests on a given file we must first create it, for that we will define generate-file function that takes its size, and optionally, a prefix and suffix as its arguments. The function will then create a file (if it doesn’t already exist) or override the contents of an existing one by filling it with random ascii (not utf!) characters as to make it of a given size.

The reason for format to only write 2 characters at a time to a file is due to 2 being the Greatest Common Divisor of all file sizes defined above.

TODO: Perhaps replace the link to the relevant sections of the posix standard? When we finish writing to the file, a newline is inserted at the end, this is the recommended thing to do when it comes to text files.

(defun generate-file (size &key (prefix "") (suffix ""))
  "Generates a file of a given size, named *prefix*size*suffix*."
  (with-open-file (stream (merge-pathnames
                           (concatenate 'string prefix (write-to-string size) suffix)
                           (uiop/os:getcwd))
                          :external-format :ASCII
                          :direction :output
                          :if-exists :supersede
                          :if-does-not-exist :create)
    (dotimes (i (- (floor size 2) 1) nil)
      (format stream "abcdef"))
    (format stream "~C~C" #\c #\linefeed)))

For this particular program we will need to create various different files, each of them of a different size and each of them for a different purpose. The bellow function accepts a list of sizes (in bytes), optionally a prefix and suffix and creates all files, or overrides their contents, if they already exist.

(defun generate-files (sizes &key (prefix "") (suffix ""))
  "Creates a set of files of a given size, named *prefi*size*suffix*."
  (dolist (size sizes nil)
    (generate-file size :prefix prefix :suffix suffix)))

TODO: return time interval (and possibly other data) in microseconds

(defun aes-encrypt (filename key cipher)
  (unless (probe-file filename)
    (format t "Error: ~s does not exist.~%" filename)
    (return nil))

  (defvar text (uiop:read-file-string filename))
  (ironclad:encrypt-in-place cipher text)
  (alexandria:write-string-into-file text filename))
(defun aes-decrypt (filename key cipher)
  (unless (probe-file filename)
    (format t "Error: ~s does not exist.~%" filename)
    (return nil))

  (defvar text (uiop:read-file-string filename))
  (ironclad:decrypt-in-place cipher text)
  (alexandria:write-string-into-file text filename))
(defun main ()
  "Main program loop."
  (multiple-value-bind (rsa-private-key rsa-public-key) (ironclad:generate-key-pair :rsa :num-bits 2048)
    (defconstant +aes-key+ (ironclad:ascii-string-to-byte-array "0123456789abcdef"))
    (defconstant +aes-cipher+ (ironclad:make-cipher :aes :mode :ecb :key +aes-key+ :padding :pkcs7))

    (dotimes (i +simulations+ nil)
      (generate-files +aes-file-sizes+ :prefix "aes" :suffix ".txt")
      (generate-files +rsa-file-sizes+ :prefix "rsa" :suffix ".txt")
      (generate-files +sha-file-sizes+ :prefix "sha" :suffix ".txt")

      (defvar aes-encryption-results '(0 0 0 0 0 0))
      (defvar aes-decryption-results '(0 0 0 0 0 0))

      ;; TODO: turn these into a function that accepts either rsa or aes
      (loop for filesize in +aes-file-sizes+ and index from 0
            do (setf (nth index aes-encryption-results)
                     (aes-encrypt
                      (concatenate 'string "aes"
                                   (write-to-string filesize) ".txt")
                      +aes-key+
                      +aes-cipher+))
               (setf (nth index aes-decryption-results)
                     (aes-decrypt
                      (concatenate 'string "aes"
                                   (write-to-string filesize) ".txt")
                      +aes-cipher+))))))

TODO: generate random aes key TODO: make sure the file generation function creates random files TODO: test aes encryption/decryption functions TODO: function to hash TODO: function to encrypt rsa TODO: function to decrypt rsa TODO: storing simulation results TODO: plotting simulation results TODO: consider changing the defvar’s inside the main function to local scope