Skip to content

Commit

Permalink
add binary-serialisation-cbor
Browse files Browse the repository at this point in the history
  • Loading branch information
@osa1
osa1 committed Nov 26, 2015
1 parent 23e81b2 commit f6df128
Show file tree
Hide file tree
Showing 5 changed files with 79 additions and 33 deletions.
3 changes: 3 additions & 0 deletions .gitmodules
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
[submodule "packman"]
path = packman
url = https://github.com/jberthold/packman
[submodule "binary-serialise-cbor"]
path = binary-serialise-cbor
url = https://github.com/well-typed/binary-serialise-cbor
69 changes: 42 additions & 27 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,14 +1,16 @@
This is a micro-benchmark that compares `binary`, `cereal` and `packman`
serialization and deserialization performances.
This is a micro-benchmark that compares `binary`, `binary-serialise-cbor`,
`cereal` and `packman` serialization and deserialization performances.

`packman` is not on Hackage, so we include it here as a Git submodule. Make sure
to initialize submodules and add `packman` as source in your Cabal sandbox.
`binary-serialise-cbor` and `packman` are not on Hackage, so we include them
here as Git submodules. Make sure to initialize submodules and add
`binary-serialise-cbor` and `packman` as sources in your Cabal sandbox.
Something like this should work:

```
$ git submodule update --init --recursive
$ cabal sandbox init
$ cabal sandbox add-source packman/
$ cabal sandbox add-source binary-serialise-cbor/
$ cabal install
```

Expand All @@ -17,47 +19,60 @@ $ cabal install
Versions:

- binary: 0.7.6.1
- binary-serialise-cbor: 0.1.1.0
- cereal: 0.5.1.0
- packman: 0.2

```
benchmarking serialization/binary
time 1.934 ns (1.928 ns .. 1.942 ns)
time 1.940 ns (1.936 ns .. 1.945 ns)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.938 ns (1.932 ns .. 1.943 ns)
std dev 18.99 ps (16.36 ps .. 22.03 ps)
variance introduced by outliers: 11% (moderately inflated)
mean 1.939 ns (1.935 ns .. 1.942 ns)
std dev 13.07 ps (11.19 ps .. 15.40 ps)
benchmarking serialization/cereal
time 1.929 ns (1.922 ns .. 1.937 ns)
time 1.934 ns (1.930 ns .. 1.937 ns)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.922 ns (1.917 ns .. 1.927 ns)
std dev 17.08 ps (14.02 ps .. 20.56 ps)
mean 1.937 ns (1.933 ns .. 1.940 ns)
std dev 10.55 ps (8.495 ps .. 14.00 ps)
benchmarking serialization/packman
time 7.980 s (NaN s .. 8.323 s)
1.000 R² (0.999 R² .. 1.000 R²)
mean 8.307 s (8.221 s .. 8.351 s)
std dev 74.09 ms (0.0 s .. 76.94 ms)
time 7.912 s (7.766 s .. 8.181 s)
1.000 R² (1.000 R² .. 1.000 R²)
mean 8.420 s (8.225 s .. 8.564 s)
std dev 219.1 ms (0.0 s .. 249.5 ms)
variance introduced by outliers: 19% (moderately inflated)
benchmarking serialization/binary-CBOR
time 1.956 ns (1.947 ns .. 1.964 ns)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.941 ns (1.935 ns .. 1.947 ns)
std dev 21.34 ps (17.49 ps .. 25.79 ps)
variance introduced by outliers: 13% (moderately inflated)
benchmarking deserialization/binary
time 1.906 ns (1.905 ns .. 1.907 ns)
time 1.937 ns (1.934 ns .. 1.940 ns)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.906 ns (1.905 ns .. 1.907 ns)
std dev 3.480 ps (2.746 ps .. 4.998 ps)
mean 1.940 ns (1.937 ns .. 1.944 ns)
std dev 12.25 ps (10.33 ps .. 15.01 ps)
benchmarking deserialization/cereal
time 704.5 ms (600.7 ms .. 787.5 ms)
0.998 R² (NaN R² .. 1.000 R²)
mean 741.0 ms (733.5 ms .. 747.0 ms)
std dev 9.351 ms (0.0 s .. 10.38 ms)
time 697.0 ms (601.6 ms .. 772.0 ms)
0.998 R² (0.992 R² .. 1.000 R²)
mean 742.9 ms (736.5 ms .. 747.8 ms)
std dev 7.562 ms (136.0 as .. 8.498 ms)
variance introduced by outliers: 19% (moderately inflated)
benchmarking deserialization/packman
time 434.1 ms (34.34 ms .. 892.5 ms)
0.882 R² (0.700 R² .. 1.000 R²)
mean 539.0 ms (480.4 ms .. 584.8 ms)
std dev 70.85 ms (0.0 s .. 79.31 ms)
variance introduced by outliers: 24% (moderately inflated)
time 518.2 ms (82.76 ms .. 845.9 ms)
0.926 R² (0.748 R² .. 1.000 R²)
mean 571.6 ms (504.1 ms .. 609.6 ms)
std dev 59.82 ms (0.0 s .. 65.78 ms)
variance introduced by outliers: 23% (moderately inflated)
benchmarking deserialization/binary-CBOR
time 1.936 ns (1.933 ns .. 1.939 ns)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.933 ns (1.929 ns .. 1.936 ns)
std dev 11.36 ps (9.370 ps .. 14.93 ps)
```
1 change: 1 addition & 0 deletions binary-serialise-cbor
Submodule binary-serialise-cbor added at 9a74b4
1 change: 1 addition & 0 deletions serialization-bench.cabal
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ executable serialization-bench
build-depends:
base >=4.8 && <4.9,
binary >=0.7.6 && <0.8,
binary-serialise-cbor,
bytestring,
cereal >=0.5.1 && <0.6,
criterion,
Expand Down
38 changes: 32 additions & 6 deletions src/Main.hs
View file
Original file line number Diff line number Diff line change
@@ -1,19 +1,23 @@
{-# LANGUAGE DeriveDataTypeable, FlexibleContexts, FlexibleInstances,
MultiParamTypeClasses #-}
LambdaCase, MultiParamTypeClasses #-}

module Main where

import Control.DeepSeq
import qualified Data.ByteString as BS
import qualified Data.ByteString.Lazy as LBS
import qualified Data.ByteString as BS
import qualified Data.ByteString.Lazy as LBS
import Data.Monoid ((<>))
import Data.Typeable
import Data.Word
import System.Random

-- Serialization libs
import qualified Data.Binary as B
import qualified Data.Serialize as C
import qualified GHC.Packing as P
import qualified Data.Binary as B
import Data.Binary.Serialise.CBOR as CBOR
import Data.Binary.Serialise.CBOR.Decoding as CBOR
import Data.Binary.Serialise.CBOR.Encoding as CBOR
import qualified Data.Serialize as C
import qualified GHC.Packing as P

-- Testing and random data generation
import Test.QuickCheck
Expand Down Expand Up @@ -68,9 +72,23 @@ instance C.Serialize a => C.Serialize (BinTree a) where
1 -> Tree <$> C.get <*> C.get
_ -> error "Serialize.get for BinTree"

instance CBOR.Serialise a => CBOR.Serialise (BinTree a) where
encode (Leaf a) =
Encoding (TkTag 0) <> encode a

encode (Tree left right) =
Encoding (TkTag 1) <> encode left <> encode right

decode =
decodeTag >>= \case
0 -> Leaf <$> decode
1 -> Tree <$> decode <*> decode
_ -> fail "CBOR.Serialise.decode for BinTree"

data Binary = Binary
data Cereal = Cereal
data Packman = Packman
data CBOR = CBOR

class Serialize lib a where
serialize :: lib -> a -> IO BS.ByteString
Expand All @@ -89,6 +107,10 @@ instance (NFData a, Typeable a) => Serialize Packman a where
fmap (force . LBS.toStrict . B.encode) . flip P.trySerializeWith (1000 * 2^(20 :: Int))
deserialize _ = fmap force . P.deserialize . B.decode . LBS.fromStrict

instance (CBOR.Serialise a, NFData a) => Serialize CBOR a where
serialize _ = return . force . LBS.toStrict . CBOR.serialise
deserialize _ = return . force . CBOR.deserialise . LBS.fromStrict

prop :: Serialize lib (BinTree Int) => lib -> Property
prop lib = forAll arbitrary (ioProperty . test)
where
Expand Down Expand Up @@ -120,6 +142,7 @@ runBench = defaultMain
[ bench "binary" $ nfIO $ serialize Binary tree
, bench "cereal" $ nfIO $ serialize Cereal tree
, bench "packman" $ nfIO $ serialize Packman tree
, bench "binary-CBOR" $ nfIO $ serialize CBOR tree
]
, bgroup "deserialization"
[ env (generateBalancedTree 22 >>= serialize Binary) $ \bs ->
Expand All @@ -128,6 +151,8 @@ runBench = defaultMain
bench "cereal" $ whnfIO (deserialize Cereal bs :: IO (BinTree Int))
, env (generateBalancedTree 22 >>= serialize Packman) $ \bs ->
bench "packman" $ whnfIO (deserialize Packman bs :: IO (BinTree Int))
, env (generateBalancedTree 22 >>= serialize CBOR) $ \bs ->
bench "binary-CBOR" $ whnfIO (deserialize CBOR bs :: IO (BinTree Int))
]
]

Expand All @@ -136,5 +161,6 @@ main = do
-- runQC Binary
-- runQC Cereal
-- runQC Packman
-- runQC CBOR

runBench

0 comments on commit f6df128

Please sign in to comment.