Airspeed velocity benchmarks

[hameerabbasi]

No description provided.

[skrah]

Thanks, this takes a bit of time to comprehend. I have a couple of remarks:

1. I find asv unsuitable for interactive use, so I prefer to leave the original files. Perhaps put everything in an asv directory and the original .py files into a scripts directory.

2. I couldn't find the reason why the tuple access result in asv differs so much from the loop result:

Accessing an element in an array of tuples
------------------------------------------

   xnd:   0.2511618137359619
   numpy: 0.7269613742828369

3. Is there some sort of make clean for asv?

[hameerabbasi]

Thanks, this takes a bit of time to comprehend. I have a couple of remarks:

It is pretty thick code, I agree.

1. I find asv unsuitable for interactive use, so I prefer to leave the original files. Perhaps put everything in an asv directory and the original .py files into a scripts directory.

Done.

2. I couldn't find the reason why the tuple access result in asv differs so much from the loop result:

I'm pretty sure it's because the Python object has to be constructed each time you do something like a[0], whereas in the tuple access, everything can be done in the C level and then accessed from Python once. It may also be the overhead of the loop itself.

3. Is there some sort of make clean for asv?

There is asv rm bit it just cleans the results for previous commits' results, not the entire environment tree.

I would just use git clean -xfd ..

[skrah]

I'd like to go to the bottom of the issue of different results for tuple element access. I don't think it's a loop issue, because all access benchmarks have the same loop.

One reason might be that xnd generates views also for elements and numpy does not (by default). So that would be an advantage for xnd that should show up in the benchmarks.

Are you sure that single elements and not subtrees are accessed (I don't have time to really dig into the code)?

[hameerabbasi]

Oh. If dim[1] < dim[0], it's a subtree access. If dim[1] == dim[0], an element is accessed (In NumPy), and an element-view (In XND).

[hameerabbasi]

Actually, in the for loop version, the last iteration of the loop in NumPy accesses a scalar (for dim[0] == dim[1]).

dim[0] == array.ndim, dim[1] is the number of integer indices.

[hameerabbasi]

To expand on the previous issue.

[hameerabbasi]

Here's the access for tuple access. Equivalent to a[0, 0, ...] where the ellipsis is a continuation and not a Python ellipsis.

[skrah]

NumPy does not appear to be able to use multi-indexing on tuples:

>>> import numpy as np
>>> lst = [('Rex', 9, 81.0)] * 100
>>> dt = np.dtype([('name', 'U10'), ('age', 'i4'), ('weight', 'f4')])
>>> x = np.array(lst, dtype=dt)
>>> y = x[0][0]
>>> type(y)
<class 'numpy.str_'>
>>> 
>>> y = x[0, 0]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
IndexError: too many indices for array

[hameerabbasi]

I don't test tuples in the element access at all, I could add that.

[skrah]

The fast benchmark measures y = x[0][0], i.e. it accesses the 'Rex' tuple element, not the array element.

[skrah]

xnd is able to access through tuples:

>>> lst = [('Rex', 9, 81.0)] * 100
>>> x = xnd(lst)
>>> x[0, 0]
xnd('Rex', type='string')

So that should be even faster for xnd.

[hameerabbasi]

Here, I cache a, then in a loop, access it. This is equivalent to a[0][0]...

[skrah]

In general I think people are likely to write benchmarks for small arrays. xnd is currently at a disadvantage because the python convenience classes in __init__.py are not optimized for small arrays.

Here is the construction benchmark when using the C Xnd constructor directly on a small one element list:

from xnd import Xnd
...
for i in range(repeat):
    x = Xnd(ndt("1 * int64"), lst)
...

Dtype provided
--------------

   xnd:   0.07504105567932129
   numpy: 0.08943438529968262

[skrah]

So I'll rewrite the class in C now before publishing the benchmarks.