Error reading with uproot root files, with TTree, written with uproot

[marinang]

Hi I tried the example in the README to write TTree in root file but when I try to read it back I get the following error.

In [1]: import uproot

In [2]: import numpy as np

In [3]: with uproot.recreate("example.root") as f:
   ...:     f["t"] = uproot.newtree({"normal": np.float64})
   ...:     f["t"].extend({"normal": np.random.normal(0, 1, 10000)})
   ...:

In [4]: t = uproot.open("example.root")["t"]

In [5]: t
Out[5]: <TTree b't' at 0x00010e1e50f0>

In [6]: t.arrays()
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-6-6502dc56ca9a> in <module>
----> 1 t.arrays()

/usr/local/lib/python3.7/site-packages/uproot/tree.py in arrays(self, branches, outputtype, namedecode, entrystart, entrystop, flatten, flatname, awkwardlib, cache, basketcache, keycache, executor, blocking)
    513
    514         # start the job of filling the arrays
--> 515         futures = [(branch.name if namedecode is None else branch.name.decode(namedecode), interpretation, branch.array(interpretation=interpretation, entrystart=entrystart, entrystop=entrystop, flatten=(flatten and not ispandas), awkwardlib=awkward, cache=cache, basketcache=basketcache, keycache=keycache, executor=executor, blocking=False)) for branch, interpretation in branches]
    516
    517         # make functions that wait for the filling job to be done and return the right outputtype

/usr/local/lib/python3.7/site-packages/uproot/tree.py in <listcomp>(.0)
    513
    514         # start the job of filling the arrays
--> 515         futures = [(branch.name if namedecode is None else branch.name.decode(namedecode), interpretation, branch.array(interpretation=interpretation, entrystart=entrystart, entrystop=entrystop, flatten=(flatten and not ispandas), awkwardlib=awkward, cache=cache, basketcache=basketcache, keycache=keycache, executor=executor, blocking=False)) for branch, interpretation in branches]
    516
    517         # make functions that wait for the filling job to be done and return the right outputtype

/usr/local/lib/python3.7/site-packages/uproot/tree.py in array(self, interpretation, entrystart, entrystop, flatten, awkwardlib, cache, basketcache, keycache, executor, blocking)
   1360             keycache = {}
   1361
-> 1362         basket_itemoffset = self._basket_itemoffset(interpretation, basketstart, basketstop, keycache)
   1363         basket_entryoffset = self._basket_entryoffset(basketstart, basketstop)
   1364

/usr/local/lib/python3.7/site-packages/uproot/tree.py in _basket_itemoffset(self, interpretation, basketstart, basketstop, keycache)
   1315         for j, key in enumerate(self._threadsafe_iterate_keys(keycache, True, basketstart, basketstop)):
   1316             i = basketstart + j
-> 1317             numitems = interpretation.numitems(key.border, self.basket_numentries(i))
   1318             basket_itemoffset.append(basket_itemoffset[-1] + numitems)
   1319         return basket_itemoffset

/usr/local/lib/python3.7/site-packages/uproot/interp/numerical.py in numitems(self, numbytes, numentries)
    150         dtype, shape = _dtypeshape(self.fromdtype)
    151         quotient, remainder = divmod(numbytes, dtype.itemsize)
--> 152         assert remainder == 0
    153         return quotient
    154

AssertionError:

[jpivarski]

Thanks for being an early adopter, but I'm going to recommend that you use the low level newbasket interface, rather than the high-level extend interface.

There were some bugs to be worked out in extend that Pratyush didn't have time to finish before a very long flight. The documentation is a little ahead of the implementation. The necessary steps to get it working are all already GitHub issues.

On the other hand, the newbasket interface is complete to the best of our knowledge.

[marinang]

HI Jim, thank you for your answer. However I am getting the same error with newbasket

In [1]: import numpy as np

In [2]: import uproot

In [3]: with uproot.recreate("example.root") as f:
   ...:     f["t"] = uproot.newtree({"norm": np.float64})
   ...:     f["t"]["norm"].newbasket(np.random.normal(0, 1, 10000))
   ...:

In [4]: t = uproot.open("example.root")["t"]

In [5]: t.arrays()
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-6-6502dc56ca9a> in <module>
----> 1 t.arrays()

/usr/local/lib/python3.7/site-packages/uproot/tree.py in arrays(self, branches, outputtype, namedecode, entrystart, entrystop, flatten, flatname, awkwardlib, cache, basketcache, keycache, executor, blocking)
    513
    514         # start the job of filling the arrays
--> 515         futures = [(branch.name if namedecode is None else branch.name.decode(namedecode), interpretation, branch.array(interpretation=interpretation, entrystart=entrystart, entrystop=entrystop, flatten=(flatten and not ispandas), awkwardlib=awkward, cache=cache, basketcache=basketcache, keycache=keycache, executor=executor, blocking=False)) for branch, interpretation in branches]
    516
    517         # make functions that wait for the filling job to be done and return the right outputtype

/usr/local/lib/python3.7/site-packages/uproot/tree.py in <listcomp>(.0)
    513
    514         # start the job of filling the arrays
--> 515         futures = [(branch.name if namedecode is None else branch.name.decode(namedecode), interpretation, branch.array(interpretation=interpretation, entrystart=entrystart, entrystop=entrystop, flatten=(flatten and not ispandas), awkwardlib=awkward, cache=cache, basketcache=basketcache, keycache=keycache, executor=executor, blocking=False)) for branch, interpretation in branches]
    516
    517         # make functions that wait for the filling job to be done and return the right outputtype

/usr/local/lib/python3.7/site-packages/uproot/tree.py in array(self, interpretation, entrystart, entrystop, flatten, awkwardlib, cache, basketcache, keycache, executor, blocking)
   1360             keycache = {}
   1361
-> 1362         basket_itemoffset = self._basket_itemoffset(interpretation, basketstart, basketstop, keycache)
   1363         basket_entryoffset = self._basket_entryoffset(basketstart, basketstop)
   1364

/usr/local/lib/python3.7/site-packages/uproot/tree.py in _basket_itemoffset(self, interpretation, basketstart, basketstop, keycache)
   1315         for j, key in enumerate(self._threadsafe_iterate_keys(keycache, True, basketstart, basketstop)):
   1316             i = basketstart + j
-> 1317             numitems = interpretation.numitems(key.border, self.basket_numentries(i))
   1318             basket_itemoffset.append(basket_itemoffset[-1] + numitems)
   1319         return basket_itemoffset

/usr/local/lib/python3.7/site-packages/uproot/interp/numerical.py in numitems(self, numbytes, numentries)
    150         dtype, shape = _dtypeshape(self.fromdtype)
    151         quotient, remainder = divmod(numbytes, dtype.itemsize)
--> 152         assert remainder == 0
    153         return quotient
    154

[reikdas]

Thanks for reporting the issue. I am looking into it.

[reikdas]

@marinang It is a compression issue.
While I am working on fixing the error, you can try setting the compression explicitly to None -

>>> import uproot
>>> import numpy as np
>>> with uproot.recreate("example.root", compression=None) as f:
...     f["t"] = uproot.newtree({"normal": np.float64})
...     f["t"].extend({"normal": np.random.normal(0, 1, 10000)})
... 
>>> t = uproot.open("example.root")["t"]
>>> t
<TTree b't' at 0x7f2320ffee10>
>>> t.arrays()
{b'normal': array([-2.04193608, -2.0257052 ,  2.04641301, ..., -0.04302636,
        2.78718844,  0.61351022])}

[marinang]

@reikdas Indeed it works :). Thanks.

[reikdas]

@marinang The issue should be fixed in master. Could you try and let me know?

[marinang]

Perfect I have just tried and it works, thanks a lot.