feat: update list of branches (#17)

* re-derive list of branches with representative data file * update notebook reading branches and processing them with coffea to use new branches * add some extra debugging utility & performance metrics
iris-hep · Apr 10, 2024 · 8ffe060 · 8ffe060
1 parent a369a0f
commit 8ffe060
Show file tree

Hide file tree

Showing 7 changed files with 3,997 additions and 3,577 deletions.
diff --git a/.gitignore b/.gitignore
@@ -168,3 +168,4 @@ cython_debug/
 servicex.yaml
 *.pstats
 servicex/test.py
+*.html
diff --git a/README.md b/README.md
@@ -10,8 +10,6 @@ ATLAS does not have released OpenData, so there isn't an AGC we can copy and try
 
 ## Description of files
 
-* `size_per_branch.ipynb`: produce breakdown of branch sizes for given file
-* `branch_sizes.json`: output of , produced by `size_per_branch.ipynb`
 * `materialize_branches.ipynb`: read list of branches, distributable with Dask (use for benchmarking)
 
 ## Usage
@@ -45,6 +43,13 @@ with additional files:
     rucio list-file-replicas mc20_13TeV:mc20_13TeV.364126.Sherpa_221_NNPDF30NNLO_Zee_MAXHTPTV500_1000.deriv.DAOD_PHYSLITE.e5299_s3681_r13145_p6026 --protocol root  --pfns --rses MWT2_UC_LOCALGROUPDISK
     ```
 
+### Branch list determination
+
+Branches to be read are determined with a 2018 data file.
+
+* `input_files/size_per_branch.ipynb`: produce breakdown of branch sizes for given file
+* `input_files/branch_sizes.json`: output of notebook above
+
 ## Acknowledgements
 
 [![NSF-1836650](https://img.shields.io/badge/NSF-1836650-blue.svg)](https://nsf.gov/awardsearch/showAward?AWD_ID=1836650)

diff --git a/branch_sizes.json b/branch_sizes.json
diff --git a/input_files/branch_sizes.json b/input_files/branch_sizes.json
diff --git a/input_files/size_per_branch.ipynb b/input_files/size_per_branch.ipynb
diff --git a/materialize_branches.ipynb b/materialize_branches.ipynb
diff --git a/size_per_branch.ipynb b/size_per_branch.ipynb