nbdime with git-lfs

[ebw44]

nbdime integration with git doesn't seem to work well when tracking *.ipynb with lfs.

$ nbdiff-web Validation.ipynb
C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\jupyter_server_mathjax\app.py:40: FutureWarning: The alias _() will be deprecated. Use _i18n() instead.
help=_("""The MathJax.js configuration file that is to be used."""),
[I nbdimeserver:422] Listening on 127.0.0.1, port 49673
[I webutil:29] URL: http://127.0.0.1:49673/difftool
[E web:1789] Uncaught exception POST /api/diff?1635392092574 (127.0.0.1)
HTTPServerRequest(protocol='http', host='127.0.0.1:49673', method='POST', uri='/api/diff?1635392092574', version='HTTP/1.1', remote_ip='127.0.0.1')
Traceback (most recent call last):
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\nbformat\reader.py", line 14, in parse_json
nb_dict = json.loads(s, **kwargs)
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\json_init_.py", line 346, in loads
return _default_decoder.decode(s)
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\tornado\web.py", line 1702, in execute
result = method(*self.path_args, **self.path_kwargs)
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\nbdime\webapp\nbdimeserver.py", line 220, in post
base_nb = self.get_notebook_argument('base')
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\nbdime\webapp\nbdimeserver.py", line 241, in get_notebook_argument
return nbformat.read(arg, as_version=4)
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\nbformat_init.py", line 143, in read
return reads(buf, as_version, **kwargs)
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\nbformat_init_.py", line 73, in reads
nb = reader.reads(s, **kwargs)
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\nbformat\reader.py", line 58, in reads
nb_dict = parse_json(s, **kwargs)
File "C:\Users\username\AppData\Local\Programs\Python\Python310\lib\site-packages\nbformat\reader.py", line 17, in parse_json
raise NotJSONError(("Notebook does not appear to be JSON: %r" % s)[:77] + "...") from e
nbformat.reader.NotJSONError: Notebook does not appear to be JSON: 'version https://git-lfs.github.com/spec...
[W handlers:622] Unhandled error
[E log:54] {
"Host": "127.0.0.1:49673",
"Connection": "keep-alive",
"Content-Length": "62",
"Sec-Ch-Ua": ""Google Chrome";v="95", "Chromium";v="95", ";Not A Brand";v="99"",
"X-Xsrftoken": "2|89cc1eca|8454f6c80863350d1ad18ef62540b91c|1634854506",
"Sec-Ch-Ua-Mobile": "?0",
"Content-Type": "text/plain;charset=UTF-8",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.54 Safari/537.36",
"Sec-Ch-Ua-Platform": ""Windows"",
"Accept": "/",
"Origin": "http://127.0.0.1:49673",
"Sec-Fetch-Site": "same-origin",
"Sec-Fetch-Mode": "cors",
"Sec-Fetch-Dest": "empty",
"Referer": "http://127.0.0.1:49673/difftool",
"Accept-Encoding": "gzip, deflate, br",
"Accept-Language": "en-US,en;q=0.9,fr;q=0.8",
"Cookie": "_xsrf=2|89cc1eca|8454f6c80863350d1ad18ef62540b91c|1634854506"
}
[E log:55] 500 POST /api/diff?1635392092574 (127.0.0.1) 56.85ms referer=http://127.0.0.1:49673/difftool

But inside visual studio I can still see the diff of uncommited changes and the HEAD.
Here is my .gitattributes:
*.ipynb filter=lfs diff=lfs merge=lfs -text
And my .git/config also contains

[diff "lfs"]
	textconv = cat

[ebw44]

If someone is looking for a partial solution to this problem you can try to manually create the head version of the Validation.ipynb file:

git show HEAD:Validation.ipynb | git lfs smudge > Validation_head.ipynb
nbdime diff-web Validation.ipynb Validation_head.ipynb

[vidartf]

Does it work if you use the --use-filter flag?

[ebw44]

I couldn't find where to use --use-filter but with the following
.gitattributes
*.ipynb filter=lfs diff=jupyternotebook

and .git/config

[diff "jupyternotebook"]
	command = git-nbdiffdriver webdiff --ip 127.0.0.1
	textconv = cat

I can use the git diff command.

[vidartf]

Thanks for sharing your working config. The --use-filter flag would be to nbdime, e.g. command = git-nbdiffdriver webdiff --ip 127.0.0.1 --use-filter (in that case, I assume textconv might not be needed, but not sure).

[ebw44]

For me the --use-filter flag does not work (either with or without texconv. But the textconv = cat seems to work.

[josueibarra95]

Have the same problem.
Setting diff=jupyternotebook in .gitattributes
and

[diff "jupyternotebook"]
	textconv = cat

in .git/config (didn't need to set command=...)
did the trick but only for using VSCode's built-in git diff tool.
I couldn't get it to work with the command git diff (it died). Adding the --use-filter flag also gave a different error:

[E nbdimeserver:115] Supplied argument cannot be read: '<nbdime.vcs.git.filter_integration.NamedStringIO object at 0x7f293214b050>'
Traceback (most recent call last):
File ".../miniconda3/envs/nbdime_env/lib/python3.7/site-packages/nbdime/webapp/nbdimeserver.py", line 89, in read_notebook
raise ValueError('Supplied argument cannot be read: %r' % arg)
ValueError: Supplied argument cannot be read: '<nbdime.vcs.git.filter_integration.NamedStringIO object at 0x7f293214b050>'
[W web:1796] 422 POST /api/diff?1658992318964 (127.0.0.1): Invalid notebook: <nbdime.vcs.git.filter_integration.NamedStringIO object at 0x7f293214b050>
[W log:59] 422 POST /api/diff?1658992318964 (127.0.0.1) 210.27ms referer=http://127.0.0.1:37355/diff?base=%2Ftmp%2Fgit-blob-VhhJdE%2FSmoothen+FRAs.ipynb&remote=%3Cnbdime.vcs.git.filter_integration.NamedStringIO+object+at+0x7f293214b050%3E
...skipping...
[I nbdimeserver:422] Listening on 127.0.0.1, port 37355
[I webutil:29] URL: http://127.0.0.1:37355/diff?base=%2Ftmp%2Fgit-blob-VhhJdE%2FSmoothen+FRAs.ipynb&remote=%3Cnbdime.vcs.git.filter_integration.NamedStringIO+object+at+0x7f293214b050%3E
[E nbdimeserver:115] Supplied argument cannot be read: '<nbdime.vcs.git.filter_integration.NamedStringIO object at 0x7f293214b050>'
Traceback (most recent call last):
File ".../miniconda3/envs/nbdime_env/lib/python3.7/site-packages/nbdime/webapp/nbdimeserver.py", line 89, in read_notebook
raise ValueError('Supplied argument cannot be read: %r' % arg)
ValueError: Supplied argument cannot be read: '<nbdime.vcs.git.filter_integration.NamedStringIO object at 0x7f293214b050>'
[W web:1796] 422 POST /api/diff?1658992318964 (127.0.0.1): Invalid notebook: <nbdime.vcs.git.filter_integration.NamedStringIO object at 0x7f293214b050>
[W log:59] 422 POST /api/diff?1658992318964 (127.0.0.1) 210.27ms referer=http://127.0.0.1:37355/diff?base=%2Ftmp%2Fgit-blob-VhhJdE%2FSmoothen+FRAs.ipynb&remote=%3Cnbdime.vcs.git.filter_integration.NamedStringIO+object+at+0x7f293214b050%3E

I'm running everything from a conda environment named 'nbdime_env' (I had to install a module named 'ipython_genutils' before being able to use nbdime's git driver)