Roundtrip conversion ipynb to md strips cell metadata

[goerz]

This is using https://github.com/qucontrol/krotov/blob/f1082fe129abd609ecac25e57e0737066a599cd7/docs/notebooks/01_example_simple_state_to_state.ipynb, which is a notebook that contains no jupytext metadata:

> jupytext --to=md 01_example_simple_state_to_state.ipynb
[jupytext] Reading 01_example_simple_state_to_state.ipynb
[jupytext] Writing 01_example_simple_state_to_state.md
> jupytext --to=notebook --update 01_example_simple_state_to_state.md
[jupytext] Reading 01_example_simple_state_to_state.md
[jupytext] Writing 01_example_simple_state_to_state.ipynb (destination file updated)
> git diff 01_example_simple_state_to_state.ipynb
nbdiff /var/folders/ll/rgszt4s525s2xtv8rr60lqz40000gn/T//fGqltc_01_example_simple_state_to_state.ipynb docs/notebooks/01_example_simple_state_to_state.ipynb
--- /var/folders/ll/rgszt4s525s2xtv8rr60lqz40000gn/T//fGqltc_01_example_simple_state_to_state.ipynb  2019-01-29 22:41:15.860990
+++ docs/notebooks/01_example_simple_state_to_state.ipynb  2019-01-29 22:41:12.259097
## deleted /cells/1/metadata/attributes:
-  classes:
-    []
-  id:
-  n: 1

## deleted /cells/3/metadata/attributes:
-  classes:
-    []
-  id:
-  n: 2

## deleted /cells/5/metadata/attributes:
-  classes:
-    []
-  id:
-  n: 3

## deleted /cells/8/metadata/attributes:
-  classes:
-    []
-  id:
-  n: 4
...

I would expect this roundtrip conversion to leave the notebook completely unchanged.

[mwouts]

Thanks for reporting @goerz . Well, currently the implementation markdown format has no support for cell metadata, and that is possibly the reason why you observe this.

And even in other formats, Jupytext has a default notebook/cell metadata filter, which will remove some metadata anyway (unless you change the filters; read more on this at the end of this paragraph).

Could you please try with the --update argument? When updating the notebook with the markdown file, Jupytext supposedly knows that it should take the metadata from the notebook.

Also, have you seen that you can test the round trip conversion with --test and --test-strict ? I did the exercice of testing on notebooks in the wild some time ago at #99 , but clearly I should automate this...

[mwouts]

Sorry I had not seen that the --update argument was already there! I will cover this with an additional test. Thanks!

[mwouts]

This is fixed now in version 1.0.0-rc1. I'll let you confirm and close. And thanks for reporting, that was a major issue!

[goerz]

Looks good, in terms of the metadata... although the conversion back to ipynb, with the latest version of jupytext, does not end in a newline. The original file does, so the roundtrip conversion still modifies the file. The newline should definitely be there: all text files (including json) should end with a newline (https://stackoverflow.com/questions/729692/why-should-text-files-end-with-a-newline)

[mwouts]

Agreed! The extra new line is added here in the nbformat library. I will implement the same (simple) rule.

[mwouts]

Should be fixed in 1.0.0-rc2:

pip install jupytext --upgrade --pre