Improve discoverability of cddatags

[jbytheway]

Summary

SUMMARY: None

Purpose of change

To help people discover cddatags.py, added in #37877 (and fix a bug in it).

Describe the solution

• Mention it in the json docs.
• Use it in the Makefile rule for creating a tags file.
• Make it more likely to work on Windows (was relying on / being the path separator).

Testing

Used make ctags. Looks good, and seems to still work.

[snipercup]

I am interested in using this but "you can run ctags -R . && tools/json_tools/cddatags.py." does not tell me much and I doubt it will mean much to the people that would read JSON_INFO.md. I suggest you move what you wrote here to /doc/DEVELOPER_TOOLING.md or /tools/readme.md and just leave a link to that page here.

Then, please describe what I should use to run the ctags command and if you think that is too elementary, please point to where I can read up on how I would do this.

[jbytheway]

I was torn between here and DEVELOPER_TOOLING. I'll move this second paragraph there.

For the first paragraph, it really should be as simple as running that script and then figuring out how your editor works with tags files. I run into trouble providing more details, for a couple of reasons:

• It's hard for me to give reasonable guidance for Windows users; I don't really know what the easiest approach is for that platform, but it's probably the one most in need of it.
• Every text editor is different. I explained what to do for Vim, but I have no idea what the most popular editors amongst CDDA JSON folks is, nor would I probably know how to configure them.
  I'm hopeful that someone who is in a better position to expand on these points will come along and add more detail.

[snipercup]

I'd like to add this text for windows:

# windows:
- Download the Source and binary for Windows 98/NT/2000/XP zipfile from here: http://ctags.sourceforge.net/
- Unzip the zip file
- Open a cmd promt and navigate to the folder where you unpacked the zipfile
- Run "ctags -R . && C:\Users\User\Documents\GitHub\Cataclysm-DDA\tools\json_tools\cddatags.py" (you should replace the path to cddatags.py with the path on your machine)

Unfortunately running this command only opens the cddatags.py in notepad without creating a tags file from what I can tell.
I tried to change this:

But that didn't work eighter. Maybe a windows user who is more familiar with this can offer help?

Here is the bit from my command prompt:

[AMurkin]

Unfortunately running this command only opens the cddatags.py in notepad

You need a python launcher to be installed. And .py files associated with it.

[snipercup]

Thanks, I associated the filetype. I have python 3.8 installed. now i'm getting this:

`Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation.

C:\Users\User\Downloads\ctags58\ctags58>ctags -R . && C:\Users\User\Documents\GitHub\Cataclysm-DDA\tools\json_tools\cddatags.py
Problem reading file texts.json, reason: 'charmap' codec can't decode byte 0x90 in position 44783: character maps to Problem reading file martialarts.json, reason: 'charmap' codec can't decode byte 0x8d in position 45837: charact
er maps to Problem reading file rotatable_symbols.json, reason: 'charmap' codec can't decode byte 0x90 in position 161: character maps to Problem reading file martial.json, reason: 'charmap' codec can't decode byte 0x8
d in position 7762: character maps to Problem reading file mutations.json, reason: 'charmap' codec can't decode byte 0x8d in position 23531: character maps to Problem reading file overmap_terrain_agricultural.json, rea
son: 'charmap' codec can't decode byte 0x90 in position 944: character maps to Problem reading file overmap_terrain_evac_center.json, reason: 'charmap' codec can't decode byte 0x90 in position 993: character maps to Pr
oblem reading file overmap_terrain_mall.json, reason: 'charmap' codec can't decode byte 0x90 in position 6473: character maps to Problem reading file overmap_terrain_necropolis.json, reason: 'charmap' codec can't decode byte 0x90
in position 5543: character maps to Problem reading file overmap_terrain_ranch_camp.json, reason: 'charmap' codec can't decode byte 0x90 in position 1327: character maps to Problem reading file overmap_terrain.json, r
eason: 'charmap' codec can't decode byte 0x90 in position 48275: character maps to Problem reading file overmap_terrain.json, reason: 'charmap' codec can't decode byte 0x90 in position 5590: character maps to Problem r
eading file ja.json, reason: 'charmap' codec can't decode byte 0x81 in position 211: character maps to Problem reading file ko.json, reason: 'charmap' codec can't decode byte 0x9d in position 9351: character maps to Pr
oblem reading file ru.json, reason: 'charmap' codec can't decode byte 0x81 in position 253: character maps to Problem reading file zh_CN.json, reason: 'charmap' codec can't decode byte 0x81 in position 67: character maps to Problem reading file zh_TW.json, reason: 'charmap' codec can't decode byte 0x8d in position 1072: character maps to
C:\Users\User\Downloads\ctags58\ctags58>`

Looks like i'm getting a tags file though:

[AMurkin]

Me too. Anyway it adds data to tags file.

[snipercup]

Okay, looks like I got it working:

To add to the windows documentation:

• Go to the plugins admin in notepad++:

• Look for the TagLEET plugin and install it:

• Select any id in the json file and press alt+space to open the references window:

Let me know if there is anything else you need for the windows documentation

[jbytheway]

Thanks for trying it out; that's very helpful.

I've tweaked the Python script to hopefully avoid the error messages you observed. Can you see if that fixed them?

I'll think about what to write in the docs.

[jbytheway]

Also, @snipercup: the ctags -R . shouldn't be necessary unless you also want tags working for the C++ code (which I'm guessing you don't). If you delete the tags file and just run the Python script (not ctags), does it still work?

I'm also curious to know if just double-clicking on cddatags.py works (though I guess even if it runs you probably won't be able to easily see any error messages it might print out).

[jbytheway]

I've updated the docs to include your findings. You can see this on my branch. What do you think?

[snipercup]

No problem jbytheway. I'll get back to you as soon as I can.

[snipercup]

I deleted the ctags file and I can run the python script by doubleclicking. It's hard to test the results because the existence of the tags file does not seem to matter, the results are the same. Pressing alt+space still shows the tagleet references window. The weird thing is that jumping to the selected location in the references window worked the first time but not after that. In https://sourceforge.net/p/tagleet/wiki/Home/ it says it uses a ctags file but for me it does not matter if it exists or not.

Edit: Tried some other things, it only seems to work for actual source files but not json. I used the cmake command in my own project that has json-schemas in the form of json and somehow that works. But that's right now, maybe after I reboot it stops working, just like what happened with the first time I tried it on the CDDA json files.

Running C:\Users\User\Documents\GitHub\Cataclysm-DDA>C:\Users\User\Documents\GitHub\Cataclysm-DDA\tools\json_tools\cddatags.py does create a tags file of 2.3mb but tagLeet does not show references for json

Edit2: I'm finally getting some results with this command: C:\Users\User\Documents\GitHub\Cataclysm-DDA\data\json>C:\Users\User\Downloads\ctags-2020-02-13_6ffdd48e-x64\ctags.exe -R --languages=JSON which uses https://github.com/universal-ctags/ctags. It makes no difference if I append . && C:\Users\User\Documents\GitHub\Cataclysm-DDA\tools\json_tools\cddatags.py. The result I get is that the ctags work for the keys but not the values.

Also, your documentation is looking good.

[jbytheway]

OK, these are strange symptoms. Let me check that I understand what you're saying:

• The first time you got it working you were able to follow the tag for rifle_auto.
• Now, if you run the script, it creates a tags file (is that in C:\Users\User\Documents\GitHub\Cataclysm-DDA?) but TagLEET isn't showing any references. Di you check the same key (rifle_auto) that was working before?
• When you use ctags's built-in json support you get something that works. How big is that file? In my case that command generates a ~400MB file that indexes every json key, since it doesn't understand the CDDA json structure, which is probably not particularly useful? Is that consistent with what you're getting?

Also, can I check: are you running cddatags.py from master, or from this PR branch?

My only idea right now is that there's something about the file format my script is generating that TagLEET doesn't like. Can you upload / attach an example of a tags file from my script that doesn't work, and one from ctags.exe that does work, and I'll see if I can discern any obvious differences?

(The latter file doesn't have to be a huge 400MB file; you can get a smaller one by running in a subdirectory with only a few files in).

Thanks again for helping try to figure this out.

[snipercup]

I changed to your pr branch and tried running this command:
C:\Users\User\Documents\GitHub\Cataclysm-DDA>C:\Users\User\Documents\GitHub\Cataclysm-DDA\tools\json_tools\cddatags.py

This gives me a tags file of 2.3mb and I get the same results as I did the first time:

It gives me references to the values and a few keys like morale and dodge. I can press space to jump to the selected reference. It works on rifle_auto too. Looks like this fixes it. I didnt know I should run the cddatags.py from the root of the cdda project so please put that in the documentation.

Running Ctags with built in json-support gave me a tags file of 100mb except I made it in /data/json so I think we are getting the same results on that one.

[jbytheway]

I didnt know I should run the cddatags.py from the root of the cdda project so please put that in the documentation.

It shouldn't matter where you run it from. It figures out where to write the tags file relative to where the script itself is. Can you double-check if it really makes a difference?

[snipercup]

It does not make a difference where the script is run from. I deleted my existing tags file and confirmed the references were gone. Then I ran this command:
C:\Users\User\Downloads\ctags-2020-02-13_6ffdd48e-x64>C:\Users\User\Documents\GitHub\Cataclysm-DDA\tools\json_tools\cddatags.py
It create a new tags file 2.7mb in the cdda root directory. I tried the references again in tagleet and I get the same results as before. I can get the references for the values and jump to the definition of that value.

[jbytheway]

Excellent. Sounds like everything is working as expected, then, and this is ready to merge.