Supporting vue (Was: require new features with {scope}) #1577

Fmajor · 2017-10-12T18:16:46Z

for scope, now we have these flags

{scope=push}
{scope=ref}
{scope=pop}
{scope=clear}
{scope=set}
{placeholder}

In my custom syntax file, i need to parse the same pattern in different way with different context (or different stack status), so i come up with some new features with {scope}, like

{scope=replace}
    replace the top of stack (pop and push, not clear and push as '{scope=set}')
{scope=push(n)}
    push current match nth times
{scope=pop(n)}
    pop current match nth times

also, i'd like to use some boolean operation for the scope, like

{scope-top-kind=blabla}, {scope-top-kind!=blabla}
{scope-level=blabla}, {scope-level>blabla}, {scope-level<blabla}
    the --regex-X will run only when the expression are true

i'm wondering if someone else have the same requirements.

The text was updated successfully, but these errors were encountered:

masatake · 2017-10-12T18:54:44Z

Interesting idea but it needs consideration.

{scope=replace}
    replace the top of stack (pop and push, not clear and push as '{scope=set}')

Implementing this is not difficult. I wonder {scope=pop}{scope=push} does the
same as {scope=replace}

{scope=push(n)}
    push current match nth times

Ctags can push a language object which is tagged to the scope stack.
Pushing nth substring directly to the stack is not possible.
Consider scope entry has not only name but also kind.
kind is not specified in your notation.

BTW, matchTagPattern() in main/lregex.c is the function where scope stack is used.
Let's think about the following code of imaginary language:

var Parent::item;

I guess you may want to make the following tag:

item  ipnut.X /^var Parent::item;/;" scope:???:Parent

We have to fill ???.

I know how useful the feature you propose is.
However, tool fill the ???, I have to implement the way to handle reference tag in regex parser.
See #1260

With the reference tag, you can capture "Parent" in the code as a reference tag.
A reference tag has a kind. So we can fill ??? with the kind.

I'm working on implementing the way to capture reference tags via regex parser very slowly.
However, it will take time.

Your proosal may be implemented on that.

Do you have any target language?
I would like to see.

#. an example input file (shorter is better)
#. expected tags output
#. IDEAL command line options to make the output from the input.

You can use any imaginary options in the last item.

BTW, matchTagPattern() is the function in which the scope stack is uesd.

Fmajor · 2017-10-12T20:20:19Z

i am writing Vue, like this (i have delete all the function body..)

<script>
import _ from 'underscore'
import MTableRow from './m-table-row'
let eachColumn = {
  'data': [],  // do not want to match this!
  'subfields': {},
}

function abyb (a, b) {
   .....
}

export default {
  name: 'm-table',
  props: {
    'data': {
      'type': [Object],
      'default': function () {
        return {
          'columns': [],
          'fixColumnCount': null
        }
      }
    }
  },
  data () { // want to match this
    return {
      'addable': false,
      'deletable': false
    }
  },
  computed: {
    style () {
    },
    headerIDStr: {
      cache: false,
      get () {
      }
    }
  },
  'methods': {
    initData () {
    },
    getColumn (index) {
    },
    getColumnShow (index) {
    },
    updateColumnSort () {
    },
    getData (data, index) {
    },
    updateRowSort (sortBy) {
    },
    onMouseOverHeader () {
    }
  },
  created () {
    this.initData()
  },
  components: {MTableRow}
}
</script>

the name, data, props, computed, methods, created in this context (<script></script> ==> export default {} ) are things i want to display, like this

--regex-vue=/^ {2}((name|'name')[^,]*)/\1/p,property/{scope=ref}
--regex-vue=/^ {2}((data) \(|('data'):|(data):)/\2\3\4/p,property/{scope=replace}
--regex-vue=/^ {2}((props) \(|('props'):|(props):)/\2\3\4/p,property/{scope=replace}
--regex-vue=/^ {2}((computed) \(|('computed'):|(computed):)/\2\3\4/p,property/{scope=replace}
--regex-vue=/^ {2}((methods) \(|('methods'):|(methods):)/\2\3\4/p,property/{scope=replace}
--regex-vue=/^ {2}((created) \(|('created'):|(created):)/\2\3\4/p,property/{scope=replace}
--regex-vue=/^ {2}((components) \(|('components'):|(components):)/\2\3\4/p,property/{scope=replace}
--regex-vue=/^ {4}(([a-zA-Z0-9_]*) \(|('[a-zA-Z0-9_]*'):|([a-zA-Z0-9_]*):)/\2\3\4/p,property/{scope=ref}

i just want to match the "data ()" in the export default {}, not before that in "let eachColumn = {}", so i want the feature of boolean expression to make a constrain, like

--regex-vue=/^(export default)/\1/e,export/{scope=push}
--regex-vue=/^ {2}((name|'name')[^,]*)/\1/p,property/{scope=ref}{scope-top-kind=export}

Fmajor · 2017-10-12T20:22:11Z

also, i tried {scope=pop}{scope=push}, it act only as {scope=push}

b4n · 2017-10-12T22:23:01Z

Looks like a job for the JavaScript parser for it to emit something for the export statements. Would that work, or is it too much a problem because the JS is embedded in something else CTags doesn't (yet?) forward that part to the JS parser?

masatake · 2017-10-13T02:03:14Z

Would that work, or is it too much a problem because the JS is embedded in something else CTags doesn't (yet?) forward that part to the JS parser?

It is possible to run js parser areas which are embedded in another language like html.

As far as seeing https://vuejs.org/images/vue-component.png,
One needs to write a vue parser which detects areas <script>...</script> and applies JS parser(or css parser) to the areas.

Writing the vue parser in C language is one of approach. More interesting one is extending regex
mline regex parser to allow wriging following command line.

--_mline-regex-Vue=/<script>(.|[\n])</script>///{area-start=1start}{area-end=1end}{guest-parser=JavsScript}

More studying vue is needed.

masatake · 2017-10-13T02:27:30Z

After thinking again, we don't have to extend the mline-regex parser right now. Even writing in C language, picking the areas (<script>...</script>) is not difficult. I did the same in yacc parser.

Fmajor · 2017-10-13T03:39:52Z

actually i do not want to run a JavaScript parser in the <script>...</script> area, that will list too many symbols, i want to focus on the overview structure of the .vue file, and list
all keys in the export default {} dict which have 3 format

export default {
  'name': 'test', // format1, this is a item with value
   props: { // format2, this is a item with value, but without quote
      'subkey': 'test'
   },
   data () { // format3, this is a function
       return {
           'result': 'test'
       }
   }
}

if we have format1 and format2(a object key)
    if the value is not a dict, extract it directly
    elif the value is a dict, extract it recursively
if we have format3 (a function)
    extract its return value (if it's a dict, extract recursively)

i don't know if it's possible without write a custom parser and only with improved regex-Vue command

=================update================
i think i can do the above things in the following logic (with several new command i want)

# first detect export default environment (use scope=replace because we are already in the <script> environment)
--regex-vue=/^export default/export default/e,export/{scope=replace}
# i want to parse only in the export environment with {if-stack-include-kind=e}
# if matched, we want to go into the { } environment either for a dict or a function
# so we need a way to correctly pop the stack after leaving the environment
# the method is push stack now, and do not push the next '{', pop all '}', and we will have the right stack status
# {skip} means not record this match
--regex-vue=/^ *(([a-zA-Z0-9_]*) \(|('[a-zA-Z0-9_]*'):|([a-zA-Z0-9_]*):)/\2\3\4/p,property/{if-stack-include-kind=e}{scope=push}{let-ignoreNext=1}{else}{skip}
--regex-vue=/\{//i,ignore/{if-stack-include-kind=e}{if-ignoreNext-ne=1}{scope=push}{else}{let-ignoreNext=0}{else}{skip}
--regex-vue=/\}//i,ignore/{if-stack-include-kind=e}{scope=pop}{else}{skip}

that is to add variables and if expression into the --regex syntax, i thing it will help a lot and make the ctags much more powerful

Fmajor · 2017-10-13T03:53:51Z

i use vim+tagbar a lot, but i just got to use ctags in this custom way yesterday. i find it a lot help to navigate vim in well structured file(e.g: automatically detect some keywords), so i want more features (like boolean operation during reg match, or even add variables in the --regex variable to help resolve complex context)

here is what i get now, it's not totally right, but still help me a lot to show me the over structure and navigate around it.

Fmajor · 2017-10-13T03:59:46Z

If the requirement are too complex, i'd like to try to write a custom parser, is there any demo or doc for that?

codebrainz · 2017-10-13T05:43:44Z

If it were me, I'd just hack the Vue code to make it emit ctags format, since it already knows how to parse the single file components and embedded languages.

Fmajor · 2017-10-13T05:51:09Z

hi @codebrainz

how to parse the single file components and embedded languages?
just as what @masatake said?

Writing the vue parser in C language is one of approach. More interesting one is extending regex
mline regex parser to allow wriging following command line.
--_mline-regex-Vue=/<script>(.|[\n])</script>///{area-start=1start}{area-end=1end}{guest-parser=JavsScript}

what does it mean to 'hack the Vue code to make it emit ctags format'?

codebrainz · 2017-10-13T05:56:55Z

how to parse the single file components and embedded languages?
just as what @masatake said?

Probably, I'm the wrong person to ask, I only ever wrote a very simple parser for a simple language for the old ctags.

what does it mean to 'hack the Vue code to make it emit ctags format'?

I mean to modify the front-end code of Vue.js and make it only emit ctags' simple tags format to a file rather than doing all the other stuff Vue.js does after the front-end. I have no idea how practical that would be, but on the surface it seems simpler.

Fmajor · 2017-10-13T06:01:49Z

Yes, you are right. i use Webpack to handle my project, so maybe i can write a preload plugin to generate ctag file using javascript, it's much more easier, thanks.

Fmajor · 2017-10-13T06:13:20Z

i read the ctags documents, it says

A regex-based parser is inherently line-oriented (i.e. the entire tag must be recognizable from looking at a single line) and context-insensitive (i.e the generation of the tag is entirely based upon when the regular expression matches a single line).

the universal-ctags add {scope} flag so that we can record the context during parsing, i think the things i talk above are to use the context variable and make the regex-based parser context-sensitive, that will make the parser much more powerful

as far as i understand, the parser try all the regex one by one for each line, so i think we can add some 'if' flag or 'set variable' flag in the syntax, the command should run before or after doing actual regex match

'if' flag decide whether or not to record the match and whether or not to change the stack
'set' flag give variable new value, which may be used in later 'if'

like this

--regex-X=/pattern1/match/n,name/{if-y-eq=1}{set-x=1}{scope=push}{record}{else}{record}
--regex-X=/pattern2/match/n,name/{if-x-gt=2}{set-x=0}{scope=pop}{record}{else}{skip}
convert to:

for eachline in file:
    for eachPattern in allPatterns:
        match = parse(eachPattern, eachline)
        if match:
            ...

# for pattern1, the code in ... may like
    if y==1:
        x = 1
        stack.push(match)
        record(match)
    else:
        record(match)

# for pattern2, the code in ... may like
    if x>2:
       x = 0
       stack.pop()
       record(match)
    else:
       continue

that is to make the {flags} a small language with only 'if' expression and basic variable assignment and operation

masatake · 2017-10-15T08:21:56Z

Of course, I would like to add the code for parsing javascript source code using vue framework to ctags.

I think adding vue parsr as a sub parser of javascript parser is the best. Reducing the information can be done after parsing. Give me time. I'm catching up the discussion and learning vue.

Fmajor · 2017-10-15T08:41:42Z

@masatake
thanks a lot!

Anyway, i still thinks it will make ctags much more powerful adding the "if" and "set" feature(i'd like to use it on some formatted file to generate navigation flags), since it will make the parser context-sensitive. This feature may cost some time to develop, you can think about it later.

Also as you said, adding the {scope=replace} flag is not difficult, so please consider to develop this feature first when you have time.

codebrainz · 2017-10-15T17:24:46Z

I think adding vue parsr as a sub parser of javascript parser is the best.

I have no idea if this makes sense in the context of ctags parsers, but a single-file view component is almost identical to a regular HTML document containing embedded CSS and JS. Does it make sense to improve the HTML parser to somehow be able to handle CSS and JSS (using subparser)?

masatake · 2017-10-15T20:55:40Z

@codebrainz, you may correct. Using HTML as a host parser will be better. Anyway, I think we can do very interesting thing with existing code.

masatake · 2017-10-19T21:24:09Z

https://vue-loader.vuejs.org/en/start/spec.html

@codebrainz, do you think we should run html parser for .vue file?

My idea is introducing a small vue parser.
vue parser may run html parser for <template>...</template> areas, JavaScript parser for <script>...</script> areas, and css parser for <style>...</style> areas.

masatake · 2017-10-19T21:54:35Z

@Fmajor, my understanding is that what you want is the top level keys in export default block.
JavaScript parser has ability to capture keys. So many code for implementing what you want is in ctags.
What we have to do is specializing JavaScript parser to Vue.

export default. Is this special phrase of Vue or standard phrase of JavaScript?

codebrainz · 2017-10-20T05:15:44Z

@masatake it's part of JavaScript proper, and I don't believe it's the only way to achieve the same thing for Vue.

masatake · 2017-10-20T06:50:17Z

Oh, @b4n already wrote about "export". If JavaScript parser handling the block well, all parts of ctags may work fine. export is something like prototype in C. I like using reference tag with "unknown" kind and ... "exported" role. Anyway a special kind for it is needed. Maybe it is what @Fmajor wants. When thinking about scope, things are not simple. If a::b is exported in c, I wonder which scope b should have: a or c. Here b is taged as a reference tag. In natural langauage, I can say b defined b is exported from c. Or c::a::b? 2017/10/20 午後2:15 "Matthew Brush" <notifications@github.com>:

…

@masatake <https://github.com/masatake> it's part of JavaScript proper <https://developer.mozilla.org/en-US/docs/web/javascript/reference/statements/export>, and I don't believe it's the only way to achieve the same thing for Vue. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1577 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAEtFTv5GsERS-SSyxhGV4IrXBRscLyAks5suCyAgaJpZM4P3arD> .

masatake · 2017-10-20T07:23:02Z

Surprisingly a function can be defiend in export block...
So using unknown/exported role/kind combination doesn't make sense (or it not enough.).

masatake · 2017-10-20T07:30:57Z

@b4n, ctags vue parser I implemented can push an area inside <script>...</script> to JavaScript parser.
I will make a branch for vue.

So could you consider how JavaScript parser deals with the export default block.
@Fmajor has interest only the sub set of the output of JavaScript parser. Such filtering can be done
in later.

VueParser calls JavaScript parser as a guest parser.
JavaScript parser parsers export default block well.
JavaScript calls VueJavaScript parser as sub parser.

VueJavaScript subparser picks up only
the name, data, props, computed, methods, created keys.

Fmajor · 2017-10-20T09:31:49Z

I tried to apply the latest version ctags install from homebrew on some js file, but it failed to extract the "export default" part

let me explain my requests (and why i need it) in more detail.
one month ago, i began to learn web front-end developing and start to write javascript code. Before that, i only write python and c code. I use vim and i have my costum folding plugin, so i can always quickly nagivate to the function/variables i want, i actually do not use ctags at that time.

But things are different when i change to .vue (javascript).

a .vue file are usually like

<template>
  <!-- some html code -->
  <div> ... </div>
  <!-- some html code with v-command (like v-if, v-for) and injected js command -->
  <div v-if="showTag === true">
    <div v-for="eachItem in data"> ... </div>
  </div>
</template>

<script>
// local functions and variables
var a = 1, b = 2
function test () {}
var d = function () {}
var e = () => {}

export default {
  name: 'name of a component',
   props: {
     'data': {
       'type': [Object],
       'default': function () {
         return {
           'columns': [],
           'fixColumnCount': null
         }
       }
     }
   },
  data () {
    return {
      'show': false
    }
  },
  computed: {
    style () {
    },
  },
  'methods': { // this is usually a LARGE dict and will take 50% of the total file
    someMethod0 () {
    },
    someMethod1 () {
    },
    someMethod0 () {
    }
  },
  created () {
  },
  components: {MTableRow}
}
</script>

<style>
</style>

vue file descript properties of a vue component, we care only about the overall structure of it

my final purpose is to quickly nagivate to the variable/functions i'm working on, in a vue file, those things are

some vue command in the part like v-if, v-for (because we have injected javascript code here)
style definitions in the <style> part
local function and variables in script (those things are easy ones if we apply a javascript parse upon the <script></script> part, in this example, they are variable a,b and function test,d,e)
the most import things, the keys (and even subkeys) in the dict we export, this part descript the properties of this vue component
in this example, they are
1. variable 'name'
2. function 'data' (this function usually immediately return a dict, what we care about is the keys in the returned dict, variable 'show' here)
3. function 'methods' (this part may take 50% lines for the whole file, we want to parse key names(which are function names) of it)
4. function 'created' (and some other functions 'updated', 'beforeUpdated'.....)
5. variabel 'props' (we also want to list all the keys of it like 4.3)
6. ...others keyword not shown in the example
the export default is a syntax sugar, it just means export the following {} as "default", we can change it to

var toExport = {
    'name':'name of it',
    data () {
    },
    ....
}
export {toExport as default };

so the export default part is just a normal dict {}, nothing special

it seems to parse the vue perfectly if use different parser in the <template>, <script> and <style>, but if so, only part 2 and part 3 is parsed, we miss part 1 and the most import part 4, becaure the export part is only a normal variable, javascript parse will NOT go into it and parse it recursively.

usually in a vue file, the export part will take 80% lines of the file, if we just parse the export dict as a single value, it's useless. what all the vue people what is to

parse the export dict like a syntax analyzer (which must be context-sensitive) and list the keys (event some sub keys like 4.2, 4.3 and 4.5) inside it

even if the 'default' dict is specially parsed, it's still not enough, we need to recursively parse the dict at least to level tow. as show in 4.3 the 'methods' dict which contains many functions usually take 50% lines of this file, we want to list these functions, so we need to parse the dict like a syntax analyzer.

because people usually use vue-cli and webpack to start and manage a vue project, which use eslint as the syntax checker, the vue file are always in good format, its easy to satisfy my requests using custom --regex-vue, i have post a figure above, its not perfect but usable for me, and save me a lot of time in nagivation these days.
if you can add some new features for the {scope} and make the parse process context-sensitive (just some 'if' features to control which tag to output, to control whether or not to modify the stack), i can make my .ctags perfect and achieve all i want. Also i'd like to use ctags later as a "general navigation tags generator" for all kinds of well formated files.

I understand these features may take a lot of time, i just give my suggestion here. Maybe it's my own minorities and niche interests, but if you think it reasonable, please add these features (maybe in your own ways).

if we just want to solve the vue parser problem, we need to write a special parser, the solution of samply applying javascript parser on the <script> part will lose useful symbols of on average 50% lines of the total file, since functions in 'method' is in the second level depth, we need a context-sensitive syntax analyzer like parser to parse it.

ps: IF the javascript parse CAN recursively parse all the dict, there will be anohter problem: toooooooooo many tags (since we only care about the overview structure). we can make some program to filter tags file later, but it's hard to integrate that program into plugins in many editer(like tagbar in vim and Symbal-tree-view in atom), these plugins will autorun "ctags" command after you modify the file and then update the visual tree view of the tags file, we need to add some 'filter' properties into ctags itself. The context-sensitive features in {scope} will help do this.

masatake · 2017-10-20T22:11:01Z

See #1581. I made a prototype that parsers inside export default block in a vue file.
Again, we can reduce tags later.

masatake · 2017-10-23T13:48:09Z

The prototype is updated. See 61d77c8 .

At least we have to do

merging main,refactor: Don't use kindDefinition type in tagEntryInfo structure #1584,
extending javascript parser to support "export" keyword.

This is the first time stacking 3 parsers for single input(Vue/JavaScript/VueJavaScript).
Surprisingly it works.

@Fmajor, your proposal about extending scope long flags of regex parser is interesting. However, I cannot find the way to implement some parts of the proposal. (replace is o.k. I will implement it when I find time.) As I showed in the commit log, implementing vue parser as a built-in parser is easier, though it is not perfect yet. I have not read your comments well yet. So I will read and tune my changes.

masatake · 2019-07-04T10:22:00Z

See above session:

[yamato@slave]/tmp% git clone https://github.com/universal-ctags/ctags.git
Cloning into 'ctags'...
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (11/11), done.
remote: Total 41440 (delta 0), reused 4 (delta 0), pack-reused 41429
Receiving objects: 100% (41440/41440), 12.76 MiB | 2.17 MiB/s, done.
Resolving deltas: 100% (26290/26290), done.
[yamato@slave]/tmp% cd ctags                                        
[yamato@slave]/tmp/ctags% git checkout -b masatake-optlib-based-vue master
Switched to a new branch 'masatake-optlib-based-vue'
[yamato@slave]/tmp/ctags% git pull https://github.com/masatake/ctags.git optlib-based-vue
remote: Enumerating objects: 457, done.
remote: Counting objects: 100% (377/377), done.
remote: Compressing objects: 100% (100/100), done.
remote: Total 295 (delta 218), reused 267 (delta 192), pack-reused 0
Receiving objects: 100% (295/295), 66.80 KiB | 453.00 KiB/s, done.
Resolving deltas: 100% (218/218), completed with 70 local objects.
From https://github.com/masatake/ctags
 * branch              optlib-based-vue -> FETCH_HEAD
Merge made by the 'recursive' strategy.
...

andbar-ru · 2019-07-04T10:29:44Z

It worked. Thanks!

masatake · 2019-07-04T10:32:28Z

@andbar-ru, thak you.

template can be not only HTML. It can be for example <template lang="pug">

This is not an issue yet because ctags hs not pug parser yet.
If one wrote a bug parser, integrating it to vue parser is not difficult.

And it can contain nested 's.

This is a critical issue. At least I have to make the guest regex flag stackable.
Could you show me a good example of nested 's?

style can ve not only CSS: less, sass, stylus.

This is not an issue yet because ctags doesn't have less, sass and stylus parsers.
If one write one of them, integrating it to vue parser is not hard.

javascript parser can collect many redundant entries.

Do you mean ctags emits too many tags?
If yes, I think that should be controlled with the command line options(.e.g. --kinds-JavaScript=-...) of ctags or external tools. Anyway we can reduce tags later development.

Just a cavil. javascript objects ({...}) are not classes in the sense of those in Java or C++, they are just objects, and arrays are objects too, and even classes class {...}

There is no choice. Exuberant-ctags introduced "class" kind ago. Universal-ctags follows it.

Anyway, I think advanced users like you can enjoy the optlib-based-vue branch.

andbar-ru · 2019-07-04T11:05:37Z

Use cases of nested <template>'s can be found on page https://vuejs.org/v2/guide/conditional.html. Nested templates are used when one needs combine multiple tags under common directive, for example v-if, v-else, v-show.
Sample from my project:

78 lines. Click to expand

<basemap

ref="basemap"

class="geotools-basemap"

:geoObjects="geoObjects"

:allowEditing="true"

:drawButtons="config.drawButtons"

:drawnLimit="1"

:drawnStyle="config.drawnStyle"

:useOnlySecuredTiles="false"

:tileSource="config.tileSource"

></basemap>

<div class="item" id="shapesSection" @click="section = 'Shapes'">Рисование</div>

<div class="item" id="spatialSection" @click="section = 'Spatial'">Данные ГПИ</div>

<div class="item" id="scriptsSection" @click="section = 'Scripts'">Сценарии работы</div>

</div>

<a class="item" id="clearCollection" :class="{disabled: !geoObjects.length}" @click="run('clearCollection', [])"><i class="icon remove"></i>clear all</a>

<a class="item" id="replaceCollection" @click="run('replaceCollection', spatialData.slice(1))"><i class="icon block layout"></i>set other</a>

<a class="item" id="setSpatialStyle" :class="{disabled: !geoObjects.length}" @click="run('setSpatialStyle', geoObjects[0].id, {color: 'lime', fillColor: 'orange'})"><i class="icon flag"></i>set style</a>

<a class="item" id="resetSpatialStyle" :class="{disabled: !geoObjects.length}" @click="run('resetSpatialStyle')"><i class="icon flag outline"></i>reset style</a>

<a class="item" id="selectItem" :class="{disabled: !geoObjects.length}" @click="run('selectItem', geoObjects[0].id)"><i class="icon plus"></i>select</a>

<a class="item" id="deselectItem" :class="{disabled: !geoObjects.length}" @click="run('deselectItem', geoObjects[0].id)"><i class="icon minus"></i>deselect</a>

<a class="item" id="replaceSelection" :class="{disabled: !selectedGeoObjects.length}" @click="run('replaceSelection', geoObjects.slice(1).map((o) => { return o.id }))"><i class="icon flag exchange"></i>reselect</a>

<a class="item" id="clearSelection" :class="{disabled: !selectedGeoObjects.length}" @click="run('clearSelection')"><i class="icon flag remove"></i>unselect</a>

<a class="item" id="loadAssetTiles" :class="{disabled: !selectedGeoObjects.length}" @click="run('loadAssetTiles', selectedGeoObjects[0].id)"><i class="icon table"></i>load tiles</a>

<a class="item" id="removeAssetTiles" :class="{disabled: !selectedGeoObjects.length}" @click="run('removeAssetTiles', selectedGeoObjects[0].id)"><i class="icon tv"></i>remove tiles</a>

</template>

<a class="item" id="addLabel" @click="run(

'addLabel',

'<myLabelUniqueIdentificator>',

[59.99204, 29.76933],

'Кронштадт. Петровский док',

undefined,

{

'color': 'lime',

'border': '1px solid yellow',

'background-color': 'rgba(0, 0, 0, 0.5)',

'padding': '5px',

}

)"><i class="icon font"></i>add label</a>

<a class="item" id="setLabelStyle" @click="run(

'setLabelStyle',

'<myLabelUniqueIdentificator>',

{'background-color': 'red', 'border': '1px solid lime'}

)"><i class="flag checkered icon"></i>label style</a>

<a class="item" id="removeLabel" @click="run('removeLabel', '<myLabelUniqueIdentificator>')"><i class="icon remove"></i>remove label</a>

<a class="item" id="deactivateDrawingTool" @click="run('deactivateDrawingTool')"><i class="icon hand paper"></i>deact tool</a>

<a class="item" id="setShapeStyle" @click="run('setShapeStyle', {'color': 'maroon', 'fillColor': 'red'})"><i class="icon flag"></i>set style</a>

<a class="item" id="resetShapeStyle" @click="run('resetShapeStyle')"><i class="icon flag outline"></i>reset style</a>

<a class="item" id="clearAllShapes" @click="run('clearAllShapes')"><i class="icon remove"></i>clear shapes</a>

<a class="item" id="getGeoJsonData" @click="testGetGeoJsonData()"><i class="icon info"></i>log geojson</a>

</template>

<a class="item" id="loopedSelection" @click="play('loopedSelection')"><i class="icon recycle"></i>loop-style</a>\

<a class="item" id="batchAddAndStyle" @click="play('batchAddAndStyle')"><i class="icon group object"></i>batch-add</a>\

<a class="item" id="addLayerAsDrawn" @click="play('addLayerAsDrawn')"><i class="icon plus object"></i>layer-add</a>\

</template>

</div>

</div>

</template>

andbar-ru · 2019-07-04T11:17:47Z

I think that vue syntax is too complex to be parsed with regular expressions, particularly with --mline-regex. <script> section can also contain inline <template>'s. Any of sections (<template>, <script>, <style>) can contain nested <section-name>: directly or in comments or in strings.

andbar-ru · 2019-07-04T11:34:50Z

Can one apply {_guest} to a table defined with --_tabledef-?

masatake · 2019-07-04T11:52:30Z

Yes. The flag is applicable to mtbale parsers.
See 0da7f2c as an example.

masatake · 2020-05-20T14:00:19Z

The original post is still very meaningful.
However, I don't want to add flags on demand. Instead, we should have a language.
Two ideas.

One is postscript alike language:

{{ 
0 scope-stack-ref dup { 
                /kind field-ref /method eq { 
                current-entry scope-replace 
           } { 
             current-entry scope-push
          }  ifelse  
} {
pop
} ifelse
}}

Another one is scheme alike language:

{(scope-action (let1 parent (ref scope-stack 0)
                           (if (eq? (ref parent 'field) 'method)
                               'replace
                               'pop) (current-entry))}

I would like to assume there is no circler dependenc. So the interpreter can manage the memory objects with reference counters.

I already use scheme alike language in readtags. So we can reuse it...however, I incline to postscript.
I cannot explain why.

It will be nice to allow users to define procedures:

--init-mylang={ 
0 scope-stack-ref dup { 
                /kind field-ref /method eq { 
                current-entry scope-replace 
           } { 
             current-entry scope-push
          }  ifelse  
} {
pop
} ifelse
}
...
--regex-mylang=/...///{{scope-push-or-replace}}

--init-mylang=(define scope-push-or-replace ()
(scope-action (let1 parent (ref scope-stack 0)
                           (if (eq? (ref parent 'field) 'method)
                               'replace
                               'pop) (current-entry)))
...
--regex-mylang=/...///{(scope-push-or-replace)}

Strahinja · 2020-05-21T15:40:36Z

What is the current status of Vue support in ctags? Any ETA?

masatake · 2020-05-21T16:50:07Z

No visible advance. However, in the user-invisible part, there are many advances.
I will rework on JavaScript parser. It is needed to support Vue.

masatake · 2020-06-14T08:13:20Z

I have implemented a postscript-alike language called optscript.
Now the optscript standalone interpreter can define a procedure.

$ ./optscript 
OPT> /add3 { 3 add } def
OPT> 4 add3
OPT<1> pstack
7

The next step is implementing operators for building control structure: if, ifelse, repeat, for, and so on.
Then we can link the library part of the optscript to ctags. I guess we can borrow many ideas from AWK.

masatake · 2020-12-13T19:29:25Z

Tonight, I linked the optscript interpreter to ctags. "hello, world" worked.

$ cat meson_options.txt
option('qemu_suffix', type : 'string', value: 'qemu', 
       description: 'Suffix for QEMU data/modules/config directories (can be empty)')
$ cat mesonOptions.ctags
--langdef=MesonOptions
--map-MesonOptions=+(meson_options.txt)

--kinddef-MesonOptions=s,string,strings
--kinddef-MesonOptions=b,boolean,booleans
--kinddef-MesonOptions=c,combo,combos
--kinddef-MesonOptions=i,integer,integers
--kinddef-MesonOptions=a,array,arrays
--kinddef-MesonOptions=f,feature,features
--mline-regex-MesonOptions=/^option[[:space:]]*\([ \t\n]*'([^']*[^\])'[[:space:]]*,[[:space:]]*type[[:space:]]*:[[:space:]]*'([a-z]+)'//{mgroup=1}{{
  /myproc { 3 mul } def
  (hello)
  (, )
  (world)
  1 2 add myproc
  pstack
}}
$ ./ctags --options=mesonOptions.ctags -o - meson_options.txt  
9
(world)
(, )
(hello)
$

masatake · 2020-12-16T20:37:29Z

Other than error handling, the optscript interpreter works well.
Now I enter the stage to design predefined operators and predefined variables.

$$ is a tag entry just created with the regular expression.
ss is the prefix for operators manipulating and accessing the scope stack.

sscount (- sscount <integer>)
sspop ( - sspop -)
ssref ( <integer> ssref <tagEntry>)
sstop (- sstop <tagEntry>)
ssexch (- ssexch -) -> /ssexch { sstop sspop sstop sspop exch sspush sspush } def`
sspush (<tagEntry> sspush -)
...

field: is an operator for accessing the field of a given tag entry. You can use this like $$ kind: => /method.
:field is an operator for setting a value to the field of a given tag entry. You an use this like $$ /method :kind

{{
    sscount 0 gt {
         sstop kind: /method eq {
         % pop the top of the scope stack only if the top is a tag for a method kind language object.
              sspop
         } if
         % fill the scope field of the newly created tag with the tag at the top of the scope stack.
         sstop $$ :scope
    } if
    % push the newly created tag to the scope stack.
    $$ sspush
}}

This looks much better than the original one:

{{ 
   0 scope-stack-ref dup { 
         /kind field-ref /method eq { 
           current-entry scope-replace 
         } { 
            current-entry scope-push
         }  ifelse  
   } {
         pop
   } ifelse
}}

How can I implement {scope=push(n)} in the first comment?
We cannot push a string to the scope stack. What we can push is a tag entry.

input.x:

var Parent::item;

I would like to capture item with scope: Parent. Here I assume Parent is a class.

--langdef-X
--map-X=.x
--kinddef-X=m,member,members
--kinddef-X=c,class,classes
--roledef-X.{class}=referenced,...

--regex-X=/var ([A-Za-z]+)::([a-z]+)/\2/m/{{
   \1 /class /referenced tagentry
   commit
   $$ exch :scope
}}

tagentry ( <string> <kind-name> <role-name> tagentry <tagentryTMP> or <string> <kind-name> tagentry <tagentryTP>)
commit (<tagentryTMP> commit <tagentry>)

After committing, we can do as we want.
Though I should take more time for the design for operators making a tag I myself want to write a parser with the operators.

masatake · 2021-01-05T15:19:35Z

I decided to use . instead of $$.
See https://ctags.io/2021/01/05/optscript/.

masatake · 2021-03-26T22:04:55Z

scope-related operators are implemented in optscript branch.

$ ./ctags --_list-operators  | grep scope
        :scope	->	int :SCOPE int true
     _scopeNth	->	index:int _SCOPENTH int
   _scopeclear	->	- _SCOPECLEAR -
   _scopedepth	->	- _SCOPEDEPTH int
     _scopepop	->	int _SCOPEPOP -
     _scopeset	->	int _SCOPESET -
     _scopetop	->	- _SCOPETOP int
        scope:	->	int int SCOPE: -

$  cat args.ctags
--langdef=X
--map-X=.unknown
--kinddef-X=p,package,packages
--kinddef-X=d,def,definitions

--regex-X=/def[ ]+([a-z]+)[ ]*:[ ]*([a-z]+)/\1/d/{{
	. [ (typename) \2 ] typeref:
    . _scopetop scope:
}}

--regex-X=/package[ ]+([A-Z]+)[ ]*/\1/p/{{
	. _scopeset
}}
$  cat input.unknown
package X

def a:int = 1
def b:str = "abc"

package Y

def c:name = name

$  u-ctags -o - --options=args.ctags input.unknown
X	input.unknown	/^package X$/;"	p
Y	input.unknown	/^package Y$/;"	p
a	input.unknown	/^def a:int = 1$/;"	d	package:X	typeref:typename:int
b	input.unknown	/^def b:str = "abc"$/;"	d	package:X	typeref:typename:str
c	input.unknown	/^def c:name = name$/;"	d	package:Y	typeref:typename:name

@Fmajor