Harbour parser... Var=... implementation

[jvguti]

(Hello everyone, I found ctags by chance and I have been very interested.
I'm trying to make a .ctags file for the xBase Harbour language, I ran into the following problem:
Thank you for contacting us.

---

The name of the parser:
My own parser: Harbour.ctags
The command line you used to run ctags:
ctags.exe --options=Harbour.ctags --sort=no -f0_prueba.txt hb.prg

$ ctags --options=NONE ...

The content of input file:

The version of ctags:

$ ctags --version
Universal Ctags 0.0.0(EXAMPLE), Copyright (C) 2015 Universal Ctags Team
Universal Ctags is derived from Exuberant Ctags.
Exuberant Ctags 5.8, Copyright (C) 1996-2009 Darren Hiebert
  Compiled: May 11 1018, 23:16:36
  URL: https://ctags.io/

=======================================================================================
Problem 1 from:
=======================================================================================
    Function FunctionName ( par1 , par2   , par3 )

 ----I need to get this tag: ·FUNCTION FunctionName(par1,par2,par3)·    
    
    I thought the next code was going to work:(The lines are cut only to better read the groups, in reality it is a single line:)
    --_mtable-regex-Harbour=function/
    ([a-z_][a-z0-9_]*)[^(]*[(]              => this part presents the function name and release spaces and left parentesis.
    ([\t ]*([a-z_][a-z0-9_]*)[\t ]*)+       => Here I tried to found (spaces+(ParameterN)+spaces)+ one or more times.
    [^)]*\)                                 => eliminates posible rest of spaces or anything that not mached with "parameter template"
    /·FUNCTION \1(\3)/f/{icase}{tleave}     => Force to function name to be in capitals and brackets like I want.
    
Results: ·FUNCTION FunctionName(par3)·      => Only last parameter.

OK, I found that next formula works:
   --_mtable-regex-Harbour=function/
    ([a-z_][a-z0-9_]*)[^(]*[(]              => this part presents the function name and release spaces and left parentesis.
 ONLY THIS CHANGES:
     ([\t ]*([a-z_][a-z0-9_, ]*))+[^)]*\)    => Here I tried to found (spaces+(ParameterN ,)+spaces)+ one or more times.
LOOK FOR NEW Parameter template:([a-z_][a-z0-9_, ]*    
============    
    [^)]*\)                                 => eliminates posible rest of spaces or anything that not mached with "parameter template"
    /·FUNCTION \1(\3)·/f/{icase}{tleave}     => Force to function name to be in capitals and brackets like I want.

Results: ·FUNCTION FunctionName(par1 ,par2   ,par3 )·                => With all the right spaces.

The valor of \3 ONLY changes if I add space and comma to it caracters list in the template.
 
 =======================================================================================
 =======================================================================================

harbour language let us to incrust comments over every place, so i have this:

    Function  ( /*coment*/ Parameter1 /*comment*/,   /*coment*/ Parameter2 /*comment*/,.., /*coment*/ ParameterN /*comment*/)
    
--- I need this result: ·FUNCTION(Parameter1,Parameter2,..,ParameterN)·

Breaking it down into several lines is very easy:

### resultados 
--_mtable-extend-Harbour=function+skipComment
--_mtable-regex-Harbour=function/(\))/ \1·<=/f/{tleave}{scope=ref}
--_mtable-regex-Harbour=function/([a-z_][a-z0-9_]*)([^(]*[(])[\t ]*(\/\*[^(\*\/)]*\*\/)*/·FUNCTION \1(/f/{icase}{tenter=fparameters}

 ## fparameters: busca los parámetros de una función o procedimiento.
 ## fparameters: searches for the parameters of a function or procedure.
### quita comentarios, busca nombre de parámetro, quita comentarios, comprueba si hay coma, quita comentarios, hasta que haya un )
### removes comments, searches for parameter name, removes comments, checks for commas, removes comments, until there is a )

--_mtable-extend-Harbour=fparameters+skipComment
--_mtable-regex-Harbour=fparameters/([a-z_][a-z0-9_]*)[\t\n; \r]*/param:·\1·/P/{icase}{scope=push}{placeholder}
--_mtable-regex-Harbour=fparameters/(\.\.\.)/param:· \1 ·/P/{scope=push}
--_mtable-extend-Harbour=fparameters+skipComment
--_mtable-regex-Harbour=fparameters/(,)[\t\n; \r]*/paramSep:· \1 ·/P/{scope=push}
--_mtable-regex-Harbour=fparameters/(\))//P/{tleave}{_advanceTo=1start}   

Now I have this result but one line each one:
·FUNCTION FunctionName(
Parameter1
,
Parameter2
,
...
,
ParameterN
)·
 =======================================================================================
 Any help?
 =======================================================================================
I thought that if you could add to the universal ctags the possibility to save data (push)
 on a variable that was adding them and at any given moment, present it as a tag formed with a push,
 the problem would be solved and help solve many more situations.

[masatake]

 =======================================================================================
 Any help?
 =======================================================================================
I thought that if you could add to the universal ctags the possibility to save data (push)
 on a variable that was adding them and at any given moment, present it as a tag formed with a push,
 the problem would be solved and help solve many more situations.

You are correct. The current version of u-ctags cannot satisfy your requirement.

I'm thinking about introducing postscript alike language called optscript to users allow to
write a parser more flexible way. See #1577.

However, it is still an idea. I cannot say when the optscript is implemented.

[jvguti]

Thank you very much for your response, you have amazed me by the speed of answering.

[masatake]

https://sourceforge.net/p/ctags/patches/68/
Is this what you want ?

[jvguti]

I need to look at it in detail, compile it and check results.
I'll be briefing.
So at first glance it looks a lot like one I saw for Exuberant ctags, but it didn't serve
for universal ctags in principle because I used File and something else that is not now supported.
You're a machine and I'm thanking you again for your quick answers.

[jvguti]

Hello, I think that someone who masters the C language, and especially if he knows the code of the ctags program, it would be very easy to add a statistic variable to the top level of each language class and then access it by means of something like this:
..... saves the regex result of the current tag to var.
.... prints the result of var as if it were a tag.
"var-clear" ... clean var.

In this way I could subdivide the tags into more accurate results by joining them by means of var and presenting them when I already have them formatted to my liking making a pop.

I hope I understood it correctly.
A cordial greeting.

[jvguti]

I have checked harbour.patch and it does not work in universal ctags because it accesses variables that are now available through functions (they are statistical variables in universal ctags) apart from not following the new specifications.

It's a shame.

[masatake]

Maybe I can apply the patch with some modifications. The question is whether it will be useful or not.
I wonder whether the parser will gather the information you want.
Even if you don't read the code written in C language, you can evaluate the patch by applying it to Exuberant-ctags; evaluation can be done with Exuberant-ctags.

[masatake]

Hello, I think that someone who masters the C language, and especially if he knows the code of the ctags program, it would be very easy to add a statistic variable to the top level of each language class and then access it by means of something like this:
..... saves the regex result of the current tag to var.
.... prints the result of var as if it were a tag.
"var-clear" ... clean var.

IMHO, it is too ad-hoc.

[jvguti]

I have compiled and evaluated harbour.c on Exuberant ctags and it seems to work with this programming language as well as it could be with any other ones that are already incorporated.
In any case, I will take advantage of the working capabilities of universal ctags with files of the type .ctags that do not have to comply with the usual and adapt almost perfectly to what I need at all times. (If you had the tool ,var-push,'var'pop', 'var'clear', the flexibility to work would be immense).
Thank you very much again. You can incorporate harbour.c as an universal ctags parser, in principle it does everything that any other parser.

[jvguti]

Hello Mr. Masatake,
I have implemented the metods var-push, var-pop, var-pop_clear and var-clear that I needed to be able to do the job I was interested in. They work perfectly for me by just touching the lregex.c file.
Now I can take parts of a line out and present them together when I'm interested.
I don't know how I can send you the code in case you might be interested.
A big hug.
Jorge V.

[masatake]

I would like to see the .ctags file utilizing var-{push,pop,pop_clear,clear}.
Please, paste it here.

[jvguti]

The workflow is:

• from main finds the word function and execute function table.
• function finds the name of the function and executes fparameters table.
  -in fparameters each parameter and the separating commas are searched until reaching )

+The word function is saved with var-push in main, a midpoint is added.
+the name of the function is saved with var-push in function, a midpoint is added.

• As you find the parameters and the commas are added with var-push when found ) returns
  to main and collects and prints the result with var-pop_clear at the same time that the variable is reset.
• In the search for each name, parameter, comma, etc. a table is executed that removes possible comments
  and/or spaces that may have.

############################
MAIN TABLE WHEN THIS MATCH
############################
--_mtable-regex-Harbour=main/func(t|ti|tio|tion)?/FUNCTION·/f/{icase}{tjump=function}{var=push}{placeholder}

############################
TABLE function, looks function name end of function ( find ) )
############################
--_mtable-extend-Harbour=function+skipComment

nombre de la función.

--mtable-regex-Harbour=function/([a-z][a-z0-9_]*)/\1·/f/{icase}{var=push}{placeholder}

busca (

--_mtable-extend-Harbour=function+skipComment
--_mtable-regex-Harbour=function/(()/\1/f/{tenter=fparameters}{var=push}{placeholder}

con ) se termina y vuelve a main.

--_mtable-regex-Harbour=function/())/ \1·/f/{tjump=main}{var=pop_clear}
--_mtable-regex-Harbour=function/(.)/===== aqui no debería llegar \1/D/

############################
TABLE fparameters, looks for parameters, comma separator and (
############################

fparameters: busca los parámetros de una función o procedimiento.

quita comentarios, busca nombre de parámetro, quita comentarios, comprueba si hay coma, quita comentarios, hasta que haya un )

encuentra )

si encuentra ) sale de la funcion.Función sin parámetros o retorno normal.

--_mtable-extend-Harbour=fparameters+skipComment
--_mtable-regex-Harbour=fparameters/())//C/{tleave}{_advanceTo=1start}

el parámetro de la función es ...

--_mtable-extend-Harbour=fparameters+skipComment
--_mtable-regex-Harbour=fparameters/(...)/ \1 /P/{var=push}{placeholder}

búsqueda de parámetros y , separadoras.

--mtable-extend-Harbour=fparameters+skipComment
--mtable-regex-Harbour=fparameters/([a-z][a-z0-9])[\t\n; \r]/\1 /P/{icase}{var=push}{placeholder}
--_mtable-extend-Harbour=fparameters+skipComment
--_mtable-regex-Harbour=fparameters/(,)/ \1 /P/{var=push}{placeholder}

##############################################
This is one posible and complicated function to works with.
##############################################
FuncTi /* multicoment1 / OneFunc (/ multicoment2 /Parameter1/ multicoment3 /,; // Line coment
/ multicoment4 / Parameter2 / multicoment5 */)

##############################################

With the power of multitables and with the vars, I get the following result... which is the one I'm looking for in this case.

FUNCTION·OneFunc·(Parameter1,Parameter2)
##############################################

[jvguti]

I dont know how to send files in GitHub, sorry El mié., 10 jun. 2020 a las 13:53, Masatake YAMATO (< notifications@github.com>) escribió:

…

I would like to see the .ctags file utilizing var-{push,pop,pop_clear,clear}. — You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub <#2561 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APZGBXVDTLP4Q4KVCDCJ2WDRV5X4XANCNFSM4NPHYBNQ> .

-- Jorge V. Gutiérrez Rubio

[masatake]

Thank you.

--_mtable-regex-Harbour=function/())/ \1·/f/{tjump=main}{var=pop_clear}

I understand that ctags emits the contents of VAR as part of the name of a tag.

The question is \1. Does ctags append or prepend \1 to the name? Or does ctags
ignores \1 if {var=pop_clear} is specified?

[jvguti]

Var preappends results of regexec, not only \1.
When program found var=pop or var=pop_clear it print var contents.
Var is independent of rest of code, if you dont use placeholder, program prints regex at normal but save var. Only affect when found var=pop to print var and regext if not placeholder.
Understand?, My english is not very good, sorry.

[jvguti]

When I said preappend....:
if you hav 1 lines of code:

function myfunction(one,two);

you make one template regex: /function/FUNCTION: //{tenter=lookforname]{placeholder}{var=push}
I'm saving "FUNTION : " to var

in lookforname i have another template regex: lookforname/([a-z_][a-z0-9_}*)/name:\1//{icase}{placeholder}{var=push}
lookforname///{var=pop}

Now in var we have: "FUNTION : name:myfunction"

So, if we make var=pop, program prints "FUNTION : name:myfunction"

[jvguti]

tested that works well:

• Push: Adds the contents of the compiled regex to the END of what is in the Varstatus variable.
• Pop : prints the contents of Varstatus without deleting the variable. If there is placeholder do NOT add the regex to the result and the result is lost. If regex is empty and there is placeholder, the placeholder is disabled so that the result can be printed.
• Pop_clear: Same as Pop but Varstatus is cleaned.
• Clear: Varstatus is cleaned without affecting any other results.

I added a size control of the Varstatus variable of a maximum of 200 characters in case too many pushes were put without ever making a clear. If that happens an error message is issued and the Varstatus variable is cleaned allowing the code to continue running.

Added and tested:

• Tables now run sequentially without reinitializing the pointer every time there is a match. He continues where he was.
• When you make a return with tleave you return to the table following the call getting it to look like the way a program is run.

[jvguti]

jvguti:
Added {var=push}{var=pop}{var=clear}{var=pop_clear} to manage one accumulative var.
Added Now execution on tables are secuential, when match does not go to first subtable.
Added Now you can make an empty pattern with any of {command}.
--_mtable-regex-name_table///{command}...{command}
Added new {tenterst=} to instruct that when found a {tleave}, execution goes to first subtable of parent table.

All things checkeds

[masatake]

ctags introduced an interpreter implementing general purpose programming language.