[Go target] cannot create a custom token in a base class lexer

[kaby76]

I am trying to port python/python3 to the Go target. Specifically, it involves creating some custom tokens for indentation. Unfortunately, given the current API, it's impossible to do so. Therefore, this is a blocking bug.

There is no way to create and have called a Python3LexerBase.EmitToken() method for custom token creation. This method should be defined for the Go target (it's defined in the other targets, sometimes as Emit(token)). The call to EmitToken(token) is wrong too. It should be b.Virt.EmitToken(t). The Lexer interface requires a definition for EmitToken(token).

Rant: I'll say it again, nesting structs like this does not make an OO data structure. It's like this person didn't ever write a compiler that implements an OO language. We had to do this kind of stuff for years with C, because C doesn't support OO directly.

(In Python3Lexer3.g4)
options {superClass=Python3LexerBase;}


(In the generated Python3Lexer.go)
type Python3Lexer struct {
	Python3LexerBase
	channelNames []string
	modeNames    []string
	// TODO: EOF string
}

It should be:

type Python3Lexer struct {
	mybase *Python3LexerBase
	channelNames []string
	modeNames    []string
	// TODO: EOF string
}

[kaby76]

The fix is slightly more than the one line changed (PR to follow). But, this code must be updated because the port to Go for python/python3 requires full control of Emit()/EmitToken().