-
-
Notifications
You must be signed in to change notification settings - Fork 1.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make emit pragma work as expected for JS target with sections #13376
Comments
I found the following on the Nim support for ES modules topic:
The Js backend support was my main draw to Nim (one ring to rule them all, back to front) so sad to see that little attention is given to the JS backend |
I would propose changing the proc genEmit(p: BProc, t: PNode) =
var options = p.options
var s = genAsmOrEmitStmt(p, t.sons[1])
var withSections = options.sections or p.prc == nil
if withSections:
# top level emit pragma?
let section = determineSection(t[1]) Then in proc gen(p: PProc, n: PNode, r: var TCompRes) =
p.options.sections = true
r.typ = etyNone |
My current best attempt at fixing this behavior In TOption* = enum # **keep binary compatible**
optNone, optObjCheck, optFieldCheck, optRangeCheck, optBoundsCheck,
optOverflowCheck, optNilCheck, optRefCheck,
optNaNCheck, optInfCheck, optStyleCheck,
optAssert, optLineDir, optWarns, optHints,
optOptimizeSpeed, optOptimizeSize, optStackTrace, # stack tracing support
optLineTrace, # line tracing support (includes stack tracing)
optByRef, # use pass by ref for objects
# (for interfacing with C)
optProfiler, # profiler turned on
optImplicitStatic, # optimization: implicit at compile time
# evaluation
optTrMacros, # en/disable pattern matching
optMemTracker,
optLaxStrings,
optNilSeqs,
optOldAst,
optSections Then in
In proc genModule(p: PProc, n: PNode) =
p.prc.options.incl optSections Then compile to js on var x = 2
Unfortunately I keep getting this weird error: nim js sandbox/try.nim
Hint: used config file '/Users/kristianmandrup/repos/nim-projects/Nim/config/nim.cfg' [Conf]
Hint: used config file '/Users/kristianmandrup/repos/nim-projects/Nim/config/config.nims' [Conf]
Hint: system [Processing]
SIGSEGV: Illegal storage access. (Attempt to read from nil?) |
@kristianmandrup you should use "./koch temp js path/to/try.nim" to (re)compile the compiler in debug mode and it will show full stack trace for crashes |
I can see where you're coming from but
These variants make no sense for JS as JS has no "type sections".
Well we keep supporting it but Nim is open source, stuff that people work on/with is what gets improved. Your PR is really welcome! |
Hi @Araq. Thanks for the quick reply. You are right, but most "real life" JS imports modules so at least the include section should be open for emit. Also, why not allows TypeScript as the target and allow us to emit interfaces. Just an idea. On that note, it would be awesome if there was an option to set the extension of the file being output. Modern node (with ES module support) requires files be named |
Any suggestions for how to best allow for this in Nim (how to pick up and pass on config/options) greatly appreciated and I will work on PR |
I would think I could leverage this part then? It calls proc myProcess(b: PPassContext, n: PNode): PNode =
result = n
let m = BModule(b)
if passes.skipCodegen(m.config, n): return n
if m.module == nil: internalError(m.config, n.info, "myProcess")
let globals = PGlobals(m.graph.backend)
var p = newProc(globals, m, nil, m.module.options)
p.unique = globals.unique
genModule(p, n) |
From what I can see, the best way would likely be to leverage ConfigRef* = ref object ## every global configuration
## fields marked with '*' are subject to
## the incremental compilation mechanisms
## (+) means "part of the dependency"
target*: Target # (+)
linesCompiled*: int # all lines that have been compiled
options*: TOptions # (+)
globalOptions*: TGlobalOptions # (+)
macrosToExpand*: StringTableRef
m*: MsgConfig
evalTemplateCounter*: int
evalMacroCounter*: int
exitcode*: int8
cmd*: TCommands # the command
selectedGC*: TGCMode # the selected GC (+)
exc*: ExceptionSystem
verbosity*: int # how verbose the compiler is
numberOfProcessors*: int # number of processors
evalExpr*: string # expression for idetools --eval
lastCmdTime*: float # when caas is enabled, we measure each command
symbolFiles*: SymbolFilesOption
cppDefines*: HashSet[string] # (*)
headerFile*: string
features*: set[Feature]
legacyFeatures*: set[LegacyFeature]
arguments*: string ## the arguments to be passed to the program that
## should be run
ideCmd*: IdeCmd
oldNewlines*: bool
cCompiler*: TSystemCC
enableNotes*: TNoteKinds
disableNotes*: TNoteKinds
foreignPackageNotes*: TNoteKinds
notes*: TNoteKinds
mainPackageNotes*: TNoteKinds
mainPackageId*: int
errorCounter*: int
hintCounter*: int
warnCounter*: int
errorMax*: int
maxLoopIterationsVM*: int ## VM: max iterations of all loops
configVars*: StringTableRef
symbols*: StringTableRef ## We need to use a StringTableRef here as defined
## symbols are always guaranteed to be style
## insensitive. Otherwise hell would break lose.
packageCache*: StringTableRef
nimblePaths*: seq[AbsoluteDir]
searchPaths*: seq[AbsoluteDir]
lazyPaths*: seq[AbsoluteDir]
outFile*: RelativeFile
outDir*: AbsoluteDir
prefixDir*, libpath*, nimcacheDir*: AbsoluteDir
dllOverrides, moduleOverrides*, cfileSpecificOptions*: StringTableRef
projectName*: string # holds a name like 'nim'
projectPath*: AbsoluteDir # holds a path like /home/alice/projects/nim/compiler/
projectFull*: AbsoluteFile # projectPath/projectName
projectIsStdin*: bool # whether we're compiling from stdin
projectMainIdx*: FileIndex # the canonical path id of the main module
command*: string # the main command (e.g. cc, check, scan, etc)
commandArgs*: seq[string] # any arguments after the main command
commandLine*: string
extraCmds*: seq[string] # for writeJsonBuildInstructions
keepComments*: bool # whether the parser needs to keep comments
implicitImports*: seq[string] # modules that are to be implicitly imported
implicitIncludes*: seq[string] # modules that are to be implicitly included
docSeeSrcUrl*: string # if empty, no seeSrc will be generated. \
# The string uses the formatting variables `path` and `line`.
docRoot*: string ## see nim --fullhelp for --docRoot
# the used compiler
cIncludes*: seq[AbsoluteDir] # directories to search for included files
cLibs*: seq[AbsoluteDir] # directories to search for lib files
cLinkedLibs*: seq[string] # libraries to link
externalToLink*: seq[string] # files to link in addition to the file
# we compiled (*)
linkOptionsCmd*: string
compileOptionsCmd*: seq[string]
linkOptions*: string # (*)
compileOptions*: string # (*)
cCompilerPath*: string
toCompile*: CfileList # (*)
suggestionResultHook*: proc (result: Suggest) {.closure.}
suggestVersion*: int
suggestMaxResults*: int
lastLineInfo*: TLineInfo
writelnHook*: proc (output: string) {.
closure.} # cannot make this gcsafe yet because of Nimble
structuredErrorHook*: proc (config: ConfigRef; info: TLineInfo; msg: string;
severity: Severity) {.closure, gcsafe.}
cppCustomNamespace*: string |
My issue now is that {.emit: "/*INCLUDESECTION*/import { x } from './abc'" .}
var x = 2
echo x Strange, why not? proc genPragma(p: BProc, n: PNode) =
for it in n.sons:
case whichPragma(it)
of wEmit: genEmit(p, it) |
My best attempt so far:
TTarget* = enum
targetC = "C"
targetCpp = "C++"
targetObjC = "ObjC"
targetJS = "JS"
targetMJS = "MJS"
targetTS = "TS"
const
targetToExt*: array[TTarget, string] = ["nim.c", "nim.cpp", "nim.m", "js", "mjs", "ts"]
targetToCmd*: array[TTarget, string] = ["c", "cpp", "objc", "js", "mjs", "ts"]
proc generatedFile(test: TTest, target: TTarget): string =
if target == targetJS:
result = test.name.changeFileExt("js")
elif target == targetMJS:
result = test.name.changeFileExt("mjs")
elif target == targetTS:
result = test.name.changeFileExt("ts")
else:
proc isSectionAvailableInTarget(sectionId: string, targetSections: seq[auto] = @[]): bool =
sectionId in targetSections
proc useSection(secStr: string, sectionMarker: string, targetSections: seq[auto] = @[]): bool =
var isAvailable = isSectionAvailableInTarget(secStr, targetSections)
secStr.startsWith("/*" & sectionMarker & "SECTION*/") and isAvailable
proc determineSection(n: PNode, targetSections: seq[auto] = @[]): TCFileSection =
result = cfsProcHeaders
if n.len >= 1 and n[0].kind in {nkStrLit..nkTripleStrLit}:
let sec = n[0].strVal
if useSection(sec, "TYPE", targetSections): result = cfsTypes
elif useSection(sec, "VAR", targetSections): result = cfsVars
elif useSection(sec, "INCLUDE", targetSections): result = cfsHeaders
proc genEmit(p: BProc, t: PNode) =
var s = genAsmOrEmitStmt(p, t[1])
echo "genEmit"
echo p.config.options
let config = p.config
echo "include sections and Include"
p.options.incl optSections
p.options.incl optIncludeSection
echo p.options
let opts = p.options
echo opts
var hasSections, hasVarSection, hasIncludeSection, hasTypeSection = false
if opts.len > 0:
echo "has options"
hasSections = optSections in opts
hasVarSection = optVarSection in opts
hasIncludeSection = optIncludeSection in opts
hasTypeSection = optTypeSection in opts
var targetSections: seq[string] = @[]
if hasIncludeSection:
targetSections.add "INCLUDE"
if hasTypeSection:
targetSections.add "TYPE"
if hasVarSection:
targetSections.add "VAR"
echo "hasSections: " & $hasSections
var hasNoPrc = p.prc == nil
echo "hasNoPrc:" & $hasNoPrc
var useSections = hasSections or hasNoPrc
echo "useSections:" & $useSections
echo targetSections
if useSections:
echo "using sections for emit"
# top level emit pragma?
let section = determineSection(t[1], targetSections)
genCLineDir(p.module.s[section], t.info, p.config)
p.module.s[section].add(s)
else:
echo "NOT using sections for emit"
genLineDir(p, t)
line(p, cpsStmts, s) |
I now understand that for js generation I see that |
yes jsgen is used , the other one-s are for other backends, also you should know nim generates a single javascript file .. so if you need many, this also requires more stuff this is one plan to refactor it .. i did some part of this refactor in a branch somewhere, but its huge, so one can do it more easily im ohttps://github.com//pull/7508 |
however i'd hesitate to target typescript.. nim is like an equivalent of typescript , also waiting for both compilers doesnt make a lot of sense to me : do you just need to use nim in typescript? it might be much easier to write a separate tool to generate typescript defs from nim ones |
Thanks, I will have a look. I see that the TJSGen = object of PPassContext
module: PSym
graph: ModuleGraph
config: ConfigRef
sigConflicts: CountTable[SigHash]
BModule = ref TJSGen
TSym* {.acyclic.} = object of TIdObj
# proc and type instantiations are cached in the generic symbol
case kind*: TSymKind
of skType, skGenericParam:
typeInstCache*: seq[PType]
of routineKinds:
procInstCache*: seq[PInstantiation]
gcUnsafetyReason*: PSym # for better error messages wrt gcsafe
transformedBody*: PNode # cached body after transf pass
of skModule, skPackage:
# modules keep track of the generic symbols they use from other modules.
# this is because in incremental compilation, when a module is about to
# be replaced with a newer version, we must decrement the usage count
# of all previously used generics.
# For 'import as' we copy the module symbol but shallowCopy the 'tab'
# and set the 'usedGenerics' to ... XXX gah! Better set module.name
# instead? But this doesn't work either. --> We need an skModuleAlias?
# No need, just leave it as skModule but set the owner accordingly and
# check for the owner when touching 'usedGenerics'.
usedGenerics*: seq[PInstantiation]
tab*: TStrTable # interface table for modules
of skLet, skVar, skField, skForVar:
guard*: PSym
bitsize*: int
alignment*: int # for alignment
else: nil
magic*: TMagic
typ*: PType
name*: PIdent
info*: TLineInfo
owner*: PSym
flags*: TSymFlags
ast*: PNode # syntax tree of proc, iterator, etc.:
# the whole proc including header; this is used
# for easy generation of proper error messages
# for variant record fields the discriminant
# expression
# for modules, it's a placeholder for compiler
# generated code that will be appended to the
# module after the sem pass (see appendToModule)
options*: TOptions
position*: int # used for many different things:
# for enum fields its position;
# for fields its offset
# for parameters its position
# for a conditional:
# 1 iff the symbol is defined, else 0
# (or not in symbol table)
# for modules, an unique index corresponding
# to the module's fileIdx
# for variables a slot index for the evaluator
offset*: int # offset of record field
loc*: TLoc
annex*: PLib # additional fields (seldom used, so we use a
# reference to another object to save space)
when hasFFI:
cname*: string # resolved C declaration name in importc decl, eg:
# proc fun() {.importc: "$1aux".} => cname = funaux
constraint*: PNode # additional constraints like 'lit|result'; also
# misused for the codegenDecl pragma in the hope
# it won't cause problems
# for skModule the string literal to output for
# deprecated modules.
when defined(nimsuggest):
allUsages*: seq[TLineInfo] The key for the sections to work for the let section = determineSection(t[1], targetSections)
genCLineDir(p.module.s[section], t.info, p.config)
p.module.s[section].add(s) But in the js context, there is no such I never thought of TypeScript as a "separate Nim target". Would be nice to have the option to emit "TypeScript like" code and then optionally set I'll have a look in your PR |
my PR is totally unfinished but mostly the conversation is useful but why would you need to produce typescript? nim doesnt generate pretty code by design, so it wouldnt be useful unless you import it in typescript |
TypeScript is just a superset of Javascript (constantly evolving). TypeScript is often at the front with support for new experimental features. Sometimes it is just convenient to be able to output a specific extension (such as Sounds great :) |
Do you know where I can find the
|
i havent tho, this was planned to be done : it shouldn't be too hard but its not existing here |
Found this: TCProc = object # represents C proc that is currently generated
prc*: PSym # the Nim proc that this C proc belongs to
flags*: set[TCProcFlag]
lastLineInfo*: TLineInfo # to avoid generating excessive 'nimln' statements
currLineInfo*: TLineInfo # AST codegen will make this superfluous
nestedTryStmts*: seq[tuple[fin: PNode, inExcept: bool, label: Natural]]
# in how many nested try statements we are
# (the vars must be volatile then)
# bool is true when are in the except part of a try block
finallySafePoints*: seq[Rope] # For correctly cleaning up exceptions when
# using return in finally statements
labels*: Natural # for generating unique labels in the C proc
blocks*: seq[TBlock] # nested blocks
breakIdx*: int # the block that will be exited
# with a regular break
options*: TOptions # options that should be used for code
# generation; this is the same as prc.options
# unless prc == nil
module*: BModule # used to prevent excessive parameter passing
withinLoop*: int # > 0 if we are within a loop
splitDecls*: int # > 0 if we are in some context for C++ that
# requires 'T x = T()' to become 'T x; x = T()'
# (yes, C++ is weird like that)
sigConflicts*: CountTable[string] Similar to BModule* = ref TCGen
TCGen = object of PPassContext # represents a C source file
s*: TCFileSections # sections of the C file. <--- HERE WE GO!!!
flags*: set[CodegenFlag]
module*: PSym
filename*: AbsoluteFile
cfilename*: AbsoluteFile # filename of the module (including path,
# without extension)
tmpBase*: Rope # base for temp identifier generation
typeCache*: TypeCache # cache the generated types
forwTypeCache*: TypeCache # cache for forward declarations of types
declaredThings*: IntSet # things we have declared in this .c file
declaredProtos*: IntSet # prototypes we have declared in this .c file
headerFiles*: seq[string] # needed headers to include
typeInfoMarker*: TypeCache # needed for generating type information
initProc*: BProc # code for init procedure
preInitProc*: BProc # code executed before the init proc
hcrCreateTypeInfosProc*: Rope # type info globals are in here when HCR=on
inHcrInitGuard*: bool # We are currently within a HCR reloading guard.
typeStack*: TTypeSeq # used for type generation
dataCache*: TNodeTable
typeNodes*, nimTypes*: int # used for type info generation
typeNodesName*, nimTypesName*: Rope # used for type info generation
labels*: Natural # for generating unique module-scope names
extensionLoaders*: array['0'..'9', Rope] # special procs for the
# OpenGL wrapper
injectStmt*: Rope
sigConflicts*: CountTable[SigHash]
g*: BModuleList
ndi*: NdiFile So essentially we could in part mirror the TCGen = object of PPassContext # represents a C source file
s*: TCFileSections # sections of the C file. <--- HERE WE GO!!! |
better interop is not bad, but you mostly need to generate compatible types to interop: not really code, if you target a language X backend, its often easier to just target a much older version if possible , because you can already create features in the source language(and for interop, you need only signatures/types to match!) imho you can enter #nim on irc as well but if its easier for you here, thats also amazing! |
OK, which irc client would you recommend for OSX? Limechat? I think the following refactoring, adding TCGen = object of PPassContext # represents a C source file
outputFiles: # HashMap of TCGen (extra files being generated as side effect of compiling any given nim file)
module*: PSym
s*: TCFileSections # sections of the C file. <--- HERE WE GO!!!
filename*: AbsoluteFile
cfilename*: AbsoluteFile # filename of the main output module (including path, without extension) For JS TJSGen = object of PPassContext # represents a C source file
outputFiles: # HashMap of TCGen (extra files being generated as side effect of compiling any given nim file)
module*: PSym
s*: TJSFileSections # sections of the JS file.
filename*: AbsoluteFile
jsfilename*: AbsoluteFile # filename of the main output module (including path, without extension) Ideally |
you can also just use discord or gitter |
please, dont change the c backend if not sure, i really doubt this would be required : there are probably many assumptions for the way files are generated there already |
I've been a bit on the main Nim gitter channel lately. OK, I won't touch the c backend |
My current proposal: TJSGen = object of PPassContext
module: PSym
s*: TJSFileSections # sections of the JS file
graph: ModuleGraph
config: ConfigRef
sigConflicts: CountTable[SigHash]
TJSFileSection* = enum # the sections a generated C file consists of
jsfsHeaders, # top file section typically used for JS require/import statements
jsfsTypes, # type section (such as interfaces/types for TypeScript interop)
jsfsMain, # main code
jsfsFooters # bottom section often used for exports
TJSFileSections* = array[TJSFileSection, Rope] # represents a generated JS file
BModule = ref TJSGen |
Then copying the |
I've come very far now. Almost there :) |
this seems like a perfect use case for #13953 |
Summary
Currently there seems to be no way to direct emits to certain sections of the output when the target is
js
.Another issue is that the pragma import form of:
Does not seem to be currently possible, as it is for
proc
.Description
Adds support for generating include statements for various compiler targets, including both C and Ecmascript/Javascript family
According to the docs, it should support the following special variants:
TYPESECTION
VARSECTION
INCLUDESECTION
I've tried the following
Compiling with
nim js -d:nodejs var.nim
Getting the output:
Additional Information
There are no unit tests for either
determineSection
orgenEmit
I believe, that
if p.prc == nil
in my example, so that it never consults if there is a section identifier.Not sure why it shouldn't always ask this question before emitting?
prc
seems to be linked to the Garbage Collector somehow? Looks like this is a very brittle approach for the emit case. Instead it should likely determine if the target supports sections.links
emit("here"): "c code"
+ other emit fixes; new module experimental/backendutils.nim #13953The text was updated successfully, but these errors were encountered: