Use the C++ roxygen parser

[kevinushey]

Wire it up where appropriate.

[hadley]

What needs to be done here?

[kevinushey]

The C++ parser introduced two main entry points, roxygen2:::preparse_file(<file>) and roxygen2:::preparse_block(<block>). The goal was for this to be a (near) drop-in replacement for parse.preref, so that those functions could potentially be used in that situation.

E.g.

lines <- "#' @param foo A parameter.\n#' @param bar Another parameter.\n"
roxygen2:::parse.preref(lines)
roxygen2:::preparse_block(lines)

Comparing output,

> lines <- "#' @param foo A parameter.\n#' @param bar Another parameter.\n"
> roxygen2:::parse.preref(lines)
$param
$param$name
[1] "foo"

$param$description
[1] "A parameter.\n#' "


$param
$param$name
[1] "bar"

$param$description
[1] "Another parameter."


> roxygen2:::preparse_block(lines)
$param
[1] "foo A parameter."
attr(,"file")
[1] ""
attr(,"row")
[1] 1

$param
[1] "bar Another parameter."
attr(,"file")
[1] ""
attr(,"row")
[1] 2

[[3]]
[1] ""
attr(,"file")
[1] ""
attr(,"row")
[1] 2

attr(,"delimiter")
[1] "#' "

I think early on you indicated that you wanted the preparse_block to do as little as possible, hence the contents of the @param tag aren't actually split up. That said, I'm open to amending the implementation of the preparse functions to generate output that's useful.

[hadley]

It seems to be a bit too accepting currently:

preparse_block("\n  # This file was generated by Rcpp::compileAttributes\n  # Generator token: 10BE3573-1514-4C36-9D1C-5A225CD40393\n")
#> $introduction
#> [1] "# This file was generated by Rcpp::compileAttributes\n# Generator token: 10BE3573-1514-4C36-9D1C-5A225CD40393\n"
#> attr(,"file")
#> [1] ""
#> attr(,"row")
#> [1] 1
#> 
#> attr(,"delimiter")
#> [1] "\n  "

This should return NULL because there's no roxygen there

[hadley]

Revised parse.file will need to look something like this:

parse_file <- function(file, env) {
  parsed <- parse(file = file, keep.source = TRUE)
  if (length(parsed) == 0) return()

  refs <- getSrcref(parsed)
  comment_refs <- comments(refs)

  extract <- function(call, ref, comment_ref) {
    comment_ref2 <- list(filename = file, lloc = as.vector(comment_ref))

    block <- preparse_block(paste(as.character(comment_ref), collapse = ""))
    if (is.null(block)) return()

    preref <- parse_elements(block)
    preref$object <- object_from_call(call, env, preref)
    preref$srcref <- list(filename = file, lloc = as.vector(ref))
    add_defaults(preref)
  }

  Map(extract, parsed, refs, comment_refs)
}

Would be nice if preparse_block() could take a srcref directly.