improve number lexer

[guwirth]

• reduce the number of regex rules to detect a number
• use only one channel to detect numbers
• improve Performance: Slow analysis of arrays #1415

---

This change is 

[ivangalkin]

Reviewed 2 of 2 files at r1.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @guwirth, @jmecosta, and @SonarOpenCommunityAdmin)

---

cxx-squid/src/main/java/org/sonar/cxx/lexer/CxxLexer.java, line 45 at r1 (raw file):

  private static final String HEX_PREFIX = "0[xX]";
  private static final String EXPONENT = "([Ee][+-]?+[0-9_]([']?+[0-9_]++)*+)";

Your change is valid. However the original regexp is not really clear to me. Does it make sense to have * followed by +? Wouldn’t + be sufficient?

[ivangalkin]

Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @guwirth, @jmecosta, and @SonarOpenCommunityAdmin)

---

cxx-squid/src/main/java/org/sonar/cxx/lexer/CxxLexer.java, line 58 at r1 (raw file):

  private static final String BINDIGIT_SEQUENCE = "[01]([']?+[01]++)*+";
  private static final String POINT = "\\.";
  

Trailing whitespaces.

[ivangalkin]

@guwirth Your change looks good to me (although I am not sure, that I understand it in detail). I am not available for reviews until the the 2nd week of January.

I wish you happy new year! My best wishes to all contributors too!

[guwirth]

@ivangalkin

Your change is valid. However the original regexp is not really clear to me. Does it make sense to have * followed by +? Wouldn’t + be sufficient?

Good explanation is here: https://docs.oracle.com/javase/tutorial/essential/regex/quant.html.
See also in the source code https://github.com/SonarSource/sslr/blob/master/sslr-core/src/main/java/com/sonar/sslr/impl/channel/RegexpChannelBuilder.java.

  public static String opt(String regexpPiece) {
    return regexpPiece + "?+";
  }

  public static String one2n(String regexpPiece) {
    return regexpPiece + "++";
  }

 public static String o2n(String regexpPiece) {
    return regexpPiece + "*+";
}