Various enhancements

[PatrickTo]

1. Fixed the wrong cursor position and treat duo UTF-16 characters as a single character for U+10000 or above.
   e.g.
   Extended Chinese characters and emoticons
   𠎀
   😁👍👌😄👏
2. Enhance the error handling in worker.js
3. Remove dead code in php.js

[nightwing]

Sorry for the late review. Unfortunately this approach adds too much complexity to Ace, since it means we can't use any of string methods, and deltas produced by Ace can't be used by other javascript code (e.g ShareJS). The better approach would be to modify selection code to not allow placing cursor in the middle of the surrogate pair.
See related discussion in #1153

[PatrickTo]

Yes, agree that it was not perfect solution.
However, if it is needed to completely solve the wrong cursor position problem, it is inevitably to solve the variable width font issues.

I am experiencing the fixing of variable font width problem with about 90% completeness.

The problem is indeed not only cursor positioning but also multiple UTF-16 characters for a single visible character (i.e. Invalid delete/move left/move right). It is not only U+10000 or above characters but also character with accent combination character such as the Thai character ก่ which consists of two UTF-16 characters.

[PatrickTo]

• Supports variable font width
• Handles combine character (e.g. ก่)
• Improve large document supports (Text mode) (Supports > 20,000,000 lines)
• Fix invisible vertical scrollbar for large documents (Firefox)
• Supports bidi
• Scrollbar highlight for annotations (Experimental)

[nightwing]

While this adds many improvements it adds many regression as well

• lines that folded and wrapped at the same time throw exceptions
• size calculation of linewidgets is broken because of changes to screenToDocumentPosition
• loops like for (var k in this.session.$foldData) { if (isFinite(k)) { have very bad performance
  to name a few.

btw seems like in 2f737da you have added files from master without merging so it shows all the changes from master branch mixed with your changes

[PatrickTo]

The principle of tuning is not to tune fast the code but to reduce the memory consumption on large document. And by the fact that Array in javascript is not actually an array.

Memory consumption:

var test = Array(100000000);
for (var i = 0; i < test.length; i++)
    test[i] = [0];
console.log(test.length);

vs

var test = Array(100000000);
for (var i = 0; i < test.length; i++)
    if (i%100==0)
        test[i] = [0];
console.log(test.length);

Retrieval

var test = Array(10000000);
for (var i = 0; i < test.length; i++)
    test[i] = [0];
var total=0;
var start = new Date();
for (var i = 0; i < test.length; i++)
    total+=test[i][0];
var end = new Date();
console.log(end.getTime()-start.getTime()); // 18ms

vs

var test = Array(10000000);
for (var i = 0; i < test.length; i++)
    if (i%1000==0)
        test[i] = [0];
var total=0;
var start = new Date();
for (var k in test)
    total+=test[k][0];
var end = new Date();
console.log(end.getTime()-start.getTime()); // 6ms

In normal usage, fold lines should be much less than the document length.
Use memory for wrap/fold only when needed.

Surely, there are rooms for further improvement such as adding another variable for keys, but it adds another memory overhead. It is to be by testing combined with actual usage scenario to find the optimal solution for the balance between memory usage and speed.

Large memory allocation = reducing speed

[nightwing]

Updating array with holes when new line is inserted is significantly slower than a normal array,
but anyway foldData isn't a sparse array of the form [ lineNumber: fold ], it is a dense array of all foldLInes

It is to be by testing combined with actual usage scenario to find the optimal solution for the balance between memory usage and speed.

yes, and documents with >20 million lines are not the intended usage scenario. Supporting them is of course nice, but that shouldn't break small documents.