Allow $$ $+ $* $/ $$= $+= ..= as operators in the parser

[ScottPJones]

This simply defines them so that people can easily play around with different syntax for string concatenation and general abstract vector concatenation, in the hope of coming to some better consensus. I believe this should have absolutely no affect on any current Julia programs.

[vtjnash]

these operators are already taken as the interpolation symbol

[ScottPJones]

@vtnash, that's not correct. Outside of a string literal, $ is used for bitwise XOR, and has nothing at all to do with string interpolation. None of the combinations I added were even being parsed in Julia as operators... that is what this change accomplishes, so that somebody could use any of them, if they so wished, in a package.

[vtjnash]

they parse for me as existing syntax:

julia> quote $+, $$, $*, $/ end
quote  # none, line 1:
    (+,$,*,/)
end

although true that they aren't valid as infix operators, there is a duality of all operators in julia such that a op b === op(a,b), and $+(a,b) does already have a meaning

I'm also pretty opposed to adding new operators to the language (ala http://en.wikipedia.org/wiki/APL_%28programming_language%29)

[ScottPJones]

@vtjnash That's not what I got... isn't the quote similar to having them in a """, where you'd expect interpolation?

julia> $+
ERROR: unsupported or misplaced expression $

julia> $$
ERROR: unsupported or misplaced expression $

I also, as you can see, was able to edit the Scheme parser and make these all work as infix operators
quite easily.

[vtjnash]

no, please read http://docs.julialang.org/en/latest/manual/metaprogramming/

[ScottPJones]

@vtjnash Yes, I read that... but it's not quite clear that that precludes using $+, $*, or $/ as tokens.
$ is also the bitwise XOR operator, and that doesn't cause any problems with interpolation, as far as I'm aware.

[ScottPJones]

@vtjnash That's a coincidence! I actually just read that yesterday! I had to do some APL programming as part of a course at MIT back 34 years ago, and after seeing the HUGE number of operators defined by the parse in Julia, I went back and Google'd APL...
Given the following, you really object to adding a few more, so that people who do string / database processing for a living can be happier?

(define prec-assignment
  '(= := += -= *= /= //= .//= .*= ./= |\\=| |.\\=| ^= .^= ÷= .÷= %= .%= |\|=| &= $= => <<= >>= >>>= ~ |.+=| |.-=|))
(define prec-conditional '(?))
(define prec-lazy-or     '(|\|\||))
(define prec-lazy-and    '(&&))
(define prec-arrow       '(-- --> ← → ↔ ↚ ↛ ↠ ↣ ↦ ↮ ⇎ ⇏ ⇒ ⇔ ⇴ ⇶ ⇷ ⇸ ⇹ ⇺ ⇻ ⇼ ⇽ ⇾ ⇿ ⟵ ⟶ ⟷ ⟷ ⟹ ⟺ ⟻ ⟼ ⟽ ⟾ ⟿ ⤀ ⤁ ⤂ ⤃ ⤄ ⤅ ⤆ ⤇ ⤌ ⤍ ⤎ ⤏ ⤐ ⤑ ⤔ ⤕ ⤖ ⤗ ⤘ ⤝ ⤞ ⤟ ⤠ ⥄ ⥅ ⥆ ⥇ ⥈ ⥊ ⥋ ⥎ ⥐ ⥒ ⥓ ⥖ ⥗ ⥚ ⥛ ⥞ ⥟ ⥢ ⥤ ⥦ ⥧ ⥨ ⥩ ⥪ ⥫ ⥬ ⥭ ⥰ ⧴ ⬱ ⬰ ⬲ ⬳ ⬴ ⬵ ⬶ ⬷ ⬸ ⬹ ⬺ ⬻ ⬼ ⬽ ⬾ ⬿ ⭀ ⭁ ⭂ ⭃ ⭄ ⭇ ⭈ ⭉ ⭊ ⭋ ⭌ ￩ ￫))
(define prec-comparison
  '(> < >= ≥ <= ≤ == === ≡ != ≠ !== ≢ |.>| |.<| |.>=| |.≥| |.<=| |.≤| |.==| |.!=| |.≠| |.=| |.!| |<:| |>:| ∈ ∉ ∋ ∌ ⊆ ⊈ ⊂ ⊄ ⊊ ∝ ∊ ∍ ∥ ∦ ∷ ∺ ∻ ∽ ∾ ≁ ≃ ≄ ≅ ≆ ≇ ≈ ≉ ≊ ≋ ≌ ≍ ≎ ≐ ≑ ≒ ≓ ≔ ≕ ≖ ≗ ≘ ≙ ≚ ≛ ≜ ≝ ≞ ≟ ≣ ≦ ≧ ≨ ≩ ≪ ≫ ≬ ≭ ≮ ≯ ≰ ≱ ≲ ≳ ≴ ≵ ≶ ≷ ≸ ≹ ≺ ≻ ≼ ≽ ≾ ≿ ⊀ ⊁ ⊃ ⊅ ⊇ ⊉ ⊋ ⊏ ⊐ ⊑ ⊒ ⊜ ⊩ ⊬ ⊮ ⊰ ⊱ ⊲ ⊳ ⊴ ⊵ ⊶ ⊷ ⋍ ⋐ ⋑ ⋕ ⋖ ⋗ ⋘ ⋙ ⋚ ⋛ ⋜ ⋝ ⋞ ⋟ ⋠ ⋡ ⋢ ⋣ ⋤ ⋥ ⋦ ⋧ ⋨ ⋩ ⋪ ⋫ ⋬ ⋭ ⋲ ⋳ ⋴ ⋵ ⋶ ⋷ ⋸ ⋹ ⋺ ⋻ ⋼ ⋽ ⋾ ⋿ ⟈ ⟉ ⟒ ⦷ ⧀ ⧁ ⧡ ⧣ ⧤ ⧥ ⩦ ⩧ ⩪ ⩫ ⩬ ⩭ ⩮ ⩯ ⩰ ⩱ ⩲ ⩳ ⩴ ⩵ ⩶ ⩷ ⩸ ⩹ ⩺ ⩻ ⩼ ⩽ ⩾ ⩿ ⪀ ⪁ ⪂ ⪃ ⪄ ⪅ ⪆ ⪇ ⪈ ⪉ ⪊ ⪋ ⪌ ⪍ ⪎ ⪏ ⪐ ⪑ ⪒ ⪓ ⪔ ⪕ ⪖ ⪗ ⪘ ⪙ ⪚ ⪛ ⪜ ⪝ ⪞ ⪟ ⪠ ⪡ ⪢ ⪣ ⪤ ⪥ ⪦ ⪧ ⪨ ⪩ ⪪ ⪫ ⪬ ⪭ ⪮ ⪯ ⪰ ⪱ ⪲ ⪳ ⪴ ⪵ ⪶ ⪷ ⪸ ⪹ ⪺ ⪻ ⪼ ⪽ ⪾ ⪿ ⫀ ⫁ ⫂ ⫃ ⫄ ⫅ ⫆ ⫇ ⫈ ⫉ ⫊ ⫋ ⫌ ⫍ ⫎ ⫏ ⫐ ⫑ ⫒ ⫓ ⫔ ⫕ ⫖ ⫗ ⫘ ⫙ ⫷ ⫸ ⫹ ⫺ ⊢ ⊣))
(define prec-pipe        '(|\|>| |<\||))
(define prec-colon       '(: |..|))
(define prec-plus        '(+ - ⊕ ⊖ ⊞ ⊟ |.+| |.-| |\|| ∪ ∨ $ ⊔ ± ∓ ∔ ∸ ≂ ≏ ⊎ ⊻ ⊽ ⋎ ⋓ ⧺ ⧻ ⨈ ⨢ ⨣ ⨤ ⨥ ⨦ ⨧ ⨨ ⨩ ⨪ ⨫ ⨬ ⨭ ⨮ ⨹ ⨺ ⩁ ⩂ ⩅ ⩊ ⩌ ⩏ ⩐ ⩒ ⩔ ⩖ ⩗ ⩛ ⩝ ⩡ ⩢ ⩣))
(define prec-bitshift    '(<< >> >>> |.<<| |.>>| |.>>>|))
(define prec-times       '(* / |./| ÷ % ⋅ ∘ × |.%| |.*| |\\| |.\\| & ∩ ∧ ⊗ ⊘ ⊙ ⊚ ⊛ ⊠ ⊡ ⊓ ∗ ∙ ∤ ⅋ ≀ ⊼ ⋄ ⋆ ⋇ ⋉ ⋊ ⋋ ⋌ ⋏ ⋒ ⟑ ⦸ ⦼ ⦾ ⦿ ⧶ ⧷ ⨇ ⨰ ⨱ ⨲ ⨳ ⨴ ⨵ ⨶ ⨷ ⨸ ⨻ ⨼ ⨽ ⩀ ⩃ ⩄ ⩋ ⩍ ⩎ ⩑ ⩓ ⩕ ⩘ ⩚ ⩜ ⩞ ⩟ ⩠ ⫛ ⊍))
(define prec-rational    '(// .//))
(define prec-power       '(^ |.^| ↑ ↓ ⇵ ⟰ ⟱ ⤈ ⤉ ⤊ ⤋ ⤒ ⤓ ⥉ ⥌ ⥍ ⥏ ⥑ ⥔ ⥕ ⥘ ⥙ ⥜ ⥝ ⥠ ⥡ ⥣ ⥥ ⥮ ⥯ ￪ ￬))
(define prec-decl        '(|::|))
(define prec-dot         '(|.|))

I think the proverbial horse left the barn many ages ago in Julia!

[ScottPJones]

@vtjnash I really don't think your objection holds, about $+ already have a meaning. Maybe it does, in a certain context, but there is quite a lot of that in Julia, things like ; mean one thing in one place, and mean something totally different within [ ].

julia> methods($+)
ERROR: unsupported or misplaced expression $

[vtjnash]

perhaps, but $+ already has a meaning in this context: it means to interpolate the variable + into the current expression. in the example above, there is no expression, so the $() is misplaced and you get an error. however, this is a runtime error, not a parser error.

Given the following, you really object to adding a few more

yes, i object to adding more unique ascii character sequences to the list. i'm largely ambivalent on the unicode operators: they are harder to google directly, but if you already know what they are (or can lookup their unicode name), they can make an algorithm much more readable.

[JeffBezanson]

An awful lot of people work with strings and databases, and there doesn't seem to be a lot of demand from them for $/.

Granted, all the unicode operators are kind of ridiculous, but consider (1) many of them are very standard math symbols like set operators, (2) there is nothing else they could possibly mean. I doubt I'm ever going to use "⋨", but why not parse it as an operator? Allowing that as an operator in no way implies allowing arbitrarily more ASCII sequences as operators.

You have to allocate your ASCII very carefully. To gain some idea of the enormity of this, take a look at https://www.python.org/dev/peps/pep-0465/, which is what numeric python users went through to get a single operator added to the language, after years.

In nested quote expressions, it makes sense to apply multiple $s to an expression, so $$x has a meaning inside quote as well. If that varied contextually it would add confusion to what is already one of the most confusing things you can write.

Since we parse .. as an operator it seems to make sense to add ..=.

[ScottPJones]

@JeffBezanson There are all sorts of databases... People working with SQL databases, or modern document databases, really wouldn't be using that sort of a slice operator (you were talking about just $/ there, right? To take a string and a character, or a string and a string, and return an array or tuple of substrings... The thing is, those aren't the only sorts of databases...
For example, b"Scott\xfcPaul\xfc\Jones\xfdOne\xfc\Memorial Drive\xfcCambridge"
is a typical record in a Pick (also known as MultiValue) database... This is also true for M[umps] databases... except there it more frequently be delimited with something like \0, \1, \2...
There is a LOT of data stored in those "legacy" formats [delimited strings stored in KVS systems] around the world.

I also understand totally about the issues of language design... did that for many years... and allocating ASCII characters very carefully... but I do wish that that care had actually been taken with Julia...
* for string concatenation doesn't pass the test of 1) being a very standard symbol for concatenation (don't know of any other language that uses it for that) or 2) there is nothing else that it could possibly mean...
Using ~ doesn't make any sense in DataFrames (and I know in some other place), if you are really trying to be careful about allocating the ASCII characters. I also would question the use of the back tick
` character for commands... a nice trick, but, wouldn't something like c"command string" have worked just as well, without using up the very small list of available characters?

I wasn't aware of using $ in quote operations, and I searched all the documentation I could find and all of the source code, and didn't find a single occurrence of $$ $+ $/ or $* anywhere...
I know $ is used in some other languages for interpolation... I guess I much prefer syntax like Swift's,
i.e. \(expression), since the bounds of the interpolation is always clear, and \ is already an escape character, and not that common, unlike $.

So, given the problems that I hadn't been aware about with using $, shall I remove those additions, add just leave in the addition of ..= as an assignment operator?

Thanks very much for the help!

[toivoh]

Maybe it's best to consider this pull request a WIP that people can try out and experiment with if they like for now. If it's not going to be merged for a while, I think it would be premature to take out those operators.

It's true that those character sequences already have a use, but I think eg $($(x)), $(+) etc is much better style than $$x and $+ anyway, making it at least somewhat more clear that interpolation is being used and what is interpolated.

If we would settle for one of these for string concatenation, I don't think many would fall into the trap of writing $$x, since you recognize the concatenation operator. (Otoh, we shouldn't add an operator of this form that wouldn't be commonly used.)

I'm quite sure that the lexer has no trouble to distinguish $$ in a context independent way. And let's be honest, there's a much wider user for strong concatenation than double interpolation or interpolation of the local + operator.

[JeffBezanson]

but I do wish that that care had actually been taken with Julia...

Ok, so you think we have used our ASCII poorly. I'm sorry to hear that. But how is the situation improved by adding a random spray of extra operators? I don't get it. The argument seems to be that the more mis-allocated syntax we already have, the more it should be ok to add?

Whether * is used for string concat is not a parser issue. I agree it doesn't pass the "nothing else it could possibly mean" test; e.g. see Stefan's comments about the generic meaning of *.

I think $* $/ and $+ are pretty obscure. I don't see why any of those, or $$, would be such a great string concatenation operator. I agree string concatenation is more important than double interpolation, but wouldn't many people find it weird if we used any of those? At least +, ++, and .. have some precedent.

[jiahao]

Using ~ doesn't make any sense in DataFrames (and I know in some other place), if you are really trying to be careful about allocating the ASCII characters.

@ScottPJones the ~ operator is used extensively in the R/statistics community to describe statistical models. Wilkinson-Rogers notation is a perfectly good reason to use ~ as an operator. Why does this not make any sense? Are you saying that giving the statistics community ~ is a bad language design decision?

[tkelman]

Are you saying that giving the statistics community ~ is a bad language design decision?

Devil's advocate, you didn't ask me, etc...

Given that only a subset of users of Julia will ever use that application-dependent meaning of ~, it is a little odd to reserve and preclude its use elsewhere in the language. Why couldn't the statistical uses of it have been restricted to a macro-based DSL where you'd be free to do almost whatever you want syntactically (as long as it parses)? For example inside JuMP the comparison operators ==, <=, etc have dramatically different meaning of specifying an optimization constraint, but the rest of the language doesn't have to be restricted in the ways it uses comparison operators as a consequence. There's also some funky uses of {} in JuMP for entirely different purposes which is probably more dangerous in terms of colliding with future language constructs, but for now the rest of the language doesn't have to be careful about allocating its ascii because of nonstandard things you may do in a macro DSL.

[ScottPJones]

@JeffBezanson "a random spray of extra operators"? I'm sorry, but as I tried to make very clear, unlike the random spray of 100s of operators that the parser already does handle... I intended for these to be a logically consistent, small set of operators... where $ indicates string (or string-like) behavior, and the second character indicates what type of operation, using what would be the most intuitive meaning (for most programmers, maybe not for numerical computing programmers).
I really don't think $+ or $* would really be that obscure, and would cause a lot less head shaking than the use of * for concatenation. I've seen a number of comments along the lines of "well, that surprised me when I started with Julia", so I don't think I'm off the mark there.
$ often indicates string operations in many languages... (even in Julia, it is used mostly for a string operation!), + usually means concatenation, * means making multiples of something (i.e. repetition), and / is dividing something up (so splitting up a string by a character or substring seems very natural).
What is so different in that, to the way . is used in Julia to indicate a matrix operation?
I do wish that it was made the only way... why do both * and .* work? Does overloading *
so much cause any performance penalties when you are dealing with type Any?
If so, that seems like a very good reason to want to move away from * and ^ from operating on strings.
As I've said a number of times, bare + has just as almost as many problems as *, and it couldn't be used as a general concatenation operator, which is one of the reasons for discussing these changes.
++ as it's own problems, of possibly confusing people a bit who are accustomed to seeing it just as a prefix or postfix unary operator. .. is fine for me, but doesn't have the advantage of indicating a family of string operators, as $ would, and also might be confused as being a syntax for ranges.

[ScottPJones]

@jiahao 1) Is the ~ operator used (in programming languages, books) outside of R (and DataFrames), for statistical models? 2) I am not saying that giving the statistics community an operator is a bad language design decision, I am saying that doing that in a way that precludes the use of that operator (used in many other languages for much more common operations) anywhere else in the language as a binary operator, is a VERY bad decision, esp. when with the great macro capabilities of Julia, it was totally unnecessary. Is putting a few more characters to indicate that it is a formula really that horrible? i.e. something like @f(y ~ 1 + x)

[ScottPJones]

@tkelman 👍 💯 Precisely the point I was trying to make all along (but much better explained!)

[tkelman]

why do both * and .* work?

Is the documentation not clear about this? If it isn't, we should improve the documentation. Matrix multiplication and elementwise multiplication are distinct operations, and pep 465 tells the story of what happens when you would like to do linear algebra without having distinct operators for the two.

[ScottPJones]

@tkelman would there be any chance of getting DataFrames to change to make the ~ formulas have to go in a macro? There have been much larger breaking changes to the language in just the few weeks I have been using the language... (Tupocalypse, for one!)

[tkelman]

would there be any chance of getting DataFrames to change to make the ~ formulas have to go in a macro?

I'm not the person to ask. There is some history of having to quote the :~ in the past that contributed to things, but I don't know the specifics.

[ScottPJones]

@tkelman I must have misunderstood some threads about trying to get people to use the .* etc. syntax... (not being a numerical computing guy, and my one linear algebra course being over 30 years in the past...)

[nalimilan]

If we decided that ~ should be the string concatenation operator, we could imagine adapting DataFrames to avoid conflicting with that. But as long as that's not the case, there seems to be little point in arguing against it if nobody would benefit from this change.

FWIW, ~ was introduced in #4882 It was previously discussed here: https://groups.google.com/d/msg/julia-dev/pB6KzaGjUNg/9RkdjCf6eGQJ and here: https://groups.google.com/d/msg/julia-dev/n3ntT4M0gwo/y8vHnFCcZjYJ

[ScottPJones]

@nalimilan What happens when you might need it for something else? My point is, it is not a good idea for packages to be written in ways that preclude changes to the base language, or preclude other packages from using syntax that would be natural for them... I think @tkelman's comments were right on, about how JuMP handles things without causing problems elsewhere (except for the use of {}, of course)

[ScottPJones]

@nalimilan I would have thought that the syntax: f"y ~ a + (1 | b)", as proposed in one of those threads, would have been the cleanest way to add it to the language, without disturbing anything else, or having the possibility of anything else disturbing DataFrames... much more explicit also, that you aren't using ~ for some other meaning.

[ScottPJones]

I agree wholeheartedly with @toivoh 's comments... much better said than I've been able to.
This PR, as stated clearly right at the top, was ONLY about allowing people to try out different things,
as currently the parser doesn't allow it... Is that such a bad thing to put in the Base?
This is not a PR to actually use one of those operators for string concatenation, or shooting off a rocket, or whatever... just having them parsed, and just having them parsed doesn't seem to conflict with anything else in the language.

[vtjnash]

just having them parsed, and just having them parsed doesn't seem to conflict with anything else in the language.

these operators are already taken by the language. parsing them differently is a breaking change.

I would have thought that the syntax: f"y ~ a + (1 | b)", as proposed in one of those threads, would have been the cleanest way to add it to the language, without disturbing anything else, or having the possibility of anything else disturbing DataFrames... much more explicit also, that you aren't using ~ for some other meaning.

The module system in Julia means that nothing else will disturb DataFrame's usage of ~, and that any other module can make use of ~ without being disturbed by DataFrames. It just comes down to the user's choice: import TheModule.@~

[ScottPJones]

@vtjnash

these operators are already taken by the language. parsing them differently is a breaking change.

That may be the case - but there are no unit tests or anything apparently for that sort of syntax,
and no occurrences I could find, searching all of the Julia code base... so it doesn't seem so breaking...

Also, I'd agree with @toivoh, that it really is not that big of a burden to make people use $(+) or $(*)
instead of $+ and $* in interpolation... I think the intent is a lot clearer also... (using $(variable) I also think now is a lot clearer... $ is too frequent a character in output strings... i.e. "The price of the shoes is $49.99", which is why I think \(variable) and \(expr) would have worked much better for string interpolation...

[ScottPJones]

@JeffBezanson If I removed all the other operators, and just added ..=, which you commented on already, could you merge this in?

Since we parse .. as an operator it seems to make sense to add ..=.

The reason I want at least some operator and corresponding assignment operator, is to be able to write a string package, utilizing all of my performance ideas, with both fast immutable and mutable strings,
that don't take up much space for small strings, with the immutable strings always validated, and yet which might be able to cache information about conversions for large strings [this is done in some other languages...], and then have people be able to try it out, with a natural syntax for string and vector manipulation, whether that be .. and ..=, or ++ and ++=, or whatever...

Does that make sense to people?

[ScottPJones]

@StefanKarpinski For concat!, I would want to just do a vcat! if you have two vectors of the same type.
If the first argument is a MutableString, then I'd want to convert as necessary... possibly changing the encoding of the MutableString (if internally, it is an ASCIIString, and you concatenate an emoji, then I'd widen it to a UTF32String before concatenation... [my idea for MutableString is that they are always DirectIndexStrings], if the MutableString is a UTF32String internally, and I concat! a ASCIIString, UTF8String orUTF16String` to it, I'd do the conversion, by first validating the strings to be added, increasing the size of the buffer if necessary, and then directly performing the conversion into the buffer, nice and fast...
This may seem complex, but it really makes life much easier for the programmer, who really doesn't want to care that much about what the optimal underlying representation of a Unicode (or binary) string is...

[StefanKarpinski]

I don't think changing the encoding of an object is going to have the performance that you might hope.

[tkelman]

IIRC @vtjnash usually posts from a phone, so he may come across as overly concise. Erring on the side of conciseness is not a bad idea though. If someone wants you to elaborate they can ask you to elaborate.

FAR too much bandwidth has been spent on string concatenation and related topics in the last few weeks. You're absolutely right about documentation leaving a lot to be desired, and we appreciate the help in making it better. Inconsistencies in base Julia's handling of null termination and encoding validity checks can be addressed one specific item at a time. But as far as major new features like mutable strings, everyone will be much happier if you first pursue development of a package outside of base Julia, where design and API decisions can be done independently as you see fit. If you announce the under-development package on the mailing list then people who are especially interested in string processing can follow along and contribute to the development. Before even coming up with a working proof of concept, you're not likely to convince the core team that these are things we absolutely need or want as part of the core language. "Better strings" sounds great in theory, but if it comes at the cost of two dozen more conversations like this one, no one has the patience for that.

[StefanKarpinski]

@ScottPJones, I think what @vtjnash is getting at is that you should endeavor to increase the ratio of code and research to commentary. You've managed to make it into the top 30 posters by volume on this repository in a few short weeks, outpacing many who have been regular, productive contributors for years. At this rate, you will have written more commentary on the repo than anyone else, including me, by end of year. I'm the current champion blabbermouth, but I've been at this for a while, and your rate dwarfs mine (I've also committed a bit more code). That rate is more than a little taxing to keep up with – I have mostly stopped reading your posts due to their volume, to be honest. Don't get me wrong – your enthusiasm and improvements are greatly appreciated – but please keep the comments more concise.

[JeffBezanson]

I want to highlight @tkelman 's suggestion to deal with issues one small item at a time. It is not efficient to reserve bug reports for when you happen to need ammunition for talking about how inconsistent julia is.

Instead of writing lengthy diatribes about how awful we are, please just open an issue for each small item, with 1-2 sentences saying what the problem is (e.g. "ascii and utf8 do not have the same methods as utf16 and utf32") and maybe giving an example if appropriate. Then we can just go through the bugs one by one. At peak rate, I've been known to close 5 or more bugs in a day, so things really can improve this way.

[tkelman]

Shortly after I wrote that comment I noticed #11140 and #11141 and #11142, which are great.

[StefanKarpinski]

Yes, those are great issues – thank you for posting them.

[ScottPJones]

@tkelman I'm sorry, but I felt @vtjnash's comment went a bit far, since it was his suggestion in the first place... I had intended to research the way string() worked in Julia as soon as I had time, to see if his suggestion had any merit... but had other matters to attend to. I did do the research as soon as I was able, and found the further issues that I immediately created new issues for, each one independently. (I haven't had time to think about wstring vs. string being inconsistent, so haven't dealt with that yet.)

I don't think changing the encoding of an object is going to have the performance that you might hope.

@StefanKarpinski Although nobody here seems to believe it, I do have some small experience with these matters, and know pretty well what performance I will see, having done so in the past, and having had decades of benchmarks that showed, at least with the applications of our customers, that my approaches got very good results. Hopefully I'll have time with Julia to be able to show you.
I think also think the comment about "increase the ratio of code and research to commentary" is simply wrong (and insulting). I have spent a lot of time researching the issues I've found, and have already contributed to changes in the C code, in Julia code, in Scheme code, and in documentation, just a few short weeks after learning the language. My "commentary" has been an attempt to understand how things are, to find out what can or should be changed if possible, and how best to get the capabilities I feel that I need into Julia. Just because you've committed a lot more code, doesn't mean you are always right either. I do try to listen carefully to what all of you have said, and do go back and research old threads (but there's quite a lot there, and it's not always easy to know just where to look), and think I've been very responsive about accepting any constructive criticism about my issues and PRs... (people wanted pure Julia functions for the conversions, instead of C, I did that, they wanted pure Julia for the length optimizations, I did that [even though @JeffBezanson had implemented it in C originally]). If I've had a lot to say, it is because there are a lot of areas that need some discussion... I'd like to say that I've actually had a positive effect in getting people to think about licensing issues (with GPL, and better headers), with having decimal arithmetic available, better documentation, and other issues. Hopefully, there won't too much more that needs discussion about...

@JeffBezanson Please! That's pretty bad to say I "reserve bug reports for when you happen to need ammuntion"... That's also wrong and insulting, and also shows you didn't pay attention to my "diatribe". I said that I discovered those just then, doing the research that @vtjnash had made a snide remark about... and I immediately created small issues, as I'd been told is the better way to do things... (I actually am able to learn new stuff, BTW). I've also not be creating bug reports willy-nilly...
I've also been attempting to fix them also... and am waiting on a number of PRs to be approved to fix said issues. I'm actually trying to save all of you time...
Also proving you don't seem to read what I write, I've never said a word about "how awful" you are... and have actually been quite glowing in my praise of all that you've accomplished... I do think you owe me an apology. My criticisms, not of any of you, have been solely meant to be constructive ones of some flaws in the language... from all the discussions on GitHub before I came, and breaking changes just in the past month, I don't believe you really think Julia is perfect as is, and cannot be improved in some ways.

Sorry for the long-winded response... but I felt it had to be said...

[carnaval]

@ScottPJones Come on man, nobody is insulting anyone. People are just trying to gently nudge you into the style of comment/contribution which is appreciated here : mostly technical, short and to the point. If you feel the forms are missing sometimes it's because there is no need for it, everyone is implicitly assumed to be friendly and with good intentions.

We are not many having the opportunity to spend time working on the project so we do value this time.

[ScottPJones]

@carnaval Sorry, but having somebody accuse me of holding back bug reports, or saying "how awful" people are, when it is totally false, does feel rather insulting and defamatory... and is not doing me the favor of implicitly assuming me to be friendly and with good intentions... (which I am!)

I also value my time, and have just been trying to work out issues that caused problems in my wanting to use Julia on a project... I hope we'll be able to (although the tuplecalypse has killed off our plan of using Julia + ODBC.jl... too much time spent trying to figure out why it was broken in 0.4... so that developer has sadly decided to go back to Python...)

[StefanKarpinski]

This is not an attack on your knowledge, abilities or intentions. It is a request: please keep your messages shorter, fewer, and more to the point.

[vtjnash]

@ScottPJones no offense intended in my comment above. as Stefan mentioned above, your replies sometimes get quite long, so reading them thoroughly takes time that most would rather be spending doing their own work. consequently, I am suggesting that waiting to post until you have a more complete story is likely to be a better use of everyone's time (including your own). conciseness and completeness will be highly valued by your readers.

reporting issues is great, even ones that just point out API inconsistencies. but talking about them in unrelated threads is unlikely to ever be appropriate. it's frustrating to other developers to be browsing for updates to a topic and instead be wading through unrelated commentary. (and as you've unwittingly discovered now, it seems to have become one of Jeff's personal pet peeves).

even though JeffBezanson had implemented it in C originally

well, to be fair, he wrote that code several years prior to the existence of Julia :)

[ScottPJones]

OK, what about adding one of the following characters as a generic concatenation operator (or do these also have some mathematical meaning that is used elsewhere that needs to be preserved [although, given that * was overloaded for this, I'm not sure that argument would even hold water])?
⊕ (circled plus, '\u2295')
⨁ (n-ary circled plus, '\u2a01') (this is my favorite now)

[StefanKarpinski]

No, that is the direct sum symbol and it's is used in mathematics all the time.

[ScottPJones]

Well, so is *! ☺
Where is it currently used in Julia or any packages then? It wasn't even in the parse table...
What is it's definition on a vector?

[ScottPJones]

@StefanKarpinski If you are think that the circled plus and n-ary circled plus might be used for mathematics direct sum (they are also used sometimes for the XOR operator), wouldn't you agree that they should be added to the parser tables? (they should also be added to the \ shortcuts, then, as well).
Also, are there any Unicode characters that you wouldn't object to (not necessarily to end up as part of a PR for general string/vector concatenation, but at least available to people to define as they wish, or to be able to experiment with)

[KristofferC]

@ScottPJones I don't know if you are aware but most of your comments ooze with passive aggressiveness. It has gotten to the point where I basically stop updating myself on the newest issues because I cringe so much. It is really frustrating...

[ScottPJones]

@KristofferC No - I'm not aware of just what is bothering you. I've been trying to get something done so that I could add something I considered important to Julia that was missing, namely a generic concatention operator (and possibly a corresponding assignment operator). So far, every possibility I've raised has been shot down, for one reason or another, the latest being:

No, that is the direct sum symbol and it's is used in mathematics all the time.

I tried to point out that that reason was not at all consistent with the idea that * was OK...
(really used all the time), in contrast to the 2 circled plus symbols, which weren't even in the operator
tables, which, considering the hundreds of Unicode characters that are in the tables, indicated either that the direct sum operation was not very important in Julia after all, and/or that it was an oversight
not having those symbols in the table.

I'd appreciate some suggestions as to what might be acceptable for operators, that don't already have a well known use on vectors or matrices (I'd only seen circled plus for XOR before, but since it does have a meaning for vectors, even if it isn't currently implemented in Julia, I wouldn't want to use it), and preferable a symbol that might make somebody think of addition or concatenation.

You could always unsubscribe to the PRs and Issues that I create, if they bother you so much. If you'd like to tell me just what I'm doing wrong in my messages, please do (but by private e-mail, so you don't end up aggravating the other people following these issues I've been raising). I'd appreciate it, because I really don't want to aggravate anybody, I'm trying as best I can to adapt myself to working well within this community, and if you'd read all my messages, I'd think you'd find I've been very appreciative of all the people who have donated their time answering my questions, reviewing my PRs, and pushing me to be a much better Julia programmer, and I've been gushingly complementary of the great work the contributors have accomplished.
I've been questioning code & architectural decisions... not doubting the intelligence of anybody.
I wouldn't trust myself to make well-informed decisions regarding algorithms/data structures for a lot of the esoteric math that julia supports, for example, but I don't think that makes me stupid.

[tkelman]

⊕ is in the operator table

julia/src/julia-parser.scm

Line 16 in 01abdc3

(define prec-plus '(+ - ⊕ ⊖ ⊞ ⊟ |.+| |.-| |\|| ∪ ∨ $ ⊔ ± ∓ ∔ ∸ ≂ ≏ ⊎ ⊻ ⊽ ⋎ ⋓ ⧺ ⧻ ⨈ ⨢ ⨣ ⨤ ⨥ ⨦ ⨧ ⨨ ⨩ ⨪ ⨫ ⨬ ⨭ ⨮ ⨹ ⨺ ⩁ ⩂ ⩅ ⩊ ⩌ ⩏ ⩐ ⩒ ⩔ ⩖ ⩗ ⩛ ⩝ ⩡ ⩢ ⩣))

[ScottPJones]

I should have been clearer, it is N-ary circled plus, \u2a01, which is the one I wanted to use, which is not in the tables.

[ScottPJones]

Also, neither is in the shortcuts either, making them not very usable.

[yuyichao]

What do you mean by shortcut?

\oplus and \bigoplus ?

[ScottPJones]

OK... well, that's rather inconsistent! Why are those \o... insted of \circled... like the rest of the similar operators, and the standard Unicode names for the characters?

\circ               \circlearrowright    \circledast          \circleddash         \circledrightdot     \circledwhitebullet  \circleonleftarrow   \circleulquad        \circlevertfill
\circeq             \circledR            \circledbullet       \circledequal        \circledstar         \circlellquad        \circleonrightarrow  \circleurquad        \cirfnint
\circlearrowleft    \circledS            \circledcirc         \circledparallel     \circledtwodots      \circlelrquad        \circletophalfblack  \circleurquadblack

Thanks very much for the information!
So, it is just the N-ary circled plus character, that isn't defined in the parser table.

julia> a ⨁ b
ERROR: syntax: extra token "⨁" after end of expression

julia> a ⊕ b
ERROR: UndefVarError: ⊕ not defined

[yuyichao]

Well, for the inconsistency, you'd better blame whoever come up with the names in latex

[hayd]

This PR is about adding more operators to the parser. Changing the string concat operator should be another issue.

---

@ScottPJones My 2 cents RE noise/passive aggressiveness: remember on the internet no-one can hear your tone so additional care should be taken to not sound like an ass. If you feel wronged by whatever was said by someone else (if they sounded like an ass), remember it probably wasn't meant as a slight (everyone here is just doing their best to improve julia).... so never escalate and never call people out for slights against you personally.

In this case, some feedback from several core julia devs was "be more concise"... two of your next posts were screen-long. So I think some reflection on your part would be worthwhile.

[ScottPJones]

@yuyichao Thanks for that information - I didn't know where the shortcuts came from originally.
@hayd I opened this PR precisely to allow for adding a general / string concatenation operator. (See my opening message).
I've tried to put emoticons to try to give an indication of tone, that didn't seem to help in this case.
I have tried to be more concise in the last month, sorry if Kristoffer's comment pushed a button!

[tkelman]

@ScottPJones since it sounds like you're not very familiar with LaTeX, just FYI the convention with \bigoplus is it's used like this:

in the same manner as a sum or product of an expression over an index set, not as an infix operator. That's probably why it's missing from the operator table, if I had to guess.

[ScottPJones]

Thanks for the info! Like my limited knowledge of linear algebra, my knowledge of Latex dates back over 30 years, and nothing since.