Julia CONFIG dictionary (closes #2430)

[vtjnash]

I figure the only way to get Jeff to comment on #2430 was to submit the code for it as a pull request. So this creates a CONFIG dictionary for arbitrary configuration parameters.

And assign configuration entries simply:

CONFIG["answer_color"] = "green"

The user can also see the current bindings:

show(CONFIG)

Julia Configuration:
OUTPUT_STREAM = TTY(connected,0 bytes waiting)
answer_color = "\e[1m"
gfx/backend = "gtk"
input_color = "\e[1m"

The standard usage of this would be to provide default values for any configuration items in the dictionary:

CONFIG["answer_color", :default] = "blue"

Deleting those values resets the value to the default:

delete!(CONFIG,"answer_color") #CONFIG["answer_color"] is reset to default (blue)

Other parameters can be deleted too as follows:

delete!(CONFIG,"answer_color", :default, :value, :getter, :setter, :delete, :entry)

The CONFIG object supports getter, setter, and delete notification methods. Here's an example from Base that uses a getter method to create compatibility with the old environment variables and converting the input color through the text_colors mapping:

CONFIG["answer_color", :default, :getter] =
    (default_color_answer,
    (key,value)->get(text_colors,symbol(get(ENV, "JULIA_ANSWER_COLOR", value)),value))
CONFIG["input_color", :default, :getter] =
    (default_color_input,
    (key,value)->get(text_colors,symbol(get(ENV, "JULIA_INPUT_COLOR", value)),value))

Here's another example from Base at using a setter method to react to changes in the value:

CONFIG["OUTPUT_STREAM", :default, :setter] =
    (STDOUT,
    (key,value,hadvalue,newvalue::IO) -> global OUTPUT_STREAM = newvalue)

Note that the setter will be called whenever the value is changed, including after a value is deleted (and reverted to default) and right after installing the setter.

[JeffBezanson]

Overall I'm not comfortable with this general idea. I don't fully understand what needs to be configured, and why. It's best to avoid configuration and options, and instead pick good defaults, that can be overridden at the point where they are used. Otherwise you get state-dependent programs that don't work for people who don't have your configuration file. Options lead to ugly and brittle systems. They also make it easier to avoid design decisions by saying "make it an option".

Considering the example of the graphics backend, we don't need a configurable backend so much as we need one that works well. And configuration is weaker than composability: display(window, plot) would ideally work for all combinations of windows and plots, so telling the plotting system where it will be drawing via a configuration option is redundant.

This particular implementation is clever, though it feels rather complex. It clearly wants keyword arguments :)
We would probably want an on-disk format. Ini file seems reasonable.
Things like OUTPUT_STREAM are a bit different though, and can't just be printed as text. We have tried to eradicate OUTPUT_STREAM entirely, but it keeps growing back like a fungus. We don't want setting that variable to be a way to do stuff. It looks like nothing ever changes it any more, so I think we should get rid of it and replace all uses with STDOUT.

Anyway I'd rather avoid adding more options. The answer color thing is frivolous and I don't want a whole subsystem for configuring it, as hilarious as that would be.

[ViralBShah]

I prefer an ini file too. CONFIG suggests that changes would be reflected immediately, but it is only a holder. The ini file will obviously have an in memory representation, which should be const.

[vtjnash]

@JeffBezanson for one thing, loading a second window manager (e.g. Gtk) can cause Tk to segfault, so composability is less helpful than a shared default for the window manager (window's themselves are a different thing -- I'm not sure that an OUTPUT_WINDOW analog to OUTPUT_STREAM is necessary since show(window, plot) should suffice). for another thing, I forgot to even include the JULIA_EDITOR configuration variable, since they are somewhat scattered. this is intended to consolidate and simplify these configuration parameters. Additionally, there's also system configuration values that the user may want to override locally, such as #2699.

While I agree that good design is important, at some point, you can't simply say "everyone just needs to use emacs", or I'll be forced to create a julia-vim fork.

Why bring in the complexity and limitations of a .ini file when we have the superior availability of the load-time .juliarc.jl file? Also, what is the point in having a const interface? That is starting to sound even more complex and silly just for setting the output colors, with the added disadvantage that I can't add conditional logic for @windows_only to make my configuration file portable.

True, OUTPUT_STREAM is a bit of an oddity here. Currently OUTPUT_STREAM is a const in all but declaration, since it cannot be changed (assignment to Base.OUTPUT_STREAM is not allowed). While good library code passes around an io object, I know Westley had been asking for a way to redirect it, to better capture the output of arbitrary user code, for better integration with the Forio IDE.

@ViralBShah This provides the option of representing changes immediately, if meaningful, since it provides getter and setter callbacks. However, I suspect that usually it is not meaningful.

[JeffBezanson]

We're using environment variables because they're the standard mechanism people already know how to work with (e.g. adding to your bashrc). Whether they are "scattered" is more of a documentation and presentation issue, which exists with any configuration system (how to know what options and values are available).

I'm ok with .juliarc.jl because it is kind of unavoidable --- either you type a few commands at the start of every session, or you automate that.

I'm not sure how to handle something like gfx/backend. The existence of the option doesn't solve anything --- packages would have to be written to respect it, gtk+tk still wouldn't work together, etc. It just seems like a really weak way to do dependency injection.

[ViralBShah]

I really do not like environment variables for stuff like answer color. They are ok for stuff like paths, I feel. Something like CONFIG would be nice to customize the REPL.

[JeffBezanson]

I think I am worried about CONFIG conflating many different things: for setting text colors it is totally fine, but something like OUTPUT_STREAM wants a dynamically-scoped variable. gfx/backend is yet another case, and might affect what libraries are linked.

There is a staging problem: we have to define when configuration changes take effect. The callback feature makes it possible for changes to take effect immediately, but this places an undue burden on many use cases. For example, if I can select whether multiprocessing uses TCP or UDP, does changing it mean that all current connections are torn down and redone using the new protocol?

Maybe we could divide options into build time, init time, and run time.

[ViralBShah]

Good point. We already use make for build time options, and we should just keep it that way. Init and run time are good to separate.

[JeffBezanson]

Julia code has a "build time" which occurs whenever the code is first loaded, for example by using a package or calling include. It is possible some programs would want to configure things at this stage, and moreso if we start statically compiling.

[ViralBShah]

Another use case of this is the choice of the BLAS library, choice of LIBM, etc.

[JeffBezanson]

But those are build-time options, so I don't see how it helps to have a julia interface to them.

[ViralBShah]

There are runtime checks too - checking the type of blas, whether 64 bit blas was built. We may even want to query versions of libraries and such.

[stevengj]

I'm fine with using ENV as the Julia configuration dictionary, since Julia-specific configurations can go into .juliarc.

However, it would be nice to have a good convention for a separate namespace for module-specific configurations. For example, I'd like to be able to do:

ENV[:PyCall, "python"] = "python3"

in my .juliarc. This could be encoded as an ordinary environment variable called "PyCall:python", i.e. this would just be syntactic sugar for a naming convention. (bash won't let you create such environment variables, but I don't see this as a big deal for Julia-specific settings.) Or it could be stored in a separate Dict within the EnvHash.

It would also be nice if Julia-specific configuration settings could be something other than strings, but that would require a separate Dict within the EnvHash.

[ViralBShah]

Close this?

[stevengj]

See also JuliaLang/Juleps#38; we really need persistent per-package configuration options.

[tkelman]

not all configuration is for packages though

[ViralBShah]

Maybe close this one?