More robust detection of user code

[timholy]

I noticed that --track-allocation=user reports allocation for a subset of files in base/ (I'm sure --code-coverage=user would do the same). This appears to fix the main problems; for some reason, a file multi.jl.mem still gets written in base, and I have no idea why that would happen.

[timholy]

It should be clarified that the problems this fixes arose in files that defined their own modules living inside Base.

[timholy]

OSX has timed out on this twice now. Anybody have insights? I'm guessing that since Linux & Windows are both working, it's probably OK.

[tkelman]

@timholy the Homebrew dependencies are downloaded from SourceForge which is frequently unreliable. @staticfloat should we maybe copy over all of the dependencies that we need on Travis to the tap, so we can keep them on more-reliable S3? Upstream Homebrew is in need of a source of funds before they'll consider switching to S3 themselves.

[timholy]

I see you must have restarted again, thanks. I'll merge.

[timholy]

Am I right in thinking that now the recommended approach is just to add the backport_pending label?

[staticfloat]

@tkelman after grepping around in the Homebrew source, it sounds like we just need to pass --root-url=<our AWS root url> to override the default bottle location. It'd be nice if we could have it fallback as a mirror, but that's asking quite a lot in this instance. :P

I don't think the Homebrew maintainers will want to add our servers as an "official mirror", since we can't really give them upload access and then ask them to upload these packages any time they're updated. I don't want to maintain a mirrored copy of their formulae (modulo a changed bottle root_url) which leaves the --root-url argument option. I just need to make a cron buildbot (old habits die hard) script to copy over new bottles to our mirror as soon as they're added, which shouldn't be impossible. Looks like the packages needed to mirror are:

$ brew deps julia | grep -v staticfloat
fftw
gcc
git
gmp
libffi
mpfr
pcre
pkg-config
python

I don't think python or pkg-config are being installed since they're already installed, but can't hurt to be cautious, right? Anything starting with staticfloat/julia/ is already mirrored, so we don't need to worry about those.

[staticfloat]

@timholy I feel like a rather large number of my github threads are hijackings of yours. I'm not sure if that's just because of the prodigious amount of Julia work you do, or instead because of some strange twist of fate that causes you to bump up against things I'm responsible for so often. ;)

[tkelman]

Am I right in thinking that now the recommended approach is just to add the backport_pending label?

Since this was a PR, I think that's preferred, yes. For commits that reference a corresponding issue number so the commit is easy to find from the issue page, I also think the backport pending label is harder to lose track of than mentioning @juliabackports. I try to reserve the latter for cases where it's the only option, commits that don't mention any issue numbers or come from a PR. We have to dig through the julia-backports list periodically to catch any mentioned-but-not-yet-backported commits. That's a little easier to do when that list stays a bit quieter, only has threads for commits that need to use the list.

[timholy]

@staticfloat, hijack away.

I'm not sure if that's just because of the prodigious amount of Julia work you do

"Takes one to know one" (there, now I feel like I'm back in 2nd grade).

[tkelman]

script to copy over new bottles to our mirror as soon as they're added, which shouldn't be impossible

That's roughly what I had in mind, yeah (also libgit2 for --HEAD). Git, gcc, and cmake are also likely to be preinstalled on Travis' OSX workers, but might not be as up-to-date as we need. The Travis OSX runs start with (as a result of brew tap staticfloat/julia; brew rm --force $(brew deps --HEAD julia); brew update; brew install -v --only-dependencies --HEAD julia)

Uninstalling cmake...
Uninstalling gcc...
Uninstalling gmp...
Uninstalling mpfr...
Uninstalling pkg-config...

[staticfloat]

@tkelman new issue created: JuliaCI/julia-buildbot#13

@timholy Welp, looks like my work here is done. ;)

[tkelman]

@timholy are you in a hurry about backporting this or can it wait until 0.3.6?

[timholy]

No hurry whatsoever. If/when it does get backported, you should include 3576665

[tkelman]

@timholy the is_submodule function that this is using does not exist on release-0.3, it was added as part of #8656. We can add it manually I guess, in the same location as in 3576665?

[timholy]

That sounds fine. Perhaps because I always use 0.4, I'm also OK with a few bugs being "wontfix on 0.3." But if you're willing, your plan sounds fine.

[tkelman]

Backported Jeff's commit, except for the parts that touched src/dump.c which do not apply to release-0.3, in 754ba07