Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mktempdir() now supports prefix. #23237

Closed
wants to merge 406 commits into from
Closed

mktempdir() now supports prefix. #23237

wants to merge 406 commits into from

Conversation

sambitdash
Copy link
Contributor

@sambitdash sambitdash commented Aug 13, 2017

The libuv is used for this implementation in a platform independent manner. This is for the issue #22922. The mktemp() or tempname() functions are not affected by this change.

This may be still acceptable as a module owner can create her own temp directory and then create files inside that. Hence, the importance of prefix on files may not be as significant.

Edit: fix #22922

@sambitdash
Copy link
Contributor Author

With this #22998 can be closed.

@sambitdash
Copy link
Contributor Author

Added test cases which show behavioral differences across OS. For example, '*' is not an accepted prefix in Windows which is perfectly permitted in Linux as part of a filename.

@ararslan ararslan added the filesystem Underlying file system and functions that use it label Aug 13, 2017
@ararslan ararslan requested review from vtjnash and kshyatt August 13, 2017 20:42
@sambitdash
Copy link
Contributor Author

Error reported in Mac OS X is completely unrelated. It's an InterruptException which this code does not deal with. The Windows 32-bit case is more like incomplete run and is not a true failure.

@sambitdash
Copy link
Contributor Author

sambitdash commented Aug 21, 2017

Closing as there is not much of activity in past week on this. If there is interest in this change let me know, I will be happy to reopen.

@sambitdash sambitdash closed this Aug 21, 2017
@StefanKarpinski
Copy link
Member

Seems a bit premature to close this. @nalimilan, you seemed engaged on the original issue here. Can you review this and see if it addresses the problem? Frankly, I don't have enough context to understand what problem this is solving, but that doesn't mean it's not a good idea.

@sambitdash
Copy link
Contributor Author

sambitdash commented Aug 21, 2017

PR #23259 affected the last test errors. Now fixed.

@sambitdash
Copy link
Contributor Author

Travis failure seems unrelated.

@StefanKarpinski
Copy link
Member

@vtjnash has said he can take a look at this in a few days. Sorry for the delay, but this is a useful change.

Copy link
Member

@nalimilan nalimilan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Looks mostly good, though I'd like a deeper review of libuv parts by somebody familiar with that code. Please improve the commit message to explain that you switch to the new uv_fs_mkdtemp function, and which libuv version it was introduced in.

req = Libc.malloc(_sizeof_uv_fs)
try
ret = ccall(:uv_fs_mkdtemp, Int32,
(Ptr{Void}, Ptr{Void}, Cstring, Ptr{Void}),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incorrect indentation.

Copy link
Contributor Author

@sambitdash sambitdash Aug 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the same indentation as readlink, although my personal preference would be to align to ccall.

+        ret = ccall(:uv_fs_mkdtemp, Int32,
+                    (Ptr{Void}, Ptr{Void}, Cstring, Ptr{Void}),

template = prefix*"XXXXXX"
tpath = joinpath(parent, template)

req = Libc.malloc(_sizeof_uv_fs)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you use the same approach as for readlink, but readdir uses a different strategy, by using zeros, which allows skipping the try... finally. @vtjnash Which approach is recommended?

Copy link
Contributor Author

@sambitdash sambitdash Aug 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess I now get the approach here. In jl_uv.c all request structures are allocated in stack (declared as local var) and passed on to the function in the c-layer. The cleanup is carried out on the address to the request like &req. In the Julia layer you cannot have a stack variable so zero is a means to create a Julia array{UInt8} which will be memory managed in the Julia layer. Hence, no special action on a throw and clean up will be called on this object after the request object is used in the ccall. The approach is a bit hard to understand initially. Are try...catch overheads very significant? I can change the code to the zero approach.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would think the simplest approach is the best one, but others must know better.

test/file.jl Outdated
@test_throws Base.UVError mktempdir(; prefix="*")
@test_throws Base.UVError mktempdir(; prefix="cdcdccd/")
else
# '/' is accepted in a prefix but affects the overall path and permissions.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That comment sounds misleading, since / is not accepted by the next line.

Copy link
Contributor Author

@sambitdash sambitdash Aug 23, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The whole behavior is very messy. I think we should just not accept '/' in prefix. Some samples:

julia> mktempdir("/";prefix="tmp/")
"/tmp/m12rZm"

julia> mktempdir("/tmp";prefix="/")
ERROR: mktempdir: permission denied (EACCES)
Stacktrace:
 [1] uv_error at ./libuv.jl:68 [inlined]
 [2] #mktempdir#12(::String, ::Function, ::String) at ./file.jl:359
 [3] (::Base.Filesystem.#kw##mktempdir)(::Array{Any,1}, ::Base.Filesystem.#mktempdir, ::String) at ./<missing>:0

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not too worried about this, as these behaviors make sense, but differences between OSes are annoying. What happens with \ on Windows?

# Behavioral differences across OS types
if Sys.iswindows()
@test_throws Base.UVError mktempdir(; prefix="*")
@test_throws Base.UVError mktempdir(; prefix="cdcdccd/")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does / without any characters before it also throw? What about \ or C:\?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Give me a couple of days. My windows VM is needs some fixing before I can test this. I am a native Linux programmer :-).

test/file.jl Outdated
end

# Unicode test
tst_prefix="\u2200x\u2203y"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Directly use the Unicode chars, that's clearer.

kshyatt and others added 15 commits August 24, 2017 08:33
* Doc checkout methods and type
Cf. problem reported there:
#22948 (comment)
Bracket pasting was not removing the indentation corresponding
to the prompt (i.e. 7 spaces for "julia> ").
* Deprecate expm in favor of exp.

* Remove reference to expm in the linalg docs
* REPL-documentation for more internals.

* Copy edit 'REPL-documentation for more internals.'

* New section "Special Types"
Previously the element type of the output was the smallest type that
would fit the union of the input's individual element types. Now the
output has an identical element type to the input. Fixes #22696.
The libuv is used for this implementation in a platform independent
manner.
The libuv is used for this implementation in a platform independent
manner.
The libuv is used for this implementation in a platform independent
manner.
The libuv is used for this implementation in a platform independent
manner.
The libuv is used for this implementation in a platform independent
manner.
The libuv is used for this implementation in a platform independent
manner.
The libuv is used for this implementation in a platform independent
manner.
@sambitdash
Copy link
Contributor Author

closing and resubmitting with rebasing.

@sambitdash sambitdash closed this Sep 8, 2017
@fredrikekre
Copy link
Member

You can rebase and force-push to the same branch and keep the same PR.

@sambitdash sambitdash deleted the I22922 branch September 8, 2017 11:50
@sambitdash sambitdash restored the I22922 branch September 8, 2017 11:56
@sambitdash sambitdash deleted the I22922 branch September 8, 2017 12:00
@sambitdash sambitdash restored the I22922 branch September 8, 2017 12:50
@sambitdash
Copy link
Contributor Author

@fredrikekre The forced push detached the PR and could not reopen it. The new PR on this is #23634.

@fredrikekre
Copy link
Member

Yea, I think you need to reopen before force-pushing in such a case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
filesystem Underlying file system and functions that use it
Projects
None yet
Development

Successfully merging this pull request may close these issues.

mktempdir() should provide an option of a prefix very similar to you can do with mkdtemp in posix