Add prefixes to results from bucket.getFiles()

[ghost]

No description provided.

[silvolu]

Lint error: https://travis-ci.org/GoogleCloudPlatform/gcloud-node/builds/46133984#L227

[mchmielarski]

Fixed

[silvolu]

@mchmielarski However it goes, could you squash the 'Fix *' commits at some point?

[stephenplusplus]

Sorry for my novice question, but could you provide a practical use of prefixes, delimiters, and handling the result returned?

[ryanseys]

Re: use case for prefixes is as I understand it a simple filter for files e.g. file search

[stephenplusplus]

I'm interested in terms of how one would use this library after merging this PR. I figured instead of guessing by the code, it would be best to let the suggester (PR-sender) help me understand :)

[mchmielarski]

with prefixes and delimiters you can use buckets as tree structure in easy way. I think that comment added to bucket.getList() explains it.

[stephenplusplus]

Would you mind demonstrating with a practical use?

[sindresorhus]

Would be nice with some practical use-cases in the docs. As a normal users I would have no idea what I should be using it for.

[ryanseys]

Not considering the added complexity of nextQuery traversal, a simple practical example for this would be the following:

Get folders in bucket

function getFolders(bucket, callback) {
  var query = { delimiter: '/' };
  bucket.getFiles(query, function(err, files, nextQuery, prefixes) {
    callback(err, prefixes);
  });
}

[stephenplusplus]

Thank you for that. I know the functionality is in the API, so it makes sense to expose it. But before I can figure out what a good API looks like for it, I need to see how it's meant to be used. Are there other use case anyone can think of? The complexity of prefixes + delimiters still seems quite expensive for such simple results -- assuming the results are "folder names".

[ryanseys]

I'm only using it with delimiter = / right now but you can combine that with the prefix to list folders in folders. Again, a very simple use case.

function getFoldersInFolder(bucket, folder, callback) {
  var query = { delimiter: '/', prefix: folder };
  bucket.getFiles(query, function(err, files, nextQuery, folders) {
    callback(err, folders);
  });
}

getFoldersInFolder(storage.bucket('myBucket'), 'my-photos/', cb);

[ryanseys]

Using delimiter as something other than / does not seem to have a useful use case unless you're mimicking a directory structure with flat files where the delimiter is a well-known value in the file names e.g. var__usr__bin__file1.png and var__usr__bin__file2.png where the delimiter would be __ in this case. The prefixes value returned in the response is definitely useful when used in combination with the delimiter value.

As for real-world applications and why-the-heck-would-anyone-do-that-anyway questions, it's beyond me.

[stephenplusplus]

As a library that isn't about being a straight API bridge, maybe we can abstract what we're interested in with delimiters and prefixes, and expose them in a more usable way. So far, it sounds like getting files from a specified folder is the benefit. We should be able to make this API work to accomplish that:

var query = {
  folder: 'images'
};

myBucket.getFiles(query, function (err, files, nextQuery) {
  // files = only files in a directory "images"
  // delimiter defaults to "/" when `folder` option is given
});

And if we think this is important, we can go so far as to create a class for folders:

myBucket.getFolders(function(err, folders, nextQuery) {})
myFolder.file("image1.jpg")

[ryanseys]

Thought experiment: Is it possible to upload a file with a filename that contains the character "/" but isn't a folder? If this is true, then this approach may not work for all cases. The only response value that isn't currently exposed is the prefixes value. Another thought would be to expose the raw response object for "advanced" users that need to do special-cases such as this.

[stephenplusplus]

I'm confused on this as well, and if I remember correctly, even the Dev Console UI was confused about it. However, as far as I know, / is a reserved character that is invalid in filenames, so a file by the name of "photo/1.jpg" wouldn't be valid, instead, we should all assume that means folder "photo", file "1.jpg" ("all" being: our library, the API, the console UI, the user)

[sindresorhus]

However, as far as I know, / is a reserved character that is invalid in filenames

Not on OS X. Creating a folder named foo/bar works fine.

[mchmielarski]

Check it: https://github.com/evenemento/gcloud-node/tree/v2
and examples/example-1.js

[ryanseys]

@sindresorhus You're right, you can name the folder with a / in OS X strangely enough, but it's actually stored as a : instead. You can see this by a simple ls or by attempting to cd into the directory. Also, when you upload this folder to google cloud storage, it's stored as a : as well.

At least, this is the case that occurs when using the "Upload Folder" feature in the Google Cloud Developer Console. Might not have anything to worry about in this case then.

[sindresorhus]

@ryanseys Yeah, weird, you're right.

[stephenplusplus]

I think supporting a Folder object / folder-specific methods is unnecessary. If we can trust / as a directory separator, it should be straightforward for the user to get a file in a folder with: bucket.file('file/in/folder.jpg'). I do think exposing a method to get only files from a folder is important, though: bucket.getFiles({ folder: 'folder/folder' }, function (err, files) {}) where we can internally use the API's prefix/delimiter options.

[ryanseys]

Yes! 100% agree.