Update hfh

[severo]

In particular: we now use the new list_repo_refs method, and the new revision parameter when we create the refs/convert/parquet branch.

[Wauplin]

Looks good to me! :)

Out of curiosity, how to you get the source_revision (which I guess is the initial commit) ? We have a user that wants to retrieve this information programmatically but for now we don't have a endpoint for that so I'd be interested if a workaround exists for it :)

[severo]

how to you get the source_revision

Oh, for now, it's hard-coded to "main" (can be set to another value with an environment variable, but still: it's a constant).

And it's not really useful to create the branch from main or from any other commit, since the first action is to delete all the contents, before adding the files we are interested in (parquet files).

The source_revision reference, in our case, is used to know the commit that will be converted to parquet.

[Wauplin]

Oh, for now, it's hard-coded to "main" (can be set to another value with an environment variable, but still: it's a constant)
And it's not really useful to create the branch from main or from any other commit, since the first action is to delete all the contents, before adding the files we are interested in (parquet files).

Oh ok, makes sense. I thought you had a trick to know the initial commit.
If (or when?) it's ever possible, it would allow you to skip the "delete all" step before uploading parquet files. But that will be for later.

[severo]

Yes, that's what I mentioned in the commit message: 5364f81

Maybe we will want to change this later once huggingface/huggingface_hub#1288 is fixed.

[julien-c]

if you just want an empty tree (and not the initial commit of the repo), you can also create a commit completely detached from any branch (i think we discussed this in the other issue but i don't remember the discusion)

[Wauplin]

you can also create a commit completely detached from any branch

Could be a solution but not implemented via the HTTP endpoint so far.
I'm personally not sure it's a good idea as the .gitattributes file would be lost which is I think a very important one.

[julien-c]

(yes, but you could re-add a "standard" gitattributes quite easily in your commit i guess)

[julien-c]

(i do agree with you, this is just for the sake of discussion)