Skip to content

Commit

Permalink
Fix various links/urls for apache/arrow-julia (#457)
Browse files Browse the repository at this point in the history
Should fix #416.

---------

Co-authored-by: Sutou Kouhei <kou@clear-code.com>
  • Loading branch information
quinnj and kou authored Jun 7, 2023
1 parent 2d1114e commit f039a5e
Show file tree
Hide file tree
Showing 4 changed files with 8 additions and 14 deletions.
10 changes: 2 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,8 @@
# Arrow

[![docs](https://img.shields.io/badge/docs-latest-blue&logo=julia)](https://arrow.juliadata.org/dev/)
[![CI](https://github.com/JuliaData/Arrow.jl/workflows/CI/badge.svg)](https://github.com/JuliaData/Arrow.jl/actions?query=workflow%3ACI)
[![codecov](https://codecov.io/gh/JuliaData/Arrow.jl/branch/master/graph/badge.svg)](https://codecov.io/gh/JuliaData/Arrow.jl)
[![CI](https://github.com/apache/arrow-julia/workflows/CI/badge.svg)](https://github.com/apache/arrow-julia/actions?query=workflow%3ACI)
[![codecov](https://app.codecov.io/gh/apache/arrow-julia/branch/main/graph/badge.svg)](https://app.codecov.io/gh/apache/arrow-julia)

[![deps](https://juliahub.com/docs/Arrow/deps.svg)](https://juliahub.com/ui/Packages/Arrow/QnF3w?t=2)
[![version](https://juliahub.com/docs/Arrow/version.svg)](https://juliahub.com/ui/Packages/Arrow/QnF3w)
Expand All @@ -40,12 +40,6 @@ The package can be installed by typing in the following in a Julia REPL:
julia> using Pkg; Pkg.add("Arrow")
```

or to use the official-apache code that follows the official apache release process, you can do:

```julia
julia> using Pkg; Pkg.add(url="https://github.com/apache/arrow", subdir="julia/Arrow.jl")
```

## Local Development

When developing on Arrow.jl it is recommended that you run the following to ensure that any
Expand Down
6 changes: 3 additions & 3 deletions docs/make.jl
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,11 @@ using Arrow

makedocs(;
modules=[Arrow],
repo="https://github.com/JuliaData/Arrow.jl/blob/{commit}{path}#L{line}",
repo="https://github.com/apache/arrow-julia/blob/{commit}{path}#L{line}",
sitename="Arrow.jl",
format=Documenter.HTML(;
prettyurls=get(ENV, "CI", "false") == "true",
canonical="https://JuliaData.github.io/Arrow.jl",
canonical="https://arrow.juliadata.org/",
assets=String[],
),
pages = [
Expand All @@ -36,6 +36,6 @@ makedocs(;
)

deploydocs(;
repo="github.com/JuliaData/Arrow.jl",
repo="github.com/apache/arrow-julia",
devbranch = "main"
)
4 changes: 2 additions & 2 deletions docs/src/manual.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ The goal of this documentation is to provide a brief introduction to the arrow d

The best place to learn about the Apache arrow project is [the website itself](https://arrow.apache.org/), specifically the data format [specification](https://arrow.apache.org/docs/format/Columnar.html). Put briefly, the arrow project provides a formal speficiation for how columnar, "table" data can be laid out efficiently in memory to standardize and maximize the ability to share data across languages/platforms. In the current [apache/arrow GitHub repository](https://github.com/apache/arrow), language implementations exist for C++, Java, Go, Javascript, Rust, to name a few. Other database vendors and data processing frameworks/applications have also built support for the arrow format, allowing for a wide breadth of possibility for applications to "speak the data language" of arrow.

The [Arrow.jl](https://github.com/JuliaData/Arrow.jl) Julia package is another implementation, allowing the ability to both read and write data in the arrow format. As a data format, arrow specifies an exact memory layout to be used for columnar table data, and as such, "reading" involves custom Julia objects ([`Arrow.Table`](@ref) and [`Arrow.Stream`](@ref)), which read the *metadata* of an "arrow memory blob", then *wrap* the array data contained therein, having learned the type and size, amongst other properties, from the metadata. Let's take a closer look at what this "reading" of arrow memory really means/looks like.
The [Arrow.jl](https://github.com/apache/arrow-julia) Julia package is another implementation, allowing the ability to both read and write data in the arrow format. As a data format, arrow specifies an exact memory layout to be used for columnar table data, and as such, "reading" involves custom Julia objects ([`Arrow.Table`](@ref) and [`Arrow.Stream`](@ref)), which read the *metadata* of an "arrow memory blob", then *wrap* the array data contained therein, having learned the type and size, amongst other properties, from the metadata. Let's take a closer look at what this "reading" of arrow memory really means/looks like.

## Support for generic path-like types

Expand Down Expand Up @@ -182,7 +182,7 @@ Again, let's break down what's going on here:
* Now in `JuliaType`, note we're using the 3-argument overload; we want the `NamedTuple` type that is the native arrow type our `Interval` is being serialized as; we use this to retrieve the 1st type parameter for our `Interval`, which is simply the type of the two `first` and `last` fields. Then we use the 3rd argument, which is whatever string we returned from `arrowmetadata`. We call `L, R = split(meta, ".")` to parse the two type parameters (in this case `Closed` and `Unbounded`), then do a lookup on those strings from a predefined `LOOKUP` Dict that matches the type parameter name as string to the actual type. We then have all the information to recreate the full `Interval` type. Neat!
* The one final wrinkle is in our `fromarrow` method; `Interval`s that are `Unbounded`, actually take `nothing` as the 2nd argument. So letting the default `fromarrow` definition call `Interval{T, L, R}(first, last)`, where `first` and `last` are both integers isn't going to work. Instead, we check if the `R` type parameter is `Unbounded` and if so, pass `nothing` as the 2nd arg, otherwise we can pass `last`.

This stuff can definitely make your eyes glaze over if you stare at it long enough. As always, don't hesitate to reach out for quick questions on the [#data](https://julialang.slack.com/messages/data/) slack channel, or [open a new issue](https://github.com/JuliaData/Arrow.jl/issues/new) detailing what you're trying to do.
This stuff can definitely make your eyes glaze over if you stare at it long enough. As always, don't hesitate to reach out for quick questions on the [#data](https://julialang.slack.com/messages/data/) slack channel, or [open a new issue](https://github.com/apache/arrow-julia/issues/new) detailing what you're trying to do.

### `Arrow.Stream`

Expand Down
2 changes: 1 addition & 1 deletion src/utils.jl
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ end

toidict(x::Base.ImmutableDict) = x

# ref https://github.com/JuliaData/Arrow.jl/pull/238#issuecomment-919415809
# ref https://github.com/apache/arrow-julia/pull/238#issuecomment-919415809
function toidict(pairs)
isempty(pairs) && return Base.ImmutableDict{String, String}()
dict = Base.ImmutableDict(first(pairs))
Expand Down

0 comments on commit f039a5e

Please sign in to comment.