Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[libc][docs][POSIX] flush out documentation for the POSIX defined headers #122006

Open
nickdesaulniers opened this issue Jan 7, 2025 · 27 comments
Assignees
Labels
documentation good first issue https://github.com/llvm/llvm-project/contribute libc

Comments

@nickdesaulniers
Copy link
Member

If you go to https://pubs.opengroup.org/onlinepubs/9799919799/, then 14. Headers in the bottom left, you get a list of headers mandated by POSIX. I'd like to document them from https://libc.llvm.org/headers/index.html. We do so by manually curating some metadata about functions and preprocessor defines in yaml files. The yaml files are currently in https://github.com/llvm/llvm-project/tree/main/libc/utils/docgen. https://github.com/llvm/llvm-project/blob/main/libc/utils/docgen/arpa/inet.yaml is an example of arpa/inet.h as per POSIX. POSIX also tries to be a superset of C (though FWICT POSIX1.2024 missed C23 additions...), so most headers we have entries for already (though we should triple check since POSIX can add functions to existing C standard headers).

Once we've added entries for the new POSIX headers, you should configure your build with the 2 additional cmake flags:

  1. -DLLVM_ENABLE_SPHINX=ON
  2. -DLIBC_INCLUDE_DOCS=ON

Then you can do ninja docs-libc-html to build html you can view locally in {build_dir}/libc/docs/html/headers/**.html.

Good pull requests should implement one header at a time, and not send a PR that implements documentation for more than one header at a time. If that means that this issue is a "meta" issue for which multiple PRs will link against, so be it.

@nickdesaulniers nickdesaulniers added documentation good first issue https://github.com/llvm/llvm-project/contribute libc labels Jan 7, 2025
@llvmbot
Copy link
Member

llvmbot commented Jan 7, 2025

Hi!

This issue may be a good introductory issue for people new to working on LLVM. If you would like to work on this issue, your first steps are:

  1. Check that no other contributor has already been assigned to this issue. If you believe that no one is actually working on it despite an assignment, ping the person. After one week without a response, the assignee may be changed.
  2. In the comments of this issue, request for it to be assigned to you, or just create a pull request after following the steps below. Mention this issue in the description of the pull request.
  3. Fix the issue locally.
  4. Run the test suite locally. Remember that the subdirectories under test/ create fine-grained testing targets, so you can e.g. use make check-clang-ast to only run Clang's AST tests.
  5. Create a Git commit.
  6. Run git clang-format HEAD~1 to format your changes.
  7. Open a pull request to the upstream repository on GitHub. Detailed instructions can be found in GitHub's documentation. Mention this issue in the description of the pull request.

If you have any further questions about this issue, don't hesitate to ask via a comment in the thread below.

@llvmbot
Copy link
Member

llvmbot commented Jan 7, 2025

@llvm/issue-subscribers-libc

Author: Nick Desaulniers (nickdesaulniers)

If you go to https://pubs.opengroup.org/onlinepubs/9799919799/, then `14. Headers` in the bottom left, you get a list of headers mandated by POSIX. I'd like to document them from https://libc.llvm.org/headers/index.html. We do so by manually curating some metadata about functions and preprocessor defines in yaml files. The yaml files are currently in https://github.com/llvm/llvm-project/tree/main/libc/utils/docgen. https://github.com/llvm/llvm-project/blob/main/libc/utils/docgen/arpa/inet.yaml is an example of arpa/inet.h as per POSIX. POSIX also tries to be a superset of C (though FWICT POSIX1.2024 missed C23 additions...), so most headers we have entries for already (though we should triple check since POSIX can add functions to existing C standard headers).

Once we've added entries for the new POSIX headers, you should configure your build with the 2 additional cmake flags:

  1. -DLLVM_ENABLE_SPHINX=ON
  2. -DLIBC_INCLUDE_DOCS=ON

Then you can do ninja docs-libc-html to build html you can view locally in {build_dir}/libc/docs/html/headers/**.html.

Good pull requests should implement one header at a time, and not send a PR that implements documentation for more than one header at a time. If that means that this issue is a "meta" issue for which multiple PRs will link against, so be it.

@llvmbot
Copy link
Member

llvmbot commented Jan 7, 2025

@llvm/issue-subscribers-good-first-issue

Author: Nick Desaulniers (nickdesaulniers)

If you go to https://pubs.opengroup.org/onlinepubs/9799919799/, then `14. Headers` in the bottom left, you get a list of headers mandated by POSIX. I'd like to document them from https://libc.llvm.org/headers/index.html. We do so by manually curating some metadata about functions and preprocessor defines in yaml files. The yaml files are currently in https://github.com/llvm/llvm-project/tree/main/libc/utils/docgen. https://github.com/llvm/llvm-project/blob/main/libc/utils/docgen/arpa/inet.yaml is an example of arpa/inet.h as per POSIX. POSIX also tries to be a superset of C (though FWICT POSIX1.2024 missed C23 additions...), so most headers we have entries for already (though we should triple check since POSIX can add functions to existing C standard headers).

Once we've added entries for the new POSIX headers, you should configure your build with the 2 additional cmake flags:

  1. -DLLVM_ENABLE_SPHINX=ON
  2. -DLIBC_INCLUDE_DOCS=ON

Then you can do ninja docs-libc-html to build html you can view locally in {build_dir}/libc/docs/html/headers/**.html.

Good pull requests should implement one header at a time, and not send a PR that implements documentation for more than one header at a time. If that means that this issue is a "meta" issue for which multiple PRs will link against, so be it.

@StarOne01
Copy link
Contributor

Looks like a good issue to start with, could I work on this?

@StarOne01
Copy link
Contributor

StarOne01 commented Jan 8, 2025

From my understanding, what i want to do is:

  • Check whether all the headers mandated by POSIX can be found in LLVM's reference.
  • If a header is missing form LLVM's reference, create a new yaml with the functions and macros of the header from POSIX reference (also create a entry in index.rs in libc/docs/headers).
  • Check and update the current status of functions and macros of all headers in POSIX.

Now, i got some questions,

  • What's the role of C23 Standard Section in the reference?

@StarOne01
Copy link
Contributor

Please correct me if i misunderstand something, somewhere. Thanks!

@nickdesaulniers
Copy link
Member Author

Looks like a good issue to start with, could I work on this?

Sure! I'll add you as an assignee. Feel free to work through as many as you'd like. Happy to add others to help out if folks are interested, since this is an embarrassingly parallel problem!

Check whether all the headers mandated by POSIX can be found in LLVM's reference.

Yep, each header should have a corresponding .yaml file under libc/utils/docgen/ (or a subdir of that for things like arpa/ sys/ etc).

also create a entry in index.rs in libc/docs/headers

s/index.rs/index.rst/ but yes, good catch! I forgot to mention that in the bug description. Please keep that list alphabetically sorted, too.

What's the role of C23 Standard Section in the reference?

The intent of that field is when a function/macro has a definition as per the ISO C spec, what section should folks look at to understand what that function's specified semantics are.

If a function is POSIX only, and not in ISO C, then the c23 standard section is optional, and may be omitted.

@StarOne01
Copy link
Contributor

StarOne01 commented Jan 9, 2025

Hey @nickdesaulniers , i just made a pull request for including (aio.h) and do i have to create a separate PR for every header ?

@marcury122
Copy link

Hi @nickdesaulniers, I would also like to work on this issue and help contribute, It would be great If you could assign me also on this, @StarOne01 Would like to work together as there are many headers to be listed. Thanks!

@StarOne01
Copy link
Contributor

Hey @marcury122 , let's work on this in parallel, please tell me which header(s) you gonna work on

@marcury122
Copy link

yeah Sure! Before that I have a doubt, any of the two can please let me know @nickdesaulniers @StarOne01,
In any of the already included file, there is a subsection as "c-definition: 7.4.1.1", I want to know how is the 2nd part i.e. 7.4.1.1 is known, because I am unable to locate it on the site listed.

If you could list me the reference, it would be helpful and we can start working on it. Thanks!

@marcury122
Copy link

marcury122 commented Jan 9, 2025

Also do we have to list all the headers from the list of 86 headers on the webpage https://pubs.opengroup.org/onlinepubs/9799919799/ and then 14.headers, that are missing from the libc/utils/docgen folder or we have to list only those which are included on the webpage https://libc.llvm.org/headers/index.html

@StarOne01
Copy link
Contributor

I want to know how is the 2nd part i.e. 7.4.1.1 is known, because I am unable to locate it on the site listed.

I did see a few mentions in the POSIX's reference itself, also I found this, to be helpful or a quick search does the job.

do we have to list all the headers from the list of 86 headers on the webpage and then 14.headers, that are missing from thelibc/utils/docgenfolder

Yes, we would need to list all the headers in the header section of the POSIX reference and check the existing functions and macros for changes.

@marcury122
Copy link

Okay thanks. I will take the headers devctl.h and dlfcn.h

@nickdesaulniers
Copy link
Member Author

It would be great If you could assign me also

Done, thanks for volunteering!

I want to know how is the 2nd part i.e. 7.4.1.1 is known, because I am unable to locate it on the site listed.

Bookmark this page: https://www.open-std.org/jtc1/sc22/wg14/www/projects#9899

The latest C standard draft is what I would consider to be the most up to date specification of the C programming language. Note: the latest standard draft is updated somewhat frequently these days, which is why I recommend bookmarking the page I did; it is updated periodically with new drafts.

Ah...7.4.1.1 is a mistake I made in ctype. It should be 7.4.2.1 (etc). Want to send a patch to fix that?

all the headers from the list of 86 headers

Ideally, yes please.

I will take the headers devctl.h and dlfcn.h

Go for it! Thanks again for volunteering.

@nickdesaulniers
Copy link
Member Author

I happen to have a local patch for pthread, so I'll finish that off and get it posted.

@StarOne01
Copy link
Contributor

Got it!

nickdesaulniers pushed a commit that referenced this issue Jan 10, 2025
#122219)

With reference to #122006 , add a new header reference (aio.yaml) to doc
nickdesaulniers pushed a commit that referenced this issue Jan 10, 2025
…X standards (#122411)

This pull request introduces the following changes to the project with
reference to ( #122006 ):

1. **Documentation Update**:
- Added a new YAML file `in.yaml` to document network protocol and
address macros.
   - The `in.yaml` file includes the following macros:
     - `IPPROTO_IP`
     - `IPPROTO_IPV6`
     - `IPPROTO_ICMP`
     - `IPPROTO_RAW`
     - `IPPROTO_TCP`
     - `IPPROTO_UDP`
     - `INADDR_ANY`
     - `INADDR_BROADCAST`
     - `INET_ADDRSTRLEN`
     - `IPV6_JOIN_GROUP`
     - `IPV6_LEAVE_GROUP`
     - `IPV6_MULTICAST_HOPS`
     - `IPV6_MULTICAST_IF`
     - `IPV6_MULTICAST_LOOP`
     - `IPV6_UNICAST_HOPS`
     - `IPV6_V6ONLY`
     - `IN6_IS_ADDR_UNSPECIFIED`
     - `IN6_IS_ADDR_LOOPBACK`
     - `IN6_IS_ADDR_MULTICAST`
     - `IN6_IS_ADDR_LINKLOCAL`
     - `IN6_IS_ADDR_SITELOCAL`
     - `IN6_IS_ADDR_V4MAPPED`
     - `IN6_IS_ADDR_V4COMPAT`
     - `IN6_IS_ADDR_MC_NODELOCAL`
     - `IN6_IS_ADDR_MC_LINKLOCAL`
     - `IN6_IS_ADDR_MC_SITELOCAL`
     - `IN6_IS_ADDR_MC_ORGLOCAL`
     - `IN6_IS_ADDR_MC_GLOBAL`

_I believe, all these macros are necessary and should be documented._

2. **CMake Configuration Update**:
- Updated the `CMakeLists.txt` file to create the necessary directory
for the `netinet` headers.
- Included the `netinet/in` documentation in the Sphinx build
configuration.

3. **Index Update**:
- Updated the `index.rst` file to include a reference to the newly added
`netinet/in` documentation.

**Purpose**:
- This pull request adds documentation for network protocol and address
macros in the `netinet/in` header.
- Updates the CMake configuration to support the new documentation.

**Testing**:
- Verified that the new YAML file is correctly referenced in the
`index.rst`.
- Ensured that the documentation builds without errors and includes the
new network interface documentation.

This pull request ensures that the `netinet/in` header macros are
documented and included in the project's documentation, and updates the
CMake configuration to support these changes.
nickdesaulniers added a commit that referenced this issue Jan 10, 2025
…ds (#122406)

This pull request introduces the following changes to the project with
reference to issue ( #122006 ):

1. **Documentation Update**:
- Added a new YAML file `if.yaml` under `net` to document network
interface functions and macros.
   - The `if.yaml` file includes the following functions and macros:
     - Functions:
       - `if_freenameindex`
       - `if_indextoname`
       - `if_nameindex`
       - `if_nametoindex`
     - Macros:
       - `IF_NAMESIZE`

2. **CMake Configuration Update**:
- Updated the `CMakeLists.txt` file to create the necessary directory
for the `net` headers.
- Included the `net/if` documentation in the Sphinx build configuration.

3. **Index Update**:
- Updated the `index.rst` file to include a reference to the newly added
`net/if` documentation.

**Purpose**:
- This pull request adds documentation for network interface functions
and macros, ensuring they are included in the project's documentation.
- Updates the CMake configuration to support the new documentation.

**Testing**:
- Verified that the new YAML file is correctly referenced in the
`index.rst`.
- Ensured that the documentation builds without errors and includes the
new network interface documentation.

Co-authored-by: Nick Desaulniers <ndesaulniers@google.com>
nickdesaulniers pushed a commit that referenced this issue Jan 17, 2025
…123000)

These changes ensure that the sys/time header is documented properly
with respect to the issue ( #122006 ) .
nickdesaulniers pushed a commit that referenced this issue Jan 17, 2025
…123004)

These changes ensure that the termios header is documented properly with
respect to the issue ( #122006 ) .
github-actions bot pushed a commit to arm/arm-toolchain that referenced this issue Jan 17, 2025
…ions docs (#123004)

These changes ensure that the termios header is documented properly with
respect to the issue ( llvm/llvm-project#122006 ) .
nickdesaulniers pushed a commit that referenced this issue Jan 17, 2025
…122997)

These changes ensure that the sys/stat header is documented properly
with respect to the issue ( #122006 ) .
nickdesaulniers added a commit to nickdesaulniers/llvm-project that referenced this issue Jan 21, 2025
Having a target named pthreads is breaking when multiple runtimes are enabled.
Disable this target for now so that the builds go back to green (and sites get
updated).

Link: llvm/llvm-zorg#359 (comment)
Link: llvm#122006
Link: llvm#122497
Link: llvm#123821
nickdesaulniers added a commit that referenced this issue Jan 22, 2025
Having a target named pthreads is breaking when multiple runtimes are enabled.
Disable this target for now so that the builds go back to green (and sites get
updated).

Link: llvm/llvm-zorg#359 (comment)
Link: #122006
Link: #122497
Link: #123821
nickdesaulniers pushed a commit that referenced this issue Jan 23, 2025
…ns doc (#123068)

These changes ensure that the unistd header is documented properly with
respect to the issue ( #122006 ) .
github-actions bot pushed a commit to arm/arm-toolchain that referenced this issue Jan 23, 2025
…lementations doc (#123068)

These changes ensure that the unistd header is documented properly with
respect to the issue ( llvm/llvm-project#122006 ) .
nickdesaulniers pushed a commit that referenced this issue Jan 24, 2025
[libc][docs] add cpio to documentation and include related functi… 
These changes ensure that the cpio header is documented properly
with respect to the issue
(#122006 ).

**Changes:**
1. **cpio.yaml**: Created a new YAML file for cpio with functions
and related macros.
2. **CMakeLists.txt**: Added cpio to the documentation
directories.
3. **index.rst**: Included `cpio` in the documentation index.

---------

Co-authored-by: siya <siya@Siya.com>
github-actions bot pushed a commit to arm/arm-toolchain that referenced this issue Jan 24, 2025
[libc][docs] add cpio to documentation and include related functi…
These changes ensure that the cpio header is documented properly
with respect to the issue
(llvm/llvm-project#122006 ).

**Changes:**
1. **cpio.yaml**: Created a new YAML file for cpio with functions
and related macros.
2. **CMakeLists.txt**: Added cpio to the documentation
directories.
3. **index.rst**: Included `cpio` in the documentation index.

---------

Co-authored-by: siya <siya@Siya.com>
bherrera pushed a commit to misttech/mist-os that referenced this issue Feb 4, 2025
… (#123798)

[libc][docs] add cpio to documentation and include related functi…
These changes ensure that the cpio header is documented properly
with respect to the issue
(llvm/llvm-project#122006 ).

**Changes:**
1. **cpio.yaml**: Created a new YAML file for cpio with functions
and related macros.
2. **CMakeLists.txt**: Added cpio to the documentation
directories.
3. **index.rst**: Included `cpio` in the documentation index.

---------

Co-authored-by: siya <siya@Siya.com>
GitOrigin-RevId: f116ea0580ab60a8d876a8e69d39e73ea3210218
Original-Revision: 4ca5d779a5a794c3a7e08e282f2822d35fd70666
Roller-URL: https://cr-buildbucket.appspot.com/build/8724818472990707457
CQ-Do-Not-Cancel-Tryjobs: true
Change-Id: I83e3d5b2390c1c6e394755dd14121c51300a165d
Reviewed-on: https://fuchsia-review.googlesource.com/c/fuchsia/+/1192690
bherrera pushed a commit to misttech/integration that referenced this issue Feb 4, 2025
[libc][docs] add cpio to documentation and include related functi…
These changes ensure that the cpio header is documented properly
with respect to the issue
(llvm/llvm-project#122006 ).

**Changes:**
1. **cpio.yaml**: Created a new YAML file for cpio with functions
and related macros.
2. **CMakeLists.txt**: Added cpio to the documentation
directories.
3. **index.rst**: Included `cpio` in the documentation index.

---------

Co-authored-by: siya <siya@Siya.com>
GitOrigin-RevId: f116ea0580ab60a8d876a8e69d39e73ea3210218
Original-Revision: 4ca5d779a5a794c3a7e08e282f2822d35fd70666
Change-Id: I140acb3c170cffbe111b450291f66699b9c64629
bherrera pushed a commit to misttech/integration that referenced this issue Feb 4, 2025
…] Add cpio.h header. (#123798)

[libc][docs] add cpio to documentation and include related functi…
These changes ensure that the cpio header is documented properly
with respect to the issue
(llvm/llvm-project#122006 ).

**Changes:**
1. **cpio.yaml**: Created a new YAML file for cpio with functions
and related macros.
2. **CMakeLists.txt**: Added cpio to the documentation
directories.
3. **index.rst**: Included `cpio` in the documentation index.

---------

Co-authored-by: siya <siya@Siya.com>
GitOrigin-RevId: 2636ca898412185939d24f825da3d94d9a13457a
Original-Revision: 4ca5d779a5a794c3a7e08e282f2822d35fd70666
Original-Reviewed-on: https://fuchsia-review.googlesource.com/c/fuchsia/+/1192690
Original-Revision: 997a3198a33fae6dd8d95b1015b13ba931b687cb
Change-Id: I34d3667c9b54e5587b3bf6beb2653d564c47fe51
nickdesaulniers pushed a commit that referenced this issue Feb 10, 2025
…126413)

These changes ensure that the sys/statvfs header is documented properly
with respect to the issue (
#122006 ) .
github-actions bot pushed a commit to arm/arm-toolchain that referenced this issue Feb 10, 2025
…initions (#126413)

These changes ensure that the sys/statvfs header is documented properly
with respect to the issue (
llvm/llvm-project#122006 ) .
bherrera pushed a commit to misttech/integration that referenced this issue Feb 11, 2025
…tation and YAML definitions (#126413)

These changes ensure that the sys/statvfs header is documented properly
with respect to the issue (
llvm/llvm-project#122006 ) .

GitOrigin-RevId: 83ead16376d0bb037abb4ce42597a48bc9d84c73
Original-Revision: 83d663f0989231081d7bc5ed47c0c548d1d9fb8c
Change-Id: Ib3d3f696ba3faa83af6b639e722a4200ac800fe3
bherrera pushed a commit to misttech/integration that referenced this issue Feb 11, 2025
…] Add sys/statvfs to documentation and YAML definitions (#126413)

These changes ensure that the sys/statvfs header is documented properly
with respect to the issue (
llvm/llvm-project#122006 ) .

GitOrigin-RevId: 275b054614a89bf6a651168b5465a1c086ab7cb5
Original-Revision: 83d663f0989231081d7bc5ed47c0c548d1d9fb8c
Original-Reviewed-on: https://fuchsia-review.googlesource.com/c/fuchsia/+/1203484
Original-Revision: f03ff8d8014dd37fce378ecad67808fad744cd28
Change-Id: Ie8d1cac6bddfd0efe8e80519eccbce1bd6ab1036
bherrera pushed a commit to misttech/mist-os that referenced this issue Feb 11, 2025
… documentation and YAML definitions (#126413)

These changes ensure that the sys/statvfs header is documented properly
with respect to the issue (
llvm/llvm-project#122006 ) .

GitOrigin-RevId: 83ead16376d0bb037abb4ce42597a48bc9d84c73
Original-Revision: 83d663f0989231081d7bc5ed47c0c548d1d9fb8c
Roller-URL: https://cr-buildbucket.appspot.com/build/8723293557253046913
CQ-Do-Not-Cancel-Tryjobs: true
Change-Id: I6c55b8adbb1d1f6d3b797da06494f90293a1e064
Reviewed-on: https://fuchsia-review.googlesource.com/c/fuchsia/+/1203484
@StarOne01
Copy link
Contributor

StarOne01 commented Feb 11, 2025

With reference to this issue, how do we figure out the removed ones ?

@nickdesaulniers
Copy link
Member Author

They appear in older versions of the POSIX docs, but I think it's out of scope for this issue to track down such removed APIs.

@StarOne01
Copy link
Contributor

They appear in older versions of the POSIX docs, but I think it's out of scope for this issue to track down such removed APIs.

I could help with that, if that's necessary.

Icohedron pushed a commit to Icohedron/llvm-project that referenced this issue Feb 11, 2025
…lvm#126413)

These changes ensure that the sys/statvfs header is documented properly
with respect to the issue (
llvm#122006 ) .
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation good first issue https://github.com/llvm/llvm-project/contribute libc
Projects
None yet
Development

No branches or pull requests

5 participants