Try to fix some issues with slowness in the MacOS build. #4280

fruffy · 2023-12-08T17:09:39Z

The MacOS CI build was fairly out-of-date, which caused slowness.

Previously dependencies took 42 minutes:
https://github.com/p4lang/p4c/actions/runs/7143318180/job/19454526195?pr=4278

Now they are taking ~1-2 minutes:
https://github.com/p4lang/p4c/actions/runs/7144321547/job/19457683307?pr=4280

vlstill

I don't see any obvious problems here, but since I have no experience with macOS and brew I don't think I should approve this (i.e. I don't know why brew link is needed for boost).

Can we safely build on macOS 12, or can there still be users on 11? (I'm not sure how OS versions and updates work for macOS :-D).

jnfoster

Getting an error about std::binary_function in json.h. It seems this is now deprecated.

/Users/nate/shared/p4c/lib/ordered_map.h:51:32: error: no template named 'binary_function' in namespace 'std'; did you mean '__binary_function'?
    class value_compare : std::binary_function<value_type, value_type, bool> {
                          ~~~~~^~~~~~~~~~~~~~~
                               __binary_function
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/include/c++/v1/__functional/binary_function.h:49:1: note: '__binary_function' declared here
using __binary_function = __binary_function_keep_layout_base<_Arg1, _Arg2, _Result>;
^
In file included from /Users/nate/shared/p4c/lib/json.cpp:17:
In file included from /Users/nate/shared/p4c/lib/json.h:31:
/Users/nate/shared/p4c/lib/ordered_map.h:65:26: error: no template named 'binary_function' in namespace 'std'; did you mean '__binary_function'?
    struct mapcmp : std::binary_function<const K *, const K *, bool> {
                    ~~~~~^~~~~~~~~~~~~~~
                         __binary_function
/Library/Developer/CommandLineTools/SDKs/MacOSX14.0.sdk/usr/include/c++/v1/__functional/binary_function.h:49:1: note: '__binary_function' declared here
using __binary_function = __binary_function_keep_layout_base<_Arg1, _Arg2, _Result>;
^
2 errors generated.

I'm on an M1 Macbook running OS X 14 (Sonoma), with this C++ compiler.

The CXX compiler identification is AppleClang 15.0.0.15000040

fruffy · 2023-12-11T13:25:17Z

Getting an error about std::binary_function in json.h. It seems this is now deprecated.

This is apparently a deprecated function which has been removed in C++17. The common advice is to just remove it. A little strange that the other compilers do not complain.

#4283

vlstill · 2023-12-11T13:47:03Z

This is apparently a deprecated function which has been removed in C++17. The common advice is to just remove it. A little strange that the other compilers do not complain.

Could be libstdc++ ABI would break if they remove it, it is still widely used in the standard library itself (as of GCC 11). Apple might care a bit less about ABI then the open-source world.

jafingerhut · 2023-12-11T14:49:50Z

@fruffy I have never used the macOS build of any open source P4 dev tools before, despite using a Mac for personal use -- I've only ever compiled it in a Linux VM.

fruffy · 2023-12-11T15:25:32Z

Can we safely build on macOS 12, or can there still be users on 11? (I'm not sure how OS versions and updates work for macOS :-D).

Me neither, but Homebrew was warning about the usage of MacOS 11. considering it deprecated.

Adding @rst0git because I believe he used the MacOS build in the past.

rst0git

LGTM

jnfoster · 2023-12-11T21:41:48Z

I still can't build on my Mac. Seems likely there are a bunch of other binary_function instances in the codebase...

For example: https://github.com/p4lang/p4c/blob/fruffy/macos_ci/lib/ordered_set.h#L56

fruffy · 2023-12-11T21:49:45Z

I still can't build on my Mac. Seems likely there are a bunch of other binary_function instances in the codebase...

For example: https://github.com/p4lang/p4c/blob/fruffy/macos_ci/lib/ordered_set.h#L56

Should have been more thorough... will try to fix this in this PR directly.

jnfoster · 2023-12-11T21:53:51Z

Seems to be working now... will let you know when it completes.

jnfoster · 2023-12-11T21:57:22Z

Worked for most backends.

Graphs had this error:

/opt/homebrew/include/boost/multi_index/detail/bucket_array.hpp:239:19: error: use of undeclared identifier 'bad_archive_exception'; did you mean 'multi_index::detail::bad_archive_exception'?
  throw_exception(bad_archive_exception());
                  ^
/opt/homebrew/include/boost/multi_index/detail/bad_archive_exception.hpp:25:8: note: 'multi_index::detail::bad_archive_exception' declared here
struct bad_archive_exception:std::runtime_error
       ^
In file included from /Users/nate/shared/p4c/backends/graphs/graphs.cpp:17:
In file included from /Users/nate/shared/p4c/backends/graphs/graphs.h:33:
In file included from /opt/homebrew/include/boost/graph/adjacency_list.hpp:36:
In file included from /opt/homebrew/include/boost/graph/named_graph.hpp:18:
In file included from /opt/homebrew/include/boost/multi_index/hashed_index.hpp:35:
/opt/homebrew/include/boost/multi_index/detail/index_node_base.hpp:120:19: error: use of undeclared identifier 'bad_archive_exception'; did you mean 'multi_index::detail::bad_archive_exception'?
  throw_exception(bad_archive_exception());
                  ^
/opt/homebrew/include/boost/multi_index/detail/bad_archive_exception.hpp:25:8: note: 'multi_index::detail::bad_archive_exception' declared here
struct bad_archive_exception:std::runtime_error
       ^
2 errors generated.
make[2]: *** [backends/graphs/CMakeFiles/p4cgraphs.dir/graphs.cpp.o] Error 1
make[1]: *** [backends/graphs/CMakeFiles/p4cgraphs.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

fruffy · 2023-12-11T22:03:34Z

Worked for most backends.

Graphs had this error:

/opt/homebrew/include/boost/multi_index/detail/bucket_array.hpp:239:19: error: use of undeclared identifier 'bad_archive_exception'; did you mean 'multi_index::detail::bad_archive_exception'?
  throw_exception(bad_archive_exception());
                  ^
/opt/homebrew/include/boost/multi_index/detail/bad_archive_exception.hpp:25:8: note: 'multi_index::detail::bad_archive_exception' declared here
struct bad_archive_exception:std::runtime_error
       ^
In file included from /Users/nate/shared/p4c/backends/graphs/graphs.cpp:17:
In file included from /Users/nate/shared/p4c/backends/graphs/graphs.h:33:
In file included from /opt/homebrew/include/boost/graph/adjacency_list.hpp:36:
In file included from /opt/homebrew/include/boost/graph/named_graph.hpp:18:
In file included from /opt/homebrew/include/boost/multi_index/hashed_index.hpp:35:
/opt/homebrew/include/boost/multi_index/detail/index_node_base.hpp:120:19: error: use of undeclared identifier 'bad_archive_exception'; did you mean 'multi_index::detail::bad_archive_exception'?
  throw_exception(bad_archive_exception());
                  ^
/opt/homebrew/include/boost/multi_index/detail/bad_archive_exception.hpp:25:8: note: 'multi_index::detail::bad_archive_exception' declared here
struct bad_archive_exception:std::runtime_error
       ^
2 errors generated.
make[2]: *** [backends/graphs/CMakeFiles/p4cgraphs.dir/graphs.cpp.o] Error 1
make[1]: *** [backends/graphs/CMakeFiles/p4cgraphs.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

This might be solved by #4147. The corresponding PR has not been reviewed yet.

fruffy force-pushed the fruffy/macos_ci branch from 0410e07 to e5ce2ae Compare December 8, 2023 17:20

fruffy requested review from vlstill, jafingerhut and jnfoster December 8, 2023 20:33

vlstill reviewed Dec 11, 2023

View reviewed changes

jnfoster approved these changes Dec 11, 2023

View reviewed changes

jnfoster self-requested a review December 11, 2023 12:07

fruffy requested a review from rst0git December 11, 2023 15:32

fruffy force-pushed the fruffy/macos_ci branch from e5ce2ae to db0fe21 Compare December 11, 2023 16:38

rst0git approved these changes Dec 11, 2023

View reviewed changes

fruffy added 3 commits December 11, 2023 16:54

Try to fix some issues with slowness in the MacOS build.

4910afb

Remove deprecated bazel arguments which cause CI to fail.

262a06d

Remove another function.

6acaba9

fruffy force-pushed the fruffy/macos_ci branch from f8873bb to 6acaba9 Compare December 11, 2023 21:54

Fix Bazel breakage.

3433648

fruffy merged commit def92ac into main Dec 12, 2023
13 checks passed

fruffy deleted the fruffy/macos_ci branch December 12, 2023 00:00

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Try to fix some issues with slowness in the MacOS build. #4280

Try to fix some issues with slowness in the MacOS build. #4280

fruffy commented Dec 8, 2023 •

edited

Loading

vlstill left a comment

jnfoster left a comment

fruffy commented Dec 11, 2023

vlstill commented Dec 11, 2023

jafingerhut commented Dec 11, 2023

fruffy commented Dec 11, 2023 •

edited

Loading

rst0git left a comment

jnfoster commented Dec 11, 2023

fruffy commented Dec 11, 2023 •

edited

Loading

jnfoster commented Dec 11, 2023

jnfoster commented Dec 11, 2023

fruffy commented Dec 11, 2023

Try to fix some issues with slowness in the MacOS build. #4280

Try to fix some issues with slowness in the MacOS build. #4280

Conversation

fruffy commented Dec 8, 2023 • edited Loading

vlstill left a comment

Choose a reason for hiding this comment

jnfoster left a comment

Choose a reason for hiding this comment

fruffy commented Dec 11, 2023

vlstill commented Dec 11, 2023

jafingerhut commented Dec 11, 2023

fruffy commented Dec 11, 2023 • edited Loading

rst0git left a comment

Choose a reason for hiding this comment

jnfoster commented Dec 11, 2023

fruffy commented Dec 11, 2023 • edited Loading

jnfoster commented Dec 11, 2023

jnfoster commented Dec 11, 2023

fruffy commented Dec 11, 2023

fruffy commented Dec 8, 2023 •

edited

Loading

fruffy commented Dec 11, 2023 •

edited

Loading

fruffy commented Dec 11, 2023 •

edited

Loading