Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(nix): Startup/configure apache for renegotiate test under nix #4592

Merged
merged 29 commits into from
Aug 15, 2024
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
cd8f596
apache2 startup under nix functional
dougch Jun 10, 2024
f69b10b
add an httpd nix store variable
dougch Jun 10, 2024
c78a311
final TODOs added
dougch Jun 10, 2024
58dad41
additonal callout
dougch Jun 10, 2024
da4261c
missing nix utils
dougch Jun 11, 2024
afca602
missing directory
dougch Jun 11, 2024
7c486cc
Backup the Ubuntu apache configs, modified to work with nixpkgs httpd
dougch Jun 13, 2024
e09b8ff
split out the site from the base apache config for nix
dougch Jun 13, 2024
052152a
Fix hardcoded module configs and handle distro specific group
dougch Jun 13, 2024
9c6baea
missing dir
dougch Jun 13, 2024
09940cb
remove nix hard coded paths
dougch Jun 13, 2024
5549e13
fix nix file paths
dougch Jun 13, 2024
6ad3101
Update nix/shell.sh
dougch Jun 17, 2024
846751c
Update nix/shell.sh
dougch Jun 17, 2024
5c82928
Update nix/shell.sh
dougch Jun 17, 2024
8933245
Reverting changes to existing integ apache files.
dougch Jun 17, 2024
bfc80e8
pr feedback
dougch Jun 18, 2024
0f32b33
Update nix/shell.sh
dougch Jun 19, 2024
a159c2a
Refactor configs
dougch Jul 25, 2024
aae0495
Add apache modules; move files around
dougch Jul 29, 2024
bb85900
consolidated apache config
dougch Jul 29, 2024
993f670
remove dup/unused sites file
dougch Jul 29, 2024
0e187d6
Merge branch 'main' into nix_apache2
dougch Jul 30, 2024
6e0e3f7
Update nix/shell.sh
dougch Aug 1, 2024
fd8406d
Reduce langs/charsets
dougch Aug 6, 2024
463ac52
Removing more from apache config
dougch Aug 14, 2024
4a7a170
Merge branch 'main' into nix_apache2
dougch Aug 14, 2024
79c09a9
Merge branch 'main' into nix_apache2
dougch Aug 15, 2024
dc29f51
Merge branch 'main' into nix_apache2
dougch Aug 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions flake.nix
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
corretto
pkgs.iproute2
pkgs.apacheHttpd
pkgs.procps
# GnuTLS-cli and serv utilities needed for some integration tests.
pkgs.gnutls
pkgs.gdb
Expand Down
44 changes: 34 additions & 10 deletions nix/shell.sh
Original file line number Diff line number Diff line change
Expand Up @@ -62,18 +62,11 @@ function unit {
}

function integ {
if [ "$1" == "help" ]; then
echo "The following tests are not supported:"
echo "- renegotiate_apache"
echo " This test requires apache to be running. See codebuild/bin/s2n_apache.sh"
echo " for more info."
return
fi
apache2_start
if [[ -z "$1" ]]; then
lrstewart marked this conversation as resolved.
Show resolved Hide resolved
banner "Running all integ tests except renegotiate_apache."
(cd $SRC_ROOT/build; ctest -L integrationv2 -E "(integrationv2_cross_compatibility|integrationv2_renegotiate_apache)" --verbose)
banner "Running all integ tests."
(cd $SRC_ROOT/build; ctest -L integrationv2 --verbose)
else
banner "Warning: renegotiate_apache is not supported in nix for various reasons integ help for more info."
for test in $@; do
ctest --test-dir ./build -L integrationv2 --no-tests=error --output-on-failure -R "$test" --verbose
if [ "$?" -ne 0 ]; then
Expand Down Expand Up @@ -159,3 +152,34 @@ function test_nonstandard_compilation {
./codebuild/bin/test_dynamic_load.sh $(mktemp -d)
}

function apache2_config(){
export APACHE_NIX_STORE=$(dirname $(dirname $(which httpd)))
export APACHE2_INSTALL_DIR=/usr/local/apache2
export APACHE_SERVER_ROOT="$APACHE2_INSTALL_DIR"
export APACHE_RUN_USER=nobody
# Unprivileged groupname differs
export APACHE_RUN_GROUP=$(awk 'BEGIN{FS=":"} /65534/{print $1}' /etc/group)
export APACHE_PID_FILE="${APACHE2_INSTALL_DIR}/run/apache2.pid"
export APACHE_RUN_DIR="${APACHE2_INSTALL_DIR}/run"
export APACHE_LOCK_DIR="${APACHE2_INSTALL_DIR}/lock"
export APACHE_LOG_DIR="${APACHE2_INSTALL_DIR}/log"
export APACHE_CERT_DIR="$SRC_ROOT/tests/pems"
}

function apache2_start(){
if [[ "$(pgrep -c httpd)" -eq "0" ]]; then
apache2_config
if [[ ! -f "$APACHE2_INSTALL_DIR/conf/apache2.conf" ]]; then
mkdir -p $APACHE2_INSTALL_DIR/{run,log,lock}
# NixOs specific base apache config
cp -R ./tests/integrationv2/apache2/nix/* $APACHE2_INSTALL_DIR
goatgoose marked this conversation as resolved.
Show resolved Hide resolved
# Integrationv2::renegotiate site
cp -R ./codebuild/bin/apache2/{www,sites-enabled} $APACHE2_INSTALL_DIR
fi
httpd -k start -f "${APACHE2_INSTALL_DIR}/conf/apache2.conf"
trap 'pkill httpd' ERR EXIT
else
echo "Apache is already running...and if \"$APACHE2_INSTALL_DIR\" is stale, it might be in an unknown state."
fi

}
254 changes: 254 additions & 0 deletions tests/integrationv2/apache2/nix/conf/apache2.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,254 @@
# Consolidated apache2.conf, mods-enabled/* and conf-enabled/*.
ServerRoot ${APACHE_SERVER_ROOT}
DefaultRuntimeDir ${APACHE_RUN_DIR}
PidFile ${APACHE_PID_FILE}
Timeout 60
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5
HostnameLookups Off
ErrorLog ${APACHE_LOG_DIR}/error.log
LogLevel info

# Inline load module
LoadModule access_compat_module ${APACHE_NIX_STORE}/modules/mod_access_compat.so
LoadModule alias_module ${APACHE_NIX_STORE}/modules/mod_alias.so
LoadModule auth_basic_module ${APACHE_NIX_STORE}/modules/mod_auth_basic.so
LoadModule authn_core_module ${APACHE_NIX_STORE}/modules/mod_authn_core.so
LoadModule authn_file_module ${APACHE_NIX_STORE}/modules/mod_authn_file.so
LoadModule authz_core_module ${APACHE_NIX_STORE}/modules/mod_authz_core.so
LoadModule authz_host_module ${APACHE_NIX_STORE}/modules/mod_authz_host.so
LoadModule authz_user_module ${APACHE_NIX_STORE}/modules/mod_authz_user.so
LoadModule autoindex_module ${APACHE_NIX_STORE}/modules/mod_autoindex.so
LoadModule dir_module ${APACHE_NIX_STORE}/modules/mod_dir.so
LoadModule deflate_module ${APACHE_NIX_STORE}/modules/mod_deflate.so
LoadModule env_module ${APACHE_NIX_STORE}/modules/mod_env.so
LoadModule filter_module ${APACHE_NIX_STORE}/modules/mod_filter.so
LoadModule log_config_module ${APACHE_NIX_STORE}/modules/mod_log_config.so
LoadModule mime_module ${APACHE_NIX_STORE}/modules/mod_mime.so
LoadModule mpm_event_module ${APACHE_NIX_STORE}/modules/mod_mpm_event.so
LoadModule reqtimeout_module ${APACHE_NIX_STORE}/modules/mod_reqtimeout.so
LoadModule rewrite_module ${APACHE_NIX_STORE}/modules/mod_rewrite.so
LoadModule setenvif_module ${APACHE_NIX_STORE}/modules/mod_setenvif.so
LoadModule socache_shmcb_module ${APACHE_NIX_STORE}/modules/mod_socache_shmcb.so
LoadModule ssl_module ${APACHE_NIX_STORE}/modules/mod_ssl.so
LoadModule status_module ${APACHE_NIX_STORE}/modules/mod_status.so
LoadModule unixd_module ${APACHE_NIX_STORE}/modules/mod_unixd.so

# Include list of ports to listen on
Include conf/ports.conf

<IfModule unixd_module>
User ${APACHE_RUN_USER}
Group ${APACHE_RUN_GROUP}
</IfModule>

DocumentRoot ${APACHE_SERVER_ROOT}/www/html
<Directory />
Options FollowSymLinks
AllowOverride None
Require all denied
</Directory>

<Directory ${APACHE_SERVER_ROOT}/www/html>
Options Indexes FollowSymLinks
AllowOverride None
Require all granted
</Directory>

AccessFileName .htaccess

LogFormat "%v:%p %h %l %u %t \"%r\" %>s \"%{Referer}i\" \"%{User-Agent}i\"" vhost_combined
LogFormat "%h %l %u %t \"%r\" %>s \"%{Referer}i\" \"%{User-Agent}i\"" combined
LogFormat "%h %l %u %t \"%r\" %>s " common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
CustomLog ${APACHE_LOG_DIR}/other_vhosts_access.log vhost_combined

# Include the virtual host configurations:
IncludeOptional sites-enabled/*.conf

# inline module configs
<IfModule mod_mime.c>
TypesConfig ${APACHE_NIX_STORE}/conf/mime.types
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddType application/x-bzip2 .bz2
AddLanguage am .amh
AddLanguage ar .ara
lrstewart marked this conversation as resolved.
Show resolved Hide resolved
AddLanguage be .be
AddLanguage bg .bg
AddLanguage bn .bn
AddLanguage br .br
AddLanguage bs .bs
AddLanguage ca .ca
AddLanguage cs .cz .cs
AddLanguage cy .cy
AddLanguage da .da
AddLanguage da .dk
AddLanguage de .de
AddLanguage dz .dz
AddLanguage el .el
AddLanguage en .en
AddLanguage eo .eo
# es is ecmascript in /etc/mime.types
RemoveType es
AddLanguage es .es
AddLanguage et .et
AddLanguage eu .eu
AddLanguage fa .fa
AddLanguage fi .fi
AddLanguage fr .fr
AddLanguage ga .ga
AddLanguage gl .glg
AddLanguage gu .gu
AddLanguage he .he
AddLanguage hi .hi
AddLanguage hr .hr
AddLanguage hu .hu
AddLanguage hy .hy
AddLanguage id .id
AddLanguage is .is
AddLanguage it .it
AddLanguage ja .ja
AddLanguage ka .ka
AddLanguage kk .kk
AddLanguage km .km
AddLanguage kn .kn
AddLanguage ko .ko
AddLanguage ku .ku
AddLanguage lo .lo
AddLanguage lt .lt
AddLanguage ltz .ltz
AddLanguage lv .lv
AddLanguage mg .mg
AddLanguage mk .mk
AddLanguage ml .ml
AddLanguage mr .mr
AddLanguage ms .msa
AddLanguage nb .nob
AddLanguage ne .ne
AddLanguage nl .nl
AddLanguage nn .nn
AddLanguage no .no
AddLanguage pa .pa
AddLanguage pl .po
AddLanguage pt-BR .pt-br
AddLanguage pt .pt
AddLanguage ro .ro
AddLanguage ru .ru
AddLanguage sa .sa
AddLanguage se .se
AddLanguage si .si
AddLanguage sk .sk
AddLanguage sl .sl
AddLanguage sq .sq
AddLanguage sr .sr
AddLanguage sv .sv
AddLanguage ta .ta
AddLanguage te .te
AddLanguage th .th
AddLanguage tl .tl
RemoveType tr
# tr is troff in /etc/mime.types
AddLanguage tr .tr
AddLanguage uk .uk
AddLanguage ur .ur
AddLanguage vi .vi
AddLanguage wo .wo
AddLanguage xh .xh
AddLanguage zh-CN .zh-cn
AddLanguage zh-TW .zh-tw
AddCharset us-ascii .ascii .us-ascii
AddCharset ISO-8859-1 .iso8859-1 .latin1
AddCharset ISO-8859-2 .iso8859-2 .latin2 .cen
AddCharset ISO-8859-3 .iso8859-3 .latin3
AddCharset ISO-8859-4 .iso8859-4 .latin4
AddCharset ISO-8859-5 .iso8859-5 .cyr .iso-ru
AddCharset ISO-8859-6 .iso8859-6 .arb .arabic
AddCharset ISO-8859-7 .iso8859-7 .grk .greek
AddCharset ISO-8859-8 .iso8859-8 .heb .hebrew
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea what most of these character sets are, but since we now only support english (and spanish?), why include the obviously non-english ones?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trimmed it down some more.

AddCharset ISO-8859-9 .iso8859-9 .latin5 .trk
AddCharset ISO-8859-10 .iso8859-10 .latin6
AddCharset ISO-8859-13 .iso8859-13
AddCharset ISO-8859-14 .iso8859-14 .latin8
AddCharset ISO-8859-15 .iso8859-15 .latin9
AddCharset ISO-8859-16 .iso8859-16 .latin10
AddCharset ISO-2022-JP .iso2022-jp .jis
AddCharset ISO-2022-KR .iso2022-kr .kis
AddCharset ISO-2022-CN .iso2022-cn .cis
AddCharset Big5 .Big5 .big5 .b5
AddCharset cn-Big5 .cn-big5
# For russian, more than one charset is used (depends on client, mostly):
AddCharset WINDOWS-1251 .cp-1251 .win-1251
AddCharset CP866 .cp866
AddCharset KOI8 .koi8
AddCharset KOI8-E .koi8-e
AddCharset KOI8-r .koi8-r .koi8-ru
AddCharset KOI8-U .koi8-u
AddCharset KOI8-ru .koi8-uk .ua
AddCharset ISO-10646-UCS-2 .ucs2
AddCharset ISO-10646-UCS-4 .ucs4
AddCharset UTF-7 .utf7
AddCharset UTF-8 .utf8
AddCharset UTF-16 .utf16
AddCharset UTF-16BE .utf16be
AddCharset UTF-16LE .utf16le
AddCharset UTF-32 .utf32
AddCharset UTF-32BE .utf32be
AddCharset UTF-32LE .utf32le
AddCharset euc-cn .euc-cn
AddCharset euc-gb .euc-gb
AddCharset euc-jp .euc-jp
AddCharset euc-kr .euc-kr
#Not sure how euc-tw got in - IANA doesn't list it???
AddCharset EUC-TW .euc-tw
AddCharset gb2312 .gb2312 .gb
AddCharset iso-10646-ucs-2 .ucs-2 .iso-10646-ucs-2
AddCharset iso-10646-ucs-4 .ucs-4 .iso-10646-ucs-4
AddCharset shift_jis .shift_jis .sjis
AddCharset BRF .brf

AddHandler type-map var
AddType text/html .shtml
<IfModule mod_include.c>
AddOutputFilter INCLUDES .shtml
</IfModule>

</IfModule>

<IfModule mod_ssl.c>
SSLRandomSeed startup builtin
SSLRandomSeed startup file:/dev/urandom 512
SSLRandomSeed connect builtin
SSLRandomSeed connect file:/dev/urandom 512
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl .crl
SSLSessionCache shmcb:${APACHE_RUN_DIR}/ssl_scache(512000)
SSLSessionCacheTimeout 300
SSLCipherSuite HIGH:!aNULL
SSLProtocol all -SSLv3
Comment on lines +121 to +122
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this configuration needed? How does it interact with the site-specific configuration?

SSLProtocol -ALL +TLSv1.2
SSLHonorCipherOrder On
SSLCipherSuite HIGH:!aNULL:!MD5
SSLCompression Off
SSLInsecureRenegotiation Off

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should only apply to the *:443/ default site

</IfModule>

<IfModule mod_negotiation.c>
LanguagePriority en ca cs da de el eo es et fr he hr it ja ko ltz nl nn no pl pt pt-BR ru sv tr zh-CN zh-TW
ForceLanguagePriority Prefer Fallback
</IfModule>
<IfModule mpm_event_module>
StartServers 2
MinSpareThreads 25
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxRequestWorkers 150
MaxConnectionsPerChild 0
</IfModule>
<IfModule reqtimeout_module>
RequestReadTimeout header=20-40,minrate=500
RequestReadTimeout body=10,minrate=500
</IfModule>

# === end module configs


# vim: syntax=apache ts=4 sw=4 sts=4 sr noet
Loading
Loading