Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAOS-12114 build: remove psm2 dependency #11637

Merged
merged 3 commits into from
Jun 6, 2023
Merged

DAOS-12114 build: remove psm2 dependency #11637

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
4319a0e
DAOS-12114 build: remove psm2 dependency
soumagne May 17, 2023
5a80380
DAOS-12114 build: remove doc section on optional components (psm2)
soumagne Jun 2, 2023
23cbb66
Merge branch 'master' into soumagne/remove_psm2
soumagne Jun 2, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 1 addition & 28 deletions site_scons/components/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,29 +108,12 @@ def define_mercury(reqs):
else:
reqs.define('rt', libs=['rt'])

reqs.define('psm2',
retriever=GitRepoRetriever('https://github.com/cornelisnetworks/opa-psm2.git'),
# psm2 hard-codes installing into /usr/...
commands=[['sed',
'-i',
'-e',
's/\\(.{DESTDIR}\\/\\)usr\\//\\1/',
'-e',
's/\\(INSTALL_LIB_TARG=\\/usr\\/lib\\)64/\\1/',
'-e',
's/\\(INSTALL_LIB_TARG=\\)\\/usr/\\1/',
'Makefile',
'compat/Makefile'],
['make', 'LIBDIR=/lib64'],
['make', 'DESTDIR=$PSM2_PREFIX', 'LIBDIR=/lib64', 'install']],
headers=['psm2.h'],
libs=['psm2'])

# pylint: disable-next=wrong-spelling-in-comment,fixme
# TODO: change to --enable-opx once upgraded to libfabric 1.17+
ofi_build = ['./configure',
'--prefix=$OFI_PREFIX',
'--disable-efa',
'--disable-psm2',
'--disable-psm3',
'--disable-opx',
'--without-gdrcopy']
Expand All @@ -140,23 +123,13 @@ def define_mercury(reqs):
else:
ofi_build.append('--disable-debug')

ofi_build.extend(include(reqs,
'psm2',
check(reqs,
'psm2',
['--enable-psm2=$PSM2_PREFIX',
'LDFLAGS=-Wl,--enable-new-dtags -Wl,-rpath=$PSM2_PREFIX/lib64'],
['--enable-psm2']),
['--disable-psm2']))

reqs.define('ofi',
retriever=GitRepoRetriever('https://github.com/ofiwg/libfabric'),
commands=[['./autogen.sh'],
ofi_build,
['make'],
['make', 'install']],
libs=['fabric'],
requires=include(reqs, 'psm2', ['psm2'], []),
config_cb=ofi_config,
headers=['rdma/fabric.h'],
package='libfabric-devel' if inst(reqs, 'ofi') else None,
Expand Down
5 changes: 0 additions & 5 deletions site_scons/prereq_tools/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,6 @@
from SCons.Script import BUILD_TARGETS
from SCons.Errors import InternalError

OPTIONAL_COMPS = ['psm2']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe we should keep this capability and just have it be empty? unless we think this was just 1off special handling for opx

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jolivier23 @ashleypittman any preferences ? should I keep OPTIONAL_COMPS = [''] ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question. I'd be fine removing it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok I kept it removed and I also removed the doc section about it.



class DownloadFailure(Exception):
"""Exception raised when source can't be downloaded
Expand Down Expand Up @@ -443,7 +441,6 @@ def __init__(self, env, opts):

RUNNER.initialize(self.__env)

opts.Add(ListVariable('INCLUDE', "Optional components to build", 'none', OPTIONAL_COMPS))
opts.Add(PathVariable('PREFIX', 'Installation path', install_dir,
PathVariable.PathIsDirCreate))
opts.Add('ALT_PREFIX', f'Specifies {os.pathsep} separated list of alternative paths to add',
Expand Down Expand Up @@ -813,8 +810,6 @@ def require(self, env, *comps, **kw):
def included(self, *comps):
"""Returns true if the components are included in the build"""
for comp in comps:
if comp not in OPTIONAL_COMPS:
continue
if not set([comp, 'all']).intersection(set(self.include)):
return False
return True
Expand Down
13 changes: 6 additions & 7 deletions src/cart/README.env
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,13 @@ This file lists the environment variables used in CaRT.

. D_PROVIDER (Deprecated: CRT_PHY_ADDR_STR)
It determines which mercury NA plugin to be used:
1: set it as "ofi+verbs;ofi_rxm" to use OFI verbs;ofi_rxm provider
2: set it as "ofi+psm2" to use OFI psm2 provider
3: set it as "ofi+gni" to use OFI gni provider
4: set it as "sm" to use SM plugin which only works within single node
5: set it as "ofi+tcp;ofi_rxm" to use OFI tcp;ofi_rxm provider.
6: set it as "ofi+sockets" to use OFI sockets provider
- set it as "ofi+verbs;ofi_rxm" to use OFI verbs;ofi_rxm provider
- set it as "ofi+gni" to use OFI gni provider
soumagne marked this conversation as resolved.
Show resolved Hide resolved
- set it as "sm" to use SM plugin which only works within single node
- set it as "ofi+tcp;ofi_rxm" to use OFI tcp;ofi_rxm provider.
- set it as "ofi+sockets" to use OFI sockets provider
NOTE: This provider is deprecated in favor of "ofi+tcp;ofi_rxm"
7: by default (not set or set as any other value) it will use ofi tcp
- by default (not set or set as any other value) it will use ofi tcp
provider.

. D_LOG_FILE
Expand Down
8 changes: 2 additions & 6 deletions src/cart/crt_hg.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,6 @@ struct crt_na_dict crt_na_dict[] = {
.nad_str = "ofi+gni",
.nad_contig_eps = true,
.nad_port_bind = false,
}, {
.nad_type = CRT_PROV_OFI_PSM2,
.nad_str = "ofi+psm2",
.nad_contig_eps = false,
.nad_port_bind = false,
}, {
.nad_type = CRT_PROV_OFI_TCP_RXM,
.nad_str = "ofi+tcp;ofi_rxm",
Expand Down Expand Up @@ -532,7 +527,8 @@ crt_provider_ip_str_get(bool primary, int provider)
static bool
crt_provider_is_block_mode(int provider)
{
if (provider == CRT_PROV_OFI_PSM2 || provider == CRT_PROV_OFI_OPX)
/* return false for providers that should busy poll */
if (provider == CRT_PROV_OFI_OPX)
return false;

return true;
Expand Down
32 changes: 1 addition & 31 deletions src/cart/crt_init.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -525,8 +525,7 @@ prov_settings_apply(bool primary, crt_provider_t prov, crt_init_options_t *opt)
return;

/* rxm and verbs providers only works with regular EP */
if (prov != CRT_PROV_OFI_PSM2 &&
prov != CRT_PROV_OFI_SOCKETS &&
if (prov != CRT_PROV_OFI_SOCKETS &&
crt_provider_is_sep(primary, prov)) {
D_WARN("set CRT_CTX_SHARE_ADDR as 1 is invalid "
"for current provider, ignoring it.\n");
Expand All @@ -544,13 +543,6 @@ prov_settings_apply(bool primary, crt_provider_t prov, crt_init_options_t *opt)

}

/* Print notice that "ofi+psm2" will be deprecated*/
if (prov == CRT_PROV_OFI_PSM2) {
D_WARN("\"ofi+psm2\" will be deprecated soon.\n");
setenv("FI_PSM2_NAME_SERVER", "1", true);
D_DEBUG(DB_ALL, "Setting FI_PSM2_NAME_SERVER to 1\n");
}

if (prov == CRT_PROV_OFI_CXI)
mrc_enable = 1;
else {
Expand Down Expand Up @@ -1004,19 +996,6 @@ crt_get_port_opx(int *port)
return rc;
}

static inline int
crt_get_port_psm2(int *port)
{
int rc = 0;
uint16_t pid;

pid = getpid();
*port = (pid << 8);
D_DEBUG(DB_ALL, "got a port: %d.\n", *port);

return rc;
}

#define PORT_RANGE_STR_SIZE 32

static void
Expand Down Expand Up @@ -1173,9 +1152,6 @@ crt_na_config_init(bool primary, crt_provider_t provider,
provider == CRT_PROV_OFI_TCP_RXM)
crt_port_range_verify(port);

if (provider == CRT_PROV_OFI_PSM2)
port = (uint16_t)port << 8;

if (provider == CRT_PROV_OFI_CXI && port_auto_adjust) {
if (port > 511) {
D_WARN("Port=%d outside of valid range 0-511, "
Expand All @@ -1187,12 +1163,6 @@ crt_na_config_init(bool primary, crt_provider_t provider,

D_DEBUG(DB_ALL, "OFI_PORT %d, using it as service port.\n", port);
}
} else if (provider == CRT_PROV_OFI_PSM2) {
rc = crt_get_port_psm2(&port);
if (rc != 0) {
D_ERROR("crt_get_port failed, rc: %d.\n", rc);
D_GOTO(out, rc);
}
} else if (provider == CRT_PROV_OFI_OPX) {
rc = crt_get_port_opx(&port);
if (rc != 0) {
Expand Down
1 change: 0 additions & 1 deletion src/include/cart/types.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,6 @@ typedef enum {
CRT_PROV_OFI_SOCKETS,
CRT_PROV_OFI_VERBS_RXM,
CRT_PROV_OFI_GNI,
CRT_PROV_OFI_PSM2,
CRT_PROV_OFI_TCP_RXM,
CRT_PROV_OFI_CXI,
CRT_PROV_OFI_OPX,
Expand Down
3 changes: 0 additions & 3 deletions src/tests/ftest/cart/util/cart_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -364,9 +364,6 @@ def build_cmd(self, env, host, **kwargs):
hostfile = write_host_file(tst_host, daos_test_shared_dir, tst_ppn)
mca_flags = ["btl self,tcp"]

if self.provider == "ofi+psm2":
mca_flags.append("pml ob1")

tst_cmd = env

tst_cont = os.getenv("CRT_TEST_CONT", "0")
Expand Down
7 changes: 3 additions & 4 deletions src/tests/ftest/util/dmg_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,14 +90,14 @@ def network_scan(self, provider=None):
# "HostFabric": {
# "Interfaces": [
# {
# "Provider": "ofi+psm2",
# "Provider": "ofi+tcp",
# "Device": "ib1",
# "NumaNode": 1,
# "Priority": 0,
# "NetDevClass": 32
# },
# {
# "Provider": "ofi+psm2",
# "Provider": "ofi+tcp",
# "Device": "ib0",
# "NumaNode": 0,
# "Priority": 1,
Expand All @@ -119,7 +119,6 @@ def network_scan(self, provider=None):
# }
# ],
# "Providers": [
# "ofi+psm2",
# "ofi+verbs;ofi_rxm",
# "ofi+tcp;ofi_rxm",
# "ofi+verbs",
Expand Down Expand Up @@ -1130,7 +1129,7 @@ def config_generate(self, access_points, num_engines=None, scm_only=False,
net_class (str): Network class preferred. Defaults to None.
i.e. "ethernet"|"infiniband"
net_provider (str): Network provider preferred. Defaults to None.
i.e. "ofi+tcp;ofi_rxm"|"ofi+psm2" etc.
i.e. "ofi+tcp;ofi_rxm" etc.
use_tmpfs_scm (bool, optional): Whether to use a ramdisk instead of PMem
as SCM. Defaults to False.

Expand Down
1 change: 0 additions & 1 deletion src/tests/ftest/util/ior_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -326,7 +326,6 @@ def get_default_env(self, manager_cmd, log_file=None):
env = self.env.copy()
env["D_LOG_FILE"] = get_log_file(log_file or "{}_daos.log".format(self.command))
env["MPI_LIB"] = '""'
env["FI_PSM2_DISCONNECT"] = "1"

# ior POSIX api does not require the below options.
if "POSIX" in manager_cmd:
Expand Down
1 change: 0 additions & 1 deletion src/tests/ftest/util/mdtest_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,6 @@ def get_default_env(self, manager_cmd, log_file=None):
env = self.env.copy()
env["D_LOG_FILE"] = get_log_file(log_file or "{}_daos.log".format(self.command))
env["MPI_LIB"] = '""'
env["FI_PSM2_DISCONNECT"] = "1"

if "mpirun" in manager_cmd or "srun" in manager_cmd:
env["DAOS_POOL"] = self.dfs_pool_uuid.value
Expand Down
5 changes: 2 additions & 3 deletions src/tests/ftest/util/server_utils_base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -662,9 +662,8 @@ def get_network_scan_info(self, host):
Returns:
dict: a dictionary of network information for the host, e.g.
{
1: {"fabric_iface": ib0, "provider": "ofi+psm2"},
2: {"fabric_iface": ib0, "provider": "ofi+verbs"},
3: {"fabric_iface": ib0, "provider": "ofi+tcp"},
1: {"fabric_iface": ib0, "provider": "ofi+verbs"},
2: {"fabric_iface": ib0, "provider": "ofi+tcp"},
}

"""
Expand Down
1 change: 0 additions & 1 deletion utils/build.config
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ ISAL_CRYPTO = v2.23.0
SPDK = v22.01.2
OFI = v1.17.1
MERCURY = v2.3.0rc5
PSM2 = PSM2_11.2.78
PROTOBUFC = v1.3.3
UCX=v1.13.0

Expand Down