Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix paddle namespace conflict when using paddle_flags #56913

Merged

Conversation

huangjiyi
Copy link
Member

@huangjiyi huangjiyi commented Sep 3, 2023

PR types

Others

PR changes

Others

Description

上述问题的原因在 paddle::xxx 命名空间下定义或者声明了 FLAG,比如:

namespace paddle {
namespace distributed {
PD_DEFINE_int32(heter_world_size, 100, "group size"); // group max size
PD_DEFINE_int32(switch_send_recv_timeout_s, 600, "switch_send_recv_timeout_s");

在对该文件进行宏展开预处理后,会得到:

namespace paddle {
namespace distributed {
namespace paddle { namespace flags { static const int32_t FLAGS_heter_world_size_default = 100; int32_t FLAGS_heter_world_size = 100; static ::paddle::flags::FlagRegisterer flag_heter_world_size_registerer( "heter_world_size", "group size", "/home/huangjiyi/code/develop/Paddle/paddle/fluid/distributed/ps/service/heter_client.cc", &FLAGS_heter_world_size_default, &FLAGS_heter_world_size); } } using paddle_flags::FLAGS_heter_world_size;
namespace paddle { namespace flags { static const int32_t FLAGS_switch_send_recv_timeout_s_default = 600; int32_t FLAGS_switch_send_recv_timeout_s = 600; static ::paddle::flags::FlagRegisterer flag_switch_send_recv_timeout_s_registerer( "switch_send_recv_timeout_s", "switch_send_recv_timeout_s", "/home/huangjiyi/code/develop/Paddle/paddle/fluid/distributed/ps/service/heter_client.cc", &FLAGS_switch_send_recv_timeout_s_default, &FLAGS_switch_send_recv_timeout_s); } } using paddle_flags::FLAGS_switch_send_recv_timeout_s;

相当于在 paddle::distributed 命名空间下又定义了一个 paddle 命名空间,这样的话后续该文件在 paddle::distributed 命名空间下的 paddle::xxx 用法都会被解析成 paddle::distributed::paddle::xxx

更底层的原因

在编译的过程中,需要对 paddle::xxx 用法进行解析,需要先找到符号 paddle 的声明或定义,由于 paddle 是一个非限定名称(没有前缀::),因此查找过程的顺序是:先在当前命名空间 paddle::distributed 进行查找,如果没找到就在上一层命名空间进行找,直到找完全局命名空间,ref: https://en.cppreference.com/w/cpp/language/unqualified_lookup

在上面的例子中,由于在 paddle::distributed 命名空间下使用 PD_DEFINE_int32 会定义一个 paddle 命名空间,编译器在解析后续的 paddle 名称时会先找到 paddle::distributed::paddle,从而所有 paddle::xxx 用法都会被解析成 paddle::distributed::paddle::xxx,然后编译器再进一步按照限定名称查找这些用法的声明或者定义,最后没找到报错:

/Paddle/paddle/fluid/distributed/ps/service/heter_client.cc:432:15: error: ‘paddle::distributed::paddle::framework’ has not been declared
  432 | using paddle::framework::TransToProtoVarType;
      |               ^~~~~~~~~

Solution

  • 更换 FLAGS 定义的命名空间
  • 这个 PR 只将之前改的一个文件改回来了以说明这个问题已经解决了,所以没改其他文件

@paddle-bot
Copy link

paddle-bot bot commented Sep 3, 2023

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot bot added the contributor External developers label Sep 3, 2023
@YuanRisheng YuanRisheng merged commit 7d8402a into PaddlePaddle:develop Sep 4, 2023
BeingGod pushed a commit to BeingGod/Paddle that referenced this pull request Sep 9, 2023
@huangjiyi huangjiyi deleted the fix_paddle_namespcae_conflict branch January 9, 2024 12:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contributor External developers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants