【快乐开源】迁移 DeepCFD 案例至 PaddleScience #529

MayYouBeProsperous · 2023-09-06T14:17:19Z

PR types

Others

PR changes

Others

Describe

题目：#439

复现的验收标准如下：
Total MSE = 2.03±0.136
Ux MSE = 0.773±0.0897
Uy MSE =0.2153±0.0186
p MSE =1.042±0.0431

本次复现的实现指标如下：
Total MSE = 2.07983
Ux MSE = 0.79234
Uy MSE = 0.21595
p MSE = 1.07154

paddle-bot · 2023-09-06T14:17:23Z

Thanks for your contribution!

HydrogenSulfate · 2023-09-07T15:40:43Z

ppsci/arch/unetex.py

+    wn=True,
+    bn=True,


===> weight_norm, batch_norm

HydrogenSulfate · 2023-09-07T15:41:44Z

ppsci/arch/unetex.py

+    activation=nn.ReLU,
+    convolution=nn.Conv2D,
+):
+    assert kernel_size % 2 == 1


对于参数的检查使用if-raise ValueError(xxx)，assert一般用于内部生成的变量，不要滥用assert来做参数的检查

ppsci/arch/unetex.py

HydrogenSulfate · 2023-09-07T15:46:29Z

ppsci/arch/unetex.py

+    wn=True,
+    bn=True,


HydrogenSulfate · 2023-09-07T16:25:41Z

examples/deepcdf/deepcdf.py

+        constraint=constraint,
+        output_dir=OUTPUT_DIR,
+        optimizer=optimizer,
+        epochs=EPOCHS,


constraint, OUTPUT_DIR, optimizer, epochs=EPOCHS,

HydrogenSulfate · 2023-09-07T16:28:26Z

examples/deepcdf/deepcdf.py

+    solver.train()
+
+    ############### evaluation after training ###############
+    output_dict = solver.predict({"input": test_x})


加一个参数 return_numpy=True，会返回numpy.ndarray

HydrogenSulfate · 2023-09-07T16:29:17Z

examples/deepcdf/deepcdf.py

+    p_MSE = ((out[:, 2, :, :] - test_y[:, 2, :, :]) ** 2).sum() / len(test_x)
+
+    logger.info(
+        f"Total MSE is {Total_MSE.detach().numpy()[0]}, Ux MSE is {Ux_MSE.detach().numpy()[0]}, Uy MSE is {Uy_MSE.detach().numpy()[0]}, p MSE is {p_MSE.detach().numpy()[0]}"


上面改完就可以删除.detach().numpy()[0]了

建议加上 xxx:.5f，保留一下位数，不然太长很难看

HydrogenSulfate · 2023-09-07T16:30:15Z

ppsci/arch/unetex.py

+from typing import Tuple
+
+import paddle
+import paddle.nn as nn


==> from paddle import nn

HydrogenSulfate · 2023-09-07T16:30:52Z

ppsci/arch/unetex.py

+import paddle
+import paddle.nn as nn
+import paddle.nn.functional as F
+from paddle.nn.utils import weight_norm


删除该行，建议使用 nn.utils.weight_norm

… into dc

HydrogenSulfate · 2023-09-11T11:29:24Z

ppsci/arch/unetex.py

+                tensor = tensors.pop()
+                size = sizes.pop()
+                ind = indices.pop()
+                # 反池化操作，为上采样


==> # upsample operations

HydrogenSulfate · 2023-09-12T03:26:01Z

@MayYouBeProsperous review好了，有时间可以修改一下代码

MayYouBeProsperous · 2023-09-12T04:50:17Z

@MayYouBeProsperous review好了，有时间可以修改一下代码

辛苦了，已修改代码

HydrogenSulfate

几处稍微修改下应该没什么问题了

另外案例文档撰写的时候，数据集下载地址可以填写如下：

HydrogenSulfate · 2023-09-12T05:58:40Z

examples/deepcdf/deepcdf.py

+    with open(os.path.join(DATASET_PATH, "dataX.pkl"), "rb") as file:
+        x = pickle.load(file)
+    with open(os.path.join(DATASET_PATH, "dataY.pkl"), "rb") as file:
+        y = pickle.load(file)


哦哦，那先不加了吧

HydrogenSulfate · 2023-09-12T05:59:08Z

examples/deepcdf/deepcdf.py

+      tensors (List[np.array]): non-empty tensor list.
+      ratio (float): split ratio. For example, tensor list A is split to A1 and A2. len(A1) / len(A) = ratio.


英文句子开头大写

HydrogenSulfate · 2023-09-12T06:02:28Z

examples/deepcdf/deepcdf.py

+            },
+            "batch_size": BATCH_SIZE,
+            "sampler": {
+                "name": "DistributedBatchSampler",


这里应该只需要用 BatchSampler 就行了，多卡的情况下会在 build_dataloader 中自动替换成 DistributedBatchSampler

HydrogenSulfate · 2023-09-13T08:03:57Z

整个项目的命名是不是错了，应该是CFD而不是CDF，整个目录全部替换一下

luotao1 · 2023-09-15T03:36:37Z

To do: 编写案例文档

记得补充下案例文档哈

HydrogenSulfate

辛苦大佬写的文档，主要有一些代码上的小问题和文档整体代码块错位，修改后应该可以合了

HydrogenSulfate · 2023-09-23T09:08:59Z

docs/zh/examples/deepcfd.md

+| dataX | [dataX.pkl](https://paddle-org.bj.bcebos.com/paddlescience/datasets/DeepCFD/dataX.pkl) |
+| dataY | [dataY.pkl](https://paddle-org.bj.bcebos.com/paddlescience/datasets/DeepCFD/dataY.pkl) |
+
+数据集官网为：https://zenodo.org/record/3666056/files/DeepCFD.zip?download=1


官网==》原始下载地址

HydrogenSulfate · 2023-09-23T09:10:42Z

docs/zh/examples/deepcfd.md

+``` py linenums="145" title="examples/deepcfd/deepcfd.py"
+--8<--
+examples/deepcfd/deepcfd.py:145:163
+--8<--
+```


代码块位置不对，

HydrogenSulfate · 2023-09-23T09:15:25Z

docs/zh/examples/deepcfd.md

+
+第二个参数是损失函数的定义，这里使用自定义的损失函数，分别计算 Ux 和 Uy 的均方误差，以及 p 的标准差，然后三者加权求和。
+
+第三个参数是约束条件的名字，方便后续对其索引。此次命名为 sup_constraint。


sup_constraint ==> "sup_constraint"

HydrogenSulfate · 2023-09-23T09:16:53Z

examples/deepcfd/deepcfd.py

+        eval_during_train=True,
+        eval_freq=50,
+        validator=validator,
+        # visualizer=visualizer,


这行注释可以删掉

HydrogenSulfate · 2023-09-23T09:17:34Z

examples/deepcfd/deepcfd.py

+    CHANNELS_WEIGHTS = np.reshape(
+        np.sqrt(
+            np.mean(
+                np.transpose(y, (0, 2, 3, 1)).reshape((981 * 172 * 79, 3)) ** 2, axis=0


981、172、79应该属于magic number，可以用全大写变量表示一下

examples/deepcfd/deepcfd.py

HydrogenSulfate · 2023-09-23T09:30:08Z

examples/deepcfd/deepcfd.py

+    return split1, split2
+
+
+def predict_and_save_plot(


同样，docstring加上返回值

这个函数没有返回值

HydrogenSulfate · 2023-09-23T09:30:44Z

examples/deepcfd/deepcfd.py

+    train_x, train_y = train_dataset[:]
+    test_x, test_y = test_dataset[:]


这里是否需要[:]浅复制操作？后续应该不涉及对输入数据的修改吧

HydrogenSulfate · 2023-09-23T09:31:41Z

examples/deepcfd/deepcfd.py

+    # evaluate after finished training
+    solver.eval()
+
+    PLOT_DIR = os.path.join(OUTPUT_DIR, "visu")


visu ==> visual

HydrogenSulfate · 2023-09-23T09:32:04Z

ppsci/arch/unetex.py

+        in_channel (int): Number of channels of input.
+        out_channel (int): Number of channels of output.
+        kernel_size (int): Size of kernel of convolution layer. Defaults to 3.
+        filters (Tuple[int, ...]): Number of filters. Defaults to [16, 32, 64].


默认值非None的参数，docstring中类型后面加上, optional，比如： Tuple[int, ...], optional

默认值为None的参数，docstring中类型用Optional[...]，比如 Optional[Type[nn.Layer]]

docstring中出现了paddle.nn.Layer，这应该是没更新导致的，可以基于autodocstring重新生成最新的docstring进行修改

默认值最好不要使用可变类型如list、dict，可以使用None然后在代码里面对入参判None，赋予默认值；或者使用小括号 (...)

好的感谢

HydrogenSulfate · 2023-09-23T09:42:50Z

docs/zh/examples/deepcfd.md

+``` py linenums="1" title="examples/deepcfd/deepcfd.py"
+--8<--
+examples/deepcfd/deepcfd.py:1:304
+--8<--
+```


完整代码可以不加 :1:304，直接使用整个文件即可

MayYouBeProsperous

好的已按照要求修改了

MayYouBeProsperous · 2023-09-24T02:36:47Z

docs/zh/examples/deepcfd.md

+``` py linenums="1" title="examples/deepcfd/deepcfd.py"
+--8<--
+examples/deepcfd/deepcfd.py:1:304
+--8<--
+```


MayYouBeProsperous · 2023-09-24T02:37:31Z

docs/zh/examples/deepcfd.md

+| dataX | [dataX.pkl](https://paddle-org.bj.bcebos.com/paddlescience/datasets/DeepCFD/dataX.pkl) |
+| dataY | [dataY.pkl](https://paddle-org.bj.bcebos.com/paddlescience/datasets/DeepCFD/dataY.pkl) |
+
+数据集官网为：https://zenodo.org/record/3666056/files/DeepCFD.zip?download=1


MayYouBeProsperous · 2023-09-24T02:37:35Z

docs/zh/examples/deepcfd.md

+``` py linenums="145" title="examples/deepcfd/deepcfd.py"
+--8<--
+examples/deepcfd/deepcfd.py:145:163
+--8<--
+```


MayYouBeProsperous · 2023-09-24T02:37:38Z

docs/zh/examples/deepcfd.md

+
+第二个参数是损失函数的定义，这里使用自定义的损失函数，分别计算 Ux 和 Uy 的均方误差，以及 p 的标准差，然后三者加权求和。
+
+第三个参数是约束条件的名字，方便后续对其索引。此次命名为 sup_constraint。


examples/deepcfd/deepcfd.py

MayYouBeProsperous · 2023-09-24T02:39:09Z

examples/deepcfd/deepcfd.py

+    CHANNELS_WEIGHTS = np.reshape(
+        np.sqrt(
+            np.mean(
+                np.transpose(y, (0, 2, 3, 1)).reshape((981 * 172 * 79, 3)) ** 2, axis=0


MayYouBeProsperous · 2023-09-24T02:39:17Z

examples/deepcfd/deepcfd.py

+        eval_during_train=True,
+        eval_freq=50,
+        validator=validator,
+        # visualizer=visualizer,


MayYouBeProsperous · 2023-09-24T02:39:36Z

examples/deepcfd/deepcfd.py

+    # evaluate after finished training
+    solver.eval()
+
+    PLOT_DIR = os.path.join(OUTPUT_DIR, "visu")


MayYouBeProsperous · 2023-09-24T02:40:11Z

ppsci/arch/unetex.py

+        in_channel (int): Number of channels of input.
+        out_channel (int): Number of channels of output.
+        kernel_size (int): Size of kernel of convolution layer. Defaults to 3.
+        filters (Tuple[int, ...]): Number of filters. Defaults to [16, 32, 64].


好的感谢

examples/deepcfd/deepcfd.py

HydrogenSulfate · 2023-09-24T08:09:37Z

docs/zh/examples/deepcfd.md

+
+$$\rho(\frac{\partial}{\partial t}  + \bf{u} \cdot  div ) \bf{u} = - \nabla p +  - \nabla \tau + \bf{f}$$
+
+其中 $\bf{u}$ 是速度差（具有 x 和 y 两个维度），$\rho$ 是密度， $p$ 是压强场，$\bf{f}$ 是体积力（例如重力）。


速度差 ==> 速度场？

HydrogenSulfate · 2023-09-24T08:11:26Z

docs/zh/examples/deepcfd.md

+
+<figure markdown>
+  ![DeepCFD](https://ai-studio-static-online.cdn.bcebos.com/288c37b569d5400aa7b2265ff13fcf0edad3115e70fe4fafb6736215355771fe){ loading=lazy}
+  <figcaption>OpenFOAM 计算结果与 DeepCFD 预测结果对比</figcaption>


==> OpenFOAM 计算结果与 DeepCFD 预测结果对比，从左到右分别为：水平速度分量（Ux），垂直速度分量（Uy）以及流体压强（p）

HydrogenSulfate · 2023-09-24T08:12:21Z

docs/zh/examples/deepcfd.md

+  ![DeepCFD](https://ai-studio-static-online.cdn.bcebos.com/288c37b569d5400aa7b2265ff13fcf0edad3115e70fe4fafb6736215355771fe){ loading=lazy}
+  <figcaption>OpenFOAM 计算结果与 DeepCFD 预测结果对比</figcaption>
+</figure>
+


图片下面加一句简单结论：可以看到DeepCFD方法与OpenFOAM的结果基本一致

HydrogenSulfate

LGTM

* add deepcdf * fix bugs * add comment * add comment * add comment * code style * add license * code style * fix * fix * fix * fix * fix * add validator * fix * add validator * add doc * fix * fix * fix * fix * refine doc

MayYouBeProsperous added 5 commits September 5, 2023 10:19

add deepcdf

a02718f

fix bugs

e437195

add comment

5e5cb2b

add comment

20862cf

add comment

50f8b4b

paddle-bot bot added the contributor label Sep 6, 2023

MayYouBeProsperous added 3 commits September 6, 2023 22:24

code style

00c73f7

add license

568ac23

code style

c97b7bd

luotao1 assigned luotao1 and HydrogenSulfate Sep 7, 2023

luotao1 added the HappyOpenSource 快乐开源活动issue与PR label Sep 7, 2023

HydrogenSulfate requested changes Sep 8, 2023

View reviewed changes

Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleScience…

4d52c6b

… into dc

HydrogenSulfate requested changes Sep 11, 2023

View reviewed changes

MayYouBeProsperous added 4 commits September 12, 2023 11:44

fix

f7d6a04

fix

7f01fca

fix

6b67e68

fix

784bbb7

HydrogenSulfate requested changes Sep 12, 2023

View reviewed changes

MayYouBeProsperous added 3 commits September 12, 2023 14:17

fix

451f12b

add validator

56edced

fix

40a5a8e

add validator

019edd6

add doc

810c31f

HydrogenSulfate requested changes Sep 23, 2023

View reviewed changes

MayYouBeProsperous added 5 commits September 23, 2023 18:25

fix

f6b796d

fix

200a88c

fix

da596eb

fix

fbdc27e

fix

7667f67

MayYouBeProsperous commented Sep 24, 2023

View reviewed changes

HydrogenSulfate requested changes Sep 24, 2023

View reviewed changes

refine doc

eeaab5d

HydrogenSulfate approved these changes Sep 24, 2023

View reviewed changes

PaddlePaddle locked as resolved and limited conversation to collaborators Sep 24, 2023

PaddlePaddle unlocked this conversation Sep 24, 2023

HydrogenSulfate merged commit 41e0424 into PaddlePaddle:develop Sep 24, 2023

MayYouBeProsperous deleted the dc branch October 18, 2023 07:16

luotao1 added HappyOpenSource Pro 进阶版快乐开源活动，更具挑战性的任务 and removed HappyOpenSource 快乐开源活动issue与PR labels Nov 20, 2023

		tensors (List[np.array]): non-empty tensor list.
		ratio (float): split ratio. For example, tensor list A is split to A1 and A2. len(A1) / len(A) = ratio.


		第二个参数是损失函数的定义，这里使用自定义的损失函数，分别计算 Ux 和 Uy 的均方误差，以及 p 的标准差，然后三者加权求和。

		第三个参数是约束条件的名字，方便后续对其索引。此次命名为 sup_constraint。

		train_x, train_y = train_dataset[:]
		test_x, test_y = test_dataset[:]


		$$\rho(\frac{\partial}{\partial t} + \bf{u} \cdot div ) \bf{u} = - \nabla p + - \nabla \tau + \bf{f}$$

		其中 $\bf{u}$ 是速度差（具有 x 和 y 两个维度），$\rho$ 是密度， $p$ 是压强场，$\bf{f}$ 是体积力（例如重力）。

【快乐开源】迁移 DeepCFD 案例至 PaddleScience #529

【快乐开源】迁移 DeepCFD 案例至 PaddleScience #529

Conversation

MayYouBeProsperous commented Sep 6, 2023 • edited Loading

PR types

PR changes

Describe

paddle-bot bot commented Sep 6, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HydrogenSulfate commented Sep 12, 2023

MayYouBeProsperous commented Sep 12, 2023

HydrogenSulfate left a comment • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HydrogenSulfate commented Sep 13, 2023 • edited Loading

luotao1 commented Sep 15, 2023

HydrogenSulfate left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MayYouBeProsperous left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HydrogenSulfate left a comment

Choose a reason for hiding this comment

MayYouBeProsperous commented Sep 6, 2023 •

edited

Loading

HydrogenSulfate left a comment •

edited

Loading

HydrogenSulfate commented Sep 13, 2023 •

edited

Loading