[Example]【快乐开源】迁移 CFDGCN 案例至 PaddleScience #609

MayYouBeProsperous · 2023-10-30T02:48:18Z

PR types

Others

PR changes

Others

Describe

【快乐开源】迁移 CFDGCN 案例至 PaddleScience #441

paddle-bot · 2023-11-02T06:26:59Z

Thanks for your contribution!

paddle-bot · 2023-11-02T06:27:03Z

✅ This PR's description meets the template requirements!
Please wait for other CI results.

MayYouBeProsperous · 2023-11-29T07:53:35Z

ppsci/data/dataset/airfoil_dataset.py

@@ -119,6 +121,8 @@ def __init__(

        self.nodes = self.mesh_graph[0]
        self.edges = self.mesh_graph[1]
+        if edges_tran:


amgnet和本模型数据集相同，但图格式不太一样吗

看了下AMGNet的代码，这里的转置主要是为了后续在获取sender、receiver以及模型里涉及edge_index的切片更方便。这里可以加这么一个参数，但是名字改一下吧，叫transpose_edges，然后Docstring：Whether transpose the edges array from (2, num_edges) to (num_edges, 2) for convenient of slicing.

HydrogenSulfate · 2023-11-29T11:34:03Z

examples/cfdgcn/su2paddle/__init__.py

+from .su2_function import SU2Module  # noqa: F401
+from .su2_function_mpi import activate_su2_mpi  # noqa: F401
+from .su2_numpy import SU2Numpy  # noqa: F401


去掉 from 后面的.

HydrogenSulfate · 2023-11-29T11:36:05Z

examples/cfdgcn/su2paddle/common.py

+    r"""Pad a list of variable length Tensors with ``padding_value``
+    ``pad_sequence`` stacks a list of Tensors along a new dimension,
+    and pads them to equal length. For example, if the input is list of
+    sequences with size ``L x *`` and if batch_first is False, and ``T x B x *``
+    otherwise.
+    `B` is batch size. It is equal to the number of elements in ``sequences``.
+    `T` is length of the longest sequence.
+    `L` is length of the sequence.
+    `*` is any number of trailing dimensions, including none.
+    Example:
+        >>> a = paddle.ones(25, 300)
+        >>> b = paddle.ones(22, 300)
+        >>> c = paddle.ones(15, 300)
+        >>> pad_sequence([a, b, c]).shape
+        paddle.Tensor([25, 3, 300])
+    Note:
+        This function returns a Tensor of size ``T x B x *`` or ``B x T x *``
+        where `T` is the length of the longest sequence. This function assumes
+        trailing dimensions and type of all the Tensors in sequences are same.
+    Args:
+        sequences (list[Tensor]): list of variable length sequences.
+        batch_first (bool, optional): output will be in ``B x T x *`` if True, or in
+            ``T x B x *`` otherwise
+        padding_value (float, optional): value for padded elements. Default: 0.
+    Returns:
+        Tensor of size ``T x B x *`` if :attr:`batch_first` is ``False``.
+        Tensor of size ``B x T x *`` otherwise
+    """


不同段落之间加一个空行

两个连续的 `` 改为一个 `

HydrogenSulfate · 2023-11-29T11:38:45Z

examples/cfdgcn/su2paddle/common.py

+            # TODO (Hui Zhang): set_value op not supprot `end==start`
+            # TODO (Hui Zhang): set_value op not support int16
+            # TODO (Hui Zhang): set_varbase 2 rank not support [0,0,...]
+            # out_tensor[i, :length, ...] = tensor


这个注释是paddle框架目前不支持吗？如果不是可以删除？

好的已删除

HydrogenSulfate · 2023-11-29T11:39:29Z

examples/cfdgcn/su2paddle/common.py

+            # TODO (Hui Zhang): set_value op not supprot `end==start`
+            # out_tensor[:length, i, ...] = tensor


HydrogenSulfate · 2023-11-29T11:40:42Z

examples/cfdgcn/su2paddle/common.py

+    scale_each: bool = False,
+    pad_value: int = 0,
+    **kwargs,
+) -> paddle.Tensor:


完善docstring

这部分代码未使用已删除

HydrogenSulfate · 2023-11-29T12:13:41Z

ppsci/arch/cfdgcn.py

+
+
+def write_graph_mesh(
+    output_filename: Union[str, PathLike],


所有路径只使用 str, 不要引入额外的库增加理解成本

好的已修改

HydrogenSulfate · 2023-11-29T12:23:19Z

ppsci/data/process/batch_transform/__init__.py

+        # graph.tensor()
+        # graph.shape = [len(batch)]
+        # return graph
+        batch = np.array([g.tensor() for g in batch])


这个改动没看懂，g.tensor()返回的是内部数据转成tensor的graph，然后又转成np.array？

np.array 中的元素是graph对象，本来想返回 list，但下面代码会报错。

PaddleScience/ppsci/solver/train.py

Line 74 in 5442d9a

total_batch_size += next(iter(input_dict.values())).shape[0]

np.array 中的元素是graph对象，本来想返回 list，但下面代码会报错。

PaddleScience/ppsci/solver/train.py

Line 74 in 5442d9a

total_batch_size += next(iter(input_dict.values())).shape[0]

你说得对，这里我觉得使用 len(next(iter(input_dict.values()))) 会更好，这样能兼容非Tensor类型的数据。

HydrogenSulfate · 2023-11-29T12:28:00Z

ppsci/arch/cfdgcn.py

+        self.output_keys = output_keys
+        meshes_temp_dir = "temp_meshes"
+        os.makedirs(meshes_temp_dir, exist_ok=True)
+        self.mesh_file = meshes_temp_dir + "/" + str(os.getpid()) + "_mesh.su2"


所有涉及路径的拼接一律使用os.path.join，不使用加号

HydrogenSulfate · 2023-11-29T12:29:12Z

ppsci/arch/cfdgcn.py

+            self.write_mesh_file(
+                nodes, self.elems, self.marker_dict, filename=self.mesh_file
+            )


每次前向都往磁盘里写数据这个操作是必要的吗？

确实不用，移到了 init 中

HydrogenSulfate · 2023-11-29T13:07:41Z

ppsci/data/dataset/airfoil_dataset.py

@@ -119,6 +121,8 @@ def __init__(

        self.nodes = self.mesh_graph[0]
        self.edges = self.mesh_graph[1]
+        if edges_tran:


看了下AMGNet的代码，这里的转置主要是为了后续在获取sender、receiver以及模型里涉及edge_index的切片更方便。这里可以加这么一个参数，但是名字改一下吧，叫transpose_edges，然后Docstring：Whether transpose the edges array from (2, num_edges) to (num_edges, 2) for convenient of slicing.

HydrogenSulfate

现有改动可能会导致AMGNet的训练、评估无法跑通

# linux
wget https://paddle-org.bj.bcebos.com/paddlescience/datasets/AMGNet/data.zip
# windows
# curl https://paddle-org.bj.bcebos.com/paddlescience/datasets/AMGNet/data.zip --output data.zip
# unzip it
unzip data.zip
python amgnet_cylinder.py mode=eval EVAL.pretrained_model_path=https://paddle-org.bj.bcebos.com/paddlescience/models/amgnet/amgnet_cylinder_pretrained.pdparams

因为把collate_fn从合并graph改成了不合并graph，需要把AMGNet的forward、案例代码里的eval_expr和loss_expr内，对graph list进行手动合并

HydrogenSulfate · 2023-12-01T13:49:05Z

ppsci/arch/cfdgcn.py

代码文件开头加一下 LICENSE 文本

HydrogenSulfate · 2023-12-01T14:03:12Z

ppsci/data/process/batch_transform/__init__.py

+        # graph.tensor()
+        # graph.shape = [len(batch)]
+        # return graph
+        batch = np.array([g.tensor() for g in batch])


np.array 中的元素是graph对象，本来想返回 list，但下面代码会报错。

PaddleScience/ppsci/solver/train.py

Line 74 in 5442d9a

total_batch_size += next(iter(input_dict.values())).shape[0]

你说得对，这里我觉得使用 len(next(iter(input_dict.values()))) 会更好，这样能兼容非Tensor类型的数据。

MayYouBeProsperous · 2023-12-08T11:03:01Z

CFDGCN 的 forward 中也会合并图，所以还是将 collate_fn 保持原样，转而修改 CFDGCN ，这样代码更整洁。

HydrogenSulfate

LGTM

HydrogenSulfate · 2023-12-10T07:24:29Z

CFDGCN 的 forward 中也会合并图，所以还是将 collate_fn 保持原样，转而修改 CFDGCN ，这样代码更整洁。

辛苦大佬

MayYouBeProsperous · 2023-12-10T07:49:21Z

案例文档还在编写中

* add cdfgcn mode * add cfdgcn example * code style * fix su2paddle * refine code * codestyle * fix bugs and add visualization * fix * fix cfdgcn model

add cdfgcn mode

d6470dd

paddle-bot bot added the contributor label Oct 31, 2023

MayYouBeProsperous added 7 commits November 7, 2023 18:54

add cfdgcn example

54f51bb

code style

2aeb91f

fix su2paddle

7491cd1

refine code

cb705ae

refine code

c0146ab

codestyle

6f6c816

fix bugs and add visualization

25b11e7

MayYouBeProsperous changed the title ~~[WIP]【快乐开源】迁移 CFDGCN 案例至 PaddleScience~~ [Example]【快乐开源】迁移 CFDGCN 案例至 PaddleScience Nov 29, 2023

MayYouBeProsperous commented Nov 29, 2023

View reviewed changes

luotao1 added the HappyOpenSource 快乐开源活动issue与PR label Nov 29, 2023

luotao1 assigned luotao1 and HydrogenSulfate Nov 29, 2023

HydrogenSulfate requested changes Nov 29, 2023

View reviewed changes

fix

0f67450

HydrogenSulfate requested changes Dec 4, 2023

View reviewed changes

fix cfdgcn model

a15970d

HydrogenSulfate approved these changes Dec 10, 2023

View reviewed changes

HydrogenSulfate merged commit e14c71d into PaddlePaddle:develop Dec 11, 2023
3 of 4 checks passed

MayYouBeProsperous mentioned this pull request Dec 25, 2023

【快乐开源】迁移 CFDGCN 案例至 PaddleScience #720

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Example]【快乐开源】迁移 CFDGCN 案例至 PaddleScience #609

[Example]【快乐开源】迁移 CFDGCN 案例至 PaddleScience #609

MayYouBeProsperous commented Oct 30, 2023 •

edited

Loading

paddle-bot bot commented Nov 2, 2023

paddle-bot bot commented Nov 2, 2023 •

edited

Loading

MayYouBeProsperous Nov 29, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Dec 1, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Nov 29, 2023

MayYouBeProsperous Dec 1, 2023

HydrogenSulfate Nov 29, 2023

HydrogenSulfate left a comment

HydrogenSulfate Dec 1, 2023

MayYouBeProsperous Dec 8, 2023

HydrogenSulfate Dec 1, 2023

MayYouBeProsperous commented Dec 8, 2023 •

edited

Loading

HydrogenSulfate left a comment

HydrogenSulfate commented Dec 10, 2023

MayYouBeProsperous commented Dec 10, 2023 •

edited

Loading

		# TODO (Hui Zhang): set_value op not supprot `end==start`
		# out_tensor[:length, i, ...] = tensor

[Example]【快乐开源】迁移 CFDGCN 案例至 PaddleScience #609

[Example]【快乐开源】迁移 CFDGCN 案例至 PaddleScience #609

Conversation

MayYouBeProsperous commented Oct 30, 2023 • edited Loading

PR types

PR changes

Describe

paddle-bot bot commented Nov 2, 2023

paddle-bot bot commented Nov 2, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

HydrogenSulfate left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MayYouBeProsperous commented Dec 8, 2023 • edited Loading

HydrogenSulfate left a comment

Choose a reason for hiding this comment

HydrogenSulfate commented Dec 10, 2023

MayYouBeProsperous commented Dec 10, 2023 • edited Loading

MayYouBeProsperous commented Oct 30, 2023 •

edited

Loading

paddle-bot bot commented Nov 2, 2023 •

edited

Loading

MayYouBeProsperous commented Dec 8, 2023 •

edited

Loading

MayYouBeProsperous commented Dec 10, 2023 •

edited

Loading