Can't pickle local object if lambdas are used

[lars-reimann]

Describe the bug

Starting the runner on a pipeline that contains lambdas leads to the following logs:

2024-04-06 19:58:54.544 [info] Launching Pipeline (ab133d54-709c-4605-ae98-e1babcb10642): file:///c%3A/Users/Lars/OneDrive/Desktop/test/test.sdstest - myPipeline
2024-04-06 19:58:54.544 [info] Sending message to python server: {"type":"program","id":"ab133d54-709c-4605-ae98-e1babcb10642","data":{"code":{"test":{"gen_test":"# Imports ----------------------------------------------------------------------\r\n\r\nimport safeds_runner\r\nfrom safeds.data.tabular.containers import Row, Table\r\n\r\n# Pipelines --------------------------------------------------------------------\r\n\r\ndef myPipeline():\r\n    titanic = safeds_runner.memoized_call(\"safeds.data.tabular.containers.Table.from_csv_file\", Table.from_csv_file, ['C:/Users/Lars/OneDrive/Desktop/test/titanic.csv'], [safeds_runner.file_mtime('C:/Users/Lars/OneDrive/Desktop/test/titanic.csv')])\r\n    safeds_runner.save_placeholder('titanic', titanic)\r\n    filtered = safeds_runner.memoized_call(\"safeds.data.tabular.containers.Table.filter_rows\", Table.filter_rows, [titanic, lambda param1: (safeds_runner.memoized_call(\"safeds.data.tabular.containers.Row.get_value\", Row.get_value, [param1, 'age'], [])) >= (18)], [])\r\n    safeds_runner.save_placeholder('filtered', filtered)\r\n","gen_test_myPipeline":"from .gen_test import myPipeline\r\n\r\nif __name__ == '__main__':\r\n    myPipeline()\r\n"}},"main":{"modulepath":"test","module":"test","pipeline":"myPipeline"}}}
2024-04-06 19:58:54.545 [info] [Runner-Err] DEBUG:root:Received Message: {"type":"program","id":"ab133d54-709c-4605-ae98-e1babcb10642","data":{"code":{"test":{"gen_test":"# Imports ----------------------------------------------------------------------\r\n\r\nimport safeds_runner\r\nfrom safeds.data.tabular.containers import Row, Table\r\n\r\n# Pipelines --------------------------------------------------------------------\r\n\r\ndef myPipeline():\r\n    titanic = safeds_runner.memoized_call(\"safeds.data.tabular.containers.Table.from_csv_file\", Table.from_csv_file, ['C:/Users/Lars/OneDrive/Desktop/test/titanic.csv'], [safeds_runner.file_mtime('C:/Users/Lars/OneDrive/Desktop/test/titanic.csv')])\r\n    safeds_runner.save_placeholder('titanic', titanic)\r\n    filtered = safeds_runner.memoized_call(\"safeds.data.tabular.containers.Table.filter_rows\", Table.filter_rows, [titanic, lambda param1: (safeds_runner.memoized_call(\"safeds.data.tabular.containers.Row.get_value\", Row.get_value, [param1, 'age'], [])) >= (18)], [])\r\n    safeds_runner.save_placeholder('filtered', filtered)\r\n","gen_test_myPipeline":"from .gen_test import myPipeline\r\n\r\nif __name__ == '__main__':\r\n    myPipeline()\r\n"}},"main":{"modulepath":"test","module":"test","pipeline":"myPipeline"}}}
2024-04-06 19:58:57.204 [info] [Runner] Message received: '{"type": "placeholder_type", "id": "ab133d54-709c-4605-ae98-e1babcb10642", "data": {"name": "titanic", "type": "Table"}}'
2024-04-06 19:58:57.205 [info] Placeholder was calculated (ab133d54-709c-4605-ae98-e1babcb10642): titanic of type Table
2024-04-06 19:58:57.205 [info] Placeholder was calculated (ab133d54-709c-4605-ae98-e1babcb10642): titanic of type Table
2024-04-06 19:58:57.205 [info] Placeholder was calculated (ab133d54-709c-4605-ae98-e1babcb10642): titanic of type Table
2024-04-06 19:58:57.205 [info] Placeholder was calculated (ab133d54-709c-4605-ae98-e1babcb10642): titanic of type Table
2024-04-06 19:58:57.245 [info] [Runner] Message received: '{"type": "runtime_error", "id": "ab133d54-709c-4605-ae98-e1babcb10642", "data": {"message": "Can't pickle local object 'myPipeli<truncated>'

To Reproduce

Run the following pipeline:

package test

pipeline myPipeline {
    val titanic = Table.fromCsvFile("C:/Users/Lars/OneDrive/Desktop/test/titanic.csv");
    val filtered = titanic.filterRows((param1) -> (param1.getValue("age") as Int) >= 18);
}

Replace the path to the CSV file with some other one.

The same issue occurs with block lambdas:

package test

pipeline myPipeline {
    val titanic = Table.fromCsvFile("C:/Users/Lars/OneDrive/Desktop/test/titanic.csv");
    val filtered = titanic.filterRows((param1) {
        yield param2 = (param1.getValue("age") as Int) >= 18;
    });
}

Expected behavior

The pipeline should run without error.

Screenshots (optional)

No response

Additional Context (optional)

It would also be nice if errors would be marked with [error] instead of [info] in the logs.

[WinPlay02]

@lars-reimann The lambda is already solved by the memoization PR I will submit later

[lars-reimann]

OK, you can overwrite my change then (PR coming soon). I need it now for some final testing.

[lars-reimann]

🎉 This issue has been resolved in version 0.8.1 🎉

The release is available on:

• v0.8.1
• GitHub release

Your semantic-release bot 📦🚀

[lars-reimann]

🎉 This issue has been resolved in version 0.9.0 🎉

The release is available on:

• v0.9.0
• GitHub release

Your semantic-release bot 📦🚀