Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

【Bug】engine启动一段时间后,spark的hdfs的tmp里面的目录丢失,所有代码执行报文件不存在 #1827

Open
freezhou2022 opened this issue Aug 28, 2022 · 1 comment

Comments

@freezhou2022
Copy link

engine启动一段时间后,spark的hdfs的tmp里面目录丢失,所有代码执行报文件不存在,下图报错
微信图片_20220828211422

@ZhengshuaiPENG ZhengshuaiPENG changed the title engine启动一段时间后,spark的hdfs的tmp里面的目录丢失,所有代码执行报文件不存在 【Bug】engine启动一段时间后,spark的hdfs的tmp里面的目录丢失,所有代码执行报文件不存在 Aug 29, 2022
@chncaesar
Copy link
Contributor

根据日志,spark.local.dir 设置为 /tmp。 该目录会被Linux 定时清理。
因而,修改配置,spark.local.dir 设置为 高速且不被清理的磁盘。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants