Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dart Analyzer runs with 100% CPU load when directory contains circular symlinks #42079

Closed
gatzsche opened this issue May 27, 2020 · 8 comments
Closed
Labels
analyzer-stability area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion.

Comments

@gatzsche
Copy link

Summary

  • Problem: CPU Load of Dart Analyzer is 160%
  • The problem is related to a circular symlink in the code folder
  • If this symlink is deleted the problem has gone away

This tracker is for issues related to:

  • Dart executable
  • Analyzer
  • Evtl. this issues belongs to Flutter

Versions

  • Dart SDK 2.8.2
  • Mac OSX 10.15.4
  • Flutter 1.17.1
  • VSCode 1.45.1
  • Plugin: dart-code.flutter 3.10.2

Reproduction

  • Create a new flutter plugin

    flutter create --template=plugin dart_issue
    
  • Open the code with VSCode

    code dart_issue
    
  • Select iPhone SE Simulator as target

  • Open example/lib/main.dart

  • Press F5 to run the example

  • Select "Dart & Flutter"

  • ... work with the project for a time

  • Open MacOSX Activity Monitor

  • --> Activity Monitor shows a dart process with 160% CPU load

  • Kill this dart process

  • Switch back to VSCode

  • VSCode tells, that Dart Analyzer has been terminated

  • Select "Restart Analyzer"

  • ** Process is again at 160% CPU load

How to remove the issue

  • The build process created a circular sym link

    dart_issue/example/ios/.symlinks/plugins -> dart_issue/example/ios/.symlinks/plugins -> dart_issue
    
  • Remove this symlink and the issue does not happen anymore

@vsmenon vsmenon added the area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion. label May 27, 2020
@DanTup
Copy link
Collaborator

DanTup commented Jun 8, 2020

@bwilkerson @devoncarew this seems like a repeat of #36919 though I'm not sure why (the original fix still seems to be there).

I can't repro this though - as I don't have a .symlinks folder when I run the steps above, and I'm not sure why.

@gatzsche
Copy link
Author

gatzsche commented Jun 8, 2020

Thanks a lot for investigating. Tried to reproduce the issue this morning and was not able to do so.
Anyway example/ios/.symlinks is created at my MacBook.

@DanTup
Copy link
Collaborator

DanTup commented Sep 17, 2020

@bwilkerson this came up again recently at Dart-Code/Dart-Code#2624. In this case it was cnpm (a JavaScript package manager) that was creating the circular links.

Pre-LSP, we could try to filter the analysis roots to only Dart projects on the client (we didn't, but it was an option). With LSP this won't be possible at all - the client will just give us the paths of open folders which may be a mix of Dart/web/others, so I think we may need to handle this in the server.

Some possible ideas:

  • Implement dart.analysisExcludedFolders (this is on my list to do anyway for a few other reasons) to allow users to exclude folders from analysis. This should work, but relies on users understand the problem (from the pegged CPU of the server).
  • Don't follow symlinks at all. This might break some things (for ex. the packages folder that used to be symlinks - I don't know if that's still supported though).
  • Try to detect the circular references (for example resolve the symlinks and skip over folders that resolve to folders that have already been analyzed). There may be a perf hit here to resolve every folder as a symlink (it seems like the info isn't available in Directory and requires another call).

Any thoughts?

@ueman
Copy link

ueman commented Jun 8, 2021

I'm seeing this issue from time to time on the Sentry Dart/Flutter repo when opening the Flutter project.

  • The Flutter project has an iOS and a macOS native integration.
  • Because iOS and macOS implementations are basically identical, the macOS integration is symlinked to the iOS implementation.
  • If you delete the iOS and macOS folders of the package, the analyzer has no problems. You can now undo the deletion of the folders and the analyzer does not use that much CPU. IIRC that fixes the problems until you run flutter clean

I can image that with stable support for desktop there are a lot of packages which are coming up with the same approach.
The audioplayers (repo) package for example uses the same approach.

I hope that helps.

@DanTup
Copy link
Collaborator

DanTup commented Jun 8, 2021

@ueman I cloned that repo and looked at the symlinks. Those in the macOS folder don't seem circular (they're to files) so I wouldn't expect them to cause any problems (and I couldn't repro this issue).

Although I did notice some circular symlinks in the example folder were created (like in the original report above).

./example/linux/flutter/ephemeral/.plugin_symlinks/sentry_flutter -> /private/tmp/sentry-dart/flutter/
./example/windows/flutter/ephemeral/.plugin_symlinks/sentry_flutter -> /private/tmp/sentry-dart/flutter/

However, these also did not seem to trigger this issue for me - the analysis server seems to analyse the workspace without any issues. I tested both on Flutter's stable channel (2.2.1) and also the latest master.

Could you confirm the version of Flutter you're using, and post the output of running find . -type l -ls in the folder you're opening in VS Code?

@DanTup
Copy link
Collaborator

DanTup commented Jun 8, 2021

Actually, I could repro this on Flutter 2.0.6. So I think it's something that's been fixed between 2.0.6 and 2.2.

If you can confirm you're also using 2.0.6 then I suspect this has been resolved in the latest stable releases and can be closed. Thanks!

@ueman
Copy link

ueman commented Jun 8, 2021

Output of find . -type l -ls:

62518248        0 lrwxr-xr-x    1 jonas            staff                  39  3 Jun 19:52 ./macos/Classes/SentryFlutterPlugin.h -> ../../ios/Classes/SentryFlutterPlugin.h
62518249        0 lrwxr-xr-x    1 jonas            staff                  39  3 Jun 19:52 ./macos/Classes/SentryFlutterPlugin.m -> ../../ios/Classes/SentryFlutterPlugin.m
62518250        0 lrwxr-xr-x    1 jonas            staff                  48  3 Jun 19:52 ./macos/Classes/SentryFlutterPluginApple.swift -> ../../ios/Classes/SentryFlutterPluginApple.swift
62518252        0 lrwxr-xr-x    1 jonas            staff                  29  3 Jun 19:52 ./macos/sentry_flutter.podspec -> ../ios/sentry_flutter.podspec
56289761        0 lrwxr-xr-x    1 jonas            staff                  15 13 Mär 17:45 ./CHANGELOG.md -> ../CHANGELOG.md
62170307        0 lrwxr-xr-x    1 jonas            staff                  52 30 Mai 11:29 ./example/macos/Flutter/ephemeral/.symlinks/plugins/sentry_flutter -> /Users/jonas/Documents/projects/sentry-dart/flutter/
62170306        0 lrwxr-xr-x    1 jonas            staff                 105 30 Mai 11:29 ./example/macos/Flutter/ephemeral/.symlinks/plugins/package_info_plus_macos -> /Users/jonas/Documents/projects/flutter/.pub-cache/hosted/pub.dartlang.org/package_info_plus_macos-1.1.1/
62704516        0 lrwxr-xr-x    1 jonas            staff                  52  4 Jun 20:44 ./example/ios/.symlinks/plugins/sentry_flutter -> /Users/jonas/Documents/projects/sentry-dart/flutter/
62704515        0 lrwxr-xr-x    1 jonas            staff                  99  4 Jun 20:44 ./example/ios/.symlinks/plugins/package_info_plus -> /Users/jonas/Documents/projects/flutter/.pub-cache/hosted/pub.dartlang.org/package_info_plus-1.0.1/
62170836        0 lrwxr-xr-x    1 jonas            staff                  31 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter/sentry_flutter.framework/sentry_flutter -> Versions/Current/sentry_flutter
62170835        0 lrwxr-xr-x    1 jonas            staff                  26 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter/sentry_flutter.framework/Resources -> Versions/Current/Resources
62170834        0 lrwxr-xr-x    1 jonas            staff                   1 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter/sentry_flutter.framework/Versions/Current -> A
62170833        0 lrwxr-xr-x    1 jonas            staff                  24 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter/sentry_flutter.framework/Headers -> Versions/Current/Headers
62170837        0 lrwxr-xr-x    1 jonas            staff                  24 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter/sentry_flutter.framework/Modules -> Versions/Current/Modules
61414740        0 lrwxr-xr-x    1 jonas            staff                  26 31 Mai 14:28 ./example/build/macos/Build/Products/Debug/FlutterMacOS.framework/Resources -> Versions/Current/Resources
61414742        0 lrwxr-xr-x    1 jonas            staff                   1 31 Mai 14:28 ./example/build/macos/Build/Products/Debug/FlutterMacOS.framework/Versions/Current -> A
61414738        0 lrwxr-xr-x    1 jonas            staff                  24 31 Mai 14:28 ./example/build/macos/Build/Products/Debug/FlutterMacOS.framework/Headers -> Versions/Current/Headers
61414739        0 lrwxr-xr-x    1 jonas            staff                  24 31 Mai 14:28 ./example/build/macos/Build/Products/Debug/FlutterMacOS.framework/Modules -> Versions/Current/Modules
62171199        0 lrwxr-xr-x    1 jonas            staff                  29 31 Mai 14:28 ./example/build/macos/Build/Products/Debug/FlutterMacOS.framework/FlutterMacOS -> Versions/Current/FlutterMacOS
62170827        0 lrwxr-xr-x    1 jonas            staff                  26 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/package_info_plus_macos/package_info_plus_macos.framework/Resources -> Versions/Current/Resources
62170828        0 lrwxr-xr-x    1 jonas            staff                  40 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/package_info_plus_macos/package_info_plus_macos.framework/package_info_plus_macos -> Versions/Current/package_info_plus_macos
62170826        0 lrwxr-xr-x    1 jonas            staff                   1 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/package_info_plus_macos/package_info_plus_macos.framework/Versions/Current -> A
62170825        0 lrwxr-xr-x    1 jonas            staff                  24 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/package_info_plus_macos/package_info_plus_macos.framework/Headers -> Versions/Current/Headers
62170829        0 lrwxr-xr-x    1 jonas            staff                  24 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/package_info_plus_macos/package_info_plus_macos.framework/Modules -> Versions/Current/Modules
61414793        0 lrwxr-xr-x    1 jonas            staff                  20 17 Mai 08:51 ./example/build/macos/Build/Products/Debug/App.framework/App -> Versions/Current/App
61414792        0 lrwxr-xr-x    1 jonas            staff                  26 17 Mai 08:51 ./example/build/macos/Build/Products/Debug/App.framework/Resources -> Versions/Current/Resources
61414791        0 lrwxr-xr-x    1 jonas            staff                   1 17 Mai 08:51 ./example/build/macos/Build/Products/Debug/App.framework/Versions/Current -> A
61415253        0 lrwxr-xr-x    1 jonas            staff                  26 31 Mai 14:28 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/FlutterMacOS.framework/Resources -> Versions/Current/Resources
61415255        0 lrwxr-xr-x    1 jonas            staff                   1 31 Mai 14:28 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/FlutterMacOS.framework/Versions/Current -> A
61415252        0 lrwxr-xr-x    1 jonas            staff                  29 31 Mai 14:28 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/FlutterMacOS.framework/FlutterMacOS -> Versions/Current/FlutterMacOS
61415277        0 lrwxr-xr-x    1 jonas            staff                  26 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/Sentry.framework/Resources -> Versions/Current/Resources
61415280        0 lrwxr-xr-x    1 jonas            staff                   1 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/Sentry.framework/Versions/Current -> A
61415278        0 lrwxr-xr-x    1 jonas            staff                  23 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/Sentry.framework/Sentry -> Versions/Current/Sentry
61415302        0 lrwxr-xr-x    1 jonas            staff                  31 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/sentry_flutter.framework/sentry_flutter -> Versions/Current/sentry_flutter
61415301        0 lrwxr-xr-x    1 jonas            staff                  26 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/sentry_flutter.framework/Resources -> Versions/Current/Resources
61415304        0 lrwxr-xr-x    1 jonas            staff                   1 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/sentry_flutter.framework/Versions/Current -> A
61415233        0 lrwxr-xr-x    1 jonas            staff                  20 17 Mai 08:51 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/App.framework/App -> Versions/Current/App
61415234        0 lrwxr-xr-x    1 jonas            staff                  26 17 Mai 08:51 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/App.framework/Resources -> Versions/Current/Resources
61415236        0 lrwxr-xr-x    1 jonas            staff                   1 17 Mai 08:51 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/App.framework/Versions/Current -> A
61415289        0 lrwxr-xr-x    1 jonas            staff                  26 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/package_info_plus_macos.framework/Resources -> Versions/Current/Resources
61415290        0 lrwxr-xr-x    1 jonas            staff                  40 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/package_info_plus_macos.framework/package_info_plus_macos -> Versions/Current/package_info_plus_macos
61415292        0 lrwxr-xr-x    1 jonas            staff                   1 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/sentry_flutter_example.app/Contents/Frameworks/package_info_plus_macos.framework/Versions/Current -> A
62170872        0 lrwxr-xr-x    1 jonas            staff                  28 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Pods_Runner.framework/Pods_Runner -> Versions/Current/Pods_Runner
62170871        0 lrwxr-xr-x    1 jonas            staff                  26 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Pods_Runner.framework/Resources -> Versions/Current/Resources
62170869        0 lrwxr-xr-x    1 jonas            staff                   1 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Pods_Runner.framework/Versions/Current -> A
62170868        0 lrwxr-xr-x    1 jonas            staff                  24 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Pods_Runner.framework/Headers -> Versions/Current/Headers
62170870        0 lrwxr-xr-x    1 jonas            staff                  24 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Pods_Runner.framework/Modules -> Versions/Current/Modules
62170783        0 lrwxr-xr-x    1 jonas            staff                  26 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Sentry/Sentry.framework/Resources -> Versions/Current/Resources
62170781        0 lrwxr-xr-x    1 jonas            staff                   1 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Sentry/Sentry.framework/Versions/Current -> A
62170780        0 lrwxr-xr-x    1 jonas            staff                  24 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Sentry/Sentry.framework/Headers -> Versions/Current/Headers
62170782        0 lrwxr-xr-x    1 jonas            staff                  23 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Sentry/Sentry.framework/Sentry -> Versions/Current/Sentry
62170784        0 lrwxr-xr-x    1 jonas            staff                  24 30 Mai 11:30 ./example/build/macos/Build/Products/Debug/Sentry/Sentry.framework/Modules -> Versions/Current/Modules

I'm using Flutter 2.2 but haven't seen the issue for a while.
I just though it would be because of

If you delete the iOS and macOS folders of the package, the analyzer has no problems. You can now undo the deletion of the folders and the analyzer does not use that much CPU. IIRC that fixes the problems until you run flutter clean

But it could also be the Flutter upgrade. I'm fine with closing it. If I run into this issue I'll just reopen it.

@DanTup
Copy link
Collaborator

DanTup commented Jun 8, 2021

@ueman thanks - I did try flutter clean and also fetching packages, but was still only able to repro on 2.0.6.

There is another issue affecting 2.2 that might trigger the other issue you commented in (Dart-Code/Dart-Code#3270) which is tracked in #45996. It's been fixed in master and there's an open request to cherry-pick/hotfix (#46249), so until/unless that happens you may still see the formatter issue, although when it occurs for that reason the CPU should not remain high.

I'll close this since all the other reports are quite old and it looks like this may have been fixed in Flutter 2.2, but do shout if you see it again in Flutter >= 2.2 with some instructions. Thanks!

@DanTup DanTup closed this as completed Jun 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
analyzer-stability area-analyzer Use area-analyzer for Dart analyzer issues, including the analysis server and code completion.
Projects
None yet
Development

No branches or pull requests

5 participants