Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: close browser after URL processing with Playwright #342

Merged

Conversation

edamame8888
Copy link
Contributor

*Issue #341

Description of changes:

This pull request enhances the resource management of our URL loading function by ensuring that Playwright pages are closed after processing each URL. It incorporates a page.close() statement within the URL processing loop and closes the browser with browser.close() at the end of the session. These changes prevent resource leaks and improve performance during bulk URL operations.

Changes:

  • Inserted page.close() within the loop to close each page after its content is processed. a8379fb

@edamame8888
Copy link
Contributor Author

edamame8888 commented Jun 6, 2024

Performance Before & After

The memory being maxed out at 100% is the cause.

Before the improvement, the memory performance in Cloud Watch Insight was maxed out and the content retrieval never finished.
After the improvement, memory performance improved and we confirmed that the embedding was completed in a short period of time.

#341 (comment)

337038006-028eeee5-daf6-4960-a09d-8dadabce21ed

@statefb
Copy link
Contributor

statefb commented Jun 6, 2024

Thank you for the investigation with clear evidence! LGTM, thank you for your contribution!! @edamame8888

@statefb statefb merged commit 7c1dfe8 into aws-samples:main Jun 6, 2024
7 checks passed
statefb added a commit that referenced this pull request Jun 6, 2024
statefb added a commit that referenced this pull request Jun 6, 2024
@statefb
Copy link
Contributor

statefb commented Jun 6, 2024

Merged v1 by #346

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants