Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues running the rfp_scraper on Windows from a .py file but not when running in .ipynb #11

Open
Arpanio opened this issue Mar 10, 2023 · 1 comment
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@Arpanio
Copy link
Collaborator

Arpanio commented Mar 10, 2023

I ran into the following error when trying to convert the rfp_scraper.ipynb notebook to a python script:

DevTools listening on ws://127.0.0.1:61525/devtools/browser/104a4d12-5f62-4e59-ab41-de5c2538219f
[7972:17088:0309/193728.241:ERROR:cert_verify_proc_builtin.cc(677)] CertVerifyProcBuiltin for accounts.google.com failed:
----- Certificate i=1 (CN=Selenium Wire CA) -----
ERROR: No matching issuer found

[7972:4932:0309/193729.351:ERROR:device_event_log_impl.cc(218)] [19:37:29.347] USB: usb_device_handle_win.cc:1046 Failed to read descriptor from node connection: A device attached to the system is not functioning. (0x1F)
[7972:17088:0309/193740.995:ERROR:cert_verify_proc_builtin.cc(677)] CertVerifyProcBuiltin for optimizationguide-pa.googleapis.com failed:
----- Certificate i=1 (CN=Selenium Wire CA) -----
ERROR: No matching issuer found

[7972:17088:0309/193827.533:ERROR:cert_verify_proc_builtin.cc(677)] CertVerifyProcBuiltin for update.googleapis.com failed:
----- Certificate i=1 (CN=Selenium Wire CA) -----
ERROR: No matching issuer found

[7972:17088:0309/193834.051:ERROR:cert_verify_proc_builtin.cc(677)] CertVerifyProcBuiltin for service.ariba.com failed:
----- Certificate i=1 (CN=Selenium Wire CA) -----
ERROR: No matching issuer found

rfp_scraper.ipynb works fine. It's probably due to some invisible Jupyter Notebook shenanigans that's missing when running from a plain .py file (so annoying).

Whether this should be fixed or not is an open question. I'm reporting this for visibility since at least 1 other developer is using Windows. I believe the goal is to deploy the scraper to a linux machine, so it would probably be a better use of time for me to setup a Linux VM instead.

@Arpanio Arpanio self-assigned this Mar 10, 2023
@Arpanio Arpanio added the help wanted Extra attention is needed label Mar 10, 2023
@Arpanio
Copy link
Collaborator Author

Arpanio commented Mar 12, 2023

I could setup Ubuntu using Windows Subsystem for Linux, and get around the Selenium issues (mostly. Scraping still stops with no error messages but at least python-selenium is able to bind with Chrome).

Combined with VSCode and the wsl plugin, I think devs with Windows machines can get unblocked from contributing. We need not worry with maintaining the project for Windows, Mac, and Linux and only focus on Mac and Linux.

Listing down instructions here for reference. I'll work on updating the readme later:

  1. Install Windows Subsystem for Linux. I chose Ubuntu 22 as the target distro, but other devs can choose differently if they see fit.
  2. Get vscode. Other options may be viable, but this is best for my productivity.
  3. Get the wsl extension. This will allow us to remotely connect to the Ubuntu backend from vscode running on Windows. ctrl + ` will then open up terminal that's zsh on Ubuntu instead of Powershell on Windows.

@Arpanio Arpanio added documentation Improvements or additions to documentation and removed help wanted Extra attention is needed labels Mar 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

1 participant