Threaded Search Execution Throws Error on Navigating to URL

[karelvysinka]

Hello,

I've been working with the Devika project, specifically with functionalities related to executing searches in separate threads. However, I've encountered an issue where the execution throws an error when attempting to navigate to a URL during the search process. The error occurs in a scenario where a search is being performed using the Playwright library. Here's a brief overview of the error encountered:

Exception in thread Thread-8 (<lambda>): Traceback (most recent call last): File "C:\Program Files\Python312\Lib\threading.py", line 1073, in _bootstrap_inner self.run() File "C:\Program Files\Python312\Lib\threading.py", line 1010, in run self._target(*self._args, **self._kwargs) File "F:\github\devika3\devika.py", line 94, in <lambda> thread = Thread(target=lambda: agent.execute(message, project_name, search_engine)) File "F:\github\devika3\src\agents\agent.py", line 336, in execute search_results = self.search_queries(queries, project_name, engine) File "F:\github\devika3\src\agents\agent.py", line 95, in search_queries browser.go_to(link) File "F:\github\devika3\src\browser\browser.py", line 23, in go_to self.page.goto(url, timeout=20000) File "C:\Users\karel\AppData\Roaming\Python\Python312\site-packages\playwright\sync_api\_generated.py", line 8641, in goto self._sync( File "C:\Users\karel\AppData\Roaming\Python\Python312\site-packages\playwright\_impl\_sync_base.py", line 113, in _sync return task.result() File "C:\Users\karel\AppData\Roaming\Python\Python312\site-packages\playwright\_impl\_page.py", line 500, in goto return await self._main_frame.goto(**locals_to_params(locals())) File "C:\Users\karel\AppData\Roaming\Python\Python312\site-packages\playwright\_impl\_frame.py", line 145, in goto await self._channel.send("goto", locals_to_params(locals())) File "C:\Users\karel\AppData\Roaming\Python\Python312\site-packages\playwright\_impl\_connection.py", line 59, in send return await self._connection.wrap_api_call( File "C:\Users\karel\AppData\Roaming\Python\Python312\site-packages\playwright\_impl\_connection.py", line 509, in wrap_api_call return await cb() File "C:\Users\karel\AppData\Roaming\Python\Python312\site-packages\playwright\_impl\_connection.py", line 97, in inner_send result = next(iter(done)).result() playwright._impl._errors.Error: Protocol error (Page.navigate): Cannot navigate to invalid URL
This error suggests that there's an issue with the URL navigation process, possibly due to an invalid URL or an issue within the Playwright handling within a threaded environment.

Could you please help me understand the cause of this error and suggest a potential fix or workaround? Any assistance or insights into handling URL navigation errors in a threaded context would be greatly appreciated.

Thank you for your support!