TypeError: Expected object_relevance_per_qid dictionary and measures set. #52
Comments
|
I'm not able to reproduce: >>> import pytrec_eval
>>> from collections import defaultdict
>>> q = defaultdict(dict)
>>> q['test']['1'] = 1
>>> measures = {'ndcg_cut.1,3,5,10,20,100,1000', 'recall.1,3,5,10,20,100,1000', 'map_cut.1,3,5,10,20,100,1000', 'P.1,3,5,10,20,100,1000'}
>>> pytrec_eval.RelevanceEvaluator(q, measures)
<pytrec_eval.RelevanceEvaluator object at 0x7f9095f3bac0>Can you try to make a minimal reproducible example? |
|
Sorry for the late reply. I tried to run your code and got the same error again. However, I noticed when reinstalling pytrec_eval that wheel was not installed. Therefore, it states: After installing wheel and installing pytrec_eval again, the code works as expected. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I am getting this error when running some evalution task with MTEB.
It seems like it has something to do with pytrec. The error occurs in the shared library (pytrec_eval_ext.cpython-310-x86_64-linux-gnu.so).
Interestingly I copied over the shared library from a different machine with similar configuratiton and the same version of pytrec_eval installed and it seems to solve the issue. Nevertheless, it would be good to find out what is going wrong.
Here is the trace:
The RelevanceEvaluator is initialized in the following way:
with
qrlesbeing:and
{map_string, ndcg_string, recall_string, precision_string}:Both machines have pytrec_eval verison 0.5, setuptools 59.6.0, Python 3.10.12, Ubuntu 22.04, and gcc 11.4.0
The text was updated successfully, but these errors were encountered: