⚡️ Speed up resolve_pairwise_criteria()
by 21% in libs/langchain/langchain/evaluation/comparison/eval_chain.py
#34
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
📄
resolve_pairwise_criteria()
inlibs/langchain/langchain/evaluation/comparison/eval_chain.py
📈 Performance went up by
21%
(0.21x
faster)⏱️ Runtime went down from
66.70μs
to55.00μs
Explanation and details
(click to show)
To optimize your program for increased runtime performance, I would suggest avoiding repetitive calls to
isinstance()
and reducing the function callresolve_pairwise_criteria(criterion)
inside the list comprehension.Here's the optimized version of the code.
In the optimized version, we store the type of criteria in a variable to avoid repeated calls which helps run the code faster. In the case where criteria is a list or tuple, instead of doing a double for loop directly in a dictionary comprehension, I have broken it down into a for loop which updates a dictionary. This can speed up the code because dictionary comprehensions are recomputed and rebuilt at runtime, whereas with dict.update(), the dictionary is updated in-place which makes it more efficient. Also, the resolution of nested criteria in case of a list is now handled by direct dict updates in place of nested dictionary comprehension, further improving speed and readability.
Correctness verification
The new optimized code was tested for correctness. The results are listed below.
✅ 18 Passed − ⚙️ Existing Unit Tests
✅ 0 Passed − 🎨 Inspired Regression Tests
✅ 4 Passed − 🌀 Generated Regression Tests
(click to show generated tests)