Fix DALI FW iterator reset for DROP last batch policy #4881
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- when the DALI FW iterator follows the DROP last batch policy, and the user iterates over it using its length, the last batch that is supposed to be dropped remains in the pipeline and is dropped in the next iteration. This PR makes sure that reset of the DALI pipeline can drop this redundant batch, so next iteration won't raise StopIteration Signed-off-by: Janusz Lisiecki <[email protected]>
|
!build |
|
CI MESSAGE: [8461064]: BUILD STARTED |
|
CI MESSAGE: [8461064]: BUILD PASSED |
szalpal
approved these changes
May 31, 2023
Signed-off-by: Janusz Lisiecki <[email protected]>
mzient
reviewed
Jun 1, 2023
| # incomplete batch, we need to extract from the pipeline and drop before rising | ||
| # StopIteration indicating the pipeline is depleted. Here we first check if that | ||
| # is the case, and if so we run the pipeline and drop the last batch | ||
| if self._last_batch_policy == LastBatchPolicy.DROP and not (): |
There was a problem hiding this comment.
Suggested change
| if self._last_batch_policy == LastBatchPolicy.DROP and not (): | |
| if self._last_batch_policy == LastBatchPolicy.DROP: |
mzient
approved these changes
Jun 1, 2023
|
!build |
|
CI MESSAGE: [8490353]: BUILD STARTED |
|
CI MESSAGE: [8490387]: BUILD STARTED |
|
CI MESSAGE: [8490353]: BUILD PASSED |
JanuszL
added a commit
to JanuszL/DALI
that referenced
this pull request
Oct 13, 2023
- when the DALI FW iterator follows the DROP last batch policy, and the user iterates over it using its length, the last batch that is supposed to be dropped remains in the pipeline and is dropped in the next iteration. This PR makes sure that reset of the DALI pipeline can drop this redundant batch, so next iteration won't raise StopIteration Signed-off-by: Janusz Lisiecki <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
and the user iterates over it using its length, the last batch
that is supposed to be dropped remains in the pipeline
and is dropped in the next iteration. This PR makes sure that
reset of the DALI pipeline can drop this redundant batch, so
next iteration won't raise StopIteration
Category:
Bug fix (non-breaking change which fixes an issue)
Description:
and the user iterates over it using its length, the last batch
that is supposed to be dropped remains in the pipeline
and is dropped in the next iteration. This PR makes sure that
reset of the DALI pipeline can drop this redundant batch, so
next iteration won't raise StopIteration
Additional information:
Affected modules and functionalities:
Key points relevant for the review:
Tests:
Checklist
Documentation
DALI team only
Requirements
REQ IDs: N/A
JIRA TASK: N/A