Python multi-dir update results in duplicate PRs

[jakecoffman]

Is there an existing issue for this?

• I have searched the existing issues

Package ecosystem

Python - pip and pip-compile

What you expected to see, versus what you actually saw

Reproducible with the following input file:

job:
  allowed-updates:
    - dependency-type: direct
      update-type: all
  package-manager: pip
  source:
    provider: github
    repo: opentracing-contrib/nginx-opentracing
    commit: 69dd153598307375363650ea301c92cb59d997f3
    api-endpoint: https://api.github.com/
    hostname: github.com
    directories:
      - "/**/*"

Run with the Dependabot CLI: dependabot update -f input.yml

The issue seems to be caused by the FileFetcher's behavior in the Python ecosystem of searching subdirectories for requirements files:

dependabot-core/python/lib/dependabot/python/file_fetcher.rb

Lines 243 to 257 in 4420d67

	def child_requirement_files
	@child_requirement_files ||=
	begin
	fetched_files = req_txt_and_in_files.dup
	req_txt_and_in_files.flat_map do |requirement_file|
	child_files = fetch_child_requirement_files(
	file: requirement_file,
	previously_fetched_files: fetched_files
	)
	fetched_files += child_files
	child_files
	end
	end
	end

This results in the same requirements file appearing twice, thus it puts up 2 PRs.

We'll need to analyze why this behavior exists and maybe remove it. Ideally Dependabot would only process the directory it was given.