[BUG] download missing latest osm file

[smellman]

Describe the bug
New planet data released but download planet file function detect older planet file

To Reproduce
Steps to reproduce the behavior:

1. run

set -e
java -Xmx24g \
  -jar planetiler.jar \
  `# Download the latest planet.osm.pbf from s3://osm-pds bucket` \
  --area=planet --bounds=world --download \
  `# Accelerate the download by fetching the 10 1GB chunks at a time in parallel` \
  --download-threads=10 --download-chunk-size-mb=1000 \
  --mbtiles=output.mbtiles \
  --nodemap-type=array --nodemap-storage=mmap 2>&1 | tee logs.txt

2. Download data from 'https://osm-pds.s3.amazonaws.com/2023/planet-230501.osm.pbf` but planet-20230529.osm.pbf is correct.

0:00:03 INF [download] - Downloading https://osm-pds.s3.amazonaws.com/2023/planet-230501.osm.pbf to data/sources/planet.osm.pbf

Expected behavior
Download latest (planet-20230529.osm.pbf)

Screenshots
If applicable, add screenshots to help explain your problem (include tile IDs or latitude/longitude for visual issues
with generated maps)

Environment (please complete the following information):

• OS: Debian GNU Linux bullesye
• Java version and distribution: OpenJDK 64-Bit Server VM (build 17.0.6+10-Debian-1deb11u1, mixed mode, sharing)
• Planetiler: 0.6.0 jar file

[erik]

Seems the osm-pds S3 bucket index page that Planetiler fetches to grab the most recent version (https://osm-pds.s3.amazonaws.com/) is either cached or not updating correctly.

$ aws s3 ls osm-pds/2023
...
2023-06-05 10:21:28 74353561106 planet-230529.osm.pbf
...

$ curl https://osm-pds.s3.amazonaws.com/ | grep planet-230529.osm.pbf 

Planetiler could be updated to list files with the S3 API directly, but maybe there's an issue with the osm-pds pipeline at the moment https://github.com/mojodna/osm-pds-pipelines/