Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue 119: download files with different extension than exe #126

Closed
wants to merge 7 commits into from
Closed

Issue 119: download files with different extension than exe #126

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3615b54
Default win extensions regex changed
Nebelhom Aug 16, 2013
1484243
Revert regex change, add manual test for win32 zip
Nebelhom Aug 20, 2013
189983b
Fixed issue in Daily Scraper
Nebelhom Sep 24, 2013
f08ddc5
Change manual test for zip extension to txt and json
Nebelhom Oct 20, 2013
1d0fd6b
Add json test
Nebelhom Nov 17, 2013
8577b17
Add txt test, clean up manifest.ini
Nebelhom Nov 18, 2013
bb35ee9
remove json file and test, adjust .gitignore
Nebelhom Nov 18, 2013
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,29 @@
build
dist
tests/venv
run_tests.sh~
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm... I didn't mean to put this in, but it might as well be. It is a constant source of aggravation for me. If I should remove it let me please know. thanks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove and adjust your editor to not create those backups for the mozdownload project.

tests/data/firefox/tinderbox-builds/mozilla-central-linux/1374568307/firefox-25.0a1.en-US.linux-i686.tar.bz2~
tests/data/firefox/tinderbox-builds/mozilla-central-linux/1374568307/firefox-25.0a1.en-US.linux-i686.txt~
tests/data/firefox/tinderbox-builds/mozilla-central-win32/1374568307/firefox-25.0a1.en-US.win32.installer.exe~
tests/data/firefox/tinderbox-builds/mozilla-central-win32/1374573725/firefox-25.0a1.en-US.win32.installer.exe~
tests/data/firefox/tinderbox-builds/mozilla-inbound-win32/1374568307/firefox-25.0a1.en-US.win32.installer.exe~
tests/data/firefox/tinderbox-builds/mozilla-inbound-win32/1374573725/firefox-25.0a1.en-US.win32.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-aurora-win32/1380318874/thunderbird-27.0a1.en-US.win32.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-aurora-win32/1380362527/thunderbird-27.0a1.en-US.win32.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-central-l10n/thunderbird-27.0a1.de.win32.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-central-l10n/thunderbird-27.0a1.pt-PT.win32.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-central-linux/1380318874/thunderbird-27.0a1.en-US.linux-i686.tar.bz2~
tests/data/thunderbird/tinderbox-builds/comm-central-linux/1380318874/thunderbird-27.0a1.en-US.linux-i686.txt~
tests/data/thunderbird/tinderbox-builds/comm-central-linux/1380362527/thunderbird-27.0a1.en-US.linux-i686.tar.bz2~
tests/data/thunderbird/tinderbox-builds/comm-central-linux/1380362527/thunderbird-27.0a1.en-US.linux-i686.txt~
tests/data/thunderbird/tinderbox-builds/comm-central-linux64/1380318874/thunderbird-27.0a1.en-US.linux-x86_64.tar.bz2~
tests/data/thunderbird/tinderbox-builds/comm-central-linux64/1380362527/thunderbird-27.0a1.en-US.linux-x86_64.tar.bz2~
tests/data/thunderbird/tinderbox-builds/comm-central-win32-debug/1380318874/thunderbird-27.0a1.en-US.win32.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-central-win32-debug/1380362527/thunderbird-27.0a1.en-US.win32.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-central-win32/1380318874/thunderbird-27.0a1.en-US.win32.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-central-win32/1380362527/thunderbird-27.0a1.en-US.win32.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-central-win64/1380318874/thunderbird-27.0a1.en-US.win64-x86_64.installer.exe~
tests/data/thunderbird/tinderbox-builds/comm-central-win64/1380362527/thunderbird-27.0a1.en-US.win64-x86_64.installer.exe~
tests/manifest.ini~
tests/release_candidate_scraper/manifest.ini~

4 changes: 2 additions & 2 deletions mozdownload/scraper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -444,8 +444,8 @@ def binary_regex(self):
'linux64': r'\.%(EXT)s$',
'mac': r'\.%(EXT)s$',
'mac64': r'\.%(EXT)s$',
'win32': r'(\.installer)%(STUB)s\.%(EXT)s$',
'win64': r'(\.installer)%(STUB)s\.%(EXT)s$'}
'win32': r'(\.installer%(STUB)s)?\.%(EXT)s$',
'win64': r'(\.installer%(STUB)s)?\.%(EXT)s$'}
regex = regex_base_name + regex_suffix[self.platform]

return regex % {'APP': self.application,
Expand Down
11 changes: 11 additions & 0 deletions ...ox/tinderbox-builds/mozilla-central-linux/1374583608/firefox-25.0a1.en-US.linux-i686.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Nam cursus. Morbi ut mi. Nullam enim leo, egestas id, condimentum at, laoreet mattis, massa. Sed eleifend nonummy diam. Praesent mauris ante, elementum et, bibendum at, posuere sit amet, nibh. Duis tincidunt lectus quis dui viverra vestibulum. Suspendisse vulputate aliquam dui. Nulla elementum dui ut augue. Aliquam vehicula mi at mauris. Maecenas placerat, nisl at consequat rhoncus, sem nunc gravida justo, quis eleifend arcu velit quis lacus. Morbi magna magna, tincidunt a, mattis non, imperdiet vitae, tellus. Sed odio est, auctor ac, sollicitudin in, consequat vitae, orci. Fusce id felis. Vivamus sollicitudin metus eget eros.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer a .txt file to download. We can make use of the ones in the latest sub folders.


Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. In posuere felis nec tortor. Pellentesque faucibus. Ut accumsan ultricies elit. Maecenas at justo id velit placerat molestie. Donec dictum lectus non odio. Cras a ante vitae enim iaculis aliquam. Mauris nunc quam, venenatis nec, euismod sit amet, egestas placerat, est. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas. Cras id elit. Integer quis urna. Ut ante enim, dapibus malesuada, fringilla eu, condimentum quis, tellus. Aenean porttitor eros vel dolor. Donec convallis pede venenatis nibh. Duis quam. Nam eget lacus. Aliquam erat volutpat. Quisque dignissim congue leo.

Mauris vel lacus vitae felis vestibulum volutpat. Etiam est nunc, venenatis in, tristique eu, imperdiet ac, nisl. Cum sociis natoque penatibus et magnis dis parturient montes, nascetur ridiculus mus. In iaculis facilisis massa. Etiam eu urna. Sed porta. Suspendisse quam leo, molestie sed, luctus quis, feugiat in, pede. Fusce tellus. Sed metus augue, convallis et, vehicula ut, pulvinar eu, ante. Integer orci tellus, tristique vitae, consequat nec, porta vel, lectus. Nulla sit amet diam. Duis non nunc. Nulla rhoncus dictum metus. Curabitur tristique mi condimentum orci. Phasellus pellentesque aliquam enim. Proin dui lectus, cursus eu, mattis laoreet, viverra sit amet, quam. Curabitur vel dolor ultrices ipsum dictum tristique. Praesent vitae lacus. Ut velit enim, vestibulum non, fermentum nec, hendrerit quis, leo. Pellentesque rutrum malesuada neque.

Nunc tempus felis vitae urna. Vivamus porttitor, neque at volutpat rutrum, purus nisi eleifend libero, a tempus libero lectus feugiat felis. Morbi diam mauris, viverra in, gravida eu, mattis in, ante. Morbi eget arcu. Morbi porta, libero id ullamcorper nonummy, nibh ligula pulvinar metus, eget consectetuer augue nisi quis lacus. Ut ac mi quis lacus mollis aliquam. Curabitur iaculis tempus eros. Curabitur vel mi sit amet magna malesuada ultrices. Ut nisi erat, fermentum vel, congue id, euismod in, elit. Fusce ultricies, orci ac feugiat suscipit, leo massa sodales velit, et scelerisque mi tortor at ipsum. Proin orci odio, commodo ac, gravida non, tristique vel, tellus. Pellentesque nibh libero, ultricies eu, sagittis non, mollis sed, justo. Praesent metus ipsum, pulvinar pulvinar, porta id, fringilla at, est.

Phasellus felis dolor, scelerisque a, tempus eget, lobortis id, libero. Donec scelerisque leo ac risus. Praesent sit amet est. In dictum, dolor eu dictum porttitor, enim felis viverra mi, eget luctus massa purus quis odio. Etiam nulla massa, pharetra facilisis, volutpat in, imperdiet sit amet, sem. Aliquam nec erat at purus cursus interdum. Vestibulum ligula augue, bibendum accumsan, vestibulum ut, commodo a, mi. Morbi ornare gravida elit. Integer congue, augue et malesuada iaculis, ipsum dui aliquet felis, at cursus magna nisl nec elit. Donec iaculis diam a nisi accumsan viverra. Duis sed tellus et tortor vestibulum gravida. Praesent elementum elit at tellus. Curabitur metus ipsum, luctus eu, malesuada ut, tincidunt sed, diam. Donec quis mi sed magna hendrerit accumsan. Suspendisse risus nibh, ultricies eu, volutpat non, condimentum hendrerit, augue. Etiam eleifend, metus vitae adipiscing semper, mauris ipsum iaculis elit, congue gravida elit mi egestas orci. Curabitur pede.

Maecenas aliquet velit vel turpis. Mauris neque metus, malesuada nec, ultricies sit amet, porttitor mattis, enim. In massa libero, interdum nec, interdum vel, blandit sed, nulla. In ullamcorper, est eget tempor cursus, neque mi consectetuer mi, a ultricies massa est sed nisl. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos hymenaeos. Proin nulla arcu, nonummy luctus, dictum eget, fermentum et, lorem. Nunc porta convallis pede.
12 changes: 6 additions & 6 deletions tests/manifest.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
[include:base_scraper/manifest.ini]
[include:daily_scraper/manifest.ini]
[include:direct_scraper/manifest.ini]
[include:directory_parser/manifest.ini]
[include:release_candidate_scraper/manifest.ini]
[include:release_scraper/manifest.ini]
#[include:base_scraper/manifest.ini]
#[include:daily_scraper/manifest.ini]
#[include:direct_scraper/manifest.ini]
#[include:directory_parser/manifest.ini]
#[include:release_candidate_scraper/manifest.ini]
#[include:release_scraper/manifest.ini]
[include:tinderbox_scraper/manifest.ini]
1 change: 1 addition & 0 deletions tests/test_cases.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ mozdownload -t tinderbox -p win32

```
mozdownload -a firefox -t tinderbox -p linux --branch=mozilla-central --extension=txt
mozdownload -a firefox -t tinderbox -p linux --branch=mozilla-central --extension=json
```
<!-- Issue #180 -->
```
Expand Down
10 changes: 9 additions & 1 deletion tests/tinderbox_scraper/test_tinderbox_scraper.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,15 @@
'platform': 'win32'},
'target': 'mozilla-inbound-firefox-25.0a1.en-US.win32.installer.exe',
'target_url': 'firefox/tinderbox-builds/mozilla-inbound-win32/'
'1374583608/firefox-25.0a1.en-US.win32.installer.exe'}
'1374583608/firefox-25.0a1.en-US.win32.installer.exe'},
# -a firefox -t tinderbox -p linux --branch=mozilla-central --extension=json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As mentioned above, lets use .txt from the latest sub folder.

{'args': {'application': 'firefox',
'branch': 'mozilla-central',
'extension': 'json',
'platform': 'linux'},
'target': 'mozilla-central-firefox-25.0a1.en-US.linux-i686.json',
'target_url': 'firefox/tinderbox-builds/mozilla-central-linux/'
'1374583608/firefox-25.0a1.en-US.linux-i686.json'}
]

thunderbird_tests = [
Expand Down