Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new PyPI release (9.4.0) #4

Closed
anarcat opened this issue Jan 31, 2017 · 45 comments
Closed

new PyPI release (9.4.0) #4

anarcat opened this issue Jan 31, 2017 · 45 comments
Labels
enhancement good first issue help wanted meta issues about project organisation and release engineering

Comments

@anarcat
Copy link
Contributor

anarcat commented Jan 31, 2017

so a key issue that is missing is a PyPI release.

i have pushed the 9.3.1 tag, but that's mostly cosmetic - the release hasn't been pushed to PyPI because we don't have access. i've reviewed the process documented here:

http://code.activestate.com/lists/python-distutils-sig/24089/

and it seems we can ask for access on ... ahem... sourceforge. so i've requested a password reset and will do that there. :)

@anarcat
Copy link
Contributor Author

anarcat commented Jan 31, 2017

i requested access in this ticket, we'll see where it goes: https://sourceforge.net/p/pypi/support-requests/709/

considering how many requests are piled up in there, i'm not holding my breath... :/

@mgedmin
Copy link
Contributor

mgedmin commented Feb 24, 2017

Suggestion: let's register linkchecker2 on PyPI for now, temporarily, so we can get fixes out to users without delay.

If PyPI admins ever decide to give us access to the original linkchecker project, we can make linkchecker2 an empty package that depends on linkchecker.

@anarcat
Copy link
Contributor Author

anarcat commented Feb 24, 2017

sounds good.

@fuzzy76
Copy link

fuzzy76 commented Apr 6, 2017

FWIW, I see these kinds of requests on their Github tracker as well.

@bexelbie
Copy link

While not ideal, continuing to cut releases on GH will prompt some distributions to package newer code. I realize this means that we would be in the weird situation where pypi was behind other forms of distribution, but at least we would be able to push the code out.

@anarcat
Copy link
Contributor Author

anarcat commented Sep 21, 2017

note that the new Warehouse deployment on PyPI.org may have different rules that could work around the issues we have with abandoned software. see:

https://pypi.org/help/#project-name-claim

there's PEP-0541 under way to fix this, apparently.

@anarcat
Copy link
Contributor Author

anarcat commented Nov 3, 2017

HEADS UP: i'll look at making a release, including on PyPI, later tomorrow or next week. thanks to @fuzzy76 I've also tried my luck at transfering PyPI ownership in pypi/legacy#725 - if that fails, I'll just use the linkcheck namespace for now.

i've also asked the folks behind the linkchecker orga on github to see if they could transfer ownership as well, not sure how i missed that, see https://github.com/linkchecker/linkchecker.github.io/issues/2

@mgoodhand
Copy link
Contributor

Sounds good. linkcheck would be better anyway, as it matches the module name.

@anarcat
Copy link
Contributor Author

anarcat commented Nov 9, 2017

yep - and it sure looks like this is what will happen - no answer from the linkchecker orga or the pypa people yet, so I guess we're on our own for that next release... i'll give it another week.

@legrostdg
Copy link

@anarcat is there any update on this issue? Thanks!

@anarcat
Copy link
Contributor Author

anarcat commented Dec 12, 2017 via email

@pomali
Copy link

pomali commented Jan 9, 2018

if there is problem with pypi namespace you could temporarily just hint people to install using
pip install git+https://github.com/linkcheck/linkchecker.git in README

@legrostdg
Copy link

Would it be possible to at least update the Debian package to use this repo?

@legrostdg
Copy link

(with a new release)

@anarcat
Copy link
Contributor Author

anarcat commented Jan 10, 2018

sure. if i had time. which i don't right now, as you might have noticed. :)

sorry - i can barely keep up with the PRs that are coming in here... dealing with administrative annoyances such as this is not so exciting, so it gets lower priority...

@anarcat
Copy link
Contributor Author

anarcat commented Oct 22, 2019

i poked the pypa people, again. sigh...

@anarcat
Copy link
Contributor Author

anarcat commented Oct 22, 2019

oh god, this unblocked! we have ownership! i logged in pypi (had to use chrome because firefox 2FA fail) and i see the project!

all of a sudden, we should make a release!!! do we want to push things out as they are now? or just push 9.4? or what do we do!

so exciting!

@mgedmin
Copy link
Contributor

mgedmin commented Oct 22, 2019

Let's twine upload an sdist built from the v9.4.0 tag and see what happens.

@mgedmin
Copy link
Contributor

mgedmin commented Oct 22, 2019

I ran check-manifest in the v9.4.0 tree and it says a bunch of files would be missing from the sdist:

missing from VCS:
  MANIFEST
missing from sdist:
  .project
  .pydevproject
  Dockerfile
  dev-requirements.txt
  doc/contributing.mdwn
  doc/development.mdwn
  doc/dot-travis-osx.yml
  doc/linkchecker.1.html.diff
  doc/linkcheckerrc.5.html.diff
  doc/web
  doc/web/Makefile
  doc/web/app.yaml
  doc/web/content
  doc/web/content/donations.md
  doc/web/content/faq.md
  doc/web/content/index.md
  doc/web/content/other.md
  doc/web/content/plugins.md
  doc/web/hooks
  doc/web/hooks/__hooks__.py
  doc/web/media
  doc/web/media/css
  doc/web/media/css/print.css
  doc/web/media/css/pygment_trac.css
  doc/web/media/css/stylesheet.css
  doc/web/media/favicon.ico
  doc/web/media/images
  doc/web/media/images/back_disabled.png
  doc/web/media/images/back_enabled.png
  doc/web/media/images/back_enabled_hover.png
  doc/web/media/images/body-bg.jpg
  doc/web/media/images/download-button.png
  doc/web/media/images/forward_disabled.png
  doc/web/media/images/forward_enabled.png
  doc/web/media/images/forward_enabled_hover.png
  doc/web/media/images/github-button.png
  doc/web/media/images/header-bg.jpg
  doc/web/media/images/highlight-bg.jpg
  doc/web/media/images/hr.png
  doc/web/media/images/logo128x128.png
  doc/web/media/images/logo16x16.png
  doc/web/media/images/logo32x32.png
  doc/web/media/images/logo48x48.png
  doc/web/media/images/logo57x57.png
  doc/web/media/images/logo64x64.png
  doc/web/media/images/shot1.png
  doc/web/media/images/shot1_thumb.jpg
  doc/web/media/images/shot2.png
  doc/web/media/images/shot2_thumb.jpg
  doc/web/media/images/shot3.png
  doc/web/media/images/shot3_thumb.jpg
  doc/web/media/images/sidebar-bg.jpg
  doc/web/media/images/sort_asc.png
  doc/web/media/images/sort_asc_disabled.png
  doc/web/media/images/sort_both.png
  doc/web/media/images/sort_desc.png
  doc/web/media/images/sort_desc_disabled.png
  doc/web/media/man1
  doc/web/media/man1/linkchecker.1.html
  doc/web/media/man1/linkchecker.1.html.rej
  doc/web/media/man5
  doc/web/media/man5/linkcheckerrc.5.html
  doc/web/templates
  doc/web/templates/default.html
  doc/web/wokconfig
  requirements.txt
  robots.txt
  tests/checker/data/Bookmarks
  tests/checker/data/file.doc
  tests/checker/data/file.markdown
  tests/checker/data/file.pdf
  tests/checker/data/file.php
  tests/checker/data/file.wml
  tests/checker/data/plist_binary/Bookmarks.plist
  tests/checker/data/plist_xml/Bookmarks.plist
  tests/checker/data/sitemap.xml
  tests/checker/data/sitemapindex.xml
  tests/checker/data/Мошкова.bin
  tests/configuration/data
  tests/configuration/data/config0.ini
  tests/configuration/data/config1.ini
  tests/configuration/data/config2.ini
  tox.ini
  windows
  windows/build.bat
  windows/clean.bat
  windows/dist.bat
  windows/linkchecker.cer
  windows/linkchecker.pfx
  windows/linkchecker.pvk
  windows/makecert.bat
  windows/makelccert.bat
  windows/test.bat

It's probably fine. I'd maybe like to download a 9.3.0 sdist from PyPI and see what files it contains and what files it omits.

@laoshaw
Copy link

laoshaw commented Nov 18, 2019

Installing 9.4.0 on ubuntu 18.04 did not work for me:

pip install --user git+https://github.com/linkchecker/[email protected]
Collecting git+https://github.com/linkchecker/[email protected]
  Cloning https://github.com/linkchecker/linkchecker.git (to revision v9.4.0) to /tmp/pip-req-build-9stjmlut
  Running command git clone -q https://github.com/linkchecker/linkchecker.git /tmp/pip-req-build-9stjmlut
  Running command git checkout -q afad6c80a2b4f3ea6f3ddf880758105fefbcf432
Requirement already satisfied: dnspython in /home/test/.local/lib/python3.6/site-packages (from LinkChecker==9.4.0) (1.16.0)
Requirement already satisfied: pyxdg in /usr/lib/python3/dist-packages (from LinkChecker==9.4.0) (0.25)
Requirement already satisfied: requests<2.15,>=2.2 in /home/test/.local/lib/python3.6/site-packages (from LinkChecker==9.4.0) (2.14.2)
Building wheels for collected packages: LinkChecker
  Building wheel for LinkChecker (setup.py) ... done
  Created wheel for LinkChecker: filename=LinkChecker-9.4.0-cp36-cp36m-linux_x86_64.whl size=407865 sha256=60f201d7ee33aea1e488fd60d7627c4b6105be456983572fdd5d11a23b85f83c
  Stored in directory: /tmp/pip-ephem-wheel-cache-qwz9czrh/wheels/a2/e3/52/a9a21270049350b19adda0c88b74a62100e30012cf3397f06b
Successfully built LinkChecker
Installing collected packages: LinkChecker
Successfully installed LinkChecker-9.4.0

linkchecker --help
Traceback (most recent call last):
  File "/home/test/.local/bin/linkchecker", line 36, in <module>
    from linkcheck.cmdline import print_version, print_usage, aggregate_url, \
  File "/home/test/.local/lib/python3.6/site-packages/linkcheck/cmdline.py", line 23, in <module>
    from . import checker, fileutil, strformat, plugins
  File "/home/test/.local/lib/python3.6/site-packages/linkcheck/checker/__init__.py", line 178, in <module>
    from . import (fileurl, unknownurl, ftpurl, httpurl, dnsurl,
  File "/home/test/.local/lib/python3.6/site-packages/linkcheck/checker/ftpurl.py", line 29, in <module>
    from . import proxysupport, httpurl, internpaturl, get_index_html
  File "/home/test/.local/lib/python3.6/site-packages/linkcheck/checker/httpurl.py", line 38, in <module>
    from ..HtmlParser import htmlsax
ImportError: /home/test/.local/lib/python3.6/site-packages/linkcheck/HtmlParser/htmlsax.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyString_FromStringAndSize

@cjmayo
Copy link
Contributor

cjmayo commented Nov 19, 2019

Installing 9.4.0 on ubuntu 18.04 did not work for me:

9.4.0 (and current Git master) do not work on Python 3. I haven't tried a pip install but from the log provided it appears to have been installed for Python 3 which would be expected to fail.

@anarcat
Copy link
Contributor Author

anarcat commented Apr 9, 2020

i'm confused here, what's our status? did we push 9.4.0 to pip yet? i think that's all that's needed to close this ticket. that it doesn't work on py3 is irrelevant.

@mgedmin did you twine upload? should i?

i opened #359 to discuss the next release

@mgedmin
Copy link
Contributor

mgedmin commented Apr 10, 2020

I never did any twine uploads.

https://pypi.org/project/LinkChecker/ doesn't have 9.4.

I would love it if you would twine upload a 9.4.0 sdist.

@phaer
Copy link
Contributor

phaer commented May 5, 2020

If I understood correctly, @anarcat could push a new release, which would be the first one since 9.3 from 2014 on pypi? Or is there anything one could help with?

Thanks a lot for your work on such a useful project! ❤️

@anarcat
Copy link
Contributor Author

anarcat commented May 11, 2020

If I understood correctly, @anarcat could push a new release, which would be the first one since 9.3 from 2014 on pypi? Or is there anything one could help with?

correct. i'm just too swamped to coordinate that work right now, sorry. :(

@anarcat
Copy link
Contributor Author

anarcat commented May 16, 2020

@linkchecker/developers - it seems i'm the bottleneck here, and don't have time to followup. can i grant one of you folks access? just give me a pypi username and i can add you to the group. that said, it seems @mgedmin is already an owner on the repo, so i'm not sure i'm the only one that can do this. :)

@mgedmin
Copy link
Contributor

mgedmin commented May 16, 2020

Yeah, I have pypi rights, but I also have burnout and a middling sense of responsibility.

I care enough that I don't want to do a bad job in making the release, but I don't care enough to invest enough effort to do a good job.

(For projects I maintain I have invested enough upfront effort that I can type make release after being woken up at 2 am with a splitting headache, and it will do all the necessary checks and tell me all the necessary things I need to do to make a good release.)

If anyone wants to be my conscience and tell me that I ought to do the following:

  • git checkout v9.4.0
  • python setup.py sdist
  • twine check dist/LinkChecker-9.4.0.tar.gz
  • virtualenv -p python2 /tmp/testenv && /tmp/testenv/bin/pip install dist/LinkChecker-9.4.0.tar.gz
  • /tmp/testenv/bin/linkchecker https://projects.gedmin.as # first website that came to mind ;)
  • twine upload dist/LinkChecker-9.4.0.tar.gz

then I will do it.

I'm a bit worried that check-manifest complains that a bunch of files are going to be dropped from the sdist: files in doc/, tests/, windows/, a few in the project root. I'd be less worried if dist/*.tar.gz did not have other files in doc/ and tests/

I'm also slightly worried that tox shows two test failures (something about clamav and something about robots.txt).

But maybe releasing a slightly broken 9.4 sdist to pypi is less bad than the current state of affairs (totally broken 9.3 that doesn't like new requests versions due to a busted version check)?

And at least 9.4.0 is tagged and released elsewhere, if not to PyPI.

So, please tell me to do it and I'll do it. (I did all the steps except the twine upload, fwiw.)

@cjmayo
Copy link
Contributor

cjmayo commented May 16, 2020

FWIW Gentoo has been using the v9.4.0 tagged archive https://github.com/linkchecker/linkchecker/archive/v9.4.0.tar.gz for over a year now with no customisation (save the bash completion) and a standard disutils based install with no bugs raised.

@mgedmin
Copy link
Contributor

mgedmin commented May 16, 2020

Yeah, but the github tarball will contain all the files that are missing from the sdist due to lack of MANIFEST.in rules.

@cjmayo
Copy link
Contributor

cjmayo commented May 16, 2020

I would say you are right to be cautious. I created an sdist tarball from the tag and adapted the Gentoo ebuild, installation failed with:

FileNotFoundError: [Errno 2] No such file or directory: b'doc/development.mdwn'

@cjmayo
Copy link
Contributor

cjmayo commented May 16, 2020

My mistake -- that was the Gentoo ebuild selecting development.mdwn. Once I remove that Gentoo instruction it finished installation, I can't actually install it on my machine because I have got rid of all the Python 2 versions of the supporting packages.

@mgedmin
Copy link
Contributor

mgedmin commented May 17, 2020

I compared the files in the 9.3 tarball on PyPI with what I get in the 9.4 tarball produced from the git tag.

The 9.4 is missing

  • doc/development.txt
  • doc/html/*
  • linkchecker-gui source files and documentation
  • thirdparty/* which had bundled copies of dnspython and miniboa

I think it's fine to release it as is.

@mgedmin
Copy link
Contributor

mgedmin commented May 17, 2020

I've uploaded https://pypi.org/project/LinkChecker/9.4.0/.

@mgedmin mgedmin closed this as completed May 17, 2020
@mgedmin mgedmin changed the title new PyPI release new PyPI release (9.4.0) May 17, 2020
@laoshaw
Copy link

laoshaw commented May 18, 2020

9.4.0 release is not working for me, after doing : linkchecker http://www.example.com I got:

Traceback (most recent call last):
  File "/home/userx/.local/bin/linkchecker", line 36, in <module>
    from linkcheck.cmdline import print_version, print_usage, aggregate_url, \
  File "/home/userx/.local/lib/python3.6/site-packages/linkcheck/cmdline.py", line 23, in <module>
    from . import checker, fileutil, strformat, plugins
  File "/home/userx/.local/lib/python3.6/site-packages/linkcheck/checker/__init__.py", line 178, in <module>
    from . import (fileurl, unknownurl, ftpurl, httpurl, dnsurl,
  File "/home/userx/.local/lib/python3.6/site-packages/linkcheck/checker/ftpurl.py", line 29, in <module>
    from . import proxysupport, httpurl, internpaturl, get_index_html
  File "/home/userx/.local/lib/python3.6/site-packages/linkcheck/checker/httpurl.py", line 38, in <module>
    from ..HtmlParser import htmlsax
ImportError: /home/userx/.local/lib/python3.6/site-packages/linkcheck/HtmlParser/htmlsax.cpython-36m-x86_64-linux-gnu.so: undefined symbol: PyString_FromStringAndSize

also during installation, my requests is downgraded:

ERROR: macaroonbakery 1.3.1 has requirement requests<3.0,>=2.18.1, but you'll have requests 2.14.2 which is incompatible.
ERROR: deadlinks 0.3.0 has requirement requests>=2.22.0, but you'll have requests 2.14.2 which is incompatible.
Installing collected packages: requests, LinkChecker
  Attempting uninstall: requests
    Found existing installation: requests 2.23.0
    Uninstalling requests-2.23.0:
      Successfully uninstalled requests-2.23.0
Successfully installed LinkChecker-9.4.0 requests-2.14.2

@mgedmin
Copy link
Contributor

mgedmin commented May 18, 2020

9.4.0 does not support Python 3; for that you need the current git master.

@mgedmin
Copy link
Contributor

mgedmin commented May 18, 2020

(I am surprised and dismayed that the installation didn't fail...)

@anarcat
Copy link
Contributor Author

anarcat commented May 19, 2020

9.4.0 release is not working for me, after doing : linkchecker http://www.example.com I got:

before anyone else jumps this issue with problems they find in 9.4.0: don't report them here! :) open a new issue and document it properly, otherwise it makes our lives needlessly hard to triage...

@mgedmin mgedmin unpinned this issue May 19, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement good first issue help wanted meta issues about project organisation and release engineering
Projects
None yet
Development

No branches or pull requests