Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a new tutorial on installing packages #369

Merged
merged 7 commits into from
Aug 31, 2017
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 29 additions & 2 deletions source/key_projects.rst
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,17 @@ Dev irc:#pypa-dev

A tool for installing Python packages.

.. _Pipfile:

Pipfile
=======

`Source <https://github.com/pypa/pipfile>`__

``Pipfile`` and its sister ``Pipfile.lock`` are a higher-level
application-centric alternative to :ref:`pip`'s lower-level
``requirements.txt`` file.


Python Packaging User Guide
===========================
Expand Down Expand Up @@ -158,8 +169,8 @@ Warehouse
Dev irc:#pypa-dev


The current codebase powering :term:`PyPI`. It is hosted at
`pypi.org <https://pypi.org/>`_.
The current codebase powering the :term:`Python Package Index (PyPI)`. It is
hosted at `pypi.org <https://pypi.org/>`_.


.. _wheel:
Expand Down Expand Up @@ -310,6 +321,22 @@ files, standalone Python environments in the spirit of :ref:`virtualenv`.
``#!/usr/bin/env python`` and special ``__main__.py``, and are designed to make
deployment of Python applications as simple as ``cp``.


.. _Pipenv:

Pipenv
======

`Docs <http://docs.pipenv.org/en/latest/>`__ |
`Source <https://github.com/kennethreitz/pipenv>`__ |
`Issues <https://github.com/kennethreitz/pipenv/issues>`__ |
`PyPI <https://pypi.python.org/pypi/pipenv>`__

Pipenv is a project that aims to bring the best of all packaging worlds to the
Python world. It harnesses :ref:`Pipfile`, :ref:`pip`, and :ref:`virtualenv`
into one single toolchain. It features very pretty terminal colors.


.. _spack:

Spack
Expand Down
5 changes: 5 additions & 0 deletions source/new-tutorials/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,8 @@ New Tutorials
.. warning:: This section is for work-in-progress tutorials! These will
eventually be promoted to the :doc:`/tutorials/index` section.
Do not link to these pages!

.. toctree::
:maxdepth: 1

installing-and-using-packages
193 changes: 193 additions & 0 deletions source/new-tutorials/installing-and-using-packages.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,193 @@
Installing and using packages
=============================

This tutorial walks you through installing and using Python packages. It will
show you how to install and use the necessary tools and make strong
recommendations on best practices. Keep in mind that Python is used for a great
many different purposes, and precisely how you want to manage your dependencies
may change based on how you decide to publish your software. The guidance
presented here is most directly applicable to the development and deployment of
network services (including web applications), but is also very well suited to
managing development and testing environments for any kind of project.

.. Note:: This guide is written for Python 3, however, these instructions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This renders as two lines of unformatted text in the preview, is that a quirk of the preview or a bug (or intentional)? It looks like this:

Note

This guide is written for Python 3, however, these instructions should work fine on Python 2.7.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah github's preview is garbage

screen shot 2017-08-30 at 11 17 20 am

should work fine on Python 2.7.


Make sure you've got Python & pip
---------------------------------

Before you go any further, make sure you have Python and that it's avalable
from your command line. You can check this by simply running:

.. code-block:: bash

python --version

You should get some output like ``3.6.2``. If you do not have Python, please
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given your discussion with @kennethreitz , perhaps we could abuse the |release| notation to make it easy to update the Python version that we use for examples in one central place?

Alternatively, we could set rst_prolog to define a custom substitution field: https://stackoverflow.com/questions/36320992/sphinx-documentation-variables/36331678#36331678

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably a good place to mention the following error with a caveat like "If you're new to programming and the command line in general, you may see an error like this":

>>> python
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'python' is not defined

and explain that all of the code blocks in the tutorial are intended to be run at an operating system prompt, rather than at the interactive Python prompt.

https://opentechschool.github.io/python-beginners/en/getting_started.html#what-is-python-exactly was the best tutorial I could find on the exact details of how to open one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried the substitution but it doesn't work in code blocks or inline code markup, so it doesn't help here. :(

Great suggestion about the error. Added. :)

install the latest 3.x version from `python.org`_ or refer to the
`Installing Python`_ section of the Hitchhiker's Guide to Python.

.. Note:: If you're newcomer and you get an error like this:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: Missing "a" before newcomer

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, done.


.. code-block:: python

>>> python
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'python' is not defined

It's because this command is intended to be run in a *shell* (also called
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might it be worth emphasising that this note also applies to all the other commands in the tutorial? For example:

It's because this command, and the other suggested commands in this tutorial, are intended ...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

a *terminal* or *console*). See the Python for Beginners
`getting started tutorial`_ for an introduction to using your operating
system's shell and interacting with Python.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

love this


Additionally, you'll need to make sure you have :ref:`pip` available. You can
check this by running:

.. code-block:: bash

pip --version

If you installed Python from source, with an installer from `python.org`_, or
via `Homebrew`_ you should already have pip. If you're on Linux and installed
using your OS package manager, you may have to install pip separately, see
:doc:`/guides/installing-using-linux-tools`.

.. _getting started tutorial: https://opentechschool.github.io/python-beginners/en/getting_started.html#what-is-python-exactly
.. _python.org: https://python.org
.. _Homebrew: https://brew.sh
.. _Installing Python: http://docs.python-guide.org/en/latest/starting/installation/


Installing Pipenv
-----------------

:ref:`Pipenv` is a dependency manager for Python projects. If you're familiar
with Node.js' `npm`_ or Ruby's `bundler`_, it is similar in spirit to those
tools. While :ref:`pip` can install Python packages, Pipenv is recommended as
it's a higher-level tool that simplifies dependency management for common use
cases.

Use ``pip`` to install Pipenv:

.. code-block:: python

pip install --user pipenv
Copy link
Contributor

@kennethreitz kennethreitz Aug 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.. code-block:: bash

    $ pip install --user pipenv



.. Note:: This does a `user installation`_ to prevent breaking any system-wide
packages. If ``pipenv`` isn't available in your shell after installation,
you'll need to add the `user base`_'s ``bin`` directory to your ``PATH``.
You can find the user base by running ``python -m site`` which will print
site information including the user base. For example, on Linux this will
return ``USER_BASE: '~/.local'`` so you'll need to add ``~/.local/bin`` to
your ``PATH``. On Linux and macOS you can set your ``PATH`` permanently
by `modifying ~/.profile`_. On Windows you can set the user
Copy link
Contributor

@kennethreitz kennethreitz Aug 30, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

`` around ~/.profile ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rst doesn't like it. :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, the annoyances or ReST. It's also impossible to have both bold and italics at the same time!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the worst option except for all of the other options.

``PATH`` permanently in the `Control Panel`_.

.. _npm: https://www.npmjs.com/
.. _bundler: http://bundler.io/
.. _user base: https://docs.python.org/3/library/site.html#site.USER_BASE
.. _user installation: https://pip.pypa.io/en/stable/user_guide/#user-installs
.. _modifying ~/.profile: https://stackoverflow.com/a/14638025
.. _Control Panel: https://msdn.microsoft.com/en-us/library/windows/desktop/bb776899(v=vs.85).aspx

Installing packages for your project
------------------------------------

Pipenv manages dependencies on a per-project basis. To install packages,
change into your project's directory (or just an empty directory for this
tutorial) and run:

.. code-block:: bash

cd myproject
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really like having $ preceding shell commands

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But I can understand why you'd want to not have that, if you don't want Windows users to feel alienated. I don't think it harms them, though.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on user research from cloud.google.com we determined that users indeed do get tripped up on this (as they tend to copy/paste). There's a css trick to add the $ in the margin so that it's not picked up on copy, but that would need to be done separately from this PR. The blocks are all annotated as bash, so if we do decide to do it at a later date it'll be easy.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha!

pipenv install requests

Pipenv will install the excellent `Requests`_ library and create a ``Pipfile``
for you in your project's directory. The :ref:`Pipfile` is used to track which
dependencies your project needs in case you need to re-install them, such as
when you share your project with others. You should get output similar to this
(although the exact paths shown will vary):

.. code-block:: text

Creating a Pipfile for this project...
Creating a virtualenv for this project...
Using base prefix '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6'
New python executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python3.6
Also creating executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python
Installing setuptools, pip, wheel...done.

Virtualenv location: ~/.local/share/virtualenvs/tmp-agwWamBd
Installing requests...
Collecting requests
Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting idna<2.7,>=2.5 (from requests)
Using cached idna-2.6-py2.py3-none-any.whl
Collecting urllib3<1.23,>=1.21.1 (from requests)
Using cached urllib3-1.22-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
Using cached chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
Using cached certifi-2017.7.27.1-py2.py3-none-any.whl
Installing collected packages: idna, urllib3, chardet, certifi, requests
Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22

Adding requests to Pipfile's [packages]...
P.S. You have excellent taste! ✨ 🍰 ✨

.. _Requests: https://python-requests.org


Using installed packages
------------------------

Now that Requests is installed you can create a simple ``main.py`` file to
use it:

.. code-block:: python

import requests

response = requests.get('https://httpbin.org/ip')

print('Your IP is {0}'.format(response.json['origin']))

Then you can run this script using ``pipenv run``:

.. code-block:: bash

pipenv run python main.py

You should get output similar to this:

.. code-block:: text

Your IP is 8.8.8.8

Using ``pipenv run`` ensures that your installed packages are available to
your script. It's also possible to spawn a new shell that ensures all commands
have access to your installed packages with ``pipenv shell``.


Next steps
----------

Congratulations, you now know how to install and use Python packages! ✨ 🍰 ✨
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

;)


There's more resources you can look at to learn about installing and using
Python packages:

.. TODO Link to additional guides and resources.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make these TODOs visible in the rendered version so there aren't just blank spaces while the link targets are being worked on?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.


If you find this approach isn't working well for you or your use case, you may
want to explore these other approaches:

.. TODO Link to alternatives
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested initial set of links:

  • briefcase: briefcase is a tool that focuses on defining applications as Python packages, and then using that package definition to generate native application installers for popular client operating systems
  • pex: PEX is a single-file distribution format for executable Python virtual environments as zip archives containing a __main__.py file. Their main runtime requirement is that a suitable Python interpreter be available on the system running the application.
  • pipsi: Similar to pex, pipsi aims to run applications in self-contained virtual environments. Unlike pex, it generates these virtual environments dynamically inside the user's home directory for applications available as Python packages.
  • pew: the underlying virtual environment manager used by pipenv (this is a lower level tool that offers more flexibility in where, when, and how virtual environments are created and updated)
  • pip-tools: automated requirements.txt file management that distinguishes between abstract dependency combinations that are expected to work, and the specific pinned versions that will be tested and/or deployed (this is a lower level tool that offers more flexibility in when and how dependencies are installed and updated)
  • conda: conda is a "cross-platform platform" that manages the Python interpreter and external binary dependencies in addition to managing Python modules. It is most useful in situations where it is easier to ensure that the conda package manager is preinstalled than it is to ensure that a suitable Python version is available.

And then make a note that the above list only covers cross-platform tools, and there are also other platform specific options available if users are only interested in a limited set of platforms.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want to put all of this in this doc, but it does make sense for a general overview guide. Moved these to #367 (comment) for tracking. :)


If you're interesting in creating and distributing Python packages, see the
tutorial on packaging and distributing packages.

.. TODO Link to packaging tutorial when it exists.