Py Packages

Early release - missing features, and will not work for some dependencies

This tool implements PEP 582 -- Python local packages directory. It manages dependencies, keeping them isolated in the project directory, and runs python in an environment which uses this directory. Per PEP 582, dependencies are stored in the project directory → __pypackages__ → 3.7(etc) → lib. A virtual environment is created in the same diretory as lib, and is used transparently.

Python ≥ 3.4 is required.

Installation

There are 2 main ways to install: - Download a binary from the releases page. On Debian or Ubuntu, download and run This deb.

On other Operating systems, download the appropriate binary, and place it somewhere accessible by the system path. For example, place it under /usr/bin in linux, or ~\AppData\Local\Programs\Python\Python37\bin in Windows.

Use

Example contents: ```toml [tool.pypackage] py_version = "3.7" name = "runcible" version = "0.1.0" author = "John Hackworth"

[tool.pypackage.dependencies] numpy = "^1.16.4" diffeqpy = "1.1.0" `` The[tool.pypackage]section is used for metadata, and isn't required unless building and distributing a package. The[tool.pypyackage.dependencies'] section contains all dependencies, and is an analog torequirements.txt. For details on how to specify dependencies in thisCargo.toml`-inspired semvar format, reference this guide.

Example use

Managing dependencies:

Running REPL and Python files in the environment:

Building and publishing:

How dependencies are resolved

Running pypackage install loads the project's requirements from pyproject.toml. Adding a package name via the CLI, eg pypackage install matplotlib simply adds that requirement before proceeding. Compatible versions of dependencies are determined using info from the PyPi Warehosue (available versions, and hash info), and the pydeps database. We use pydeps, which is built specifically for this project, due to inconsistent dependency information stored on pypi. A dependency graph is built using this cached database. We attempt to use the newest compatible version of each package, but older ones are used if needed to satisfy the dependency occuring with different requirements.

This tool downloads and unpacks wheels from pypi, or builds wheels from source if none are availabile. It verifies the integrity of the downloaded file against that listed on pypi using SHA256, and the exact versions used are stored in a lock file.

If a lockfile already exists, package versions stored in it which are compatible with those in pyproject.toml and resolved subdependencies are used.

Important caveat: There appears to be no way install multiple versions of a package simultaneously without renaming them; this is a factor when encountering incompatible sub-dependencies. Perhaps this can be sorted around through behind-the-scenes renaming and import-line edits, but for now may result in unresolvable trees.

When a dependency is removed from pyproject.toml, it, and its subdependencies not also required by other packages are removed from the __pypackages__ folder.

Why?

Using a Python installation directly when installing dependencies can become messy. If using a system-level Python, which is ubiqutious in Linux, altering dependencies may break the OS. Virtual environments correct this, but are cumbersome to use. An example workflow:

Setup: bash cd ~/.virtualenvs python -m venv "myproject" cd myproject/bin source activate cd ~/myproject python install -r requirements.txt deactivate Use: bash cd ~/.virtualenvs/myproject/bin source activate cd ~/myproject python main.py deactivate

This signifcantly impacts the usability of Python, especially for new users. IDEs like PyCharm abstract this away, but are a specific solution to a general problem. See this section of PEP 582.

If multiple versions of Python are installed, verifying you're using the one you want may be difficult.

When building and deploying packages, a set of other, redudant files are traditionally used: setup.py, setup.cfg, and MANIFEST.in

Why add another Python dependency manager?

Pipenv and Poetry both address this problem. Some reasons why this tool is different:

Conda addresses this as well, but focuses on maintining a separate repository of binaries from PyPi.

Not-yet-implemented

Building and uploading your project to PyPi.

In order to build and publish your project, additional info is needed in pyproject.toml, that mimics what would be in setup.py. Example: ```toml [tool.pypackage] name = "everythingkiller" pyversion = "3.6" version = "0.1.0" author = "Fraa Erasmas" authoremail = "raz@edhar.math" description = "Small, but packs a punch!" homepage = "https://everything.math" repository = "https://github.com/raz/everythingkiller" license = "MIT"

[tool.pypackage.dependencies] numpy = "^1.16.4" django = "2.0.0" ```

Building this from source

If you’d like to build from source, download and install Rust, clone the repo, and in the repo directory, run cargo build --release.

Ie on Linux:

bash curl https://sh.rustup.rs -sSf | sh git clone https://github.com/david-oconnor/pypackage.git cd pypackage cargo build --release

Updating

If installed via Cargo, run cargo install pypackage --force.

Contributing

If you notice unexpected behavior or missing features, please post an issue, or submit a PR. There are probably multiple problems with the dependency resolver. If you see unexpected behavior, it's probably a bug! Post an issue listing the dependencies that did not install correctly.

Dependency cache repo:

Gotchas

References