Embeddable Python distribution hack: include code outside the embedded distro path

Here’s a method for tweaking the “embeddable zip file” distributions of Python to allow you to import code outside of the extracted Python environment. This may be useful for a number of reasons, such as wanting to keep your code separate from the Python “core” code.

In my particular case, I have a project I’m trying to port to Python 3. However, user content may reference and run via Python 2 in subprocesses. Simply unzipping the embedded environment into the root of my project won’t work since that’ll result in Python 2 trying to import Python 3 .pyd files from the current working directory. Thus, I need to keep Python in a separate directory. However, moving my code into that separate directory is not ideal either since so many assumptions have been made regarding where the sources lie. Ideally I should be able to keep my code where it is and just have the Python executor bundled in a subdirectory with my program. But that doesn’t work out of the box since the embedded Python environment is configured not to pull from the PATH, nor any sitecustomize or usercustomize modules.

The official documented solution for this use case is to write a custom launcher which includes the appropriate paths. I can do that if I must, but I’d prefer a purely Python solution to avoid needing to install Visual Studio to build my project.

Thankfully, I found a way. Here it is.

  1. Write a placeholder module with the same name as the top-level package you want to import, and place it in your embedded Python directory.
  2. Have the placeholder module add the directory containing the real package to the beginning of sys.path.
  3. Remove the placeholder module from sys.modules.
  4. Repeat the import of the top-level package in question. This should pull the real package in.

Put in code, assuming the real package is located in the parent directory of the embedded Python with the name “mypackage”, here’s what to write:

import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), os.pardir)))
del sys.modules['mypackage']
import mypackage

(For those worried about license, consider this trivial snippet to be public domain.)

Note that even if you are importing a subpackage such as mypackage.foo, Python will first try to import mypackage.py or mypackage/__init__.py. That’s why this works – we put a placeholder which does our path tweaks and then replaces itself with the correct version, after which Python can find the subpackages and modules within that package without issues.

Hope this helps someone! And if there is a better way to accomplish this, please let me know.

Leave a Reply

Your email address will not be published. Required fields are marked *