Byte-compile Python libraries

PythonServer Side ProgrammingProgramming

Python is an interpreter based language. However it internally compiles the source code to byte code when a script (.py extension) is run and afterwards the bytecode version is automatically removed. When a module (apart from the precompiled built-in modules) is first imported, its compiled version is also automatically built but saved with .pyc extension in __pycache__ folder. Subsequent calls to import same module again won't recompile the module instead uses the one already built.

However, a Python script file with .py extension can be compiled expilicitly without running it. The 'py_compile' module contains 'compile()' function for that purpose. Name of resultant .pyc file is appended by version number of Python. In case of Python 3.7, the name has 'cpython-37' suffix.

You won't need this module normally. However it can be useful when installing modules for shared use, especially if some users don't have permission to write the byte-code cache files in the directory containing the source code.

The module has a command line interface. It can be imported using –m switch as follows

E:\python37>python -m py_compile

This will create and store 'hello.cpython-37.pyc' file in __pycache__ subfolder.

The byte-compilation can also be done programmatically using compile() function in py_compile module.

import py_compile

The bytecode file with .pyc extension can be directly executed from command line just as normal script with .py extension.

E:\python37>python __pycache__/hello.cpython-37.pyc

However, it can be executed by interpreter of same version with which the bytecode file was compiled. If we try to run it with any other version of Python following error will be displayed.

C:\python36>python e:hello.cpython-37.pyc
RuntimeError: Bad magic number in .pyc file

The py_compile module has another function main() which compiles several source files. The files named as arguments are compiled and the resulting byte-code is cached in the normal manner. This function does not search a directory structure to locate source files; it only compiles files named explicitly.

The module also defines PyCompileError exception which is raised when an error occurs while attempting to compile the file. (This will be raised only if doraise option is set to True for compile() function.

The optimize option of compile() function controls the optimization level and is passed to the built-in compile() function. The default of -1 selects the optimization level of the current interpreter.

Published on 17-Jan-2019 11:42:54