Metadata-Version: 2.4
Name: multiprocessing-logging
Version: 0.4.0
Summary: Logger for multiprocessing applications
Home-page: https://github.com/jruere/multiprocessing-logging
Author: Javier Ruere
Author-email: javier@ruere.com.ar
License: LGPLv3
Keywords: multiprocessing,logging,logger,handler
Platform: POSIX
Classifier: Development Status :: 5 - Production/Stable
Classifier: Intended Audience :: Developers
Classifier: License :: OSI Approved :: GNU Lesser General Public License v3 or later (LGPLv3+)
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: Python :: 3.13
Classifier: Programming Language :: Python :: 2
Classifier: Programming Language :: Python :: 2.7
Classifier: Programming Language :: Python :: Implementation :: CPython
Classifier: Operating System :: POSIX
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Requires-Python: >=2.7
Description-Content-Type: text/markdown
License-File: LICENSE
Provides-Extra: test
Requires-Dist: mock~=3.0; python_version < "3.3" and extra == "test"
Dynamic: author
Dynamic: author-email
Dynamic: home-page
Dynamic: license-file
Dynamic: platform

# multiprocessing-logging

[![Supported Python versions](https://img.shields.io/pypi/pyversions/multiprocessing-logging.svg)](https://pypi.python.org/pypi/multiprocessing-logging/)
[![License](https://img.shields.io/pypi/l/multiprocessing-logging.svg)](https://pypi.python.org/pypi/multiprocessing-logging/)


When using the `multiprocessing` module, logging becomes less useful since
sub-processes should log to individual files/streams or there's the risk of
records becoming garbled.

This simple module implements a `Handler` that when set on the root
`Logger` will handle tunneling the records to the main process so that
they are handled correctly.

It's currently tested in Linux on CPython 2.7, CPython 3.9+, and PyPy 2.7 and 3.

Only works on POSIX systems and only Linux is supported. It does not work on Windows.

# Origin

This library was taken verbatim from a [StackOverflow post](http://stackoverflow.com/questions/641420/how-should-i-log-while-using-multiprocessing-in-python)
and extracted into a module so that I wouldn't have to copy the code in every
project.

Later, several improvements have been contributed.

# Usage

Before you start logging but after you configure the logging framework (maybe with `logging.basicConfig(...)`), do the following:

```py
import multiprocessing_logging

multiprocessing_logging.install_mp_handler()
```

and that's it.

## With multiprocessing.Pool

When using a Pool, make sure `install_mp_handler` is called before the Pool is instantiated, for example:

```py
import logging
from multiprocessing import Pool
from multiprocessing_logging import install_mp_handler

logging.basicConfig(...)
install_mp_handler()
pool = Pool(...)
```

# Problems
The approach of this module relies on
[fork](https://docs.python.org/3.9/library/multiprocessing.html#multiprocessing.set_start_method)
being used to create new processes. This start method
[is basically unsafe when also using threads](https://bugs.python.org/issue37429),
as this module does.

The consequence is that there's a low probability of the application hanging
when creating new processes.

As a palliative, don't continuously create new processes. Instead, create a
Pool once and reuse it.
