Releasing GIL in C extension:
Give a C extension code and one needs to concurrently execute it with other threads in the Python interpreter. For this Global Interpreter Lock (GIL) has to be released and reacquired.
Code #1 : Releasing and Reacquiring GIL by inserting following macros
#include "Python.h" ... PyObject *pyfunc(PyObject *self, PyObject *args) { ... Py_BEGIN_ALLOW_THREADS // Threaded C code. // Must not use Python API functions ... Py_END_ALLOW_THREADS ... return result; } |
Mixing Threads from C and Python:
Considering a situation, that a given program involves a mix of C, Python and threads. But some of the given threads are created from C outside the control of the Python interpreter and also certain threads utilize functions in the Python C API.
What’s solution then?
If there is a mix of Python, C and threads, proper initialization and management of Python’s Global Interpreter Lock (GIL) is important. Given code below explains the situation, their code can be used anywhere (prior to the creation of threads) in the C program.
Code #2 :
#include <Python.h> ... if (!PyEval_ThreadsInitialized()) { PyEval_InitThreads(); } ... |
For any C involving Python objects or the Python C API, GIL needs to be acquired and released first. This can be performed using PyGILState_Ensure() and PyGILState_Release() as in the code given below. Every call to PyGILState_Ensure() must have a matching call to PyGILState_Release().
Code #3 :
... // Make sure we own the GIL // Use functions in the interpreter PyGILState_STATE state = PyGILState_Ensure(); ... // Restore previous GIL state and return PyGILState_Release(state); ... |
Note :
It is not so easy to have so much things going at once, involving a mix of a Python code, C code and their threads. To perform so, it is to be taken care that the interpreter is properly initialized and that C code involving the interpreter has the proper GIL management calls, it all should work.
Also, the PyGILState_Ensure() call does not immediately preempt or interrupt the interpreter. If other code is currently executing, this function will block until that code decides to release the Global Interpreter Lock (GIL).
