RecursionError pyinstaller – Artificial Intelligence

The project is connected with artificial intelligence. Reinstalling libraries or other versions did not help.

There is a recursion error when packing, exactly when you specify that you need to pack the torchvision library. Namely:

pyinstaller --collect-all torchvision "/path/main-file.py"

I checked all the code, the problem occurs because of the mention of the import torch library. I checked the code thoroughly, and it just came out that I commented on parts of the code or even the entire code, leaving only the libraries. I figured it out by isklbcheniya.

Error log:

================================================= ===========
A RecursionError (maximum recursion depth exceeded) occurred.
For working around please follow these instructions
================================================= ===========
 
1. In your program's .spec file add this line near the top::
 
     import sys ; sys.setrecursionlimit(sys.getrecursionlimit() * 5)
 
2. Build your program by running PyInstaller with the .spec file as
   argument::
 
     pyinstaller myprog.spec
 
3. If this fails, you most likely hit an endless recursion in
   PyInstaller. Please try to track this down has far as possible,
   create a minimal example so we can reproduce and open an issue at
   https://github.com/pyinstaller/pyinstaller/issues following the
   instructions in the issue template. many thanks.
 
Explanation: Python's stack-limit is a safety-belt against endless recursion,
eating up memory. PyInstaller imports modules recursively. If the structure
how modules are imported within your program is awkward, this leads to the
nesting being too deep and hitting Python's stack-limit.
 
With the default recursion limit (1000), the recursion error occurs at about
115 nested imported, with limit 2000 at about 240, with limit 5000 at about
660.

info:

pyinstaller 5.7.0,
pyinstaller-hooks-contrib 2022.15,
torch 1.13.1,
torchvision 0.14.1,
Python 3.10.6
Desktop: Xfce 4.16.0 Distro: Linux Mint 21 Vanessa base: Ubuntu 22.04 jammy

If you follow the proposed decision points in the log, then after packaging the application will start, but it is not possible to close it using the standard method.

Packaging works if you add to main.spec:

import sys
sys.setrecursionlimit(sys.getrecursionlimit() + 30)

But I described the consequences above.

Without the proposed solution, I initially indicated in the main script of the project: sys.setrecursionlimit (10000) I tried to lower it to 1000 – it did not help, I will try to increase it.

Could there be a reason due to the fact that the project is large and the calculation is used in the project more than once.

Log terminal:

317 INFO: PyInstaller: 5.7.0
317 INFO: Python: 3.10.6
319 INFO: Platform: Linux-5.15.0-60-generic-x86_64-with-glibc2.35
320 INFO: wrote /home/odinson/PycharmProjects/Panning/Video/all/main.spec
322 INFO: UPX is not available.
323 INFO: Extending PYTHONPATH with paths
['/home/odinson/PycharmProjects/Panning/Video/all',
 '/home/odinson/PycharmProjects/Panning/Video/all/yolov5']
pygame 2.1.2 (SDL 2.0.16, Python 3.10.6)
Hello from the pygame community. https://www.pygame.org/contribute.html
1109 INFO: checking Analysis
1109 INFO: Building Analysis because Analysis-00.toc is non existent
1109 INFO: Initializing module dependency graph...
1111 INFO: Caching module graph hooks...
1121 WARNING: Several hooks defined for module 'numpy'. Please take care they do not conflict.
1128 INFO: Analyzing base_library.zip ...
2661 INFO: Loading module hook 'hook-encodings.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
3551 INFO: Loading module hook 'hook-heapq.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
4249 INFO: Loading module hook 'hook-pickle.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
8649 INFO: Caching module dependency graph...
8863 INFO: running Analysis Analysis-00.toc
8941 INFO: Analyzing /home/odinson/PycharmProjects/Panning/Video/all/main.py
9487 INFO: Loading module hook 'hook-PIL.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
9633 INFO: Loading module hook 'hook-PIL.Image.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
10682 INFO: Loading module hook 'hook-numpy.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/numpy/_pyinstaller'...
11565 INFO: Loading module hook 'hook-numpy._pytesttester.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
12199 INFO: Loading module hook 'hook-difflib.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
12491 INFO: Loading module hook 'hook-multiprocessing.util.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
12697 INFO: Loading module hook 'hook-xml.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
13436 INFO: Loading module hook 'hook-platform.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
13676 INFO: Loading module hook 'hook-sysconfig.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
14339 INFO: Loading module hook 'hook-xml.etree.cElementTree.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
15427 INFO: Loading module hook 'hook-packaging.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
15482 INFO: Loading module hook 'hook-PIL.ImageFilter.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
16106 INFO: Loading module hook 'hook-torch.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
16466 INFO: Loading module hook 'hook-pkg_resources.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
20955 INFO: Processing pre-safe import module hook six.moves from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-six.moves.py'.
22528 INFO: Loading module hook 'hook-sqlite3.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
23107 INFO: Processing pre-safe import module hook tensorflow from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/pre_safe_import_module/hook-tensorflow.py'.
23155 INFO: Loading module hook 'hook-tensorflow.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
2023-02-24 10:58:38.868196: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-24 10:58:39.997247: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2023-02-24 10:58:39.997357: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2023-02-24 10:58:39.997378: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-02-24 10:58:47.118986: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-02-24 10:58:48.254075: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2023-02-24 10:58:48.254181: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2023-02-24 10:58:48.254196: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
34543 INFO: Processing pre-find module path hook distutils from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks/pre_find_module_path/hook-distutils.py'.
34916 INFO: Loading module hook 'hook-distutils.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
35302 INFO: Loading module hook 'hook-xml.dom.domreg.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
45894 INFO: Processing pre-safe import module hook urllib3.packages.six.moves from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-urllib3.packages.six.moves.py'.
46690 INFO: Loading module hook 'hook-charset_normalizer.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
46914 INFO: Loading module hook 'hook-certifi.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
52242 INFO: Loading module hook 'hook-pandas.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
53868 INFO: Loading module hook 'hook-pytz.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
55878 INFO: Loading module hook 'hook-scipy.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
56376 INFO: Loading module hook 'hook-matplotlib.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
56862 INFO: Processing pre-safe import module hook gi from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.py'.
56949 INFO: Loading module hook 'hook-gi.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
57053 INFO: Processing pre-safe import module hook gi.repository.GLib from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks/pre_safe_import_module/hook-gi.repository.GLib.py'.
57934 INFO: Loading module hook 'hook-jinja2.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
59090 INFO: Loading module hook 'hook-matplotlib.backends.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
61255 INFO: Loading module hook 'hook-IPython.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
63139 INFO: Loading module hook 'hook-pygments.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
65766 INFO: Loading module hook 'hook-lib2to3.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
67179 INFO: Loading module hook 'hook-wcwidth.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
69467 INFO: Loading module hook 'hook-jedi.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
70433 INFO: Loading module hook 'hook-parso.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
74493 INFO: Loading module hook 'hook-scipy.linalg.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
75033 INFO: Loading module hook 'hook-scipy.sparse.csgraph.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
77484 INFO: Loading module hook 'hook-scipy.spatial.transform.rotation.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
81973 INFO: Loading module hook 'hook-pandas.io.formats.style.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
84467 INFO: Loading module hook 'hook-pandas.plotting.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...
84918 INFO: Loading module hook 'hook-numba.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
85203 INFO: Loading module hook 'hook-llvmlite.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/_pyinstaller_hooks_contrib/hooks/stdhooks'...
87354 INFO: Loading module hook 'hook-distutils.util.py' from '/home/odinson/PycharmProjects/Panning/Video/all/venv/lib/python3.10/site-packages/PyInstaller/hooks'...

=============================================================
A RecursionError (maximum recursion depth exceeded) occurred.
For working around please follow these instructions
=============================================================

1. In your program's .spec file add this line near the top::

     import sys ; sys.setrecursionlimit(sys.getrecursionlimit() * 5)

2. Build your program by running PyInstaller with the .spec file as
   argument::

     pyinstaller myprog.spec

3. If this fails, you most probably hit an endless recursion in
   PyInstaller. Please try to track this down has far as possible,
   create a minimal example so we can reproduce and open an issue at
   https://github.com/pyinstaller/pyinstaller/issues following the
   instructions in the issue template. Many thanks.

Explanation: Python's stack-limit is a safety-belt against endless recursion,
eating up memory. PyInstaller imports modules recursively. If the structure
how modules are imported within your program is awkward, this leads to the
nesting being too deep and hitting Python's stack-limit.

With the default recursion limit (1000), the recursion error occurs at about
115 nested imported, with limit 2000 at about 240, with limit 5000 at about
660.

UPD: I change the description of the question, as I find some more details. sorry