skip to Main Content

For the last 5 days, I am trying to make Keras/Tensorflow packages work in R. I am using RStudio for installation and have used conda, miniconda, virtualenv but it crashes each time in the end. Installing a library should not be a nightmare especially when we are talking about R (one of the best statistical languages) and TensorFlow (one of the best deep learning libraries). Can someone share a reliable way to install Keras/Tensorflow on CentOS 7?

Following are the steps I am using to install tensorflow in RStudio.

Since RStudio simply crashes each time I run tensorflow::tf_config() I have no way to check what is going wrong.

enter image description here

devtools::install_github("rstudio/reticulate")
devtools::install_github("rstudio/keras") # This package also installs tensorflow
library(reticulate)
reticulate::install_miniconda()
reticulate::use_miniconda("r-reticulate")
library(tensorflow)
tensorflow::tf_config() **# Crashes at this point**

sessionInfo()


R version 3.6.0 (2019-04-26)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] tensorflow_2.7.0.9000 keras_2.7.0.9000      reticulate_1.22-9000 

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.7      lattice_0.20-45 png_0.1-7       zeallot_0.1.0  
 [5] rappdirs_0.3.3  grid_3.6.0      R6_2.5.1        jsonlite_1.7.2 
 [9] magrittr_2.0.1  tfruns_1.5.0    rlang_0.4.12    whisker_0.4    
[13] Matrix_1.3-4    generics_0.1.1  tools_3.6.0     compiler_3.6.0 
[17] base64enc_0.1-3


Update 1
The only way RStudio does not crash while installing tensorflow is by executing following steps –

First, I created a new virtual environment using conda

conda create --name py38 python=3.8.0
conda activate py38
conda install tensorflow=2.4

Then from within RStudio, I installed reticulate and activated the virtual environment which I earlier created using conda

devtools::install_github("rstudio/reticulate")
library(reticulate)
reticulate::use_condaenv("/root/.conda/envs/py38", required = TRUE)
reticulate::use_python("/root/.conda/envs/py38/bin/python3.8", required = TRUE)
reticulate::py_available(initialize = TRUE)
ts <- reticulate::import("tensorflow")

As soon as I try to import tensorflow in RStudio, it loads the library /lib64/libstdc++.so.6 instead of /root/.conda/envs/py38/lib/libstdc++.so.6 and I get the following error –

Error in py_module_import(module, convert = convert) : 
  ImportError: Traceback (most recent call last):
  File "/root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/pywrap_tensorflow.py", line 64, in <module>
    from tensorflow.python._pywrap_tensorflow_internal import *
  File "/home/R/x86_64-redhat-linux-gnu-library/3.6/reticulate/python/rpytools/loader.py", line 39, in _import_hook
    module = _import(
ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /root/.conda/envs/py38/lib/python3.8/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)


Failed to load the native TensorFlow runtime.

See https://www.tensorflow.org/install/errors

for some common reasons and solutions.  Include the entire stack trace
above this error message when asking for help.

Here is what inside /lib64/libstdc++.so.6

> strings /lib64/libstdc++.so.6 | grep GLIBC

GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBCXX_3.4.11
GLIBCXX_3.4.12
GLIBCXX_3.4.13
GLIBCXX_3.4.14
GLIBCXX_3.4.15
GLIBCXX_3.4.16
GLIBCXX_3.4.17
GLIBCXX_3.4.18
GLIBCXX_3.4.19
GLIBC_2.3
GLIBC_2.2.5
GLIBC_2.14
GLIBC_2.4
GLIBC_2.3.2
GLIBCXX_DEBUG_MESSAGE_LENGTH

To resolve the library issue, I added the path of the correct libstdc++.so.6 library having GLIBCXX_3.4.20 in RStudio.

system('export LD_LIBRARY_PATH=/root/.conda/envs/py38/lib/:$LD_LIBRARY_PATH')

and, also

Sys.setenv("LD_LIBRARY_PATH" = "/root/.conda/envs/py38/lib")

But still I get the same error ImportError: /lib64/libstdc++.so.6: version `GLIBCXX_3.4.20'. Somehow RStudio still loads /lib64/libstdc++.so.6 first instead of /root/.conda/envs/py38/lib/libstdc++.so.6

Instead of RStudio, if I execute the above steps in the R console, then also I get the exact same error.

Update 2:
A solution is posted here

2

Answers


  1. Chosen as BEST ANSWER

    Update on 29 July, 2022 After months of solving this problem, I feel so stupid to have wasted time coding R on CentOS. The most popular and stable OS to code R is Ubuntu. By default, CentOS supports only the 3.6 version of R while the most stable current version of R is 4.2. With the default 3.6 version of R on CentOS, most of the libraries are outdated and they conflict with other libraries which are updated for R 4.2+. From my experience, you are going to avoid a lot of misery and frustration if you start coding R on Ubuntu. I am not sponsoring Ubuntu, the above statement is just from my experience and others might have different experiences.

    Original Answer Took me more than 15 days and I finally solved this problem.

    Boot up a clean CentOS 7 VM, install R and dependencies (taken from Jared's answer) -

    yum install epel-release
    yum install R
    yum install libxml2-devel
    yum install openssl-devel
    yum install libcurl-devel
    yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
    

    Now, create a conda environment

    yum install conda
    conda clean -a     # Clean cache and remove old packages, if you already have conda installed
    # Install all the packages together and let conda handle versioning. It is important to give a Python version while setting up the environment. Since Tensorflow supports python 3.9.0, I have used this version 
    conda create -y -n "tf" python=3.9.0 ipython tensorflow keras r-essentials r-reticulate r-tensorflow
    conda activate tf
    

    Open a new port (7878 or choose any port number you want) on the server to access RStudio with new conda environment libraries

    iptables -A INPUT -p tcp --dport 7878 -j ACCEPT
    /sbin/service iptables save
    

    then launch RStudio as follows -

    /usr/lib/rstudio-server/bin/rserver 
       --server-daemonize=0 
       --www-port 7878 
       --rsession-which-r=$(which R) 
       --rsession-ld-library-path=$CONDA_PREFIX/lib
    

    You will have your earlier environment intact on default port 8787 and a new environment with Tensorflow and Keras on 7878.

    The following code now works fine in RStudio

    install.packages("reticulate")
    install.packages("tensorflow")
    library(reticulate)
    library(tensorflow)
    ts <- reticulate::import("tensorflow")
    

  2. Perhaps my failed attempts will help someone else solve this problem; my approach:

    • boot up a clean CentOS 7 vm
    • install R and some dependencies
    sudo yum install epel-release
    sudo yum install R
    sudo yum install libxml2-devel
    sudo yum install openssl-devel
    sudo yum install libcurl-devel
    sudo yum install libXcomposite libXcursor libXi libXtst libXrandr alsa-lib mesa-libEGL libXdamage mesa-libGL libXScrnSaver
    
    • Download and install Anaconda via linux installer script
    • Create a new conda env
    conda init
    conda create --name tf
    conda activate tf
    conda install -c conda-forge tensorflow
    

    **From within this conda env you can import tensorflow in python without error; now to access tf via R

    • install an updated gcc via devtoolset
    sudo yum install centos-release-scl
    sudo yum install devtoolset-7-gcc*
    
    • attempt to use tensorflow in R via the reticulate package
    scl enable devtoolset-7 R
    install.packages("remotes")
    remotes::install_github('rstudio/reticulate')
    reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
    reticulate::repl_python()
    # This works as expected but the command "import tensorflow" crashes R
    # Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
    
    # Also tried:
    install.packages("devtools")
    devtools::install_github('rstudio/tensorflow')
    devtools::install_github('rstudio/keras')
    library(tensorflow)
    install_tensorflow() # "successful"
    tensorflow::tf_config()
    # Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
    
    • try older versions of tensorflow/keras
    devtools::install_github('rstudio/[email protected]')
    devtools::install_github('rstudio/[email protected]')
    library(tensorflow)
    tf_config()
    # Error: *** caught segfault *** address 0xf8, cause 'memory not mapped'
    
    • Try an updated version of R (v4.0)
    # deactivate conda
    sudo yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm 
    export R_VERSION=4.0.0
    curl -O https://cdn.rstudio.com/r/centos-7/pkgs/R-${R_VERSION}-1-1.x86_64.rpm
    sudo yum install R-${R_VERSION}-1-1.x86_64.rpm
    
    scl enable devtoolset-7 /opt/R/4.0.0/bin/R
    install.packages("devtools")
    devtools::install_github('rstudio/reticulate')
    reticulate::use_condaenv("tf", conda = "~/anaconda3/bin/conda")
    reticulate::repl_python()
    # 'import tensorflow' resulted in "core dumped"
    

    I guess the issue is with R/CentOS, as you can import and use tensorflow via python normally, but I’m not sure what else to try.

    I would also like to say that I had no issues with Ubuntu (which is specifically supported by tensorflow, along with macOS and Windows), and I came across these docs that might be some help: https://wiki.hpcc.msu.edu/display/ITH/Installing+TensorFlow+using+anaconda / https://wiki.hpcc.msu.edu/pages/viewpage.action?pageId=22709999

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search