Amazon Web Services Question posted in
The official Amazon Web Services documentation can be found here.

Amazon web services – ERROR : XGBoostError: XGBoost Library (libxgboost.so) could not be loaded

Full Error:

[ERROR] XGBoostError: XGBoost Library (libxgboost.so) could not be loaded.
Likely causes:

  • OpenMP runtime is not installed (vcomp140.dll or libgomp-1.dll for Windows, libgomp.so for UNIX-like OSes)
  • You are running 32-bit Python on a 64-bit OS
    Error message(s): [‘libgomp.so.1: cannot open shared object file: No such file or directory’]

Infrastructure:

  • AWS Lambda Function

  • I am importing the XGBoost in the Lambda function and I get above error when I Try to run it.

  • This error has started when I upgraded the Lambda Function Runtime version from Python 3.6 to Python 3.9

  • I am importing the XGBoost from the zip I have kept in the S3 bucket and Lambda has a code to import it.

  • I also tried packaging the libgomp.so.1 with the xgboost zip but that also did not worked.

Tags:

2

Answers


  1. 0

    try using an amazon linux compatible prebuilt wheel

    you can using this container as it has same env as lambda

    docker run -it --rm amazonlinux:2

    in docker container install requirements
    make sure to use venv

    yum install -y gcc gcc-c++ python3 python3-devel
    Login or Signup to reply.
  2. 0

    use the correct amazon linux base,
    aws python 3.9 runtime is based on Amazon linux 2
    so you need to build and package dependencies in the same environment

    run the following to start with the correct environment

    docker run -it --rm amazonlinux:2

    when you are inside the docker container run this

    yum update -y
yum install -y gcc gcc-c++ make python3 python3-devel
python3 -m venv /app/venv
source /app/venv/bin/activate
pip install --upgrade pip
pip install xgboost

    now do this

    find / -name "libgomp.so.1"

    Copy this library to your Lambda deployment package

    cp /usr/lib64/libgomp.so.1 /app/venv/lib/python3.9/site-packages/

    now zip the deployment package

    cd /app/venv/lib/python3.9/site-packages/
zip -r /tmp/xgboost_package.zip .

    now upload to S3 and deploy

    I tried producing your error, but yeah these things that i mentioned above are fixed

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search