skip to Main Content

I am using Pychromeless repo with success at AWS lambda.

Now I need to use NaCL dependency, to decrypt a string, but I am getting

Unable to import module 'lambda_function': /var/task/lib/nacl/_sodium.abi3.so

with a complement of

invalid ELF header
 

when running the function on AWS Lambda.

I know it is a problem specific related to AWS Lambda environment, because I can run the function on my Mac inside docker.

Here’s my requirements.txt file

boto3==1.6.18
botocore==1.9.18
selenium==2.53.6
chromedriver-install==1.0.3
beautifulsoup4==4.6.1
certifi==2018.11.29
chardet==3.0.4
editdistance==0.5.3
future==0.17.1
idna==2.7
python-telegram-bot==10.1.0
requests==2.19.1
soupsieve==1.7.3
urllib3==1.23
PyNaCl==1.3.0

Here is the dockerfile

FROM lambci/lambda:python3.6
MAINTAINER [email protected]

USER root

ENV APP_DIR /var/task

WORKDIR $APP_DIR

COPY requirements.txt .
COPY bin ./bin
COPY lib ./lib

RUN mkdir -p $APP_DIR/lib
RUN pip3 install -r requirements.txt -t /var/task/lib

And the makefile:

clean:
    rm -rf build build.zip
    rm -rf __pycache__

fetch-dependencies:
    mkdir -p bin/

    # Get chromedriver
    curl -SL https://chromedriver.storage.googleapis.com/2.37/chromedriver_linux64.zip > chromedriver.zip
    unzip chromedriver.zip -d bin/

    # Get Headless-chrome
    curl -SL https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-37/stable-headless-chromium-amazonlinux-2017-03.zip > headless-chromium.zip
    unzip headless-chromium.zip -d bin/

    # Clean
    rm headless-chromium.zip chromedriver.zip

docker-build:
    docker-compose build

docker-run:
    docker-compose run lambda src/lambda_function.lambda_handler


build-lambda-package: clean fetch-dependencies
    mkdir build
    cp -r src build/.
    cp -r bin build/.
    cp -r lib build/.
    pip install -r requirements.txt -t build/lib/.
    cd build; zip -9qr build.zip .
    cp build/build.zip .
    rm -rf build

Without the decryption part, the code works great. So the issue is 100% related to PyNaCl.

Any help on solving this?

2

Answers


  1. Chosen as BEST ANSWER

    Ok, this is how I did it. I had to build everything on an EC2 AMI Linux 2 instance.

    amzn2-ami-hvm-2.0.20190823.1-x86_64-gp2 (ami-0a1f49a762473adbd)

    After launching the instance, I use this script to install Python 3.6 (and pip) and to create and activate a virtual environment.

    For the docker part, I followed this tutorial, not without some troubles on the way (had to

    sudo yum install polkit

    and

    sudo usermod -a -G docker ec2-user

    and

    reboot

    and

    sudo curl -L https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose

    and

    sudo chmod +x /usr/local/bin/docker-compose).

    But anyway, I manage to work with docker on the EC2 instance, building the zip file and uploading it to Lambda environment, where everything worked fine, as I expected.

    I thought docker was an environment independent from the host, but I guess that is not the case.


  2. I think you may try to setup PyNaCl like so:

    SODIUM_INSTALL=system pip3 install pynacl
    

    that will force PyNaCl to use the version of libsodium provided by AWS

    see this

    in the last version of PyNaClis updated to libsodium 1.0.16. so maybe it is not compatible with AWS

    so you may remove PyNaCl from requirements.txt and add this to your Dockerfile:

    RUN SODIUM_INSTALL=system pip3 install pynacl -t /var/task/lib
    

    or maybe setup the dockerfile like this and keep PyNaCl in requirements.txt:

    ARG SODIUM_INSTALL=system
    

    Try also to setup sodium before installing PyNaCI:


    RUN wget https://download.libsodium.org/libsodium/releases/libsodium-1.0.15.tar.gz
    && tar xzf libsodium-1.0.15.tar.gz
    && cd libsodium-1.0.15
    && ./configure
    && make install

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search