I am using Pychromeless repo with success at AWS lambda.
Now I need to use NaCL dependency, to decrypt a string, but I am getting
Unable to import module 'lambda_function': /var/task/lib/nacl/_sodium.abi3.so
with a complement of
invalid ELF header
when running the function on AWS Lambda.
I know it is a problem specific related to AWS Lambda environment, because I can run the function on my Mac inside docker.
Here’s my requirements.txt file
boto3==1.6.18
botocore==1.9.18
selenium==2.53.6
chromedriver-install==1.0.3
beautifulsoup4==4.6.1
certifi==2018.11.29
chardet==3.0.4
editdistance==0.5.3
future==0.17.1
idna==2.7
python-telegram-bot==10.1.0
requests==2.19.1
soupsieve==1.7.3
urllib3==1.23
PyNaCl==1.3.0
Here is the dockerfile
FROM lambci/lambda:python3.6
MAINTAINER [email protected]
USER root
ENV APP_DIR /var/task
WORKDIR $APP_DIR
COPY requirements.txt .
COPY bin ./bin
COPY lib ./lib
RUN mkdir -p $APP_DIR/lib
RUN pip3 install -r requirements.txt -t /var/task/lib
And the makefile:
clean:
rm -rf build build.zip
rm -rf __pycache__
fetch-dependencies:
mkdir -p bin/
# Get chromedriver
curl -SL https://chromedriver.storage.googleapis.com/2.37/chromedriver_linux64.zip > chromedriver.zip
unzip chromedriver.zip -d bin/
# Get Headless-chrome
curl -SL https://github.com/adieuadieu/serverless-chrome/releases/download/v1.0.0-37/stable-headless-chromium-amazonlinux-2017-03.zip > headless-chromium.zip
unzip headless-chromium.zip -d bin/
# Clean
rm headless-chromium.zip chromedriver.zip
docker-build:
docker-compose build
docker-run:
docker-compose run lambda src/lambda_function.lambda_handler
build-lambda-package: clean fetch-dependencies
mkdir build
cp -r src build/.
cp -r bin build/.
cp -r lib build/.
pip install -r requirements.txt -t build/lib/.
cd build; zip -9qr build.zip .
cp build/build.zip .
rm -rf build
Without the decryption part, the code works great. So the issue is 100% related to PyNaCl.
Any help on solving this?
2
Answers
Ok, this is how I did it. I had to build everything on an EC2 AMI Linux 2 instance.
amzn2-ami-hvm-2.0.20190823.1-x86_64-gp2 (ami-0a1f49a762473adbd)
After launching the instance, I use this script to install Python 3.6 (and pip) and to create and activate a virtual environment.
For the docker part, I followed this tutorial, not without some troubles on the way (had to
sudo yum install polkit
and
sudo usermod -a -G docker ec2-user
and
reboot
and
sudo curl -L https://github.com/docker/compose/releases/download/1.22.0/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose
and
sudo chmod +x /usr/local/bin/docker-compose
).But anyway, I manage to work with docker on the EC2 instance, building the zip file and uploading it to Lambda environment, where everything worked fine, as I expected.
I thought docker was an environment independent from the host, but I guess that is not the case.
I think you may try to setup
PyNaCl
like so:that will force
PyNaCl
to use the version of libsodium provided byAWS
see this
in the last version of
PyNaCl
is updated tolibsodium 1.0.16.
so maybe it is not compatible withAWS
so you may remove
PyNaCl
fromrequirements.txt
and add this to yourDockerfile
:or maybe setup the
dockerfile
like this and keepPyNaCl
inrequirements.txt
:Try also to setup sodium before installing PyNaCI:
RUN wget https://download.libsodium.org/libsodium/releases/libsodium-1.0.15.tar.gz
&& tar xzf libsodium-1.0.15.tar.gz
&& cd libsodium-1.0.15
&& ./configure
&& make install