Question 18851 in Amazon Web Sevices Question posted in
The official Amazon Web Services documentation can be found here.

AWS Lambda Python Serverless error: "unable to import module … cannot important name 'etree' from 'lxml' – Amazon Web Sevices

"Unable to import module ‘launcher’: cannot import name ‘etree’ from ‘lxml’ (/var/task/lxml/init.py)"

I have been trying to schedule a Python scraper to run and drop data to AWS S3 using AWS Lambda and Serverless. I have successfully deployed a function within AWS Lambda via Serverless, but I’m getting the error referenced above when I go to run it.

I’ve already worked through troubleshooting steps referenced here: https://aws.amazon.com/premiumsupport/knowledge-center/lambda-import-module-error-python/

Th result of this troubleshooting was setting up a Cloud9 IDE that allowed me to create a Python package with lxml installed among other modules. I created a layer on my Lambda function with that package hoping it would pick up the module, but I got the same error.

I am now wondering if the problem is that the Python scraper was built locally and Serverless was deployed locally. Would I be better off migrating my code to Cloud9 and trying to deploy the job from there?

Any help would be appreciated.

Tags:

2

Answers


  1. 0

    I was having this same issue when attempting to use the simple-salesforce library on a lambda function. I created a ticket with AWS and the steps they provided worked for me. Posting the steps w/ commands below:

    [If you are using python3.9 for your Lambda function]

    1. Launch an EC2 instance using [Amazon Linux 2 AMI]
    2. SSH to this EC2 instance

    a. Install python3.9

    sudo yum groupinstall "Development Tools" -y
sudo yum install openssl-devel libffi-devel bzip2-devel -y
gcc --version
sudo yum install wget -y
wget https://www.python.org/ftp/python/3.9.16/Python-3.9.16.tgz
tar xvf Python-3.9.16.tgz
cd Python-3.9.16/
./configure --enable-optimizations
sudo make altinstall # install python3.9
python3.9 --version
pip3.9 --version
/usr/local/bin/python3.9 -m pip install --upgrade pip

    b. create simple-salesforce package or whatever package is using lxml

    cd ~
mkdir layer-name
cd layer-name
mkdir python
cd python
pip3.9 install simple-salesforce -t .
cd ..
zip -r simple-salesforce-layer.zip .

    create Lambda layer with the package(you can use AWS CLI ‘publish-layer-
    version’)

    aws lambda publish-layer-version --layer-name desired-name --description 
"description wanted" --zip-file fileb://layer.zip

    Voila! Hope this works for you.

    Login or Signup to reply.
  2. 0

    Hi I came across the same error actually it is really simple you should just create AWS lambda layer with scrapy installed in the layer

    #1 Create a new folder with a requirements.txt

    #2 Add scrapy to your requirements.txt

    Once you have followed this steps

    #3 Follow the rest of the steps here

    https://aws.amazon.com/fr/premiumsupport/knowledge-center/lambda-layer-simulated-docker/

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search