Problem
Docker image compiles successfully, however fails when ran from Lambda because of its read only file system.
Summary
Luminati-proxy has a docker integration for their proxy manager. I copied over their docker file and appended it to my own docker file for pushing out a script to AWS Lambda. The building of the docker image was successful, but when pushed off to Lambda, it failed because of a read only file system error:
Failed to create directory /home/sbx_user1051/proxy_manager: [code=EROFS] Error: EROFS: read-only file system, mkdir '/home/sbx_user1051'
2022-02-28 19:37:22.049 FILE (8): Failed to create directory /home/sbx_user1051/proxy_manager: [code=EROFS] Error: EROFS: read-only file system, mkdir '/home/sbx_user1051'
Analysis
Upon examining the trackback, the error is focused on the proxy_manager installation and fails with directory changes (mkdir, mk_work_dir …). These changes are made within the .js files of the GitHub which is pulled from the docker file as the proxy_manager installation. Obviously the only mutable directory on Lambda is the /tmp directory, but is there a workaround for getting this set up without resorting to putting everything under the /tmp directory as it wipes itself upon runtime? Reinstalling a proxy_manager each run is not at all ideal…
Answer?
Could this be as simple as setting environment stipulations such as:
ENV PATH=...
ENV LD_LIBRARY_PATH=...
If so, I how should they be configured? I am adding the docker file below for quick reference:
FROM node:14.18.1
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
&& sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
&& apt-get update
&& apt-get install -y google-chrome-stable fonts-ipafont-gothic fonts-wqy-zenhei fonts-thai-tlwg fonts-kacst fonts-freefont-ttf
--no-install-recommends
&& rm -rf /var/lib/apt/lists/*
USER root
RUN npm config set user root
RUN npm install -g [email protected]
RUN npm install -g @luminati-io/luminati-proxy
ENV DOCKER 1
CMD ["luminati", "--help"]
I appreciate the insight!
2
Answers
AWS Lambda provides /tmp folder for users to write files on lambda, as I don’t about your question context but hope this help.
You can write files to AWS Lambda at /tmp folder
eg. I want to create a file demo.txt at runtime/programmatically using AWS lambda, then i can write the file to /txt/demo.txt
TL;DR:
Explanation:
According to the documentation of AWS
Lambda functions:
Since containers based on Linux based images are already supposed to have a folder called
/tmp
you should pretty much be able to access that folder any time from your code to read( remember, read-only FS)If you are looking to store content amazon’s solution for that is for you to have any content created and manage over an S3 bucket, buckets are as easy to use as if you read a file locally but will remain accessible after the lambda instance finishes the workload
Please refer to Read file from aws s3 bucket using node fs and Upload a file to Amazon S3 with NodeJS for more details on how to use an S3 bucket. There are plenty of ways to achieve it regardless of the language been used.
This is all based on a best practice promoted by AWS over their platform. Where lambdas remain stateless