How to create AWS Lambda Layer

Install Python 3.11

To get started, we need to install Python 3.11 using Homebrew. Open your terminal and run the following command:

brew install python@3.11

Once installed, verify that you're running the correct version of Python by typing:

python3 -V

Install pip & Virtual Environment

Next, we need to install pip, the package installer for Python, and create a virtual environment using venv. Run the following commands:

python3.11 -m pip install --upgrade pip
python3.11 -m pip install virtualenv

Set up Lambda Layer Directory

Create a new directory to hold your lambda layer files, and navigate into it. Create a new virtual environment using venv.

mkdir lambda_layers
cd lambda_layers

python3.11 -m venv env
source env/bin/activate
pip list
pip install --upgrade pip

Install Dependencies & Create Lambda Layer

Install your dependencies, as per your need, and create the lambda layer. An example as below:

mkdir python
cd python

pip install snowflake-connector-python \
    "urllib3<2" \
    certifi==2023.11.17 \
    packaging \
    --no-cache-dir \
    --python-version 3.11 \
    --platform manylinux2014_x86_64 \
    --target=package \
    --implementation cp \
    --no-user \
    --only-binary=:all: \
    --upgrade \
    -t .

Remove Cache and DistInfo

To reduce the size of your lambda layer, remove any cache and distinfo files:

repo=$(pwd)
root=`basename ${repo}`
dir=$(pwd)

for d in $(find ${repo} -maxdepth 10 -type d)
do
    dname=`basename ${d}`
    pdname=`basename $(dirname ${d})`
    if [[ ( "${dname}" == "__pycache__" ) || ( "${dname}" == *".dist-info" ) || ( "${dname}" == *"tests"* ) ]]
    then
        echo "${d}"
        rm -rf "${d}"
    else
        continue
    fi
done

cd ${dir}

Create the Zip File

Finally, create a zip file containing your lambda layer:

cd ..
zip -r custom-lambda-layer.zip python


deactivate

Once you've created your lambda layer, deactivate your virtual environment

And that's it! You now have a Python-based AWS Lambda layer ready for use. With this guide, you should be able to create your own custom layers and reuse code across multiple functions in AWS Lambda.