Resolve AWS Lambda Layers compatibility issues using CodeBuild

Many of us use windows laptops for development work and face runtime issues while executing AWS Lambda functions which uses lambda layers created using windows laptop.

These issues are due to simple fundamental reason that some modules/libraries are different for windows and unix/linux.

For example let us take python 3.x as our programming language and you might have seen “ImportError: No module named xxxx” error when executing lambda functions (which are referring to Lambda Layers). Now folks with windows laptop won’t be able to fix above issues as they will need Linux environment to download required packages and then create compatible Lambda Layers.

To overcome above issue and make sure that lambda layer being deployed on AWS is compatible with actual Lambda runtime we can follow below process of usign CodeBuild to create and deploy lambda layer version:

1. Create below buildspec.yml and upload to S3 bucket (we considering python for this example, you can modify for nodejs etc.)

# Buildspec Reference Doc: https://docs.aws.amazon.com/codebuild/latest/userguide/build-spec-ref.html
version: 0.2
phases:
  install:
    runtime-versions:
      python: 3.9
  pre_build:
    commands:
      - echo "[+] Updating PIP...."
      - pip install --upgrade pip
      - echo "[+] Installing dependencies...."
      - mkdir python
      - ls
      - cd python
      - pip install <package name/version> -t .
      - ls
      - cd ..
      - yum install zip unzip -y -q
      - zip -r test-lambda-layer.zip python
      - ls
      - pip install awscli --upgrade
  build:
    commands:
      - echo "Starting build `date` in `pwd`"
      - echo "Build completed `date` in `pwd`"
      
  post_build:
    commands:
      - echo "Starting Lambda Layer Deployment `date` in `pwd`"
      - >
        bash -c '
        aws lambda publish-layer-version
        --layer-name test-pymysql-layer
        --description "test lambda layer for python 3.8 and 3.9"
        --license-info "MIT"
        --zip-file fileb://test-lambda-layer.zip
        --compatible-runtimes python3.8 python3.9'
      - echo "Completed Lambda Layer Deployment `date` in `pwd`"

1. Create New Code Build project

2. Provide bucket details of buildspec file

3. Select amazon linux image for better compatibility

4. Select Code Build role with required permissions to S3 bucket and layer creation etc.

5. Provide buildspec filename created in Step 1

6. Finish creating Code Build

7. Go to Code Build, select newly created Code Build project and start build

8. Monitor Code Build step to be complete

9. Go to Lambda Functions -> Layers and check if layer is created

10. Use this layer to resolve/avoid compatibility issues and get rid of “ImportError: No module named xxxx" errors

Conclusion : Always prepare of target runtime along with local runtime and using above steps you can easily prepare Lambda Layers which are compatible with target runtime even if local runtime is different.