Serverless - Include/Exclude

Serverless Development with AWS Lambda and NodeJS

44 Lectures 7.5 hours

AWS LAMBDA- Serverless Computing

31 Lectures 3 hours

AWS Serverless Applications

25 Lectures 1 hours

We've already seen in the 'Deploying a function' chapter that to deploy functions from an existing project to AWS lambda, you need to modify the functions to take event and context as arguments and you need to add a serverless.yml file in the project folder, with the functions defined.Then hitting serverless deploy does the job.

Quite frequently, especially if you are asked to migrate some functions from an extensive existing project to AWS Lambda, you are faced with a size challenge.If your project is large enough, you may very well exceed the size limits imposed by AWS on Lambda functions (250 MB, including the application code and its dependencies).

And some dependencies like NumPy occupy a lot of space themselves. For instance, NumPy is ~80 MB large, SciPy is also in the same ballpark, and so on.In such scenarios, you have very limited space left for the application code and require a method of excluding unnecessary files from the lambda deployment package. Luckily, serverless makes this quite easy.

Include and exclude fields

As you would have guessed, you can specify files and folders that you want to exclude from the deployment build using the 'exclude' tag. All the files/ folders not specified in the exclude section are included by default. Then what is the use of the 'include' tag? Well, if you want a folder to be excluded in general, but want to include just a few files or sub-folders within that folder, you can specify those files/sub-folders within the 'include' tag. This way, all other files within that folder will be excluded, and only those specified within the 'include' section will remain. The example below will explain this better.

service: influx-archive-pipeline

provider:
name: aws
runtime: python3.6
stage: prod
region: us-east-2
profile: yash-sanghvi
timeout: 900
memorySize: 1024

# you can add packaging information here
package:
include:
- src/models/config.py
- src/models/lambda_apis/**
- src/models/scheduled_lambdas/**

exclude:
- docs/**
- models/**
- notebooks/**
- references/**
- reports/**
- src/data/**
- src/visualization/**
- src/models/**
- utils/**

functions:
user_details_api:
handler: src/models/lambda_apis/user_details_api.sync_user_details
events:
- http:
path: details/{user_id}
method: get
integration: lambda
cors: true

monitoring_lambda:
handler: src/models/scheduled_lambdas/monitoring_lambda.periodic_monitoring
events:
- schedule: cron(15 16 * * ? *)


As you can from the above severless.yml file, we are excluding most of the folders within the root folder that contains the serverless.yml. We have even excluded the src/models folder. However,we want to include 2 sub-folders and 1 file within src/models. Therefore, those have been specifically added in the 'include' section. Please note that any file/ folder that is not a part of the exclude section will be included by default.

Note the path of the two lambda functions. They both lie in src/models. While src/models is excluded by default, these functions specifically lie in sub-folders that are mentioned in the include section. Therefore they will execute without any issues. If I added a function lying in, say, src/data, that would not be allowed, because all the contents of src/data have been excluded.

Please note that specifying /** indicates that all content (files/sub-folders) within that folder gets covered. Thus if the docs folder contains 10 sub-folders and 12 files, all of which need to be excluded, then -docs/** does the job. We don't need to mention each file/folder separately.