VPC Endpoint : Creating S3 VPC endpoint connection with a Lambda function.

Amazon Virtual Private Cloud (Amazon VPC) enables you to launch Amazon Web Services (AWS) resources into a virtual network that you’ve defined. This virtual network resembles a traditional network that you’d operate in your own data center.

A VPC endpoint allows you to privately connect your VPC to supported AWS services. It also doesn’t require an internet gateway, NAT device, VPN connection, or AWS Direct Connect connection. Endpoints are virtual devices that are horizontally scaled, redundant, and highly available VPC components. They allow communication between instances in your VPC and services without imposing availability risks or bandwidth constraints on your network traffic. VPC endpoints use IP addresses allocated from within your VPC address space.

There are two types of VPC endpoints: Interface endpoints and Gateway endpoints. 

Amazon Simple Storage Service (S3) and Amazon DynamoDB are accessed using gateway endpoints. You can configure resource policies on both the gateway endpoint and the AWS resource that the endpoint provides access to. A VPC endpoint policy is an AWS Identity and Access Management (AWS IAM) resource policy that you can attach to an endpoint. It is a separate policy for controlling access from the endpoint to the specified service. This enables granular access control and private network connectivity from within a VPC.

Interface endpoints allow you to connect to services powered by AWS PrivateLink. This includes a large number of AWS services. It also can include services hosted by other AWS customers, and AWS Partner Network (APN) partners in their own VPCs, by using AWS partner services through AWS PrivateLink, you no longer have to rely on access to the public internet. Like gateway endpoints, interface endpoints can be secured using resource policies on the endpoint itself and the resource that the endpoint provides access to. Interface endpoints allow the use of security groups to restrict access to the endpoint.

Below is an example architecture showing the use of VPC endpoint works to have a better understanding:

This doesn’t work, because the Lambda function is in a private subnet, so has no internet access. Therefore, the Lambda function can’t reach the AWS S3 URL, and the request will time out.

To resolve the issue a VPC endpoint is created. Here, an VPC endpoint provides a way for an S3 request to be routed through to the Amazon S3 service, without having to connect a subnet to an internet gateway. The VPC endpoint is what’s known as a gateway endpoint. It works by adding an entry to the route table of a subnet, forwarding S3 traffic to the VPC endpoint.

Now let us set up the IAM role for the lambda function and create the VPC endpoint to make the whole thing work.

A) Setting the IAM role for the Lambda Function:

   Go to the Lambda function Console in the AWS management console.

1.     Open the function
2.     Click on the Configuration Tab
3.     From the left side panel select the Permission.
4.     You will be able to see the Role attached under the  Execution Role.

5.      Open the Role by clicking on it and click on the permission

6. Under permission, all the attached policies are listed. Make sure that you give the lambda function an access to s3 by including the correct policies.

  #AmzasonS3FullAccess with policy full access to all the resources of  S3.

             #AllowS3Get Customize policy will limited access to a particular bucket

B) Setting the VPC Endpoint with endpoints type to Gateway so that the lambda function is able to connect to the S3 bucket.

1. Go to VPC Console
2. On the left side panel scroll down to endpoints and click on it to open.

3. Click on the create endpoints 

4. Here Tick on the AWS Service.

5. Select the S3 service with Gateway Type.

6. Select the VPC where the lambda function is deployed.

7. Configure the Route table by selecting the best route available for accessing the AWS services within the VPC.

8. In Policy let it be full access. If there is a need a custom policy can be written.

9. Then add tags to Name it properly and then click create endpoint to create the endpoint.

After this wait for a few minutes to allow the newly created endpoints to be available for use. 

All the required configuration has been completed. Now, the lambda function will be able to access the S3 bucket with out any issues.