Lambda can't access HTTP endpoint in the same security group

Question

I'm trying to figure out the following:

1. I have an API service deployed in the default VPC, and I have a ELB configured to access the service.
2. The ELB is attached to a security group sg-XXXXXXXX1 to restrict inbound traffic (open to all for outbound)
3. I'm now trying to create a lambda function that can call the API service. (using python requests)

I've tried the following and failed to succeed:

1. In the configuration > VPC section, I added sg-XXXXXXXX1 as the security group and the 4 default subnets
2. Added AWSLambdaVPCAccessExecutionRole to lambda role
3. Created a new security group sg-XXXXXXXX2 (all traffic for inbound and outbound), attached it to lambda, and added sg-XXXXXXXX2 to the inbound list (all traffic) for sg-XXXXXXXX1

Any ideas on what I did wrong? and how I can fix it?

Answer 1

Your ELB is most likely a public ELB (which is the default setting). The ELB DNS will resolve to a public IP address, not a VPC IP address. That means that the traffic going to the ELB will exit the VPC, go out to the Internet, and then back into AWS and into the ELB. When that happens, any association with the Lambda function's security group gets lost. Also, if your Lambda function isn't running in a subnet with a route to a NAT gateway it won't be able to access the Internet at all, so it is just going to timeout and fail when it tries to access the ELB.

To fix this, change the load balancer scheme to private, which will give the load balancer a private VPC IP address, and make it accessible only within the VPC.