No traffic getting to EKS pods through ALB

I’m hoping someone can point out where I’ve messed something up with my setup on AWS.

The architecture of the application is:

• 1 VPC with 2 public + 2 private subnets
• 1 RDS MySQL database within the same VPC
• 1 Elasticache Redis store within the same VPC
• 2 node groups in two availability zones – eu-west-2 a and b

All of the above seems to be working fine. I’ve deployed the application using helm and I can see my pods running, can access them directly, they’re able to communicate with the database/Redis correctly so I don’t THINK there are any issues in this part of the system.

Then I came to setting up the ingress/load balancer and I’m not having any luck at all. I’ve basically followed the guide here to set things up, and got through the guide without any issues, and on the surface things appear to be setup correctly.

The ingress controller logs suggest that it’s working correctly

I0802 05:46:45.130113       1 rules.go:102] default/my-app-ingress: rule 1 modified with conditions [{    Field: "path-pattern",    PathPatternConfig: {      Values: ["/*"]    }  }]
I0802 05:46:45.130576       1 recorder.go:53] kubebuilder/manager/events "level"=1 "msg"="Normal"  "message"="rule 1 modified with conditions [{    Field: "path-pattern",    PathPatternConfig: {      Values: ["/*"]    }  }]" "object"={"kind":"Ingress","namespace":"default","name":"my-app-ingress","uid":"02ce21a7-fba1-4605-9339-509bf3745538","apiVersion":"extensions/v1beta1","resourceVersion":"400603"} "reason"="MODIFY"
I0802 05:46:45.440737       1 controller.go:236] kubebuilder/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="alb-ingress-controller" "request"={"Namespace":"default","Name":"my-app-ingress"}
W0802 06:38:42.924349       1 reflector.go:270] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:95: watch of *v1.Secret ended with: too old resource version: 400605 (410306)
I0802 06:39:54.486822       1 controller.go:236] kubebuilder/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="alb-ingress-controller" "request"={"Namespace":"default","Name":"my-app-ingress"}
I0802 06:41:37.739524       1 controller.go:236] kubebuilder/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="alb-ingress-controller" "request"={"Namespace":"default","Name":"my-app-ingress"}
I0802 06:41:38.149869       1 controller.go:236] kubebuilder/controller "level"=1 "msg"="Successfully Reconciled"  "controller"="alb-ingress-controller" "request"={"Namespace":"default","Name":"my-app-ingress"}

My ingress definition I think is ok (worked fine when testing with Digital Ocean at least) and I’ve added various annotations for the ALB

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: my-app-ingress
  labels:
    app.kubernetes.io/component: web
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/healthcheck-path: /healthcheck
spec:
  rules:
  - http:
      paths:
      - path: /*
        backend:
          serviceName: my-app-service
          servicePort: 3000

My service is a NodePort as required by ALB

apiVersion: v1
kind: Service
metadata:
  name: my-app-service
  labels:
    app.kubernetes.io/component: web
spec:
  type: NodePort
  ports:
    - port: 80
      targetPort: 3000
      protocol: TCP
      name: http
    - port: 443
      targetPort: 3000
      protocol: TCP
      name: https
  selector:
    app: my-app-web

I followed all the other steps in the guide. Nodes in the target group appear healthy, can’t see any errors anywhere.

When I try and connect to my load balancer URL however, I get an immediate failure. No status or anything, more like it simply can’t find the URL. The DNS resolves correctly, if I try a ping on the URL it resolves to the IP address that I can find in AWS, but the browser just terminates immediately giving a "We can’t find that site" page but there’s not 404 response from the server itself that I can see.

The only thing I can think of though not certain of, is that I haven’t yet setup an SSL certificate (haven’t got that far). When I try and load the URL in the browser, it does immediately switch it to an https request so it seems as though something somewhere is triggering that switchover, so is it possible that the ALB is immediately closing the connection due to there being no certificate setup?

Any pointers in the right direction would be appreciated, or ways of debugging this as I’m a bit lost at this point