# 99 Problems but Load Balancing ain't one > A short story on DiscrimiNAT OTF's integration with AWS' Gateway Load Balancer that enabled auto scaling, high availability and load balancing. :::caution lie It's actually 5 less problems. But that wouldn't have made a catchy title. ::: **DiscrimiNAT OTF now integrates with AWS' Gateway Load Balancer. Deploying this version alleviates five distinct problems.** 1. [Exfiltration](https://attack.mitre.org/tactics/TA0010/) and [Command & Control](https://attack.mitre.org/tactics/TA0011/) TTPs 2. Discovery and maintenance of outbound [FQDNs'](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) allowlists 3. High Availability 4. Load Balancing 5. Auto Scaling Read on for more details. ## Things Have Changed [Log4j](https://www.fastly.com/blog/digging-deeper-into-log4shell-0day-rce-exploit-found-in-log4j), [SolarWinds](https://www.crn.com/slide-shows/security/12-lessons-learned-from-the-solarwinds-breach-rsa-conference?page=6) and [Codecov](https://www.bleepingcomputer.com/news/security/popular-codecov-code-coverage-tool-hacked-to-steal-dev-credentials/) have shown us that having high quality exfiltration controls are more important than ever. And nonetheless, ➟ [Two-pizza product teams](https://docs.aws.amazon.com/whitepapers/latest/introduction-devops-aws/communication-collaboration.html) want to move faster than ever and independently. ➟ SREs have [SLOs](https://www.atlassian.com/incident-management/kpis/sla-vs-slo-vs-sli) to meet and their managers have SLAs to meet. ➟ Developers will only work with nimble tools that fit with their methodology, such as Terraform. ➟ Deployment times are in minutes and optimised to the second. ➟ Each deployment has become an unmapped, and unrestrained, mesh of microservices and exotic CDNs. As security engineers would happily tell you, the controls to defend against these, and similar vulnerabilities have existed for some time: 🇺🇸 NIST SP 800-53, [AC-4 Information Flow Enforcement](https://csrc.nist.gov/projects/cprt/catalog#/cprt/framework/version/SP_800_53_5_1_0/home?element=AC-4) 🇺🇸 NIST SP 800-53, [SC-7 Boundary Protection](https://csrc.nist.gov/projects/cprt/catalog#/cprt/framework/version/SP_800_53_5_1_0/home?element=SC-7) 💳 PCI DSS v4.0, [1.3.2 Outbound traffic from the CDE is restricted](https://www.pcisecuritystandards.org/document_library/) But these controls couldn't have been imposed easily on engineering squads because the available solutions were simply not up to scratch. ⨯ Allowlists for genuine outbound traffic were cumbersome to work out. ⨯ Shared allowlists, which contain CDNs with wildcarded base domains, defeated the purpose. ⨯ Allowlists managed centrally by 'security' were a point of friction. ⨯ 'Appliances' were incredibly complicated to bootstrap with XML payloads. ⨯ Some offered no protection against even simple evasion techniques such as DNS spoofing. (Have you seen our [litmus test](/discriminat/comparison/aws-network-firewall/#litmus-test)?) ⨯ Others were susceptible to false-positives, such as in the case of FWAAS-1501. ⨯ Apps & microservices' code would need to be changed to work with explicit proxies, and configurations maintained for _dev_ and _prod_ environments. They were a recipe for non-adoption. Until now. ## Enter DiscrimiNAT with AWS' GWLB :::caution spoiler The remaining 94 problems are not to do with DiscrimiNAT. ::: Over the last few months we've collaborated with AWS to integrate DiscrimiNAT with the Gateway Load Balancer, in the [two-arm mode](https://aws.amazon.com/blogs/networking-and-content-delivery/best-practices-for-deploying-gateway-load-balancer/), that delivers, ✓ High Availability, Load Balancing, Auto Scaling ✓ Zero-Downtime Upgrades ✓ Reduced number of hops, hence low latency ✓ Source NAT to the internet in the same hop ✓ Simple, two-subnet design We're glad to see that AWS has completely thought this through! ➟ A GWLB Endpoint is set as the target for destination 0.0.0.0/0 in the Private subnets' routing tables. This Endpoint does not change. ➟ The Endpoint encapsulates the original packets, in [Geneve](https://www.rfc-editor.org/rfc/rfc8926), from the apps and delivers them to one of the DiscrimiNAT instances in the associated Auto Scaling Group. ➟ Flow stickiness is maintained with the 5-tuple of each packet, i.e. protocol, source IP, source port, destination IP, destination port. ➟ The load balancer health checks each firewall instance at regular intervals. ➟ Should an instance fail the health check, traffic is diverted away from it to healthy instances and the Auto Scaling Group begins its replacement process. ➟ With routing applied to the original packet (decapsulated), Source NAT is achieved within the firewall instances. ### Two Reference Architectures #### Cross-Zone In the `cross-zone` mode, the GWLB distributes traffic evenly across all deployed AZs. This reduces the number of DiscrimiNAT instances that will have to run for high-availability but increases AWS' data-transfer costs and a bit of latency. :::note Terraform variable `high_availability_mode` should be set to `cross-zone` to implement this. This is also the default. The Terraform module is [here](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws). ::: ![](/img/gwlb-cross-zone-v2.drawio.png) #### Intra-Zone In the `intra-zone` mode, the GWLB distributes traffic evenly across all DiscrimiNAT instances in the same AZ as the client. For effective high-availability, this mode will need at least two instances per deployed AZ. Please note this does not fully protect you against the failure of an entire AZ on the Amazon side, however your other services in the zone would potentially be impacted too and therefore not sending egress traffic. :::note Terraform variable `high_availability_mode` should be set to `intra-zone`. The Terraform module is [here](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws). ::: ![](/img/gwlb-intra-zone-v2.drawio.png) :::tip Both architectures are discussed in our reference documentation [here](/docs/discriminat/aws/reference-architectures/). ::: ## This is DevSecOps :::note   The trinity of ‘developer experience + security standards + operational efficiency’ is greater than the sum of its parts. We call it **‘ergonomic cybersecurity’**. ::: ### For the Dev ✓ Figure out [FQDNs](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) to be allowed with clever annotation in the Security Groups directly. ![](/img/aws-see-thru.gif) ✓ [Ready-made queries for CloudWatch](/docs/discriminat/aws/logs-ref/#filters) to tabulate the observed [FQDNs](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) in normal traffic. ![](/img/log-insights-after-see-thru-log-capture.gif) ✓ Add allowed [FQDNs](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) to **existing** Security Groups as annotations directly. ![](/img/aws-protocol-tls.gif) ✓ Completely [Terraform-driven](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws/latest) (or any other IaC) with just the AWS provider. ✓ Never too late to implement PCI-DSS or NIST SP 800 controls. ✓ Simple, two-subnet design. ✓ Free, expert [support](/support/) always at hand. One of our team will readily pair on a screen-sharing call and get the job done. ### For the Sec ✓ Allowlists can be stored in the AWS Parameter Store, and shared & enforced via AWS RAM across accounts, VPCs and Security Groups – least privilege egress [FTW](https://en.wiktionary.org/wiki/for_the_win). ✓ Everything visible and settable from AWS console itself. ✓ IAM permissions on Security Groups themselves determine who can change them. ✓ TLS 1.2+, SSH v2 enforced and cannot be downgraded. ✓ Robust defence against DNS over HTTPS, SNI spoofing, Encrypted Client Hello (ESNI), etc. ✓ Updates on emerging threats and the CIS-hardened base 3–5 times a year. ✓ Critical updates in 10 days. ✓ Data processing, with all the DNS and TLS metadata, is within customer-owned VPCs and not in another company's SaaS. ✓ Structured and filterable audit (allowlist change, etc.) logs in CloudWatch. ✓ Structured and filterable egress traffic logs in CloudWatch for full visibility. ### For the Ops ✓ [Terraform module](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws/latest) preconfigured with high availability, load balancing and auto scaling. ✓ Preconfigured health checks at 2 consecutive failures, 5 seconds apart. (Most aggressive possible on AWS.) ✓ Terraform picks up upgrades automatically and applies with zero-downtime. ✓ No new providers or vendor UIs/dashboards; everything in AWS APIs. ✓ Works well on a t3.small and downscales automatically saving license costs. ✓ No need for another NAT solution, let alone a _Managed NAT_ solution. ✓ No data processing fees. ✓ Same Elastic IP association support when allocated. ✓ No problem with round-robin and low-TTL DNS destinations in the allowlists. ✓ Structured and filterable flow logs in CloudWatch. ## Next Steps Now that you've discovered the simplest [northbound](https://learn.microsoft.com/en-gb/archive/blogs/tip_of_the_day/tip-of-the-day-demystifying-software-defined-networking-terms-the-cloud-compass-sdn-data-flows) firewall solution around, Get a Demo Explore the Docs Explore the Terraform module