# DiscrimiNAT Firewall > DiscrimiNAT Firewall documentation for AWS and GCP This file contains all documentation content in a single document following the llmstxt.org standard. ## Configuration Reference With DiscrimiNAT, the Security Groups attached to the applications that need filtered access to the Internet can be annotated with the set of allowed FQDNs. This allows creating **least privilege egress policies** from the protected workload to the Internet. No need to create VPC-wide allowlists that allow discrete applications access to what they needn't. :::tip DiscrimiNAT's own Security Groups need not be altered. ::: On this page, we cover the syntax for the Cloud-native Security Groups' **description** fields, where the annotations live. ### See-Thru mode The `see-thru` mode is a non-blocking configuration that allows one to [build an appropriate allowlist 🎥](/docs/discriminat/aws/logs-ref/#building-an-allowlist-from-scratch-video-version) up to a specified date. The annotation is added to the description fields of **outbound rules of protected applications to-be**. #### Syntax `discriminat:see-thru:YYYY-mm-dd` :::caution The rule will automatically stop working **after** the `YYYY-mm-dd` date. ::: #### Examples `discriminat:see-thru:2026-12-31` `discriminat:see-thru:2026-09-09` Configuring this mode in the **AWS web console** would look like: ![](img/aws-see-thru-security-group-example.png) In **Terraform**, this would be along the lines of: ``` resource "aws_security_group" "some-app" { name = "monitor-some-app" vpc_id = var.vpc_id egress { from_port = 0 to_port = 0 protocol = "-1" cidr_blocks = ["0.0.0.0/0"] description = "discriminat:see-thru:2026-12-31" } } ``` #### Logs The **flow** logs stream under the DiscrimiNAT log group in CloudWatch will have three additional fields for traffic from any workload that has a Security Group attached with a `see-thru` Rule in it: 1. `see_thru_exerted`: **true** when no `tls` or `ssh` protocol rules allowed this connection through and it's only because of the `see-thru` rule that the connection has been let through. **false** when an already present `tls` or `ssh` protocol rule would otherwise allow this connection through. The rest of the fields in the log line remain as they would without the `see-thru` rule, hence indicating which `tls` or `ssh` protocol rule matched to let this connection through. 1. `see_thru_gid`: The Security Group ID where the enacted `see-thru` was found. 1. `see_thru_days_remaining`: The number of remaining days this rule will continue to work. Runs into negative if the current date goes past the specified date. These log fields are still emitted if a `see-thru` rule with any date, past or future, remains specified. ### TLS and SSH mode The `tls` and `ssh` modes are a blocking configuration that will only allow FQDNs set out in the allowlist. Destination protocols, [FQDNs](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) and IP Addresses are added to the description fields of **outbound rules of the protected applications**. :::caution If transitioning from a `see-thru` rule, note that `tls` and `ssh` rules **require a specific port number** to be specified in the Security Group Rules. ::: #### Syntax `discriminat::` ([ARN variant](#arn-variant)) `discriminat::[,,]` ([Inline variant](#inline-variant)) - Valid protocols are `ssh` and `tls` only. - There can be other text leading and trailing this specification, separated by whitespace. - All of it is case insensitive, except the ARN if specified. - [Wildcards](/docs/discriminat/aws/default-prefs/#wildcard_exposure) are only supported for the TLS protocol. :::caution **Please familiarise yourself with [operation and caveats](/blog/wildcards-and-system-2-thinking/#operation) on the behaviour of wildcard rules before using them.** ::: #### ARN variant - The \ is the full ARN of an SSM Parameter or a Secret. It can be in another AWS Account and/or Region as well, as long as DiscrimiNAT's IAM Instance Profile has [permissions](/docs/discriminat/aws/iam-instance-profile/) to read it. - If using our Terraform modules, consider adding the full ARNs of SSM Parameters to the `iam_get_additional_ssm_params` list variable and of Secrets to the `iam_get_additional_secrets` list variable to automatically update the IAM Instance Profile with requisite _Get_ permissions. - The JSON format for list of FQDNs and IP Addresses in the payload of an SSM Parameter or a Secret is: ```json { "addrs": [ "fqdn1.com", "*.github.com", "203.0.113.9", ... ] } ``` ##### Examples If the above payload were to be stored in an SSM Parameter called `team-foo-allowed-fqdns`, its full ARN could be `arn:aws:ssm:eu-west-2:111111111111:parameter/team-foo-allowed-fqdns`. Therefore, what goes in description field will be: `discriminat:tls:arn:aws:ssm:eu-west-2:111111111111:parameter/team-foo-allowed-fqdns` And if in a Secret: `discriminat:tls:arn:aws:secretsmanager:eu-west-2:111111111111:secret:service-foo-allowed-fqdns` Configuring this variant in the **AWS web console** would look like: ![](img/aws-arn-security-group-example.png) #### Common constraints to both variants - Under `Type`, choose any TCP protocol type and **specify exactly one port number** in `Port range`. To use another port for an alternative set of addresses, simply create another Security Group Rule with that port number. See the [Connecting to MS/Azure SQL Server over TLS](/docs/how-to/tls-upgrade/sql-server/#on-aws) guide for an example. - For your typical **HTTPS** outbound connection, the port number would be `443`. - A typical **SFTP** connection has the port number of `22`. - **Kafka** typically, as another example, is on port `9092`. - `Destination` must be set to the narrowest IP block you can think of. Now if you don't have anything narrower than `0.0.0.0/0` that is absolutely fine! The firewall takes care of validating the destination IP address of actual packets in many ways, but the outbound rule at the AWS networking and security level must allow the packet to leave the protected workload at all. **`0.0.0.0/0` is a safe choice with the DiscrimiNAT Firewall in the way.** - Should you wish to use the same port number again with a different Rule, set the destination to `0.0.0.0/1` or `0.0.0.0/2` or `0.0.0.0/3` and so on. - Multiple such Rules can be added to one Security Group and all of them will be recognised. :::tip You can attach multiple Security Groups to EC2 instances. It's a great way to bunch together related FQDNs. ::: #### Inline variant - There can be as many FQDNs and IP Addresses as you can fit separated by commas. There cannot be whitespace, though. - Multiple Security Groups can be created to accommodate more addresses, or group them by some criteria. - Alternatively, multiple outbound Rules within the same Security Group can be created with destination CIDRs varying like `0.0.0.0/0`, `0.0.0.0/1`, `0.0.0.0/2` and so on. ##### Examples - `DiscrimiNAT:TLS:api.foo.example.com,downloads.example.net,203.0.113.9` - `DiscrimiNAT:SSH:sftp.txs.example.org,ssh.github.com` - `discriminat:tls:api-v_.example.com discriminat:tls:www.example.org` - `lorem ipsum discriminat:ssh:ssh.github.com,gitlab.com,203.0.113.9 dolor sit amet` Configuring this variant in the **AWS web console** would look like: ![](img/aws-inline-security-group-example.png) In **Terraform**, this would be along the lines of: ``` resource "aws_security_group" "acme" { name = "acme-reports-upload" vpc_id = var.vpc_id egress { from_port = 443 to_port = 443 protocol = "tcp" cidr_blocks = ["0.0.0.0/0"] description = "discriminat:tls:api.acme.com,auth.acme.com,203.0.113.9,foo.example.com,bar.example.net" } } ``` ### Complete Bypass If you would like to discuss alternate protocols or know how to completely bypass DiscrimiNAT whilst still routing traffic, get in touch with our [DevSecOps Support](/support/) to discuss the best option for your use case. --- ## Default Preferences From v2.9.0 of DiscrimiNAT, certain behavioural preferences can be set at global level per deployment. These are defined in JSON format in an SSM Parameter named `DiscrimiNAT` (or `DiscrimiNAT_`) (case-sensitive) and the default value if not defined or not accessible is: ```json { "%default": { "wildcard_exposure": "prohibit_public_suffix", "flow_log_verbosity": "full", "see_thru": null, "x509_crls": "ignore" } } ``` :::tip Preferences you wish to leave at defaults can be excluded. Also, any additional JSON keys will simply be ignored. ::: :::caution Preferences are refreshed **once every five minutes** from the SSM Parameter. ::: ## Preferences ### wildcard_exposure Generally, `?` represents one character match exactly which isn't a `.`, and `*` matches any number of characters except `.`. #### Options `none`: no wildcard characters such as `?` or `*` are allowed at all `prohibit_asterisk`: `?` is allowed; `*` is not allowed `prohibit_public_suffix`: `?` is allowed; `*` is allowed as long as the possibility of resulting domain names are not publicly registrable (aka Effective TLDs). For example, `*.github.com` is okay but `*.github.io` is not. Another example would be `*.cloudflare.net` being effectively public. This is determined using [Mozilla's Public Suffix List (PSL)](https://publicsuffix.org/) bundled into DiscrimiNAT. The version of the PSL bundled will be in the [Release Notes](/docs/discriminat/aws/release-notes/) and will be updated with every release. :::tip If a wildcard pattern is not accepted because it could have led to a Public Suffix List match, a _config_ log message like the following will be emitted: ``{addr: "*.github.io", cat: "addr", outcome: "publicsuffix[.]org list matched with `github.io`"}`` ::: `nuclear`: `?` is allowed; `*` is allowed with no public suffix list safeguard #### Default `prohibit_public_suffix` ### flow_log_verbosity This setting only impacts the _flow_ logs. _config_ logs verbosity and behaviour will not be affected. #### Options `full`: all _allowed_ and _disallowed_ flows for both the client requests and the server responses are logged in the _flow_ logs `only_disallowed`: only _disallowed_ flows will be logged `none`: no logs at all #### Default `full` ### see_thru The `see_thru` non-blocking, monitoring mode can be enabled for an entire deployment rather than per Security Group from this level. This is only the default and does not override a _see-thru_ mode defined at a more granular level – even if the one the granular level may have expired. :::tip See [see-thru](/docs/discriminat/aws/config-ref/#see-thru-mode) mode docs for understanding this non-blocking, monitoring mode in detail. ::: #### Options `yyyy-mm-dd` formatted date `null` to not enable _see_thru_ at the default level. `null` value to be specified without quotes or this top-level key can be left out completely. #### Default `null` ### x509_crls #### Options `auto_allow`: Automatically allow plaintext HTTP CRL Endpoints of x509 SSL certificates for all TLS FQDNs allowlisted. `ignore`: Do not allow CRL Endpoints automatically. #### Default `ignore` ## Examples It may be useful to turn on the non-blocking, monitoring `see_thru` mode on until a specified date on a new deployment of DiscrimiNAT. This would ensure apps continue to work while data is captured for building an allowlist from eventually: `{"%default":{"see_thru":"2026-11-19"}}` :::tip Also see [building an allowlist from scratch video recipe](/docs/discriminat/aws/logs-ref/#building-an-allowlist-from-scratch-video-version). ::: Ultimately, when allowlists are built and enforced, you may want to turn off _flow_ logs for _allowed_ connections, saving on logging costs: `{"%default":{"flow_log_verbosity":"only_disallowed"}}` To also enable automatic allowing of X509 CRL Endpoints: `{"%default":{"flow_log_verbosity":"only_disallowed","x509_crls":"auto_allow"}}` ## System System-level preferences are passed directly to VM instance creation, are stored as files on the operating system disk, and are only read on start. These are placed within the instance because they represent critical customer choices that must be respected whether external APIs are accessible or not and the right IAM permissions on the profile are assigned or not. ### BYOL A licence key supplied by Chaser Support if using the [BYOL version](https://aws.amazon.com/marketplace/pp/prodview-7hafqjdkvnfyk) from AWS Marketplace. The pay as you go version does not require this. The value can be passed to the `byol` variable in our [Terraform modules](https://registry.terraform.io/namespaces/ChaserSystems) and [CloudFormation templates](https://github.com/orgs/ChaserSystems/repositories?q=cloudformation). The contents are Base64 decoded and written to the path `/etc/chaser/licence-key.der`. ### ASHR Automated System Health Reporting. 10 minutes after boot and then at around 0200 UTC every day, each instance of DiscrimiNAT collects its OS internals & system logs since instance creation, config changes & traffic flow information from last two hours and uploads it to a Chaser-owned cloud bucket. This information is encrypted at rest with a certain public key so only relevant individuals with access to the corresponding private key can decrypt it. The transfer is encrypted over TLS. Access to this information is immensely useful to create a faster and more reliable DiscrimiNAT as we add new features. We also get to learn about how users interact with the product in order to further improve the usability of it as they embark on a very ambitious journey of fully accounted for and effective egress controls. We understand if certain environments within your deployment would rather not have this turned on. To disable it, a file at the path `/etc/chaser/disable_automated-system-health-reporting` should exist. This can be achieved by setting the `ashr` variable to `false` in our [Terraform modules](https://registry.terraform.io/namespaces/ChaserSystems). --- ## IaC & Terraform :::info Ensure you've visited our [marketplace page](https://aws.amazon.com/marketplace/pp/prodview-7ulmdnoq5jnwu) and accepted the terms & conditions, and any private offers you may have been extended for your AWS account, first. ::: ## IMAGE IDENTIFIERS | key | value | | ------------ | ------------------------- | | owner-alias | aws-marketplace | | product-code | bz1yq0sc5ta99w5j7jjwzym8g | | architecture | x86_64 | These filters will result in all the historic versions too. The _latest_ among these is recommended. ### BYOL In case you have a licence key from us, there's a [dedicated BYOL marketplace page](https://aws.amazon.com/marketplace/pp/prodview-7hafqjdkvnfyk) to accept the terms at, and product-code: `a7z5gi2mkpzvo93r2e8csl2ld` . ## TERRAFORM MODULES You may want to consider our fully-working [modules at the Terraform Registry](https://registry.terraform.io/modules/ChaserSystems), which include **preconfigured high availability, load balancing and auto scaling**. The examples within extend from the canonical [terraform-aws-modules/vpc/aws](https://registry.terraform.io/modules/terraform-aws-modules/vpc/aws) module at the registry. :::info Contact us for expert help at `devsecops@chasersystems.com` at any stage of your journey – we'll jump on a screen-sharing call right away! ::: ## CLOUDFORMATION Equivalent CloudFormation templates are available on [our GitHub profile](https://github.com/orgs/ChaserSystems/repositories?q=cloudformation). ## DEPLOYMENT ESSENTIALS For effective functioning, the DiscrimiNAT Firewall will need: 1. A machine type with at least 2 vCPU and 2 GiB RAM. A `t3.small` should suffice where throughput requirements are basic and allowlists small. Otherwise a `c6a.large` (or `c6i.large`) makes a good choice for constant throughput. [Talk to our DevSecOps](/support) to get the sizing right! 1. An IAM policy that can read some EC2 metadata, write logs and set instance health (of itself). See the [IAM Instance Profile](../iam-instance-profile/) page for details. 1. Ability to forward IP packets with the `SourceDestCheck` property of the network interface turned off. 1. A public IP with routing to the Internet via the AWS supplied Internet Gateway. ## aws CLI EXAMPLE #### Lookup ```plaintext aws ec2 describe-images \ --query \ 'sort_by(Images, &CreationDate)[-1].ImageId' \ --filters \ "Name=owner-alias,Values=aws-marketplace" \ "Name=product-code,Values=bz1yq0sc5ta99w5j7jjwzym8g" \ "Name=architecture,Values=x86_64" \ --no-cli-pager --output text ``` --- ## IAM Instance Profile The DiscrimiNAT Firewall requires certain permissions to work effectively and these are included with our CloudFormation templates & Terraform modules. The following sections describe these in more detail for your review. ## TL;DR The following policy document is taken from our [Terraform module](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws/) and enables DiscrimiNAT for all its functions. ```json { "Statement": [ { "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams" ], "Effect": "Allow", "Resource": [ "arn:aws:logs:*:*:log-group:DiscrimiNAT:log-stream:*" ] }, { "Action": [ "ec2:DescribeNetworkInterfaces", "ec2:DescribeSecurityGroups", "ec2:DescribeAddresses" ], "Effect": "Allow", "Resource": "*" }, { "Action": [ "ssm:GetParameter" ], "Effect": "Allow", "Resource": [ "arn:aws:ssm:*:*:parameter/DiscrimiNAT*" ] }, { "Action": [ "ec2:ModifyInstanceAttribute", "ec2:AssociateAddress" ], "Condition": { "Null": { "aws:ResourceTag/discriminat": false } }, "Effect": "Allow", "Resource": "*" } ], "Version": "2012-10-17" } ``` ## LOGGING The DiscrimiNAT automatically logs config & flow events to CloudWatch. The following statement allows its instances to create and write to a log group called `DiscrimiNAT`. ```json { "Effect": "Allow", "Action": [ "logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents", "logs:DescribeLogStreams" ], "Resource": [ "arn:aws:logs:*:*:log-group:DiscrimiNAT:log-stream:*" ] } ``` ## ALLOWLIST CONFIG To form a complete picture of the VM instances and any serverless workloads in the VPC, such as Lambda, and their associated Security Groups' Rules, the DiscrimiNAT needs read access to these resources. ```json { "Effect": "Allow", "Action": [ "ec2:DescribeNetworkInterfaces", "ec2:DescribeSecurityGroups" ], "Resource": "*" } ``` Allowlists may further be stored as Parameters in SSM or as Secrets in Secrets Manager. The following two examples may be used to allow DiscrimiNAT to read in the configuration stored in these. ```json { "Action": [ "ssm:GetParameter" ], "Effect": "Allow", "Resource": [ "arn:aws:ssm:eu-west-2:111111111111:parameter/team-foo-allowed-fqdns", "arn:aws:ssm:eu-west-2:111111111111:parameter/service-foo-allowed-fqdns" ] } ``` ```json { "Action": [ "secretsmanager:GetSecretValue" ], "Effect": "Allow", "Resource": [ "arn:aws:secretsmanager:eu-west-2:111111111111:secret:team-foo-allowed-fqdns", "arn:aws:secretsmanager:eu-west-2:111111111111:secret:service-foo-allowed-fqdns" ] } ``` ## DEFAULT PREFERENCES Global level, behavioural [preferences](/docs/discriminat/aws/default-prefs/) are stored in an SSM Parameter named DiscrimiNAT. To allow for some creative flexibility with other Parameters that you may conceive prefixed with the word DiscrimiNAT (to store allowlists), a trailing wildcard to the name has been included in the supplied IAM Policy. ```json { "Action": [ "ssm:GetParameter" ], "Effect": "Allow", "Resource": [ "arn:aws:ssm:*:*:parameter/DiscrimiNAT*" ] } ``` ## ELASTIC IPs If a Public IP is not found attached to a DiscrimiNAT instance, it will look for any allocated but unassociated Elastic IPs that have a tag-key named `discriminat` set to the value of the same tag-key as on itself, else anything but blank. One of such Elastic IPs will be attempted to be associated with itself then. :::tip This allows you to have a stable set of static IPs to share with your partners, who may wish to allowlist them. ::: ```json { "Effect": "Allow", "Action": [ "ec2:DescribeAddresses" ], "Resource": "*" }, { "Effect": "Allow", "Action": [ "ec2:AssociateAddress" ], "Resource": "*", "Condition": { "Null": { "aws:ResourceTag/discriminat": false } } } ``` ## NAT Since the DiscrimiNAT NATs (is the clue in the name?), it needs [Source/Destination checks disabled](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_NAT_Instance.html#EIP_Disable_SrcDestCheck). Sadly, this property cannot be set in the Launch Template of an Auto Scaling Group. Therefore, the DiscrimiNAT sets this property to the desired value of _false_ on boot. The DiscrimiNAT instances themselves are tagged with the key `discriminat` to limit the scope of this permission. ```json { "Effect": "Allow", "Action": [ "ec2:ModifyInstanceAttribute" ], "Resource": "*", "Condition": { "Null": { "aws:ResourceTag/discriminat": false } } } ``` ## SELF-HEAL :::info This permission is only applicable and present in the ENI [architecture](/docs/discriminat/aws/reference-architectures/), where a load balancer conducting health checks is absent. It is absent and not needed in an [architecture](/docs/discriminat/aws/reference-architectures/) with the Gateway Load Balancer (GWLB). ::: The DiscrimiNAT monitors its own health in an Auto Scaling Group. To report bad health to the platform, so the instance can be recycled, it needs a specific permission. It is restricted to Auto Scaling Groups with an expected name. ```json { "Effect": "Allow", "Action": "autoscaling:SetInstanceHealth", "Resource": "arn:aws:autoscaling:*:*:autoScalingGroup:*:autoScalingGroupName/discriminat-*" } ``` --- ## Installation Overview ### [IaC & TERRAFORM âžœ](/docs/discriminat/aws/iac/) If you require complex or custom routing, and have Infrastructure-as-Code for your deployments already, this is where you will find key information. ### [CONFIG REFERENCE âžœ](/docs/discriminat/aws/config-ref/) The various ways AWS-native Security Groups can be annotated with appropriate FQDN allowlists and traffic monitoring rules after deployment of DiscrimiNAT. ### [DEFAULT PREFERENCES âžœ](/docs/discriminat/aws/default-prefs/) Global level preferences for controlling logs verbosity, wildcards exposure and monitoring mode. ### [LOGS REFERENCE âžœ](/docs/discriminat/aws/logs-ref/) Fields, filters and recipes to find what you need from DiscrimiNAT's config and flow logs in AWS CloudWatch. ### [REFERENCE ARCHITECTURES âžœ](/docs/discriminat/aws/reference-architectures/) The DiscrimiNAT Firewall supports three deployment topologies: **GWLB Cross-Zone**, **GWLB Intra-Zone** and **ENI per AZ**. See their high-level diagrams and read more about their non-functional properties such as the RTO. ### [QUICK START (from Marketplace) âžœ](/docs/discriminat/aws/quick-start/) For a 5-minute deployment, assuming secure defaults, from within the marketplace console itself. This will totally lock down the private subnets in your VPC from the get-go unless monitoring mode or appropriate FQDN-based egress firewall rules exist for outbound connections. --- ## Logs Reference ## Fields ### flow logs Flow Logs can be found in CloudWatch under the `DiscrimiNAT` log group, in the `flow` log stream. A typical flow log looks like: ![](img/typical-cloudwatch-flow-log-from-discriminat.png) Here's a summary of the possible fields: `dhost`: destination hostname/FQDN or IP Address `cat`: packet origin – client or server `outcome`: allowed or disallowed `src`: source IP address `spt`: source port `proto`: tls, ssh, http or unknown protocol `proto_v`: version of the identified protocol `dst`: destination IP address `dpt`: destination port `reason`: reason for the outcome – such as in which Security Group was a matching protocol rule found, cache not ready yet for a newly discovered allowlist entry, or which protocol anomaly led to the connection being disallowed `instance`: the EC2 instance ID of the DiscrimiNAT instance this log line was generated from `pqc`: indicates if a TLS handshake used a known [PQC TLS Supported Group](https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-8). The value may be true or false in case of TLS; or absent in the case of SSH. If X.509 CRLs are [set to be allowed](/docs/discriminat/aws/default-prefs/#x509_crls), or if there is some plaintext HTTP egress traffic with requisite outbound ports open, there may be HTTP flow logs with the following fields. Note that HTTP is generally not allowed unless X.509 CRLs were allowed and the requested URL was of a CRL Endpoint. `http_code`, `http_method`, `http_path`, `http_user_agent`: these fields can be useful to understand which application the plaintext HTTP traffic is originating from And if a Security Group with a `see-thru` rule in it is found attached to a client application, the following fields will be also present: `see_thru_days_remaining`: number of days remaining while the `see-thru` rule will be non-blocking, as defined in the `see-thru` rule itself `see_thru_gid`: the Security Group ID where the `see-thru` rule was found, or _%default_ if set via [preferences](/docs/discriminat/aws/default-prefs/#see_thru) `see_thru_exerted`: when _true_, the `see-thru` rule had to let this connection through and there was no protocol rule that would have allowed it; when _false_, there was a protocol rule that would have let the connection through anyway and its presence will be identified in the `reason` field; or the `see-thru` rule has expired Watch this **4 minute video** to understand these three fields, and their interplay with the `outcome` field, when viewed in the Cloud web console. Config reference for the `see-thru` rules can be found [here](/docs/discriminat/aws/config-ref#see-thru-mode). ### config logs Config Logs can be found in CloudWatch under the `DiscrimiNAT` log group, in the `config` log stream. They form an audit trail of changes made to rules and Security Group attachments as clients come and go. Here's a summary of the possible fields: `cat`: fqdn (addr), wildcard pattern (wildcard-addr), client, see-thru or self change category `outcome`: accepted something new in the internal configuration, removed something because it had been removed from external configuration sources, or rejected something (because of reason), or warning about a syntactically correct configuration input but one which wasn't going to work as expected (because of reason) `client`: IP address of an affected client `subnet`: block of IP addresses in the `a.b.c.d/e` notation of an affected subnet (mutually exclusive with `client`) `gid`: the Security Group ID that relates to this change `addr`: the FQDN, wildcard pattern or IP Address found in a Security Group rule `proto`: tls or ssh protocol as identified in the rule `reason`: reason for usually a warning or a rejected outcome `instance`: the EC2 instance ID of the DiscrimiNAT instance this log line was generated from If X.509 CRLs are [set to be allowed](/docs/discriminat/aws/default-prefs/#x509_crls) there will be an additional field: `http_path`: the document path over HTTP for CRL data And if a Security Group with a `see-thru` rule in it is found attached to a client application, there will be an additional field: `thru_date`: the date until which the non-blocking `see-thru` mode should remain effective; past this date such a rule would lose effect ## Filters The log lines are JSON-structured so each field can be addressed specifically. More of CloudWatch syntax can be explored on its [Filter and pattern syntax page](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/FilterAndPatternSyntax.html). #### _disallowed_ connections `{ $.outcome = "disallowed" }` #### _allowed_ but protocol is not TLS `{ $.outcome = "allowed" && $.proto != "tls" }` #### connections from a specific client where the destination host is not api.github.com `{ $.cat = "client" && $.src = "172.16.1.9" && $.dhost != "api.github.com" }` #### security groups where the see-thru mode exception has only 2 or less days left `{ $.see_thru_days_remaining <= 2 }` #### connections where the see-thru mode had to be used, but not for perhaps telemetry data `{ $.see_thru_exerted is true && $.dhost != "ec2-instance-connect.*" }` ## Recipes ### building an allowlist from scratch (video version) Watch this **3½ minute video** to build the allowlists super quick! The two commands used in the video are also documented here, underneath the video. Command 1: ``` aws logs filter-log-events --output json --no-paginate --log-group-name DiscrimiNAT --log-stream-names flow --filter-pattern '{$.dhost != "" && $.see_thru_gid = '$SECURITY_GROUP_ID' && $.see_thru_exerted is true}' --start-time $(date --date '2 hours ago' +%s000) | jq '[.events[].message | fromjson]' | dd of=frequency-fqdns.json ``` Command 2: ``` cat frequency-fqdns.json | jq --raw-output 'group_by(.dhost) | map(.[0] + {"count": length}) | sort_by(.count) | reverse | [.[].dhost] | @csv' | tr --delete '"' ``` ### building an allowlist from scratch (text version) :::info This method is now deprecated in favour of the CloudShell based method above. ::: 1. Create a new Security Group, with a `see-thru` rule in it. See the config reference and an example [here](/docs/discriminat/aws/config-ref#see-thru-mode). Give it a thru date sufficient enough to capture all stages of the application's lifecycle, such as deployment, restart, the occassional uploading of reports, telemetry, monitoring, etc. Note the Security Group ID. 2. Attach this Security Group to the application. In AWS this could be either attaching the Security Group to the EC2 instances in addition to other attachments, or in the case of serverless workloads, to the Network Interface (ENI) that is in the VPC. 3. Let the application follow its normal course of lifecycle. This could last a few hours, days or weeks depending on the application. 4. Go to **CloudWatch Log Insights** and select the DiscrimiNAT log group. Then enter the following query, select an appropriate time range, and run it. _Note: You'll have to replace the Security Group ID in this example with the one that was created above._ ``` filter see_thru_exerted AND see_thru_gid = "sg-00replaceme00" | stats count() by see_thru_exerted, see_thru_gid, dhost, proto, dpt ``` This will produce a table of results like: ![](img/recipe-discovering-rules-to-be-made.png) 5. From the information in the results table, specific allowlist protocol rules can be created. For example: `discriminat:tls:ec2.us-west-2.amazonaws.com,ssm.us-west-2.amazonaws.com` and `discriminat:tls:api.github.com` with protocol TCP and destination port 443. ![](img/egress-fqdn-rule-made-from-log-metrics-of-see-thru-mode.png) Full reference for creating these protocol rules is [here](/docs/discriminat/aws/config-ref#tls-and-ssh-mode). 6. Once these protocol rules are attached to the application, run the full application lifecycle again, and then the query at #4 to ensure that `see_thru_exerted` did not have to be used to let any disallowed traffic through. There may be the case where you choose to disallow certain destinations anyway, such as endpoints that just collect telemetry data. 7. Detach or remove the `see-thru` rule from the application and give it another full lifecycle run to ensure it works smoothly. --- ## Quick Start (from Marketplace) So you've heard about DiscrimiNAT, and would like to take it for a test drive? This is the page for you! Here you'll find easy instructions on how to deploy a brand new VPC in AWS, secured by the [DiscrimiNAT Firewall](/discriminat/) for egress traffic filtering right from the onset, and instructions on how to evaluate its capabilities in order to see if it fits your business needs. :::info Contact us for expert help at `devsecops@chasersystems.com` at any stage of your journey – we'll jump on a screen-sharing call right away! ::: --- ### i. MARKETPLACE SUBSCRIBE ![AWS Marketplace Subscription](img/01_subscribe.png) Proceed to our [AWS Marketplace page](https://aws.amazon.com/marketplace/pp/prodview-7ulmdnoq5jnwu) and click the **Continue to Subscribe** button. This step is entirely self-service. --- ![AWS Marketplace Subscription](img/01_accept-terms.png) You will need to press the **Accept Terms** button next and wait until it's ready for configuration. --- ![AWS Marketplace Subscription](img/01_subscribed.png) Once ready, do **not** proceed with the configuration, instead read on and press the **Launch Stack** button present in the next section here. --- ### ii. CLOUDFORMATION TEMPLATE – DEMO ENVIRONMENT :::tip If you have [deployed through Terraform](/docs/discriminat/aws/iac/#terraform-modules) or are considering, you may jump straight to [Configuring an Allowlist](#vi-configuring-an-allowlist). ::: Our self-service demo (using CloudFormation) includes a couple of instances to make it easy to evaluate the capabilities of DiscrimiNAT. Ensure you are logged into the AWS console and have selected the intended region for the demo before you click the following button. [![Launch Stack](img/launch-stack.svg)](https://console.aws.amazon.com/cloudformation/home?#/stacks/create/review?templateURL=https://cloudformation-aws-discriminat-gwlb.s3.eu-west-2.amazonaws.com/2az_demo.json&stackName=plugleak) --- ### iii. STACK PARAMETERS ![CloudFormation Stack Parameters](img/03_stack-parameters.png) :::info Contact us for expert help at `devsecops@chasersystems.com` at any stage of your journey – we'll jump on a screen-sharing call right away! ::: You may want to change the **Stack name**, everything else has safe defaults. **Stack name:** This is a unique identifier for your deployment. In our case, and the case of this documentation, we have chosen _plugleak_. This name will prefix a lot of the resources that the stack will create. If you choose a different name and are following this demo through, don't forget to replace _plugleak_ with _your-chosen-name_ in the relevant steps. #### EXAMPLE APPLICATION EGRESS ALLOWLIST CONFIGURATION This section contains optional customisation for the demo allowlist. **MyAppHTTPSAllowlist:** A comma-separated list of domains (without whitespace) that you wish your demo app to be able to connect to via a HTTPS or TLS connection. Note: Most services now use TLS for end-to-end encryption. It's not just for web page traffic, this also covers a number of other services such as SMTP, Databases, Kafka, etc. **MyAppSSHAllowlist:** A comma-separated list of domains (without whitespace) that you wish your demo app to be able to connect to via SSH. Note: This will also cover SFTP, SCP and GitLab/GitHub runners. :::info Feel free to change these to whatever you like, but for the rest of the documentation for this demo, I'll assume you left the defaults in place. If you're following along and have changed these, don't forget to substitute the domains shown for your domains. ::: #### OPTIONAL Example VPC Network Configuration If you wish to customise the network configuration, you can do so here. But it is not necessary for the purposes of this demo. The fields are fairly self explanatory, but I'll briefly describe them now. **VpcCidr:** The VPC-level CIDR block to deploy in this region. **PrivateSubnetAZ1:** CIDR for the Private Subnet in Availability Zone 1. **PrivateSubnetAZ2:** CIDR for the Private Subnet in Availability Zone 2. **PublicSubnetAZ1:** CIDR for the Public Subnet in Availability Zone 1. **PublicSubnetAZ2:** CIDR for the Public Subnet in Availability Zone 2. #### OPTIONAL DiscrimiNAT ADVANCED CONFIGURATION **InstanceType:** this, by default is set to `t3.small`. This is completely adequate for the purposes of this demo, as it will only require a small instance. In production, we'd suggest a more powerful instance type. Valid values are `t3.small` , `c6i.large` , `c6i.xlarge` , `c6a.large` , `c6a.xlarge` . :::info The default of `t3.small` should suffice for light to medium levels of usage. Anything less than 2 CPU cores and 2 GB of RAM is not recommended. For performance testing, faster access to the Internet or for accounts with a large number of instances (ENIs, rather), you may want to choose a machine type with dedicated CPU cores. ::: Finally, just tick on the checkbox that says _I acknowledge that AWS CloudFormation might create IAM resources with customised names_ then click on **Create stack** and the stack will begin to build. --- ### iv. SMALL WAIT ![CloudFormation Deployment Wait](img/08_cloudformation_05.png) The stack is now being deployed, it should only take a few minutes, and assuming your IAM permissions are correct, it should complete successfully like so: ![CloudFormation Stack Deployed](img/09_cloudformation_06.png) :::caution We will be using AWS Session Manager to connect to demo instances later, and due to a race condition between when the DiscrimiNAT Firewall comes up and when the SSM agent checks its connection, it could be **around 5 minutes** after creation before you're able to connect to them due to the agent back-off time period, so it is advisable to wait a few minutes after creation before continuing. ::: --- ### v. OVERVIEW OF WHAT'S BEEN CREATED #### v(a). THE ARCHITECTURE OF DiscrimiNAT INSIDE THE VPC The very short version is DiscrimiNAT replaces your NAT Gateways. If that's all you want to know, feel free to jump to [section v(b)](#vb-the-demo-environment) otherwise, let's take a look. First you'll see we have the VPC which has both private and public subnets. These subnets are set up with the CIDR ranges given by the stack variables. You'll also see the route tables for the public and private subnets ![VPC Layout](img/10a-vpc.png) You can see from here that the public subnet route tables go through the Internet Gateway as usual. This is also where the DiscrimiNAT instances sit, much like NAT Gateways. However the private routing is slightly different, so lets take a look at the private route tables. ![Private Route](img/10a_private_route.png) As we can see here the route to `0.0.0.0/0` is going out via a VPC endpoint. ![GWLB Endpoint](img/10a_endpoint.png) And that endpoint is a GatewayLoadBalancer (GWLB) type. :::info Due to the way GWLBs and AWS Networking interact, GWLBs require a VPC Endpoint per subnet. ::: Looking in Load Balancers under EC2 we see that there is only one in this VPC and it's of type `gateway`. ![Gateway Load Balancer](img/10a_gwlb.png) And looking at the load balancer, it's forwarding to this target group, which contains the DiscrimiNAT instances. ![Target Group](img/10a_target_group.png) The short version of this is the following: The private subnets (where your protected instances will live) route via a VPC endpoint to a GWLB which then forwards on to the DiscrimiNAT instances. This means that, unlike other architectures, if a DiscrimiNAT instance becomes unhealthy there will be no break in service while it's replaced. You can read more about this in our [99 Problems but Load Balancing ain't one](/blog/99-problems-but-load-balacing-aint-one/) blog post, or the [Reference Architectures](/docs/discriminat/aws/reference-architectures/) page. An even shorter version is that functionally, DiscrimiNAT replaces your NAT Gateways. :::info You *can* run NAT Gateways along with DiscrimiNAT, but you will be both double NATing and paying for extra services. We haven't found a practical benefit to running both. ::: #### v(b). The demo environment Looking at the instances we can see there are 4 instances here. ![Instances](img/10b_instances.png) The top two are instances of DiscrimiNAT Firewall, but we also have two other instances. These are here to emulate protected instances. **MySecuredApp-plugleak:** This instance has allowlists applied to it, both for TLS traffic and SSH traffic. We will look into how this is done shortly. **MyMonitoredApp-plugleak:** This instance has been put into our _see-thru_ mode which allows you to monitor the traffic the instance is generating without affecting the connections in any way. :::info Monitoring, or _see-thru_ mode, is exceptionally useful in the discovery phase of your implementation. Unlike a lot of solutions, we allow you to examine the traffic going out, so you can add what's needed, know if needed rules are present, and then switch over to a secured mode with confidence. See our video guide 🎥 on that [here](/docs/discriminat/aws/logs-ref/#building-an-allowlist-from-scratch-video-version). ::: --- ### vi. CONFIGURING AN ALLOWLIST #### vi(a) LOCATION AND SYNTAX If we look at the instance called **MySecuredApp-plugleak** we will see it has three Security Groups attached. ![Security Group list](img/11_secgroups_list.png) DiscrimiNAT checks what domains each instance is allowed to access by seeing what Security Groups are attached to the instance, and then looking at the description of the Outbound Security Group Rules. Let's take a look at one. ![zoomed out view](img/11_secgrp_zoomout.png) If we look at the highlighted section more closely, we can see that the outbound rule has text in the description field. Specifically some colon-separated values. This is where DiscrimiNAT reads its config from. ![config view](img/11_secgrp_zoomin.png) We can break this down as follows: **discriminat:** this tells DiscrimiNAT that this description field should be parsed and the rules found within should be applied to anything attached to this Security Group. **tls:** this tells DiscrimiNAT that this is for TLS connections rather than SSH. **api.github.com,icanhazdadjoke.com:** this is a comma-separated list of domains that any instance attached to this Security Group can have access to. Alternatively, this list can be stored in an [SSM Parameter in JSON format](/docs/discriminat/aws/config-ref/#tls-and-ssh-mode). As you can see, this allows for least-privilege allowlisting, each instance only needs to have the list of domains that **it** needs to access in its allowlist. We encourage you to create Security Groups as you would normally. Tight in scope and **attached to their respective applications**. #### vi(b) EFFECTIVE USE OF DiscrimiNAT USING SECURITY GROUPS This section is to give you a basic grounding on how to configure Security Groups for use with DiscrimiNAT, if you are simply interested in going through the demo, you can carry on to [section vii](#vii-discriminat-in-action). For an effective use of the DiscrimiNAT Firewall: - Add destination protocols and [FQDNs](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) to the description field of each **outbound rule**. The format of this text is `DiscrimiNAT::[,]` . Alternatively, the list can be stored in an [SSM Parameter in JSON format](/docs/discriminat/aws/config-ref/#tls-and-ssh-mode). - Valid protocols are `ssh` and `tls` only. - There can be as many FQDNs as you can fit separated by commas. There cannot be whitespace, though. - Some examples of what can go in the description field: - `DiscrimiNAT:TLS:api.foo.example.com,downloads.example.net` - `DiscrimiNAT:SSH:sftp.txs.example.org,ssh.github.com` - All DiscrimiNAT rules must be under `Outbound rules`. Inbound Rules are not read. - Under `Type`, choose any TCP protocol type and **specify exactly one port number** in `Port range`. Multiple ports, ranges, other protocols, and so on are **not** supported. - For your typical **HTTPS** outbound connection, the port number would be `443`. - A typical **SFTP** connection has the port number of `22`. - `Destination` must be set to the narrowest IP block you can think of. Now if you don't have anything narrower than `0.0.0.0/0` that is absolutely fine! The firewall takes care of validating the destination IP address of actual packets against the given domain names' DNS etc., but the Outbound Rule at the AWS networking and security level must allow the packet to leave the EC2 Instance. **`0.0.0.0/0` is a safe choice with the DiscrimiNAT Firewall in the way.** - That's it! Although you may want to check the [full reference](/docs/discriminat/aws/config-ref/) for more possibilities. --- ### vii. DiscrimiNAT IN ACTION Now we're going to connect to one of our instances using AWS Session Manager. In the AWS EC2 console, find the instance called **MySecuredApp-plugleak** and click on its name, then click on **Connect**. :::tip It is advisable to **wait for over 5 minutes** after stack creation to do this, as the SSM agent might not have had the time to reconnect. The instances come up at the same time as the DiscrimiNAT instances in this CloudFormation stack and there can be a race condition as described earlier. ::: On the next page, ensure you have chosen to connect via **Session Manager**. If this is disabled, wait for a few minutes for the agent to reconnect. ![SSM Connect](img/12_ssm_connect.png) Now we're connected, let's test `curl`ing to an FQDN we have configured in the allowlist. Then let's test an FQDN we haven't configured in the allowlist. First run `curl https://istio.io` (an FQDN not in our allowlist). We will immediately see a connection reset. This is because the rules attached to this machine do not include this FQDN. :::tip We give a connection reset rather than a hang as it gives fast feedback to developers. A hang (drop & timeout), especially if connecting to multiple domains, or the same domain multiple times, can waste developer time or stall applications. ::: Now run `curl https://icanhazdadjoke.com` (an FQDN in our allowlist) and we'll see a dad joke appear in the command line because this FQDN is allowed. :::info Chaser Systems are not responsible for the quality of dad joke you are given. Dad jokes are read at your own risk. ::: ![Command Output](img/12_curl_view.png) Let's take the view that our organisation has decided they need this machine to be able to connect to **istio.io**. It's been agreed by everyone, so let's add **istio.io** to our Security Group Rule Description to add it to DiscrimiNAT's config. ![Add istio to security group](img/12_add_istio.png) We will now need to wait around a minute for the rule to be picked up. :::info DiscrimiNAT polls the AWS API at 1-3 times a minute, and as there is no global store, each instance picks up the config individually, so new rules usually take around a minute to go live. The lack of a global store means no databases are required and makes the machines stateless and easily replaceable. ::: Now we will try `curl https://istio.io` and we'll get a very large HTML response. ![istio now works](img/12_istio_works.png) As you can see, DiscrimiNAT is simple to configure and does exactly what it's supposed to do. However one thing we haven't shown you yet is something most of our competitors _can't_ do. And that is: Combat **SNI Spoofing**. :::info SNI Spoofing put simply is when an attacker attempts to make a connection by claiming they're connecting to a specific DNS host in [TLS SNI](https://www.cloudflare.com/en-gb/learning/ssl/what-is-sni/) but are actually using an IP address completely unrelated to that domain name. A [lot of firewalls](/discriminat/comparison/aws-network-firewall/) just check to see if the TLS SNI contains the correct domain name, without checking whether the IP is actually valid for that domain name. The other competitors that *do* check the IP is valid are prone to false positives, and the destination only works *some* of the time. DiscrimiNAT combats this using our proprietary **Wormhole DNS** technology. ::: One of the simplest ways to spoof SNI is to edit the *hosts* file, so let's do that. * Run `sudo vi /etc/hosts` * Once the file is open press the **o** key. * insert the following line `8.8.8.8 istio.io` * press **escape** then the following keys **:wq** then **return** Now, when you try to `curl https://istio.io`, it will use the IP address in the *hosts* file (which is a Google IP) instead of doing a DNS query. [Lesser firewalls would let this through](/discriminat/comparison/aws-network-firewall/#litmus-test), but DiscrimiNAT won't. ![You Shall Not Pass!](img/12_sni_proof.png) As you can see, DiscrimiNAT has blocked the traffic instantly. It will even report on this on the logs. Talking of which... --- ### viii. HOW DiscrimiNAT LOGS WORK DiscrimiNAT has no UI of its own, instead it ties into your cloud service, so in this case we'll be looking at CloudWatch. :::tip DiscrimiNAT can send its logs to wherever you need, and can support most Linux-based logging agents such as Splunk, Rapid7, etc. ::: In CloudWatch, browse to **Logs -> Log Management -> DiscrimiNAT -> flow**. The flow log shows all hosts going through the DiscrimiNAT instances and whether the traffic was allowed out or not, along with some other information. Let's search in here specifically for **istio**. ![Cloudwatch Search Results](img/13_istio_search.png) You should see at least 3 results here, one where you `curl`ed at first and it was disallowed, one after the rule was in place and it worked, and one where we tried to SNI spoof. Let's look at these one by one. ![Istio wasn't allowed](img/13_istio_disallow.png) We can see here that this was an attempt to connect to **istio.io** that was **disallowed** and the reason is because there were no matching rules found. Now let's have a look at the working one. ![Istio was allowed through](img/13_istio_allowed.png) We can see here that this was an attempt to connect to **istio.io** that was **allowed** and the reason is because there was a matching rule in Security Group **sg-0daabff1c073ac1f7** that was attached to the instance. So let's take a look at when we tried to trick the firewall into letting spurious traffic through. ![No Spoofing Allowed](img/13_istio_spoofed.png) We can see here that this was an attempt to connect to **istio.io** that was **disallowed** and the reason is because there was **spoofing detected**. As you can imagine, this is something your security team would be very interested in as it indicates an exfiltration or a C2 attempt. Let’s now look at the configuration logs. Browse to **CloudWatch -> Logs -> Log Management -> DiscrimiNAT -> config**. Here we can see any config changes and errors. For example here we can see our *istio* rule being picked up by both DiscrimiNAT instances. ![Config logs](img/13_config_log.png) This is useful not only for fault diagnosis, but also for compliance reasons as any changes to the allowlists are logged. :::tip As you can see, all our logs are JSON and therefore you can query them and pull information out easily. You can even do it on the Cloud Shell CLI using a combination of the **aws** command and **jq**. For an example, see the recipe [here](/docs/discriminat/aws/logs-ref/#building-an-allowlist-from-scratch-video-version). ::: --- ### ix. MONITORING (see-thru) MODE One of the toughest parts of adopting egress filtering is the initial phase. Planning, knowing what to actually put in your allowlists, trying to make sure they're not too open, while making sure your app and the underlying OS have everything they need to function correctly. Our monitoring mode is designed specifically to help with this and aid with the adoption. We have an instance configured in _see-thru_ in our demo environment, so let's look at the Security Groups attached to the instance called **MyMonitoredApp-plugleak**. ![Security Groups attached to the monitored instance](img/14_sec_groups.png) As we can see here, one of the groups is called **MyApp-Monitored-Egress-plugleak**, lets take a look at this, and the description on the outbound rule. ![The configuration for monitoring mode](img/14_sec_group_config.png) So we can see here that we have _discriminat_ to indicate the DiscrimiNAT should read this config, followed by _see-thru_ which tells DiscrimiNAT that the instances attached to this Security Group are in monitoring mode, followed by _2026-12-31_ which is the date when the monitoring mode will expire. :::caution When monitoring mode expires, the rule is disregarded and the traffic goes back to being blocked. ::: Let's connect to the monitored box using SSM (using the same method as the other instance) and run the following commands. ``` curl --head https://yahoo.com curl --head https://graph.facebook.com curl --head https://apple.com curl --head https://youtube.com ``` You'll find they all work. It's as if DiscrimiNAT isn't there. So let's take a look at the logs. Navigate to **CloudWatch -> Logs -> Log Management -> DiscrimiNAT -> flow**, then search for **yahoo.com**. ![log results](img/14_yahoo.png) Let's expand that log entry. ![The log entry for yahoo.com](img/14_yahoo_log_entry.png) You may notice that there are extra fields in this log entry compared to the others we've looked at. These are **see_thru_exerted**, **see_thru_gid**, and **see_thru_days_remaining**. This is an easy way to tell that this log entry comes from a monitored group. Let's go through what these mean. **see_thru_gid**: this tells you which Security Group this log entry has come from. So if you have multiple apps, all in monitor mode, you can still tell which is which. **see_thru_days_remaining**: How many days there are until the monitoring shuts off and the traffic starts being blocked as normal. _(Note, we do not use 0. The last day is 1 and the first non-working day is -1.)_ **see_thru_exerted**: Did the object (eg. an instance or a lambda) needed to have the monitoring mode on to be able to access this host? For example, if there is another Security Group attached to it allowing it through, then this value would be `false`. If this has been set to `true` then this traffic would not have been allowed through under normal circumstances. :::tip Using an `aws logs` command, you can pull through all the hosts that the machine connects to, and even more than that, using **see_thru_exerted**, you can choose to see which hosts haven't been allowed through via other Security Groups yet. For a fully worked out example, see the recipe [here](/docs/discriminat/aws/logs-ref/#building-an-allowlist-from-scratch-video-version). ::: ### x. TEARDOWN Teardown is just as simple as deleting the CloudFormation stack. Navigate to **CloudFormation** in the AWS console, find the stack named **plugleak** and delete it. It will then tear down all the resources in the demo environment. --- ### xi. NEXT STEPS Hopefully now you've seen what DiscrimiNAT can do for your organisation, and how easy it is to configure. Get a Demo Explore the Terraform module :::tip DiscrimiNAT has an ever-growing feature set. If there is a feature you think is missing and that is blocking your adoption, please get in touch. It may already be on our roadmap. If it isn't and we think it's a good idea, it can be added to our roadmap. ::: --- Troubleshooting Discovering FQDNs Recipe --- ## Reference Architectures The DiscrimiNAT Firewall supports three deployment topologies. You are free, however, to deploy it as you see fit and consult our [DevSecOps Support](/support/) as needed. :::info In a **Shared VPC** setup, DiscrimiNAT must be deployed in the VPC Owner account. ::: The key differentiators between the topologies are laid out in the table below. |Topology|FailureTime to Detection|FailureTime to Recovery|Min Instancesper AZ|Inter-AZData Transfer|Auto Scaling|Max Gbps| |-|-|-|-|-|-|-| |[GWLB in Cross-Zone](#gwlb-in-cross-zone-mode)|10 seconds|60 seconds†|1|yes|yes|100| |[GWLB in Intra-Zone](#gwlb-in-intra-zone-mode)|10 seconds|60 seconds†|2|no|yes|100| |[ENI per AZ](#eni-per-az)|2 seconds|150 seconds‡|1 (also max)|no|no|5| **†** This is GWLB behaviour as quoted below. The health checks' interval, though, is set to 5 seconds in our Terraform modules therefore reducing the 20-second delay from them to 10 seconds. > The minimum duration to start re-routing new flow is up to 70 seconds. It is a sum of 20 seconds for health checks (Min. Interval: 10s, Min. threshold: 2) and 50 seconds for GWLB backend to detect and re-route. Source: [https://aws.amazon.com/.../best-practices-for-deploying-gateway-load-balancer/](https://aws.amazon.com/blogs/networking-and-content-delivery/best-practices-for-deploying-gateway-load-balancer/) **‡** In practice, this is how long EC2 Auto Scaling takes to replace a DiscrimiNAT Instance upon receiving `Unhealthy` _HealthStatus_ on the [SetInstanceHealth API](https://docs.aws.amazon.com/autoscaling/ec2/APIReference/API_SetInstanceHealth.html). The GWLB topologies do not use the SetInstanceHealth API for recovery, but instead are purely load balancer health check managed. ## GWLB in Cross-Zone mode In the `cross-zone` mode, the Gateway Load Balancer (GWLB) will distribute traffic evenly across all deployed AZs. This reduces the number of DiscrimiNAT Firewall instances you will have to run for high-availability but increases data-transfer costs. :::note Terraform variable `high_availability_mode` should be set to `cross-zone`. This is also the default. The Terraform module is [here](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws). ::: :::caution Minimum number of allocated Elastic IPs for high-availability (=2) with headroom for auto-scaling (+1) is **3 per region**. ::: ![](/img/gwlb-cross-zone-v2.drawio.png) ## GWLB in Intra-Zone mode In the `intra-zone` mode, the GWLB will distribute traffic evenly across all DiscrimiNAT Firewall instances in the same AZ as the client. For effective high-availability, this mode will need at least two instances per deployed AZ. Please note this does not fully protect you against the failure of an entire AZ on the Amazon side, however your other services in the zone would potentially be impacted too and therefore not sending egress traffic. :::note Terraform variable `high_availability_mode` should be set to `intra-zone`. The Terraform module is [here](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws). ::: :::caution Traffic will not be balanced to other zones, even in case of failure of all instances in one zone, therefore minimum high-availability numbers (=2) have to be configured per AZ. ::: :::caution Minimum number of allocated Elastic IPs for high-availability (=2) with headroom for auto-scaling (+1) is **3 per AZ**; and therefore 6 for two AZs. ::: ![](/img/gwlb-intra-zone-v2.drawio.png) ## ENI per AZ A single DiscrimiNAT instance, wrapped in an Auto Scaling Group of its own, is deployed per AZ without any load balancers. Routing is directly to the ENI (Network Interface) of this instance. The ENI is statically bound to the Launch Template in the provided CloudFormation & Terraform modules, therefore only one active instance per AZ can exist at a time. Should the built-in health check fail, the instance will be terminated and a new one brought in its place by its Auto Scaling Group. This process takes up to 150 seconds. The Terraform module for this topology is [here](https://registry.terraform.io/modules/ChaserSystems/discriminat-eni/aws). ![](img/20200523_1discrimiNAT_160.png) --- ## Release Notes ## version 2.40 (2026-05-06) * added a Post-Quantum Cryptography `pqc` key exchange field in flows logs that indicates if a TLS handshake used a known [PQC TLS Supported Group](https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-8). The value may be _true_ or _false_ in case of TLS; or absent in the case of SSH. This feature, although in preview, can be used to assess PQC readiness in your outbound connections. * TLS SNI matching against user-defined rules is now case-insensitive to support client-side libraries that did not lowercase user input before initiating a new outbound connection * broadened support for some legacy SSH ciphers * lowered disk usage and telemetry data collection volume (when left enabled) * a mitigation for [CVE-2026-31431](https://ubuntu.com/security/CVE-2026-31431) (aka [Copy.Fail](https://copy.fail/)) is included with package _kmod_ version _31+20240202-2ubuntu7.2_, [as advised by Canonical](https://ubuntu.com/blog/copy-fail-vulnerability-fixes-available) * version of bundled Public Suffix List: [8b4345f](https://github.com/publicsuffix/list/blob/8b4345f9a2513011b21e6fc7b8a7197a849be52c/public_suffix_list.dat) **Terraform Updates** * `custom_deployment_id` added to Terraform modules to afford parallel but distinct deployments in the same AWS account. External IPs, the CloudWatch Log Group name and Default Preferences can now have a unique identifier tied into a DiscrimiNAT deployment of the same ID. * DiscrimiNAT instances are now able to read their own tags via the metadata service. This is necessary for the feature above to work. * full diff between previous and this version can be found [here](https://github.com/ChaserSystems/terraform-aws-discriminat-gwlb/compare/2.30.0...2.40.0) for the **-gwlb** module, and [here](https://github.com/ChaserSystems/terraform-aws-discriminat-eni/compare/2.30.0...2.40.0) for the **-eni** module ## version 2.30 (2026-03-12) * improved and quicker new ENI detection. This should improve new ECS Task detection as a side-effect. * a new log message `undiscovered client` in case an ENI has not yet been polled and discovered by DiscrimiNAT, separating this circumstance from the `no matching rules found` log message * Post-Quantum Cryptography (PQC) TLS cipher exchange in ClientHello support **in preview** even if the handshake is split over multiple IP packets * fixed a bug where ENIs of type 'branch' were causing all known ENIs (clients) to be dropped * version of bundled Public Suffix List: [7ef6384](https://github.com/publicsuffix/list/blob/7ef6384612e1b48bb8b6023716cc9a493ac25d8a/public_suffix_list.dat) ## version 2.20 (2025-08-27) * wildcard matched FQDNs no longer cause a `cache not ready` flow log event on DiscrimiNAT instance restart or upgrade. This is accomplished by sharing knowledge of such FQDNs between DiscrimiNATs: * by discovering peers which either have label key or label value set to `discriminat` (case-insensitive) * exchanging knowledge continuously over port 1042 UDP with peers * incoming peers learn from outgoing peers during a rolling restart/upgrade * related and necessary changes have been made in all our Terraform (and other IaC) modules * messaging between peers is encrypted and authenticated, and does not contain policy * caveats: * upgrading from any version older than 2.20 will, for once and the last time, cause wildcard matching FQDNs to not work for a few seconds until the cache is warmed up again * from v2.20 onwards, this should not occur as long as one-at-a-time rolling upgrades are made (default behaviour of our IaC modules) * single-instance deployments will continue to lose memory of wildcard matched FQDNs and require a few seconds to allow them again after first encounter * new config log _reason_: `malformed fqdn`. This is emitted in case a syntactically invalid domain name is specified in an allowlist * improvement in how quickly new clients (Elastic Network Interfaces) are detected * version of bundled Public Suffix List: [44211b0](https://github.com/publicsuffix/list/blob/44211b0fbb8b4e09b89a553f5b26c6a02f889c4b/public_suffix_list.dat) * underlying OS has been changed from Ubuntu 20.04 LTS to Ubuntu 24.04 LTS * version number strategy has changed: * DiscrimiNAT itself will only use Major and Minor in its version numbers, and not use the Patch level from the [semantic versioning scheme](https://semver.org/). For example, this release is 2.20 instead of 2.20.0 * related components to a specific version of DiscrimiNAT will follow the full semantic versioning scheme, where the Major and Minor will correspond to the recommended version of DiscrimiNAT. For example, Terraform modules' version 2.20.0, 2.20.1, 2.20.2 and so on will be best suited for DiscrimiNAT 2.20 **Terraform Updates** * allowed UDP port 1042 to and from DiscrimiNATs' security group for messaging between peers, as mentioned above * added tags to launch template network interfaces * added tags to a few other resources * full diff between previous version and this can be found [here](https://github.com/ChaserSystems/terraform-aws-discriminat-gwlb/compare/2.9.0...2.20.0) for the **-gwlb** module, and [here](https://github.com/ChaserSystems/terraform-aws-discriminat-eni/compare/2.9.0...2.20.0) for the **-eni** module ## version 2.9.0 (2024-11-28) * default preferences (see [docs](/docs/discriminat/aws/default-prefs/)), of which there are four at this time, can now be stored in an SSM Parameter named `DiscrimiNAT` (case-sensitive): * `wildcard_exposure`: control whether `*` is accepted in FQDNs or not, and if accepted, with prohibit [public suffix](https://publicsuffix.org/) safeguard or not. **Please familiarise yourself with [operation and caveats](/blog/wildcards-and-system-2-thinking/#operation) on the behaviour of wildcard rules before using them.** * `flow_log_verbosity`: control whether all logs are emitted, just _disallowed_ or none at all * `see_thru`: set non-blocking, monitoring _see-thru_ mode as a default (useful with new deployments so traffic is not blocked upon deployment) * `x509_crls`: whether to automatically allow CRL Endpoints of x509 SSL certificates for all TLS FQDNs allowlisted. This was a feature introduced in v2.7.0 * list of FQDNs for allowlisting, in JSON format, can now be read in from an SSM Parameter or a Secret in Secrets Manager * format is `{"addrs": ["fqdn1.com", "*.github.com", ...]}` * a symbolic reference of the Parameter/Secret, using its full ARN, will need to be added to the Security Group Rules' description field instead of comma-separated FQDNs. For example, `discriminat:tls:arn:aws:ssm:eu-west-2:111111111111:parameter/team-foo-allowed-fqdns` and `discriminat:tls:arn:aws:secretsmanager:eu-west-2:111111111111:secret:service-foo-allowed-fqdns` * DiscrimiNAT's IAM Role's Policy will need _ssm:GetParameter_ (for SSM Parameters) and _secretsmanager:GetSecretValue_ (for Secret) permissions on the ARNs referred * **previous method of storing comma-separated FQDNs in Security Group Rules' description fields is preserved and will continue to work** * version of bundled Public Suffix List: [931546b](https://github.com/publicsuffix/list/blob/931546b3beb45b544d0692aa116b420fb34b9dfa/public_suffix_list.dat) * improvement in startup time with a large allowlist (>500 FQDNs) * new config log _reason_: `rejected`. This is emitted in case a wildcard FQDN is specified but the _wildcard\_exposure_ preference value does not allow its inclusion. For example, ``{addr: "*.github.io", cat: "addr", outcome: "publicsuffix[.]org list matched with `github.io`"}`` * raw packet captures (PCAP) for ~10 seconds may be included in telemetry data if Automated System Health Reporting is not opted out of * fixed a bug where wildcard matched FQDNs would not be allowed, until the cache for them was warmed up, in spite of _see-thru_ mode being set * fixed occassional "spoofing detected" flow logs disallowing connections to Cloudinary and Azure Cloud CDN FQDNs * TLS and SSH connectivity improvements to some hosts that would not acknowledge trailing zeroes in padding bytes of a handshake * absence of leading zeroes in month and date components of a _see-thru_ date now works. For example, previously, `2024-9-1` would not have worked, however, `2024-09-01` would have. Both work now. * wildcard matched connections are now timed out on first attempt, instead of connection reset, until the cache has warmed up for them. This reduces the number of attempts made by an app/client when accessing a wildcarded FQDN for the first time. **Breaking Changes** * plaintext HTTP CRL Endpoints of x509 SSL certificates for all TLS FQDNs allowlisted are no longer allowed automatically. Set the preference `x509_crls` to `auto_allow` to restore previous behaviour. If you did not know about this, you are likely not affected. We have proactively informed the customers we definitely knew were relying on this. **Terraform Updates** * [preferences' defaults](/docs/discriminat/aws/default-prefs/) in JSON format are deployed automatically, along with the IAM Policy permission for DiscrimiNAT to be able to read them, from v2.9.0 of our Terraform module to serve as a starting point. Can be overriden from the new `preferences` variable. * option to disable automatic updates to the Launch Template when a new DiscrimiNAT AMI version is available with the `ami_auto_update` boolean variable * renamed variable `ami_name` to `ami_version`. It now accepts the _semver_ for DiscrimiNAT. * list-type variables `iam_get_additional_ssm_params` and `iam_get_additional_secrets` introduced to automatically add IAM Get permissions on DiscrimiNAT's IAM Role's Policy for the ones specified. This enables DiscrimiNAT to read allowlists specified in these resources. * full diff between previous version and this can be found [here](https://github.com/ChaserSystems/terraform-aws-discriminat-gwlb/compare/2.8.0...2.9.0) for the **-gwlb** module, and [here](https://github.com/ChaserSystems/terraform-aws-discriminat-eni/compare/2.8.0...2.9.0) for the **-eni** module ## version 2.8.0 (2024-08-20) * wildcard support is now in preview: * they are supported for the TLS protocol only * the character `_` may be used to substitute one wild character in an FQDN to be allowed * the set of wild characters is from `a` to `z`, `0` to `9` and the `-` (hyphen or minus) only; the `.` (period, dot or fullstop) character is not included * you may use any number of wildcards in a single FQDN address (in the allowlist) * see our [dedicated page on using wildcards](/blog/wildcards-and-system-2-thinking/) with examples and the caveats expected in this preview; ensure you've read the __Operation section__ as well * further improvements are expected in the next version of DiscrimiNAT; please [write to us](/support/) with your experience on using this feature * suppressed repetitive warning log messages `no ip addresses resolved` about CRL endpoint `crl.comodo.net` * improved compatibility with proprietary SSH server-side implementations, such as GoAnywhere, that send a larger than normal list of ciphers during the initial handshake * Terraform module [v2.8.0](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws/2.8.0) will need to be used to deploy the v2.8.x AMIs of DiscrimiNAT ## version 2.7.1 (2024-02-12) * `warning` type messages in `config` logs now do not repeat before 10 minutes. This will reduce the frequency of `no ip addresses resolved`, etc. log messages significantly. * disabling **Automated System Health Reporting** can now be done by setting variable `ashr` to `false` from Terraform module [v2.7.1](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws/2.7.1) onwards. The earlier method will continue to work. * automated system health reporting, if left enabled, now runs at shutdown too. * automatically allowed CRL Endpoints of x509 SSL certificates now emit the name of the otherwise allowed FQDN and the issuing CA in the certificate chain, from which the CRL Endpoint was determined, in the `reason` field of the `config` log. For example, `crl endpoint from issuer GTS Root R1 in certificate chain of trends.google.com`. ## version 2.7.0 (2024-01-14) * HTTP `flow` Logs: additional log fields of `http_method`, `http_user_agent` and `http_path` will be present for plaintext HTTP traffic to aid in determining the source of unencrypted traffic. The traffic will always be denied with the message `insecure protocol, use https`, though. * CRL Endpoints of x509 SSL certificates, which are over plaintext HTTP, are now automatically allowed for all TLS FQDNs allowlisted. Only HTTP methods `HEAD` and `GET` are allowed to these URLs from only the clients that otherwise have the TLS FQDNs (to which these CRL Endpoints belong) allowed. * `no ip addresses resolved` warning message in `config` log for FQDNs found in the allowlist but for which a DNS lookup did not resolve any IP addresses. This is useful in spotting typos and domain names not configured yet by third parties. * Terraform module [v2.7.0](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws/2.7.0) update: variable ~~`startup_script_base64`~~ has been renamed `user_data_base64`. * **Automated System Health Reporting:** 10 minutes after boot and then at around 0200 UTC every day, each instance of DiscrimiNAT will collect its OS internals & system logs since instance creation, config changes & traffic flow information from last two hours and upload it to a Chaser-owned cloud bucket. This information is encrypted at rest with a certain public key so only relevant individuals with access to the corresponding private key can decrypt it. The transfer is encrypted over TLS. Access to this information will be immensely useful to create a faster and more reliable DiscrimiNAT as we add new features. We also aim to learn about how users interact with the product in order to further improve the usability of it as they embark on a very ambitious journey of fully accounted for and effective egress controls. We understand if certain environments within your deployment would rather not have this turned on. **To disable it,** a file at the path `/etc/chaser/disable_automated-system-health-reporting` should exist. From our Terraform module v2.7.0 onwards, this can be accomplished by including the following statement: ``` user_data_base64 = "I2Nsb3VkLWNvbmZpZwp3cml0ZV9maWxlczoKLSBwYXRoOiAvZXRjL2NoYXNlci9kaXNhYmxlX2F1dG9tYXRlZC1zeXN0ZW0taGVhbHRoLXJlcG9ydGluZwo=" ``` The _base64_ value above decodes to: ``` #cloud-config write_files: - path: /etc/chaser/disable_automated-system-health-reporting ``` Which is a [cloud-init](https://cloudinit.readthedocs.io/en/latest/reference/examples.html) way of creating that file in the instance. * Instance types `c5.large`, `c5.xlarge`, `c5.2xlarge`, `c6i.2xlarge` and `c6a.2xlarge` are no longer supported. Please choose from one of `t3.small`, `c6i.large`, `c6i.xlarge`, `c6a.large` and `c6a.xlarge`. * DiscrimiNAT Firewall's product code on the AWS Marketplace has changed from ~~`a83las5cq95zkg3x8i17x6wyy`~~ to `bz1yq0sc5ta99w5j7jjwzym8g`. This has no impact on users unless they had been using the product code to lookup its AMI ID, for example. Terraform modules version 2.7.0 onward encapsulate this change. ## version 2.6.1 (2023-10-13) * health check logic now also waits for the firewall cache to build up a bit before giving a green light to the load balancer (and therefore accepting traffic on new VMs) * two new warning log message types which indicate if the configured port in a Firewall Rule has a connection-level issue: * for example `test for TLS on 203.0.113.5:80 failed` – port 80 was not listening with TLS * and for example `timed out testing connection to 203.0.113.6:443` – port 443 on that IP address is not open (from DiscrimiNAT's public IP point of view) ## version 2.6.0 (2023-07-24) * the TLS notation for allowlisting now supports IP v4 addresses besides FQDNs (i.e. without SNI), for example `discriminat:tls:203.0.113.9` * the _see-thru_ monitoring mode now accepts all specifications of IP addresses, Protocols & Ports on Security Groups. Previously, it had required IP addresses to be set to `0.0.0.0/0` and Ports & Protocols to be to set to _all_. * two new `flow` log `reason` messages when a network packet is `disallowed` have been introduced: * `cache not ready`: this message is logged when a new address is added in the allowlist but the firewall has not yet warmed up its cache for it. Expected to occur for up to 2 minutes after adding a new address (FQDN or IP.) * `spoofing detected`: logged when TLS SNI has been manipulated and a connection is attempted to an IP address that doesn't otherwise belong to the given FQDN (in the SNI.) * enabled Finite Field Diffie–Hellman ciphers for TLS 1.2, for example DHE as opposed to ECDHE * enabled ciphers **without** Forward Secrecy for TLS 1.2 * instance types have been updated to `t3.small`, `c6i.large`, `c6i.xlarge`, `c6i.2xlarge`, `c6a.large`, `c6a.xlarge`, `c6a.2xlarge`, `c5.large`,`c5.xlarge` and `c5.2xlarge`. Note that the `c6a` AMD types, although cheaper and equally performant, may not be available in all Availability Zones. **Breaking Changes** * the `config` log has its `fqdn` field name changed to `addr`. Field names have not changed in the `flow` log. * in the `config` log, association of a public IP for egress from the firewall had its category (`cat` field) set to `static-ip`. This is now set to `egress-ip`. ## version 2.5.3 (2023-05-31) * increased tolerance towards some rare TLS servers that otherwise resulted in DiscrimiNAT logging connection test failures and not allowing connections to them ## version 2.5.2 (2023-03-27) * change of base OS from Ubuntu 18.04 to Ubuntu 20.04 The CIS Ubuntu Linux 20.04 LTS Benchmark v1.1.0 Level 2 - Server report is available upon request by contacting support. The image scored 215/219. An explanation will be attached for the unmet 4. ## version 2.5.1 (2023-02-03) * general OS updates ## version 2.5.0 (2022-11-07) * DiscrimiNAT now supports load balancing, high availability and auto scaling with [AWS' Gateway Load Balancer (GWLB)](https://aws.amazon.com/elasticloadbalancing/gateway-load-balancer/). * With the GWLB, the [RTO](https://en.wikipedia.org/wiki/Disaster_recovery#Recovery_Time_Objective) for DiscrimiNAT is reduced from ~120 seconds to 10 seconds! * [New `-gwlb` Terraform module published at the registry](https://registry.terraform.io/modules/ChaserSystems/discriminat-gwlb/aws) to deploy a load balancing, highly available and an auto scaling set of DiscrimiNAT Firewalls. **Breaking Changes** * CloudWatch log group name changed from `discrimiNAT` to `DiscrimiNAT`. The first letter of the word **D**iscrimiNAT is, and going-forwards will be, in upper-case. * IAM Policy has been updated to reflect the upper-case letter **D**. * AMI name has seen the same change. * A new patch version, [2.4.1, of the ENI Terraform module](https://registry.terraform.io/modules/ChaserSystems/discriminat-eni/aws/2.4.1) has been released that constrains it to DiscrimiNAT version 2.4.x AMIs only. The ENI Terraform module will be updated in due course to support DiscrimiNAT version 2.5.x and onwards. ## version 2.4.2 (2022-10-11) * improved connection handling for very short lived TLS connections with specific server-side implementations (such as Envoy Proxy) ## version 2.4.1 (2022-05-11) * fixed a sporadic connection reset issue, that emitted `unexpected response` in the logs, and which only occurred in the `see-thru` monitoring mode while connecting to a destination at very high latency ## version 2.4.0 (2022-03-01) * new warning message in config logs when a connection test to an FQDN, carried out by discrimiNAT itself, in any allowlist fails * added support for [self-attaching an allocated Elastic IP](/docs/discriminat/aws/iam-instance-profile#elastic-ips) * discrimiNAT's own instance ID added to every log line under the key `instance`, indicating which instance the log line was emitted from * updated [TLS ECH draft extension](https://datatracker.ietf.org/doc/draft-ietf-tls-esni/) identifiers ## version 2.2.0 (2021-09-06) * [see-thru mode](/docs/discriminat/aws/config-ref#see-thru-mode) introduced; build allowlists super-quick by putting a Security Group in monitor mode first * serverless support introduced; Lambdas etc. with an interface in the VPC will have their outbound traffic filtered * full bypass hook added; please reach out to [support](/support) for instructions on this ## version 2.1.0 (2021-08-19) * improved handling for a large number of FQDNs in the allowlists * updated [TLS ECH draft extension](https://datatracker.ietf.org/doc/draft-ietf-tls-esni/) identifiers ## version 2.0.5 (2021-05-11) * restricted firewall rule scanning to the same VPC as discrimiNAT firewall was deployed in ## version 2.0.4 (2021-04-07) * updated [TLS ECH draft extension](https://datatracker.ietf.org/doc/draft-ietf-tls-esni/) identifiers ## version 2.0.3 (2020-11-10) * v2 launch * completely new architecture addressing the potential for mismatch of IPs addresses as looked up by a protected workload from the VPC resolver and as looked up by the discrimiNAT firewall * rewritten in Rust ## version 20200524 (2020-05-27) _available on request; v1 is now deprecated; please upgrade to v2_ ## version 20200516 (2020-05-20) _available on request; v1 is now deprecated; please upgrade to v2_ ## version 20191207 (2019-12-10) _available on request; v1 is now deprecated; please upgrade to v2_ ## version 20191108 (2019-11-12) _available on request; v1 is now deprecated; please upgrade to v2_ ## version 20190911 (2019-09-13) * v1 launch --- ## Creating a Bastion for SSH access on AWS On some occasions you may want a [bastion host](https://en.wikipedia.org/wiki/Bastion_host) present in your VPC. For example: - accessing a DiscrimiNAT instance over SSH - accessing a host without a public IP ### Creating a Bastion Host ![EC2 instance for Bastion Host](img/01-vpc-public-subnet.png) Let's create a new instance pretty much as usual, taking care that: - it's in the right VPC (one that has DiscrimiNAT in it) - it's in the _Public_ subnet so that direct public IP is routing is available - _Auto-assign public IP_ is _Enabled_ --- ![EC2 tag for Bastion name](img/02-name-my-bastion.png) Perhaps give it an appropriate name. --- ![New Security Group for Bastion SSH](img/03-ssh-security-group.png) Let's create a new security group while we're at it, for the exclusive purpose of letting you connect to this bastion host. Choosing _My IP_, in most cases, should accurately determine your IP address. **This will enable you to directly connect to this instance, from your public IP to its public IP.** --- ![EC2 Key Pair for Bastion SSH](img/04-ssh-bastion-keypair.png) Associate an SSH Key Pair so we can log in at all. Your bastion host on AWS is now ready! ### Accessing a DiscrimiNAT instance through the Bastion :::caution SSH Key If DiscrimiNAT was deployed via the provided CloudFormation templates or Terraform modules, an SSH Key Pair should've been set at that stage; otherwise, you won't be able to authenticate into its instances. ::: The DiscrimiNAT image is hardened per _The CIS Ubuntu Linux 24.04 LTS Benchmark Level 2 - Server_. Therefore, SSH access to it requires your posture to be sound and secure. Let's check with a few commands on your machine: 1. The SSH Agent should have only one identity loaded. This is to prevent it from trying one identity after another to the server, causing the server to block the user after too many failures. So it's safer to just have the one identity that will work. `ssh-add -L` If the output shows more than one line, you may clear all of them out with the command `ssh-add -D`. If the previous command still shows some lines, check the `.ssh` directory in your home directory for unexpected private key files. 2. To add a specific private key to the SSH Agent, run the command: `ssh-add /path/to/private-key` And then check with `ssh-add -L` whether only one line in the output is present. 3. The username to use for login will depend on the Linux image chosen for this function. :::tip Contact our [DevSecOps Support](/support) for help with the usernames. ::: ![EC2 Bastion Public IP](img/05-public-ip-bastion.png) ![EC2 DiscrimiNAT Private IP](img/06-private-ip-discriminat.png) 4. Finally, SSH into the DiscrimiNAT instance using the bastion host as _ProxyJump_. This is needed because DiscrimiNAT will only allow SSH connections from private IPs (i.e. within the VPC), so you cannot connect to it from a public IP. You will need the public IP address of the bastion, the private IP address of the target DiscrimiNAT instance, and this command: `ssh -J @ @` An example of a fully formed command from the example deployment in the screenshots is: `ssh -J @3.122.51.213 @10.10.6.90` If you manage to lock yourselves out of a DiscrimiNAT instance due to repeated authentication failures, either terminate the instance and let the AutoScaling Group bring back a new one, or wait 15 minutes. ### Accessing a host without a public IP through the Bastion This could be much simpler if your target instances are not fussy about where you connect from and how many keys you present during authentication. However, since the targets do not have a public IP address at all in this case, you still need to use a bastion host, so there is connectivity to and from them. The commands are the same as above: 1. Verify the right keys are loaded in your SSH Agent: `ssh-add -L` 2. Connect using the bastion host as _ProxyJump_: `ssh -J @ @` --- ## Troubleshooting ### Video for End Users If you are an application developer, a service owner or an end-user of DiscrimiNAT tasked with getting new domain names working, this \<5m video is for you and covers the most common issue along with solutions. ### Search for _warning_ messages from DiscrimiNAT The firewall may have tried to warn you about an issue it can foresee. Search for `{$.outcome = "warning"}` in CloudWatch -> Logs -> Log Management -> DiscrimiNAT -> **config** log stream. You may find a descriptive clue in the resultant log lines from the last few minutes/hours. ### Elastic IPs not attached to DiscrimiNAT VMs If a Public IP is not found attached to a DiscrimiNAT instance, it will look for any allocated but unassociated Elastic IPs that have a tag-key named `discriminat` (set to any value.) One of such Elastic IPs will be attempted to be associated with itself then. An EC2 VPC Endpoint is needed in DiscrimiNAT's subnets for this mechanism to work though – since making the association needs access to the EC2 API. See the [Terraform](https://github.com/ChaserSystems/terraform-aws-discriminat-gwlb#deployment-examples) or [CloudFormation](https://github.com/ChaserSystems/cloudformation-aws-discriminat-gwlb/blob/main/2az_demo.json#L814-L845) examples. The IAM permissions needed to do this are already a part of our Terraform modules and CloudFormation templates. ### Logs not appearing in CloudWatch It could be one of these three potential issues: 1. DiscrimiNAT VMs do not have an Elastic IP associated. 1. There is a _logs_ VPC endpoint in the VPC but not in the subnets where DiscrimiNAT is deployed. 1. From v2.5.0 onwards of DiscrimiNAT, the CloudWatch log group name changed from `discrimiNAT` to `DiscrimiNAT`. ~~While we're working on getting the permissions updated on the AWS Marketplace deployment templates,~~ please ensure the IAM Policy for DiscrimiNAT has the right capitalisation for `DiscrimiNAT` log group. ### Clients' network traffic sometimes allowed, sometimes not. #### Resolution Please upgrade the instance size from `t3.small` to `c6a.large` or `c6i.large`. This is done via the `instance_size` Terraform variable. The `t3.small` instance size only suffices for light loads and a few clients. The `c6` series offers the best CPU to egress bandwidth and price ratios for the kind of work involved. Use of other instance size classes, such as `m` and `r`, is not recommended because the DiscrimiNAT is not a memory-intensive application, therefore making machines with more memory than needed not a cost-optimal choice. ### _see-thru_ mode not working If the _config_ logs do not show a log line picking up a _see-thru_ rule, you may have a problem with the annotation's implied syntax. #### Resolution The _see-thru_ mode requires a Security Group Outbound Rule (or the _see\_thru_ [default preference](/docs/discriminat/aws/default-prefs/#see_thru)) to have a valid calendar date. For example, `discriminat:see-thru:2022-02-29` is NOT a valid date (because 2022 was not a leap year) but `discriminat:see-thru:2022-02-28` is. --- ## YouTube Playlist --- ## Cloud Composer v2 GKE Clusters ## Concepts >Cloud Composer is a fully managed workflow orchestration service, enabling you to create, schedule, monitor, and manage workflows that span across clouds and on-premises data centers. >Cloud Composer is built on the popular Apache Airflow open source project and operates using the Python programming language. Source: https://docs.cloud.google.com/composer/docs/composer-2/composer-overview **From v2.4.2** onwards, the DiscrimiNAT supports filtering traffic egressing from an Autopilot mode VPC-native Google Kubernetes Engine cluster, when Network Tags are applied, in a Composer v2 [Private IP architecture](https://docs.cloud.google.com/composer/docs/composer-2/environment-architecture#private-ip-psc) environment. ## Network tags A Composer v2 environment's cluster is an Autopilot mode VPC-native Google Kubernetes Engine cluster. Network Tags to it, however, can be applied at the time of the Composer environment's creation. ![GCP Marketplace Launch](img/composer-v2-gke-autopilot-network-tags.gif) :::caution When Network Tags are applied to a Composer v2 environment, DiscrimiNAT will apply corresponding Firewall Rules' FQDN allowlists to the **entire primary subnetwork**. With SNAT enabled in such environments (default), after translation from the Pods, any Internet-bound traffic egresses from the managed Nodes in the primary subnetwork. This means if you were running any other compute in the same subnetwork, the FQDN allowlist for the Composer v2 environment would apply to it as well. The [config logs](/docs/discriminat/gcp/logs-ref#config-logs) show the association of such rules to such a compute workload (a VM, for example) too. Therefore, it is highly recommended to use dedicated primary subnetworks for Composer v2 environments. ::: ## Role Add the following permissions to DiscrimiNAT's service account's role to allow it to apply corresponding Firewall Rules to the clusters' subnets. ``` container.clusters.list compute.subnetworks.get ``` :::tip Also see the [Service Account](../service-account/) page for other permissions of interest. ::: --- ## Configuration Reference(Gcp) With DiscrimiNAT, the Firewall Rules attached to the applications that need filtered access to the Internet can be annotated with the set of allowed FQDNs. This allows creating **least privilege egress policies** from the protected workload to the Internet. No need to create VPC-wide allowlists that allow discrete applications access to what they needn't. :::info DiscrimiNAT's own Firewall Rules need not be altered. ::: On this page, we cover the syntax for the Cloud-native Firewall Rules' **description** fields, where the annotations live. ### See-Thru mode The `see-thru` mode is a non-blocking configuration that allows one to [build an appropriate allowlist 🎥](/docs/discriminat/gcp/logs-ref/#building-an-allowlist-from-scratch-video-version) up to a specified date. The annotation is added to the description fields of **egress rules of protected applications to-be**. #### Syntax `discriminat:see-thru:YYYY-mm-dd` :::caution The rule will automatically stop working **after** the `YYYY-mm-dd` date. ::: #### Examples `discriminat:see-thru:2026-12-31` `discriminat:see-thru:2026-09-09` Configuring this mode in the **GCP web console** would look like: ![](img/gcp-firewall-rule-see-thru-example-discriminat.png) In **Terraform**, this would be along the lines of: ``` resource "google_compute_firewall" "some-app" { name = "monitor-some-app" network = var.network direction = "EGRESS" destination_ranges = ["0.0.0.0/0"] allow { protocol = "all" } target_tags = ["new-app"] description = "discriminat:see-thru:2026-12-31" } ``` #### Logs The **flow** logs at `logName="projects//logs/discriminat-flow"` in Logs Explorer will have three additional fields for traffic from any workload that has a Firewall Rule with a `see-thru` annotation in it: 1. `see_thru_exerted`: **true** when no `tls` or `ssh` protocol rules allowed this connection through and it's only because of the `see-thru` rule that the connection has been let through. **false** when an already present `tls` or `ssh` protocol rule would otherwise allow this connection through. The rest of the fields in the log line remain as they would without the `see-thru` rule, hence indicating which `tls` or `ssh` protocol rule matched to let this connection through. 1. `see_thru_gid`: The Firewall Rule where the enacted `see-thru` was found. 1. `see_thru_days_remaining`: The number of remaining days this rule will continue to work. Runs into negative if the current date goes past the specified date. These log fields are still emitted if a `see-thru` rule with any date, past or future, remains specified. ### TLS and SSH mode The `tls` and `ssh` modes are a blocking configuration that will only allow FQDNs set out in the allowlist. Destination protocols, [FQDNs](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) and IP Addresses are added to the description fields of **egress rules of the protected applications**. :::caution If transitioning from a `see-thru` rule, note that `tls` and `ssh` rules **require a specific port number** to be specified in the Firewall Rules. ::: #### Syntax `discriminat::` ([Secret Path variant](#secret-path-variant)) `discriminat::[,,]` ([Inline variant](#inline-variant)) - Valid protocols are `ssh` and `tls` only. - There can be other text leading and trailing this specification, separated by whitespace. - All of it is case insensitive, except the Secret Path if specified. - [Wildcards](/docs/discriminat/gcp/default-prefs/#wildcard_exposure) are only supported for the TLS protocol. :::caution **Please familiarise yourself with [operation and caveats](/blog/wildcards-and-system-2-thinking/#operation) on the behaviour of wildcard rules before using them.** ::: #### Secret Path variant - The \ is the full path of a Secret. It can be in another GCP Project as well, as long as DiscrimiNAT's Service Account has _Secret Manager Secret Accessor_ Role to read it. - If using our Terraform modules, see the [Terraform section](/docs/discriminat/gcp/service-account/#terraform) on the Service Account page for example code. - The JSON format for list of FQDNs and IP Addresses in the payload of a Secret is: ```json { "addrs": [ "fqdn1.com", "*.github.com", "203.0.113.9", ... ] } ``` ##### Examples If the above payload were to be stored in a Secret called `service-foo-allowed-fqdns`, its full path could be `projects/000000000000/secrets/service-foo-allowed-fqdns`. Therefore, what goes in description field will be: `discriminat:tls:projects/000000000000/secrets/service-foo-allowed-fqdns` Configuring this variant in the **GCP web console** would look like: ![](img/gcp-firewall-rule-secret-path-example.png) #### Common constraints to both variants - `Direction of traffic` must be `Egress`. - `Targets` must be set to `Specified target tags`. **And then specify the same Network Tags as on the virtual machines that you'd like this firewall rule to be applied to.** Precisely like you would anyway. - Under `Protocols and ports`, only choose `tcp` and **specify exactly one port number**. To use another port for an alternative set of addresses, simply create another Firewall Rule with that port number. See the [Connecting to MS/Azure SQL Server over TLS](/docs/how-to/tls-upgrade/sql-server/#on-gcp) guide for an example. - For your typical **HTTPS** outbound connection, the port number would be `443`. - A typical **SFTP** connection has the port number of `22`. - **Kafka** typically, as another example, is on port `9092`. - `Destination filter` must be set to `IP range` and the range set to the narrowest you can think of. Now if you don't have anything narrower than `0.0.0.0/0` that is absolutely fine! The firewall takes care of validating the destination IP address of actual packets in many ways, but the Firewall Rule at the Google Cloud networking and security level must allow the packet to leave the virtual machine at all. **`0.0.0.0/0` is a safe choice with the DiscrimiNAT Firewall in the way.** - Multiple Firewall Rules can be associated with the virtual machines with the use of Network Tags, and they will all be recognised. #### Inline variant - There can be as many FQDNs and IP Addresses as you can fit separated by commas. There cannot be whitespace, though. - Multiple Firewall Rules can be created to accommodate more addresses, or group them by some criteria. ##### Examples - `DiscrimiNAT:TLS:api.foo.example.com,downloads.example.net,203.0.113.9` - `DiscrimiNAT:SSH:sftp.txs.example.org,ssh.github.com` - `discriminat:tls:api-v?.example.com discriminat:tls:www.example.org` - `lorem ipsum discriminat:ssh:ssh.github.com,gitlab.com,203.0.113.9 dolor sit amet` Configuring this variant in the **GCP web console** would look like: ![](img/gcp-firewall-rule-tls-ssh-example-discriminat.png) In **Terraform**, this would be along the lines of: ``` resource "google_compute_firewall" "acme" { name = "acme-reports-upload" network = var.network direction = "EGRESS" destination_ranges = ["0.0.0.0/0"] allow { protocol = "tcp" ports = ["443"] } target_tags = ["acme"] description = "discriminat:tls:api.acme.com,auth.acme.com,203.0.113.9,foo.example.com,bar.example.net" } ``` ### Complete Bypass If you would like to discuss alternate protocols or know how to completely bypass the DiscrimiNAT whilst still routing traffic, get in touch with our [DevSecOps Support](/support/) to discuss the best option for your use case. --- ## Default Preferences(Gcp) From v2.9.0 of DiscrimiNAT, certain behavioural preferences can be set at global level per deployment. These are defined in JSON format in a Secret named `DiscrimiNAT_` (case-sensitive) in Secret Manager and the default value if not defined or not accessible is: ```json { "%default": { "wildcard_exposure": "prohibit_public_suffix", "flow_log_verbosity": "full", "see_thru": null, "x509_crls": "ignore" } } ``` :::tip Preferences you wish to leave at defaults can be excluded. Also, any additional JSON keys will simply be ignored. ::: :::caution Preferences are refreshed **once every five minutes** from the Secret. ::: ## Preferences ### wildcard_exposure Generally, `?` represents one character match exactly which isn't a `.`, and `*` matches any number of characters except `.`. #### Options `none`: no wildcard characters such as `?` or `*` are allowed at all `prohibit_asterisk`: `?` is allowed; `*` is not allowed `prohibit_public_suffix`: `?` is allowed; `*` is allowed as long as the possibility of resulting domain names are not publicly registrable (aka Effective TLDs). For example, `*.github.com` is okay but `*.github.io` is not. Another example would be `*.cloudflare.net` being effectively public. This is determined using [Mozilla's Public Suffix List (PSL)](https://publicsuffix.org/) bundled into DiscrimiNAT. The version of the PSL bundled will be in the [Release Notes](/docs/discriminat/gcp/release-notes/) and will be updated with every release. :::tip If a wildcard pattern is not accepted because it could have led to a Public Suffix List match, a _config_ log message like the following will be emitted: ``{addr: "*.github.io", cat: "addr", outcome: "publicsuffix[.]org list matched with `github.io`"}`` ::: `nuclear`: `?` is allowed; `*` is allowed with no public suffix list safeguard #### Default `prohibit_public_suffix` ### flow_log_verbosity This setting only impacts the _flow_ logs. _config_ logs verbosity and behaviour will not be affected. #### Options `full`: all _allowed_ and _disallowed_ flows for both the client requests and the server responses are logged in the _flow_ logs `only_disallowed`: only _disallowed_ flows will be logged `none`: no logs at all #### Default `full` ### see_thru The `see_thru` non-blocking, monitoring mode can be enabled for an entire deployment rather than per Firewall Rule from this level. This is only the default and does not override a _see-thru_ mode defined at a more granular level – even if the one the granular level may have expired. :::tip See [see-thru](/docs/discriminat/gcp/config-ref/#see-thru-mode) mode docs for understanding this non-blocking, monitoring mode in detail. ::: #### Options `yyyy-mm-dd` formatted date `null` to not enable _see_thru_ at the default level. `null` value to be specified without quotes or this top-level key can be left out completely. #### Default `null` ### x509_crls #### Options `auto_allow`: Automatically allow plaintext HTTP CRL Endpoints of x509 SSL certificates for all TLS FQDNs allowlisted. `ignore`: Do not allow CRL Endpoints automatically. #### Default `ignore` ## Examples It may be useful to turn on the non-blocking, monitoring `see_thru` mode on until a specified date on a new deployment of DiscrimiNAT. This would ensure apps continue to work while data is captured for building an allowlist from eventually: `{"%default":{"see_thru":"2026-11-19"}}` :::tip Also see [building an allowlist from scratch video recipe](/docs/discriminat/gcp/logs-ref/#building-an-allowlist-from-scratch-video-version). ::: Ultimately, when allowlists are built and enforced, you may want to turn off _flow_ logs for _allowed_ connections, saving on logging costs: `{"%default":{"flow_log_verbosity":"only_disallowed"}}` To also enable automatic allowing of X509 CRL Endpoints: `{"%default":{"flow_log_verbosity":"only_disallowed","x509_crls":"auto_allow"}}` ## System System-level preferences are passed directly to VM instance creation, are stored as files on the operating system disk, and are only read on start. These are placed within the instance because they represent critical customer choices that must be respected whether external APIs are accessible or not and the right IAM permissions on the Service Account are assigned or not. ### BYOL A licence key supplied by Chaser Support if using a BYOL version. The value can be passed to the `byol` variable in our [Terraform modules](https://registry.terraform.io/namespaces/ChaserSystems). The contents are Base64 decoded and written to the path `/etc/chaser/licence-key.der`. ### ASHR Automated System Health Reporting. 10 minutes after boot and then at around 0200 UTC every day, each instance of DiscrimiNAT collects its OS internals & system logs since instance creation, config changes & traffic flow information from last two hours and uploads it to a Chaser-owned cloud bucket. This information is encrypted at rest with a certain public key so only relevant individuals with access to the corresponding private key can decrypt it. The transfer is encrypted over TLS. Access to this information is immensely useful to create a faster and more reliable DiscrimiNAT as we add new features. We also get to learn about how users interact with the product in order to further improve the usability of it as they embark on a very ambitious journey of fully accounted for and effective egress controls. We understand if certain environments within your deployment would rather not have this turned on. To disable it, a file at the path `/etc/chaser/disable_automated-system-health-reporting` should exist. This can be achieved by setting the `ashr` variable to `false` in our [Terraform modules](https://registry.terraform.io/namespaces/ChaserSystems). --- ## DIY & Terraform :::info Ensure you've visited our [marketplace page](https://console.cloud.google.com/marketplace/details/chasersystems-public/discriminat) and accepted the terms & conditions, and any custom offers you may have been extended for your GCP account, first. ::: ## IMAGE IDENTIFIERS | key | value | | ------- | -------------------- | | family | discriminat | | project | chasersystems-public | ## DEPLOYMENT ESSENTIALS For effective functioning, DiscrimiNAT will need: 1. A machine type with at least 2 vCPU and 2 GiB RAM. An `e2-small` should suffice where throughput requirements are basic and allowlists small. Otherwise an `n2-highcpu-2` makes a good choice for constant throughput. [Talk to our DevSecOps](/support/) to get the sizing right. 1. A service account that can read/write compute metadata, and write to logging and monitoring: ```plaintext serviceAccounts: - email: -compute@developer.gserviceaccount.com scopes: - https://www.googleapis.com/auth/compute - https://www.googleapis.com/auth/logging.write - https://www.googleapis.com/auth/monitoring.write ``` 1. Ability to forward IP packets with the `canIpForward` property turned on. 1. A public IP. 1. SSD disk type is recommended. When deploying the instance(s), you may configure high-availability through a Managed Instance Group (see reference implementation [here](https://github.com/ChaserSystems/terraform-google-discriminat-ilb/blob/main/discriminat.tf#L300)), routing and tagging as per your desired architecture. For example, the route to the Internet will be for destination `0.0.0.0/0` for instances without a public IP and should pass through the DiscrimiNAT instance – whether by tagging or by internal load-balancing. **For monitoring the logs and configuring the FQDN-based firewall egress rules, see the [logs](/docs/discriminat/gcp/logs-ref/) and [config](/docs/discriminat/gcp/config-ref/) references.** ## TERRAFORM MODULES Before you dive into the DIY code that follows, you may want to consider our fully-working [modules at the Terraform Registry](https://registry.terraform.io/namespaces/ChaserSystems), which include **preconfigured high-availability** and further examples. In fact, one of the examples extends from the canonical [terraform-google-modules/network/google](https://registry.terraform.io/modules/terraform-google-modules/network/google/) module at the registry. You will find [two modules](https://registry.terraform.io/namespaces/ChaserSystems) relevant to Google Cloud at the registry, which can be briefly described as: ### discriminat-ilb Architecture with internal TCP load balancers as next hops set as the default route, and tag based opt-out control. ### discriminat-ntag Architecture with Network Tags in VPCs for fine-grained, opt-in control over routing. :::info Drop us an email at `devsecops@chasersystems.com` to receive quarterly version update release notes one week prior to GA. Also for a demo, best practices and architecture review. ::: ## gcloud CLI EXAMPLE #### Lookup ```plaintext gcloud compute images describe-from-family \ --format="value(selfLink)" \ --project chasersystems-public \ discriminat ``` --- ## Installation Overview(Gcp) ### [DIY & TERRAFORM âžœ](/docs/discriminat/gcp/diy/) If you require complex or custom routing, and have Infrastructure-as-Code for your deployments already, this is where you will find key information. ### [QUICK START âžœ](/docs/discriminat/gcp/quick-start/) For a 5-minute deployment, assuming secure defaults, from within the marketplace console itself. This will totally lock down your VPC from the get-go unless monitoring mode or appropriate FQDN-based egress firewall rules exist for outbound connections, and VMs needing to bypass DiscrimiNAT are network-tagged with `bypass-discriminat`. ### [CONFIG REFERENCE âžœ](/docs/discriminat/gcp/config-ref/) The various ways GCP-native Firewall Rules can be annotated with appropriate FQDN allowlists and traffic monitoring rules after deployment of DiscrimiNAT. ### [DEFAULT PREFERENCES âžœ](/docs/discriminat/gcp/default-prefs/) Global level preferences for controlling logs verbosity, wildcards exposure and monitoring mode. ### [LOGS REFERENCE âžœ](/docs/discriminat/gcp/logs-ref/) Fields, filters and recipes to find what you need from DiscrimiNAT's config and flow logs in GCP Logs Explorer. ### ARCHITECTURE Pertaining to the Quick Start only. If you go down the DIY route, the architecture will be as per your design. A DiscrimiNAT Firewall becomes the NAT solution for your VPC, making the use or presence of other NAT gateways redundant. It is architected along the [_Internal TCP/UDP load balancers as next hops_](https://cloud.google.com/load-balancing/docs/internal/ilb-next-hop-overview) guide from Google Cloud. As the guide suggests, these NAT instances are simply a [_bump-in-the-wire_](https://en.wikipedia.org/wiki/Bump-in-the-wire) with: - no explicit client configuration - load balancing - proactive health-checks ### MARKETPLACE [![Launch Free Trial on Google Cloud](img/GC-Partner-no_outline-H_small.png)](https://console.cloud.google.com/marketplace/details/chasersystems-public/discriminat) The DiscrimiNAT Firewall is [listed on the Google Cloud Marketplace](https://console.cloud.google.com/marketplace/details/chasersystems-public/discriminat). The architecture discussed above is available as a Deployment Manager template to launch on subsequent steps from marketplace subscription. --- ## Logs Reference(Gcp) ## Fields ### flow logs Flow Logs can be found in Logs Explorer under Logging, with the filter `logName="projects//logs/discriminat-flow"`. A typical flow log looks like: ![](img/typical-stackdriver-flow-log-from-discriminat.png) Here's a summary of the possible fields: `dhost`: destination hostname/FQDN or IP Address `cat`: packet origin – client or server `outcome`: allowed or disallowed `src`: source IP address `spt`: source port `proto`: tls, ssh, http or unknown protocol `proto_v`: version of the identified protocol `dst`: destination IP address `dpt`: destination port `reason`: reason for the outcome – such as in which Firewall Rule was a matching protocol rule found, cache not ready yet for a newly discovered allowlist entry, or which protocol anomaly led to the connection being disallowed `instance`: the VM instance name of the DiscrimiNAT instance this log line was generated from `pqc`: indicates if a TLS handshake used a known [PQC TLS Supported Group](https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-8). The value may be true or false in case of TLS; or absent in the case of SSH. If X.509 CRLs are [set to be allowed](/docs/discriminat/gcp/default-prefs/#x509_crls), or if there is some plaintext HTTP egress traffic with requisite outbound ports open, there may be HTTP flow logs with the following fields. Note that HTTP is generally not allowed unless X.509 CRLs were allowed and the requested URL was of a CRL Endpoint. `http_code`, `http_method`, `http_path`, `http_user_agent`: these fields can be useful to understand which application the plaintext HTTP traffic is originating from And if a Firewall Rule with a `see-thru` rule in it is found associated to a client application, the following fields will be also present: `see_thru_days_remaining`: number of days remaining while the `see-thru` rule will be non-blocking, as defined in the `see-thru` rule itself `see_thru_gid`: the Firewall Rule name where the `see-thru` rule was found, or _%default_ if set via [preferences](/docs/discriminat/gcp/default-prefs/#see_thru) `see_thru_exerted`: when _true_, the `see-thru` rule had to let this connection through and there was no protocol rule that would have allowed it; when _false_, there was a protocol rule that would have let the connection through anyway and its presence will be identified in the `reason` field; or the `see-thru` rule has expired Watch this **4½ minute video** to understand these three fields, and their interplay with the `outcome` field, when viewed in the Cloud web console. Config reference for the `see-thru` rules can be found [here](/docs/discriminat/gcp/config-ref/#see-thru-mode). ### config logs Config Logs can be found in Logs Explorer under Logging, with the filter `logName="projects//logs/discriminat-config"`. They form an audit trail of changes made to rules and Firewall Rule associations as clients come and go. Here's a summary of the possible fields: `cat`: fqdn (addr), wildcard pattern (wildcard-addr), client, see-thru or self change category `outcome`: accepted something new in the internal configuration, removed something because it had been removed from external configuration sources, or rejected something (because of reason), or warning about a syntactically correct configuration input but one which wasn't going to work as expected (because of reason) `client`: IP address of an affected client (mutually exclusive with `subnet`) `subnet`: block of IP addresses in the `a.b.c.d/e` notation of an affected subnet (mutually exclusive with `client`) `gid`: the Firewall Rule name that relates to this change `addr`: the FQDN, wildcard pattern or IP Address found in a Firewall Rule rule `proto`: tls or ssh protocol as identified in the rule `reason`: reason for usually a warning or a rejected outcome `instance`: the VM instance name of the DiscrimiNAT instance this log line was generated from If X.509 CRLs are [set to be allowed](/docs/discriminat/gcp/default-prefs/#x509_crls) there will be an additional field: `http_path`: the document path over HTTP for CRL data And if a Firewall Rule with a `see-thru` rule in it is found attached to a client application, there will be an additional field: `thru_date`: the date until which the non-blocking `see-thru` mode should remain effective; past this date such a rule would lose effect ## Filters The log lines are JSON-structured so each field can be addressed specifically under the `jsonPayload` object. More of Logs Explorer syntax can be explored on its [Logging query language page](https://docs.cloud.google.com/logging/docs/view/logging-query-language). #### _disallowed_ connections ``` logName="projects//logs/discriminat-flow" jsonPayload.outcome="disallowed" ``` #### _allowed_ but protocol is not TLS ``` logName="projects//logs/discriminat-flow" jsonPayload.outcome="allowed" -jsonPayload.proto="tls" ``` #### connections from a specific client where the destination host is not api.github.com ``` logName="projects//logs/discriminat-flow" jsonPayload.cat="client" jsonPayload.src="192.168.101.3" -jsonPayload.dhost="api.github.com" ``` #### firewall rules where the see-thru mode exception has only 2 or less days left ``` logName="projects//logs/discriminat-flow" jsonPayload.see_thru_days_remaining<=2 ``` #### connections where the see-thru mode had to be used, but not for perhaps telemetry data ``` logName="projects//logs/discriminat-flow" jsonPayload.see_thru_exerted="true" -jsonPayload.dhost=~".*\.ubuntu\.com" ``` ## Recipes ### building an allowlist from scratch (video version) Watch this **3½ minute video** to build the allowlists super quick! The two commands used in the video are also documented here, underneath the video. Command 1: ``` gcloud logging read --freshness=1h --format='json(jsonPayload.dhost,jsonPayload.src,jsonPayload.see_thru_gid)' 'logName=projects/'"$GOOGLE_CLOUD_PROJECT"'/logs/discriminat-flow AND jsonPayload.dhost:* AND jsonPayload.see_thru_exerted=true AND jsonPayload.see_thru_gid="'$RULE_NAME'"' | dd of=frequency-fqdns.json ``` Command 2: ``` cat frequency-fqdns.json | jq --raw-output 'group_by(.jsonPayload.dhost) | map(.[0] + {"count": length}) | sort_by(.count) | reverse | [.[].jsonPayload.dhost] | @csv' | tr --delete '"' ``` ### building an allowlist from scratch (text version) :::info This method is now deprecated in favour of the Cloud Shell based method above. ::: 1. Create a new Firewall Rule, with a `see-thru` rule in it. See the config reference and an example [here](/docs/discriminat/gcp/config-ref/#see-thru-mode). Give it a thru date sufficient enough to capture all stages of the application's lifecycle, such as deployment, restart, the occassional uploading of reports, telemetry, monitoring, etc. Note the Firewall Rule name. 2. Attach this Firewall Rule to the application. In GCP this would be adding the target network tags in the Firewall Rule to the network tags of the workload/VMs. 3. In the Logs Explorer under Logging, enter the following query and run it. _Note: You'll have to replace the Firewall Rule name in this example with the one that was created above._ ``` logName="projects//logs/discriminat-flow" jsonPayload.see_thru_exerted="true" jsonPayload.see_thru_gid="firewall-rule-replace-me-name" ``` - Now create a metric out of this query from the Actions menu. ![](img/stackdriver-create-metric-from-logs-query.png) - Add these three labels to the metric, and save it. | Name | Type | Field Name | | ----- | ------ | ----------------- | | dhost | STRING | jsonPayload.dhost | | proto | STRING | jsonPayload.proto | | dpt | STRING | jsonPayload.dpt | ![](img/see-thru-stackdriver-metric-labels.png) 4. Now let the application follow its normal course of lifecycle. This could last a few hours, days or weeks depending on the application. 5. When you feel some useful traffic may have been logged, visit the Metrics Explorer under Monitoring, and select the metric you saved earlier. Choose an appropriate timespan, and group the results by the `dpt`, `proto` and `dhost` fields. ![](img/gcp-flow-log-metrics-see-thru-group-by.png) This will produce a table of results like: ![](img/gcp-flow-log-metrics-see-thru-results-table.png) 6. From the information in the results table, specific allowlist protocol rules can be created. For example: `discriminat:tls:entropy.ubuntu.com,motd.ubuntu.com` and `discriminat:tls:logging.googleapis.com` with protocol TCP and destination port 443. Although you may want to reconsider allowing any telemetry endpoints. For the full reference on creating these protocol rules along with an example, see the [config reference](/docs/discriminat/gcp/config-ref/#tls-and-ssh-mode). 7. Once these protocol rules are attached to the application, run the full application lifecycle again, and then the query at #3 to ensure that `see_thru_exerted` did not have to be used to let any disallowed traffic through. There may be the case where you choose to disallow certain destinations anyway – those are to be expected and can be selectively filtered out from view. 8. Detach or remove the `see-thru` rule from the application and give it another full lifecycle run just to ensure it works smoothly. 9. The metric created at #3 can now be deleted. --- ## Quick Start If you are planning to deploy a new VPC network secured by egress filtering, think about the IP address ranges you would like to deploy. A good place to start is the [Virtual Private Cloud (VPC) network overview page over at Google Cloud documentation](https://docs.cloud.google.com/vpc/docs/vpc). Whether something wholly new or the _default_ network, just follow the visual guide below. :::info Contact us for expert help at `devsecops@chasersystems.com` at any stage of your journey – we'll jump on a screen-sharing call right away! ::: --- ### i. MARKETPLACE LAUNCH ![GCP Marketplace Launch](img/01-launch.png) Proceed to our [Google Cloud Platform Marketplace](https://console.cloud.google.com/marketplace/details/chasersystems-public/discriminat) page and click the **Launch** button. This step is entirely self-service. --- :::info If you have [deployed through Terraform](/docs/discriminat/gcp/diy/#terraform-modules) or are considering, jump straight to the [Key Information](#iv-key-information) section. You may skip everything before that. ::: --- ### ii. DEPLOYMENT PARAMETERS ![DiscrimiNAT Launch Parameters](img/02-deploy.png) At the deployment configuration page, the defaults should suffice for a non-Production environment. The parameters are explained below for help with multiple deployments and considerations for Production environments. **Deployment name:** This is a unique identifier for the deployment. In the case of this example, we have chosen the name _plugleak_. **Number of Instances:** This many number of instances will spread evenly over all Zones in the selected Region. A number of 1 should suffice for non-critical environments such as those for Test & Development purposes. For Production, a number of at least 2 will provide rapid High Availability. A number equal to the number of Zones in the selected Region will provide a nice, even spread of the instances in all Zones and sufficient headroom for baseline Production throughput in case of an incident. **Machine Type:** A machine type with at least 2 vCPU and 2 GiB RAM. An `e2-small` should suffice where throughput requirements are basic and allowlists small. Otherwise an `n2-highcpu-2` makes a good choice for constant throughput. [Talk to our DevSecOps](/support/) to get the sizing right! **Zone:** The Zone is only used to infer the Region for this deployment. **Network:** This is the VPC Network for this deployment. The VM Instances that you wish to protect (filter the egress traffic of) would have to be a part of this VPC Network. **Subnetwork:** This is the Subnetwork for this deployment. The VM Instances that you wish to protect (filter the egress traffic of) would have to be a part of this Subnetwork. Just hit **Deploy** and the firewall instance(s) will be ready in a few minutes! --- ### iii. SMALL WAIT ![GCP Marketplace Launch Wait](img/03-deploying.png) Shouldn't be more than a few minutes. --- ### iv. KEY INFORMATION ![GCP Marketplace DiscrimiNAT Launched](img/04-deployed.png) You're all set! Let's review some key information that is now visible: **Manage Firewall Rules**: Brief example of what could be specified in the firewall rules now this firewall is deployed. More on that in the next section. **Tag for bypassing firewall:** This tag, `bypass-discriminat`, is what can be added to the network tags of any virtual machines if you would like them to simply side-step this firewall. **Watch the Configuration Logs:** This string is useful to directly filter for logs, in StackDriver, that reveal any changes to the firewall configuration. It is always constructed as `logName="projects//logs/discriminat-config"` . **Watch the Flow Logs:** This string is useful to directly filter for logs that reveal traffic metadata for all accepted and rejected connections through the firewall. It is always constructed as `logName="projects//logs/discriminat-flow"` . --- ### v. FIREWALL RULES :::tip Don't know which FQDNs to allow? Watch our 3½ minute [video](/docs/discriminat/gcp/logs-ref/#recipes) on how to build an allowlist super quick with DiscrimiNAT's monitor mode! ::: ![GCP Create Firewall Rule with FQDN](img/05-create-firewall-rule.png) :::info Contact us for expert help at `devsecops@chasersystems.com` at any stage of your journey – we'll jump on a screen-sharing call right away! ::: We encourage you to create firewall rules as you would normally. Tight in scope and tagged in a granular fashion so they attach with the intended applications' instances only. FQDN based egress filtering enhancements for GCP follow! For an effective use of the DiscrimiNAT firewall: - Add destination protocols and [FQDNs](https://en.wikipedia.org/wiki/Fully_qualified_domain_name) to the description field. The format of this annotation is `DiscrimiNAT::[,]` . - Valid protocols are `ssh` and `tls` only. - There can be as many FQDNs as you can fit separated by commas. There cannot be whitespace, though. - There can be other text leading and trailing this specification, separated by whitespace or newlines. - All of it is case insensitive. - Some examples of what can go in the description field: - `DiscrimiNAT:TLS:api.foo.example.com,downloads.example.net` - `DiscrimiNAT:SSH:sftp.txs.example.org,ssh.github.com` - `discriminat:tls:api-v2.example.com discriminat:tls:www.example.org` - `lorem ipsum discriminat:ssh:ssh.github.com,gitlab.com dolor sit amet` - lorem ipsum discriminat:ssh:ssh.github.com discriminat:ssh:gitlab.com dolor sit amet - `Direction of traffic` must be `Egress`. - `Targets` must be set to `Specified target tags`. And then specify the same network tags as on the virtual machines that you'd like this firewall rule to be applied to. Precisely like you would anyway. - `Destination filter` must be set to `IP range` and the range set to the narrowest you can think of. Now if you don't have anything narrower than `0.0.0.0/0` that is absolutely fine! The firewall takes care of validating the destination IP address of actual packets in many ways, but the firewall rule at the Google Cloud networking and security level must allow the packet to leave the virtual machine at all. **Therefore, `0.0.0.0/0` is a safe choice with the DiscrimiNAT firewall in the way.** - Under `Protocols and ports`, only choose `tcp` and **specify exactly one port number**. Ranges, multiple ports, other protocols, and so on are **not** supported. - For your typical **HTTPS** outbound connection, the port number would be `443`. - A typical **SFTP** connection has the port number of `22`. - That's it! Although you may want to check the [full reference](/docs/discriminat/gcp/config-ref/) for more possibilities. --- ### vi. CONFIGURATION LOG ![StackDriver DiscrimiNAT config log](img/06-observe-config-log.png) Let's now look at the configuration logs. Browse to **Logging -> Logs Explorer** and paste the log filter for watching the configuration logs from the deployments page, which is constructed as `logName="projects//logs/discriminat-config"` and hit Run Query. You will see each instance of the firewall pick up the changes! Logs are structured (JSON) so filtering them in any way you like should be a walk in the park! :::tip Can't see any DiscrimiNAT logs? Have a look at our [Troubleshooting](/docs/discriminat/gcp/troubleshooting/) section for common issues. ::: --- ### vii. TEST A VIRTUAL MACHINE ![GCP New VM for test](img/07-create-test-vm.png) Time to test with a virtual machine. Create one as usual, in the same region as the firewall was deployed in. Customise the `Management, security, disks, networking, sole tenancy` section though. :::info Contact us for expert help at `devsecops@chasersystems.com` at any stage of your journey – we'll jump on a screen-sharing call right away! ::: --- ### viii. PRIVATE IP AND TAGS ![GCP VM Private IP](img/08-vm-network-settings.png) Under the `Networking` tab, ensure that: - `Network tags` match any or some firewall rules with the same target tags, as usual. - `Network` and `Subnetwork` are set to where the firewall was deployed. - `External IP` is set to `None`. These virtual machines do not need a Public IP address at all and will be NAT'ed if firewall rules permit through the DiscrimiNAT firewall. The logs of flow activity from this virtual machine can be viewed in the Log Explorer with a filter that is constructed as `logName="projects//logs/discriminat-flow"`. --- ### ix. Next Steps TroubleshootDiscover FQDNs --- ## Release Notes(Gcp) ## version 2.40 (2026-05-07) * added a Post-Quantum Cryptography `pqc` key exchange field in flows logs that indicates if a TLS handshake used a known [PQC TLS Supported Group](https://www.iana.org/assignments/tls-parameters/tls-parameters.xhtml#tls-parameters-8). The value may be _true_ or _false_ in case of TLS; or sometimes absent in the case of SSH. This feature, although in preview, can be used to assess PQC readiness in your outbound connections. * TLS SNI matching against user-defined rules is now case-insensitive to support client-side libraries that did not lowercase user input before initiating a new outbound connection * broadened support for some legacy SSH ciphers * lowered disk usage and telemetry data collection volume (when left enabled) * a mitigation for [CVE-2026-31431](https://ubuntu.com/security/CVE-2026-31431) (aka [Copy.Fail](https://copy.fail/)) is included with package _kmod_ version _31+20240202-2ubuntu7.2_, [as advised by Canonical](https://ubuntu.com/blog/copy-fail-vulnerability-fixes-available) * version of bundled Public Suffix List: [8b4345f](https://github.com/publicsuffix/list/blob/8b4345f9a2513011b21e6fc7b8a7197a849be52c/public_suffix_list.dat) **Terraform Updates** * full diff between previous and this version can be found [here](https://github.com/ChaserSystems/terraform-google-discriminat-ilb/compare/2.30.0...2.40.0) for the **-ilb** module, and [here](https://github.com/ChaserSystems/terraform-google-discriminat-ntag/compare/2.30.0...2.40.0) for the **-ntag** module ## version 2.30 (2026-03-12) * added support for _Job_ and _WorkerPool_ type of Cloud Run objects, besides _Service_, to be able to use Direct VPC Egress. [More permissions](/docs/discriminat/gcp/service-account/) will need to be granted for these to be discovered by DiscrimiNAT. * a new log message `undiscovered client` in case a workload has not yet been polled and discovered by DiscrimiNAT, separating this circumstance from the `no matching rules found` log message * Post-Quantum Cryptography (PQC) TLS cipher exchange in ClientHello support **in preview** even if the handshake is split over multiple IP packets * _warning_ and _notice_ level log messages can now indicate their severity with the right icon and colour in Logs Explorer * version of bundled Public Suffix List: [7ef6384](https://github.com/publicsuffix/list/blob/7ef6384612e1b48bb8b6023716cc9a493ac25d8a/public_suffix_list.dat) ## version 2.20 (2025-08-27) * wildcard matched FQDNs no longer cause a `cache not ready` flow log event on DiscrimiNAT instance restart or upgrade. This is accomplished by sharing knowledge of such FQDNs between DiscrimiNATs: * by discovering peers which either have label key or label value set to `discriminat` (case-insensitive) * exchanging knowledge continuously over port 1042 UDP with peers * incoming peers learn from outgoing peers during a rolling restart/upgrade * related and necessary changes have been made in all our Terraform (and other IaC) modules * messaging between peers is encrypted and authenticated, and does not contain policy * caveats: * upgrading from any version older than 2.20 will, for once and the last time, cause wildcard matching FQDNs to not work for a few seconds until the cache is warmed up again * from v2.20 onwards, this should not occur as long as one-at-a-time rolling upgrades are made (default behaviour of our IaC modules) * single-instance deployments will continue to lose memory of wildcard matched FQDNs and require a few seconds to allow them again after first encounter * added support for Cloud Composer v3; this needs the `composer.environments.list` permission (see [docs](/docs/discriminat/gcp/service-account/)) * added support for [Direct VPC Egress in Cloud Run](https://docs.cloud.google.com/run/docs/configuring/vpc-direct-vpc); this needs the `run.services.list` permission (see [docs](/docs/discriminat/gcp/service-account/)) * new config log _reason_: `malformed fqdn`. This is emitted in case a syntactically invalid domain name is specified in an allowlist * improvement in how quickly new clients are detected * more resilient handling of upstream Cloud APIs erroring or timing out – for knowledge on new clients * version of bundled Public Suffix List: [44211b0](https://github.com/publicsuffix/list/blob/44211b0fbb8b4e09b89a553f5b26c6a02f889c4b/public_suffix_list.dat) * underlying OS has been changed from Ubuntu 20.04 LTS to Ubuntu 24.04 LTS * version number strategy has changed: * DiscrimiNAT itself will only use Major and Minor in its version numbers, and not use the Patch level from the [semantic versioning scheme](https://semver.org/). For example, this release is 2.20 instead of 2.20.0 * related components to a specific version of DiscrimiNAT will follow the full semantic versioning scheme, where the Major and Minor will correspond to the recommended version of DiscrimiNAT. For example, Terraform modules' version 2.20.0, 2.20.1, 2.20.2 and so on will be best suited for DiscrimiNAT 2.20 **Terraform Updates** * allowed UDP port 1042 to and from DiscrimiNATs' firewall rule for messaging between peers, as mentioned above * added a new variable `bypass_cidrs` which allow you to create routes that should bypass DiscrimiNAT (using higher priority). By default, it contains the [gRPC Direct Connectivity range](https://cloud.google.com/storage/docs/direct-connectivity) * default preferences, which are stored as a secret, now replicate to only the deployment region * added labels to a few resources * full diff between previous version and this can be found [here](https://github.com/ChaserSystems/terraform-google-discriminat-ilb/compare/2.9.0...2.20.0) for the **-ilb** module, and [here](https://github.com/ChaserSystems/terraform-google-discriminat-ntag/compare/2.9.0...2.20.0) for the **-ntag** module ## version 2.9.0 (2024-11-28) * default preferences (see [docs](/docs/discriminat/gcp/default-prefs/)), of which there are four at this time, can now be stored in a Secret named `DiscrimiNAT_` (case-sensitive) in Secret Manager: * `wildcard_exposure`: control whether `*` is accepted in FQDNs or not, and if accepted, with prohibit [public suffix](https://publicsuffix.org/) safeguard or not. **Please familiarise yourself with [operation and caveats](/blog/wildcards-and-system-2-thinking/#operation) on the behaviour of wildcard rules before using them.** * `flow_log_verbosity`: control whether all logs are emitted, just _disallowed_ or none at all * `see_thru`: set non-blocking, monitoring _see-thru_ mode as a default (useful with new deployments so traffic is not blocked upon deployment) * `x509_crls`: whether to automatically allow CRL Endpoints of x509 SSL certificates for all TLS FQDNs allowlisted. This was a feature introduced in v2.7.0 * list of FQDNs for allowlisting, in JSON format, can now be read in from a Secret * format is `{"addrs": ["fqdn1.com", "*.github.com", ...]}` * a symbolic reference of the Secret, using its full path, will need to be added to the Firewall Rules' description field instead of comma-separated FQDNs. For example, `discriminat:tls:projects/000000000000/secrets/service-foo-allowed-fqdns` * DiscrimiNAT's Service Account will need _Secret Manager Secret Accessor_ Role for the Secret paths referred (see [docs](/docs/discriminat/gcp/service-account/)) * **previous method of storing comma-separated FQDNs in Firewall Rules' description fields is preserved and will continue to work** * version of bundled Public Suffix List: [931546b](https://github.com/publicsuffix/list/blob/931546b3beb45b544d0692aa116b420fb34b9dfa/public_suffix_list.dat) * improvement in startup time with a large allowlist (>500 FQDNs) * new config log _reason_: `rejected`. This is emitted in case a wildcard FQDN is specified but the _wildcard\_exposure_ preference value does not allow its inclusion. For example, ``{addr: "*.github.io", cat: "addr", outcome: "publicsuffix[.]org list matched with `github.io`"}`` * raw packet captures (PCAP) for ~10 seconds may be included in telemetry data if Automated System Health Reporting is not opted out of * fixed a bug where wildcard matched FQDNs would not be allowed, until the cache for them was warmed up, in spite of _see-thru_ mode being set * fixed occassional "spoofing detected" flow logs disallowing connections to Cloudinary and Azure Cloud CDN FQDNs * TLS and SSH connectivity improvements to some hosts that would not acknowledge trailing zeroes in padding bytes of a handshake * absence of leading zeroes in month and date components of a _see-thru_ date now works. For example, previously, `2024-9-1` would not have worked, however, `2024-09-01` would have. Both work now. * wildcard matched connections are now timed out on first attempt, instead of connection reset, until the cache has warmed up for them. This reduces the number of attempts made by an app/client when accessing a wildcarded FQDN for the first time. **Breaking Changes** * plaintext HTTP CRL Endpoints of x509 SSL certificates for all TLS FQDNs allowlisted are no longer allowed automatically. Set the preference `x509_crls` to `auto_allow` to restore previous behaviour. If you did not know about this, you are likely not affected. We have proactively informed the customers we definitely knew were relying on this. **Terraform Updates** * [preferences' defaults](/docs/discriminat/gcp/default-prefs/) in JSON format are deployed automatically from v2.9.0 of our Terraform module, in a Secret named _DiscrimiNAT_\_, to serve as a starting point. Can be overriden from the new `preferences` variable. * option to disable automatic updates to the Instance Template when a new DiscrimiNAT image version is available with the `image_auto_update` boolean variable * full diff between previous version and this can be found [here](https://github.com/ChaserSystems/terraform-google-discriminat-ilb/compare/2.8.0...2.9.0) for the **-ilb** module, and [here](https://github.com/ChaserSystems/terraform-google-discriminat-ntag/compare/2.7.2...2.9.0) for the **-ntag** module ## version 2.8.0 (2024-08-20) * wildcard support is now in preview: * they are supported for the TLS protocol only * the character `?` (or `_`) may be used to substitute one wild character in an FQDN to be allowed * the set of wild characters is from `a` to `z`, `0` to `9` and the `-` (hyphen or minus) only; the `.` (period, dot or fullstop) character is not included * you may use any number of wildcards in a single FQDN address (in the allowlist) * see our [dedicated page on using wildcards](/blog/wildcards-and-system-2-thinking/) with examples and the caveats expected in this preview; ensure you've read the __Operation section__ as well * further improvements are expected in the next version of DiscrimiNAT; please [write to us](/support/) with your experience on using this feature * suppressed repetitive warning log messages `no ip addresses resolved` about CRL endpoint `crl.comodo.net` * improved compatibility with proprietary SSH server-side implementations, such as GoAnywhere, that send a larger than normal list of ciphers during the initial handshake * Terraform module [source](https://github.com/ChaserSystems/terraform-google-discriminat-ilb) includes a script `rmig-update-maxUnavailable-1.sh` that could be run in Cloud Shell or as part of your CI/CD pipeline to update DiscrimiNAT instances separately from a version update of the instance template for their Managed Instance Group. This has the added advantage of keeping under the number of allocated External IPs during a rolling update * the script will need setting of `GOOGLE_CLOUD_PROJECT`, `MIG_NAME`, `MIG_EXPECTED_SIZE` and `MIG_REGION` environment variables. Examples are provided within the script * variable `mig_update_policy_type` should be set to `OPPORTUNISTIC` for this to be useful ## version 2.7.1 (2024-02-12) * `warning` type messages in `-config` logs now do not repeat before 10 minutes. This will reduce the frequency of `no ip addresses resolved`, etc. log messages significantly. * disabling **Automated System Health Reporting** can now be done by setting variable `ashr` to `false` from Terraform module [v2.7.1](https://registry.terraform.io/modules/ChaserSystems/discriminat-ilb/google/2.7.1) onwards. The earlier method will continue to work. * automated system health reporting, if left enabled, now runs at shutdown too. * automatically allowed CRL Endpoints of x509 SSL certificates now emit the name of the otherwise allowed FQDN and the issuing CA in the certificate chain, from which the CRL Endpoint was determined, in the `reason` field of the `-config` log. For example, `crl endpoint from issuer GTS Root R1 in certificate chain of trends.google.com`. * `syslog` is no longer sent from DiscrimiNAT VMs to StackDriver (Logs Explorer). This will reduce the level of logging from the operating system. * resolved an issue where large Microsoft updates over plaintext HTTP, while in see-thru mode, could cause DiscrimiNAT to drop packets after a few weeks of passing such traffic through – needing a restart. ## version 2.7.0 (2024-01-03) * HTTP `-flow` Logs: additional log fields of `http_method`, `http_user_agent` and `http_path` will be present for plaintext HTTP traffic to aid in determining the source of unencrypted traffic. The traffic will always be denied with the message `insecure protocol, use https`, though. * CRL Endpoints of x509 SSL certificates, which are over plaintext HTTP, are now automatically allowed for all TLS FQDNs allowlisted. Only HTTP methods `HEAD` and `GET` are allowed to these URLs from only the clients that otherwise have the TLS FQDNs (to which these CRL Endpoints belong) allowed. * Alias IP Ranges assigned to VM Instances now get recognised and functionally work the same way as a VM's primary IP address. Firewall Rules get applied to the ranges as well, just as they do to the primary IP address of a VM, based on Network Tags. * `no ip addresses resolved` warning message in `-config` log for FQDNs found in the allowlist but for which a DNS lookup did not resolve any IP addresses. This is useful in spotting typos and domain names not configured yet by third parties. * Terraform module [v2.7.0](https://registry.terraform.io/modules/ChaserSystems/discriminat-ilb/google/2.7.0) update: `user_data_base64` supersedes ~~`startup_script_base64`~~. See diff [here](https://github.com/ChaserSystems/terraform-google-discriminat-ilb/commit/887f5576300c37fb3a3a80f1a9b1b6d4f5994718). * **Automated System Health Reporting:** 10 minutes after boot and then at around 0200 UTC every day, each instance of DiscrimiNAT will collect its OS internals & system logs since instance creation, config changes & traffic flow information from last two hours and upload it to a Chaser-owned cloud bucket. This information is encrypted at rest with a certain public key so only relevant individuals with access to the corresponding private key can decrypt it. The transfer is encrypted over TLS. Access to this information will be immensely useful to create a faster and more reliable DiscrimiNAT as we add new features. We also aim to learn about how users interact with the product in order to further improve the usability of it as they embark on a very ambitious journey of fully accounted for and effective egress controls. We understand if certain environments within your deployment would rather not have this turned on. **To disable it,** a file at the path `/etc/chaser/disable_automated-system-health-reporting` should exist. From our Terraform module v2.7.0 onwards, this can be accomplished by including the following statement: ``` user_data_base64 = "I2Nsb3VkLWNvbmZpZwp3cml0ZV9maWxlczoKLSBwYXRoOiAvZXRjL2NoYXNlci9kaXNhYmxlX2F1dG9tYXRlZC1zeXN0ZW0taGVhbHRoLXJlcG9ydGluZwo=" ``` The _base64_ value above decodes to: ``` #cloud-config write_files: - path: /etc/chaser/disable_automated-system-health-reporting ``` Which is a [cloud-init](https://cloudinit.readthedocs.io/en/latest/reference/examples.html) way of creating that file in the instance. ## version 2.6.1 (2023-10-10) * operating system kernel tuning parameters tweaked (to match with the AWS build) resulting in improved performance of the see-thru mode * health check logic now also waits for the firewall cache to build up a bit before giving a green light to the load balancer (and therefore accepting traffic on new VMs) * fixed a bug where a certain configuration of a Firewall Rule would have DiscrimiNAT drop all rules * two new warning log message types which indicate if the configured port in a Firewall Rule has a connection-level issue: * for example `test for TLS on 203.0.113.5:80 failed` – port 80 was not listening with TLS * and for example `timed out testing connection to 203.0.113.6:443` – port 443 on that IP address is not open (from DiscrimiNAT's public IP point of view) * minor update to [Terraform modules](https://registry.terraform.io/namespaces/ChaserSystems) increasing the health check initial delay from 120 seconds to 360 seconds ## version 2.6.0 (2023-07-11) * the TLS notation for allowlisting now supports IP v4 addresses besides FQDNs (i.e. without SNI), for example `discriminat:tls:203.0.113.9` * the _see-thru_ monitoring mode now accepts all specifications of IP addresses, Protocols & Ports on Firewall Rules. Previously, it had required IP addresses to be set to `0.0.0.0/0` and Ports & Protocols to be to set to _all_. * two new `-flow` log `reason` messages when a network packet is `disallowed` have been introduced: * `cache not ready`: this message is logged when a new address is added in the allowlist but the firewall has not yet warmed up its cache for it. Expected to occur for up to 2 minutes after adding a new address (FQDN or IP.) * `spoofing detected`: logged when TLS SNI has been manipulated and a connection is attempted to an IP address that doesn't otherwise belong to the given FQDN (in the SNI.) * enabled Finite Field Diffie–Hellman ciphers for TLS 1.2, for example DHE as opposed to ECDHE * enabled ciphers **without** Forward Secrecy for TLS 1.2 **Breaking Changes** * the `-config` log has its `fqdn` field name changed to `addr`. Field names have not changed in the `-flow` log. * in the `-config` log, association of a public IP for egress from the firewall had its category (`cat` field) set to `static-ip`. This is now set to `egress-ip`. ## version 2.5.3 (2023-05-22) * increased tolerance towards some rare TLS servers that otherwise resulted in DiscrimiNAT logging connection test failures and not allowing connections to them ## version 2.5.2 (2023-03-16) * change of base OS from Ubuntu 18.04 to Ubuntu 20.04 The CIS Ubuntu Linux 20.04 LTS Benchmark v1.1.0 Level 2 - Server report is available upon request by contacting support. The image scored 215/219. An explanation will be attached for the unmet 4. ## version 2.5.1 (2023-01-29) * External IPs can now be pinned to specific deployments. If the value of a DiscrimiNAT instance VM label `discriminat` matches the label-key & value of an External IP, that External IP will be preferred for self-attaching. Fallback behaviour remains to self-attaching any allocated but unassociated External IPs with the label-key `discriminat` set to any value. * Terraform module updated to support custom deployment IDs (see variable `custom_deployment_id`), to optionally override the randomly generated ones. This allows fine-grained control over naming and can also be used for matching allocated External IPs to a specific fleet of DiscrimiNAT instances. * replaced google-fluentd with ops-agent. Ops Agent requires the Monitoring Metric Writer predefined role to be added to the service account. See our [Service Account](/docs/discriminat/gcp/service-account/#ops-agent) page for more details. ## version 2.4.2 (2022-10-05) * added support for [Network Tags in Cloud Composer v2 GKE Clusters](/docs/discriminat/gcp/cloud-composer-v2-gke-clusters) * improved audit (config) logging for subnets * improved connection handling for very short lived TLS connections with specific server-side implementations (such as Envoy Proxy) ## version 2.4.1 (2022-05-05) * fixed an excessive retries issue with automatic config building where the service account's role allowed querying of Service Projects but the Host Project did not have shared VPC setup enabled * fixed a sporadic connection reset issue, that emitted `unexpected response` in the logs, and which only occurred in the `see-thru` monitoring mode while connecting to a destination at very high latency ## version 2.4.0 (2022-03-13) * new warning message in config logs when a connection test to an FQDN, carried out by discrimiNAT itself, in any allowlist fails * [serverless support introduced](/docs/discriminat/gcp/serverless-vpc-access-connectors); VPC connectors from Cloud Functions etc. will have their outbound traffic filtered * added support for [self-attaching an allocated External IP](/docs/discriminat/gcp/service-account#external-ips) * change of one of the default scopes, when the service account is not overridden,from `compute-ro` to `compute-rw`; this is to support self-assignment of labelled external IPs * discrimiNAT's own instance ID added to every log line under the key `instance`, indicating which instance the log line was emitted from * updated [TLS ECH draft extension](https://datatracker.ietf.org/doc/draft-ietf-tls-esni/) identifiers ## version 2.3.0 (2021-11-02) * added support for shared VPC; now Service Projects can use a discrimiNAT instance deployed in their Host Project ## version 2.2.0 (2021-08-31) * [see-thru mode](/docs/discriminat/gcp/config-ref#see-thru-mode) introduced; build allowlists super-quick by putting a Firewall Rule in monitor mode first * improved handling for a large number of FQDNs in the allowlists * full bypass hook added; please reach out to [support](/support) for instructions on this * updated [TLS ECH draft extension](https://datatracker.ietf.org/doc/draft-ietf-tls-esni/) identifiers ## version 2.0.5 (2021-05-03) * restricted firewall rule scanning to the same VPC as discrimiNAT firewall was deployed in ## version 2.0.4 (2021-04-03) * set compute image family to `discriminat` * updated [TLS ECH draft extension](https://datatracker.ietf.org/doc/draft-ietf-tls-esni/) identifiers ## version 2.0.3 (2020-11-05) * v2 launch * completely new architecture addressing the potential for mismatch of IPs addresses as looked up by a protected workload from the VPC resolver and as looked up by the discrimiNAT firewall * rewritten in Rust ## version 20200529 (2020-05-29) _available on request; v1 is now deprecated; please upgrade to v2_ ## version 20200524 (2020-05-24) _available on request; v1 is now deprecated; please upgrade to v2_ ## version 20200516 (2020-05-16) _available on request; v1 is now deprecated; please upgrade to v2_ ## version 20191207 (2019-12-07) _available on request; v1 is now deprecated; please upgrade to v2_ ## version 20191107 (2019-11-07) * v1 launch --- ## Serverless VPC Access Connectors ## Concepts >Serverless VPC Access makes it possible for you to connect directly to your Virtual Private Cloud network from serverless environments such as Cloud Run, App Engine, or Cloud Functions. Source: https://docs.cloud.google.com/vpc/docs/serverless-vpc-access **From v2.4** onwards, the DiscrimiNAT supports filtering traffic egressing from Serverless VPC Access Connectors. ## Network tags The network tags on Firewall Rules that apply to Serverless VPC Access Connectors have preset names defined by Google Cloud Platform. >Serverless VPC Access network tags let you refer to VPC connectors in firewall rules and routes. > >Every Serverless VPC Access connector automatically receives two network tags (sometimes called instance tags): > * Universal network tag: `vpc-connector` Applies to all existing connectors and any connectors made in the future > * Unique network tag: `vpc-connector-REGION-CONNECTOR_NAME` Applies to the connector `CONNECTOR_NAME` in `REGION` > >These network tags cannot be deleted. New network tags cannot be added. Source: https://docs.cloud.google.com/vpc/docs/serverless-vpc-access#network-tags ### Examples 1. For a connector named `foo` in `europe-west2` region, the network tag applicable to just the traffic from this connector will be `vpc-connector-europe-west2-foo`. 2. The network tag applicable to all connectors, regardless of name or region, will be `vpc-connector`. ## Requirements ### Serverless The egress settings of the serverless platform need to be configured to send all traffic through a connector rather than just the traffic to internal addresses. This is accomplished during the setup of the serverless workload, and in the following ways. Choose Route all traffic through the VPC connector when selecting a connector. Reference: https://docs.cloud.google.com/run/docs/configuring/connecting-vpc#connectors Choose Route all traffic through the VPC connector under Egress settings. Reference: https://docs.cloud.google.com/run/docs/configuring/networking-best-practices#direct-vpc-throughput Set the egress_settings property to all-traffic. vpc_access_connector:{'\n'}   name: projects/PROJECT_ID/locations/REGION/connectors/CONNECTOR_NAME{'\n'}   egress_setting: all-traffic Reference: https://docs.cloud.google.com/appengine/docs/standard/connecting-vpc#egress ### Firewall Rules Firewall Rules from NAT Ranges and Health Check Ranges, as described at ¹, must be created for the VPC Connectors. Additionally, firewall rules to allow connections from serverless workloads to DiscrimiNAT instances would have to be created. If using Terraform, the variable `client_cidrs` accepts a list of IP ranges for this. The subnets allocated for VPC Connectors should be added to this list. ¹ https://docs.cloud.google.com/vpc/docs/configure-serverless-vpc-access#restrict-access ## Role The following permissions must be added to the Role meant to be granted to the service account for DiscrimiNAT instances, in order for it to be able to pick up the connector subnets: ```plaintext vpcaccess.connectors.list compute.subnetworks.get ``` :::tip If Serverless VPC Access Connectors are deployed in Service Projects of a Shared VPC , see the [Shared VPC setup](../shared-vpc/) page and grant permissions as discussed in the [Service Account](../service-account/) page. ::: --- ## Service Account The DiscrimiNAT Firewall requires different permissions to work effectively across differing setups. The most basic of the setups is a single project with only VM instances to protect within the same project. The permissions for this are included with our Deployment Manager template and Terraform modules as allowed scopes on the DiscrimiNAT instances. For the rest, and if you'd rather manage the service account with discrete permissions, the following should come in handy. ## TL;DR For all features and topologies enabled, 1. Create an IAM service account, in a _host_ project, for use by DiscrimiNAT instances. 2. Create an organisation-level IAM Role with the following permissions. 3. Create a folder-level† (or organisation-level) IAM binding granting the above IAM Role to the Principal of the above service account. :::caution † The chosen folder should contain the _host_ project and all _service_ projects either directly or through subfolders. Bindings can also be created on multiple folders. ::: ``` logging.logEntries.create compute.firewalls.list compute.instances.list compute.addresses.list compute.addresses.use compute.subnetworks.useExternalIp compute.instances.addAccessConfig compute.projects.get vpcaccess.connectors.list compute.subnetworks.get container.clusters.list composer.environments.list run.jobs.list run.services.list run.workerpools.list roles/monitoring.metricWriter roles/secretmanager.secretAccessor° ``` :::tip `roles/monitoring.metricWriter` is the Google [predefined `Monitoring Metric Writer` role](https://gcp.permissions.cloud/predefinedroles/monitoring.metricWriter). It can be associated with the service account as a whole. ::: :::caution ° The _Secret Manager Secret Accessor_ Role can be conditionally bound to only the Secret(s) that access is required to with the CEL Expression `resource.name == "projects//secrets/DiscrimiNAT_/versions/latest"`. Also see Terraform examples below. :::caution ## LOGGING The DiscrimiNAT automatically logs config & flow events to StackDriver. The following permission allows its instances to log at `logName="projects//logs/discriminat-config"` and `logName="projects//logs/discriminat-flow"`. ``` logging.logEntries.create ``` ## ALLOWLIST CONFIG To form a complete picture of the VM instances in the VPC and their associated Firewall Rules, the DiscrimiNAT needs read access to these resources. ``` compute.firewalls.list compute.instances.list ``` ## DEFAULT PREFERENCES Global level, behavioural [preferences](/docs/discriminat/gcp/default-prefs/) are stored in a Secret named DiscrimiNAT in Secret Manager. This is best permitted through a built-in Role and a Condition limiting the scope to only that named Secret. ``` roles/secretmanager.secretAccessor ``` Condition Expression: `projects//secrets/DiscrimiNAT_/versions/latest` This is worked out and the interpolation automated for Terraform in the [examples below](#terraform). ## SHARED VPC In addition to the above, to discover VM instances in service projects, the following permission is required. See the [Shared VPC](../shared-vpc/) page for more information. ``` compute.projects.get ``` ## SERVERLESS VPC ACCESS CONNECTORS In addition to the above, the following will be required to form a complete picture of any Serverless VPC Access Connectors. If permissions for Shared VPC are included, then any Serverless VPC Access Connectors defined in service projects will also be picked up. ``` vpcaccess.connectors.list compute.subnetworks.get ``` See the [Serverless VPC Access Connectors](../serverless-vpc-access-connectors/) page for more information. ## CLOUD COMPOSER v2 GKE CLUSTERS A Composer v2 environment's cluster is an Autopilot mode VPC-native Google Kubernetes Engine cluster. Network Tags to it, however, can be applied at the time of Composer environments creation. Add the following permissions to the service account's role to allow the DiscrimiNAT to apply corresponding Firewall Rules to the clusters' subnets. ``` container.clusters.list compute.subnetworks.get ``` ## CLOUD COMPOSER v3 ``` composer.environments.list ``` ## CLOUD RUN DIRECT VPC EGRESS ``` run.services.list ``` ## EXTERNAL IPs If a Public IP is not found attached to a DiscrimiNAT instance, it will look for any allocated but unassociated External IPs that have a label-key named `discriminat` – the value which should be set to the value of the variable `custom_deployment_id` of the [Terraform module](https://registry.terraform.io/namespaces/ChaserSystems), if that was set, else anything but blank. One of such External IPs will be attempted to be associated with itself then. :::tip This allows you to have a stable set of static IPs to share with your partners, who may wish to allowlist them. ::: **Private Google Access** enabled on the subnet DiscrimiNAT is deployed in is needed for this mechanism to work though – since making the association needs access to the Compute API. In the [google_network example](https://github.com/ChaserSystems/terraform-google-discriminat-ilb/blob/main/examples/google_network), this is demonstrated by setting `subnet_private_access = true`. ``` compute.addresses.list compute.addresses.use compute.subnetworks.useExternalIp compute.instances.addAccessConfig ``` ## OPS AGENT The DiscrimiNAT comes with Google's [Ops Agent](https://docs.cloud.google.com/stackdriver/docs/solutions/agents/ops-agent) pre-installed. Should you require detailed metrics on the operating system's performance and utilisation, such as memory and load average, the permissions granted by the Monitoring Metric Writer role would be required. ``` roles/monitoring.metricWriter ``` :::tip `roles/monitoring.metricWriter` is the Google [predefined `Monitoring Metric Writer` role](https://gcp.permissions.cloud/predefinedroles/monitoring.metricWriter). It can be associated with the service account as a whole. ::: ## TERRAFORM If using Terraform, a new variable from v2.3 onwards has been introduced to let the user override the service account used by DiscrimiNAT instances. This is the `custom_service_account_email` variable and if left unset, defaults to the default compute engine service account with limited scopes of `compute-rw`, `logging-write` and `monitoring-write`. If set to the Principal of any service account, assigns that service account to the instances and changes the scope to `cloud-platform`. ### Sample code to create a Service Account, the IAM Role and an IAM Binding for a simple, single Project only deployment ```terraform variable "project_id" { type = string description = "Project ID for the Service Account to reside in. Think about the actual Project the DiscrimiNAT Firewall will be deployed in (where the VPC Network will be.) You must have privileges to create a new Service Account in this Project." } variable "discriminat_deployment_id" { type = string description = "As output from DiscrimiNAT's Terraform modules for its deployment. This is the suffix to `DiscrimiNAT_` Secret (in Secret Manager) for building an IAM Conditional Access expression and scope down the access to one the Secret (`DiscrimiNAT_`) which stores Preferences for this deployment of DiscrimiNAT only." } data "google_project" "current" { project_id = var.project_id } resource "google_project_iam_custom_role" "discriminat" { role_id = "DiscrimiNAT" title = "DiscrimiNAT Firewall Role" project = var.project_id description = "Permissions needed for DiscrimiNAT Firewall to read metadata from VMs, Subnetworks, Firewall Rules, etc., write Logs & Metrics, and attach an External IP to itself. See upstream docs for full details: https://chasersystems.com/docs/discriminat/gcp/service-account/" permissions = ["logging.logEntries.create", "compute.firewalls.list", "compute.instances.list", "compute.addresses.list", "compute.addresses.use", "compute.subnetworks.useExternalIp", "compute.instances.addAccessConfig", "compute.projects.get", "vpcaccess.connectors.list", "compute.subnetworks.get", "container.clusters.list", "composer.environments.list", "run.services.list"] } resource "google_service_account" "discriminat" { account_id = "discriminat" project = var.project_id display_name = "DiscrimiNAT Firewall Service Account" description = "For DiscrimiNAT Firewall. See upstream docs for full details: https://chasersystems.com/docs/discriminat/gcp/service-account/" } resource "google_project_iam_member" "discriminat_custom_role" { project = var.project_id role = google_project_iam_custom_role.discriminat.id member = "serviceAccount:${google_service_account.discriminat.email}" } resource "google_project_iam_member" "discriminat_metric_writer" { project = var.project_id role = "roles/monitoring.metricWriter" member = "serviceAccount:${google_service_account.discriminat.email}" } resource "google_project_iam_member" "discriminat_prefs_access" { project = var.project_id role = "roles/secretmanager.secretAccessor" member = "serviceAccount:${google_service_account.discriminat.email}" condition { title = "specific_secret" description = "Restrict access to the specific Secret for DiscrimiNAT Preferences" expression = < Shared VPC allows an organization to connect resources from multiple projects to a common Virtual Private Cloud (VPC) network, so that they can communicate with each other securely and efficiently using internal IPs from that network. When you use Shared VPC, you designate a project as a **host project** and attach one or more other **service projects** to it. The VPC networks in the **host project** are called Shared VPC networks. Eligible resources from **service projects** can use subnets in the Shared VPC network. Source: https://docs.cloud.google.com/vpc/docs/shared-vpc **Shared VPCs** in Google Cloud Platform have emerged as a robust design pattern for scaling business units and their services in an enterprise setting. The DiscrimiNAT firewall, **from v2.3** onwards, supports the Shared VPC topology. ## Requirements * An IAM service account, in the host project, must be created for use by DiscrimiNAT instances. * An organisation-level IAM Role must exist that allows getting information about firewall rules, associated service projects and compute instances within. Details in the **Role** section below. * A folder-level (or organisation-level) IAM binding granting that IAM Role to the Principal of the service account must exist. * The DiscrimiNAT firewall must be deployed in the host project. * The service account for DiscrimiNAT instances must be explicitly set to the Principal of that service account. * The egress-filtered workload may be deployed in the host project and/or any attached service projects. * Firewall Rules to allow connections from the egress-filtered workloads' subnets to DiscrimiNAT instances would have to be created. If using Terraform, the variable `client_cidrs` accepts a list for this. ## Role :::caution IMPORTANT This Role must be created at the organisation-level. ::: The following permissions must be added to the Role meant to be granted to the service account for DiscrimiNAT instances, in order for it to be able to pick up the firewall rules, the associated projects and the compute instances within: ```plaintext logging.logEntries.create compute.firewalls.list compute.instances.list compute.projects.get ``` :::tip Also see the [Service Account](../service-account/) page for other permissions of interest. ::: ## Terraform If using Terraform, a new variable from v2.3 onwards has been introduced to let the user override the service account used by DiscrimiNAT instances. This is the `custom_service_account_email` variable and if left unset, defaults to the default compute engine service account with limited scopes of `compute-rw`, `logging-write` and `monitoring-write`. If set to the Principal of any service account, assigns that service account to the instances and changes the scope to `cloud-platform`. :::tip Firewall Rules to allow connections from the egress-filtered workloads to DiscrimiNAT instances would have to be created. If using Terraform, the variable `client_cidrs` accepts a list for this. DiscrimiNAT instances are deployed with a `discriminat-itself` network tag for ease in defining any other network rules. ::: ## FAQ #### Does DiscrimiNAT work without a custom service account? Yes. In a non-Shared VPC setting, it is able to work for private IP workloads in the same project as itself. In a Shared VPC setting, it will behave in a similar fashion and work for private IP workloads in the same project as itself. However, traffic arriving from other Projects will simply be rejected. #### Is there a setting in the DiscrimiNAT to be toggled for Shared VPC support? No. The DiscrimiNAT figures out the best possible setting looking at the service account, the permissions that come with it, and whether it is indeed in a host project with a Shared VPC. #### Where are the Firewall Rules defined in a Shared VPC setting? The Firewall Rules are defined at the VPC-level in a Shared VPC. With that constraint, they can only be defined in the host project. The Firewall Rules can be defined upfront to apply to only certain targets with Network Tags. These Network Tags can then be applied and used in the service projects on any compute workloads with only private IPs, and the DiscrimiNAT will figure out the FQDN egress rules to apply to each client as usual. --- ## Creating a Bastion for SSH access on GCP On some occasions you may want a [bastion host](https://en.wikipedia.org/wiki/Bastion_host) present in your VPC. For example: - accessing a DiscrimiNAT instance over SSH - accessing a host without a public IP _psst... should you ever need a great alternative to a [~~shoal~~ squad](https://www.change.org/p/scientific-community-call-a-group-of-squids-a-squad-instead-of-a-shoal) of Squid proxies for outbound filtering, consider [DiscrimiNAT](/discriminat) for a cloud-native solution_ ### Creating a Bastion Host ![GCP Create a Bastion Host VM](img/bastion-01.png) Let's create a new instance pretty much as usual, taking care that: - it's in the right region - it doesn't need too much grunt as it will only pass minimal traffic through We will need to add a small detail in the networking section, though. See the next image. ![GCP VM Network Tags to bypass DiscrimiNAT](img/bastion-02.png) Add the network tag `bypass-discriminat` to this instance. The presence of this network tag enables the traffic to and from this instance to have higher precedence than the control laid out by the DiscrimiNAT Firewall, therefore _bypassing_ it. **This will allow you to directly connect to this instance, from your public IP to its public IP.** Your bastion host on GCP is now ready! ### Accessing a DiscrimiNAT instance through the Bastion The DiscrimiNAT image is hardened per _CIS Ubuntu Linux 24.04 LTS Benchmark Level 2 - Server_. Therefore, SSH access to it requires your posture to be sound and secure. Let's check with a few commands on your machine: 1. The SSH Agent should have only one identity loaded. This is to prevent it from trying one identity after another to the server, causing the server to block the user after too many failures. So it's safer to just have the one identity that will work. `ssh-add -L` If the output shows more than one line, you may clear all of them out with the command `ssh-add -D`. If the previous command still shows some lines, check the `.ssh` directory in your home directory for unexpected private key files. 2. To add a specific private key to the SSH Agent, run the command: `ssh-add /path/to/private-key` And then check with `ssh-add -L` whether only one line in the output is present. 3. Check the Compute Metadata -> SSH Keys in Google Cloud (GCP) console for the username of the key loaded above. ![GCP VM External and Internal IPs](img/bastion-03.png) 4. Finally, SSH into the DiscrimiNAT instance using the bastion host as _ProxyJump_. This is needed because DiscrimiNAT will only allow SSH connections from private IPs (i.e. within the VPC), so you cannot connect to it from a public IP. You will need the public IP address of the bastion, the private IP address of the target DiscrimiNAT instance, and this command: `ssh -J @ @` An example of a fully formed command from the example deployment in the screenshots is: `ssh -J don@34.122.172.195 don@10.128.0.22` If you manage to lock yourselves out of a DiscrimiNAT instance due to repeated authentication failures, either terminate the instance and let the managed instance group bring back a new one, or wait 15 minutes. ### Accessing a host without a public IP through the Bastion This could be much simpler if your target instances are not fussy about where you connect from and how many keys you present. However, since the targets do not have a public IP address at all in this case, you still need to use a bastion host, so there is connectivity to and from them. The commands are the same as above: 1. Verify the right keys are loaded in your SSH Agent: `ssh-add -L` 2. Connect using the bastion host as _ProxyJump_: `ssh -J @ @` --- ## Troubleshooting(Gcp) ### Video for End Users If you are an application developer, a service owner or an end-user of DiscrimiNAT tasked with getting new domain names working, this \<5m video is for you and covers the most common issue along with solutions. ### Search for _warning_ messages from DiscrimiNAT The firewall may have tried to warn you about an issue it can foresee. Use the following query in Logs Explorer (replacing _project name_ with one where DiscrimiNAT is deployed): ``` logName="projects//logs/discriminat-config" jsonPayload.outcome="warning" ``` You may find a descriptive clue in the resultant log lines from the last few minutes/hours. ### There are no `discriminat-config` logs at all. ![](img/no-discriminat-config-log.gif) #### Resolution 1. Check if the [service account](https://console.cloud.google.com/iam-admin/serviceaccounts) for DiscrimiNAT instances is enabled. It is the default Compute Engine account if DiscrimiNAT is deployed through the Marketplace directly. If deployed through Terraform, the default service account is the same unless overridden with the `custom_service_account_email` variable. 2. Check if the [Cloud Logging API](https://console.cloud.google.com/apis/api/logging.googleapis.com/metrics) is enabled. ### Clients' network traffic sometimes allowed, sometimes not. #### Resolution Please upgrade the machine type from `e2-small` to `n2-highcpu-2`. This is done via the `machine_type` Terraform variable. The `e2-small` machine type only suffices for light loads and a few clients. The `n2-highcpu-2` offers the best CPU to egress bandwidth and price ratios for the kind of work involved. Use of `n2-standard-*` machine types is not recommended because the DiscrimiNAT is not a memory-intensive application, therefore making machines with more memory than needed not a cost-optimal choice. ### There are no `discriminat-flow` logs from certain clients. Certain, or all, clients may not even be able to access the Internet in this case. Requests would be timing out instead of being quickly terminated with a reset. #### Resolution 1. Check if the subnet, where the affected clients are, is allowed for ingress in a [firewall rule](https://console.cloud.google.com/networking/firewalls/list) named "discriminat-\-from-clients". If deployed via our Terraform module, additional subnets can be added to this firewall rule through the `client_cidrs` variable. ### _see-thru_ mode not working If the `discriminat-config` logs do not show a log line picking up a _see-thru_ rule, you may have a problem with the annotation's implied syntax. #### Resolution The _see-thru_ mode requires a Firewall Rule (or the _see\_thru_ [default preference](/docs/discriminat/gcp/default-prefs/#see_thru)) to have a valid calendar date. For example, `discriminat:see-thru:2022-02-29` is NOT a valid date (because 2022 was not a leap year) but `discriminat:see-thru:2022-02-28` is. ### Service Projects' network traffic not going through If the `discriminat-config` logs do not show log lines picking up VM Instances' IPs or other managed services' subnets from the Service Projects, you may have a problem with how the service account was configured. #### Resolution For Service Projects to work through the DiscrimiNAT, ensure that: 1. DiscrimiNAT is running with a custom service account. This is overridden with the `custom_service_account_email` variable in Terraform. 1. The Role to be associated with that service account is defined at the Organisation level in your Google Cloud and NOT at the Project level. 1. The IAM binding of the service account to that Role should be defined at the Folder† level (or the Organisation level) and NOT at the Project level. † The chosen Folder should contain the Host project and all Service projects either directly or through subfolders. For more details on the service account and the Role required, please see the [Service Account](/docs/discriminat/gcp/service-account/) page. --- ## YouTube Playlist(Gcp) --- ## CentOS 7 yum HTTPS upgrade This guide aims to help in pinning down the repository location to a known FQDN, so it can be set in the allowlists, without affecting the performance of downloads. ## Virtual Machine ### AWS User Data ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%#baseurl=http://mirror.centos.org/centos/%baseurl=https://mirrors.edge.kernel.org/centos/%g' /etc/yum.repos.d/CentOS-*.repo sed --in-place --regexp-extended 's%^mirrorlist=%#mirrorlist=%g' /etc/yum.repos.d/CentOS-*.repo ``` The commands simply replace CentOS's default mirror redirector with a known, reliable CDN, making a backup file (with extension _.orig_) in the process. ### GCP Startup Script ```bash #!/bin/bash -ex # centos sed --in-place=.orig --regexp-extended 's%#baseurl=http://mirror.centos.org/centos/%baseurl=https://mirrors.edge.kernel.org/centos/%g' /etc/yum.repos.d/CentOS-*.repo sed --in-place --regexp-extended 's%^mirrorlist=%#mirrorlist=%g' /etc/yum.repos.d/CentOS-*.repo # epel sed --in-place=.orig --regexp-extended 's%#baseurl=http://(download.example|download.fedoraproject.org)/pub/epel/%baseurl=https://mirrors.edge.kernel.org/fedora-epel/%g' /etc/yum.repos.d/epel*.repo sed --in-place --regexp-extended 's%^metalink=%#metalink=%g' /etc/yum.repos.d/epel*.repo ``` The first set of commands simply replace CentOS's default mirror redirector with a known, reliable CDN, making a backup file (with extension _.orig_) in the process. The second set of commands simply replace Fedora's default mirror redirector with a known, reliable CDN, making a backup file (with extension _.orig_) in the process. ## Container ### Dockerfile ```Dockerfile FROM centos:7 RUN sed --in-place=.orig --regexp-extended 's%#baseurl=http://mirror.centos.org/centos/%baseurl=https://mirrors.edge.kernel.org/centos/%g' /etc/yum.repos.d/CentOS-*.repo && \ sed --in-place --regexp-extended 's%^mirrorlist=%#mirrorlist=%g' /etc/yum.repos.d/CentOS-*.repo && \ yum clean expire-cache ``` ## Allowlist ### FQDNs ``` mirrors.edge.kernel.org ``` ### DiscrimiNAT Annotation ``` discriminat:tls:mirrors.edge.kernel.org ``` ### Alternative Mirrors Although `mirrors.edge.kernel.org` is a CDN with geo-located caches, you may want to pick a specific `https` mirror from CentOS's [official mirrors list](https://www.centos.org/download/mirrors/). --- ## CentOS 8 Stream yum HTTPS upgrade This guide aims to help in pinning down the repository location to a known FQDN, so it can be set in the allowlists, without affecting the performance of downloads. ## Virtual Machine ### AWS User Data / GCP Startup Script ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%#baseurl=http://mirror.centos.org/%baseurl=https://mirrors.edge.kernel.org/%g' /etc/yum.repos.d/CentOS-Stream-*.repo sed --in-place --regexp-extended 's%^mirrorlist=%#mirrorlist=%g' /etc/yum.repos.d/CentOS-Stream-*.repo ``` The commands simply replace CentOS's default mirror redirector with a known, reliable CDN, making a backup file (with extension _.orig_) in the process. ## Allowlist ### FQDNs ``` mirrors.edge.kernel.org ``` ### DiscrimiNAT Annotation ``` discriminat:tls:mirrors.edge.kernel.org ``` ### Alternative Mirrors Although `mirrors.edge.kernel.org` is a CDN with geo-located caches, you may want to pick a specific `https` mirror from CentOS's [official mirrors list](https://www.centos.org/download/mirrors/). --- ## Debian 10 apt HTTPS upgrade Upgrading package downloads from HTTP to HTTPS can help with meeting standards such as PCI DSS and NIST SP 800-53. Use of secure transport can also mitigate against future zero-day, remote code vulnerabilities with package managers themselves, such as [CVE-2019-3462](https://security-tracker.debian.org/tracker/CVE-2019-3462), [CVE-2016-1252](https://security-tracker.debian.org/tracker/CVE-2016-1252) and [CVE-2014-6273](https://security-tracker.debian.org/tracker/CVE-2014-6273) in the past. This short guide aims to help in upgrading the default repositories' URLs from plaintext to encrypted. ## Virtual Machine The commands simply change the URL scheme from `http` to `https`, making a backup file (with extension _.orig_) in the process. ### AWS User Data ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%http://(cdn-aws.)*(deb|security).debian.org%https://\2.debian.org%g' /etc/apt/sources.list ``` ### GCP Startup Script ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%http://(deb|security).debian.org%https://\1.debian.org%g' /etc/apt/sources.list sed --in-place=.orig --regexp-extended 's%http://packages.cloud.google.com%https://packages.cloud.google.com%g' /etc/apt/sources.list.d/*.list ``` ## Container ### Dockerfile ```Dockerfile FROM debian:10-slim RUN echo 'Acquire::https::Verify-Peer "false";' > /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf && \ sed --in-place=.orig --regexp-extended 's%http://(deb|security).debian.org%https://\1.debian.org%g' /etc/apt/sources.list && \ apt-get update && \ apt-get install --assume-yes ca-certificates && \ rm /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf ``` These commands first turn off SSL certificate verification because (a) the container build process may be behind [DiscrimiNAT](/discriminat/) already and (b) the CA certificates bundle needs to be downloaded for the verification process to work. [DiscrimiNAT](/discriminat/) independently verifies each connection and will ensure _apt_ connected to the CDN specified in spite of not verifying the certificate. The configuration that turned the verification off is removed after the bundle is installed. ## Allowlist ### FQDNs ``` deb.debian.org,security.debian.org,packages.cloud.google.com ``` ### DiscrimiNAT Annotation ``` discriminat:tls:deb.debian.org,security.debian.org,packages.cloud.google.com ``` --- ## Debian 11 apt HTTPS upgrade Upgrading package downloads from HTTP to HTTPS can help with meeting standards such as PCI DSS and NIST SP 800-53. Use of secure transport can also mitigate against future zero-day, remote code vulnerabilities with package managers themselves, such as [CVE-2019-3462](https://security-tracker.debian.org/tracker/CVE-2019-3462), [CVE-2016-1252](https://security-tracker.debian.org/tracker/CVE-2016-1252) and [CVE-2014-6273](https://security-tracker.debian.org/tracker/CVE-2014-6273) in the past. This short guide aims to help in upgrading the default repositories' URLs from plaintext to encrypted. ## Virtual Machine The commands simply change the URL scheme from `http` to `https`, making a backup file (with extension _.orig_) in the process. ### AWS User Data ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%http://(cdn-aws.)*(deb|security).debian.org%https://\2.debian.org%g' /etc/apt/sources.list ``` ### GCP Startup Script ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%http://(deb|security).debian.org%https://\1.debian.org%g' /etc/apt/sources.list sed --in-place=.orig --regexp-extended 's%http://packages.cloud.google.com%https://packages.cloud.google.com%g' /etc/apt/sources.list.d/*.list ``` ## Container ### Dockerfile ```Dockerfile FROM debian:11-slim RUN echo 'Acquire::https::Verify-Peer "false";' > /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf && \ sed --in-place=.orig --regexp-extended 's%http://(deb|security).debian.org%https://\1.debian.org%g' /etc/apt/sources.list && \ apt-get update && \ apt-get install --assume-yes ca-certificates && \ rm /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf ``` These commands first turn off SSL certificate verification because (a) the container build process may be behind [DiscrimiNAT](/discriminat/) already and (b) the CA certificates bundle needs to be downloaded for the verification process to work. [DiscrimiNAT](/discriminat/) independently verifies each connection and will ensure _apt_ connected to the CDN specified in spite of not verifying the certificate. The configuration that turned the verification off is removed after the bundle is installed. ## Allowlist ### FQDNs ``` deb.debian.org,security.debian.org,packages.cloud.google.com ``` ### DiscrimiNAT Annotation ``` discriminat:tls:deb.debian.org,security.debian.org,packages.cloud.google.com ``` --- ## Fedora EPEL 7 mirror pinning This guide aims to help in pinning down the repository location to a known FQDN, so it can be set in the allowlists, without affecting the performance of downloads. ## Virtual Machine ### AWS User Data / GCP Startup Script ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%#baseurl=http://(download.example|download.fedoraproject.org)/pub/epel/%baseurl=https://mirrors.edge.kernel.org/fedora-epel/%g' /etc/yum.repos.d/epel*.repo sed --in-place --regexp-extended 's%^metalink=%#metalink=%g' /etc/yum.repos.d/epel*.repo ``` The commands simply replace Fedora's default mirror redirector with a known, reliable CDN, making a backup file (with extension _.orig_) in the process. ## Container ### Dockerfile ```Dockerfile FROM centos:7 RUN yum install --assumeyes epel-release && \ sed --in-place=.orig --regexp-extended 's%#baseurl=http://(download.example|download.fedoraproject.org)/pub/epel/%baseurl=https://mirrors.edge.kernel.org/fedora-epel/%g' /etc/yum.repos.d/epel*.repo && \ sed --in-place --regexp-extended 's%^metalink=%#metalink=%g' /etc/yum.repos.d/epel*.repo && \ yum clean expire-cache ``` ## Allowlist ### FQDNs ``` mirrors.edge.kernel.org ``` ### DiscrimiNAT Annotation ``` discriminat:tls:mirrors.edge.kernel.org ``` ### Alternative Mirrors Although `mirrors.edge.kernel.org` is a CDN with geo-located caches, you may want to pick a specific `https` mirror from Fedora's [official mirrors list](https://admin.fedoraproject.org/mirrormanager/mirrors/EPEL). --- ## Fedora EPEL 8 mirror pinning Even though Fedora EPEL repository's configuration points to an HTTPS location, the default location of `mirrors.fedoraproject.org` is a redirect service that redirects to a close-by but non-deterministic mirror. This guide aims to help in pinning down the repository location to a known FQDN, so it can be set in the allowlists, without affecting the performance of downloads. ## Virtual Machine ### AWS User Data / GCP Startup Script ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%#baseurl=https://download.example/pub/epel/%baseurl=https://mirrors.edge.kernel.org/fedora-epel/%g' /etc/yum.repos.d/epel*.repo sed --in-place --regexp-extended 's%^metalink=%#metalink=%g' /etc/yum.repos.d/epel*.repo ``` The command simply replaces Fedora's default mirror redirector with a known, reliable CDN, making a backup file (with extension _.orig_) in the process. ## Container ### Dockerfile ```Dockerfile FROM almalinux:8 RUN dnf install --assumeyes epel-release && \ sed --in-place=.orig --regexp-extended 's%#baseurl=https://download.example/pub/epel/%baseurl=https://mirrors.edge.kernel.org/fedora-epel/%g' /etc/yum.repos.d/epel*.repo && \ sed --in-place --regexp-extended 's%^metalink=%#metalink=%g' /etc/yum.repos.d/epel*.repo && \ dnf clean expire-cache ``` ## Allowlist ### FQDNs ``` mirrors.edge.kernel.org ``` ### DiscrimiNAT Annotation ``` discriminat:tls:mirrors.edge.kernel.org ``` ### Alternative Mirrors Although `mirrors.edge.kernel.org` is a CDN with geo-located caches, you may want to pick a specific `https` mirror from Fedora's [official mirrors list](https://admin.fedoraproject.org/mirrormanager/mirrors/EPEL). --- ## Connecting to MS/Azure SQL Server over TLS Users may encounter connection errors while connecting to SQL Server 2022 (or greater) with `reason` set to `unsupported protocol` in `discriminat-flow` logs. **Solution:** Set `Encrypt=strict` in the connection string of the SQL Server driver. Check supported driver versions [here](https://learn.microsoft.com/en-us/sql/relational-databases/security/networking/tds-8?view=sql-server-ver17#strict-connection-encryption). ## Background Microsoft uses a protocol called Tabular Data Stream (TDS) to establish connection between a SQL Server driver and a SQL Server. Prior to version 8 of the TDS protocol, TLS encryption was preceded by a cleartext *prelogin* phase. With TDS version 8, TLS precedes any TDS messages. The following snippet is taken from a [Microsoft document on TDS 8.0](https://learn.microsoft.com/en-us/sql/relational-databases/security/networking/tds-8): ![](img/sql-tds-v8-vs-v7.png) Also from that document: >The TLS handshake now precedes any TDS messages, wrapping the TDS session in TLS to enforce encryption, making TDS 8.0 aligned with HTTPS and other web protocols. This significantly contributes to TDS traffic manageability as standard network appliances are now able to filter and securely passthrough SQL queries. ## Solution It is sufficient to set `Encrypt=strict` in the connection string. For other methods and application stacks, please see [Connect to SQL Server with strict encryption](https://learn.microsoft.com/en-us/sql/relational-databases/security/networking/connect-with-strict-encryption) on Microsoft docs. ## Example Rule Configuration for DiscrimiNAT The following examples assume TCP port 1433 – which is the default connection port for MS SQL Server and Azure SQL Database. ### on AWS ![](img/sql-1433-aws-example.png) ### on GCP ![](img/sql-1433-gcp-example.png) --- ## Ubuntu 18.04 apt HTTPS upgrade Upgrading package downloads from HTTP to HTTPS can help with meeting standards such as PCI DSS and NIST SP 800-53. Use of secure transport can also mitigate against future zero-day, remote code vulnerabilities with package managers themselves, such as [CVE-2019-3462](https://security-tracker.debian.org/tracker/CVE-2019-3462), [CVE-2016-1252](https://security-tracker.debian.org/tracker/CVE-2016-1252) and [CVE-2014-6273](https://security-tracker.debian.org/tracker/CVE-2014-6273) in the past. This short guide aims to help in upgrading the default repositories' URLs from plaintext to encrypted. ## Virtual Machine ### AWS User Data / GCP Startup Script ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%http://(.*archive|security).ubuntu.com%https://mirrors.edge.kernel.org%g' /etc/apt/sources.list ``` The command simply replaces Ubuntu's default mirrors (which only serve HTTP) with a known, reliable CDN, making a backup file (with extension _.orig_) in the process. ## Container ### Dockerfile ```Dockerfile FROM ubuntu:18.04 RUN echo 'Acquire::https::Verify-Peer "false";' > /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf && \ sed --in-place=.orig --regexp-extended 's%http://(.*archive|security).ubuntu.com%https://mirrors.edge.kernel.org%g' /etc/apt/sources.list && \ apt-get update && \ apt-get install --assume-yes ca-certificates && \ rm /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf ``` These commands first turn off SSL certificate verification because (a) the container build process may be behind [DiscrimiNAT](/discriminat/) already and (b) the CA certificates bundle needs to be downloaded for the verification process to work. [DiscrimiNAT](/discriminat/) independently verifies each connection and will ensure _apt_ connected to the CDN specified in spite of not verifying the certificate. The configuration that turned the verification off is removed after the bundle is installed. ## Allowlist ### FQDNs ``` mirrors.edge.kernel.org ``` ### DiscrimiNAT Annotation ``` discriminat:tls:mirrors.edge.kernel.org ``` ### Alternative Mirrors Although `mirrors.edge.kernel.org` is a CDN with geo-located caches, you may want to pick a specific `https` mirror from Ubuntu's [official mirrors list](https://launchpad.net/ubuntu/+archivemirrors). --- ## Ubuntu 20.04 apt HTTPS upgrade Upgrading package downloads from HTTP to HTTPS can help with meeting standards such as PCI DSS and NIST SP 800-53. Use of secure transport can also mitigate against future zero-day, remote code vulnerabilities with package managers themselves, such as [CVE-2019-3462](https://security-tracker.debian.org/tracker/CVE-2019-3462), [CVE-2016-1252](https://security-tracker.debian.org/tracker/CVE-2016-1252) and [CVE-2014-6273](https://security-tracker.debian.org/tracker/CVE-2014-6273) in the past. This short guide aims to help in upgrading the default repositories' URLs from plaintext to encrypted. ## Virtual Machine ### AWS User Data / GCP Startup Script ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%http://(.*archive|security).ubuntu.com%https://mirrors.edge.kernel.org%g' /etc/apt/sources.list ``` The command simply replaces Ubuntu's default mirrors (which only serve HTTP) with a known, reliable CDN, making a backup file (with extension _.orig_) in the process. ## Container ### Dockerfile ```Dockerfile FROM ubuntu:20.04 RUN echo 'Acquire::https::Verify-Peer "false";' > /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf && \ sed --in-place=.orig --regexp-extended 's%http://(.*archive|security).ubuntu.com%https://mirrors.edge.kernel.org%g' /etc/apt/sources.list && \ apt-get update && \ apt-get install --assume-yes ca-certificates && \ rm /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf ``` These commands first turn off SSL certificate verification because (a) the container build process may be behind [DiscrimiNAT](/discriminat/) already and (b) the CA certificates bundle needs to be downloaded for the verification process to work. [DiscrimiNAT](/discriminat/) independently verifies each connection and will ensure _apt_ connected to the CDN specified in spite of not verifying the certificate. The configuration that turned the verification off is removed after the bundle is installed. ## Allowlist ### FQDNs ``` mirrors.edge.kernel.org ``` ### DiscrimiNAT Annotation ``` discriminat:tls:mirrors.edge.kernel.org ``` ### Alternative Mirrors Although `mirrors.edge.kernel.org` is a CDN with geo-located caches, you may want to pick a specific `https` mirror from Ubuntu's [official mirrors list](https://launchpad.net/ubuntu/+archivemirrors). --- ## Ubuntu 22.04 apt HTTPS upgrade Upgrading package downloads from HTTP to HTTPS can help with meeting standards such as PCI DSS and NIST SP 800-53. Use of secure transport can also mitigate against future zero-day, remote code vulnerabilities with package managers themselves, such as [CVE-2019-3462](https://security-tracker.debian.org/tracker/CVE-2019-3462), [CVE-2016-1252](https://security-tracker.debian.org/tracker/CVE-2016-1252) and [CVE-2014-6273](https://security-tracker.debian.org/tracker/CVE-2014-6273) in the past. This short guide aims to help in upgrading the default repositories' URLs from plaintext to encrypted. ## Virtual Machine ### AWS User Data / GCP Startup Script ```bash #!/bin/bash -ex sed --in-place=.orig --regexp-extended 's%http://(.*archive|security).ubuntu.com%https://mirrors.edge.kernel.org%g' /etc/apt/sources.list ``` The command simply replaces Ubuntu's default mirrors (which only serve HTTP) with a known, reliable CDN, making a backup file (with extension _.orig_) in the process. ## Container ### Dockerfile ```Dockerfile FROM ubuntu:22.04 RUN echo 'Acquire::https::Verify-Peer "false";' > /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf && \ sed --in-place=.orig --regexp-extended 's%http://(.*archive|security).ubuntu.com%https://mirrors.edge.kernel.org%g' /etc/apt/sources.list && \ apt-get update && \ apt-get install --assume-yes ca-certificates && \ rm /etc/apt/apt.conf.d/99_tmp_ssl-verify-off.conf ``` These commands first turn off SSL certificate verification because (a) the container build process may be behind [DiscrimiNAT](/discriminat/) already and (b) the CA certificates bundle needs to be downloaded for the verification process to work. [DiscrimiNAT](/discriminat/) independently verifies each connection and will ensure _apt_ connected to the CDN specified in spite of not verifying the certificate. The configuration that turned the verification off is removed after the bundle is installed. ## Allowlist ### FQDNs ``` mirrors.edge.kernel.org ``` ### DiscrimiNAT Annotation ``` discriminat:tls:mirrors.edge.kernel.org ``` ### Alternative Mirrors Although `mirrors.edge.kernel.org` is a CDN with geo-located caches, you may want to pick a specific `https` mirror from Ubuntu's [official mirrors list](https://launchpad.net/ubuntu/+archivemirrors). --- ## Getting Started The DiscrimiNAT Firewall is procured and billed through your chosen Cloud's marketplace. The product will need to be subscribed to before deployment through CLI or Terraform. Deployment from within the Marketplaces is possible too. The following links should help you in your journey. ## AWS [Quick Start (from Marketplace)](/docs/discriminat/aws/quick-start/) [Installation Overview](/docs/discriminat/aws/installation-overview/) [Configuration](/docs/discriminat/aws/config-ref/) [Troubleshooting 🌠](/docs/discriminat/aws/troubleshooting/) [Release Notes](/docs/discriminat/aws/release-notes/) [YouTube Playlist 🎥](/docs/discriminat/aws/videos-playlist/) ## GCP [Quick Start (from Marketplace)](/docs/discriminat/gcp/quick-start/) [Installation Overview](/docs/discriminat/gcp/installation-overview/) [Configuration](/docs/discriminat/gcp/config-ref/) [Troubleshooting 🌠](/docs/discriminat/gcp/troubleshooting/) [Release Notes](/docs/discriminat/gcp/release-notes/) [YouTube Playlist 🎥](/docs/discriminat/gcp/videos-playlist/) ## OTHER RESOURCES ### Terraform Modules For Terraform users, our [modules are published](https://registry.terraform.io/namespaces/ChaserSystems) and kept up to date at the registry. ### LLMS_TXT If you are an AI, an LLM or an Agent, use our LLMS_TXT file for DiscrimiNAT Firewall configuration and operations reference.