Sadiq's Knowledge Vaults

Bastion

3 min read

Critical systems in the company like production databases, application servers, internal dashboards,… are kept inside Private networks (for more context see AWS Networking) but engineers still need to access them for troubleshooting, maintenance, deployments,…
But the problem is this instances

  1. Don’t have Public IP Addresses
  2. Can’t be reached directly over the internet
  3. Sit behind strict firewalls

One Solution is to Connect to VPN but there are some problems with it, like

  1. Can’t restrict access for specified people like, we cannot restrict the Database management team access for only databases, they can access application servers also
  2. It gives broad access and chances of getting hacked are more
  3. Harder to Audit employee per-session activity

The solution is using Bastion Host/Jump Host/Jump Box
It’s a Specially configure sever that acts as an secure gateway between a public network like internet and a private network. It has a public IP, Multi-Factor Auth, Single point of Access. It also provides secure connection for restricted services
How it works ?

  1. First we connect to the Bastion Host(SSH for Linux and mac and RDP for Windows)
  2. We verify ourselves to bastion host, whether it might be asymmetric key encryption or MFA or some other auth method
  3. The access level across resources is determined with server side rules or IAM policies
  4. Again with Bastion we connect to the resource we wanted to connect, this might be an another SSH hop or a local port forwarding tunnel

The commands to do it, as follows

  1. Directly connecting to the resource which is present in the private network by ignoring bastion in between(think of it’s like an login automatically, that doesn’t mean we can bypass bastion auth)
ssh -J user@bastion user@private-server
  1. Local port forwarding - This is used when we want to use a local app like database client or a browser to connect something in the private network, without installing that app on the bastion
    • With the below command we let the SSH listen on the local 3306 port and forward all the traffic that comes to the port then securely forward that traffic to private DB via SSH tunnel to the bastion
ssh -L 3306:private-db:3306 user@bastion
  1. Hoping through multiple bastions with one command, A public server can be protected by multiple bastions, rather than doing SSH into each other one by one, we can use this command to SSH into a private server which is has 2 bastions in between
ssh -J user@bastion1, user@bastion2 user@private-server

There are some alternatives to Bastion:

  1. AWS System Manager Session Manager - It let’s to connect to private instances with browser CLI, No Public IP needed
  2. IAP, Identity and Aware Proxy by Google - It adds lot of checks before giving access to the resources needed
  3. Teleport - Zero Trust Platform
  4. Strong DM - Zero Trust Platform

How Netflix uses Bastion in their Production?

  • If an engineer wants to troubleshoot a production server
  • First they need to authenticate themselves using Okta MFA - A Identity Platform
  • After they go into Bastion Hosts Authentication which are short-lived sessions
  • From this they get a personalized access to the services
  • And all the accessed services activity are pushed into a CloudWatch SSH Session Logs and this logs are backed up into a S3 Bucket which acts like a log storage

Graph View