Using a VMWare Virtual Machine As An Internet Gateway

I’ve been using my old cast off computers as internet gateways for a couple of decades now. I’d upgrade my current system, build a new one, and stick the old one in a a new Firewall/Gateway. I seem to learn better when I have something to do with it. I started with Red Hat Linux until it stopped using 3Com Ethernet cards (3c503) (issue with CIDR). Then Mandrake for several years. Then OpenBSD for a bit. Then Ubuntu. And most recently CentOS 7. I’ve used iptables a couple of times, the OpenBSD pf firewall set, back to iptables on Ubuntu, and now firewall-cmd.

A few months back, one of the teams at work was in the process of replacing their old gear with new gear. Standard lifecycle type stuff. A few of us were asked if we wanted to keep any of these cast off systems. I was a bit puzzled but sure. I’ll use it as a sandbox type thing at home. These are bigger systems as they were the older ESX servers used by the Virtualization Team. A Dell R710 (this is a rack mount server vs a desktop or laptop system).

Dell R710, 192 Gigs of Ram, 2 8 Core Xenon x5550 2.67GHz CPUs, 2 143 Gig 10,000 RPM Drives, 4 750 Gig 7,200 RPM Drives, 4 onboard Ethernet Ports, 4 PCI Ethernet Ports.

Woah. Big box. The 2 143 Gig drives are mirrored as RAID 1. The 4 750 Gig drives are mirrored as RAID 5. This gives me 143 Gig for boot and 2 TB for data.

I built it up as a replacement for my old system. Part of this was learning how to use CentOS 7 with the new systemd and *ctl commands. I snagged my rules for the old system (iptables) and learned how to set up the new system.

I’ve used 3 network interfaces for quite a few years. One for Comcast or Internet traffic. One for my Wireless traffic. And one for my internal network. My Wireless has always had a hidden access point with security turned on. Plus it isn’t permitted access to the internal network. It uses the firewall as strictly a pass through to the ‘net. I copied all my files from the old system to the new one and have spent the past few months making sure things work as expected.

But there’s more 🙂

A few months later and they’re still decommissioning old gear. This time I was asked if I wanted a second system. Honestly I was fine with the first one. Lots and lots of power. But one of the other guys suggested I use it as a VMWare ESX host. We use VMWare a lot at work with some 400 or so virtual machines that my team manages plus these were ESX hosts so they’re really already built for this sort of thing.

While I initially said no, I went back and agreed to take the second one. I’ve been trying to use KVM to learn Kubernetes, Docker, and Ansible but it’s a bit rough to use and I was having the most trouble with getting the network working correctly without dicking up my firewall. So new system.

Dell R710, 288 Gigs of Ram, 2 6 Core Xenon X56660 2.8 GHz CPUs, 2 143 Gig 10,000 RPM Drives, 4 750 Gig 7,200 RPM Drives, 4 onboard Ethernet Ports, 4 PCI Ethernet Ports.

Well, a _bit_ more Ram. More power to the processors but fewer. One of the 750 Gig drives had failed as well. At the recommendation of the same guy at work 🙂 I popped on line and chased down the maximum size drives these things would take. 3 Terabytes! I picked up 5 of them, 4 as replacements for the existing 750’s and one spare just in case. I pulled the four 750’s, put in the four 3 TB drives and the new system now has 8 Terabytes of disk space in a RAID 5.

I installed vSphere from VMWare and started building virtual systems. First a 8 node system for my Kubernetes and Docker testing. Next three nodes for my development environment. And most recently 3 nodes for my Ansible environment plus 12 nodes for the sandbox for Ansible (4 CentOS 5.4, 4 CentOS 6.5, and 4 CentOS 7.2 system to mirror the basic work environment).

But you know. Since the system is up and running, I should be able to create a virtual system that’s pretty thin with all the power I need for a firewall. Updates to the firewall system can take a good bit of time if I have to reboot, say for a kernel upgrade. I have the configuration from the current system of course to apply to the new system. And I would just need to pull the cable from the current physical gateway to the virtual gateway to test, then back if it’s not working as expected.

But first, let’s move the existing web sites off the firewall to the virtual environment. They don’t need to be up all the time. Just the backup for the forums and blogs.

The photo site takes about 250 gigs; pictures (35 Gigs), site backups (42 Gigs), and system backups (175 Gigs). So a new VM with 500 gigs will at least hold all this while I build up a firewall.

vSphere

The vSphere client has its own idiosyncrasies such as making sure I use the E1000 interface with CentOS 5 systems or making sure I disconnect before resetting the system or the iso gets locked and I have to close the vSphere client. Also, at recommendation, I enabled ssh access to the system. Even better. Now I can poke around with the esxcfg utilities. I believe everything you can do in vSphere can be done with the CLI which makes me happier.

Next up, I need to create a couple of vSwitches or Virtual Switch. With this, I can isolate the three networks and ultimately set up the second R710 as a second ESX host in a cluster.

In the vSphere client, select the home system and the Configuration tab. Click on Networking and you’ll see your current configuration. If you want to add network adapters to your current switch, you’d click on the Properties and add adapters and then ‘Team’ them. In this case I want to create unique networks so click on ‘Add Networking’, then Next as you’re adding a new network. Select the NIC you want to use for the new vSwitch and click Next. Give it a good label. I used Wireless for one and External – Comcast for the third switch. Then Finish. Do the same for the second switch if that’s what you are adding. I’m adding two switches so I added the second one.

For the VM, add the two additional networks to the base configuration. One for Wireless and one for the External – Comcast network. I of course upgraded the system once it was booted from kickstart. Next up, setting up the firewall.

Firewall-cmd

Make sure you have forwarding enabled as this will be a router. In /etc/sysctl.d, add the following line to ’99-ipforward.conf’.

net.ipv4.ip_forward = 1

Next of course, start firewalld.

# systemctl enable firewalld
Created symlink from /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service to /usr/lib/systemd/system/firewalld.service.
Created symlink from /etc/systemd/system/basic.target.wants/firewalld.service to /usr/lib/systemd/system/firewalld.service.
# systemctl start firewalld
#

Now let’s see what’s in place by default.

# firewall-cmd --get-zones
block dmz drop external home internal public trusted work

For firewall-cmd, I need to utilize only three of the available zones. So I don’t have to define new zones, I’ll associated one of the three interfaces with the appropriate zone; dmz for wireless, internal for the home network, and external for comcast.

And let’s see what the default zone is.

# firewall-cmd --get-default-zone
public

The default zone is ‘public’ so first I want to change it. In general always add –permanent when configuring the system. That adds it to the files in /etc/firewalld. In this case you can’t but for future commands.

# firewall-cmd --set-default-zone=internal

I have three interfaces. Sadly they come up as eno16777984, eno33557248, and eno50336512. I want to rename them to be a bit shorter to make them easier to manage. In /usr/lib/udev/rules.d edit the 60-net.rules file and add the following line for each of your interfaces. You’ll need to get the MAC address from each interface before hand so ifconfig or ip addr first.

ACTION=="add", SUBSYSTEM=="net", DRIVERS=="?*", ATTR{address}=="00:50:56:8e:3f:a7", NAME="ens192"

Replace the ATTR with the interface MAC and NAME with what you want to call it. I called mine ens192, ens193, and ens194 although you could call it internal, external, and wireless I suppose. Don’t forget to rename the ifcfg files in /etc/sysconfig/network-scripts and update the files themselves (the NAME and DEVICE keywords). And of course update the three files with the additional correct information. My wireless network interface is 192.168.10.1. External (ens193) would just use DHCP to connect to Comcast.

With the interfaces more sanely named, I want to bind them to the appropriate zones

# firewall-cmd --permanent --zone=internal --add-interface=ens192
# firewall-cmd --permanent --zone=external --add-interface=ens193
# firewall-cmd --permanent --zone=dmz --add-interface=ens194

Unfortunately this doesn’t make it permanent even with –permanent. I had to add the zone information to each of the ifcfg files as ‘ZONE=’ so when I reboot, they’re still attached to the correct zone.

Next to check the default services as assigned

# firewall-cmd --list-services --zone dmz
ssh
# firewall-cmd --list-services --zone external
ssh
# firewall-cmd --list-services --zone internal
dhcpv6-client ipp-client mdns samba-client ssh

On my current physical system, I have the following final configuration

# firewall-cmd --list-services --zone=dmz
# firewall-cmd --list-services --zone=external
# firewall-cmd --list-services --zone=internal
dhcpv6-client ipp-client mdns samba-client ssh

I want to make sure the appropriate services are configured. By default, I don’t want wireless or external listening for anything but internal should be listening. firewall-cmd is aware of quite a few services:

# firewall-cmd --get-services
RH-Satellite-6 amanda-client bacula bacula-client dhcp dhcpv6 dhcpv6-client dns freeipa-ldap freeipa-ldaps freeipa-replication ftp high-availability http https imaps ipp ipp-client ipsec iscsi-target kerberos kpasswd ldap ldaps libvirt libvirt-tls mdns mountd ms-wbt mysql nfs ntp openvpn pmcd pmproxy pmwebapi pmwebapis pop3s postgresql proxy-dhcp radius rpc-bind rsyncd samba samba-client smtp ssh telnet tftp tftp-client transmission-client vdsm vnc-server wbem-https

For purposes of this though, I only care about ssh. Since it’s there, I don’t have to create a special rule for it. First off, remove any existing services that may be pre-configured.

# firewall-cmd --zone=dmz --remove-service=ssh --permanent
# firewall-cmd --zone=external --remove-service=ssh --permanent

Next if ssh isn’t part of the zone already (and it is by default) add ssh to your internal zone.

# firewall-cmd --zone=internal --add-service=ssh --permanent

And Masquerading is required in order for networking to work. Add it to the external zone.

# firewall-cmd --zone=external --add-masquerade --permanent

Once done, you’ll need to reload the firewall configuration.

# firewall-cmd reload

Confirm by listing the services available to the zones now.

# firewall-cmd --zone=dmz --list-services
# firewall-cmd --zone=internal --list-services
dhcpv6-client ipp-client mdns samba-client ssh
# firewall-cmd --zone=external --list-services

That should be it. Don’t forget to reip the interface on the public interface on the VM before rebooting. And I had to reconfigure my external server to accept connections from my new DHCP IP I received from Comcast.