Need your iOS devops in the cloud? We can help.
Setup a VPN server with El Capitan server 10.11 | Macminicolo Blog - Tips, tutorials and reviews on running a Mac mini server

Setup a VPN server with El Capitan server 10.11

We have a lot of customers who use their Mac mini as a VPN server. This works great when you need an IP address in the US, or a secure internet connection on the road, or a number of other reasons. When Apple released Lion, they changed the setup a bit. This continued in Mountain Lion and Mavericks, Yosemite, and remains the case in El Capitan. By default, El Capitan Server VPN will distribute IP addresses in the same range the Mac itself uses. This doesn’t work well in a facility like Macminicolo where each Mac mini has a static WAN IP address.

We asked Rusty Ross (@ConsultantRR) to help us put together a tutorial that will help Macminicolo customers setup their Mac minis to serve as VPNs. He’s broken it down in a few parts so be sure to take the steps that are best for your situation:

PART I: VLAN and DNS
PART II: Internet Routing (OPTIONAL)
PART III: VPN
PART IV: Client Setup

If you are simply looking to enable VPN service on your OS X Server for secure connection(s) between your server and client(s), you can skip PART II. That’s right: you can jump straight from PART I to PART III. The procedures discussed in PART II are intended for those who are looking to route internet traffic from their VPN clients over the VPN and out to the internet via their server’s public internet connection at Macminicolo.
Also, it should be mentioned that server administration (particularly at the command line level) can be tricky. If you do proceed beyond this point, which shall be exclusively at your own risk, then please proceed carefully, and as always, don’t ever proceed without a backup of your server and other irreplaceable data.
Still here? Okay, let’s get started.


PART I: VLAN and DNS


First, let’s set up a VLAN.
In System Preferences, go to Network, and choose “Manage Virtual Interfaces…”

vpn1

Then choose “New VLAN…”

vpn2

Let’s just name our VLAN something like “LAN”, and all other defaults here should be fine:

vpn3

After pressing “Create”, you’ll see this:

vpn4

After pressing “Done”, you’ll be able to enter network info for your new VLAN. Make sure to choose “Manually” for “Configure IPv4”, and set the IP Address, Subnet Mask, and Router as shown below.
(Advanced: We’ll be using a 10.0.0.1 private IP for the server and 10.0.0.0/24 private network in this walkthrough, but note that the technique documented here will work with any private IP addressing scheme. To accomplish that, you’d substitute that alternate network info here, as well as a few other places further along in this walkthrough.)

vpn5

After pressing “Apply”, you should see an something like this, indicating that your newly-created VLAN is active:

vpn6

Nice work. Now, let’s get basic DNS up and running. Launch Server.app, and click on the “DNS” section of the sidebar, under “Advanced”:

vpn7

All DNS defaults in Server.app should be fine, so let’s switch DNS service on:

vpn8

Great. Now, once again, if you are NOT interested in routing public internet traffic from your VPN client(s) over the VPN and out to the internet via your server’s public internet connection at Macminicolo, you should SKIP from here to PART III.

PART II: Internet Routing (OPTIONAL)


So far, so good. Now things get a little trickier, as we need to dive into the command line a bit to get NAT and routing set up. First, we’ll need to edit two privileged text files, so we are going use a command line text editor called nano. Breathe easy, we've got each other's backs here, and we will take this step by step.

As we proceed, it is important to remember that in the command line, typos aren't the least bit welcome, and also, uppercase/lowercase needs to match exactly, so it is extremely important to enter text into Terminal.app exactly as it is described here.

One additional (but important!) point: The quotation marks used here in terminal commands are "straight" quotes. Some web browsers and text editors may automatically convert these marks to smart (curly) quotes, particularly when copying and pasting. It's important to use straight quotes when entering the commands from this tutorial in Terminal.app.

Okay? Let's forge ahead.
Launch Terminal.app:

vpn9

Inside the terminal window that appears, enter the following command (as a single line), and press return:
sudo nano /etc/pf.anchors/com.apple

vpn10

You’ll be prompted for your password, and if you’ve not used sudo on this Mac in the past, you may see a warning about using sudo, which is fine. Enter your password (you wont see the cursor move while you type your password) and press return:

vpn11

Upon doing this, you should expect to see the following text file open inside of nano, our friendly command line text editor:

vpn12

Okay, we are now going to add three custom lines to this document. Red arrows in the picture below indicate where these lines should go. To move the cursor into the correct place to add these lines, simply use the arrow keys on your keyboard.
The three lines you’ll be adding are:
nat-anchor "100.customNATRules/*"
rdr-anchor "100.customNATRules/*"
load anchor "100.customNATRules" from "/etc/pf.anchors/customNATRules"
vpn13

Great. Now we need to tell nano to save the changes you made to this text file. This will be a three step process, and we'll take it one step at a time.

First you'll press the “control” key and the “X” key simultaneously (that's right, “control-X”) to tell nano you are done editing. Upon doing so, you will see this:

vpn14

Now press the “Y” key to let nano know that, yes indeed, you do want to save changes:
vpn15

And finally, nano is already suggesting the proper location to save the file you edited, so simply press return to accept its suggestion:

vpn16

Well done! You are back where you started, having edited a text file in nano.
Since you are now a nano expert, let's use it once more, this time to create an entirely new text file. Start nano up again as follows:
sudo nano /etc/pf.anchors/customNATRules

vpn17

Upon entering the command above and pressing return, you will be reunited with your friend nano, now editing a new text file which, thus far, contains no text:

vpn18

We are simply going to add two lines of text here:
nat on en0 from 10.0.0.0/24 to any -> (en0)
pass from {lo0, 10.0.0.0/24} to any keep state
(Advanced: If you are using private IP addressing other than 10.0.0.0/24, you should customize these two lines to match your chosen network.)

vpn19

And now that these two lines have been added, well, you know the drill, the old three-step save: “control-X”, then “Y”, and then press return.
Again, that's “control-X”:

vpn20

Then “Y”:
vpn21

And then press return:

vpn22

Excellent. Though nano has served us well, we won't need to use it again during this tutorial.
Now, just a couple more commands in Terminal.app, and we’ll be done with the command line altogether.
Enter this command (as a single line) into your Terminal.app window and press return:
sudo launchctl load -w /System/Library/LaunchDaemons/com.apple.pfctl.plist

vpn23

And finally, enter this command (as a single line) into your Terminal.app window and press return:
echo 'net.inet.ip.forwarding=1' | sudo tee -a /etc/sysctl.conf
The Terminal should respond with “net.inet.ip.forwarding=1”, which is what we want.

vpn24

Well done. You can now close your Terminal.app window entirely.
Okay, you have now set up NAT and routing for your private network. The last piece of the puzzle on the server will be to configure and enable VPN service.
Before you proceed, though: RESTART your server now. (We’ll wait…)
Now that you have restarted your server, let’s continue.

PART III: VPN


Whether or not you have just completed Part II or skipped to this point straight from Part I, rest assured that everyone is welcome here in Part III.
First, open Server.app and click on the “VPN” section of the sidebar:

vpn25

Several default settings here are already in place as we’d want them, so we’ll just edit a few.
Enter your Shared Secret as desired:

vpn26

Now press “Edit…” next to DNS Settings. You will likely see the Macminicolo DNS IP addresses here, which is NOT what we want in this particular place:

vpn27

Instead, change this to 10.0.0.1 as follows:

vpn28

(Advanced: If you are using an alternate private network, customize the above appropriately.)
Press “OK” and we’re back here:

vpn29

Now press “Edit…” next to Client Addresses, and enter settings as pictured below:

vpn30

(Advanced: If you are using an alternate private network, or have different needs in terms of address pool size, customize appropriately.)
Press “OK” and once again, we are back here:

vpn31

...and should be all set to go. Switch the VPN service on:

vpn32

Brilliant. Wait about 30 seconds for the VPN service to become fully active, and your Mac mini server should now be ready to serve VPN clients and (optionally, if you completed Part II) route their public internet traffic over its connection.

PART IV: Client Setup


Now that your server’s VPN is configured, enabled, and (optionally) ready to route public internet traffic for its clients, you may want a little guidance on how best to configure a client.
Let’s set up an El Capitan client as an example.
In System Preferences, go to Network, and press the “+” in the lower-lefthand corner:

vpn33

Choose “VPN”, make sure you are using “L2TP over IPSec”, and give your service a name:

vpn34

Press “Create”, and then make sure your new VPN is selected in the sidebar on the left, so you can edit its details on the right:

vpn35

As shown above, enter the IP address or DNS name for your server in the “Server Address” field. In the “Account Name” field, enter the username for the account on the server that you want to use to log in from the client.
Press “Authentication Settings…” and you’ll see this:

vpn36

Enter the Password for the account you just specified, and the Shared Secret exactly as you set it up on the server.
Press “OK”, and you are back to:

vpn37

Now press “Advanced…” and you should see this panel:

vpn38

If you chose to complete optional “Part II: Internet Routing” section earlier:
Then you should check the option to “Send all traffic over VPN connection” so that your client will, um, send all its traffic (including public internet-bound traffic) over the VPN when the VPN connection is active.
Otherwise, if you skipped the optional “Part II: Internet Routing” section, make sure to un-check “Send all traffic over VPN connection” (unlike the picture above).
Press “OK”, and you are back to:

vpn39

Press “Apply” to save changes.
And now your client should be ready to connect to your server’s VPN.
Just press “Connect” when you want make this happen.
Well done.

As I mentioned, this tutorial came from Rusty Ross (@ConsultantRR), a great hands-on consultant that works with a bunch of happy MMC customers on a wide range of topics, including setup, migration, troubleshooting, maintenance, networking, strategic planning, and creative thinking. He’s available for a quick-fix, a specific project, or a longer-term relationship. If you have questions, you can find us on Twitter @macminicolo. And if you’re looking for somewhere safe and connected to place a VPN server, check out our prices to host a Mac mini with us.