Setting up a VPN (or Virtual Private Networking) does not have to be difficult. In fact using Apple’s OSX, it can be down right easy. VPNs should never be taken lightly. IT is the door to your protected network. If they’re not set up correctly it could leave you and your network assets at risk. There are two main types of VPNs that on can implement on OSX server, PPTP and L2TP. There are pluses and minuses to each and depending on how you/what you’re looking to support will determine which implementation you will use. It’s interesting to note that neither of the two mentioned VPN protocols provide encryption. They are considered tunneling protocols and thus need to rely on other methods to provide the encryption.
PPTP – Is the older of the two most popular tunneling protocols. It relies on either on either MSCHAP-v2 or EAP-TLS for authentication. Additionally, Apple has built in support for both Kerberos authentication and RADIUS. PPTP uses Microsoft Point-to-Point Encryption (MPPE) to encrypt the data that passes through the tunnel. Originally MPPE was only offered with support for a 40bit key. It was later expanded to a 128bit key!
L2TP – Is the newer comer, its latest version (RFC 3931) having been published in 2005. L2TPv3 makes use of IPSec for securing the connection. This is preformed through the use of pre-shared secrets, symmetrical keys or digital certificates. As with any secure connection the hardest part of maintaining the SA is the managing of the keys used. However, once the first connection is made and security confirmed. The passing of pre-shared secrets, keys or digital certificates becomes trivial.
NOTE: It should be noted that that PPTP and L2TP are not the only players in the VPN game. There is two other methods as well, PPP Over SSL and PPP Over SSH.
Configuring your server
Open Server Admin and select the host you wish to administer. Select VPN and click save.
Figure 1. Services Activation Pane
Turn down the triangle to reveal the VPN configuration pane.
Configuring L2TP Settings
It is as this point that you can decide which tunneling protocol you’re going to support. Setting up the server is pretty simple. Select the check box to enable L2TP. You need to allocate an IP range (remember this is still a point to point connection). Under PPP Authentication select if you want to use the built-in Directory Service plug-ins for user ID and password lookups (you can also chose between MS-CHAPv2 or Kerberos) or point the VPN service to look at a RADIUS server for authentication lookups. Lastly, you need to specify whether you want to use a pre-Share secret or a digital certificate for IPSec Authentication.
Figure 2. L2TP Configuration Pane
Configuring PPTP Settings
If you need to support older VPN clients PPTP may be a better choice for you. Many experts still contend the PPTP is vulnerable to compromise but with anything else strong passwords make for strong security. Depending on the client that you need to support you may need to allow 40bit encryption keys. This should be avoided if at all possible as 40 bit keys are easily cracked.
Figure 3. PPTP Configuration Pane
Configuring Client Information Settings
Lastly, you need to “tell” your clients about the network they have just connected to. This could be done on the client side, and may be desirable is some situations. In a lot of ways this is very similar to setting up a DHCP server.
NOTE: If you are running DHCP on the same subnet, make sure that the allocated IP address ranges do not conflict!
Figure 4. Client Information Settings
NOTE: If no information is added to Network Routing Definitions all traffic is routed through the VPN connection. This may not always be desirable. If bandwidth is a concern, define a network that is private and force all non-private traffic over the client’s Internet connection.
Ports on your Firewall
One thing you must make sure to perform before your VPN will work is to open the required ports on your firewall. Both protocols make use of different ports can it can be confusing which ports are actually needed. Not just on the host (if you’re running IPFW on the host) but on the network perimeter. So what ports are used?
500 UDP ISAKMP/IKE
1701 UDP L2TP
1723 TCP PPTP
4500 UDP IKE NAT Traversal *
* NOTE: Port 4500 is also used for Back to My Mac (MobileMe, Mac OS X 10.5 or later)
In Mac OSX Server 10.3 the VPN service uses the following:
1. PPTP uses the IP-GRE protocol (IP protocol 47).
2. L2TP/IPsec uses the IP-ESP protocol (IP protocol 50, ESP).
Resources:
http://manuals.info.apple.com/en_US/Network_Services_Admin_v10.5.pdf
http://support.apple.com/kb/TS1629



