VPN Basics

a Virtual Private Network is a technology that provides secure communication through an insecure and untrusted network (like the internet). It achieves this through authentication, encryption, compression and tunneling.

What is tunneling?

Tunneling is a technique that encapsulates packet headers and data within the payload of another protocol. This way the packet can traverse the network in a manor it would otherwise not be capable of traversing.

How is it done?

The most common ways of creating a VPN are IPsec and SSL/TLS.

IPsec

IPsec provides authentication,encryption and compression at the network level. IPsec is actually a suite of protocols developed in the mid 90's and supports both IPv4 and IPv6. However it is mandatory in IPv6 and optional in IPv4. To implement IPsec two new protocols were added: Authentication Header (AH) and Encapsulating Security Payload (ESP). Handshaking and exchanging session keys are done using the the Internet Key Exchange (IKE) protocol.
The Authentication Header is protocol number 51 and it authenticates both the header and the payload. The AH however does not use encryption so it is almost never used.
Encapsulating Security Payload is protocol number 50. It enables you to add a security policy to the packet and optionally encrypt it. The encryption is done in the kernlet via the cryptoAPI. When two machines are connected via the ESP protocol, a unique number identifies the connection. This number is called the SPI (Security Parameter Index). Each packet contains a sequence number and a checksum which is called the ICV (Integrity Check Value). The checksum is calculated using a secret key which is known only to these two machines.

IPsec has two modes: Transport and Tunneling. When creating a VPN, we use tunnel mode. This means that each IP packet is fully encapsulated in the new IPsec packet. The payload of this new packet is the original packet before it was encapsulated.

The problem with this model is that in networks where the peer is behind a NAT (Network Address Translation) device. Using a NAT is a common way of connecting machines that are not directly accessable to the outside world. The NAT is performed on a machine that does have access, this is usually a gateway. The NAT modifies the IP packet and as a result the peer rejects it because the signature is wrong. The solution is commonly known as NAT-T (Network Address Translation Traversal) and works by encapsulating the IPsec packet in UDP packets so that they'll be able to pass through the NAT routers without being dropped.

OpenSwan is an open-source project that provides an implementation of IPsec. The SWAN name comes from Secure Wide Area Network which is actually a trademark of RSA. OpenSwan supports Opportunistic Encryption (OE), which enables the creation of IPsec based VPNs by advertising and fetching public keys from a DNS server.

SSL/TLS

OpenVPN is a VPN solution based on SSL/TLS. It is simpler in comparision with IPsec and OpenVPN supports RSA authentication, Diffie-Hellman key agreement, integrity checks etc. When running in server mode, it can support up to 128 clients over the same port. You can setup your own Certificate Authority and generate certificates and keys for an OpenVPN server and multiple clients.

OpenVPN operates in user-space mode; this makes it easy to port to other operating systems.