A question I get asked a lot by customers and resellers alike is how to provide web filtering on deployed customer networks. Although Peplink devices do have basic web filtering built into their firmware, this only filters on http traffic, and with more and more social media and video streaming services working exclusively on https its effectiveness in real world deployments is somewhat restricted.
If you would like to skip the next bit where I talk about the types of web filters that are out there I understand :). Follow this link to my suggestion for easy Peplink network web filtering.
How do web filters work?
Now there are loads of approaches to web filtering, for example:
- Real time traffic scanning for keywords and image scans
- Website databases (a categorized list of websites including those flagged for malware content / phishing)
- Manually managed domain whitelists and blacklists
- URL and content keyword scanning
How and where these methods are implemented can vary considerably. In educational / enterprise environments there tends to be a dedicated web filtering appliance. This will physically or logically sit in the routing path between client device browsers and the internet connectivity (acting either as an enforced proxy or as the internet gateway), and depending on its capability can do real time scanning of all internet access or will perform redirects (to a ‘you are not allowed’ page) based on domain/url/content keywords.
Deep Packet Inspection is awesome
By far the most comprehensive way to do web filtering on any network is with deep packet inspection in combination with SSL hand-off / interception. This is where all all web traffic is analysed at a content level, with the full knowledge of destination domain/IPs available. The SSL interception allows the filter to see inside of encrypted traffic too, and more advanced recent solutions will use a combination of all methods of web filtering on the same appliance using the visibility of traffic flowing across it that is available due to the DPI.
Commercial web filtering appliances aren’t cheap
All of this advanced web filtering capability comes at a cost though since not only is there a high hardware processing overhead to content scanning (especially when involving DPI & SSL), but the databases and intelligence used to spot sites that should be filtered have a high maintenance cost to the vendor/provider as well. An entry level commercial web filter then (like a smoothwall 210) can easily cost $1500 and at this price might only support 5-10Mbs of throughput. Larger appliances capable of supporting today’s typical internet connection bandwidth (and the large number of sessions we see on a typical network of any size) will run into many thousands of whatever currency is local to you.
DNS based web filters are a good middle ground
When you don’t have the budget kicking about for a full blown web filtering appliance another approach is DNS based web filtering. A DNS web filter sits either in your ISP network or on a 3rd party network outside of your environment and does all the things a DNS server should do – resolves web urls to IPs so your browser can request the content. The twist is that these DNS services have a backend database of domains/URLS that have been categorised and tagged when appropriate as sites that are full of nasties and malware.
How DNS web filters work
When your browser requests a url your device sends that request to your gateway/router which then forwards the DNS request on to the DNS based web filters servers. At this point the web filter service checks the requested domain against its database and any additional lists of blocked sites it might have available, and then if its allowed, responds with the correct IP as normal. If the site is not allowed, the service responds with one of its own IPs and the browser gets redirected to a ‘you are not allowed’ page.
The Limitations of DNS based web filters
Since DNS based filters work on the DNS requests from client devices, the easiest way to bypass them is to change the client configuration to either use alternative DNS servers (like Googles DNS) or by manually adding host file entries for the blocked sites you want to visit – since if URL resolution happens locally in the device’s host file a remote DNS server isn’t queried at all and can’t redirect the device browser.
Mitigating the Limitations
These limitations can be mitigated to a certain extent in corporate environments where device policies can restrict network configuration changes (and protect the host file) from users who are not device administrators. However a user that brings their own device into work (or school), connects it to the network and uses either their local host file for DNS resolution or any of the publicly available DNS servers will still be able to access the internet uninhibited. It is still possible to block the access of these clients by adding firewall rules that deny DNS requests (on port 53) to any unapproved internet DNS server (whilst allowing requests to the 3rd party DNS web filter service), but there is nothing much that can be easily done about local address resolution via a hosts file.
There are a number of technologies that are easily available that have been designed from the outset to bypass internet filtering and censorship attempts – and for good reason. Everything from TOR to public https proxies, to in browser VPN clients. Blocking these services either becomes a game of cat and mouse, with super geek level automated firewall ACL scripting, or requires a network infrastructure change where connected devices can only access the internet through a http proxy (so https can not be used), or its a commitment to a commercial solution that does https interception.
Open source & software only web filters
I would be amiss not to mention the vast swath of open source and software only solutions to web filtering. These broadly fall into the following two categories, client device software and server installed software. Some of the biggest names in this sector are DansGuardian, Untangle, PFSense, smoothwall, Net Nanny etc. When configured correctly these can be very effective (although since I’m a lazy admin I tend to stay away from client device software as there is too much management involved). I’m a big fan of Untangle with their web-filtering and Https inspector add-ons – if you haven’t checked it out then its well worth a look, as is PFSense.
Although I am a big fan of the opensource solutions available specifically – and software based web filters generally – its not always feasible to have another appliance or additional server hardware at a remote or client site for web filtering. When it does make sense to -for example when all web access traffic is being routed to a central location over SpeedFusion, I will happily install a software based solution (normally Untangle as this is really easy for my customers to self manage), although so far in my experience these larger customers tend to already have a commercial product in place, so its not something I tend to get involved in from a Peplink solution design perspective.
The Long and The Short of It
There is a lot to web content filtering – especially when you consider the challenges around blocking technologies that have been specifically designed to bypass them (like TOR). Any web filtering technology can be bypassed by someone with enough technical knowledge and time. Deciding what level of web content filtering / blocking you need by taking your user base demographic into account is key. If you are looking to apply web-filtering for a school or for an environment where children access the internet for example, every commercial and technical effort should be made to protect them, the same for government, healthcare and banking institutions where data security is key, but in all of these cases, web content filtering should only ever be one of many tools used to manage and control the flow and access to data (on or off the internet).
So, what is a nice easy way to add web filtering to a Peplink Network?
As described above – there are loads of different ways to add web filtering to a network, however my favourite low cost, quick way to do so is to use DNS based Web Filtering. Why? because its a low risk, low impact way to restrict easy access to most nasty websites and at its most basic is free to use. Its not infallible – no web filter is, but it is largely very successful.
This is what I do
- I use DynDNS for dynamic host name resolution and they have a great web filtering service called Internet Guide that is free for hosts using their service.
- If you have static IPs on your WAN, or you don’t want to use their service for Dynamic Host Resolution you can still buy into their service ($30/year) and add the ability to use static IPs ($5/month)
- I make sure my WAN ports are using dynamic DNS
- I set the DNS servers on my WANs to all use dyndns servers for DNS resolution (18.104.22.168 , 22.214.171.124)
- I then login to my control panel and configure what level of blocking I want at that site/device.
- Additionally I add a firewall rule to the Peplink device that blocks all outbound DNS requests from client devices – this means that they have to use the Peplink Router for DNS resolution.
This works great for me, as I can easily block not only a large percentage of web nastiness using the default settings but I can also block streaming web services like BBC iPlayer. youtube and even facebook if I need to when a remote business site has limited bandwidth.
I recommend you try it too. There are other similar services of course – like OpenDNS that are well worth considering also.