Deny Page API Specifications

The Lenovo NetFilter deny page framework allows you to configure the location where your deny pages will reside, within your overall Lenovo NetFilter solution. This section outlines how to configure your deny pages to reside externally from the Lenovo NetFilter system, and what parameters are available to you when creating externally hosted deny pages.

Policy Service Configuration Settings

Deny Page Host

The string $HOST in the deny_page_http and deny_page_https configuration settings is replaced with the deny_page_host parameter. This allows multiple policy servers to point to either individual deny pages or to point to a common deny page.

The deny_page_host needs to be defined before the deny_page_http and deny_page_https settings are defined.

The value of $HOSTNAME will use the local machine's hostname value. If this setting is not set, it will default to the $HOSTNAME value.

It is also possible to automatically retrieve the IP address of an Ethernet interface. Entering a value of $IPADDR_ followed by a device name will use the IPv4 address for that interface at the time when the configuration is loaded. Some valid examples of using setting include:

·       $IPADDR_eth0

·       $IPADDR_en0

·       $IPADDR_wlan0

Note that an interface's address is read only when the configuration is loaded. This means the IP is only read at startup and when configuration is applied (i.e. remote admin command #9)

Format:        deny_page_host filter.company.org

deny_page_host $IPADDR_eth0

Deny Page for HTTP traffic

This defines the location of the deny page for http traffic. The following information is passed to this deny page as GET request parameters.

Available Variables for the Deny / Portal Pages

A Lenovo NetFilter Deny Page will be requested in the following format:

http://192.168.5.129:8080/webadmin/deny/index.php?dpid=1&dpruleid=3&cat=101&ttl=0&groupname=kkkkk&policyname=-&username=-&userip=127.0.0.1&connectionip=127.0.0.1&nsphostname=localhost.localdomain&protocol=squid25&dplanguage=-&url=http%3a%2f%2fwww%2estbernard-school%2ecom%2fmoodle%2fenrol%2fAlibaba%2falibaba%2falibaba%2falibaba%2falibaba%2ecom%2fLogin%2ehtm

The Lenovo NetFilter auth redirect portal page will be requested i the following format:

http://192.168.5.129/webadmin/authportal/login.php?cat=23&dpid=portal&rememberme=0&rememberme_expire=0&ttl=-200&groupname=default&policyname=default&username=%2d&userip=192.168.4.100&connectionip=127.0.0.1&nsphostname=localhost.localdomain&protocol=squid25&dplanguage=-&error=0&url=http%3a%2f%2fwww%2esex%2ecom

The requests include values for certain variables, such as url, dpid. These variables may be used in the deny page managed/created by the webadmin users. However, they are tweaked/extended for flexible usage.

The WebAdmin users can use the tools, such as templates from the tinymce, to apply these variable values. For advanced user and other purposes, knowing available variables are helpful.

WebAdmin User Interfaces

• Portal page editor under Policies > Groups > Authentication Redirect

• Deny/Portal Page content editor under Tools > Deny Pages

Web Requests

• Deny and portal Page Requests can be made in similar format with a similar dpid CGI argument:

• http://webadminserver/deny/index.php?dpid=XX

• http://webadminserver/authportal/login.php?dpid=XX

• The main page content for both types are saved in webadmin/config/denypages.php

Available Variables for Deny Pages

See 'Available Variables for Deny Page' for more information.

Deny Page for HTTPS traffic

Secure webpages operate differently than unsecured webpages. When a browser makes a connection to a secured webpage, the header requests contains nothing but a request to make a connection. Then a secure connection is created between the browser and the webpage. Any information transferred after the connection cannot be viewed, therefore it is not possible to see what file the browser is requesting. The Lenovo NetFilter is only able to intelligently block a secure request on the first connection to the webserver. Since the connection types are different, the only kind of block page that can be displayed is another secured block page. Unfortunately, no specific information about the type of block can be delivered. The following parameter tells the NSD where to redirect the connection.

Format:

deny_page_https $HOST:443

deny_page_https https://$HOST:443

Maximum length of the request URL to be passed to the deny page after encoding.

This setting allows you to adjust and limit the length of the URL to pass to the deny page. Some http surrogate devices can only handle a limited size of URL. If this is the case, the request_url can become up to 3 times larger due to character encoding. This setting should be set to 1/3 - the size of the deny_page_http{s} settings. By default this is set to 1024 or 1k.

Format:

deny_page_request_url_length 1024

Send full information about request processing to the deny page

With this flag set the Policy Server sends a lot of information in CGI parameters to the deny page, including Client name, IP address, Policy name, etc. It allows for an explanation of the deny reason with all details but can be considered insecure. Set it to off if you want to hide information that is not essential for the deny page.

Default:

deny_page_send_full_info yes

Format:

deny_page_send_full_info {yes|no}

Automatic loading of all local IP Addresses into the Filter Bypass List

This will automatically load all local IP Addresses into the Filter Bypass List so that the local server can automatically serve deny pages. This feature should be disabled on production systems with the Filter Bypass List properly populated. However, on systems with DHCP or on some systems this feature may be required for proper operation.

Format: filterbypasslist_ipscan_interval_secs SECONDS

Default: filterbypasslist_ipscan_interval_secs 1 hour

Deny Page Request URL Length

The maximum length of the request URL to be passed to the deny page after encoding is set using the deny page request URL length setting. This setting allows you to limit the length of the URL to pass to the deny page. Some HTTP surrogate devices can only handle a limited size of URL. If this is the case, the request_url can become up to 3 times larger due to character encoding. This setting should be set to 1/3 - the size of the deny_page_http{s} settings. By default, this is set to 1024 or 1k. These settings apply to both HTTP and HTTPS deny pages.

Format:

deny_page_request_url_length 1024

Up2Date Service for Deny Pages

Deny pages can be synchronized across multiple remote deny page hosts using the Up2Date service on each host. This downloads the denypages.php file from the central WebAdmin server. Under Administration > Services, click on your remote host to open the Server Management window. Click the Enabled link and then click Start.

Up2Date Service for Category Names

To convert the comma separated list of categories from numbers to names, you will need to synchronize categories.php from your central WebAdmin server using the Up2Date service on each remote deny page host.

Understanding a Dedicated Deny Page Host

Overview

Hosting deny pages on a dedicated web server allows for delivery of deny pages to all end users without requiring any open ports on a Lenovo NetFilter WebAdmin server. This means that the web port (80, 8080, 443) can be closed and the WebAdmin/HTTPD services can be stopped on any publicly accessible Lenovo NetFilter system.

Reasons for Deployment

There are two reasons why you might want to deploy a Dedicated Deny Page Host.

·       To reduce load on the WebAdmin server, host the deny pages on a dedicated webserver.

·       To serve deny pages to filtered client workstations while restricting their access to the WebAdmin interface.

It is possible to use a Policy Server as a deny page server. To configure this you must enable the Up2Date service on the Policy Servers hosting and serving deny pages.

Redirecting Deny Pages

This procedure illustrates how to redirect to a dedicated webserver. Please note that you can also use this procedure to redirect to other locations or websites.

1.    Go to Tools > Deny Pages and click the Rules tab.

2.    Click the ‘Default Deny Page’ link.

3.    Click ‘Redirect to URL’. The page changes to display a field for the name of the dedicated webserver.

4.    Enter the name of the dedicated webserver (the URL must begin with http://) and click Submit.

5.    The ‘Deny page has been successfully saved’ message displays.

Deploying Dedicated Webserver in a DMZ

To serve deny pages to filtered client workstations while restricting their access to the WebAdmin interface; you can use a DMZ configuration.

Both internal LAN users as well as external WAN users can access deny page content served from the DMZ System.

About Deny Page Sizing

One important aspect of sizing the Lenovo NetFilter content filtering solution is the sizing of the deny page server and network resources required to serve deny pages. The following calculations can be made to determine the appropriate sizing.

·       Denying adult content generally will deny 1% of total traffic

·       Generally, a single user generates 2000 transactions per day

·       The average size of a deny page with images and graphic resources is around 20 Kbyte

Using the above you can calculate both the network resources and server resources required to serve deny pages. Remember, changing the Categories or types of content you are denying will change your deny page requirements.

Network Resource Requirements Example 1

Example: 500,000 users – Amount of network bandwidth on average required to serve a deny page.

·       500,000 users * 2,000 transactions per day = 1,000,000,000 transactions per day

·       1% deny page hit rate * 1,000,000,000 transactions per day = 10,000,000 deny page hits per day

·       10,000,000 deny page hits per day = 116 deny page hits per second

·       116 deny page hits per second * 20Kbyte deny page = 2.32 Mbyte/second

·       2.32 Mbyte/second = 19Mbit/second

Remember, changing the Categories or types of content you are denying will change your deny page requirements.

Network Resource Requirements Example 2

Example: 2Gbps of Total Internet Traffic – Amount of network bandwidth on average required to serve a deny page.

·       2Gbps of Total Internet Traffic

·       Estimate of 30% outgoing/upstream traffic of 600Mbps

·       Given, 1Gbps of upstream/uplink traffic generates around 32,000 transactions/second

·       600Mbps/1Gbps * 32,000 transaction/second = 19,200 transactions/second

·       1% deny page hit rate = 19,200 transaction/second * 1% = 192 deny pages/second

·       20Kbyte deny page = 2.84Mbyte/second = 30.72Mbits/second

Remember, it is always good practise to use the Lenovo NetFilter product to monitor your Network traffic prior to implementing the deny page sizing.  Your bandwidth and transactions/second along with the deny page size will change your sizing requirements.

Server Resource Requirements

If the Lenovo NetFilter deny page system is used for serving deny pages the appropriate number of servers must be used based on the deny page transaction rate. Lenovo NetFilter can handle approximately the following in the different versions:

The above tests used 100 clients running 1,000 to 10,000 requests in total to determine the average Transactions/second. Depending on your server requirements your results may differ. Refer to performance results below to run your own independent tests to confirm results on your hardware.

Performance Results

Intel(R) Xeon(R) Gold 5122 CPU @ 3.60GHz 8 core, 16 threads

# ab -n 10000 -c 100 'http://localhost/webadmin/deny/?dpid=1&cat=23,18'

This is ApacheBench, Version 2.3 <$Revision: 655654 $>

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)

Completed 1000 requests

Completed 2000 requests

Completed 3000 requests

Completed 4000 requests

Completed 5000 requests

Completed 6000 requests

Completed 7000 requests

Completed 8000 requests

Completed 9000 requests

Completed 10000 requests

Finished 10000 requests

Server Software:    Apache

Server Hostname:    localhost

Server Port:      80

Document Path:     /webadmin/deny/?dpid=1&cat=23,18

Document Length:    7650 bytes

Concurrency Level:   100

Time taken for tests:  3.440 seconds

Complete requests:   10000

Failed requests:    0

Write errors:      0

Total transferred:   79740000 bytes

HTML transferred:    76500000 bytes

Requests per second:  2906.84 [#/sec] (mean)

Time per request:    34.402 [ms] (mean)

Time per request:    0.344 [ms] (mean, across all concurrent requests)

Transfer rate:     22635.85 [Kbytes/sec] received

Connection Times (ms)

       min mean[+/-sd] median  max

Connect:    0  0  0.1   0    1

Processing:   5  34  2.1   34   43

Waiting:    4  34  2.1   34   43

Total:     5  34  2.0   34   43

Percentage of the requests served within a certain time (ms)

 50%   34

 66%   35

 75%   35

 80%   36

 90%   36

 95%   37

 98%   37

 99%   38

 100%   43 (longest request)

Dual Socket - Intel(R) Xeon(R) Gold 6136 CPU @ 3.00GHz 12 core, 24 threads

# ab -n 10000 -c 100 'http://localhost/webadmin/deny/?dpid=1&cat=23,18'

This is ApacheBench, Version 2.3 <$Revision: 655654 $>

Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/

Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)

Completed 1000 requests

Completed 2000 requests

Completed 3000 requests

Completed 4000 requests

Completed 5000 requests

Completed 6000 requests

Completed 7000 requests

Completed 8000 requests

Completed 9000 requests

Completed 10000 requests

Finished 10000 requests

Server Software:    Apache

Server Hostname:    localhost

Server Port:      80

Document Path:     /webadmin/deny/?dpid=1&cat=23,18

Document Length:    7650 bytes

Concurrency Level:   100

Time taken for tests:  3.518 seconds

Complete requests:   10000

Failed requests:    0

Write errors:      0

Total transferred:   79740000 bytes

HTML transferred:    76500000 bytes

Requests per second:  2842.86 [#/sec] (mean)

Time per request:    35.176 [ms] (mean)

Time per request:    0.352 [ms] (mean, across all concurrent requests)

Transfer rate:     22137.65 [Kbytes/sec] received

Connection Times (ms)

       min mean[+/-sd] median  max

Connect:    0  0  0.1   0    1

Processing:   4  35  5.4   33   87

Waiting:    3  35  5.4   33   87

Total:     4  35  5.4   33   87

Percentage of the requests served within a certain time (ms)

 50%   33

 66%   37

 75%   39

 80%   40

 90%   43

 95%   44

 98%   46

 99%   49

 100%   87 (longest request)