Restrict access with basic authentication

Protect an Apache site on TurboStack with HTTP basic authentication while letting trusted IP addresses through without a login, including the Varnish case.

In this article we tackle the problem of how to decide whether a visitor should or should not log in on a server with basic authentication enabled, based on their IP address.

So what is the result we want to achieve? We want to implement an .htpasswd file so visitors need a valid login, except when the request comes from a whitelisted IP address. In that case no login is asked and the visitor is taken straight to the site. Like a VIP that would skip the waiting queue for a club.

This method is used for Apache (apache2).

Method 1: Server without Varnish enabled

There is a difference when a server has or does not have Varnish enabled. For now we keep it simple and assume there is no interruption from any service like Varnish. In that case we use the following setup.

For best practice, put this code at the top of your .htaccess file:

AuthType Basic
AuthName "Restricted content"
AuthUserFile /var/www/prod/apache2/.htpasswd

# Whitelisted IPs are granted access without a login prompt
Require ip 203.0.113.10
# Only a person with valid credentials is let in
Require valid-user

Apache treats the two Require lines as "either one is enough": a request from 203.0.113.10 is allowed without a prompt, and every other request must supply a valid login.

Method 2: Server with Varnish enabled

For a server with Varnish enabled, a different approach is needed. All requests that go through Varnish pass the X-Forwarded-For header, but it may contain some tampered information about the visitor's IP. Because of this, the request for immediate access is denied and the visitor is asked to log in. To make sure this does not happen, we add a variable for the header that contains the whitelisted IP address.

The code below does the trick (the IP should be written between quotes):

AuthType Basic
AuthName "Restricted content"
AuthUserFile /var/www/prod/apache2/.htpasswd

# Only a person with valid credentials is let in
Require valid-user
# Create the variable for the header (the IP should be written between quotes)
SetEnvIf X-Forwarded-For 203.0.113.10 AllowIP
# Include the env variable
Require env AllowIP

A request whose X-Forwarded-For header contains 203.0.113.10 is let through; everyone else is asked to log in.

Block the infamous Bytespider bot

Sometimes a server can go high in load due to the infamous Bytespider bot. This one can be excluded by adding this piece of code inside the .htaccess:

<IfModule mod_rewrite.c>
    RewriteEngine On
    RewriteCond %{HTTP_USER_AGENT} Bytespider [NC]
    RewriteRule .* - [F]
</IfModule>

For broader protection against abusive traffic, prefer the Firewall over per-site rules.