Restrict access with basic authentication
In this article we tackle the problem of how to decide whether a visitor should or should not log in on a server with basic authentication enabled, based on their IP address.
So what is the result we want to achieve? We want to implement an .htpasswd file so visitors need a
valid login, except when the request comes from a whitelisted IP address. In that case no login is
asked and the visitor is taken straight to the site. Like a VIP that would skip the waiting queue for
a club.
This method is used for Apache (apache2).
Note
The login file is a .htpasswd file created with the htpasswd tool. htpasswd ships in the
apache2-utils package; add it with
os_extra_packages if it is not present. For
example, htpasswd -c /var/www/prod/apache2/.htpasswd admin creates the file and adds the first
user (use -c only for the first user).
Method 1: Server without Varnish enabled
There is a difference when a server has or does not have Varnish enabled. For now we keep it simple and assume there is no interruption from any service like Varnish. In that case we use the following setup.
For best practice, put this code at the top of your .htaccess file:
AuthType Basic
AuthName "Restricted content"
AuthUserFile /var/www/prod/apache2/.htpasswd
# Whitelisted IPs are granted access without a login prompt
Require ip 203.0.113.10
# Only a person with valid credentials is let in
Require valid-user
Apache treats the two Require lines as "either one is enough": a request from 203.0.113.10 is
allowed without a prompt, and every other request must supply a valid login.
Method 2: Server with Varnish enabled
For a server with Varnish enabled, a different approach is needed. All requests that go through
Varnish pass the X-Forwarded-For header, but it may contain some tampered information about the
visitor's IP. Because of this, the request for immediate access is denied and the visitor is asked to
log in. To make sure this does not happen, we add a variable for the header that contains the
whitelisted IP address.
The code below does the trick (the IP should be written between quotes):
AuthType Basic
AuthName "Restricted content"
AuthUserFile /var/www/prod/apache2/.htpasswd
# Only a person with valid credentials is let in
Require valid-user
# Create the variable for the header (the IP should be written between quotes)
SetEnvIf X-Forwarded-For 203.0.113.10 AllowIP
# Include the env variable
Require env AllowIP
A request whose X-Forwarded-For header contains 203.0.113.10 is let through; everyone else is
asked to log in.
Note
Not sure whether Varnish is enabled? Check the host's caching setting, or start with the Varnish version above - it also works as a fallback because it does not rely on the direct client IP.
Warning
A syntax error in .htaccess can return 500 Internal Server Error for the whole directory. Change
one rule at a time and test. If you are unsure, contact support.
Block the infamous Bytespider bot
Sometimes a server can go high in load due to the infamous Bytespider bot. This one can be excluded
by adding this piece of code inside the .htaccess:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Bytespider [NC]
RewriteRule .* - [F]
</IfModule>
For broader protection against abusive traffic, prefer the Firewall over per-site rules.
Related
- Use .htaccess overrides
- Run Apache behind Nginx
- Configure Apache
- Installing extra OS packages
- Firewall