How to restrict access with basic authentication
In this article we tackle the problem of how to decide whether a visitor should or should not log in on a server with basic authentication enabled, based on their IP address.
So what is the result we want to achieve? We want to implement an .htpasswd file so visitors need a
valid login, except when the request comes from a whitelisted IP address. In that case no login is
asked and the visitor is taken straight to the site. Like a VIP that would skip the waiting queue for
a club.
This method is used for Apache (apache2).
Note
The login file is a .htpasswd file created with the htpasswd tool. htpasswd ships in the
apache2-utils package; add it with
os_extra_packages if it is not present. For
example, htpasswd -c /var/www/prod/apache2/.htpasswd admin creates the file and adds the first
user (use -c only for the first user).
Method 1: Server without Varnish enabled
There is a difference when a server has or does not have Varnish enabled. For now we keep it simple and assume there is no interruption from any service like Varnish. In that case we use the following setup.
For best practice, put this code at the top of your .htaccess file:
AuthType Basic
AuthName "Restricted content"
AuthUserFile /var/www/prod/apache2/.htpasswd
# Whitelisted IPs are granted access without a login prompt
Require ip 203.0.113.10
# Only a person with valid credentials is let in
Require valid-user
Apache treats the two Require lines as "either one is enough": a request from 203.0.113.10 is
allowed without a prompt, and every other request must supply a valid login.
Method 2: Server with Varnish enabled
For a server with Varnish enabled, a different approach is needed. All requests that go through
Varnish pass the X-Forwarded-For header, but it may contain some tampered information about the
visitor's IP. Because of this, the request for immediate access is denied and the visitor is asked to
log in. To make sure this does not happen, we add a variable for the header that contains the
whitelisted IP address.
The code below does the trick (the IP should be written between quotes):
AuthType Basic
AuthName "Restricted content"
AuthUserFile /var/www/prod/apache2/.htpasswd
# Only a person with valid credentials is let in
Require valid-user
# Create the variable for the header (the IP should be written between quotes)
SetEnvIf X-Forwarded-For 203.0.113.10 AllowIP
# Include the env variable
Require env AllowIP
A request whose X-Forwarded-For header contains 203.0.113.10 is let through; everyone else is
asked to log in.
Note
Not sure whether Varnish is enabled? Check the host's caching setting, or start with the Varnish version above - it also works as a fallback because it does not rely on the direct client IP.
Warning
A syntax error in .htaccess can return 500 Internal Server Error for the whole directory. Change
one rule at a time and test. If you are unsure, contact support.
Method 3: Ask for the login in front of Varnish
Methods 1 and 2 put the rules in .htaccess, which Apache reads from the document root. With Varnish
enabled that document root belongs to the backend: Apache serves a second, plain virtual host that
Varnish requests when it has nothing cached, while the virtual host on port 443 only passes traffic
to Varnish and has no document root of its own.
Two things follow from that. The front virtual host never reads .htaccess, and a page Varnish
already holds is returned without asking Apache. The login prompt is only enforced on a cache miss.
To ask for the login before the cache, put the rules in ~/apache2/outside/50vhost_post.conf
instead:
<Location "/">
AuthType Basic
AuthName "Restricted content"
AuthUserFile /var/www/prod/apache2/.htpasswd
# Whitelisted IPs are granted access without a login prompt
Require ip 203.0.113.10
# Only a person with valid credentials is let in
Require valid-user
</Location>
That file is read as part of the front virtual host, so the directives need a <Location> container -
unlike .htaccess, which is already scoped to the directory it sits in. The front virtual host also
sees the visitor's own address, so Require ip takes that address directly and the SetEnvIf from
Method 2 is not needed here.
Apply the change:
tscli apache reload
Note
The ~/apache2/outside/ directory only exists when Varnish or a reverse proxy is enabled. Its
50vhost_post.conf is created empty and is never overwritten, so what you put in it survives a
publish.
Important
A login prompt in front of the cache applies to every visitor, including uptime monitoring. A
monitoring_url pointing at the protected
site reports it as down. Give the check its own login by including the credentials in the address
it calls: https://monitor:password@example.com/.
Block the infamous Bytespider bot
Sometimes a server can go high in load due to the infamous Bytespider bot. This one can be excluded
by adding this piece of code inside the .htaccess:
<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Bytespider [NC]
RewriteRule .* - [F]
</IfModule>
For broader protection against abusive traffic, prefer the Firewall over per-site rules.
Related
- Use .htaccess overrides
- How Apache and Nginx relate
- Configure Apache
- Configure Varnish - check whether the cache is enabled
- Installing extra OS packages
- Firewall