Removing file extensions from URLs is a common task that improves SEO and is user-friendly. The best way to do this is with your .htaccess file. Along with URI rewriting, there are a lot of things the .htaccess file can do.

What is .htaccess?

An .htaccess file is a directory specific configuration file for Apache servers. When a file is served from the directory (or a child directory) that the file is placed in, the configuration settings in the file are implemented by Apache. The name of the file is .htaccess, that’s not just it’s file extension. It’s not config.htaccess, just .htaccess

What can .htaccess do?

With .htaccess, the following actions are available:

• Changing Error Pages
• Redirects
• Password Protection
• Deny users by IP address and referrer
• Hot-link prevention
• Bot blocking
• MIME types
• Enable Server-side includes
• Enable CGI outside of cgi-bin
• Disable directory listings
• Set server timezone
• Change server signature
• Prevent access to files
• Force scripts to display
• Force downloads instead of streams of media files
• Set up associations
• Prevent requests with invalid characters

That’s a lot of stuff. The most commonly used actions include setting error pages, redirects, password protection, user denial, hot-link prevention, and bot-blocking. In this article, we’re going to cover what I just mentioned. I’ll delve in to the rest of the list in another post.

Removing File Extensions

Removing file extensions is user-friendly and often helps to improve SEO. The below code block directs the Apache Web Server to serve all files ending with the PHP file extension as the filename without the extension.

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^\.]+)$ $1.php [NC,L]

It’s possible to do the same thing with HTML files, too:

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^\.]+)$ $1.html [NC,L]

It’s also possible to add a trailing slash, but in a slightly different way:

RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^([^/]+)/$ $1.php
RewriteRule ^([^/]+)/([^/]+)/$ /$1/$2.php
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(\.[a-zA-Z0-9]{1,5}|/)$
RewriteRule (.*)$ /$1/ [R=301,L]

Setting Error pages

Custom error pages are a great way to encourage users to continue using your site after they experience an error. 404 (file not found) and 500 (internal service error) errors are most common. Having a way for users to go back to where they were without having to use the browser’s back button is a good way to encourage them to stay on your site.

Setting it up

In your .htaccess file, you need to set the document to be served for the error experienced. That code looks like this:

ErrorDocument 404 /error_pages/404.html

Redirects

Redirects are also an important function the .htaccess can serve. Sometimes sites get moved and rather than showing a 404, a redirect is possible. Redirects are also more SEO friendly than 404s and tend to preserve users better. There are a lot of redirect codes, but the most common and useful is the 301, which is a permanent redirect. A redirect can either (1) show the user a different page from a URL or (2) direct the user to an entirely different URL.

Showing the user a different page from the same URL

The code block below will tell the Apache Web Server that if a visitor requests a document located in the directory ‘some_website’, then to display the document ‘index.html’ located in the directory ‘new_website_directory’.

Redirect /some_website/ http://www.yourdomain.com/new_website_directory/index.html

Directing the user to a different URL

The code block below will tell the Apache Web Server to forward the visitor to the specified URL if they request the document “old.html”.

Redirect 301 /old.html http://www.example.com/newpage.html

Password Protection

Password protecting a URL or directory is common when you need to share privileged information with people. I use it frequently to set up staging sites for my clients. The code below tells the Apache Web Server the secure directory is called ‘Secure Directory’. The name will be displayed when the pop-up login prompt appears. The second line specifies the location of the password file. The third line specifies the authentication type – the example is using ‘Basic’ because we are using basic HTTP authentication. The fourth line specifies that we require valid login credentials. This line can also be used to specify a specific username: require user username would require the username username. This would be used if you were password protecting an administration area, rather than setting up a public password protected directory.

AuthName "Secure Directory"
AuthUserFile /passwords/.htpasswd
AuthType Basic
require valid-user

The password file would look something like this:

username:encryptedpassword
example_user:oCF9Pam/MXJg2

User Denial

Denying users can be helpful to reduce spam, attacks, and unwanted traffic. There are two ways to deny access to users with .htaccess: by their IP, or by referrer.

By IP address

Blacklisting users by IP can be great. This feature of Apache also allows you to whitelist users. The below code block tells the Apache Web Server to block visitors from the IP address 255.0.0.0 and 123.45.6. Note that the second IP address is missing the fourth set of digits – this means any IP address which matches the firth three set of digits will be blocked (123.45.6.10 and 123.45.6.255 would be blocked.

order allow,deny
deny from 255.0.0.0
deny from 123.45.6.
allow from all

Block by Referrer

If you look at your logs and notice that there is an increase in traffic but not file requests, chances are that (1) content is getting “pinched” (like a CSS file) or (2) someone is attempting to hack your website (trying to access non-public files/directories). To prevent this, you can block by referrer. Note that to block by referrer, mod_rewrite must be enabled. The below code block tells the Apache Web Server to block traffic from the URL suspicious.com. The ‘[NC]’ text after the referrer specifies it as not case-sensitive. Which prevents traffic from SuSpicious.com, suspicious.com, SUSPICIOUS.COM, etc.

RewriteEngine on
# Options +FollowSymlinks
RewriteCond %{HTTP_REFERER} suspicious\.com [NC]
RewriteRule .* - [F]

Hot-link Prevention

Hot-link prevention refers to the practice of preventing sites that are not your own from displaying your content. It’s commonly used to prevent other sites from using your images, but can also be used to prevent use of CSS and JavaScript files. To do this, mod_rewrite must be enabled. The code block below instructs the Apache Web Server to block all links to .gif, .png, and .css files which are not from https://www.yourdomain.com.

RewriteEngine on
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !^https://(www\.)?yourdomain.com/.*$ [NC]
RewriteRule \.(gif|jpg|css)$ - [F]

Bot-blocking

Offline browsers allow users to visit your site while they don’t have an internet connection. Though the purpose is often innocent, if your site is large, the cost could be high to allow these services to continue. Additionally, sometimes these services have malicious purposes, like searching for email addresses. These programs are known as “bad bots”. The below code block instructs the Apache Web Server to block requests from some known “bad bots”.

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} ^eCatch [OR]
RewriteCond %{HTTP_USER_AGENT} ^FlashGet [OR]
RewriteCond %{HTTP_USER_AGENT} ^Custo [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mass\ Downloader [OR]
RewriteCond %{HTTP_USER_AGENT} ^NetSpider [OR]
RewriteRule ^.* - [F,L]

Conclusion

By no means is this list all-inclusive or conclusive. But I think this overview pretty much overs the common use cases of the .htaccess file. In the near future, I’ll have another post going into further detail about what .htaccess can be used for.

I sourced information from the following links: alexcican.com | htaccess-guide.com

No Comments...yet