Preventing some types of User Agents from accessing my website

[jungleist]

My website sometimes gets hit with recursive wgets and HTTrack downloads that can cripple the web server.

I'd like restrict access from these programs. Is user agent filtering for these types of scans worthwhile? Here is another example I found.

Do you know of any other ways to prevent server overload at the Apache / OS level? I don't mind limiting connections, I just don't want the server to crash.

[jungleist]

I fould my solution. I just added the examples in these links to my httpd.conf instead of .htaccess. I prefer httpd.conf because its not as easy to forget as .htaccess when doing a website restoration, or troubleshooting.

Comprehensive guide to .htaccess
Hijacking - Some Advice for Webmasters

Looks like it's working so far.

[jungleist]

Sorry, it needs to be in .htaccess.

[NipponDSM]

I use modsecurity, which is described as an application firewall. This may be a bit much for your needs, though.