Another note, the facebookexternalhit and other similar meta useragents coming from Facebook IPs in the USA and Europe are bots from META stealing your content to feed there new AI engine. Best to block them or at the least only serve them the bare necessary data that is needed for Facebook shares as blocking will also prevent Facebook sharing. There are all tied into the same bots.No need to pay Cloudfare or another third party. Just implement a set of logical checks in your board's code. Our phpbb2 board uses a multilayered security approach to stop most non-desireable traffic including the more recent deluge of AI harvesters. The same approach should be easy to implement on any phpbb3 board as well.I look after a website that is getting a lot of bots, everytime i visit the site it has over 200 bots online at any one time.
The site is allocated 100GB of Bandwidth each month and every month it is used up (sometimes making the site go down)
The site has been going for over 10 years and used to be very busy but now only gets a handful of members on each day (less than 10) so i know it's not that.
The site is run through Cloudflare (free version) but with only basic settings as i am not sure how to set things up on it.
It used to be a lot of chinese bots but lately it is more of these:
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
Lots of random IP's so hard to ban.
Any suggestions on how i can reduce the bandwidth usage?
Statistics: Posted by JLA — Wed Sep 18, 2024 9:08 pm