Most are parasitic (GPTBot, ImageSift bot, Yandex, etc) but I’ve even blocked Google’s crawler (and its ActivityPub cralwer bot) since it now feeds their LLM models. Most of my content can be found anyway because instances it federated to don’t block those, but the bandwidth and processing savings are what I’m in it for.
I’ve seen some when I appended “Lemmy” just like “Reddit”. But it relies on lemmy being in the domain name.
Also I assume even when people click on those results, they don’t get ranked much higher because it’s so many different domains while reddit is just one.
I’m inclined to think due to the nature of the platform, contents are constantly duplicated to the eyes of search engines, which hurts authoritativeness of each instance thereby hurts ranking.
The problem with that is, lemmy.world is only one of many different instances. Too bad there isn’t a way to add a modifier that searches the entire fediverse.
from the top of my head, that won’t include lemm.ee, sopuli, beehaw, szmer.info, slrpnk.net, sh.itjust.works, or other threadiverse instances like kbin/mbin.
tbh I’ve never seen a Lemmy link when searching for stuff. Is it too small to show up? Or do search engines not index Lemmy instances?
A lot of Fediverse admins are just normal people like you and me with a budget, and disallowing bots and spiders helps save bandwidth, and the budget.
Yep. I block all bots to my instance.
Most are parasitic (GPTBot, ImageSift bot, Yandex, etc) but I’ve even blocked Google’s crawler (and its ActivityPub cralwer bot) since it now feeds their LLM models. Most of my content can be found anyway because instances it federated to don’t block those, but the bandwidth and processing savings are what I’m in it for.
Searx will show Lemmy results, at least on some Searx instances.
One of the major problems with Lemmy is that many posts get deleted and that nukes the comment section (which is where most of the answers will be).
I wish Lemmy deleted posts closer to how Reddit deletes posts - the post content should be deleted, but leave the comments alone.
I’ve seen some when I appended “Lemmy” just like “Reddit”. But it relies on lemmy being in the domain name.
Also I assume even when people click on those results, they don’t get ranked much higher because it’s so many different domains while reddit is just one.
I’m inclined to think due to the nature of the platform, contents are constantly duplicated to the eyes of search engines, which hurts authoritativeness of each instance thereby hurts ranking.
You can always add “site:lemmy.world” to your search (remove the quotes). I commonly do that, as well as the same for reddit or stack overflow.
The problem with that is, lemmy.world is only one of many different instances. Too bad there isn’t a way to add a modifier that searches the entire fediverse.
yea i’ve been doing “inurl:lemmy” for that reason
from the top of my head, that won’t include lemm.ee, sopuli, beehaw, szmer.info, slrpnk.net, sh.itjust.works, or other threadiverse instances like kbin/mbin.