• 0 Posts
  • 104 Comments
Joined 10 months ago
cake
Cake day: August 10th, 2024

help-circle











  • Zexks@lemmy.worldtoScience Memes@mander.xyzBlack Mirror AI
    link
    fedilink
    English
    arrow-up
    3
    ·
    edit-2
    8 days ago

    No. Only very selective people joined message boards. The rest were on AOL, compact or not at all. You’re taking a very select group of.people and expecting the Facebook and iPad generations to be able to do that. Not going to happen. I also noticed some people below talking about things like geocities and other minor free hosting and publishing site that are all gone now. They’re not coming back.


  • Lol. And they’ll delist you. Unless you’re really important, good luck with that.

    robots.txt

    Disallow: /some-page.html

    If you disallow a page in robots.txt Google won’t crawl the page. Even when Google finds links to the page and knows it exists, Googlebot won’t download the page or see the contents. Google will usually not choose to index the URL, however that isn’t 100%. Google may include the URL in the search index along with words from the anchor text of links to it if it feels that it may be an important page.