• General_Effort@lemmy.world
    link
    fedilink
    English
    arrow-up
    6
    arrow-down
    4
    ·
    3 months ago

    Shocked? You’d think all the people outraged at having their websites scraped would be delighted. That’s probably the real reason for this.

    • subignition@fedia.io
      link
      fedilink
      arrow-up
      3
      ·
      3 months ago

      It’s not the scraping itself, but the purpose of the scraping, that can be problematic. There are good reasons for public sites to allow scraping.

      • General_Effort@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        3 months ago

        I have the distinct impression that a number of people would object to the purpose of re-hosting their content as part of a commercial service, especially one run by Google.

        Anyway, now no one has to worry about Google helping people bypass their robots.txt or IP-blocks or whatever counter-measures they take. And Google doesn’t have to worry about being sued. Next stop: The Wayback Machine.