Related:
Major cyber attack could cost the world $3.5 trillion - Power Grid, Internet Outage
The one database/file/zip to save humanity, what is it?
Show Lemmy the downloadable URL of a Database or AI you know of so we can have a local backup copy that will improve the resilience and availability of Human Knowledge.
Given the state of AI being Corporatized I think we could definitely use links for whatever comes closest to a fully usable Open Source, fully self-contained downloadable AI.
Starter Pack:
- Wikipedia Single 100GB File
- http://sci-hub.wf/
- Arxiv Download Script
- https://wholeearth.info/
- https://the-eye.eu/
- Endless OS “Offline Library”
- scikit-learn AI with External Databases
- ScienceFair
Is it possible to download an archive of scihub?
Sci-Hub is ENORMOUS, about 100TB. If you want to help preserve it, you can torrent and seed one of their many 100GB chunks.
What a fantastic resource, this is exactly what is needed. I also found about The Standard Template Construct Library:
“Learn about how to access large corpus of high-quality scholarly texts using Python and use them in AI apps”
Super cool never knew about this. I got probably 1-2tb I can spare for the effort.
Does anyone know if a LLM has been trained on something like scihub?