Quote:
Originally Posted by rowan
I do search engine research and data mining so I'm interested in this. Wouldn't it be kind of geeky cool to have a search engine dedicated to indexing stale websites from geocities? :D
Question is whether 1TB is the size of the raw data, or the (heavily?) compressed archive...
|
I'd kill to get raw access to archive.org's database. Talk about the ultimate in data mining. They used to offer it and I recall seriously considering it and reading some of the details but the links to more information about it seem dead.
Quote:
Can I search the Archive?
Using the Internet Archive Wayback Machine, it is possible to search for the names of sites contained in the Archive (URLs) and to specify date ranges for your search. We hope to implement a full text search engine at some point in the future.
|
Quote:
Who has access to the collections? What about the public?
Anyone can access our collections through our website archive.org. The web archive can be searched using the Wayback Machine.
The Archive makes the collections available at no cost to researchers, historians, and scholars. At present, it takes someone with a certain level of technical knowledge to access collections in a way other than our website, but there is no requirement that a user be affiliated with any particular organization.
|
http://www.archive.org/about/faqs.php
http://www.archive.org/web/researche...nded_users.php (dead link - but if I remember right it gave a lot of information on how the collection worked and gaining access after requesting it. I believe they took it offline mainly due to funding and technical issues -- at least that's what I think the page claimed last when I read it?)
They also tried to grab a lot of Geocities stuff:
http://www.archive.org/web/geocities.php
Hopefully they merge this new collection with their own.
</rambling about archive.org since someone got me thinking of it from another topic>