Crawls performed by Internet Archive on behalf of the National Library of Australia. This data is currently not publicly accessible.
92.1M
92M
Dec 19, 2017
12/17
by
Internet Archive Web Group
A series of open web crawls targeting journal articles, technical memos, essays, datasets, and other research publications. This collection contains WARC and CDX files that end up in Wayback ( https://web.archive.org ). See also bibliographic metadata corpuses at https://archive.org/details/ia_biblio_metadata
WARCs from internal crawl testing.
Topics: web, cctld
Data collected by Internet Archive on behalf of the National Library of Spain. This data is currently not publicly accessible.
This collection contains collaborative Election crawls performed by IA.
Topics: elections, web
This crawl was performed in Summer & Fall of 2012 to archive the US Federal Elections.
Topics: US, federal, elections, web, 2012
Crawls of the french domain space performed by Internet Archive on behalf of Bibliotheque Nationale de France. This data is currently not publicly accessible.
National Archives and Records Administration crawl performed by Internet Archive. This data is currently not publicly accessible.
Domain crawl of the Australian web domain (.au) performed by Internet Archive on behalf of the National Library of Australia in March-April, 2022.
Topic: crawldata
National Library of Luxembourg
Topic: Luxembourg
National Library of Austrailia crawl. This data is currently not publicly accessible.
Data collected by Internet Archive on behalf of the National Library of Israel. This data is currently not publicly accessible.
Topic: nlil
this data is currently not publicly accessible.
This collection includes all collaborative Olympic crawls performed by IA for the IIPC.
Topics: olympics, IIPC, web
13.5M
13M
Jul 17, 2018
07/18
by
Internet Archive Web Group
Web archive data from a crawl of open access PDF URLs provided by Unpaywall.
Crawls performed by the Internet Archive in 2017 on behalf of the National Library of Australia.
Topic: nla web 2017
this data is currently not publicly accessible.
These crawls were performed by IA on behalf of the IIPC in Summer 2012 during and prior to the 2012 Summer Olympics held in London, UK.
Topics: London, olympics, web, 2012, IIPC
this data is currently not publicly accessible.
This crawl of the .au domain was performed on behalf of the National Library of Australia in of 2016.
Topics: nla, australia, web
Domain crawl of the New Zealand web domain (.nz) performed by Internet Archive on behalf of the National Library of New Zealand in January-March, 2022.
Topic: crawldata
These crawls of the .es domain were performed in 2011 on behalf of the National Library of Spain (BNE).
Topics: bne, spain, web, 2011
ccTLD crawl for .br domain
Topics: br, web, 2018, cctld
4.7M
4.7M
Apr 26, 2019
04/19
by
Internet Archive Web Group
This crawl of online resources of the 114th US Congress was performed on behalf of The United States National Archives & Records Administration (NARA).
Domain crawl of the Australian web domain (.au) performed by Internet Archive on behalf of the National Library of Australia in March-April, 2021.
Topic: crawldata
Crawls performed by Internet Archive on behalf of the National Library of Ireland. This data is currently not publicly accessible.
Domain crawl of the New Zealand web domain (.nz) performed by Internet Archive on behalf of the National Library of New Zealand in January-February, 2018.
Topics: web, nlnz, 2018
This crawl of the .au domain was performed on behalf of the National Library of Australia in of 2015.
Topics: nla, web, 2015
Web archive data collected by Internet Archive from a domain harvest of the Ukrainian ccTLD.
Topic: crawldata
Data collected by Internet Archive on behalf of the National Library of Sweden. This data is currently not publicly accessible.
this data is currently not publicly accessible.
This crawl of online resources of the 112th US Congress was performed in Fall of 2012 and early winter of 2013 on behalf of NARA.
Topics: nara, 112th, web
Crawls performed by the Internet Archive in 2018 on behalf of the National Library of Australia.
Topics: nla, web, 2018
This crawl of the .au domain was performed on behalf of the National Library of Australia in of 2014.
Topics: nla, web, 2014
Topics: bne, spain, web, 2013
this data is currently not publicly accessible.
016-2022-Spring domain crawl of the Luxembourg web domain (.lu) performed by Internet Archive March - May 2022 on behalf of the National Library of Luxembourg / Bibliothèque nationale de Luxembourg.
Topic: crawldata
8.8M
8.8M
Jul 6, 2020
07/20
by
Internet Archive Web Group
10.1M
10M
Apr 9, 2018
04/18
by
Internet Archive Web Group
this data is currently not publicly accessible.
Crawls performed by the Internet Archive in 2019 on behalf of the National Library of Australia.
Topics: nla, web, 2019
This crawl of the .au domain was performed on behalf of the National Library of Australia in Spring of 2013.
Topics: nla, web, 2013
10.9M
11M
Aug 4, 2017
08/17
by
Internet Archive Web Group
Microsoft Academic Graph public corpus (Feb 2016) PDF URLs, filtered to remove large sites (pubmed, citeseerx, arxiv) and already-crawled URLs.
Topics: papers, journals
this data is currently not publicly accessible.
Domain crawl of the New Zealand web domain (.nz) performed by Internet Archive on behalf of the National Library of New Zealand in January-March, 2021.
Topic: crawldata
2015 crawl of museum websites listed in the IMLS Museum Universe Data File. More about the IMLS MUDF can be found at https://www.imls.gov/research-evaluation/data-collection/museum-universe-data-file
Topic: AIT
2017 domain crawl for National Library of Ireland.
Topics: ireland, web
This crawl of the .es domain was performed in 2012 on behalf of the National Library of Spain (BNE).
Topics: bne, spain, web, 2012
Crawls performed by the Internet Archive in 2020 on behalf of the National Library of Australia.
Topics: nla, web, 2020
3.3M
3.3M
Mar 5, 2020
03/20
by
Internet Archive Web Group
this data is currently not publicly accessible.
This crawl of online resources of the 115th US Congress was performed on behalf of The United States National Archives & Records
Topic: crawldata
this data is currently not publicly accessible.
this data is currently not publicly accessible.
4M
4.0M
May 28, 2020
05/20
by
Internet Archive Web Group
This crawl of the .nz domain was performed on behalf of the National Library of New Zealand in Spring of 2017.
Topics: nlnz, web, 2017
Crawl 00001 of the IMLS Museum Universe Date File.
26.2M
26M
Oct 3, 2013
10/13
by
dominic@archive.org
This crawl of the .il domain was performed in 2013 on behalf of the National Library of Israel (NLIL).
Topics: nlil, israel, web, 2013
Domain crawl of the Israel web domain (.il) performed by Internet Archive in October-November 2021 on behalf of the National Library of Israel.
Topic: crawldata
this data is currently not publicly accessible.
This collection includes content harvested from the Web on behalf of the National Library & Archives New Zealand in January 2016.
Topics: new zealand, web, domain
Domain crawl of the New Zealand web domain (.nz) performed by Internet Archive on behalf of the National Library of New Zealand in January-February, 2019.
Topics: web, nlnz, 2019
Crawls performed by the Internet Archive in 2020 on behalf of the National Library of Israel .
Topic: web
this data is currently not publicly accessible.
3M
3.0M
Feb 5, 2020
02/20
by
Internet Archive Web Group
This collection includes content harvested from the Web on behalf of the National Library & Archives New Zealand in February 2013.
Topics: web, domain
p.p1 {margin: 0.0px 0.0px 0.0px 0.0px; line-height: 17.0px; font: 12.8px Menlo; color: #161516; background-color: #ffffff} span.s1 {font-kerning: none} WARCS from Whole Earth Web Archive (WEWA) Domain Crawls
Topic: web
This crawl of the .il domain was performed in 2015 on behalf of the National Library of Israel (NLIL).
Topics: nlil, israel, web, 2015
3.4M
3.4M
Jan 24, 2020
01/20
by
Internet Archive Web Group
Data collected by Internet Archive on behalf of Biblioteca Nazionale Centrale di Firenze. This data is currently not publicly accessible.
13.8M
14M
Aug 4, 2014
08/14
by
dominic@archive.org
This crawl of the .il domain was performed in 2014 on behalf of the National Library of Israel (NLIL).
Topics: nlil, israel, web, 2014
4.8M
4.8M
Feb 15, 2019
02/19
by
Internet Archive Web Group
This crawl of online resources of the 111th Congress of the United States was performed in Fall of 2010 and Winter of 2011 on behalf of NARA.
Topics: nara, 111th, congress, web
KB Curated List Crawl 2019. This data is not currently publicly accessible.
Topic: web
