Complete Routed-Space DNS Lookups

This dataset contains PTR and SOA DNS lookups of every routed IPv4 address using a recursive resolver from hosts at CAIDA.

| Data Sources: Passive Active Other External |

Complete Routed-Space DNS Lookups

We performed the lookups using one of two tools. For older datasets, we used a custom parallel DNS lookup tool based on libunbound, a component of the Unbound validating, recursive, caching DNS resolver. For newer datasets, beginning with 2018-08-23, we use zdns in iterative mode. To determine routed addresses, we use BGP tables collected by RIPE and RouteViews via BGPstream. Lookups of the full routed space take 2-3 weeks to complete.

In the 2014-08-22 dataset, we queried 2.7 billion addresses in 10.6 million /24 prefixes (for this particular dataset, we did not query the .0 and .255 addresses in each /24, but all future datasets will query these as well).

Currently available datasets are

2019-01-25

2018-08-23

2017-09-08

2015-09-17

2014-08-22

File format

Result files are in a simple tab-separated, line-oriented text format.

A PTR result has the following fields:
   timestamp   IP_address      hostname
   1537482068  192.172.226.123 cider.caida.org
An SOA result has the following fields:
   timestamp   IP_address       name                      ns         mbox                  serial    refresh retry expire  ttl
   1537482068  192.172.226.123  226.172.192.in-addr.arpa. caida.org. postmaster.caida.org. 201808220 86400   300   3600000 86400
Acceptable Use Agreement

Access to these data is subject to the terms of the following CAIDA Acceptable Use Agreement (printable version in PDF format)
CAIDA ACCEPTABLE USE AGREEMENT (ver 071814) The following terms comprise the Acceptable Use Policy (AUP) and Data License Agreement (collectively, the "AGREEMENT", or, "Acceptable Use Agreement (AUA)" for all datasets made available to You by the Center for Applied Internet Data Analysis (CAIDA), a research unit at the University of California San Diego (UCSD) and governed by The Regents of the University of California. Certain datasets may have additional Supplemental provisions. References to this AGREEMENT shall include any and all relevant Supplemental provisions. CAIDA has the authority and reserves the right, in its sole discretion, to refuse requests for dataset(s) or discontinue further access and use to anyone. If You feel Your request is inappropriately denied please contact CAIDA by sending a message to data-info@caida.org. In consideration for requesting and receiving access to CAIDA dataset(s), You acknowledge that You understand and agree to be bound by the terms and conditions of this AGREEMENT. Any violation of this AGREEMENT may result in the immediate suspension or termination of this AGREEMENT and/or other action entitled by law such as injunctive or equitable relief. You are individually liable and responsible for compliance with this AGREEMENT. This AGREEMENT is legally binding under the laws of the State of California, United States. You may terminate this Agreement by contacting CAIDA in writing and receiving acknowledgement of such request. 1. LICENSE CAIDA's authorization to access the data grants You a limited, non-exclusive, non-transferable, non-assignable, and terminable license to copy, modify, and use the data in accordance with this AGREEMENT solely for the purpose of non-profit research, non-profit education, commercial internal testing and evaluation of the data, or for government purposes by or on behalf of the U.S. Government. No license is granted for any other purpose and there are no implied licenses in this Agreement. This Agreement shall become effective as of the date of approval by CAIDA and shall remain in force for a period of one year, unless terminated earlier or amended in writing. CAIDA shall have the right to use any of Your feedback received during the license period solely for its non-profit educational and/or research purposes. 2. GENERAL CONDITIONS (i) You will not impersonate any individual or entity, misrepresent any affiliation with another person, entity or association, use false information, or otherwise conceal Your identity from CAIDA at any time for any purpose. (ii) You consent that CAIDA can make public or otherwise disclose Your name as the registered requestor, the name of Your Affiliated Institution, the name of the dataset(s) that CAIDA has made available to You under this AUA, and the brief description of the type of research being undertaken that You provided to CAIDA. (iii) You will abide by any and all modifications. If a modification occurs, it will be explicitly communicated to You via Your registered email address and shall become effective fifteen (15) days after the transmission of such notification. Your continued access to or use of the data after such time shall indicate Your assent to any and all modification(s). If You do not agree to comply with the modification(s), You agree that You will: (a) inform CAIDA immediately, at at which point Your access to the data will be terminated and no longer authorized; and, (b) no longer access or use this data. 3. USE RESTRICTIONS If You have any concerns or questions about these restrictions, You are encouraged to contact CAIDA management via email to data-info@caida.org. To obtain an exemption from any of these restrictions, You will need a written authorization from CAIDA management. (i) While using non-anonymized data set(s), You will respect the privacy of persons that may be identified in the data. For any publication or other disclosure, You will anonymize or de-identify personally-identifiable information, IP addresses, and other data identified in Supplemental provisions (if any) by using commonly accepted techniques such as one of the methods recommended by CAIDA (https://www.caida.org/projects/predict/anonymization/). (ii) While using anonymized data set(s), You will not attempt to reverse engineer, decrypt, de-anonymize, derive or otherwise re-identify anonymized information. (iii) You will not distribute, disclose, transfer or otherwise make available the dataset(s) to any person other than those employed by your institute who are assisting or collaborating with You using the dataset(s). Other entities with whom You are collaborating in research using the dataset(s) must request access to the dataset(s) separately and directly from CAIDA. (iv) All conditions, restrictions and obligations attached to this data shall accompany any and all subsequent uses and disclosures of this dataset by You. Therefore, You are personally and fully responsible for communicating this AUA and ensuring its compliance as to any and all users described above to whom You make the data available. 4. USE OBLIGATIONS (i) If You create a publication (including web pages, papers published by a third party, and publicly available presentations) using data from this dataset, You must provide CAIDA with a copy of (or a link to) the publication and You must cite the data as follows: The CAIDA UCSD [DataSet Name] - [dates used], https://www.caida.org/data/[dataset-URL] (ii) At the end of the research, or semi-annually (whichever is sooner), You will report a summary of the research and any findings/conclusions to CAIDA. This information is used in reports to our funding agencies. (iii) You agree to expunge any and all copies of the received Dataset(s) upon completion or termination of stated research and/or termination of data access or use. Completion of stated research shall allow for a reasonable period of time that You may need to retain the dataset(s) in order to satisfy scientific reproducibility obligations. 5. ACCOUNTABILITY AND ENFORCEMENT (i) You agrees to safeguard any and all sensitive data as required, or may be required, by law by using at least the same degree of care that You uses for its own data of a like nature but no less than a reasonable degree of care, to protect the confidentiality of data and/or the privacy of any identifiable person and to prevent its unauthorized disclosure and use. Data is confidential if it is marked as such, if by its nature or content is reasonably distinguishable as confidential, or if You have reasonable cause to believe that its disclosure to a third party would cause harm or damage. Data is not confidential, and therefore not sensitive if: (a) You already knew the data before it was disclosed to You by CAIDA; (b) You gain subsequent knowledge of the data by either lawfully obtaining it from another source under no obligation of confidentiality, or You develop it independently; (c) the data is or becomes generally available to the public through no wrongful act of You or any other party; (e) it is required to be disclosed under applicable law, regulation or court order provided You notify CAIDA prior to making such a disclosure so that CAIDA may take appropriate action. (ii) You will notify CAIDA immediately of all relevant details if: (a) confidentiality or privacy is compromised; or (b) You receive any legal, investigatory, or other government demand to reverse engineer, decrypt, de-anonymize or otherwise disclose anonymized or confidential data. (iii) If requested, You will provide CAIDA evidence of compliance with this AGREEMENT, such as a written description of Your data protection plan or a written affirmation that You have disposed of any and all copies of the received dataset from all systems reasonably known to you. (iv) DISCLAIMER OFWARRANTIES. CAIDA USES ITS BEST EFFORTS TO PROVIDE DATA IN ACCORDANCE WITH ETHICAL PRINCIPLES AND SCIENTIFIC INTEGRITY. HOWEVER, THE DATA PROVIDED HEREIN IS ON AN "AS IS" BASIS. NEITHER CAIDA, ITS RESEARCHERS, RESEARCH PARTNERS, LICENSORS, AND DATA PROVIDERS, NOR THE UNIVERSITY OF CALIFORNIA AND ITS TRUSTEES, OFFICERS, EMPLOYEES, AND AGENTS MAKE ANY WARRANTY, EITHER IMPLIED OR EXPRESS, OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, INCLUDING, BUT NOT LIMITED TO, THE ACCURACY, TIMELINESS, COMPLETENESS, RELIABILITY, OR AVAILABILITY OF CAIDA DATA, APPLICATIONS, OR SERVICES ACCESSIBLE THROUGH OR MADE AVAILABLE BY CAIDA. TO THE EXTENT ALLOWED BY LAW, YOU WILL INDEMNIFY, DEFEND AND HOLD HARMLESS CAIDA AND THE UNIVERSITY OF CALIFORNIA FROMALL LIABILITIES, CLAIMS, DEMANDS, COSTS, JUDGMENTS, DAMAGES, LOSSES AND EXPENSES ARISING OUT OF OR IN CONNECTION WITH ANY USE OF THE DATA BY YOU. IN NO EVENT SHALL CAIDA AND THE UNIVERSITY OF CALIFORNIA BE LIABLE TO YOU OR ANY THIRD PARTY FOR ANY INDIRECT, CONSEQUENTIAL, INCIDENTAL, SPECIAL OR PUNITIVE DAMAGES, ARISING FROM YOUR USE OF THE DATA. 6. INTELLECTUAL PROPERTY This section applies only if You are licensing the data for the purpose of commercial internal testing and evaluating the data. (i) Except as specifically provided in paragraphs (ii), (iii) or (iv) of this Section, all right, title and interest in and to any "Intellectual Property" any invention, discovery, improvement, computer or other program, related documentation and work of authorship that is developed for commercial use by You or Your employer, employees or agents, either individually or jointly with CAIDA, using any information provided by CAIDA under this Agreement, shall be the joint property of UCSD/CAIDA and You, and that shared portion is hereby assigned to UCSD/CAIDA, solely for its non-profit educational and/or research purposes. As between CAIDA and You, You retain exclusive and all right, title and interest in and to "Your Intellectual Property" the information originating with or provided by You. (ii) You obtain an exemption from the default Intellectual Property provisions in Section (i) for specific developments via a written authorization from CAIDA management; (iii) You obtain a full Commercial License from the UC San Diego Technology Transfer Office that grants alternative rights in the Intellectual Property; or (iv) You make the Intellectual Property freely available and without restrictions to the public, provided such disclosure complies with obligations regarding sensitive information described in Section 5(i). 6.1 UCSD COPYRIGHT Permission to use, print, copy, and modify any copyrightable part of this UCSD CAIDA Data for educational, research and non-profit purposes, as set forth in the Researcher MOA without fee, and without a written agreement is hereby granted, provided that this paragraph and the following copyright notice and paragraphs appear in all copies: Copyright 2006-2014 The Regents of the University of California. All Rights Reserved. If You desire to use or otherwise incorporate any copyrightable part of this data for commercial purposes, You should contact the UC San Diego Technology Transfer Office, (858) 534-5815, fax: (858) 534-7345.

When referencing this data (as required by the AUA), please use:
The CAIDA UCSD Complete Routed-Space DNS Lookups Dataset - <dates used>,
https://www.caida.org/data/active/complete_dns_lookups_dataset.xml.
Also, please, report your publication to CAIDA.

Request Data Access

Request Access to the CAIDA Complete Routed-Space DNS Lookups Dataset

Topology Datasets

Freely Available Datasets

The Ark IPv4 Routed /24 Topology Dataset (data older than one year only)

The Ark IPv4 Routed /24 DNS Names Dataset (data older than one year only)

IPv4 TNT MPLS Topology Dataset (data older than one year only)

Ark Internet Topology Data Kits (ITDK) (data older than one year only)

The Ark IPv6 Topology Dataset

The Ark IPv6 DNS Names Dataset

The IPv6 Routed /48 Topology Dataset

IPv4 Routed /24 AS Links (September 2007 - ongoing)

IPv6 AS Links (December 2008 - ongoing)

AS Rank

AS Relationships

Skitter Macroscopic Topology Data

Skitter Internet Topology Data Kits (ITDK) - April 2002 and April/May 2003

Skitter AS Links (January 2000 - February 2008)

Skitter Router Adjacencies

AS Taxonomy

PAM 2010 "Improving AS Annotations" Supplement

Restricted Access Datasets

The Ark IPv4 Routed /24 Topology Dataset (incl. most recent one year)

The Ark IPv4 Routed /24 DNS Names Dataset (incl. most recent one year)

IPv4 TNT MPLS Topology Dataset (incl. most recent one year)

The Ark IPv4 Prefix-Probing Dataset (incl. most recent one year)

Ark Internet Topology Data Kits (ITDK) (incl. most recent one year)

Complete Routed-Space DNS Lookups

References

For more information on CAIDA topology measurements, see:

Internet Topology Data Comparison

The Archipelago Project (Ark)

Macroscopic Topology Measurements

For more information on topology measurements in general see:

University of Oregon Route Views Project

Paris traceroute