CAIDA NGI Quarterly Report

CAIDA NGI Quarterly Report - October 1999

RECIPIENT'S PROGRESS STATUS AND MANAGEMENT REPORT

Predictability and Security of High Performance Networks

for the period 01 July 1999 to 30 September 1999

Report #5

CDRL A001

CONTRACT N66001-98-2-8922

29 October 1999

Quarterly Status Report

Predictability and Security of High Performance Networks

for the period 01 July 1999 to 30 September 1999

Contract N66001-98-2-8922

CDRL A001

1.0 Purpose of Report

This status report is the quarterly cooperative agreement report (CDRL A001) which summarizes the effort expended by the UCSD's Cooperative Association for Internet Data Analysis (CAIDA) program in support of SPAWARSYSCEN-SAN DIEGO and DARPA on Agreement N66001-96-2-8922.

2.0 Project Members

UCSD utilized (07/01/99 to 09/30/99):

Dr. KC Claffy                   414 hours
Andrew Gross                     88 hours
Daniel McRobb                   264 hours
David Moore                     528 hours
Other Technical Staff         2,137 hours
Management                      748 hours

GA Technical Staff: 366 hours
University of Waikato actual hours unavailable

3.0 Project Description

UCSD/CAIDA is focusing on advancing the capacity to monitor, depict, and predict

traffic behavior on current and advanced networks, through developing and deploying tools to better engineer and operate networks and to identify traffic anomalies in real time. CAIDA will concentrate efforts in the development of tools to automate the discovery and visualization of Internet topology and peering relationships, monitor and analyze Internet traffic behavior on high speed links, detect and control resource use (security), and provide for storage and analysis of data collected in aforementioned efforts.

4.0 Performance Against Plan

A. Task 1. Coral OC48mon

Modification of Task 1 occurred this quarter to include provisions for initiating work on a Gig-Ether Monitor. Based on discussions with Mari Maeda on July 20, 1999, CAIDA will expand the scope of Task 1 to include development of a Gigabit-Ethernet monitor expanding upon Waikato's DAG-4 design. CAIDA will also build and deploy additional DAG-4 OC48 monitors. No funds will be expended on the Gig-Ether development or additional OC48 monitors until a successful DAG-4 prototype card is completed. No additional funding is required for these efforts beyond those funds already allocated under the base Task 1 budget.

Due to delays in obtaining the Vitesse chipsets (reported in the April 30, 1999 Quarterly Report), Task 1 is approximately two quarters behind schedule.

B. Task 2. Tomography

Task 2 is proceeding according to plan, with the following Program Plan directives completed:

Expanded collection of data from distributed sites throughout the Internet, approximately 30,000 hosts for the global dataset (additional hosts under Option 1 below
Continued briefings to Internet community on purpose and initial results of Skitter and solicited their inputs (see section 11.2, Significant Events)
Added additional measurement host through the DNS Root name server initiative

C. Task 3. Security

Andrew Gross, head of SDSC's Pacific Institute for Computer Security (PICS), left UCSD in August of 1999 for career reasons. Due to the lack of available CAIDA personnel to dedicate to the security task and the change in leadership, this task is approximately one quarter behind schedule.

D. Task 4. Data Storage/Analysis

This task is performing according to plan, with the following Program Plan directives completed:

Expanded collection/storage of data
Enhanced analysis code and reporting formats
Created new analysis scripts (see section 11, Summary of Activity)
Made Asia Pacific study dataset available to collaborators for analysis

E. Option 1

Option 1, encompassing DNS Root Name Server measurement and analysis, visualization of massive data sets, and expansion of the tomography task to include enhancements and expansion of the database, storage system, and analysis code, was initiated in September of 1999 based on approval by the DARPA program manager. DARPA and SPAWAR have not yet made funds for this effort available; therefore UCSD is proceeding "at risk".

5.0 Major Accomplishments to Date

Under Task 1, the Coral OC48 Monitor DAG4 board design was completed by the Waikato development team and sent to a fabricator this quarter.

Under Task 2, a total 16 skitter monitors have been deployed in the field and were operational by the end of September 1999. [Note that transit bandwidth for all of these machines is being donated by ISPs or collaborating organizations.]

A 2100 destination list dataset was created as a result of a successful 10-day data collection (August 29-September 8, 1999) from skitter source machines deployed at the Asia Pacific locations. We made the this destination list available to Asia Pacific collaborators for analysis, e.g., validation of its `representativeness' with respect to AP region as well as the larger Internet.

Accomplishments under Task 3 include the completion of packet re-assembly code.

Task 4 accomplishments include development of scripts for analyzing the Asia Pacific data collected under Task 2. (See section 11, Summary of Activity)

Under Option 1, a skitter machine was successfully deployed at the "F" DNS root server located at the Palo Alto Internet eXchange (PAIX) facility, and is collecting data. A dataset of 36,000 clients using the "F" root server was developed from server logs; skitter data collection began in August of 1999. Preliminary analyses incorporating the scripts developed under Task 4 for the Asia Pacific study are described at https://www.caida.org/tools/measurement/skitter/RSSAC.

6.0 Artifacts Developed During the Past Quarter

No artifacts of note were developed during this quarter.

7.0 Issues

None.

7.1 Open issues with no plan, as yet, for resolution:

None.

7.2 Open issues with plan for resolution:

Based on written approval provided by DARPA PM Mari Maeda dated September 1, 1999, CAIDA began work on Option 1 of this award. However, the funding for this award is not yet obligated, and UCSD is operating "at risk" at this time.By the end of Year 2, Quarter 2, we estimate that $77,300 will be expended in support of the DNS Root Name Server and Visualization of Massive Datasets effort.

7.3 Issues resolved:

The project leader of Task 3 (Security), Andrew Gross of SDSC's PICS group, left his position with UCSD for career reasons. David Moore, CAIDA's Technical Manager, and CAIDA's Programmer/Analyst Ken Keys assumed leadership of the Security Task.

MCI Worldcom, an original sub-contractor for development of the OC48 monitor under Task 1, withdrew from collaboration under this award. MCI Worldcom agreed to provide their initial work on the OC48 monitor specifications at no cost to this project. As a result, previously reported expenditures for this subcontract ($37, 488) are no longer included in CAIDA's project expenditure totals.

Due to the Task 1 savings expected through the use of Waikato as a subcontractor and adoption of their designs for the DAG4, an under-run of up to $300,000 is envisioned on the Task 1 budget. Based on discussions with Mari Maeda on July 20, 1999, these funds will be applied towards development of the Gig Ether Monitor and building additional OC48 monitors.

8.0 Near-term Plan

The material below reflects the activities planned during Year 2, Quarter 2 of this project. It is organized according to the categories identified in the Project Program Plan (see https://www.caida.org/funding/progplan/NGIprogplan98.xml).

A. General/Administrative Outreach and Reporting

Quarterly meeting with SPAWAR Officials
Deliver technical presentations to organizations interested in the technical aspects of this NGI research, including the USENIX LISA 99' conference on November 12 in Seattle Washington, Stardust's IBAND conference on October 22, MCI/UUnet engineers on October 27^th, an CISAC/SAIC conference on International Cooperation to Combat Cyber Crime and Terrorism on December 5-7, 1999, the DARPA ITO PI meeting in Washington, DC, December 15-17
Nevil Brownlee, Director of Technology Development at the University of Auckland, New Zealand, will begin sabbatical with CAIDA in San Diego in November, 1999, assisting with efforts under Tasks 1, 2 and 4
Submit Quarterly Report to SPAWAR covering progress, status and management
Submit Quarterly Financial Status Report (UCSD Extramural Funds Dept. submits)
Submit Quarterly Report of Federal Cash Transactions (UCSD Extramural Funds Dept. submits)

B. Task 1. Coral OC48mon

Board testing of DAG4 cards by Waikato team in New Zealand
DAG4 card testing on OC48 networks by Waikato staff member David Miller and CAIDA team in San Diego
Continue discussions of OC48mon development and use with the community
Modify OC48 card designs, as required, based on test results and based on new Vitesse 2212 chipset and PCI 66/64 bus
C. Task 2. Tomography Mapping/Modeling
- Continue briefings to Internet community on purpose and initial results of skitter and solicit their inputs
- Increase the number of skitter datasets available to the community on CAIDA ftp site, and continue to solicit collaborative involvement by third parties
- Refine and expand global and Asia Pacific destination target lists based on initial study results
D. Task 3. Security
- Complete and test prototype security module during November 1999
- Begin work on the security enforcement module in OC12mon
- Make the prototype security module code available for beta testing by collaborators
E. Task 4. Storage/Analysis
- Expand collection/storage of data
- Enhance analysis code and reporting formats
- Continue analysis of data collected through Asia Pacific study
F. Option 1
- Deploy 2-4 additional skitter machines at DNS root server sites
- Continue and refine analysis of data collected from skitter machines located at DNS root server locations
- Deploy 30 gigabit visualization machine on the CAIDA network that will host NetGeo services to facilitate geographic-based analyses of skitter data and enable remote researchers to map IP address and domain names to geographic coordinates for network visualization purposesi
9.0 Completed Travel

David Moore traveled to the University of Waikato in New Zealand August 23, 1999 through September 18, 1999 to work with the DAG team on development of the OC48 DAG4 cards.

Dr. Claffy traveled to New Jersey to present "Internet Measurement and Data Analysis: Topology, Workload, Performance and Routing Statistics" to the American Statistical Association (ASA), August 7-10, 1999.

Tracie Monk and Dr. Claffy traveled Washington, DC to meet with the DARPA program manager July 20, 1999 to discuss the CAIDA's NGI project status.

Additional travel that occurred but was not charged to this award include:
- Dr. Claffy's and Tracie Monk's travel to New Jersey to make a Internet measurement and analysis presentation to Telcordia's Technical Advisory Board and to Internet Research staff on July 19, 1999
- Dr. Claffy's travel to London, England to make a skitter presentation to the Technology Transfer Institute's Vanguard Conference, "Business Discontinuities. Within the Ubiquitous Internet" on July 28-29, 1999
- Several trips to the Bay area by Claffy and Monk for collaboration discussions with commercial vendors and Internet providers
10.0 Equipment Purchases and Description

Five additional skitter host machines were purchased on UCSD purchase order number

10178109 from Computer Parts Plus computers under Option 1.

Total Cost: $11,300.00

One 30-Gigabit Visualization host was purchased on UCSD purchase order number 10178107 from ASA computers under Option 1.

Total Cost: $7,995.00

11.0 Summary of Activity

11.1 Work Focus:

General/Administrative Outreach and Reporting

A meeting with SPAWAR/DARPA representative Steve Spendlove was held October 14, 1999 to discuss progress for Year 2, Quarter 1. A paper entitled "Internet Measurement and Data Analysis: Topology, Workload, Performance and Routing Statistics" was published in "Statistical Computing and Graphics", Volume 10 No.1, and presented at the American Statistical Association's August conference. Dr. Claffy made several additional presentations on skitter and Coral (see Section 11.2, Significant Events).

Task 1. Coral OC48mon

MCI withdrew as sub-contractor on this award (See section 7.3, Issues Resolved).

The first OC48 monitor board design using the Vitesse 2211 chipset and PCI 34/33 bus is complete and was sent to Sydney for fabrication. After basic testing in New Zealand is conducted, the University of Waikato's David Miller will come to San Diego to work with CAIDA staff on final OC48 network testing on the prototype DAG 4 card. Note that this prototype is for proof of concept only. The chipsets the card uses are not capable of full OC48 line rates, nor can they handle packet over sonet (POS) which is critical for the final monitors. Following these tests, we will begin the second prototype DAG4 card, which will include the Vitesse 2212 chipset that is both ATM and POS capable.

Task 2. Tomography Mapping/Modeling

Skitter datasets were made available to multiple researchers in the community via CAIDA's ftp web site. The researchers' use of the skitter datasets are briefly summarized below:
- Carnegie Mellon University
  
  Yang-hua Chu, Sanjay Rao with Professor Hui Zhang are conducting a simulation study of multicast routing using the public skitter dataset for "realistic" topology in the simulator, see http://www.cs.cmu.edu/~kunwadee/research/mcastindex.html
- Boston University
  Assistant Professor John Byers is using the skitter dataset to 1) help identify heuristics which can classify links as being part of a backbone or being outside a backbone and 2) devising efficient strategies for mapping backbone links with minimum end-to-end measurements, see www.cs.bu.edu/faculty/byers
- UCLA
  Lixia Zhang, Beichuan Zhang, Jin Yixin are conducting a project entitled "Internet Data Maps Services", with the goal of building a self-organizing and adaptive Internet service to provide distance information in a scalable, robust way. They are using the skitter dataset to analyze delay and path information, see http://idmaps.eecs.umich.edu/
- University of Illinois, Chicago
  Robert Grossman is using the skitter data to test a new protocol called Data Space Transfer Protocol (DSTP) which will allow easy publishing of data on the web, see http://www.ncdm.uic.edu/dstp/
A number of presentations on skitter data and tools were made during Quarter 1, and one paper was published (See Section 11.2, Significant Events).

We also conducted a review of the global and Asia Pacific destination target list this quarter, and determined that for topology related analyses, the dataset will need significant expansion. Our goal is to cover a significant portion of currently routed IPv4 address prefixes throughout the world.

Task 3. Security

Task leadership responsibilities passed from Andrew Gross of PICS to CAIDA's David Moore and Ken Keys (see Section 7.3, Issues resolved)

We completed algorithms permitting efficient re-assembly of packet headers during this quarter and incorporated them into the CoralReef software package. Packet header filtering code is being finalized and will be tested during the coming quarter.

Due to the loss of two PICS security task leaders and the shortage of available security personnel at CAIDA, efforts on the security task, including development of the security compliance module, will be limited during the next quarter.

Task 4. Storage/Analysis

We created a dataset based on a data collection spanning August 29-September 8, 1999 from Skitter machines deployed at the Asia Pacific locations. This dataset was released to Asia Pacific collaborators for analysis. Asia Pacific Study Participants currently include Japan (APAN, KDDLabs, AboveNet), Korea (KAIST), Canada (CANARIE), Singapore (SingaREN), New Zealand (Waikato), and the US (CAIDA). We will post this dataset to the CAIDA public ftp site during Quarter 2 to allow skitter collaborators access to additional data for their research.

Extensive analysis code was developed and preliminary summary graphics were made available to Mari Maeda for DARPA presentations in Europe and at the PITAC review meeting. An abstract describing the research was also submitted for the INET 2000 conference. This *UPDATED* paper is now available at https://www.caida.org/publications/papers/2000/asia_paper/ *END OF UPDATE*.

Option 1.

DNS Root Server:

CAIDA successfully deployed a skitter host machine at the "F" DNS root server located at the Palo Alto Internet eXchange (PAIX). Preliminary analysis of the data collected from this machine is available at https://www.caida.org/Tools/Skitter/RSSAC/.

The primary near-term goal of the measurement effort is to assess two metrics of connectivity: round trip time and hop count from the root name server to the client machine in the target set to evaluate how well the clients are being served by the root name server. Later analyses will include consideration of the following aspects of topology:
1. Clusters of hosts that are particularly far, measured by latency, from all of the roots, and that might thus suggest a region that merits a new root server.
2. Insufficient redundancy in the root server architecture might be reflected in skitter topologies from multiple roots that suggest that the failure of a strategic intermediate router or sub-path would render many end hosts unable to reach any root.
3. Conversely, excessive redundancy in the infrastructure might be reflected in a set of skitter topologies from different roots where a large set of destination hosts are quite close to several of these roots.
  
  We are preparing to send three skitter host machines to additional root sever locations in November, the "A" root server at ISI in Marina Del Ray, California, the "B" root server at Network Solutions in Herndon, VA, and the "K" root server at Swedish University Network in Sweden. We will continue to refine analysis of data from each monitor.
  
  Visualization of Massive Datasets
  
  CAIDA purchased a 30-Gigabit visualization host machine that will be used to facilitate the analysis and visualization of massive network-related data sets. Analyses conducted during year 1 of this NGI project included preliminary visualizations of skitter measurement and routing data. Under this option, visualization initiatives will be expanded significantly to develop techniques and code designed to facilitate aggregation, correlation, analysis and layout of massive datasets.
  
  11.2 Significant Events:
  
  10/14/99 San Diego, CA- Dr. Claffy, Tracie Monk, and Amy Blanchard presented updates on CAIDA NGI project tasks for Year 2, Quarter 1 at a meeting with Steve Spendlove of SPAWAR. Topics of discussion included completion of OC48 DAG4 network interface card, community use of skitter data, security module progress, Asia Pacific data collection, DNS Root sever placement, and budgetary issues.
  
  08/13/99 San Diego, CA - Dr. Claffy, Sean McCreary and David Moore presented a traffic analysis tutorial for networking faculty from around the United States at CAIDA's Internet Engineering Curriculum workshop, see https://www.caida.org/Iec/workshop/9908/
  
  08/07/99- 08/10/99 Washington, DC- Dr. Claffy traveled to New Jersey to present the "Internet Measurement and Data Analysis: Topology, Workload, Performance and Routing Statistics" paper to the American Statistical Association (ASA). The Paper is available at https://www.caida.org/Papers/Nae/4hansen.html
  
  07/28/99 - 07/29/99 London, England - Dr. Claffy made an Internet measurement and analysis presentation to the Technology Transfer Institute's Vanguard Conference, "Business Discontinuities within the Ubiquitous Internet". Slides are available at https://www.caida.org/Presentations/Soa9905/
  
  07/19/99 New Jersey- Claffy and Monk traveled to New Jersey to make an Internet measurement and analysis presentation to the Telcordia Group. Slides are available at https://www.caida.org/Presentations/Soa9905/
  
  07/20/99 Washington, DC - Monk and Claffy met with DARPA program manager Mari Maeda about current status of NGI tasks, no-cost expansion of the SOW for Task 1, and activation of Option 1 under this award.
  
  Publications:
  
  A CAIDA skitter image was used in an article entitled "Internet Mapping creates it Own Atlas of Landmarks" in Sony In-Side, Straight From the Source Volume No. 11, Autumn/Winter 1999/2000.
  
  An article entitled "Internet maps Help Plot Network's Future" featuring skitter was published in the Vol. 7 No. 10 issue of Today's Science on File.
  
  CAIDA's measurement, analysis and visualization efforts were mentioned in the New York Times on-line article dated September 30, 1999 entitled "Beyond Geography: Mapping Unknown of Cyberspace" by Pamela Licalzi O'Connell, see http://www.nytimes.com/library/tech/99/09/circuits/articles/30maps.html.
  
  FINANCIAL INFORMATION:
  
  Contract #: N66001-98-2-8922
  
  Contract Period of Performance: 16/07/1998 to 15/07/2000
  
  Ceiling Value: $6,655,449
  
  Current Obligated Funds: $1,663,659
  
  Reporting Period: 07/01/1999 to 09/30/1999
  Actual Costs Incurred:
  Current Period:
```
UCSD 
Labor               Hours 4179            Cost   $   217,987
ODC's                                     Cost   $    20,858
IDC's                                     Cost   $      1179
General Atomics     Hours 366             Cost   $    10,791
Waikato                                   Cost   $    20,000
MCI                                       Cost   $   -37,488

TOTAL:              Hours 4545            Cost   $   233,328 
```
  Cumulative to date:
  UCSD Labor Hours 9223 Cost $ 334,307 ODC's Cost $ 231,578 IDC's Cost $ 179,137 GA 81,106 Waikato Cost $ 47,908 TOTAL: Hours 10,071 Cost $ 874,036
```
Note: additional financial information in tabular form, including breakdown by subcontract and estimated expenditures for Quarter 5, is attached to this report.
```