CAIDA : funding : ngi1998 : content : reports

RECIPIENT'S PROGRESS STATUS AND MANAGEMENT REPORT

Predictability and Security of High Performance Networks

For the period 01 January 2000 to 31 March 2000

Report #8

CDRL A001

CONTRACT N66001-98-2-8922

April 30, 2000

SUBMITTED TO Receiving Officer
SPAWARSYSCEN - SAN DIEGO
e-mail address: spendlov@spawar.navy.mil

Richard Laverty
PHONE 619-553-2918
FAX 619-553-1690
laverty@spawar.navy.mil

Frank Schindler
PHONE 619-553-2845
FAX 619-553-1690
schindl@spawar.navy.mil

SUBMITTED BY
University of California, San Diego (UCSD)
9500 Gilman Drive
La Jolla, CA 92093

Principal Investigator
Dr. Kimberly Claffy
PHONE 858-534-8333
FAX 858-822-0861
kc@caida.org

Contract/Financial Contact
Lynnelle Gehrke
PHONE 858-534-0243
FAX 858-534-0280
lgehrke@ucsd.edu

Quarterly Status Report

Predictability and Security of High Performance Networks

For the period 01 January 2000 to 31 March 2000

Contract N66001-98-2-8922

CDRL A001

1.0 Purpose of Report

This status report is the quarterly cooperative agreement report (CDRL A001) which summarizes the effort expended by the UCSD's Cooperative Association for Internet Data Analysis (CAIDA) program in support of SPAWARSYSCEN-SAN DIEGO and DARPA on Agreement N66001-96-2-8922.

2.0 Project Members

UCSD utilized (01/01/00-03/31/00):

Dr. KC Claffy 328 hours

Daniel McRobb 172 hours

David Moore 588 hours

Other Staff 3081 hours

Total Hours 4169

University of Waikato actual hours unavailable(see financial information)

3.0 Project Description

UCSD/CAIDA is focusing on advancing the capacity to monitor, depict, and predict

traffic behavior on current and advanced networks, through developing and deploying tools to better engineer and operate networks and to identify traffic anomalies in real time. CAIDA will concentrate efforts in the development of tools to automate the discovery and visualization of Internet topology and peering relationships, monitor and analyze Internet traffic behavior on high speed links, detect and control resource use (security), and provide for storage and analysis of data collected in aforementioned efforts.

4.0 Performance Against Plan

A. Task 1. Coral OC48mon

The fabrication of the Dag4.0 capture card was completed in February 2000, and testing commenced by the University of Waikato development team. An OC48 Broadband Series Test System on loan from Sprint was used for testing.

We are approximately 3 quarters behind schedule on Task 1 due to the fabrication delays reported in the January 2000 quarterly report, but have completed the following Program Plan directives:

-Continued development and enhancement of OC48mon software and firmware code

-Begun testing of prototype cards

-Continued discussions of OC48mon development and use with the community

B. Task 2. Tomography

Lack of sufficient storage space on the DEC raid array that houses collected skitter data, and complications in collecting destination lists from skitter machines collocated with DNS root server machines has slowed the DNS root server initiative. Program Plan objectives that were completed during Quarter 3 include:

- Collection of data from the skitter machine co-located at the "L" root server

- Continued refinement and analysis of data collected from skitter machines located at "F" DNS root server location

-Visualization of Core Internet ASes completed

- Developed and refined skitter destination list

- Continued briefings to Internet community on purpose and initial results of skitter, and

solicited their inputs

- Enhanced the skitter/Tomography analysis/visualization tools

- Created animation describing how skitter works

C. Task 3. Security

The following Task 3 Program Plan objectives were complete this quarter:

-Development of a set of recommendations for additional security applications/implementations of Coral or related traffic monitoring tools

-Porting third-party security applications with existing CoralReef security applications.

Due to the lack of available personnel to focus on this task (as reported in the January 2000 quarterly report), we are approximately one quarter behind schedule. CAIDA employee Ryan Koga has been assigned to work on the Security task.

D. Task 4. Data Storage/Analysis

This task is performing according to plan, with the following Program Plan directives completed:

-Expanded collection/storage of data

-Enhanced analysis code and reporting formats

-Continued comprehensive analysis of skitter data (see section 11.1, Work Focus)

5.0 Major Accomplishments to Date

The following major accomplishments were achieved during Quarter 3:

Under Task 1, the first prototype of the OC48 monitor hardware is now operating, and was demonstrated at the Passive and Active Measurement Workshop held in New Zealand in early April. This prototype is ATM-only, and uses 32-bit 33 MHz PCI. The DAG 4.1 POS/ATM capture card development is underway, with circuit design completed, and layout commenced.

Task 2 accomplishments include major work on revision of the skitter destination list, release of additional skitter data to the research community, and completion of a skitter animation. Accomplishments also include successful data collection from the "L" root server, and development of a visualization of "Core Internet Autonomous Systems".

6.0 Artifacts Developed During the Past Quarter

No artifacts were developed this past quarter.

7.0 Issues

7.1 Open issues with no plan, as yet, for resolution:

None.

7.2 Open issues with plan for resolution:

A formal re-budget of remaining NGI funds and formal request for a one year extension of the NGI award will be submitted to SPAWAR/DARPA in early April. The current project end date is 07/15/00, and we are asking for a 1 year extension.

The primary reason for extension is due to significant delays in obtaining critical components required for development of the prototype OC48 capture cards under Task 1 of this award. As explained in the quarterly reports dated April and October 1999, there were delays in finalizing the subcontract with the University of Waikato, the entity chosen to do the development of the OC48 capture card. There were additional delays in obtaining Vitesse's new OC48 ATM/POS 2211 chipset with the PCI 34/33 bus, a critical component for development of the OC48 DAG 4.0 capture card. The Vitesse chipset became available in September 1999, and the DAG 4.0 was immediately sent to Jawed in Los Angeles for fabrication (see quarterly report dated October 31, 1999). These delays have put us nine months behind schedule on Task 1.

There have also been delays on Task 3 (Security) due to loss of technical personnel. As noted in the April 1999 quarterly report, Andrew Gross, head of SDSC's Pacific Institute of Computer Security (PICS) replaced Glenn Sager who left SDSC in March 1999 for career reasons (see quarterly report dated October 31, 1999) as Coral Security applications task leader. Andrew Gross also left UCSD in August of 1999 for career reasons (see quarterly report dated October 31, 1999). Work on Task 3 is continuing under Ken Keys and David Moore's leadership, however, progress remains slower than anticipated. We have now assigned Ryan Koga, a CAIDA employee, work on the Security task.

For these reasons, we are requesting a one year no-cost extension to complete work under this award.

7.3 Issues resolved:

Incremental funding for Fiscal Year 2000 for this award in the amount of $819,538.00 arrived at UCSD in March, 2000.

Funding for Option 1 of the DARPA-NGI award, including visualization of massive datasets and the DNS root server initiative, was sent by DARPA to SPAWAR for official activation in March. The funding, totaling $488,615.00, arrived at SDSC and was obligated in early April. **Now that this funding is obligated, all work on the DNS root server initiative and visualization of massive data sets will be moved from Task 2 to Option 1.

8.0 Near-term Plan

The material below reflects the activities planned during Year 2, Quarter 4 of this project, April 1, 2000- June 30, 2000. It is organized according to the categories identified in the Project Program Plan (see https://www.caida.org/NGI/progplan98.html).

A.General/Administrative Outreach and Reporting

The following Administrative Outreach and Reporting items are planned for Quarter 4:

-Dr. Claffy will be featured as the NGI distinguished lectured on April 13, 2000 at the National Center for Supercomputing Applications Building in Arlington, VA, and will give a presentation entitled "Tracking a Metamorphic Infrastructure: observations on our (in)ability to predict, analyze, or even measure conditions on the global Internet", see https://www.caida.org/publications/presentations/Soa0005/mgp00001.html.

-Tracie Monk and Claffy will meet with DARPA Program Manager Mari Maeda on April 13, 2000, in Washington D.C. to discuss NGI progress and status.

-Claffy, Monk, David Moore and Amy Blanchard will meet with Steve Spendlove of SPAWAR to discuss NGI progress and Status on April 24, 2000 at the San Diego Supercomputer Center.

-Claffy present "Tracking a Metamorphic Infrastructure: observations on our (in)ability to accurately predict, analyze or even measure conditions on the global Internet" at the TERENA conference in Lisbon, Portugal, May 22-25, 2000, http://www.terena.nl/tnc2000.

-Claffy and Evi Nemeth will give a presentation entitled "Internet State of the Union" on May 25 at the SANE 2000 conference in Maastricht, The Netherlands, http://www.nluug.nl/events/sane2000/daily/thursday.html.

-Submit Quarterly Report to SPAWAR covering progress, status and management

-Submit Quarterly Financial Status Report (UCSD Extramural Funds Dept. submits)

-Submit Quarterly Report of Federal Cash Transactions (UCSD Extramural Funds Dept. submits)

B. Task 1. Coral OC48mon

The following work for Task 1 is planned for Quarter 4:

-Complete testing of the DAG 4.0 OC48 capture card

-Complete design of the DAG 4.1

-Continue to refine the CoralReef requisite software suite, including the CoralReef Report Generator tool

-Waikato's David Miller will come to San Diego to do additional OC48mon testing at SDSC with CAIDA personnel

-Continue discussions of OC48mon development and use with the community

C. Task 2. Tomography Mapping/Modeling

Task 2 initiatives for Quarter 4 include:

-Deploy additional 1-2 skitter measurement hosts

-Continue to make skitter data available to third party researchers in the community

-Continue briefings to Internet community on purpose and initial results of skitter and solicit their inputs; prepare technical paper on initial results from analysis of skitter/tomographic data

-Enhance the skitter/Tomography analysis/visualization tools

D. Task 3. Security

Quarter 4 plans for Task 3 include:

-Testing and refinement of CoralReef's implementation of Bro security software

-Development of a white paper describing how to push security filtering operations into the firmware of a capture card

E. Task 4. Storage/Analysis

Task 4 focus for Quarter 4 includes:

-Expand collection/storage of data

-Enhance analysis code and reporting formats

-Continued in-depth analysis of skitter data and dissemination of results to the networking community

F. Option 1/DNS Root Server and Visualization of Massive Datasets

-Deployment of skitter host to be co-located with the "K" DNS root server maintained by RIPE in London

-Additional visualizations of massive skitter datasets

9.0 Completed Travel

The following travel occurred during Quarter 3:

-Claffy and Monk attended NANOG 18 February 7 and 8 in San Jose, California, http://www.nanog.org/mtg-0002/

-Amy Blanchard and Nevil Brownlee demonstrated CAIDA measurement and analysis tools at the Optical Fiber Communication Conference and Exhibit 2000 (OFC 2000) in Baltimore, Maryland, March 6-10, 2000, see http://www.osa.org/mtg_conf/OFC/

-Monk traveled to Anne Arbor, Michigan to meet with Daniel McRobb to discuss donation of Juniper equipment to CAIDA and to discuss NGI status and progress on March 20-23.

-David Moore traveled to Adelaide, Australia, to attend the Internet Engineering Task Force (IETF) on March 24, 2000. He continued on to the University of Waikato on Friday, March 31 to meet with the subcontractors for NGI Task 1, Coral monitors.

Other related travel that was not charged to this award:

-KC Claffy and Tracie Monk traveled to San Jose on January 19, 2000 to present to the Cisco University Research Program (URP) on CAIDA measurement, research and analysis activities.

-Nevil Brownlee presented to the XIWT/PERF group on February 24 in San Jose, California on the CAIDA Passive Measurement/Metrics Working Group, see https://www.caida.org/Presentations/iperf0002/sld001.html

-Brownlee, Claffy and Monk met with CAIDA members and members of the Internet engineering community at the AboveNet facility in San Jose for the first meeting of the metrics working group on Tuesday, March 2.

11.0 Work Focus

Task 1. Coral OC48 Monitors

Major work focus on Task 1 centered on completing the fabrication of the ATM compatible DAG 4.0 capture card and design of the POS/ATM DAG 4.1 capture card. We also spent time developing and improving the CoralReef report generator that is used for analyzing workload data at OC3-OC48 speeds.

Jawed completed fabrication of the DAG4.0 OC48 capture card in February, and returned it to the University of Waikato Development team in New Zealand for initial testing. An OC48 Broadband Series Test System on loan from Sprint was used to run preliminary tests. The prototype is operating, and was demonstrated at the Passive and Active Measurement Workshop held in New Zealand in early April. This prototype is ATM-only, and uses 32-bit 33 MHz PCI.

Development of the final DAG 4.1 prototype is approaching its final stages. This board will be POS and ATM capable, and will use a 64-bit 66 MHz PCI interface. The circuit design of this board was completed, and layout commenced. We believe that initial problems of component supply have been overcome, and that the next prototype will be ready for testing in June.

The original plan for the development of OC48 monitor hardware called for three prototype phases:

1. ATM only, 32-bit 33 MHz PCI

2. ATM/POS 32-bit 33 MHz PCI

3. ATM/POS 64-bit 66 MHz PCI

The design and board layout for the first prototype was completed in September of 1999, and the board sent to Jawed for fabrication. Delays in obtaining components, and some fabrication problems, resulted in Waikato not receiving the completed board until February 2000.

The industry has moved on since the original development plan, and there are no IP/ATM OC48 networks accessible for testing the OC48mon. This factor influenced our decision to forego live tests of this prototype and to move as rapidly as possible to the next stage of completing development of a POS capable device.

PCI bus technology advanced during the construction of the first prototype. We originally thought that it would be necessary to purchase a large server-class PC, costing more than $15,000, in order to obtain 64-bit 66 MHz PCI and a corresponding high memory bandwidth. However, a new chipset from INTEL, the 82840-QP memory controller hub, allowed the development of a new class of high performance desktop PCs with Rambus DRAM and very high speed I/O. We purchased two examples of the Hewlett Packard versions of these machines, the Kayak, at less than $4,000 each.

Xilinx also developed a new PCI kernel for its Virtex series of FPGAs, and we purchased a license to use this. With this kernel we should achieve data transfer rates of more than 400 Mbyte/second.

These developments made it possible to skip the construction of the second prototype, and to move directly to the third es.

Task 2. Tomography Mapping/Modeling

Tomography Task work during Quarter 2 centered on revision of the skitter destination list and release of additional skitter datasets to the research community.

A comprehensive CAIDA meeting was held on March 6, 2000 to discuss and conclude issues concerning establishing a final skitter destination list(s). Multiple needs and categories were defined, and the following topics were discussed: Anomalies, Dynamics, Geopolitics, Routing, Performance, Topology, Trends, Location, and Validation.

Paul Barford of Boston University is nearing completion of his research using multiple skitter datasets. Under the direction of Assistant Professor John Byers, they are using the skitter datasets to 1) help identify heuristics which can classify links as being part of a backbone or being outside a backbone and 2) devising efficient strategies for mapping backbone links with minimum end-to-end measurements. Results on this research are expected next quarter and will be reported in the July 31, 2000 Technical report.

We are preparing to upgrade the versions of FreeBSD that currently exist on skitter machines in the field from version 2.2.5 to version 3.4.stable in order to allow for compatibility between current versions of skitter software and the OS. Upgrades will be done remotely by CAIDA staff during Quarter 4.

An animation demonstrating how skitter works was completed, and is available at http://flicks.caida.org/skitter_animation/.

DNS Root Server/Visualization of Massive Datasets

Both "L" and "F" root servers are collecting data with preliminary analyses available at: https://www.caida.org/analysis/content/skitterdaily/.

Preparations are currently underway to send a skitter monitor to the "K" root server in London, maintained by RIPE.

Visualization of Massive Datasets

A visualization of core Internet Autonomous Systems was created during Quarter 3, see https://www.caida.org/analysis/content/visualization/as_core_network/.

This visualization represents a macroscopic snapshot of the Internet for sixteen days in mid-January 2000. The graph includes 220,533 nodes obtained by merging three datasets: one from Bill Cheswick on January 14, 2000; one gathered from traceroutes to Cheswick destinations from CAIDA's network in San Diego during January 15-22, 2000; and the last using data from 14 skitter monitors taken on January 08, 16, and 24, 2000.

The view of the network is then abstracted into a topology of Autonomous Systems (ASes), each of which approximately maps to an Internet Service Provider, or "ISP". This abstraction occurs by mapping each IP address to its best-match origin AS in Border Gateway Protocol (BGP) routing tables collected by the University of Oregon's

RouteViews project. The abstracted graph consists of 5107 AS nodes. Finally, we remove "non-core" ASes by recursively removing all ASes with an outdegree of 0, as well as removing any two ASes that connect only to each other ("leaf" nodes), until no further nodes can be removed. This leaves a set of well-connected ASes, which we label an "AS core" snapshot. The final AS core has 1516 nodes, or 29.7% of the total ASes seen in the data sets.

The position of each AS node is plotted in polar coordinates. The "outdegree" of an AS reflects the number of other ASes that accepted traffic from that AS.

Thus, the closer an AS node is to the center of the graph, the richer is its peering. Radii of the circle naturally cluster ASes into approximate geopolitical areas. An intuitive way to consider the geographical structure in this graph is as a view of the Earth from the North Pole. The large white "pie slices" map to oceans or other non-populated areas of the globe.

By graphing dimensions of peering richness versus geographic information, the graph reveals the "highly core-connected" nature of ASes based in the United States. The top 15 ASes are all headquartered in the U.S. Also, while both Europe and Asia have many peering relationships with the U.S., there are far fewer links between Asia and Europe.

Task 3. Security

Task 3 efforts centered on porting Vern Paxson's Bro security software to the CoralReef software suite and development of a white paper.

CAIDA remedied the problem of being short-staffed on the security task by assigning CAIDA staff member Ryan Koga to work on Task 3. Koga focused of porting Vern Paxson's Bro security software to the CoralReef software suite during Quarter 3. Bro is a stand-alone system for detecting intruders in real time by passively monitoring the link over which the intruder's traffic transits. It focuses on monitoring FDDI rates, with provisions for real-time notification, clear separation between mechanism and policy, and extensibility, see http://www-nrg.ee.lbl.gov/nrg-papers.html.

We are exploring a potential collaboration with SDSC's Operational Security Group to do additional security testing on the CoralReef's implementation of Bro during Quarter 4.

A white paper is under development around the topic of pushing security filtering operations into the firmware of a capture card. The white paper should be completed by the end of Quarter 4.

Task 4. Storage/Analysis

Task 4 efforts during Quarter 3 focused on backing up skitter data stored on the raid array, a synopsis of trends in Internet traffic, and in-depth analysis on the dynamic and static properties of skitter data.

New disk space, controllers and memory were purchased for the DEC raid array that stores skitter data, and the .25 Terabyte of existing skitter data was backed up on SDSC's HPSS storage system. This data back up allowed us to add new disk space to the raid array without worrying about compromising the existing data. There is now enough disk space to continue collecting from all 17 skitter machines currently deployed in the field and to collect data from new skitter machines co-located at additional DNS root-server sites.

CAIDA provided DARPA program manager Mari Maeda with the following synopsis on Trends in Internet Traffic for publication at DARPA during Quarter 3:

"Trends in Internet Traffic Point to Rapid Growth of Napster, Declines in Some Other Internet Applications: Networking researchers at the San Diego Supercomputer Center's Cooperative Association for Internet Data Analysis (CAIDA) report a marked increase in Internet traffic associated with a new music application known as Napster, topping 4% of total bytes measured at the Ames Internet Exchange (AIX) facility in March 2000 (double January's levels). The traditional bulk transfer protocol, FTP, continues its steady decline in proportion to other traffic at AIX, with FTP dropping from roughly 6% of traffic bytes in June through October 1999 to less than 4% in March 2000. RealAudio traffic also is declining at this site, representing 1% of total traffic bytes, less than half of its June 1999 levels. Strong periodic behavior patterns are associated with certain applications, including a near doubling of on-line game traffic on weekends and high email levels in December. Packet sizes at the AIX exchange point seem relatively stable, averaging about 400 bytes; the TCP protocol continues to represent more than 80% of total Internet packets, see https://anala.caida.org/AIX/.

Characterization of Internet traffic is becoming more difficult as link speeds increase and technology evolves. NASA AIX's upgrade this week from OC3 ATM to OC12 POS, for example, temporarily leaves researchers without a window into core Internet traffic patterns. DARPA is funding CAIDA to develop monitors capable of analyzing traffic at OC48 speeds and real-time detection of some security threats at OC12 speeds."

CAIDA researcher Andre Broido performed in-depth analysis on the static and dynamic characteristics of skitter data including analysis of power laws, distances and hop counts, path run lengths, and load balancing.

Power Laws: Power laws, the mathematical concept of inverse proportionality, help demonstrate the concept of network topology "fan-out". This information is potentially important to developers building large-scale network simulators. Simulators should be designed to have the same properties as real networks, and analysis of power laws allows us to emulate this. Fan-out also demonstrates the richness of an ISP's connectivity at different points in the Internet, a factor important to customers.

Distances and Hop Counts: We are doing comparative work on shortest path distances on topology graphs created from data collected from skitter and from Bill Cheswick's traceroute database. This work is intended to help analyze how useful subsets of topology data are to assumptions about the global infrastructure. Shortest Path Distance provides a conceptual framework for analyzing the infrastructure; it does reflect routing policy or routing tables.

Path Run Lengths: Skitter does multiple probes of each destination on the destination list each day, and each of these probes could result in either the same forward path or multiple forward paths being taken (data on reverse paths are not available through skitter). Multiple forward paths could be caused by changes in routing, load balancing or failures in the network. Examination of the path run lengths is important because it is an indicator of route stability.

Load Balancing: Three years ago, research by Vern Paxson found that the number of paths being dynamically load-balanced was relatively small. This assumption is being used in many of today's simulators. Given changes in the commercial Internet, we feel that it is important to revisit this assumption and attempt to quantify the extent to which load-balancing is being used by Industry.

11.2 Significant Events

A paper entitled "Measurements of Internet Topology in the Pacific Rim Countries" concerning CAIDA's DARPA research in this arena was accepted by INET2000 to be held July 18-21 in Japan.

Tracie Monk made a presentation entitled "Predictability and Security of high Performance Networks" about CAIDA'S DARPA project to the Science Applications International Corporation (SAIC) at the Marriott Hotel in La Jolla on January 12.

Dr. Claffy was named one of the top 25 Woman of the Web by the San Francisco Women of the Web society, and attended the award presentation ceremony in San Francisco on January 31st.

Publications:

The skitter hypview animation was used in a short video for an exhibition in the museum

"la Citi des Sciences et de l'Industrie" in Paris, France. The main goal of this work is to show how the Internet and related networks can be considered as new territories under construction.

FINANCIAL INFORMATION:

Contract #: N66001-98-2-8922

Contract Period of Performance: 16/07/1998 to 15/07/2000

Ceiling Value: $6,655,449

Current Obligated Funds: $2,971,812

Reporting Period: 01/01/00 - 03/31/00

Actual Costs Incurred:

Current Period:

UCSD

Labor Hours 4169
Cost $ 141,111.01
ODC's Cost $ 13,306.00
IDC's Cost $ 88,273.00
Waikato Cost $ 40,579.70
TOTAL: Hours 4169
Cost $ 283, 269.00
Cumulative to date:
UCSD Labor Hours 18532
Cost $ 631,735.00
ODC's Cost $ 471,411.00
IDC's Cost $ 369,097.00
GA ( $ 84,102.23 included above)
Waikato ( $ 119,689.70 included above)
TOTAL: Hours 18,898
Cost $ 1,472,243.00
Note: additional financial information in tabular form, including breakdown by subcontract and estimated expenditures for Quarter 8, is attached to this report.