NSF Annual Report: Year One

Internet Engineering Repository

Annual Progress Report, August 15, 1997 - July 31, 1998

NSF Award Number NCR-9706181

Kimberly C. Claffy, PI

CAIDA's IEC Repository is based at the University of California's San Diego Supercomputer Center (UCSD/SDSC).

I. Introduction
II. IEC Repository
- Design of the Site
- Automated Tools for Consistency and Maintenance
- Population of the Site
- The Hard Parts
III. Outreach and Advertising
- Workshops
- University Survey
- Presentations
IV. Staffing
V. Futures
- Scheduling Workshops
- Course Review Process
- Advisory Committee Meeting
- Distributed Students Experiment
- Additional Content
VI. Attachments
- A - Course Cover Page Template
- B - Sample Navigational Frame
- C - University Letter and Survey Form
- D - Vendor Equipment Support Letter
- E - Distributed Course Annoucement
- F - Sample Concept Graph

I. Introduction

The Internet Engineering Curriculum Repository (IEC) is a project of CAIDA (Cooperative Association for Internet Data Analysis) in conjunction with NLANR (National Laboratory for Applied Network Research) to help educators and others interested in Internet technology keep up with developments in the field.

Motivation

The technology surrounding the Internet is so new and is changing so fast that many Universities do not have networking courses and those that do find it hard to keep the courses current. Often faculty were trained before the Internet was so pervasive. Most networking textbooks are at least 3 years out of date. The IEC repository is a collection of teaching materials from University courses, vendor training materials, tutorials, etc. that have been contributed by individual authors. These authors include regular University faculty, industry research labs personnel, training professionals, and even the occasional MIT freshman dropout who is deep in the trenches making the Internet work. The repository is intended to provide a resource to University faculty and others designing networking courses for their institution.

Mechanism

A repository of teaching materials doesn't independently produce new University courses or trained network engineers. We will hold workshops in order to facilitate new faculty's use of the repository. The workshops will be in Boulder, Colorado in the summertime and in San Diego, California in the winter beginning in early 1999. University faculty are not the only potential customers of this repository; engineers seeking re-training or wishing to keep current in this fast moving field will also benefit both from the materials and from workshops. Additional workshops targetted at industry engineers and training specialists will be held if demand exists.

The Repository

The repository contains a collection of links to University course materials, conference tutorial materials, research reports, white papers, tech reports, FAQs, and other online information related to networking.

In addition to these raw teaching materials, we have added:

an index of materials present
a profile of each course
- instructor
- textbook
- pre-requisite knowledge required
- difficulty level
- duration
- description
a navigational tool for easy browsing

This annual report represents a clarification and re-focusing of IEC efforts from the original description in the proposal. In the sections that follow, we discuss not only what we have accomplished in the IEC repository project this past year, but also our vision for the future. The next two sections detail the repository and our outreach efforts. That is followed by a short section on staffing and then our future plans both in terms of new features in the repository and new ideas for its use.

II. IEC Repository

In building the IEC Repository, we feel that our mission is not to create materials, but to collect, index, and review existing materials and then do technology transfer by putting them into the hands of educators and users. With this in mind, our design process has concentrated on a uniform look for each course, a quick way to browse a course, ways to automate the addition of courses and maintenance of the site.

Design of the Site

The key ideas behind our design are to leave the raw course materials located at their original site and not modified in any way. This means that the author does not need to undertake any further effort for their materials to be included in the repository.

To each course in the repository, we add a frame based cover page that includes a navigational frame and an index frame. The index frame contains the following headings:

Course:
Instructor:
Department:
Institution:
Textbook:
Level:
Duration:
Pre-Requisites:
Offered:
URL to Course Home Page:
Course Description:

An instructor looking for relevant material can quickly filter out the courses that are not the appropriate level of difficulty or duration. A typical cover page is included as Attachment A; it is for Steve McCanne's graduate course at Berkeley. The navigational tool facilitates browsing of the course site; shortcuts to lectures, labs, homeworks, etc. have been added where possible. Attachment B is a sample navigational frame for the course by Henning Schulzrinne from Columbia University. A bit of information is lost in printing from Netscape; online, there are several levels of hierarchy indicated by indentation.

In general, materials will remain on their home machine and only be mirrored at the IEC repository if bandwidth is an issue or if an author requests it. Authors whose materials evolve from semester to semester might want to use the IEC as a stable archive of the earlier courses. An author on sabbatical or changing jobs might need a home for course materials.

Automation and Maintenance

We have built cgi-bin scripts to automate parts of the construction of the site. This ensures uniformity of the index cover page and navigational frame for course materials and allows us to separate the task of reviewing a potential course from the task of entering it into the archive. We intend to build more such scripts for accepting reviews from authors and users.

We will run the usual fleet of tools to check HTML syntax and find dead links on a regular basis. Authors will be notified of bad links in their course materials.

Populating the Site

The original IEC repository concept site is iec.caida.org. We have built a parallel site at iec.caida.org/iec.evi that includes the new design described above. It is populated with sample University courses and industry workshops. We have used this prototype to explore different formats and finalize our design.

Over the next month we expect to add many more courses to the site. Our editorial committee will select the courses and student interns will enter the materials using an automation script.

Challenges

We see several challenges in the development and maintenance of the IEC Repository:

keeping the site current
making the navigation frame consistently useful
getting vendors to contribute their training materials
providing consistent review (editorial) of courses

To keep the site current we need to worry about not only stale links, but also the fact that the wealth of information about the network is growing almost as fast as the net is growing. Course material becomes obsolete very quickly, and the situation is exacerbated by instructors going on sabbatical, rotating teaching schedules, or being lured into higher paying industry positions.

The navigational frame is a set of hierarchical links to lectures, homeworks, labs, etc. within the course itself. Some courses accommodate this structure well, others map less gracefully into the navigational index. For example, in the prototype archive, Steve McCanne's course does not work well while Henning Schulzrinne's course fits naturally. In these cases we intend to show the authors the navigational tool, in case they want to use it to structure their course materials.

Many vendors (eg. Cisco) have wonderful training materials that are used for their public and private classes. We would like to include these materials in the archive, but so far have been unable to get approval from the companies. We are also approaching some individuals who teach tutorials professionally with requests to include their notes in the repository. This approach has shown more promise than dealing with companies.

Our final item on the list is the problem of providing consistent and correct reviews of the courses. The key to the usefulness of the archive will be its utility to University faculty seeking relevant teaching materials at an appropriate level of difficulty for their students. The cover page and reviews of a course will give faculty a quick feel for the course. As the archive grows and users must peruse more courses, it becomes essential that this quick summary provides an accurate snapshot of the course.

Outreach and Advertising

One of the deliverables for Year 1 of the IEC Repository grant is to solicit course materials for the archive and to advertise its existence. We have done three things toward this end:

designed technology transfer workshops
surveyed Universities for contributions
made presentations at various conferences

We will describe each in more detail.

Workshops

Using someone elses teaching materials is not easy, especially if the lectures are Power Point style outlines of intented content. To improve the technology transfer aspects of the IEC Repository, we will hold workshops to help faculty wishing to build networking courses for their institution learn to use the materials in the repository. The authors of the materials serve as the workshop faculty (whenever possible) and the University faculty and their Teaching Assistants will be the students. There is a bit of funding in the IEC to provide financial support for US University attendees and honoraria for instructors. We envision the workshops as being one week long and covering about 10 discrete day-long topics. Attendees would choose one topic per day and attend lecture and lab sessions on the topic itself, the materials supporting that topic, and typical homework assignments. Some topics may require two days to cover properly. We feel that a lecture/lab format will maximize the amount of technology transferred. Typical topics might be: using the ns simulator, routing including a lab with Cisco equipment, or IP/ATM interaction.

University Survey

We sent a letter in August 1998 introducing the IEC Repository project to over 100 Universities in the US surveying their networking courses and laboratories and asking for faculty contacts. From this we hope to get both Universities who would provide their course materials, and Universities who do not yet have networking curricula and would be customers of the archive and our workshops. Copies of the letter and the survey are included as Attachment C.

We also intend to solicit vendors to donate equipment to be used in the courses and perhaps be sent back to participating institutions to build laboratories. Some vendors have upgrade programs whereby older equipment is returned and credit is given toward newer replacement equipment. Usually these vendors then use the trade-ins for spare parts, or crush them and recycle the metal. We are discussing with vendors various ways to put this older equipment in University teaching laboratories rather than recycling as scrap. We are volunteering the IEC's services to coordinate the donation and distribution of some of this equipment.

Presentations

Evi Nemeth and Tracie Monk have both given presentations about the IEC Repository. A partial list of recipients follows: the NLANR I2 meeting at UCSD in January, APRICOT (the Asia Pacific version of NANOG) in Manila in February, and the Usenix conference in June. We hope to get a short slot at the upcoming IETF Plenary since that meeting is rich with potential content contributors.

Staffing

The project was funded in August, 1997 and expected to gear up by fall. However, the project coordinator, Evi Nemeth, could not join the effort until January, 1998 due to other committments at her home institution, the University of Colorado. To make up for the late start we hired two summer interns, Alex Ma and Mary Kalra, both undergraduate computer science students at UCSD. Alex is building the cgi-bin scripts to automate entering courses and reviews; Mary is finding materials on the web for inclusion in the archive.

In addition to staff, we intend to have an advisory board and an editorial board. We have made informal contact with many well-known folks in the networking world with mostly positive results.

Futures

We have several activities planned for the near future or just getting underway. Some are directly related to the IEC Repository and some are only peripherally involved.

Scheduling Workshops

We would like to hold the first workshop January, 1999 in San Diego. This would be a prototype with only 2 or 3 days of instruction and a smaller group of faculty-students. We are targeting the time after New Years and before many semester-based schools begin again.

Course Review Process

The review process has not yet been formalized. We would like to solicit a few reviews of individual courses from the community and then use those reviews to structure and design our system for providing consistent, timely reviews of course materials.

The course materials get an initial cursory review when they are scanned for quality and completeness prior to their inclusion in the archive. Once entered into the archive, they are assigned a difficulty level. In addition to this process, we would also like to include reviews by the authors themselves, by our editorial board, and by any faculty who use the course materials. We will build a web form to make it easier to submit reviews.

Advisory Committee Meeting

We are in the process of forming two advisory committees, one for the entire IEC Repository and one for editorial/review of materials. We would like to schedule an advisory committee meeting within the next 6 months.

Distributed Students Experiment

This fall, the computer networks class (CSE 123a at UCSD, CS 4273/5273 at Colorado) will be taught using the Internet to students enrolled at both the University of California, San Diego and the University of Colorado, Boulder. The instructor, Evi Nemeth, is on sabbatical leave at San Diego this year and both the department at San Diego and the department in Boulder thought she would be at their site for the fall. The result of this double-booking is an experimental course using distance learning via the Internet with the multicast video conferencing tools under UNIX. Both the San Diego Supercomputer Center and the University of Colorado are on the vBNS, a fast (OC3 or 155mb/sec), lightly-loaded research and education network supported by the National Science Foundation. The course announcement that was transmitted to students at both campuses is included as Attachment E.

The first 5 weeks of that 10 weeks of instruction, Evi will be in San Diego and will teach live to California students and remotely to Colorado students. The second 5 weeks she will teach from Boulder to Colorado students and remotely to San Diego students. Tests will be given before the course starts, and at the end of each 5 week period. We hope to learn just how important it is to have the instructor participate in person in the classroom.

Sessions will be recorded digitally and saved on disk for students to replay if they miss a lecture or want to review a point. This will also be used in advance to cover lectures when the instructor is out of town. We would like to put these lectures in the IEC Repository as a prototype for general distance learning delivery and use the log file to determine how often students access the materials for review.

Additional Content

There are several additional items that we would like to include in the repository for each course:

a concept graph representing the pre-requisite structure of a course
a concept graph of the whole collection
an annotated bibliography

The concept graph was introduced to us by Prof. William M. Waite at the University of Colorado and Rosemary Simpson from Brown University. It is a representation of an index for a course that contains knowledge about the lecture materials, the textbooks, and the hierarchical structure of the concepts. The nodes of the graph represent concepts or topics in the course and the edges and represent component relationships. For example, an edge from concept A to concept B means that concept B is one of the sub-concepts making up concept A.

We would like to use this concept but modify it to express "pre-requisite" relationships, rather than the "contains" relationship. Hopefully, the resulting graph is close to acyclic! A concept graph of the entire collection helps the user browse its contents quickly.

The CAIDA network visualization project includes a java based tool called Otter, which is capable of displaying nodes and edges of a graph and coloring by attributes of those objects. In addition to the concept graph described above and shown as Attachment F, we would like to use Otter to display course data as well.

Follow-on Projects

One of the items mentioned in the original proposal but not included in the budget is the idea of producing "Readers Digest" versions of the IETF working groups progress. This idea has proven extraordinarily useful in the USENIX Association, which has for many years used a system of "snitches" who reported on the progress of various POSIX and X-Open standards committees. IETF meetings would be ideally suited for such a model for articulating such "cliff notes" and archiving them on both the ietf.org site and the IEC Repository. Although beyond the scope of the existing IEC proposal framework, we are investigating the possiblity of engaging resources to pursue such a program from interested IETF sponsors.