arts++

ARTS is a binary file format specification for storing network data. Initially developed at ANS by David Bolen in 1992, ARTS was licensed to CAIDA in March of 1998.

CAIDA has developed a C++ class library for ARTS. This work is separate from the software licensed from ANS. The C++ class library is used by the CAIDA packages cflowd and skitter. In addition to the class library, CAIDA distributes some simple applications for viewing and manipulating ARTS data. The entire package is called arts++.

ARTS Download

The most recent version of the ARTS class library is available for download.

ARTS Data

The following types of data are stored in ARTS files by CAIDA software:

forward IP path
This data type is used by skitter. You can think if this data type as holding the results of traceroute, minus the round-trip times for each hop. This data type does contain the round trip time from the source to the final destination, but only stores the IP addresses of the intermediate hops.
AS matrix
This data type is used by cflowd. It contains packet and byte counters for traffic from source ASes to destination ASes.
net matrix
This data type is used by cflowd. It contains packet and byte counters for traffic from source networks to destination networks. Networks are identified by their address and netmask.
port matrix
This data type is used by cflowd. It contains input and output packet and byte counters for traffic from each source transport port number to each destination transport port number.
protocol table
This data type is used by cflowd. It contains packet and byte counters for each IP protocol (TCP, UDP, ICMP, et. al.).
interface matrix
This data type is used by cflowd. It contains packet and byte counters for traffic from source (input) interfaces to destination (output) interfaces on a router.
nexthop table
This data type is used by cflowd. It contains packet and byte counters for traffic to each IP nexthop for a router.
TOS table
This data type is used by cflowd. It contains packet and byte counters for each IP TOS (from the TOS field in the IP header).
RTT time series
This data type is used by skping. It contains RTT and packet loss data for a single source/destination pair.

NOTE: the author often refers to an 'ARTS data object'. In most contexts, the author is referring to the data stored in ARTS files and *not* an instance of one of the ARTS C++ classes. This distinction is important and historical: the original ANS ARTS software was written in C.

ARTS Data Commonality

ARTS data objects are generally composed of three parts: a header, a list of attributes and a data section. The data section is specific to the data object type, while the format of the header and attributes are common to all ARTS data objects.

All ARTS data objects contain a header section. The header identifies the type of data contained, the version number of the object, the number of attributes in the object, and the length of the data portion of the object.

All ARTS data objects may contain a number of attributes. Attributes can be used to attach common information to a data object in a common format. There is a set of pre-defined attributes commonly used by CAIDA applications. These include a host attribute (an IP address), a creation time attribute (seconds since the UNIX epoch), and a period attribute (two timestamps defining an interval in time, both in seconds since the UNIX epoch).

ARTS Data Collection

ARTS data is collected and archived at regular intervals. In the case of cflowd, a program named cfdcollect is responsible for this activity. In the case of skitter, a program called skcollectd performs this activity.

The result is time-series data for each of the data types listed above.

General Functionality

efficient data archival
One of the primary goals of the ARTS file format is storage space efficiency. When faced with a tradeoff between disk space and performance, ARTS leans toward disk space efficiency. The reason for this is fairly simple: we wish to permit the storage of large datasets without undue disk space requirements.
In most cases ARTS is also fairly CPU efficient for a given operation, but the author admits to preferring clean code leveraging existing mechanisms (for example, the Standard Template Library) over special code written only for the sake of optimization. This makes the code easier to maintain and port to different flavors of UNIX, and typically makes it easier for application developers to use the class library.
aggregation in the time domain
The C++ class library has support for aggregating each of the following data types in the time domain:
- AS matrix
- net matrix
- port table
- port matrix
- protocol table
- interface matrix
- nexthop table
- TOS table
There is currently no support for time domain aggregation of the IP path object. However, this functionality will be added in the near future.
There are simple applications in the arts++ package that can perform time domain aggregation for supported data objects. These are artsasagg, artsnetagg, artsportmagg, artsprotoagg, artsintfmagg and artsnexthopagg.
version-specific formats
Some ARTS data objects may support more than one format for storage. While the C++ class library currently only supports a single format for each data object, it is possible to support different versions of each data object with changes that should be relatively painless to application developers. Currently where a version number is required, the library uses arguments with default values in the member functions.
support for iostreams and UNIX file descriptors
The class library supports reading and writing from iostreams as well as UNIX file descriptors. The latter may be used for client/server applications using the BSD socket interface.

ARTS Download

ARTS Data

ARTS Data Commonality

ARTS Data Collection

General Functionality

ARTS References