These instructions are intended to help you start using the Endace DAG cards. They are not endorsed by Endace, and if you're stuck then you should refer to their manuals or contact their technical support people (who are quite helpful). These instructions are based on the DAG7.5G4 PCI Express card.

Interface Modules (SFPs)

Endace only support the Finisar SFPs. Others do work, and I've used a Cisco 10/100/1000 SFP that was borrowed from the IT department.

The DAG cards do not support 100BASE-FX SFPs -- the only fibre interfaces supported are 1000BASE-X. If you need to monitor 100Mb/s fibre links then you will need a media converter, but any timing measurements need to take into account the latency of the media converter (which are often two port switches).

Basic Capture

If all you want to do is capture frames then set the card up as receive only, fix the ports to the correct rate and you're done. Most of the time auto-negotiation of duplex works fine, but things do get messy with taps and half duplex devices.

dagconfig default Resets the card to default values

dagconfig 100 rxonly mem=128:0 Sets all ports to 100Mb/s and allocates all the memory to a single receive stream

Now it is time to capture some frames. dagsnap does this for you, but a trap is that it can also store data that has arrived in the meantime. The dagbits command can be used to clean the buffer out first.

The verbose (-v) option of dagsnap shows the total accumulated data, how much of the buffer memory is used and the incoming data rate (in that order). The recording time is specified with the --s option and output file with the --o option. A 15 minute capture to example.erf would be:

• dagbits –d0 –cv –S2 Runs for two seconds and clears the buffer
• dagsnap –s 900 –v –o example.erf Runs for 900 seconds and saves into example.erf

Viewing Files

Wireshark is a very easy way of viewing ERF files since it supports them directly. Converting the ERF file to a PCAP file with dagconvert reduces the time stamp accuracy. If you capture from multiple ports at once all the frames end up in one file, but the receive interface is stored in the ERF header. Adding a custom column to Wireshark that has the custom field erf.flags.cap makes it very clear where the frames came from. Colour coding based on the interface also helps.

Filtered Capture

Sometimes there is more happening on the network than you care about. DSM filtering is a nice way of doing this, but you need to use DSM firmware. The factory firmware in the DAG7.5G4 is edag75g4pci_dsm_v2_2, and this supports DSM. The latest version I used was edag75g4pci_bfs_v2_4 and this doesn't support DSM. If you have upgraded to a new version and want to enable DSM then run dagreset.

A DSM filter file is an XML file with special tags. Check the Endace documentation (EDM04-07 dsm_loader User Guide) for details. I want to capture data only if the Ethertype is 0x88BA, so my filter file is:

    <?xml version="1.0"?>
    <dsm-config version="1.0">
        <!-- SV filter -->
        <filter>
            <name>filter0</name>
            <number>0</number>
            <ethernet>
               <ethertype hex="true">88BA</ethertype>
            </ethernet>
        </filter>
        <filter>
            <name>filter1</name>
            <number>1</number>
            <ethernet-vlan>
                <ethertype hex="true">88BA</ethertype>
            </ethernet-vlan>
        </filter>
        <partial>
            <name>partial0</name>
            <partial-component>filter0</partial-component>
            <partial-component>filter1</partial-component>
        </partial>
        <stream>
             <number>0</number>
             <stream-component>partial0</stream-component>
        </stream>
    </dsm-config>

Two filters are needed to ensure frames that are tagged with 802.1Q are also captured. Filters can be set up for particular source or destination addresses too.

The capture process is just like a simple capture, but dsm_loader is run first

    dsm_loader -f filter.xml dagbits -d0 -cv 
                     -S2 dagsnap -v -s 15 -o capture_file.erf

The only way of going back to the user/updated firmware is to reboot the computer.

Transmitting Data

The DAG7.5G4 is capable of transmitting and receiving data at the same time. There are some extra setup steps required though. These involve setting up the buffer memory and disabling receive for an interface or two. The data to be transmitted needs to be prepared properly.

The DAG card transmits ERF files, and this means it can transmit from multiple interfaces. The simplest way is to replay a captured file, but you can also convert a PCAP file to ERF and specify which interface should be used. A trap is that the ERF data needs to be aligned to 64 byte boundaries, and this is managed by dagconvert. I haven't replayed ERF captures, so this example will take some synthetic PCAP data.

To convert input_file.pcap to output_file.erf, with interface 1 (the second port) to be used, run:

    dagconvert -i input_file.pcap -o output_file.erf -T pcap:erf -A 64 -p 1

The maximum buffer memory is 128MB, and this needs to be shared with receiving too. The DAG card is very efficient, and even 1MB of receive buffer is enough to capture 100Mb/s data on two ports (total of 200Mb/s going to the disc). The maximum ERF file size I've reliably been able to transmit is 125Mb. There may be tricks to work your way through a larger file, but I haven't had much luck. To setup the card to receive on interface 0 and transmit on interface 1, both at 100Mb/s and with the transmitted data to follow the timestamps of the ERF file run:

    dagconfig default
    dagconfig enablea disableb disablec disabled
    dagconfig 100 auto_neg rxtx mem=1:127 relative

To transmit and receive, you will need two console windows.

On the transmit one run

    dagflood –f output_file.erf –v –r2

On the receive window do your normal capture, such as:

    dagbits -d0 -cv -S2
    dagsnap -v -s 15 -o capture_file.erf

It is a good idea to wait until the dagbits command has finished before running the dagflood command. I give the dagsnap an extra 10 seconds or so, I wait until it has started capturing some data and then start dagflood.

Complicated Setup

To capture data on interface 0 at 1000Mb/s, capture on interface 1 at 100Mb/s and transmit on interface 2 at 1000Mb/s the following commands would be used:

    dagconfig -d0 relative rxtx mem=1:127 enablea enableb disablec disabled  slen=1600 varlen
    dagconfig -1 1000 auto_neg
    dagconfig -2 100 auto_neg
    dagconfig -3 1000 auto_neg

In the receive window:

    dsm_loader -f filter.xml
    dagbits -d0 -cv -S2
    dagsnap -v -s 15 -o capture_file.erf

In the transmit window:

    dagflood –f output_file.erf –v –r2

This runs for two seconds and clears the buffer.