Frequently asked questions about Monarch

How do I use Monarch to emulate a non-TCP flow?
What format does Monarch use for its trace files?
Can I run multiple instances of Monarch concurrently?
Why does Monarch report 'unexpected' packets on the TUN device?
Why does the IPID sanity check fail whenever I use the tcp-ack probe?


How do I use Monarch to emulate a non-TCP flow?

You need to run Monarch in standalone mode. In this mode, Monarch does not generate its own TCP flow; instead, it just creates a TUN device and listens for packets to intercept. However, it is necessary to tell Monarch the exact IP addresses and port numbers of the packets it should intercept. This is done using four command line options: --sender and --virtual-receiver specify the source and destination address of the packets that are sent by the sender, and --virtual-sender and --receiver specify the source and destination address of the packets that are sent by the receiver.

Here is an example: We first pick an unused subnet (say 10.0.0.0/8) and then choose two IP addresses from that subnet: One (say 10.0.0.1) for the local host, and another (say 10.0.0.2) for a 'virtual' host that will be the communication partner of the local host. Neither of these two addresses must currently be assigned to any host. Of course, we also need the address of the remote host where Monarch should send the probe packets (say foo.bar.net). Next, we use these addresses to configure Monarch in standalone mode as follows:

      ./monarch -m standalone -i eth0 -o auto -k auto --sender 10.0.0.1:5001
          --virtual-receiver 10.0.0.2:5002 --virtual-sender 10.0.0.2:5003 
	  --receiver 10.0.0.1:5004 foo.bar.net &
Monarch will create a TUN device, bind it to 10.0.0.1, and then wait in the background for packets to intercept. Now we can start the sender and the receiver:
       ./my-sender-app --bind-to 10.0.0.1:5001 --send-udp-packets-to 10.0.0.2:5002 --num-packets 100 &
       ./my-receiver-app --bind-to 10.0.0.1:5004 --receive-udp-packets-from 10.0.0.2:5003 &
Once the transfer has completed, we can terminate Monarch (killall -SIGTERM monarch), which will cause it to write the trace files and to perform self-diagnosis as usual.

You may wonder why it is necessary to have two IP addresses; could Monarch not simply be configured with --sender 10.0.0.1:5001 and --receiver 10.0.0.1:5004? The answer is that in that case, Monarch would not be able to intercept any packets, since the kernel would recognize 10.0.0.1 as a local IP address and simply deliver the packets, instead of routing them to the TUN device. Note that the sender and the receiver must communicate using either TCP or UDP, since Monarch relies on port numbers to identify the packets it should intercept.

What format does Monarch use for its trace files?

The standard Monarch trace file (the one that is created with the -o option) contains all packets Monarch has sent or received, including probe and response packets and any packets intercepted from the sender or the receiver. If you choose the PCAP format (-t pcap), the trace file can be read with standard tools such as tcpdump. The MOUT format (-t mout) contains essentially the same information, but uses a textual representation and partially decodes the packet headers. This is useful for script processing.

The kernel trace file is created with the -k option and is only available in flow mode. It contains periodic snapshots of several TCP state variables, such as the congestion window or the kernel's RTT estimate. Each file contains a header that lists the names of all the state variables, which correspond to the definitions in /usr/include/linux/tcp.h. For the exact meaning of these variables, please refer to the Linux kernel documentation.

Can I run multiple instances of Monarch concurrently?

Yes, but you cannot run multiple concurrent flows between the same pair of hosts. The reason is that each Monarch instance uses the source and destination address in the IP header to filter out responses to its own probe packets. If two instances on the same host were to use the same destination address, they would receive not only their own response packets, but those of the other instance as well.

Why does Monarch report 'unexpected' packets on the TUN device?

Monarch creates a TUN device to intercept packets between the TCP sender and the TCP receiver (or, if in standalone mode, between two other protocol instances). Normally, these should be the only packets that appear on the TUN device. If Monarch intercepts a packet that comes neither from the TCP sender nor from the TCP receiver, it ignores that packet but prints the above warning message. In most cases, the emulation will still work, but since the 'unexpected' packets may be interfering with the TCP flow, it is better to remove them.

The 'unexpected' packets can have several reasons:

  1. A previous instance of Monarch is still running in the background. Run ps fax and check for processes called monarch.
  2. The kernel is routing other packets to the TUN device. Run route -n and check for routing table entries to the TUN device. Normally, there should be only one entry (mask 255.255.255.255, flags UH). If there are other entries, use the route command to remove them.
  3. Monarch is looking for the wrong packets, e.g. due to a configuration problem in standalone mode. Add --verbose to the command line and check for the following output:
            Probe exchange: 139.19.131.1:10334 <-> 139.19.131.2:10334
            Sender: 10.0.0.1:48575 <-> 10.0.0.2:10001
            Receiver: 10.0.0.1:55742 <-> 10.0.0.2:10002
    The last two lines show the expected IP addresses and port numbers for packets from the sender and the receiver. If either of them is bound to a different address or sends its traffic to some other destination, Monarch will not recognize the packets.
If none of these cases apply, we suggest invoking tcpdump -i tun0 -n -x while Monarch is running.

Why does the IPID sanity check fail whenever I use the tcp-ack probe?

Please check whether your measurement host is behind a firewall. Some firewalls will intercept the ACK probes before they even leave the local network, and send back a RST packet immediately. To find out whether this is happening, run a Monarch flow to a host that is far away, create a pcap trace, and look at the trace with tcpdump -r. If you see very low round-trip times (at most a few milliseconds), the responses are probably coming from a firewall and not from the remote host.

Back to the
Monarch home page