IP over Web-avian Carriers

The Internet is notoriously hungry for bandwidth and much effort goes into increasing the throughput of Internet Protocol carrying media. One particularly successful approach in throughput optimization has been IP over Avian Carriers which, thanks to pleasant developments in storage media density, managed to stay ahead of the game by a factor of three. However, speeding up existing protocols is not enough to meet increased bandwidth demands. New protocols and unconventional link media have to be considered to keep the lights on and the bytes flowing.

Today I would like to propose an adaption of the IP over Avian Carriers protocol to make use of so-called web-avian carriers. Similar to homing pigeons, which have been the preferred medium used by traditional IP over Avian Carriers implementations, web-avian carriers have wings and a beak. The most common web-avian carrier is the blue-winged jack-bird. It is a shy, yet extroverted creature. It is easy to spot, yet only a few people had the pleasure to see it from other perspectives than side-face.

This article introduces the Internet Protocol over Web-avian Carriers (IPoWAC). Furthermore, it provides a proof of concept implementation written in Go.

IPoWAC Network Stack

IPoWAC is a link-layer protocol utilizing a publicly available microblogging platform as transport medium.

IPoWAC protocol stack

IPoWAC is specified for IPv4 datagrams but it is imaginable to extend the functionality to include other popular network-layer protocols, such as IPv6.

The Medium

The web-avian carrier network is a planet-scale data transmission system piggybacking on other technologies, such as mobile data networks, Digital Subscriber Lines, and possibly traditional IP over Avian Carriers installations.

The medium uses globally unique node identifiers consisting of at least one and up to 15 characters, the so-called handles. A handle is identified in the protocol by its preceding @ character. Handles are given out on a first come first serve basis by a central handle handling authority. Some nodes got luckier than others in getting their favorite handle.

Nodes modulate data onto the medium in data units of 280 characters called messages or sometimes tweets. In previous iterations, the web-avian carrier network facilitated an MTU of 140 characters. The highly debated limitation was removed in 2017. While this bold move doubled the MTU, legacy clients using the Short Message Service-based implementation of web-avian carriers are now forced to fragment submissions. The outrage was loud and voiceful but without real-life impact. A pattern we often observe in non-IP-carrying web-avian communication streams.

The web-avian carrier network features common link-layer aspects, however, it has subtle differences:

Destination Labels (Addressing)

An IPoWAC edge router attaches one or more destination labels (hashtags) to each message it modulates onto the medium. The destination label is derived from the site’s globally unique IP space allocation. The destination label begins with the # (hash) character to indicate the start of a label. It is followed by the four octets of the smallest (most specific) IP space allocation that contains the destination IP address. Octets are written in decimal notation and separated by the _ (underscore) character. Finally, the prefix length of the allocation is appended, once again using the _ (underscore) character for separation.

IPoWAC address format

Label Example

The IP address 193.160.39.100 is part of the RIPE-managed IP space allocation 193.160.39.0/24. The IPoWAC message encapsulating an IP packet addressed to 193.160.39.100 is therefore labeled #193_160_39_0_24.

Wire Format

The IPoWAC message format was designed with simplicity in mind. Each message starts with the base64-encoded IP datagram. Separated by spaces, one or more (when multicasting) destination labels follow.

IPoWAC wire format

Link Establishment

IPoWAC edge router operators configure their instances so that they track all labels for all IP space allocations that fall under their administrative domain. For filtering purposes, operators may decide to additionally limit message tracking by only accepting messages from nodes with high trust values. A high-trust node is identified by a blue checkmark sign next to the node name.

Tweets containing labels and encoded payloads

Additional Medium Features

The web-avian carrier network offers a feature-rich medium and additional value via meta information:

All mentioned features are available to the operator via web interface.

A tweet

Introducing WACky: A Proof Of Concept Implementation

WACky is an IPoWAC proof-of-concept implementation written in Go. WACky acts as an edge router accepting Ethernet frames. It extracts the IP payload from ethertype 0x0800 frames and converts it to the IPoWAC wire format.

WACky consists of to parts that run in parallel:

Let’s discuss each of these separately.

From Ethernet To Web-avian Carrier

On startup, WACky tries to get a hold on the Ethernet interface it is supposed to listen on for IP datagrams.

ifi, err := net.InterfaceByName(*ifname)
if err != nil {
    fmt.Fprintf(os.Stderr, "interface %q: %v", *ifname, err)
    os.Exit(1)
}

Once the interface is under WACky’s control it listens for Ethernet frames containing an IP datagram using a raw PacketConn.

cfg, err := raw.ListenPacket(ifi, 0x0800, &raw.Config{
    LinuxSockDGRAM: false,
})
if err != nil {
    fmt.Fprintf(os.Stderr, "listen: %v", err)
    os.Exit(1)
}

Then variables are allocated for use in a worker loop later. The buffer for incoming Ethernet frames is computed by guestimating how much space is left in the IPoWAC message once base64 encoding and destination label length have been accounted for.

var frame ethernet.Frame
buf := make([]byte, ((280-len(*destinationLabel)-1)/4)*3) // ¯\_(ツ)_/¯

The worker loop reads from the Ethernet device and encodes the message (if it is half-way decent in format).

for {
    // read from device
    n, addr, err := cfg.ReadFrom(buf)
    if err != nil {
        fmt.Fprintf(os.Stderr, "read: %v\n", err)
        time.Sleep(100 * time.Millisecond)
        continue
    }

    // parse ethernet frame to extract payload
    if err := (&frame).UnmarshalBinary(buf[:n]); err != nil {
        fmt.Fprintf(os.Stderr, "unmarshal: %v\n", err)
        time.Sleep(100 * time.Millisecond)
        continue
    }

    // base64 encode the packet and discard malformed frames (using a
    // stupid but surprisingly effecting method: decode base64)
    encoded := base64.StdEncoding.EncodeToString(frame.Payload)
    _, err = base64.StdEncoding.DecodeString(encoded)
    if err != nil || strings.Contains(encoded, "/") {
        fmt.Fprint(os.Stderr, "received malformed frame\n")
        continue
    }
    tweet := encoded + " " + *destinationLabel
    fmt.Printf("from %v: %v\n", addr.String(), tweet)

When the message is ready to be modulated onto the wire, it is handed over to the wak0 virtual interface. What is that, the interested reader may wonder? Just a function call using Aditya Mukerjee’s awesome Twitter API library anaconda. 🤪

    // send to web-avian carrier network
    _, err = api.PostTweet(tweet, url.Values{})
    if err != nil {
        fmt.Fprintf(os.Stderr, "post tweet: %v\n", err)
        time.Sleep(10 * time.Second)
        continue
    }
    // twitter loves us to rate limit API usage
    time.Sleep(750 * time.Millisecond)
}

The Ethernet-reading worker loop runs inside a goroutine so that it does not block the receving side of the IPoWAC implementation.

By the way: A big thank you to Matt Layher for providing the ethernet and raw golang packages for easy link-layer programming!

From Web-avian Carrier To Ethernet

The purpose of the IPoWAC-to-Ethernet part of the WACky router is to track a label on the global medium and emit received messages via a local interface.

For that, WACky opens a raw IP socket by using the syscall package.

fd, err := syscall.Socket(syscall.AF_INET, syscall.SOCK_RAW, syscall.IPPROTO_RAW)
if err != nil {
    fmt.Fprintf(os.Stderr, "socket: %v\n", err)
    os.Exit(1)
}

Then a stream of messages is created by subscribing to the tracking label via Twitter API.

stream := api.PublicStreamFilter(url.Values{"track": []string{*trackLabel}})
defer stream.Stop()

While the stream lasts, messages are received and decoded. If a message does not decode well, it is discarded.

for v := range stream.C {
    t, ok := v.(anaconda.Tweet)
    if !ok {
        fmt.Fprintf(os.Stderr, "malformed tweet")
        continue
    }

    encoded := strings.Split(t.FullText, " ")[0]
    data, err := base64.StdEncoding.DecodeString(encoded)
    if err != nil || len(data) < 60 { // 60 = reasonable packet size ;)
        fmt.Fprint(os.Stderr, "received malformed tweet\n")
        continue
    }
    hdr := ipv4.Header{}
    hdr.Parse(data)

From the decoded message the destination IP address is extracted and used in the sockaddr that the sendto() syscall requires. Also, hello sockaddr, my old friend…

    // create socket address for sendto() syscall
    ip := hdr.Dst.To4()
    addr := syscall.SockaddrInet4{
        Port: 0,
        Addr: [4]byte{ip[0], ip[1], ip[2], ip[3]},
    }

After that there is not much left to do but to actually send the datagram.

    // make it happen, send the packet!
    fmt.Printf("to %v: %v\n", ip.String(), encoded)
    err = syscall.Sendto(fd, data, 0, &addr)
    if err != nil {
        fmt.Fprintf(os.Stderr, "sendto: %v\n", err)
        continue
    }
}

And that’s all there is to it. Simple, but effective. The full source code is available at Github in the WACky repository.

Limitations

In good tradition of PoC implementations, WACky lacks features that one would expect from a production-grade IPoWAC edge router:

World’s First IPoWAC Tansmission

Join me in the world’s first IPoWAC data transmission! For that I prepared a setup consisting of two clients and two WACky routers. Each client is connected to one edge router. Both routers use the handle @ipowac on their virtual wky0 interfaces but track different labels each.

The demo setup explained graphically.

First I started the WACky router process on each of the router machines.

[email protected]:~# go run main.go \
    -track-label="#1_3_3_0_24" \
    -destination-label="#2_4_4_0_24"

And…

[email protected]:~# go run main.go \
    -track-label="#2_4_4_0_24" \
    -destination-label="#1_3_3_0_24"

Then I ran ping on wacky-1 with a generous wait time of 120 seconds. IPoWAC turns out to not be the fastest protocol. Surprise!

[email protected]:~# ping -c 1 -W 120 2.4.4.8

The WACky program on wacky-1-router immediately cought the frame and sent the corresponding IPoWAC message with the destination label set to #2_4_4_0_24.

      from 08:00:27:a5:2b:1e: RQAAVAxZQABAASQ7AQMDBwIEBAgIAEoCAdwAAUJT2lsAAAAAx54JAAAAAAAQERITFBUWFxgZGhscHR4fICEiIyQlJicoKSorLC0uLzAxMjM0NTY3 #2_4_4_0_24

On wacky-2-router the message was received after a short delay.

      to 2.4.4.8: RQAAVAxZQABAASQ7AQMDBwIEBAgIAEoCAdwAAUJT2lsAAAAAx54JAAAAAAAQERITFBUWFxgZGhscHR4fICEiIyQlJicoKSorLC0uLzAxMjM0NTY3

The echo request was answered by wacky-2 with an echo reply, which was also seen by both routers and finally delivered to the ping process on wacky-1.

[email protected]:~# ping -c 1 -W 120 2.4.4.8
PING 2.4.4.8 (2.4.4.8): 56(84) bytes of data.
64 bytes from 2.4.4.8: icmp_seq=1 ttl=64 time=11933 ms

--- 2.4.4.8 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 11933.267/11933.267/11933.267/0.000 ms

Woohoo! It works!

This is the whole transmission as archived by the web-avian carrier network:

A IPoWAC transmission

The world’s first successful IPoWAC transmission is also available in video!

Outlook

The PoC implementation of the WACky router shows that transmitting IP over Web-avian Carriers is generally possible. Further improvements could dramatically increase the throughput:

Compression could be applied to the data before it hits the web-avian carrier network. A more efficient encoding could be used instead of base64. Additional payload data may be encapsulated in image tweets. Rumor has it that up to 15 MB of image data can be attached in certain situations. An MTU that puts even Ethernet Jumbo Frames to shame.

Conclusion

IPoWAC is a powerful link-layer protocol running on top of the existing Internet and completely dependent on the mercy of a single company. Stacking layers of complexity on top of each other has a long tradition in software and network engineering alike. By combining the worst of both professions, the urge to make something work no matter what and the total disrespect for network layers, a beautiful piece of technology abuse is created. It should have never happened, yet, IPoWAC had to be made. Someone out there, at some point in time, will start to use the protocol to access Twitter over Twitter… Because, why not?