skip to Main Content

I tried to add an extension header after the ipv6 message through ebpf (tc egress)

I successfully added the extension header. During the initial test, I found that the addition of the extension header did not cause device communication abnormalities.

I didn’t discover this problem until my ebpf program was running on an openwrt (23.05.3 ramips/mt7621) router and the outgoing packets were lost by the other party due to checksum.

I didn’t have this problem when I first tested on an x86 virtual machine (Ubuntu 22.04). I found that the checksum printed by the virtual machine’s ebpf program was inconsistent with the checksum I actually sent. In fact, the checksum sent was correct. The virtual machine seemed to Automatically corrected

Moreover, even if bpf_skb_store_bytes is called with the BPF_F_RECOMPUTE_CSUM mark, the checksum before and after does not change.

This is part of my code to add the ipv6 extension header, the checksum is consistent before and after calling bpf_skb_store_bytes

    protocol = ip6->nexthdr;   // store original protocol
    ip6->nexthdr = ip6nexthdr; // update protocol

    // off is the offset of the last nexthdr field (off = sizeof(struct ipv6hdr) + sizeof(struct ethhdr))
    // bytes_len is the length of the new bytes
    ip6->payload_len = bpf_htons(skb->len - off + bytes_len); // update payload_len

    // Print the checksum before bpf_skb_store_bytes
    if (protocol == IPPROTO_TCP) {
        struct tcphdr *tcp_hdr = (struct tcphdr *)((char *)ip6 + sizeof(*ip6));
        if ((char *)(tcp_hdr + 1) > (char *)(long)skb->data_end)
            return TC_ACT_SHOT;
        BPF_PRINT("TCP checksum before bpf_skb_store_bytes: %x", tcp_hdr->check);
    } else if (protocol == IPPROTO_UDP) {
        struct udphdr *udp_hdr = (struct udphdr *)((char *)ip6 + sizeof(*ip6));
        if ((char *)(udp_hdr + 1) > (char *)(long)skb->data_end)
            return TC_ACT_SHOT;
        BPF_PRINT("UDP checksum before bpf_skb_store_bytes: %x", udp_hdr->check);
    }
    /* Make room for new bytes and insert them.
     */
    if (bpf_skb_adjust_room(skb, bytes_len, BPF_ADJ_ROOM_NET, 0)) {
        BPF_PRINT("Failed to adjust room");
        return TC_ACT_SHOT;
    }
    /* Store the new bytes.
     */
    if (bpf_skb_store_bytes(skb, off, exthdr->bytes, bytes_len, BPF_F_RECOMPUTE_CSUM)) {
        BPF_PRINT("Failed to store bytes");
        return TC_ACT_SHOT;
    }
    /* Update last Extension Header's nexthdr field.
     */
    if (bpf_skb_store_bytes(skb, off + off_last_nexthdr, &protocol, sizeof(protocol), BPF_F_RECOMPUTE_CSUM)) {
        BPF_PRINT("Failed to store last nexthdr");
        return TC_ACT_SHOT;
    }

    // Print the checksum after bpf_skb_store_bytes
    if (protocol == IPPROTO_TCP) {
        ip6 = ipv6_header(skb, &off);
        if (!ip6) {
            BPF_PRINT("Failed to check IPv6 header");
            return TC_ACT_SHOT;
        }
        struct tcphdr *tcp_hdr = (struct tcphdr *)((char *)ip6 + sizeof(*ip6) + bytes_len);
        if ((char *)(tcp_hdr + 1) > (char *)(long)skb->data_end)
            return TC_ACT_SHOT;
        BPF_PRINT("TCP checksum after bpf_skb_store_bytes: %x", tcp_hdr->check);
    } else if (protocol == IPPROTO_UDP) {
        ip6 = ipv6_header(skb, &off);
        if (!ip6) {
            BPF_PRINT("Failed to check IPv6 header");
            return TC_ACT_SHOT;
        }
        struct udphdr *udp_hdr = (struct udphdr *)((char *)ip6 + sizeof(*ip6) + bytes_len);
        if ((char *)(udp_hdr + 1) > (char *)(long)skb->data_end)
            return TC_ACT_SHOT;
        BPF_PRINT("UDP checksum after bpf_skb_store_bytes: %x", udp_hdr->check);
    }

Why does my ebpf program set the BPF_F_RECOMPUTE_CSUM flag when calling bpf_skb_store_bytes, but it does not update the checksum? Did I write it somewhere wrong?

Why is the checksum of the packet sent by the virtual machine (Ubuntu 22.04 x86_64) automatically corrected?

2

Answers


  1. Chosen as BEST ANSWER

    I understand why the checksum is consistent because my modification did not actually cause a modification to the ipv6 pseudo-header used to calculate the checksum.

    The checksum of the sent data packet will be automatically modified because of checksum offload. The checksum printed on the egress will be modified by the hardware later.

    The receiver of the packet sent out by my openwrt router (MIPS) will report a bad checksum error. It seems that the checksum offload has failed. I captured the packet and looked at the sending and receiving ends. Their checksums are consistent.

    Does anyone know what could be the cause? At least I haven't had this problem on x86_64 devices.

    This is my system information:

    NAME="OpenWrt"
    VERSION="23.05.3"
    ID="openwrt"
    ID_LIKE="lede openwrt"
    PRETTY_NAME="OpenWrt 23.05.3"
    VERSION_ID="23.05.3"
    HOME_URL="https://openwrt.org/"
    BUG_URL="https://bugs.openwrt.org/"
    SUPPORT_URL="https://forum.openwrt.org/"
    BUILD_ID="r23809-234f1a2efa"
    OPENWRT_BOARD="ramips/mt7621"
    OPENWRT_ARCH="mipsel_24kc"
    OPENWRT_TAINTS=""
    OPENWRT_DEVICE_MANUFACTURER="OpenWrt"
    OPENWRT_DEVICE_MANUFACTURER_URL="https://openwrt.org/"
    OPENWRT_DEVICE_PRODUCT="Generic"
    OPENWRT_DEVICE_REVISION="v0"
    OPENWRT_RELEASE="OpenWrt 23.05.3 r23809-234f1a2efa"
    

  2. Moreover, even if bpf_skb_store_bytes is called with the BPF_F_RECOMPUTE_CSUM mark, the checksum before and after does not change.

    BPF_F_RECOMPUTE_CSUM tells bpf_skb_store_bytes to update the packet checksum, not the TCP/UDP checksum. So given you’re on IPv6, there’s nothing to update here.

    IPv6 doesn’t have a checksum and the TCP/UDP checksum doesn’t cover the IPv6 extension headers. Unless you changed the IPv6 addresses or the IPv6 payload, your TCP/UDP checksum shouldn’t change.


    Isn’t the IPv6 Next Header value also part of the TCP/UDP Checksum?

    The IPv6 specification (RFC8200) discusses the TCP/UDP checksum in section 8.1. In particular, it mentions a pseudo-header for IPv6 that is part of the checksum. That header contains the IPv6 addresses and the Next Header value. However, as the RFC later explains, this "Next Header value" doesn’t depend on IPv6 extension headers:

    The Next Header value in the pseudo-header identifies the upper-layer protocol (e.g., 6 for TCP or 17 for UDP). It will differ from the Next Header value in the IPv6 header if there are extension headers between the IPv6 header and the upper-layer header.

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search