How do I expose a UDP service externally on Google Kubernetes Engine (GKE) using the ingress-nginx controller?

I’m looking for some guidance on exposing UDP services on GKE using the ingress-nginx controller. After following the instructions on https://kubernetes.github.io/ingress-nginx/user-guide/exposing-tcp-udp-services/ I was able to access it when deploying to a local minikube VM using the ConfigMap method. However, when I deploy to GKE the services are unreachable over the IP of the ingress controller service.

I see the ports (1053 and 15353) on the controller are mapped correctly:

NAMESPACE       NAME                                 TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                                                     AGE
ingress-nginx   ingress-nginx-controller             LoadBalancer   10.51.252.115   <redacted>      80:32307/TCP,443:30514/TCP,1053:32764/UDP,15353:31385/UDP   54d

The cluster itself was created using the google_container_cluster Terraform module with default settings and the controller works well handling HTTPS traffic. One thing I did notice is that the auto-generated firewall rules omit UDP for the specified ports, and use TCP instead. Manually adding a UDP firewall rule for those ports didn’t work.

NAME                                          NETWORK                        DIRECTION  PRIORITY  ALLOW                              DENY  DISABLED
k8s-fw-a62a982b26a034e0e97258af6717b8b0       cluster-network-labs-us-west1  INGRESS    1000      tcp:80,tcp:443,tcp:1053,tcp:15353        False

I’ve deployed a simple UDP ping-pong server which works both locally on bare metal and on minikube as a kubernetes service using the ingress-nginx controller. That same controller with an identical configuration causes client requests to time out.

Server

package server

import (
    "fmt"
    "net"
    "os"
    "time"

    "github.com/spf13/cobra"
)

func response(udpServer net.PacketConn, addr net.Addr, buf []byte) {

    fmt.Println("msg", string(buf))

    time := time.Now().Format(time.ANSIC)
    responseStr := fmt.Sprintf("%v. msg: %v", time, string(buf))

    udpServer.WriteTo([]byte(responseStr), addr)
}

var Command = &cobra.Command{
    Use:   "server",
    Short: "Debug UDP server.",
    Long:  `Provides a UDP server endpoint which responds to pings.`,
    RunE: func(cmd *cobra.Command, args []string) error {

        udpServer, err := net.ListenPacket("udp", fmt.Sprintf(":%d", serverPort))
        if err != nil {
            return err
        }
        defer udpServer.Close()

        fmt.Fprintf(os.Stdout, "listening :%dn", serverPort)

        for {
            buf := make([]byte, 1024)
            _, addr, err := udpServer.ReadFrom(buf)
            if err != nil {
                continue
            }
            go response(udpServer, addr, buf)
        }

    },
}

var serverPort int

func init() {
    Command.PersistentFlags().IntVar(&serverPort, "port", 1053, "Port to open an listen for UDP packets")
}

Client

package client

import (
    "fmt"
    "net"
    "os"

    "github.com/spf13/cobra"
)

var Command = &cobra.Command{
    Use:   "client",
    Short: "Debug UDP client.",
    Long:  `Provides a UDP client endpoint which responds to pings.`,
    RunE: func(cmd *cobra.Command, args []string) error {

        udpServer, err := net.ResolveUDPAddr("udp", fmt.Sprintf("%s:%d", queryHost, queryPort))

        if err != nil {
            return err
        }

        conn, err := net.DialUDP("udp", nil, udpServer)
        if err != nil {
            return err
        }
        defer conn.Close()

        _, err = conn.Write([]byte(msg))
        if err != nil {
            return err
        }

        received := make([]byte, 1024)
        _, err = conn.Read(received)
        if err != nil {
            println("Read data failed:", err.Error())
            os.Exit(1)
        }

        println(string(received))

        return nil
    },
}

var msg string
var queryHost string
var queryPort int

func init() {

    Command.PersistentFlags().StringVar(&msg, "msg", "echo", "Message used to send ping/pong requests over UDP")
    Command.PersistentFlags().StringVar(&queryHost, "host", "127.0.0.1", "Host used to send ping/pong requests over UDP")
    Command.PersistentFlags().IntVar(&queryPort, "port", 1053, "Port used to send ping/pong requests over UDP")
}

Has anyone seem something similar to this or have any ideas on where I can dig in further?
Thanks

Versions:

• ingress-nginx helm chart – 4.4.0
• ingress-nginx – 1.5.1
• Kubernetes – v1.24.5-gke.600
• registry.terraform.io/hashicorp/google – v4.43.0