How to allow for tcp service (not http) on custom port inside kubernetes – Nginx

I have a container running an OPC-server on port 4840. I am trying to configure my microk8s to allow my OPC-Client to connect to the port 4840. Here are examples of my deployment and service:

(No namespace is defined here but they are deployed through azure pipelines and that is where the namespace is defined, the namespace for the deployment and service is "jawcrusher")

deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jawcrusher
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jawcrusher
  strategy: {}
  template:
    metadata:
      labels:
        app: jawcrusher
    spec:
      volumes:
        - name: jawcrusher-config
          configMap:
            name: jawcrusher-config
      containers:
      - image: XXXmicrok8scontainerregistry.azurecr.io/jawcrusher:#{Version}#
        name: jawcrusher
        ports:
          - containerPort: 4840
        volumeMounts:
          - name: jawcrusher-config
            mountPath: "/jawcrusher/config/config.yml"
            subPath: "config.yml"
      imagePullSecrets:
        - name: acrsecret

service.yml

apiVersion: v1
kind: Service
metadata:
  name: jawcrusher-service
spec:
  ports:
  - name: 7070-4840
    port: 7070
    protocol: TCP
    targetPort: 4840
  selector:
    app: jawcrusher
  type: ClusterIP
status:
  loadBalancer: {}

I am using a k8s-client called Lens and in this client there is a functionality to forward local ports to the service. If I do this I can connect to the OPC-Server with my OPC-Client using the url localhost:4840. To me that indicates that the service and deployment is set up correctly.

So now I want to tell microk8s to serve my OPC-Server from port 4840 "externally". So for example if my dns to the server is microk8s.xxxx.internal I would like to connect with my OPC-Client to microk8s.xxxx.internal:4840.

I have followed this tutorial as much as I can: https://minikube.sigs.k8s.io/docs/tutorials/nginx_tcp_udp_ingress/.

It says to update the tcp-configuration for the ingress, this is how it looks after I updated it:

nginx-ingress-tcp-microk8s-conf:

apiVersion: v1
kind: ConfigMap
metadata:
  name: nginx-ingress-tcp-microk8s-conf
  namespace: ingress
  uid: a32690ac-34d2-4441-a5da-a00ec52d308a
  resourceVersion: '7649705'
  creationTimestamp: '2023-01-12T14:12:07Z'
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
      {"apiVersion":"v1","kind":"ConfigMap","metadata":{"annotations":{},"name":"nginx-ingress-tcp-microk8s-conf","namespace":"ingress"}}
  managedFields:
    - manager: kubectl-client-side-apply
      operation: Update
      apiVersion: v1
      time: '2023-01-12T14:12:07Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:metadata:
          f:annotations:
            .: {}
            f:kubectl.kubernetes.io/last-applied-configuration: {}
    - manager: kubectl-patch
      operation: Update
      apiVersion: v1
      time: '2023-02-14T07:50:30Z'
      fieldsType: FieldsV1
      fieldsV1:
        f:data:
          .: {}
          f:4840: {}
  selfLink: /api/v1/namespaces/ingress/configmaps/nginx-ingress-tcp-microk8s-conf
data:
  '4840': jawcrusher/jawcrusher-service:7070
binaryData: {}

It also says to update a deployment called ingress-nginx-controller but in microk8s it seems to be a daemonset called nginx-ingress-microk8s-controller. This is what it looks like after adding a new port:

nginx-ingress-microk8s-controller:

spec:
      containers:
        - name: nginx-ingress-microk8s
          image: registry.k8s.io/ingress-nginx/controller:v1.2.0
          args:
            - /nginx-ingress-controller
            - '--configmap=$(POD_NAMESPACE)/nginx-load-balancer-microk8s-conf'
            - >-
              --tcp-services-configmap=$(POD_NAMESPACE)/nginx-ingress-tcp-microk8s-conf
            - >-
              --udp-services-configmap=$(POD_NAMESPACE)/nginx-ingress-udp-microk8s-conf
            - '--ingress-class=public'
            - ' '
            - '--publish-status-address=127.0.0.1'
          ports:
            - name: http
              hostPort: 80
              containerPort: 80
              protocol: TCP
            - name: https
              hostPort: 443
              containerPort: 443
              protocol: TCP
            - name: health
              hostPort: 10254
              containerPort: 10254
              protocol: TCP
            ####THIS IS WHAT I ADDED####
            - name: jawcrusher
              hostPort: 4840
              containerPort: 4840
              protocol: TCP

After I have updated the daemonset it restarts all the pods. The port seem to be open, if I run this script is outputs:

 Test-NetConnection -ComputerName microk8s.xxxx.internal -Port 4840                                                                                                                                                                                                                                                                             
 ComputerName     : microk8s.xxxx.internal                                                                            
 RemoteAddress    : 10.161.64.124                                                                                        
 RemotePort       : 4840                                                                                                 
 InterfaceAlias   : Ethernet 2                                                                                           
 SourceAddress    : 10.53.226.55
 TcpTestSucceeded : True

Before I did the changes it said TcpTestSucceeded: False.

But the OPC-Client cannot connect. It just says:
Could not connect to server: BadCommunicationError.

Does anyone see if I made a mistake somewhere or knows how to do this in microk8s.

Update 1:
I see an error message in the ingress-daemonset-pod logs when I try to connect to the server with my OPC-Client:

2023/02/15 09:57:32 [error] 999#999: *63002 connect() failed (111:
Connection refused) while connecting to upstream, client:
10.53.225.232, server: 0.0.0.0:4840, upstream: "10.1.98.125:4840", bytes from/to client:0/0, bytes from/to upstream:0/0

10.53.225.232 is the client machines ip address and 10.1.98.125 is the ip number of the pod running the OPC-server.

So it seems like it has understood that external port 4840 should be proxied/forwarded to my service which in turn points to the OPC-server-pod. But why do I get an error…

Update 2:
Just to clearify, if I run kubectl port forward command and point to my service it works. But not if I try to connect directly to the 4840 port. So for example this works:

kubectl port-forward service/jawcrusher-service 5000:4840 -n jawcrusher --address='0.0.0.0'

This allows me to connect with my OPC-client to the server on port 5000.