Configure NGINX/traefik for socket.io when using docker, nestjs gateway, react, and vite on a remote server

I am trying to figure out how to get my nginx config setup to allow socket.io connections from my react client to my node backend that is using NestJS Gateways.
Both the client and the backend are running inside their own docker containers on staging/production servers.
When testing things on my local computer, using vite to run the client and node to run the backend (not running in docker) I can connect both no problem. So I know I have the backend and frontend configured and working correctly.
When I deploy to our staging server, I can’t get the socket.io connection to work. I am 99% sure it has to do with my nginx configuration.

Here is my setup:

1. In my backend docker file I have exposed port 8889 (this is my socket gateway port)

EXPOSE 8889

2. In my backend, using NestJS, I have created a ws.gateway.ts file. This is it:

@WebSocketGateway(8889, { cors: { origin: "*" }, path: "/socket", namespace: "/api/frontend" })
export class EventsGateway implements OnGatewayInit, OnGatewayConnection, OnGatewayDisconnect {
  @WebSocketServer()
  server: Server;

  afterInit(server: Server) {
    console.log("Websocket initialized.");
  }

  handleDisconnect(client: Socket) {
    console.log(`Client disconnected: ${client.id}`);
  }

  handleConnection(client: Socket, ...args: any[]) {
    console.log(`Client connected: ${client.id}`);
  }

}

The class is setup in a module and connected to the NestJS module system.

3. In my client app, which is a react app using vite to serve it locally. I am aware that vite doesn’t have any impact on the client when running on the staging server as nginx is serving the client and note vite, but I am noting it here just in case. I have the following files and code:

vite.config.mjs

export default defineConfig({
   ...
   server: {
    host: "127.0.0.1",
    port: 3000,
    proxy: {
      "/api": {
        target: "http://127.0.0.1:8888/",   // <-- this is for the backend api's.
        changeOrigin: true,
        secure: false,
        ws: true,
        rewrite: path => path.replace(/^/api/, ""),
      },
      "/socket": {
        target: "http://127.0.0.1:8889/",   // <-- This is for the socket connections.
        changeOrigin: true,
        secure: false,
        ws: true,
        // rewrite: path => path.replace(/^/socket/, ""),
      },
    },
  },
  ...
})

This is part of my socket.tsx file, which is added to my app.tsx file and where I try to make a test socket connection:

const WSClient: React.FC<WSClientProps> = props => {
   const uri = `${import.meta.env.VITE_APP_WS}/api/frontend`; // <-- this translates to https://stage.mycompany.io/api/frontend

   useEffect(() => {
      console.log("Attempting to open socket to:", uri);
      const options: Partial<ManagerOptions & SocketOptions> = {
        reconnection: true,
        reconnectionDelay: 100,
        reconnectionAttempts: 3,
        path: "/socket",
      };
      const newSocket = io(uri, options);
      setSocket(newSocket);
      newSocket.on("connect", () => {
        console.log("Socket connected to backend server. Socket id:", newSocket.id, "on uri:", uri);
      });
      newSocket.on("disconnect", () => {
        console.log(`Socket id ${newSocket.id} has disconnected.`);
      });
      newSocket.io.on("reconnect_attempt", attempt => {
        console.log(`Socket reconnect_attempt ${attempt}.`); 
        if (attempt > 2) {
          console.log("Failed to disconnect the socket.");
        }
      });
   }, []);

   ...
}

4. And this is my nginx default.conf file. This, I’ll admit, is not my strong suit and I am not really sure how to properly set it up and where I need some help.

server {
    listen       80;
    server_name  localhost;
    root   /usr/share/nginx/html;
    index  index.html index.htm;
    resolver 127.0.0.11 ipv6=off valid=10s;
    access_log  /etc/nginx/conf.d/access.log;
    error_log  /etc/nginx/conf.d/errors.log debug;

    location @index {
        root   /usr/share/nginx/html;
        add_header Cache-Control 'private, no-cache, no-store, must-revalidate';
        add_header Expires 'Sat, 01 Jan 2000 00:00:00 GMT';
        add_header Pragma no-cache;
        try_files $uri $uri/ /index.html;
    }

    ##### SETUP FOR SOCKET.IO ######
    location /socket {
      proxy_set_header Host $host;
      proxy_set_header X-Real-IP $remote_addr;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_pass http://127.0.0.1:8889/;
      proxy_http_version 1.1;
      proxy_set_header Upgrade $http_upgrade;
      proxy_set_header Connection "upgrade";
    }
    #### END SETUP FOR SOCKET.IO ######

    location / {
        try_files $uri @index;
    }

    error_page   500 502 503 504  /50x.html;

    location = /50x.html {
        root   /usr/share/nginx/html;
    }

}

The react part of the client and communication to the back end api is working perfectly as it was before. In order to hit any of the api’s I need to do an axios or fetch call to https://stage.mycompany.io/api/ and https://stage.mycompany.io/api/graphql/ for my queries. Everything does what it’s supposed to.

In my console, I see that the socket is attempting to connect to https://stage.mycompany.io/api/frontend as defined in Gateway namespace. I honestly am not sure where the breakdown is and I have tried a ton of different things in my nginx config file but I can’t seem to figure it out.

I think it may be the proxy_pass, since the backend is in another docker container therefor using a localhost or home ip is not going to work. I have tried using the server url and a bunch of other things to no avail.

I have been banging my head against the keyboard for a couple of days trying to figure this out. Any help would be greatly appreciated.

Update:
Here are my docker files by request

Backend docker file:

FROM node:18.19-alpine3.18 as builder

ARG NPM_TOKEN
ARG PACKAGE_VERSION

RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > ~/.npmrc

RUN for i in $(seq 1 3); 
  do [ $i -gt 1 ] && sleep 5; 
  npm cache clean --force; 
  npm add @mycompany/mycompany-backend@$PACKAGE_VERSION --global && s=0 && break || s=$?; 
  done; (exit $s)

FROM node:18.19-alpine3.18

ARG PACKAGE_VERSION
ARG ENVIRONMENT

COPY --from=builder /usr/local/lib/node_modules/@mycompany/mycompany-backend /app/

WORKDIR /app/


#RUN --name some-redis -d redis

EXPOSE 8889

ENTRYPOINT ["npm"]

CMD ["start"]

Frontend docker file:

FROM node:18.19-alpine3.18 as builder
#WORKDIR '/builddir'
ARG NPM_TOKEN
ARG PACKAGE_VERSION

RUN echo "//registry.npmjs.org/:_authToken=$NPM_TOKEN" > ~/.npmrc

RUN for i in $(seq 1 3); 
  do [ $i -gt 1 ] && sleep 5; 
  npm cache clean --force; 
  npm add @mycompany/mycompany-frontend@$PACKAGE_VERSION --global && s=0 && break || s=$?; 
  done; (exit $s)

FROM nginx:1.25.3-alpine
RUN rm -f /etc/nginx/conf.d/default.conf
RUN rm -f /etc/nginx/default.conf
COPY /conf/default.conf /etc/nginx/conf.d/default.conf

COPY --from=builder /usr/local/lib/node_modules/@mycompany/mycompany-frontend/dist/ /usr/share/nginx/html/

base.yml is my docker compose file which is run when I deploy to stage.

version: '3.4'
networks:
  traefik-net:
  app-net:
volumes:
  certs:
services:
  proxy:
    image: traefik:1.7.5
    deploy:
      replicas: 1
      placement:
        constraints:
          - node.role == manager
    environment:
      CF_API_EMAIL: "${CLOUDFLARE_EMAIL}"
      CF_API_KEY: "${CLOUDFLARE_API_KEY}"
    command: [
      "--loglevel=ERROR",
      "--web.address=:8080",
      "--retry",
      "--docker.swarmmode=true",
      "--docker.watch=true",
      "--docker.endpoint=unix:///var/run/docker.sock",
      "--defaultentrypoints=http,https",
      "--entryPoints=Name:http Address::80 Redirect.EntryPoint:https",
      "--entryPoints=Name:https Address::443 TLS",
      "--acme.dnschallenge.provider=cloudflare",
      "--acme.dnschallenge.delaybeforecheck=30",
      "--acme.acmelogging=true",
      "[email protected]",
      "--acme.entrypoint=https",
      "--acme.onhostrule=true",
      "--acme.storage=/certs/acme.json"
    ]
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - certs:/certs/
    networks:
      - traefik-net
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"
      - "9229:9229"
      - "8889:8889"
  server:
    image: mycompany/mycompany-frontend:$MYCOMPANY_FRONTEND
    deploy:
      replicas: 1
      update_config:
        delay: 20s
        failure_action: rollback
      restart_policy:
        condition: any
      labels:
        traefik.backend: 'server'
        traefik.docker.network: 'mycompany_traefik-net'
        traefik.enable: 'true'
        traefik.port: '80'
    networks:
      - traefik-net
  api:
    image: mycompany/mycompany-backend:$MYCOMPANY_BACKEND
    deploy:
      replicas: 1
      update_config:
        delay: 20s
        failure_action: rollback
      restart_policy:
        condition: any
      labels:
        traefik.backend: 'api'
        traefik.docker.network: 'mycompany_traefik-net'
        traefik.enable: 'true'
        traefik.port: '8888'
    networks:
      - traefik-net
      - app-net
    dns:
      - 1.1.1.1
      - 8.8.8.8
      - 8.8.4.4

I am starting to think the problem may be in the traefik config section of base.yml. I see it’s defining the backend port of 8888, but 8889 is not defined for the socket connections. I am uncertain how to configure it.