CentOS Question posted in
The official documentation can be found here.

Why is my pre-built cloud foundry node app trying to fetch modules that are already in node_modules? – CentOS

I have a pre-built node app that must not attempt to access the network if npm install is run prior to startup – the intent is for everything to already be present in the node_modules directory. It is being deployed in a cloud foundry environment.

This is not a proxy problem – there must be no attempt to access registry url’s during cloud foundry app staging. I’m looking for ideas why it’s trying to do this.

When this app is deployed to cloud foundry, even though it detects the presence of the node_modules directory during staging, it still tries to retrieve basic dependency modules (like @node/types) that are already present in node_modules, and of course it times out trying to reach a registry that is not permitted in these environments. There are hundreds of other dependencies that it does not attempt to retrieve, but for some reason it thinks it does need some modules. For example:

   2021-03-17T16:29:57.71-0700 [STG/0] OUT        Installing any new modules (package.json + package-lock.json)
   2021-03-17T16:32:31.78-0700 [STG/0] OUT npm ERR! code ECONNREFUSED
   2021-03-17T16:32:31.78-0700 [STG/0] OUT npm ERR! errno ECONNREFUSED
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR! FetchError: request to https://<registry-fqdn>/@types%2flong failed, reason: connect ECONNREFUSED 10.x.x.x:443
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at ClientRequest.<anonymous> (/tmp/contents784086672/deps/0/node/lib/node_modules/npm/node_modules/node-fetch-npm/src/index.js:68:14)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at ClientRequest.emit (events.js:315:20)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at TLSSocket.socketErrorListener (_http_client.js:469:9)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at TLSSocket.emit (events.js:315:20)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at emitErrorNT (internal/streams/destroy.js:106:8)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at emitErrorCloseNT (internal/streams/destroy.js:74:3)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at processTicksAndRejections (internal/process/task_queues.js:80:21)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!  FetchError: request to https://<registry-fqdn>/@types%2flong failed, reason: connect ECONNREFUSED 10.x.x.x:443
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at ClientRequest.<anonymous> (/tmp/contents784086672/deps/0/node/lib/node_modules/npm/node_modules/node-fetch-npm/src/index.js:68:14)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at ClientRequest.emit (events.js:315:20)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at TLSSocket.socketErrorListener (_http_client.js:469:9)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at TLSSocket.emit (events.js:315:20)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at emitErrorNT (internal/streams/destroy.js:106:8)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at emitErrorCloseNT (internal/streams/destroy.js:74:3)
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!     at processTicksAndRejections (internal/process/task_queues.js:80:21) {
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!   type: 'system',
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!   errno: 'ECONNREFUSED',
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!   code: 'ECONNREFUSED',
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!   parent: 'app'
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR! }
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR!
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR! If you are behind a proxy, please make sure that the
   2021-03-17T16:32:31.79-0700 [STG/0] OUT npm ERR! 'proxy' config is set properly.  See: 'npm help config'
   2021-03-17T16:39:12.54-0700 [STG/0] OUT npm ERR! A complete log of this run can be found in:
   2021-03-17T16:39:12.54-0700 [STG/0] OUT npm ERR!     /home/vcap/.npm/_logs/2021-03-17T23_32_31_800Z-debug.log
   2021-03-17T16:39:12.56-0700 [STG/0] OUT        **ERROR** Unable to build dependencies: exit status 1
   2021-03-17T16:39:13.07-0700 [STG/0] ERR Failed to compile droplet: Failed to run all supply scripts: exit status 14
   2021-03-17T16:39:13.09-0700 [STG/0] OUT Exit status 223
   2021-03-17T16:39:13.28-0700 [STG/0] OUT Cell 5cee670a-6f6c-4510-a274-5584f197038c stopping instance 59dda306-be2f-4d08-830c-77c08ffab3f5
   2021-03-17T16:39:13.28-0700 [STG/0] OUT Cell 5cee670a-6f6c-4510-a274-5584f197038c destroying container for instance 59dda306-be2f-4d08-830c-77c08ffab3f5
   2021-03-17T16:39:13.76-0700 [API/1] ERR Failed to stage build: staging failed

Any ideas?

Edit #1
Other facts:

  1. the app is pushed as a zip archive
  2. the zip file system includes the node_modules directory resulting from an npm install command run during the "build" to construct the zip
  3. the zip file system includes package-lock.json (from source)
  4. there is no .cfignore file anywhere in the zip archive
  5. the zip ‘build’ used to be taking place on a windows machine, and did not previously have this problem when subsequently pushed to cf
  6. the zip ‘build’ recently migrated to a gitlab ci runner from a k8s cluster, which may be using a centos-derived image
  7. the buildpack version is 1.7.32
  8. the version of node (14.14.0) on the build machine matches the version used by the buildpack, but the npm version (7.6.1) does not (buildpack uses 6.14.8)
  9. changing the build image where the zip is assembled to Ubuntu Bionic Beaver (18.04.4 LTS) makes no difference
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    This problem turned out to be caused by mismatched npm versions on the machine where the package-lock.json file is generated, the machine where the node_modules directory is populated for inclusion in the zip distribution (npm v7.x), and what the cloud foundry build pack is using (npm v6.x).

    Changing the dev and build machines to use npm v6.x to generate both the package-lock.json file and populate the node_modules directory resulted in a successfully-vendored nodejs app.


  2. 0

    Not 100% sure this will resolve the issue, but here’s what I’ve seen commonly trip people up trying to vendor Node.js dependencies:

    First, review the instructions from the docs: https://docs.cloudfoundry.org/buildpacks/node/index.html#vendoring

    1. Run npm install (you’ve done this)
    2. Make sure you have a package-lock.json file (you probably have this). That locks in the versions to use and should guarantee what’s in node_modules/ matches what will be installed. If you don’t have this, you’ll see Warning: package-lock.json not found. output from the buildpack.
    3. Make sure you’re pushing up package.json, package-lock.json, and the full node_modules/ directory, along with all your application code.

    These are not documented, but some tips from what I’ve observed working with others on this:

    1. Make sure you don’t have a .cfignore file, as that could accidentally cause node_modules/ to not be pushed. You can typically tell if node_modules/ is not being pushed because the number of files & size of what is being pushed will be much, much larger with this. You will also see the message It is recommended to vendor the application's Node.js dependencies if the node_modules/ directory doesn’t exist.

    2. When you run npm install locally, you need to be running it in an Ubuntu Bionic VM or container. This is because NPM will often install modules that require native code. It will automatically handle this, but what’s in node_modules/ is specific for the OS & architecture where you run npm install. Thus If you run npm install on a non-Ubuntu Bionic OS (say Windows or MacOS), it will compile the native code for your local machine. When you push this up, it won’t match and NPM will try to reinstall the package which can trigger access to the Internet.

    3. Make sure you’re using the latest Node.js buildpack & Node.js version as possible. It always helps to have the latest code with bug fixes.

    4. NODE_ENV=production will be set by the buildpack, and this can sometimes cause difference in behavior. It also skips installing dev dependencies. Not likely the issue here, but worth mentioning as this trips up some people.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search