Twitter API Question posted in
The official Twitter API documentation can be found here.

delete and copy files using shx in package.json – Twitter API

I have two npm projects, one is a Gatsby project called web-project and another one is Express project called server-project.

This is the file structure of the projects:

- projects
   - web-project
      - public
      - package.json
   - server-project
      - package.json
      - public

If I run command npm run build in web-project/package.json, I would like to generate build files for web-project, delete content in folder server-project/public and copy content from web-project/public to server-project/public.

I tried to use shx, which is installed locally in web-project, but it doesn’t work. Gatsby generates build files, but shx doesn’t copy them.

web-project/package.json

"scripts": {
    "build": "gatsby build && (shx find ../server-project/public -type f -delete && shx cp -R public/* ../server-project/public)"
    ...
  },

npm debug log file:

0 info it worked if it ends with ok
1 verbose cli [
1 verbose cli   '/usr/local/Cellar/node/12.1.0/bin/node',
1 verbose cli   '/usr/local/bin/npm',
1 verbose cli   'run',
1 verbose cli   'build'
1 verbose cli ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prebuild', 'build', 'postbuild' ]
5 info lifecycle [email protected]~prebuild: [email protected]
6 info lifecycle [email protected]~build: [email protected]
7 verbose lifecycle [email protected]~build: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]~build: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/and/devel/tau/web-project/node_modules/.bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
9 verbose lifecycle [email protected]~build: CWD: /Users/and/devel/tau/web-project
10 silly lifecycle [email protected]~build: Args: [
10 silly lifecycle   '-c',
10 silly lifecycle   'gatsby build && (shx find ../server-project/public -type ' +
10 silly lifecycle     'f -delete && shx cp -R public/* ../server-project/public)'
10 silly lifecycle ]
11 silly lifecycle [email protected]~build: Returned: code: 1  signal: null
12 info lifecycle [email protected]~build: Failed to exec build script
13 verbose stack Error: [email protected] build: `gatsby build && (shx find ../server-project/public -type f -delete && shx cp -R public/* ../server-project/public)`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
13 verbose stack     at EventEmitter.emit (events.js:196:13)
13 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at ChildProcess.emit (events.js:196:13)
13 verbose stack     at maybeClose (internal/child_process.js:1000:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:267:5)
14 verbose pkgid [email protected]
15 verbose cwd /Users/and/devel/tau/web-project
16 verbose Darwin 18.7.0
17 verbose argv "/usr/local/Cellar/node/12.1.0/bin/node" "/usr/local/bin/npm" "run" "build"
18 verbose node v12.1.0
19 verbose npm  v6.9.0
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] build: `gatsby build && (shx find ../server-project/public -type f -delete && shx cp -R public/* ../server-project/public)`
22 error Exit status 1
23 error Failed at the [email protected] build script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

I tried to change build command to only copy files from one folder to another folder:

web-project/package.json

{
  "name": "tau",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "develop": "gatsby develop",
    "build": "shx cp -R public/* ../server-project/public",
    "serve": "gatsby serve",
    "start": "node server.js",
    "gh-pages": "gatsby build --prefix-paths && gh-pages -d public",
    "lint": "eslint src --fix",
    "dev": "(shx --silent rm -rf public .cache || shx true) && gatsby develop",
    "server": "cross-env NODE_ENV=development DEBUG=api nodemon server.js"
  },
  "author": "PI",
  "license": "MIT",
  "dependencies": {
    "@fortawesome/fontawesome-svg-core": "^1.2.25",
    "@fortawesome/free-brands-svg-icons": "^5.12.0",
    "@fortawesome/free-solid-svg-icons": "^5.11.2",
    "@fortawesome/react-fontawesome": "^0.1.7",
    "@kunukn/react-collapse": "1",
    "@material-ui/core": "^4.9.5",
    "@popperjs/core": "^2.3.3",
    "add": "^2.0.6",
    "axios": "^0.19.0",
    "basic-auth": "^2.0.1",
    "bootstrap": "4.2.1",
    "chart.js": "^2.9.3",
    "d3-node": "^2.2.1",
    "debug": "^4.1.1",
    "font-awesome": "4.7.0",
    "framer-motion": "^1.10.3",
    "gatsby": "^2.18.12",
    "gatsby-plugin-canonical-urls": "^2.3.0",
    "gatsby-plugin-dark-mode": "^1.1.0",
    "gatsby-source-prismic-graphql": "^3.5.0",
    "gatsby-transformer-remark": "^2.6.53",
    "lodash.get": "^4.4.2",
    "lodash.groupby": "^4.6.0",
    "lodash.pickby": "^4.6.0",
    "marked": "^0.8.0",
    "moment": "^2.24.0",
    "prismic-reactjs": "^1.3.1",
    "prop-types": "^15.7.2",
    "react-chartjs-2": "^2.8.0",
    "react-collapsible": "^2.6.3",
    "react-d3-components": "^0.9.1",
    "react-d3-library": "^1.1.8",
    "react-headroom": "^3.0.0",
    "react-helmet": "^5.2.0",
    "react-lazyload": "^2.6.5",
    "react-moment": "^0.9.6",
    "react-onclickout": "^2.0.8",
    "react-popper": "^2.2.3",
    "react-popper-tooltip": "^2.11.1",
    "react-responsive": "^8.0.1",
    "react-scroll-to": "^3.0.0-beta.3",
    "react-sidebar": "^3.0.2",
    "react-slick": "^0.25.2",
    "react-svg-donuts": "^1.0.0",
    "react-telegram-embed": "^0.0.10",
    "react-toastify": "^5.4.1",
    "react-twitter-embed": "^3.0.3",
    "react-window": "^1.8.5",
    "reactstrap": "^8.4.1",
    "slick-carousel": "^1.8.1",
    "underscore": "^1.9.1",
    "yarn": "^1.21.1"
  },
  "devDependencies": {
    "@prototype-interactive/eslint-config": "^0.1.1",
    "autoprefixer": "^9.4.4",
    "dotenv": "^8.2.0",
    "eslint": "^5.12.0",
    "gatsby-plugin-google-analytics": "^2.3.0",
    "gatsby-plugin-postcss": "^2.0.2",
    "gatsby-plugin-postcss-sass": "^1.0.22",
    "gatsby-plugin-react-helmet": "^3.0.5",
    "gatsby-plugin-sass": "^2.0.7",
    "gatsby-source-filesystem": "^2.2.2",
    "gatsby-transformer-json": "^2.1.7",
    "gh-pages": "^2.0.1",
    "husky": "^1.3.1",
    "prettier": "^1.15.3",
    "pretty-quick": "^1.8.0",
    "react": "^16.7.0",
    "react-dom": "^16.7.0",
    "shx": "^0.3.2",
    "svg-sprite-loader": "^4.1.3"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/PI/gatsby-react-boilerplate.git"
  },
  "bugs": {
    "url": "https://github.com/PI/gatsby-react-boilerplate/issues"
  },
  "homepage": "https://github.com/PI/gatsby-react-boilerplate#readme"
}

npm debug log file:

0 info it worked if it ends with ok
1 verbose cli [
1 verbose cli   '/usr/local/Cellar/node/12.1.0/bin/node',
1 verbose cli   '/usr/local/bin/npm',
1 verbose cli   'run',
1 verbose cli   'build'
1 verbose cli ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prebuild', 'build', 'postbuild' ]
5 info lifecycle [email protected]~prebuild: [email protected]
6 info lifecycle [email protected]~build: [email protected]
7 verbose lifecycle [email protected]~build: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]~build: PATH: /usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/Users/and/devel/tau/web-project/node_modules/.bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
9 verbose lifecycle [email protected]~build: CWD: /Users/and/devel/tau/web-project
10 silly lifecycle [email protected]~build: Args: [ '-c', 'shx cp -R public/* ../web-project-project/public' ]
11 silly lifecycle [email protected]~build: Returned: code: 1  signal: null
12 info lifecycle [email protected]~build: Failed to exec build script
13 verbose stack Error: [email protected] build: `shx cp -R public/* ../web-project-project/public`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:301:16)
13 verbose stack     at EventEmitter.emit (events.js:196:13)
13 verbose stack     at ChildProcess.<anonymous> (/usr/local/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at ChildProcess.emit (events.js:196:13)
13 verbose stack     at maybeClose (internal/child_process.js:1000:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:267:5)
14 verbose pkgid [email protected]
15 verbose cwd /Users/and/devel/tau/web-project
16 verbose Darwin 18.7.0
17 verbose argv "/usr/local/Cellar/node/12.1.0/bin/node" "/usr/local/bin/npm" "run" "build"
18 verbose node v12.1.0
19 verbose npm  v6.9.0
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] build: `shx cp -R public/* ../web-project-project/public`
22 error Exit status 1
23 error Failed at the [email protected] build script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]
Tags:

3

Answers


  1. 0

    You can drop the wildcard from shx cp -R public/* ../server-project/public, as the recursive flag will pick up everything in the source directory. This should get you what you are after – 

    shx cp -R public ../server-project/public
    Login or Signup to reply.
  2. 0

    It looks like you are already using find in your build system. I leverage tar and find heavily in some of the build systems I maintain, and this works well for me. tar handles a really wide variety of edge cases very nicely, since it was originally written as a backup utility.

    in tar, – is the filename that means stdin or stdout, -T specifies the list of files to backup, and -C specifies its working directory. This lets you glue a bunch of stuff together with the pipeline and some pretty cool stuff done. For example,

    (cd /path/to/src/files && find . -type f -name) 
| tar -T - -cf - 
| tar -C /path/to/dest -xvf -
    • The subshell lets me the directory for find without affecting other commands
    • find produces a list of files on stdout
    • the first tar reads the list of files to archive on stdin
    • the first tar produces the archive on stdout
    • the second tar changes directory to /path/to/dest
    • then it extracts the archive on stdin
    • and ‘v’ prints the list of files it is extract while it does it

    This will preserve ownership, group membership, permissions, symlinks, directory structure, the whole nine yards for you. Even filenames with spaces won’t mess it up. 🙂

    Login or Signup to reply.
  3. 0

    The main issue exists in this part:

    shx find ../server-project/public -type f -delete

    The -type f flag is not supported in shx (see issue #177). Presumably, the -delete flag is also not supported.

    Consider replacing that part with a node.js script that does the following:

    1. Utilizes the shelljs find method.
    2. Iterates the Array of found pathnames using forEach().
    3. In the body of the forEach callback checks whether it’s a file and deletes accordingly.

    For instance, putting it altogether into a single npm script named build:

    "scripts": {
  ...
  "build": "gatsby build && node -e "var sh = require('shelljs'), fs = require('fs'); sh.find('../server-project/public').forEach(function (file) { if (!fs.statSync(file).isDirectory()) { sh.rm(file); }});" && shx cp -R "public" "../server-project/public""
  ...
}

    Notes:

    1. Your attempted usage of find ../server-project/public -type f -delete in your OP implies that your intention is to delete all files (only) from ../server-project/public (n levels deep) – but don’t delete any folders in ../server-project/public (again n levels deep). This is what the node -e "..." command in the aforementioned example replicates.

    2. The nodejs command line option -e is utilized to evaluate the inline Javascript, which is essentially this:

      var sh = require('shelljs'),
    fs = require('fs');

sh.find('../server-project/public').forEach(function (file) {
    if (!fs.statSync(file).isDirectory()) {
        sh.rm(file);
    }
});

      • As you can see, a combination of the Logical NOT (!) operator, the fs.statSync method and isDirectory() are utilized to ascertain whether the pathname is a file and not a directory/folder.

      • The file is deleted using the shelljs rm method.

    3. The globstar (*) in your final shx cp -R ... command, and the parentheses (...) encasing the compound commands, have been omitted.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search