I’d like to utilize Github Actions to pull specific files from another repo whenever they get updated.
What I tried to do is:
- Pull specific files from someone else’s repo (and use a token if it’s a private repo).
- Rename some of those files.
- Create a PR with them in my repo.
- Allow to control whether to do manually or also automatically (when the source repo updates itself).
If I somehow missed older questions about it, remember it needs to use the latest Node version due to Github’s blockage of older Node versions.
Sample codes (trying to pull from a public repo) is below (with v3 in both actions/checkout@v3 and actions/github-script@v3 in an attempt to use the latest Node), but it triggers the following errors, could you help me figure out what’s wrong?
Sample code 1:
pull-file Unhandled error: SyntaxError: Invalid or unexpected token
pull-file Node.js 12 actions are deprecated. Please update the
following actions to use Node.js 16: actions/github-script@v3. For
more information see:
https://github.blog/changelog/2022-09-22-github-actions-all-actions-will-begin-running-on-node16-instead-of-node12/.
name: Pull File from Another Repository on Push
env:
SOURCE_REPO: https://github.com/some/repo
AUTOMATIC_MONITOR: true
FILES: "*.txt,LICENSE" # seperate multiple entries with ,
FILES_NEW: "*.txt,LICENSE*" # seperate multiple entries with ,
EXTRA: mv LICENSE LICENSE_some_repo # do an extra command like renaming
# TOKEN_NAME: ACCESS_TOKEN # uncomment to use a secret with that name
# SOURCE_BRANCH: source_alternate # uncomment to not monitor the default branch
# TARGET_BRANCH: target_alternate # uncomment to not push to default branch
on: # Remove # below for non default branches
workflow_dispatch:
# inputs:
# source_branch:
# description: '${{ env.SOURCE_BRANCH }}'
# required: false
push:
# branches:
# - ${{ env.SOURCE_BRANCH }}
paths:
- ./source-repo/**
jobs:
pull-file:
runs-on: ubuntu-latest
steps:
- name: Check whether to automatically monitor
if: ${{ github.event_name != 'workflow_dispatch' && env.AUTOMATIC_MONITOR == false }}
run: |
exit 1
- uses: actions/checkout@v3
- name: Pull File
uses: actions/github-script@v3
with:
script: |
#!/bin/sh
if [ -n "${{ env.TOKEN_NAME }}" ]; then
REPO_URL=https://${{ secrets[env.TOKEN_NAME] }}@${{ env.SOURCE_REPO }}.git
else
REPO_URL=${{ env.SOURCE_REPO }}
fi
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
if [ -n "${{ github.event.inputs.source_branch }}" ]; then
git clone -b ${{ github.event.inputs.source_branch }} $REPO_URL source-repo
else
git clone $REPO_URL source-repo
fi
else
git clone -b ${{ github.ref }} $REPO_URL source-repo
fi
cp source-repo/{${{ env.FILES }}} .
${{ env.EXTRA }}
- name: Commit
run: |
git add ${{ env.FILES_NEW }}
git commit -m "Pulled files from ${{ env.SOURCE_REPO }}."
- name: Push
run: |
if [ -n "${{ env.TARGET_BRANCH }}" ]; then
git push origin ${{ env.TARGET_BRANCH }}
else
git push
fi
Sample code 2:
Error:
git add $FILES_NEW
git commit -m "Pulled files from $SOURCE_REPO."
shell: /usr/bin/bash -e {0}
Author identity unknown
*** Please tell me who you are.
Run
git config --global user.email "[email protected]"
git config --global user.name "Your Name"
to set your account's default identity.
Omit --global to set the identity only in this repository.
fatal: empty ident name (for <runner@...>) not allowed
Error: Process completed with exit code 128.
Code:
name: Pull File from Another Repository on Push
env:
SOURCE_REPO: https://github.com/some/repo
AUTOMATIC_MONITOR: true
FILES: "*.txt,LICENSE" # seperate multiple entries with ,
FILES_NEW: "*.txt,LICENSE*" # seperate multiple entries with ,
EXTRA: mv LICENSE LICENSE_some_repo # do an extra command like renaming
# TOKEN_NAME: ACCESS_TOKEN # uncomment to use a secret with that name
# SOURCE_BRANCH: source_alternate # uncomment to not monitor the default branch
# TARGET_BRANCH: target_alternate # uncomment to not push to default branch
on: # Remove # below for non default branches
workflow_dispatch:
# inputs:
# source_branch:
# description: '${{ env.SOURCE_BRANCH }}'
# required: false
push:
# branches:
# - ${{ env.SOURCE_BRANCH }}
paths:
- ./source-repo/**
jobs:
pull-file:
runs-on: ubuntu-latest
steps:
- name: Check whether to automatically monitor
if: ${{ github.event_name != 'workflow_dispatch' && env.AUTOMATIC_MONITOR == false }}
run: |
exit 1
- name: Checkout
uses: actions/checkout@v3
- name: Pull File
env:
# THE_TOKEN: ${{ secrets.$TOKEN_NAME }}
EVENT_NAME: ${{ github.event_name }}
SOURCE_BRANCH: ${{ github.event.inputs.source_branch }}
REF: ${{ github.ref }}
run: |
#!/bin/sh
if [ -n "$TOKEN_NAME" ]; then
REPO_URL=https://$THE_TOKEN@$SOURCE_REPO.git
else
REPO_URL=$SOURCE_REPO.git
fi
if [ "$EVENT_NAME" = "workflow_dispatch" ]; then
if [ -n "$SOURCE_BRANCH" ]; then
git clone -b $SOURCE_BRANCH $REPO_URL source-repo
else
git clone $REPO_URL source-repo
fi
else
git clone -b $REF $REPO_URL source-repo
fi
cd source-repo
$EXTRA
mv $FILES_NEW ../
cd ..
- name: Commit
run: |
git add $FILES_NEW
git commit -m "Pulled files from $SOURCE_REPO."
- name: Push
run: |
if [ -n "$TARGET_BRANCH" ]; then
git push origin $TARGET_BRANCH
else
git push
fi
2
Answers
You’re left with needing a safe way to pass the credentials to Git.
You can set a custom credential manager to pass these details from the environment:
That will allow you to pass the credentials safely from the environment variables into git, without having to inject the token into the remote url:
You’ll need to pass the token to the push step too, it might be simpler to declare them at the job level instead:
Your commit step should beleverage the custom credential manager we’ve setup in the pull step.
I use a simple
runcommand to run a script. Albeit, it won’t update when they get an update, but will fetch latest file when your github actions triggers.