Question 301008 in Ubuntu Question posted in
The official Ubuntu documentation can be found here.

Ubuntu – Azure Devops – untracked file preventing newer version of the file from being pushed

My goal is to run a DevOps pipeline (in my repo’s main branch) that does the following:

  • run a Python script (in my repo’s main branch) that returns a json called myJson.json
  • push myJson.json to another branch in the same repo called DIN_grafana_json_archives.
  • I would like to be able to run the exact same script periodically, so that an updated version of myJson.json is pushed to DIN_grafana_json_archives every time the script runs. That would allow me to keep an archive of the different versions of the json.

Initial situation

This is my main branch:
enter image description here

23 11 24 myTestPipeline.yml is my pipeline.

myTestPythonFile.py is the script called within the pipeline.

The DIN_grafana_json_archives branch already exists and is empty.

Here is my pipeline:


trigger: none

pool:
  vmImage: ubuntu-latest

steps:
    # Check out source files and persist the Git Credentials for use in next steps.
    - checkout: self
      persistCredentials: true

    # Generate a new 'myJson.json' file with the new content in the Python script. 
    - script: python3 -m pip install --upgrade pip
      displayName: 'upgrade pip'
    - script: python3 -m pip install pandas
      displayName: 'install pandas'
    - script: python3 -m pip install requests
      displayName: 'install requests'
    - script: python3 -m pip install datetime
      displayName: 'install datetime'
    
    - task: PythonScript@0
      inputs:
        scriptSource: 'filePath'
        scriptPath: '$(System.DefaultWorkingDirectory)/myTestPythonFile.py'
    
    # Commit and push the updated 'myJson.json' file to remote 'DIN_grafana_json_archives' branch.
    - task: Bash@3
      displayName: 'Push updates to remote'
      inputs:
        targetType: inline
        script: |
          git config --global user.name {myUsername}
          git config --global user.email {[email protected]}
          git checkout DIN_grafana_json_archives
          git add myJson.json
          git commit -m "Update file myJson.json"
          git push

The script runs fine and myJson.json is successfully pushed to DIN_grafana_json_archives.

Problem when I run the script a 2nd time

Objective: I would like the script to push a new version of myJson.json (this version could potentially be identical to the previous one) to DIN_grafana_json_archives, even though it already contains (an older version of) this file.

Problem: the script fails at th last stage with this error message:

error: The following untracked working tree files would be overwritten by checkout:
    myJson.json
Please move or remove them before you switch branches.
Aborting
[detached HEAD db4eeee] Update file myJson.json
 1 file changed, 285 insertions(+)
 create mode 100644 myJson.json
fatal: You are not currently on a branch.
To push the history leading to the current (detached HEAD)
state now, use

    git push origin HEAD:<name-of-remote-branch>

##[error]Bash exited with code '128'.
Finishing: Push updates to remote

If I navigate to the DIN_grafana_json_archives branch and delete myJson.json, the script runs fine.
What do I need to do for my script to be able to run multiple times? Thanks.

EDIT

Replacing

git checkout DIN_grafana_json_archives

with

git checkout -f DIN_grafana_json_archives

prevents the last step from failing. The logs for this step read:

Previous HEAD position was bdf49df Update 23 11 24 myTestPipeline.yml for Azure Pipelines
Switched to a new branch 'DIN_grafana_json_archives'
branch 'DIN_grafana_json_archives' set up to track 'origin/DIN_grafana_json_archives'.
On branch DIN_grafana_json_archives
Your branch is up to date with 'origin/DIN_grafana_json_archives'.

nothing to commit, working tree clean
Everything up-to-date
Finishing: Push updates to remote

However, no updated version of myJson.json is pushed to the target repo. Why is that? Why do the logs say "nothing to commit" even though the next lines are git add / commit / push ?

Tags:

2

Answers


  1. 0

    Instead of checking DIN_grafana_json_archives out which is causing the error you can just push directly on it using

    git push origin HEAD:DIN_grafana_json_archives

    Just leave out checkout step in your script and replace git push with this command.

    Login or Signup to reply.
  2. 0

    You’re seeing this because checkout command works with the commit at the time the pipeline was queued rather than the latest values in the branch. This enables the pipeline to be "re-run" deterministically from a specific point in time. Under the hood, git refers to this scenario is as a detached head.

    If you look at the output from the checkout command, you’ll see something like this:

    snippet from pipeline

    Keep in mind that the branch may have changed from the time that the pipeline was originally queued, so you may need to fetch the latest changes to prevent your push from being rejected. If your pipeline runs automatically as a batched trigger, this is less likely to occur as you should always have the latest.

    git fetch
git checkout $branch
git pull

    The second thing to consider is you should checkout and update before you modify the contents of the repository as git will prevent you from switching with untracked changes. As you discovered, this is why you had to add the --force parameter.

    Lastly, as an optimization, you should commit and push only if there are modifications. For example, suppose your script produces a result that is identical to the current copy of your file.

    Assuming you are only modifying existing files, the following should work:

    changes=$(git ls-files --modified --deleted)

if [ -n "$changes" ]; then
  # stage and add all changes
  git commit --all -m"Updated from pipeline [skip ci]"
  git push
fi
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search