Ubuntu - how to update a file within an ADO pipeline

Eeuk
February 12, 2024
197 views
0 votes
2 Answers

I’m trying to develop a simple pipeline with azure as a bit of a POC.

The aim is to do the following:

trigger when someone commits an updated json file
read the json file and identify a "version" attribute and increment it by 1
write the update file back
commit the changes back to git

The pipleline appears to trigger correctly and creates a git repo with a detached HEAD in the first step. This is all standard code from azure.

I’ve used a simple python script to read the json file, update the attribute and write it back.

What I’m struggling to do is get those changes committed back to the origin git repo, a the minute they just disappear at the end of the pipleline.

I’ve tried a few things, latest attempt is to use:

git add ./Content/basic.json
git commit -m 'update version number [skip ci]'
git push origin Development
(These are run from the python script using subprocess.run following the update to the json file)

The add and commit appear to work but the push fails with this:

error: src refspec Development does not match any
error: failed to push some refs to ‘url for my source repo’

When I’ve run git branch commands its seems I can’t see any of the branches in either the remote origin repo or the headless clone.

any assistance appreciated, is there an easier way of getting the file changes made in the origin repo?

UPDATE:

I have a repo called "Simple". This has three branches, main (default) in ADO, Development and Dev.

I’m working in the Development branch in ADO and vscode. When i make changes to files in the Development branch the pipeline triggers and runs through the following steps:

initialise job: this is a standard pipeline step
Checkout Simple@Development to s

this runs a number of git commands that have been autogenerated by Azure. I suspect the output of that step maybe providing the answer, I just don’t know what it means.

Here’s the git commands and output from that step:


==============================================================================

Task         : Get sources
Description  : Get sources from a repository. Supports Git, TfsVC, and SVN repositories.`
Version      : 1.0.0`
Author       : Microsoft`
Help         : [More Information](https://go.microsoft.com/fwlink/?LinkId=798199)`
==============================================================================
Syncing repository: Simple (Git)
git version
git version 2.43.0
git lfs version
git-lfs/3.4.1 (GitHub; linux amd64; go 1.21.1)
git init "/home/vsts/work/1/s"
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint:   git config --global init.defaultBranch <name>
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint:   git branch -m <name>
Initialized empty Git repository in /home/vsts/work/1/s/.git/
git remote add origin <url to repo>
git config gc.auto 0
git config --get-all <url to repo>
git config --get-all http.extraheader
git config --get-regexp .*extraheader
git config --get-all http.proxy
git config http.version HTTP/1.1
git --config-env=http.extraheader=env_var_http.extraheader fetch --force --tags --prune --prune-tags --progress --no-recurse-submodules origin --depth=1 +8520646c06ef567d0494494cc5d2dcc70cb76116:refs/remotes/origin/8520646c06ef567d0494494cc5d2dcc70cb76116
remote: Azure Repos        
remote: 
remote: Found 13 objects to send. (0 ms)        
From <url to repo>
 * [new ref]         8520646c06ef567d0494494cc5d2dcc70cb76116 -> origin/8520646c06ef567d0494494cc5d2dcc70cb76116
git --config-env=http.extraheader=env_var_http.extraheader fetch --force --tags --prune --prune-tags --progress --no-recurse-submodules origin --depth=1 +8520646c06ef567d0494494cc5d2dcc70cb76116
remote: Azure Repos        
remote: 
remote: Found 0 objects to send. (16 ms)        
From <url to repo>
 * branch            8520646c06ef567d0494494cc5d2dcc70cb76116 -> FETCH_HEAD
git checkout --progress --force refs/remotes/origin/8520646c06ef567d0494494cc5d2dcc70cb76116
Note: switching to 'refs/remotes/origin/8520646c06ef567d0494494cc5d2dcc70cb76116'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 8520646 now with a prune
Finishing: Checkout Simple@Development to s

I’m thinking the comment towards the end about git switch – c is aimed at helping me…. I’m just unsure what it means and how I’d use it effectively.

I assume I’d do a git switch -c Development before I update the JSON file, then add/commit it. But how do I get that change back to the origin Development branch, will the push now work?

For completeness other steps in the pipeline are just default ones that I updated with some commands to confirm the default working directory and a call to the python script:

YAML
# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
   - Development

pool:
   vmImage: ubuntu-latest

steps:
 - script: echo Hello, world!
   displayName: 'Run a one-line script'

- script: |
  echo Add other tasks to build, test, and deploy your project.
  echo See https://aka.ms/yaml
  echo ls -la
  ls -la
  echo ls -la ./Scripts
  ls -la ./Scripts
  echo ls -la ./Scripts/update_json_file.py
  ls -la ./Scripts/update_json_file.py
  which git
 
 displayName: 'Run a multi-line script'


- task: PythonScript@0
  displayName: 'Run a python script'
  inputs:
    scriptSource: 'filePath'
    scriptPath: './Scripts/update_json_file.py'```

Tags: azure-pipelines git

Answers

- ravenwing
- February 9, 2024 at 1:52 pm
- 0 votes
0
You try to publish Development branch, which does not exist locally. You have few options there:
- As first step checkout new branch git checkout -b Development – it checkouts new branch named Development. Then make changes and make commit.
- You can specify which local branch should be pushed to remote: git push main:Development – if you work on main on CI, you’ll push your local (local for CI) branch main to your remote Development branch.
But there probably will be more problems further.

If you look for changes on main, and push on Development then on second try you’ll have conflicts.
Login or Signup to reply.

- wadezhouMSFT
- February 12, 2024 at 10:53 am
- 0 votes
0
any assistance appreciated, is there an easier way of getting the file changes made in the origin repo?

You can change the json file in your python script, and add a new task, use git command to push the file back to the repo. Please notice the comment in the yaml below:
```
pool:
  vmImage: ubuntu-latest

steps:
- checkout: self
  persistCredentials: true # This allows later steps to use the system-provided token to execute commands

- script: echo test1 >> basic.json    # you can change the file with python scirpt

- script: |
    git config --global user.email "[email protected]"
    git config --global user.name "test1"
    git add basic.json             # you can use "git add ." to add all files
    git commit -m "Update version number [skip ci]"
    git push origin HEAD:$(Build.SourceBranchName)
  displayName: 'Commit changes'
```
The yaml above use build service scoped identity to push the file to repo, hence, please go to project settings -> Repositories -> target repo -> security -> grant contribute role for the projectname build servie(orgname) account.

The json file updated:
Login or Signup to reply.

Please signup or login to give your own answer.

Click here to cancel reply.

Ubuntu – how to update a file within an ADO pipeline

Answers