cPanel Question posted in
The official documentation can be seen here.

Concatenation of variable to end of URL in cURL for linux shell script – CPanel

(I have made several edits to this in response to comments)

My goal is to have a shell script that goes to an external server, gets a batch of files from a directory, prints them on a thermal receipt printer, then deletes the same files from the remote server (so that when a cron job runs they don’t get printed again). My problem is in the concatenation of a variable to the end of the URL for cURL. The rest of the script is working but I will show the entire script for clarity

I’ve done several searches for solutions and many seem to involve more complex situations, e.g. this one that tries to solve for a hidden carriage return since the variable is appended to the middle of the URL (c.f. Bash curl and variable in the middle of the url). I tried that solution (and several others) and they didn’t work. My situation is simpler (I think) so maybe those answers added unnecessary complications and that’s my problem? Anyways…

Here’s the full code with placeholders for sensitive info:

!/bin/bash
# step 1 - change to the directory where we want to temporarily store tickets
cd Tickets
# step 2 - get all the tickets in the target directory on the external server and put them in the current temporary local directory
wget --secure-protocol TLSv1_2 --user=<placeholder> --password='<placeholder>d' ftps://<placeholder>/public_html/tickets/*.txt
# step 3 - print each of the tickets in the current temporary local directory
for i in *.txt
do lp $i
done
# step 4 - reach out to the target directory and delete each of the files that we previously downloaded (not the entire directory; might have new files)
for i in *.txt
do curl --verbose --ftp-ssl --user <placeholder>:<placeholder> 'ftp://<placeholder>/public_html/tickets' -Q "DELE /public_html/tickets/$i"
done
# empty the current local directory where we temporarily stored files during the execution of this script
for i in *.txt
do rm $i
done
# should be done now.

I have used all of the following variations for step 4:

for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q 'DELE /public_html/tickets/'$i
done

for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q 'DELE /public_html/tickets/'${i}
done

for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q 'DELE /public_html/tickets/'"$i"
done

for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q 'DELE /public_html/tickets/'"${i}"
done

Output for all four of those is:

curl: (21) QUOT command filed with 550

I was able to confirm that the code works without a variable by testing this:

curl --ftp-ssl --user <placeholder>:<placeholder> ftp://<placeholder>/public_html/tickets/ -Q 'DELE /public_html/tickets/14.txt'

*** EDIT **
I re-read the comments and I think I initially misunderstood some of them. I was able to use echo in front of the curl command to see the output with the variable. This was very helpful, thanks @Bodo. The suggestion from @NationBoneless for the verbose tag was also useful and yielded the following:

< 220-You are user number 1 of 50 allowed.
< 220-Local time is now 18:34. Server port: 21.
< 220-This is a private system - No anonymous login
< 220-IPv6 connections are also welcome on this server.
< 220 You will be disconnected after 30 minutes of inactivity.
AUTH SSL
< 500 This security scheme is not implemented
AUTH TLS
< 234 AUTH TLS OK.
* successfully set certificate verify locations:
*   CAfile: none
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / <placeholder>
* Server certificate:
*  subject: CN=<placeholder>
*  start date: Nov 16 00:00:00 2020 GMT
*  expire date: Nov 16 23:59:59 2021 GMT
*  subjectAltName: host "<placeholder>" matched cert's "<placeholder>"
*  issuer: C=US; ST=TX; L=Houston; O=cPanel, Inc.; CN=cPanel, Inc. Certification Authority
*  SSL certificate verify ok.
USER <placeholder>
< 331 User <placeholder> OK. Password required
PASS <placeholder>
< 230 OK. Current restricted directory is /
PBSZ 0
< 200 PBSZ=0
PROT P
< 200 Data protection level set to "private"
PWD
< 257 "/" is your current location
* Entry path is '/'
DELE /public_html/tickets/14.txt
* ftp_perform ends with SECONDARY: 0
< 550 Could not delete /public_html/tickets/14.txt: No such file or directory
* QUOT command failed with 550
* Closing connection 0
Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    First, I want to thank @Bodo and @NationBoneless. I would not have figured this out without their comments. Thank you very much to both of you.

    My problem had two parts - one was the problem with concatenation (which I knew from the start) but also in how I was setting up my URL path for curl (which I didn't realize until later).

    The correct form for the concatenation is:

    curl --ftp-ssl --user myID:mypassword 'ftp://host.mydomain.com' -Q "DELE /public_html/tickets/$i"

    Some salient points that I learned along the way and which might be useful to others with similarly limited experience include:

    • Try exactly what the commenters tell you to try, even if you think you understand their reasoning and you think they're on the wrong track - they probably aren't wrong.
    • Use double quotes for shell scripts if you plan to have a variable inside the quotes. Single quotes behave very differently and don't play nice with variables.
    • echo is your friend; it was extremely helpful for debugging because it showed me that my variable was being parsed as $i instead of as the value of $i because of those pesky single quotes. Thanks @Bodo
    • the --verbose flag is very helpful. It let me see that curl was connecting to the server and that my script was failing because it couldn't find the file to delete. My URL path was wrong and I didn't realize it. Thanks @NationBoneless
    • I originally put the full path in the ftp: section and then again after -Q but that doesn't work. After much trial and error I got it to work with just ftp:hostname before the -Q and the rest of the path after the -Q.
    • This next part might not apply to anyone else but the server I'm working on is using WHM/cPanel. The domain to which I'm trying to connect is actually not the domain I am using in my code. I had to connect to host.myserverdomain.com (the domain that I use for root admin stuff in WHM) instead of ftp.thisprojectdomain.com (the domain on which I'm actually working). Apparently WHM/cPanel can be set up many ways and that's how my data center set up mine. Not sure how common this is but if you're tearing your hair out, try switching to the WHM domain.
    • For those wondering why my main script uses both wget and curl; I started with wget because I thought I was more familiar with it. wget can't delete files on the server. I switched to curl for deleting the files but never switched the first statement to curl because it worked. There's probably a way to use curl instead of wget; I leave that exercise to the comments section as I'll stick with what works.

  2. 0

    I would think something like this would work. When I tested it locally the curl command seemed to work correctly:

    for i in *.txt
do curl --ftp-ssl --user (myftpid):(mypasswd) ftp://(myhostname)/public_html/tickets/ -Q "DELE /public_html/tickets/$i"
done
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search