awk does not print desired df output - Ubuntu

learner
September 10, 2022
229 views
2 votes
3 Answers

I want to identify the ebs volumes which have less than 100 [ Or any other value which I would like to pass as a parameter(variable) ]

Less than X GB is from the "Available" Column.

Note:

Question 2: I am not asking here but giving input just so responder would know what is the end goal I am trying to achieve. I would like to use those ebs or backup volumes and expand them using aws cli.

Not a working host. Ubuntu 16.04

df -Ph
Filesystem                       Size  Used Avail Use% Mounted on
udev                             241G     0  241G   0% /dev
tmpfs                             49G  4.1G   44G   9% /run
/dev/xvda1                        20G   15G  5.3G  73% /
tmpfs                            241G     0  241G   0% /dev/shm
tmpfs                            5.0M     0  5.0M   0% /run/lock
tmpfs                            241G     0  241G   0% /sys/fs/cgroup
/dev/md1                         6.0T  4.1T  2.0T  68% /var/logt/backup
/dev/mapper/vg0-lv_lookups_swap  406G  161G  245G  40% /var/logt/data
/dev/mapper/vg0-lv_ssd5          834G  372G  463G  45% /var/logt/ssd5
/dev/mapper/vg0-lv_ssd2          834G  298G  537G  36% /var/logt/ssd2
/dev/mapper/vg0-lv_ssd0          834G  281G  553G  34% /var/logt/ssd0
/dev/mapper/vg0-lv_ssd3          834G  243G  592G  30% /var/logt/ssd3
/dev/mapper/vg0-lv_ssd1          834G  270G  564G  33% /var/logt/ssd1
/dev/mapper/vg0-lv_ssd4          834G  304G  531G  37% /var/logt/ssd4
/dev/mapper/vg0-lv_ssd7          834G  302G  533G  37% /var/logt/ssd7
/dev/mapper/vg0-lv_ssd6          834G  392G  443G  47% /var/logt/ssd6
/dev/xvdcj                       4.8T  4.6T  135G  98% /var/logt/ebs4/b00
/dev/xvdcl                       4.8T  4.6T  136G  98% /var/logt/ebs5/b00
/dev/xvdcf                       5.0T  4.8T  202G  97% /var/logt/ebs2/b00
/dev/xvdcb                       4.8T  4.7T   74G  99% /var/logt/ebs0/b00
/dev/xvdci                       5.0T  4.8T  223G  96% /var/logt/ebs3/b01
/dev/xvdce                       4.8T  4.6T  132G  98% /var/logt/ebs1/b01
/dev/xvdcn                       5.9T  4.7T  1.2T  80% /var/logt/ebs6/b00
/dev/xvdcy                       5.9T  4.7T  1.3T  80% /var/logt/ebs7/b00
/dev/xvdcd                       4.8T  4.6T  132G  98% /var/logt/ebs1/b00
/dev/xvdcc                       5.9T  4.9T  975G  84% /var/logt/ebs0/b01
/dev/xvdck                       4.8T  4.6T  136G  98% /var/logt/ebs4/b01
/dev/xvdcx                       5.9T  4.8T  1.2T  81% /var/logt/ebs2/b01
/dev/xvdch                       5.9T  4.7T  1.2T  81% /var/logt/ebs3/b00
/dev/xvdco                       5.9T  4.7T  1.2T  80% /var/logt/ebs6/b01
/dev/xvdcq                       5.9T  4.7T  1.3T  80% /var/logt/ebs7/b01
/dev/xvdcm                       4.8T  4.6T  141G  98% /var/logt/ebs5/b01
tmpfs                             49G     0   49G   0% /run/user/1011
tmpfs                             49G     0   49G   0% /run/user/1074
tmpfs                             49G     0   49G   0% /run/user/1081
tmpfs                             49G     0   49G   0% /run/user/1035
tmpfs                             49G     0   49G   0% /run/user/4002
tmpfs                             49G     0   49G   0% /run/user/1002
tmpfs                             49G     0   49G   0% /run/user/4031
tmpfs                             49G     0   49G   0% /run/user/4030
tmpfs                             49G     0   49G   0% /run/user/4043

df -Ph|grep ebs
/dev/xvdcj                       4.8T  4.6T  135G  98% /var/logt/ebs4/b00
/dev/xvdcl                       4.8T  4.6T  136G  98% /var/logt/ebs5/b00
/dev/xvdcf                       5.0T  4.8T  202G  97% /var/logt/ebs2/b00
/dev/xvdcb                       4.8T  4.7T   74G  99% /var/logt/ebs0/b00
/dev/xvdci                       5.0T  4.8T  223G  96% /var/logt/ebs3/b01
/dev/xvdce                       4.8T  4.6T  132G  98% /var/logt/ebs1/b01
/dev/xvdcn                       5.9T  4.7T  1.2T  80% /var/logt/ebs6/b00
/dev/xvdcy                       5.9T  4.7T  1.3T  80% /var/logt/ebs7/b00
/dev/xvdcd                       4.8T  4.6T  132G  98% /var/logt/ebs1/b00
/dev/xvdcc                       5.9T  4.9T  975G  84% /var/logt/ebs0/b01
/dev/xvdck                       4.8T  4.6T  136G  98% /var/logt/ebs4/b01
/dev/xvdcx                       5.9T  4.8T  1.2T  81% /var/logt/ebs2/b01
/dev/xvdch                       5.9T  4.7T  1.2T  81% /var/logt/ebs3/b00
/dev/xvdco                       5.9T  4.7T  1.2T  80% /var/logt/ebs6/b01
/dev/xvdcq                       5.9T  4.7T  1.3T  80% /var/logt/ebs7/b01
/dev/xvdcm                       4.8T  4.6T  141G  98% /var/logt/ebs5/b01
root@datanode-3-pro-cloud-shared-aws-eu-west-1:~# df -Ph | grep ebs|awk 'NR > 1 { gsub("%","",$5); if ($4 < 100) { system("echo " $6 " ebs volumes has less than 100GB disk space") }}'
/var/logt/ebs6/b00 ebs volumes has less than 100GB disk space
/var/logt/ebs7/b00 ebs volumes has less than 100GB disk space
/var/logt/ebs2/b01 ebs volumes has less than 100GB disk space
/var/logt/ebs3/b00 ebs volumes has less than 100GB disk space
/var/logt/ebs6/b01 ebs volumes has less than 100GB disk space
/var/logt/ebs7/b01 ebs volumes has less than 100GB disk space

As you would see the output is incorrect.

The correct output I could achieve is the below host ( Ubuntu 20.04 ):


df -Ph | grep ebs|awk 'NR > 1 { gsub("%","",$5); if ($4 > 270) { system("echo " $6 " ebs volumes has less than 100GB disk space") }}'
/var/logt/ebs6/b00 ebs volumes has less than 100GB disk space
/var/logt/ebs6/b01 ebs volumes has less than 100GB disk space
/var/logt/ebs7/b01 ebs volumes has less than 100GB disk space
/var/logt/ebs7/b00 ebs volumes has less than 100GB disk space

df -Ph
Filesystem                       Size  Used Avail Use% Mounted on
udev                             241G     0  241G   0% /dev
tmpfs                             49G  1.7M   49G   1% /run
/dev/sda1                         29G   13G   17G  43% /
tmpfs                            241G  672K  241G   1% /dev/shm
tmpfs                            5.0M     0  5.0M   0% /run/lock
tmpfs                            241G     0  241G   0% /sys/fs/cgroup
/dev/loop1                        68M   68M     0 100% /snap/lxd/22753
/dev/mapper/vg0-lv_lookups_swap  5.0T  119G  4.9T   3% /var/logt/data
/dev/md1                         4.9T  3.6T  1.4T  73% /var/logt/backup
/dev/mapper/vg0-lv_ssd0          495G  125G  370G  26% /var/logt/ssd0
/dev/mapper/vg0-lv_ssd1          495G  127G  369G  26% /var/logt/ssd1
/dev/mapper/vg0-lv_ssd2          495G  126G  370G  26% /var/logt/ssd2
/dev/mapper/vg0-lv_ssd3          495G  125G  371G  26% /var/logt/ssd3
/dev/mapper/vg0-lv_ssd4          495G  126G  370G  26% /var/logt/ssd4
/dev/mapper/vg0-lv_ssd5          495G  128G  368G  26% /var/logt/ssd5
/dev/mapper/vg0-lv_ssd6          495G  211G  284G  43% /var/logt/ssd6
/dev/mapper/vg0-lv_ssd7          495G  208G  288G  42% /var/logt/ssd7
/dev/sdh                         1.3T  1.1T  261G  80% /var/logt/ebs2/b01
/dev/sdd                         1.3T  1.1T  252G  81% /var/logt/ebs0/b01
/dev/sdg                         1.3T  1.1T  263G  80% /var/logt/ebs2/b00
/dev/sdc                         1.3T  1.1T  246G  82% /var/logt/ebs0/b00
/dev/sdj                         1.3T  1.1T  258G  81% /var/logt/ebs3/b01
/dev/sdf                         1.3T  1.1T  252G  81% /var/logt/ebs1/b01
/dev/sde                         1.3T  1.1T  246G  82% /var/logt/ebs1/b00
/dev/sdi                         1.3T  1.1T  258G  81% /var/logt/ebs3/b00
/dev/sdk                         1.3T  1.1T  259G  81% /var/logt/ebs4/b00
/dev/sdl                         1.3T  1.1T  259G  81% /var/logt/ebs4/b01
/dev/sdp                         1.3T  1.1T  274G  79% /var/logt/ebs6/b00
/dev/sdm                         1.3T  1.1T  261G  80% /var/logt/ebs5/b00
/dev/sdq                         1.3T  1.1T  274G  79% /var/logt/ebs6/b01
/dev/sdo                         1.3T  1.1T  269G  80% /var/logt/ebs5/b01
/dev/sds                         1.3T 1023G  277G  79% /var/logt/ebs7/b01
/dev/sdr                         1.3T  1.1T  274G  79% /var/logt/ebs7/b00
/dev/sda15                       105M  5.2M  100M   5% /boot/efi
tmpfs                            241G  6.5M  241G   1% /tmp
/dev/loop6                        47M   47M     0 100% /snap/snapd/16010
/dev/loop7                        47M   47M     0 100% /snap/snapd/16292
/dev/loop3                        62M   62M     0 100% /snap/core20/1611
/dev/loop4                        64M   64M     0 100% /snap/core20/1623
tmpfs                             49G     0   49G   0% /run/user/4038
tmpfs                             49G     0   49G   0% /run/user/25

df -Ph |grep "/var/logt/ebs6/b00"
/dev/sdp                         1.3T  1.1T  274G  79% /var/logt/ebs6/b00

df -Ph |grep "/var/logt/ebs6/b01"
/dev/sdq                         1.3T  1.1T  274G  79% /var/logt/ebs6/b01

df -Ph |grep "/var/logt/ebs7/b01"
/dev/sds                         1.3T 1023G  277G  79% /var/logt/ebs7/b01

df -Ph |grep "/var/logt/ebs7/b00"
/dev/sdr                         1.3T  1.1T  274G  79% /var/logt/ebs7/b00

Need some guidance please.

Tags: awk linux

Answers

You need to convert units of Terabyte and Megabyte to Gigabyte before making a comparison. You may use this awk script:

cat df.awk

NR > 1 && index($NF, "/ebs") {
   u = substr($4, length($4), 1)
   x = 1
   if (u == "T") x = 1024;
   else if (u == "M") x=1/1024;
   if (($4 * x) < sz)
      print $NF, "ebs volumes has less than", sz "GB disk space", $4
}

# run it with 100GB limit
df -Ph | awk -f df.awk -v sz='100'

/var/logt/ebs0/b00 ebs volumes has less than 100GB disk space 74G

# or with 200GB
df -Ph | awk -f df.awk -v sz='200'

/var/logt/ebs4/b00 ebs volumes has less than 200GB disk space 135G
/var/logt/ebs5/b00 ebs volumes has less than 200GB disk space 136G
/var/logt/ebs0/b00 ebs volumes has less than 200GB disk space 74G
/var/logt/ebs1/b01 ebs volumes has less than 200GB disk space 132G
/var/logt/ebs1/b00 ebs volumes has less than 200GB disk space 132G
/var/logt/ebs4/b01 ebs volumes has less than 200GB disk space 136G
/var/logt/ebs5/b01 ebs volumes has less than 200GB disk space 141G

- JukkaMatilainen
- September 10, 2022 at 9:18 am
- 0 votes
0
You can make it easier by making df always output sizes in gigabytes:
```
df -P -BG | awk -v sz=100 'NR > 1 && $4+0 < sz {print $6 " ebs volumes has less than " sz "GB disk space"}'
```
From the df man page:

-B, –block-size=SIZE

scale sizes by SIZE before printing them; e.g., ‘-BM’ prints sizes in
units of 1,048,576 bytes; see SIZE format below

[…]

The SIZE argument is an integer and optional unit (example: 10K is
10*1024). Units are K,M,G,T,P,E,Z,Y (powers of 1024) or KB,MB,…
(powers of 1000).

The GNU Awk User’s Guide has the tip for converting strings with suffixes like "100G" to plain numbers like 100 by adding zero:

To force a string to be converted to a number, add zero to that string. A string is converted to a number by interpreting any numeric prefix of the string as numerals: "2.5" converts to 2.5, "1e3" converts to 1,000, and "25fix" has a numeric value of 25.
Login or Signup to reply.

no need to force convert anything to a single base unit – e.g. to get all drives with more than 270 GB (1024^3) free space :

df -Ph |

{m,n,g}awk -v __='270G' 'BEGIN {
  __*=(____=(_+=_+=_^=___ = "KMGTPEZY")^_*_
       )    ^ index(___, substr(__, length(__)))
} __ <(____ ^ index(___, substr($_, length($_))) * $_)'

/dev/md1                         6.0T  4.1T  2.0T  68% /var/logt/backup
/dev/mapper/vg0-lv_ssd5          834G  372G  463G  45% /var/logt/ssd5
/dev/mapper/vg0-lv_ssd2          834G  298G  537G  36% /var/logt/ssd2
/dev/mapper/vg0-lv_ssd0          834G  281G  553G  34% /var/logt/ssd0
/dev/mapper/vg0-lv_ssd3          834G  243G  592G  30% /var/logt/ssd3
/dev/mapper/vg0-lv_ssd1          834G  270G  564G  33% /var/logt/ssd1
/dev/mapper/vg0-lv_ssd4          834G  304G  531G  37% /var/logt/ssd4
/dev/mapper/vg0-lv_ssd7          834G  302G  533G  37% /var/logt/ssd7
/dev/mapper/vg0-lv_ssd6          834G  392G  443G  47% /var/logt/ssd6
/dev/xvdcn                       5.9T  4.7T  1.2T  80% /var/logt/ebs6/b00
/dev/xvdcy                       5.9T  4.7T  1.3T  80% /var/logt/ebs7/b00
/dev/xvdcc                       5.9T  4.9T  975G  84% /var/logt/ebs0/b01
/dev/xvdcx                       5.9T  4.8T  1.2T  81% /var/logt/ebs2/b01
/dev/xvdch                       5.9T  4.7T  1.2T  81% /var/logt/ebs3/b00
/dev/xvdco                       5.9T  4.7T  1.2T  80% /var/logt/ebs6/b01
/dev/xvdcq                       5.9T  4.7T  1.3T  80% /var/logt/ebs7/b01

That same code can also be leveraged to convert any file size suffixes into bytes :

     1   934.11062Y        1129270446894229419454874381
     2   682.89266Y         825566568699314292028705247
     3   472.80939Y         571591479327242855169030503
     4   400.19753Y         483809126963000147583350110
     5   161.27283Y         194966888189320756405429421
     6   681.31735Z            804357554459391768108885
     7   557.23389Z            657865661313767691336925

     8   381.27249Z            450127106904062994010614
     9   303.09825Z            357835254204111110598033
    10    43.21891Z             51023883002539934555664
    11   995.74610E              1148017091818399909648
    12   950.81981E              1096220605955196366419
    13   767.43677P                  864056987850604260
    14   444.38438T                     488605793012028

    15   425.44362T                     467780207153113
    16   320.48011T                     352371607415931
    17    83.03899T                      91302335063774
    18    77.53498T                      85250612069379
    19   622.37739G                        668272633954
    20   437.59758G                        469866823727
    21   216.17984G                        232121335713

    22   819.04005M                           858825739
    23   301.55676M                           316205181
    24   859.38582K                              880011
    25   637.11727K                              652408
    26          153                                 153

Please signup or login to give your own answer.

Click here to cancel reply.

awk does not print desired df output – Ubuntu

Answers