Question 361372 in Redis Question posted in
The official documentation can be found here.

Redis – Capturing first line from shell command output and continue processing rest

I’m trying to capture and "store away" (preferably in a variable) the first line of a command output and forward the remainder of the output back to stdin of the next command in a pipe. As an FYI, the purpose is to capture and store the cursor reference of a redis-cli SCAN invocation, while processing the data returned.

This is what I came up with:

command | { read first_line ; cat ; } | process_rest
echo $first_line

This seems to work as intended in zsh, but not in Bourne Shell ("/bin/sh") or bash. It appears that piping to the command grouping causes a subshell in sh/bash, "swallowing" the first_line variable.

What is the best way to achieve what I’m trying to do? Will I need to "pipe" the first line through a file or fifo instead of trying to set a variable? I also tried using a while read line ; do construct to filter out the first line and continue processing the rest, but it appears it has the same effect (any variable changes within the do/done block are not reflected in the outer scope).

Tags:

3

Answers


  1. Chosen as BEST ANSWER
    0

    I think I found a way that works:

    first_line=$(command | { read first_line ; echo $first_line ; process_rest })
echo $first_line

    This will capture the first line while processing the remainder of the data. The main drawback of this approach is that I cannot easily redirect stdout of process_rest as part of the whole construct anymore. E.g.

    get_data() {
  first_line=`command | { read first_line ; echo $first_line ; process_rest }`  
  echo 2>&1 $first_line
}

get_data | do_something_else # derp

    One possible solution to this might be to use a fifo:

    get_data() {
  FIFO=$(mktemp -u)
  mkfifo $FIFO
  first_line=$(cat | { read first_line ; echo $first_line ; process_rest > $FIFO})
  echo 2>&1 first_line
  cat $FIFO
  rm $FIFO
}
get_data | do_something_else

  2. 0

    You may consider using a temporary file :

    tempfile=$(mktemp)
command | { read -r first_line ; printf "%sn" "$first_line" > "$tempfile"; cat ; } | process_rest
    Login or Signup to reply.
  3. 0

    It’s not the braces. In a POSIX shell, any command sequence involving pipes ( cmd1 | cmd2 ) runs both in subshells.

    To retain the value of first_line you must not run the read in a subshell.

    With bash process substitution you can do:

    while
    IFS= read -r first_line
    consumer
do
    break
done < <( producer )

    or

    { IFS= read -r first_line; consumer; } < <( producer )

    Bash also has a lastpipe option which is available when job control is disabled (typically scripts / non-interactive shells, or when disabled explicitly):

    set +m
shopt -s lastpipe

producer | {
    IFS= read -r first_line
    consumer
}

    BashFAQ 024 has more information and options.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search