Unix & Linux Asked by tiny on December 13, 2020
Is it possible to use bash function inside AWK somehow?
Example file (string, int, int, int)
Mike 247808 247809 247810
Trying to convert values from decimal to hexadecimal.
Function defined either in .bashrc
or in shell script.
$ awk '{print $1 ; d2h($2)}' file
awk: calling undefined function d2h
input record number 1, file file
source line number 1
Just a quick example to demonstrate @HaukeLaging's command|getline
:
Dear friends
my name is `id -nu` and
today is `date "+%Y-%m-%d"`.
where we are following shell syntax, in the input,
`command`
is used to denote inline commands to be replaced by the result of his execution.
#!/usr/bin/gawk -f
BEGIN { FS ="`"; }
NF==3 { $2 | getline $2 }
{ print }
$ expand-inline input
Dear friends
my name is jjoao and
today is 2018-01-15.
Answered by JJoao on December 13, 2020
Converting from decimal to hexadecimal is something that awk
can very well do itself. And you could define an awk
function to do it:
function d2h(d) {
return sprintf("%x", d)
}
Now to answer the question in the general case, for awk
to run bash
functions, you'd need awk
to execute a bash
shell, that bash
to interpret the definition of that function, and call that function, with the value extracted by awk
passed as arguments.
Not trivial.
bash
supports exporting functions via the environment, so it's available in subsequent invocations of bash
, so that's one way to pass the definition of the function to the bash
invoked by awk
:
export -f d2h
The only ways for awk
to execute a command (bash
here) are with its system("cmd")
, or print... | "cmd"
or "cmd" | getline
. In all cases, awk
runs a shell to interpret that cmd
, but it will be sh
, not bash
. So you need to construct a command line for sh
that is a bash
invocation that interprets a bash
command line to invoke the function, so you need to be careful with quoting:
export -f d2h
<file awk -v q="'" '
function shquote(s) {
gsub(q, q "\" q q, s)
return q s q
}
{print $1; system("exec bash -c '''d2h "$1"''' bash " shquote($2))}'
If you want to get the output of the function back into awk
, you'd need to transfer it back via a pipe. For that, you'd use cmd | getline
instead of system(cmd)
(which leaves cmd
's stdout untouched).
cmd | getline line
stores one line (strictly speaking one record, records being lines by default), so to get the whole output in the cases where it's made of several lines, you'd need a loop such as:
awk '...
cmd = "exec bash -c '''d2h "$1"''' bash " shquote($2)
output = ""
while ((cmd | getline line) > 0) {
output = output line RS
}
sub(RS "$", "", output) # remove the last newline
...'
That does mean running one sh
and one bash
for each each invocation of the function, so is going to be quite inefficient. That would end up being even significantly more inefficient than having bash
do the reading and splitting with a while read loop
:
(unset -v IFS; while read -r a b rest; do
printf '%sn' "$a"
d2h "$b"
done < file)
Also note that since shellshock, bash
now export functions in environment variables that are named like BASH_FUNC_d2h%%
. Some sh
implementations including mksh
and newer versions of dash
remove those environment variables from the environment:
$ env 'foo%%=bar' dash -c 'printenv foo%%'
$ env 'foo%%=bar' mksh -c 'printenv foo%%'
$ env 'foo%%=bar' zsh -c 'printenv foo%%'
bar
$ env 'foo%%=bar' bash -c 'printenv foo%%'
bar
So instead of relying on the flimsy function export feature, you could pass the function definition some other way. It could be via an environment variable with a usual name:
BASH_FUNCTIONS=$(typeset -f d2h) awk '
...
cmd = "exec bash -c '''eval "$BASH_FUNCTIONS";"
"d2h "$1"''' bash " shquote($2)
...'
Answered by Stéphane Chazelas on December 13, 2020
This will give you good chances.
cat ../logs/em2.log.1 |grep -i 192.168.21.15 |awk '{system("date"); print $1}'
system function enables you to parse bash command within awk stream.
Answered by Mansur Ali on December 13, 2020
Disclaimer: I realize this is not what the OP is trying to do, but Google will lead others like me to this answer.
You have a bash
script that is organized with functions (because you do not hate yourself or [most] coworkers) and at least 1 of those functions needs to call another from within awk
.
Script
#!/bin/env bash
# The main function - it's a sound pattern even in BASH
main(){
# In the awk command I do some tricky things with single quotes. Count carefully...
# The first $0 is outside the single quotes so it is the name of the current bash script.
# The second $0 is inside the single quotes so it is awk's current line of input.
awk '{printf("%s. ", ++c); system("'$0' --do"); print $0}'<<-PRETEND_THIS_IS_AN_INPUT_STREAM
and
and
well
PRETEND_THIS_IS_AN_INPUT_STREAM
}
# functionized to keep things DRY
doit(){
echo -n "doin' it "
}
# check for a command switch and call different functionality if it is found
if [[ $# -eq 1 && $1 == "--do" ]];
then
doit
else
main
fi
Output
$ ./example.sh
1. doin' it and
2. doin' it and
3. doin' it well
Answered by Bruno Bronosky on December 13, 2020
Try to use system()
function:
awk '{printf("%s ",$1); system("d2h " $2)}' file
In your case system
will call d2h 247808
and then append output of this command to printf
output:
Mike 3C800
EDIT:
As system
uses sh
instead of bash
I can't find a way to access .bashrc
. But you can still use functions from your current bash script:
#!/bin/bash
d2h() {
# do some cool conversion here
echo "$1" # or just output the first parameter
}
export -f d2h
awk '{printf("%s ",$1); system("bash -c '''d2h "$2"'''")}' file
EDIT 2:
I don't know why, but this is not working on my Ubuntu 16.04. This is strange, because it used to work on Ubuntu 14.04.
Answered by akarilimano on December 13, 2020
You can call bash from awk and use its output. That is obviously dangerous from a performance perspective if it happens too often. Quoting the man page:
command | getline [var]
Run command piping the output either into $0 or var,
command would be a bash script that contains the function definition and executes the function.
Answered by Hauke Laging on December 13, 2020
Try doing this :
awk '{print $1 ; printf "%xn", $2}' file
AFAIK, you can't use a bash function in awk but only a script. You can use an awk function if needed.
Answered by Gilles Quenot on December 13, 2020
Get help from others!
Recent Answers
Recent Questions
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP