gawk "readdir" dynamic lib errors

[jeorge]

Hello,
I'm wanting to take advantage of the readdir.so dynamic library that
ships with gawk but I'm having an issue on the target system which the
following test code reveals:

#-- test code --
@load "readdir"

BEGIN{
d="/proc/sys"
f=d"/kernel/version"
FS="/"
while(getline<d)
if($3=="d")
print $2
close(d)

FS=" "
while(getline<f)
print $0
close(f)

print "\nGNU Awk",PROCINFO["version"]
}
#-- ---- ---- --

Output on my home system look fine but on the target system I get the
following errors/warnings right after the /proc/sys directory listing
which I don't understand:

#-- error msg --
awk: cmd. line:1: warning: dir_take_control_of: opendir/fdopendir
failed: Not a directory
awk: cmd. line:1: warning: input parser `readdir' failed to open
`/proc/sys/kernel/version'
#-- ----- --- --

Versioning (last line in code):
- home system: GNU Awk ver: 5.1.0
- target system: GNU Awk ver: 5.2.2

Other info for target machine:
- OS = Arch Linux (64bit)
- AWKLIBPATH = /usr/lib/gawk
- $ gawk --version |head -n1
=> GNU Awk 5.2.2, API 3.2, PMA Avon 8-g1, (GNU MPFR 4.2.0-p9, GNU MP
6.2.1)

Does this look like a fixable configuration quirk not requiring
recompiling or something else? I'm not admin on the target system but
benign configuration tweaks would likely be an option.

-J

[Kenny McCormack]

Post by jeorge
Hello,
I'm wanting to take advantage of the readdir.so dynamic library that
ships with gawk but I'm having an issue on the target system which the

Works fine here, but then again, that helps you not at all.

I doubt there's anything wrong with the readdir library (although it is
certainly possible that there is). More likely, something strange about
the contents of the /proc/sys directory on the "target" system.

Probably just the usual debugging techniques need to be applied. Keep
trying things until you figure out what step is not doing what you expect.

[Manuel Collado]

Post by jeorge
Hello,
I'm wanting to take advantage of the readdir.so dynamic library that
ships with gawk but I'm having an issue on the target system which the
[snip]
Output on my home system look fine but on the target system I get the
following errors/warnings right after the /proc/sys directory listing
#-- error msg --
awk: cmd. line:1: warning: dir_take_control_of: opendir/fdopendir
failed: Not a directory
awk: cmd. line:1: warning: input parser `readdir' failed to open
`/proc/sys/kernel/version'
#-- ----- --- --

Some weird with /proc/sys/kernel/version in the target machine. Please
post the relevant part of the output of

ls -l /proc/sys/kernel/

Maybe .../version is a link instead of a regular directory.

HTH.

[jeorge]

Actually the warning occurs when gawk hits ## while(getline<f) ## as
seen below. I changed my test dir and file to "/etc" and "/etc/hosts"
to eliminate potential permissions issues in /proc/sys/:

$ cat readdir_test2.awk
#-- test code --
@load "readdir"
BEGIN{
d="/etc"
f=d"/hosts"
while(getline<f)
print $0
close(f)
}
#-- ---- ---- --

Run & output:

$ gawk -f readdir_test2.awk
gawk: readdir_test.awk:6: warning:\
dir_take_control_of: opendir/fdopendir failed: Not a directory
gawk: readdir_test.awk:6: warning:\
input parser `readdir' failed to open `/etc/hosts'
=># Static table lookup for hostnames.
=># See hosts(5) for details.
=>
=>

It does output the contents of the hosts file as (=>) indicated.

Basically my goal is to use the "readdir" functionality to obtain the
paths to particular files which are then read in and processed. I found
additionally loading "readfile" provides a warning-free work-around:

$ cat readdir_test3.awk
#-- test code --
@load "readfile"
@load "readdir"
BEGIN{
d="/etc/"
f=d"hosts"
FS="/"
while(getline<d)
if($2~/[A-Z]/ && $3=="d")
print d$2
close(d)
str=readfile(f)
split(str,arr,"\n")
for(i in arr)
print arr[i]
}
#-- ---- ---- --

Run & output:

$ gawk -f readdir_test3.awk
/etc/ODBCDataSources
/etc/R
/etc/X11
# Static table lookup for hostnames.
# See hosts(5) for details.


This seems an okay solution if the files to be processes aren't huge;
it would be better to only read as much as needed from the files.

I'm wondering it there is a way to UNload a dynamic library once one is
done with it, something like ## @unload "readdir" ## ? Might be useful.

-J

[Aharon Robbins]

Hello.

Mack The Knife forwarded your report to me. This kind of
question would have been better sent to the bug-gawk list,
as you've found an actual bug. (The first real bug for 5.2.2,
congratulations! :-)

The bug is due to a change in the readdir extension that
occurred with the 5.2.2 release. Here is a patch.

Thanks,

Arnold
-----------------------------------------------
diff --git a/extension/readdir.c b/extension/readdir.c
index e367ff12..4ae1626f 100644
--- a/extension/readdir.c
+++ b/extension/readdir.c
@@ -249,7 +249,7 @@ dir_can_take_file(const awk_input_buf_t *iobuf)
if (iobuf == NULL)
return awk_false;

- return (iobuf->fd != INVALID_HANDLE || S_ISDIR(iobuf->sbuf.st_mode));
+ return (S_ISDIR(iobuf->sbuf.st_mode));
}

/*
-----------------------------------------------

Post by jeorge
Hello,
I'm wanting to take advantage of the readdir.so dynamic library that
ships with gawk but I'm having an issue on the target system which the
#-- test code --
@load "readdir"
BEGIN{
d="/proc/sys"
f=d"/kernel/version"
FS="/"
while(getline<d)
if($3=="d")
print $2
close(d)
FS=" "
while(getline<f)
print $0
close(f)
print "\nGNU Awk",PROCINFO["version"]
}
#-- ---- ---- --
Output on my home system look fine but on the target system I get the
following errors/warnings right after the /proc/sys directory listing
#-- error msg --
awk: cmd. line:1: warning: dir_take_control_of: opendir/fdopendir
failed: Not a directory
awk: cmd. line:1: warning: input parser `readdir' failed to open
`/proc/sys/kernel/version'
#-- ----- --- --
- home system: GNU Awk ver: 5.1.0
- target system: GNU Awk ver: 5.2.2
- OS = Arch Linux (64bit)
- AWKLIBPATH = /usr/lib/gawk
- $ gawk --version |head -n1
=> GNU Awk 5.2.2, API 3.2, PMA Avon 8-g1, (GNU MPFR 4.2.0-p9, GNU MP
6.2.1)
Does this look like a fixable configuration quirk not requiring
recompiling or something else? I'm not admin on the target system but
benign configuration tweaks would likely be an option.
-J

[jeorge]

Post by Aharon Robbins
Mack The Knife forwarded your report to me. This kind of
question would have been better sent to the bug-gawk list,
as you've found an actual bug. (The first real bug for 5.2.2,
congratulations! 😄
The bug is due to a change in the readdir extension that
occurred with the 5.2.2 release. Here is a patch.
Thanks,
Arnold

LOL, thanks -- glad I could be of service! And thanks for your years of
gawk development; inspiring.

Cheers,
-J