FS#9396 - [namcap] fails to detect dependencies on uninstalled shared libs

Attached to Project: Arch Linux
Opened by Kevin Monceaux (dokpm0) - Tuesday, 29 January 2008, 19:36 GMT
Last edited by Jelle van der Waa (jelly) - Friday, 11 August 2023, 15:50 GMT
Task Type Bug Report
Category Arch Projects
Status Closed
Assigned To Rémy Oudompheng (remyoudompheng)
Architecture All
Severity Medium
Priority Normal
Reported Version 2007.08-2
Due in Version Undecided
Due Date Undecided
Percent Complete 100%
Votes 2
Private No

Details

Description:

I've created a package(c3270) which should at the very least be dependent on openssl. On my box, readelf -d /usr/bin/c3270 reports:

0x00000001 (NEEDED) Shared library: [libssl.so.0.9.8]
0x00000001 (NEEDED) Shared library: [libcrypto.so.0.9.8]
0x00000001 (NEEDED) Shared library: [libnsl.so.1]
0x00000001 (NEEDED) Shared library: [libreadline.so.5]
0x00000001 (NEEDED) Shared library: [libncursesw.so.5]
0x00000001 (NEEDED) Shared library: [libutil.so.1]
0x00000001 (NEEDED) Shared library: [libc.so.6]

If I list any dependencies at all in the package, namcap reports them as not needed.


Additional info:
* package version(s): 2.1-1
This task depends upon

Closed by  Jelle van der Waa (jelly)
Friday, 11 August 2023, 15:50 GMT
Reason for closing:  Moved
Additional comments about closing:  https://gitlab.archlinux.org/pacman/namc ap/issues/37
Comment by Roman Kyrylych (Romashka) - Tuesday, 29 January 2008, 22:07 GMT
@Jason: I (and I know other users are too) was observing that namcap (both 1.x and 2.x) doesn't detect dependencies correctly in some cases.
You've said it doesn't use ldd. What does it use then? Is there a way to improve its correctness?
Comment by Jason Chu (jason) - Tuesday, 29 January 2008, 22:21 GMT
Can I also get output from namcap -i <package>? That will tell me more about what namcap thinks internally.

@Roman: It uses readelf -d. Ldd and readelf both look at things the same way, it's just that ldd reads dependencies farther down and readelf just reads the dependencies for that single binary. We can improve each situation as they come up. This particular situation seems more like a bug than a deficiency of namcap. I get the proper results when I run it on my system.
Comment by Kevin Monceaux (dokpm0) - Tuesday, 29 January 2008, 22:41 GMT
namcap -i c3270-3.3.7p1-1-i686.pkg.tar.gz

c3270 W: Dependency included and not needed (ncurses)
c3270 W: Dependency included and not needed (openssl)
c3270 W: Dependency included and not needed (readline)
c3270 I: Depends as namcap sees them: depends=()
Comment by Jason Chu (jason) - Tuesday, 29 January 2008, 23:14 GMT
Wow... it doesn't even find the binaries...

Can you just attach the package file that you get?
Comment by Kevin Monceaux (dokpm0) - Tuesday, 29 January 2008, 23:24 GMT
Attached is the package file.
Comment by Jason Chu (jason) - Tuesday, 29 January 2008, 23:46 GMT
Ok, my namcap detects that openssl is a dependency.

How's your python? ;)

The file that we need to inspect is /usr/lib/python2.5/site-packages/Namcap/depends.py. The code I need to figure out is in the scanlibs function. It'd be great to see what files come up in the names list, which names are considered files (isfile), and the result of the readelf -d call on each of those files. If you need me to give you a modified depends.py that does all that, just ask.
Comment by Roman Kyrylych (Romashka) - Wednesday, 30 January 2008, 13:51 GMT
Here's a fresh example that namcap fails to detect dependencies.
With the attached PKGBUILD I get this:
$ namcap ~/packages/gnotime-2.2.3-1-i686.pkg.tar.gz
gnotime W: Dependency included and not needed (qof)
gnotime W: Dependency included but already satisfied (gconf)
gnotime W: Dependency included and not needed (desktop-file-utils)

$ readelf -d pkg/usr/bin/gnotime | grep qof
0x00000001 (NEEDED) Shared library: [libqof.so.1]

Without 'qof' in depends array I get:
$ gnotime
gnotime: error while loading shared libraries: libqof.so.1: cannot open shared object file: No such file or directory
Comment by Pierre Schmitz (Pierre) - Wednesday, 30 January 2008, 17:25 GMT
I had similar problems with php which needs libxml2, ncurses and readline. Those are reported by ldd but not namcap.
Comment by Jason Chu (jason) - Friday, 01 February 2008, 03:13 GMT
@Pierre: Php is a little bit weird, because all the plugins have their own requirements. The problem is that namcap finds those .so files and assumes they're needed for running the package. In finding the deps for those, libxml2, ncurses, and readline are all already covered.

@Roman: I just built a gnotime package and it does depend on qof on my system. Can you send me your package?

I'm beginning to think this might be something else...

Can you send me the output of namcap -i?

Also, Kevin, when you sent the namcap output, was that all of it? Or just the lines you thought relevant?
Comment by Jason Chu (jason) - Friday, 01 February 2008, 03:17 GMT
For those of you waiting for a depends.py, try this one and upload the output of namcap -i.
Comment by Roman Kyrylych (Romashka) - Friday, 01 February 2008, 10:24 GMT
@Jason: the package is in Extra (i686 version). Will send -i output later today.
Comment by Roman Kyrylych (Romashka) - Friday, 01 February 2008, 13:17 GMT
Here's output with old and new depends.py (i've renamed depends.pyc to make sure python won't use it instead of new depends.py)
Comment by Kevin Monceaux (dokpm0) - Friday, 01 February 2008, 13:35 GMT
Attached is the output from the modified depends.py on my c3270 package file.
Comment by Jason Chu (jason) - Friday, 01 February 2008, 15:27 GMT
Next can I get the output of ldconfig -p?
Comment by Roman Kyrylych (Romashka) - Friday, 01 February 2008, 15:33 GMT
note that I don't have gnotime & qof installed at the moment. Do you want ldconfig -p with these packages installed?
Comment by Roman Kyrylych (Romashka) - Friday, 01 February 2008, 15:46 GMT
ok, here it is, with gnotime and qof installed
Comment by Roman Kyrylych (Romashka) - Saturday, 02 February 2008, 20:43 GMT
Another one:
$ namcap ~/packages/lightscribe-1.12.29.2-1-i686.pkg.tar.gz
lightscribe W: Dependency included and not needed (libstdc++5)
$ readelf pkg/usr/lib/liblightscribe.so.1 -d | grep NEEDED
0x00000001 (NEEDED) Shared library: [libstdc++.so.5]
0x00000001 (NEEDED) Shared library: [libm.so.6]
0x00000001 (NEEDED) Shared library: [libc.so.6]

The PKGBUILD is available in AUR.
Comment by Jason Chu (jason) - Sunday, 03 February 2008, 05:00 GMT
Roman and Kevin, I would like to ssh into your machines and inspect things. I'm running out of ideas.
Comment by Kevin Monceaux (dokpm0) - Sunday, 03 February 2008, 16:00 GMT
Jason, as soon as I finish entering this comment I'll send you a private e-mail with login details.
Comment by Roman Kyrylych (Romashka) - Monday, 04 February 2008, 12:04 GMT
Jason, unfortunately this is not possible due to my silly provider. :-(
Comment by Roman Kyrylych (Romashka) - Wednesday, 27 February 2008, 21:05 GMT
fresh one:
$ namcap ~/packages/libbtctl-0.10.0-1-i686.pkg.tar.gz
libbtctl W: Dependency included and not needed (openobex)
$ readelf -d pkg/usr/lib/libbtctl.so.4.1.0 | grep NEEDED
0x00000001 (NEEDED) Shared library: [libgobject-2.0.so.0]
0x00000001 (NEEDED) Shared library: [libglib-2.0.so.0]
0x00000001 (NEEDED) Shared library: [libbluetooth.so.2]
0x00000001 (NEEDED) Shared library: [libopenobex.so.1]
0x00000001 (NEEDED) Shared library: [libc.so.6]

Jason, any ideas how to fix this bug?
Comment by Jason Chu (jason) - Wednesday, 27 February 2008, 21:24 GMT
Nothing else is coming to mind. I don't know what else I would look for and I can't get access to your machine.

What does the namcap -i output look like?
Comment by Roman Kyrylych (Romashka) - Wednesday, 27 February 2008, 21:38 GMT
$ namcap -i ~/packages/libbtctl-0.10.0-1-i686.pkg.tar.gz
libbtctl W: Dependency included and not needed (openobex)
libbtctl I: File ['usr/lib/libbtctl.so', 'usr/lib/libbtctl.so.4.1.0', 'usr/lib/python2.5/site-packages/btctl.so', 'usr/lib/libbtctl.so.4'] link-level dependence on glibc
libbtctl I: File ['usr/lib/libbtctl.so', 'usr/lib/libbtctl.so.4.1.0', 'usr/lib/python2.5/site-packages/btctl.so', 'usr/lib/libbtctl.so.4'] link-level dependence on glib2
libbtctl I: Dependency covered by dependences from link dependence (kernel-headers)
libbtctl I: Dependency covered by dependences from link dependence (pcre)
libbtctl I: Dependency covered by dependences from link dependence (gcc-libs)
libbtctl I: Dependency covered by dependences from link dependence (ncurses)
libbtctl I: Dependency covered by dependences from link dependence (glibc)
libbtctl I: Dependency covered by dependences from link dependence (tzdata)
libbtctl I: Dependency covered by dependences from link dependence (readline)
libbtctl I: Dependency covered by dependences from link dependence (bash)
libbtctl I: Depends as namcap sees them: depends=(glib2)
libbtctl I: Symlink (usr/lib/libbtctl.so) found that points to libbtctl.so.4.1.0
libbtctl I: Symlink (usr/lib/libbtctl.so.4) found that points to libbtctl.so.4.1.0

I wonder why namcap cannot parse "readelf -d" output correctly :-/
Comment by Glenn Matthys (RedShift) - Thursday, 11 December 2008, 07:25 GMT
What's the status of this issue?
Comment by Roman Kyrylych (Romashka) - Monday, 26 January 2009, 12:21 GMT
assigned to Hugo who is the new maintainer
Comment by Allan McRae (Allan) - Monday, 13 April 2009, 07:43 GMT
This appears to be due to namcap not knowing what to do with libraries not found on the system. e.g.

From libbtctl-0.10.0-2-i686.pkg.tar.gz:
> readelf -d usr/lib/libbtctl.so.4.1.0 | grep NEEDED
0x00000001 (NEEDED) Shared library: [libopenobex.so.1]

> namcap libbtctl-0.10.0-2-i686.pkg.tar.gz
libbtctl W: Dependency included and not needed ('openobex')
libbtctl W: Dependency included and not needed ('bluez')

Note openobex provides libopenobex.so.1 so openobex is needed.

> pacman -S openobex
> namcap /home/pkgcache/pkg/libbtctl-0.10.0-2-i686.pkg.tar.gz
libbtctl W: Dependency included but already satisfied ('glib2')
libbtctl W: Dependency included but already satisfied ('bluez')

Now namcap knows about openobex...

Namcap should output that the package needs a library provided by an unknown package and warnings that it does not know what files are contained in uninstalled dependencies.
Comment by Dan McGee (toofishes) - Wednesday, 24 February 2010, 04:37 GMT
I made a partial fix to this in git- I'm not completely happy but it should be a hell of a lot less misleading then it currently is.

$ ./namcap.py --tags ./namcap-tags /var/cache/pacman/pkg/libbtctl-0.11.1-1-x86_64.pkg.tar.gz
libbtctl W: Referenced library 'libopenobex.so.1' is an uninstalled dependency
libbtctl W: Dependency included and not needed ('openobex')
Comment by Eric Belanger (Snowman) - Sunday, 13 February 2011, 17:13 GMT
An improvement to this fix would be to check if the library is in the package itself. For example:

$ namcap /var/cache/pacman/pkg/libhangul-0.0.12-1-i686.pkg.tar.xz
libhangul W: Referenced library 'libhangul.so.0' is an uninstalled dependency
libhangul W: Dependency 'glibc' on your system is a testing release

but libhangul.so.0 is in the package itself:

$ tar -tJvf /var/cache/pacman/pkg/libhangul-0.0.12-1-i686.pkg.tar.xz |grep libhangul.so.0
-rwxr-xr-x root/root 69608 2011-02-12 19:31 usr/lib/libhangul.so.0.1.4
lrwxrwxrwx root/root 0 2011-02-12 19:31 usr/lib/libhangul.so -> libhangul.so.0.1.4
lrwxrwxrwx root/root 0 2011-02-12 19:31 usr/lib/libhangul.so.0 -> libhangul.so.0.1.4

In this case, the warning message is misleading therefore shouldn't be displayed.
Comment by Eric Belanger (Snowman) - Sunday, 13 February 2011, 17:16 GMT
Added Rémy to notifications as he develops namcap now.
Comment by Rémy Oudompheng (remyoudompheng) - Sunday, 27 February 2011, 15:34 GMT
Having a common python module with the pkgfile utility might also help to fix this problem.
Comment by Dan McGee (toofishes) - Wednesday, 25 May 2011, 22:29 GMT
pkgfile is perhaps part of the answer, but just checking some base cases would help. At least today I saw the false errors in a very easy way, and not unheard of- checking an i686 package on an x86_64 machine.

There are two main cases here that are easy to eliminate:
1. deps on libraries included in the package- should be easy to fix.
2. deps on whatever we deem as "base". libc, libm, libpthread fall into this category for sure, while you could also stretch it to libssl, libcrypto, etc. This would take care of every single false error below.

dan@alderaan ~/svn-packages/postgresql/trunk
$ namcap postgresql-libs-9.0.4-4-i686.pkg.tar.xz
postgresql-libs W: Referenced library 'libecpg.so.6' is an uninstalled dependency
postgresql-libs W: Referenced library 'libcrypto.so.1.0.0' is an uninstalled dependency
postgresql-libs W: Referenced library 'libpgtypes.so.3' is an uninstalled dependency
postgresql-libs W: Referenced library 'libpq.so.5' is an uninstalled dependency
postgresql-libs W: Referenced library 'libssl.so.1.0.0' is an uninstalled dependency
postgresql-libs W: Referenced library 'libpthread.so.0' is an uninstalled dependency
postgresql-libs W: Referenced library 'libm.so.6' is an uninstalled dependency
postgresql-libs W: Referenced library 'libc.so.6' is an uninstalled dependency
postgresql-libs W: Dependency included and not needed ('openssl')

dan@alderaan ~/svn-packages/postgresql/trunk
$ uname -a
Linux alderaan 2.6.38-ARCH #1 SMP PREEMPT Mon May 23 22:02:08 CEST 2011 x86_64 Intel(R) Core(TM) i7 CPU 920 @ 2.67GHz GenuineIntel GNU/Linux

Comment by Antony Lee (anntzer) - Wednesday, 15 June 2016, 04:28 GMT
Missing depends against not installed libraries are now marked, but it would be nice if namcap could automatically run pkgfile on those to report the missing package name.
Comment by Buggy McBugFace (bugbot) - Tuesday, 08 August 2023, 19:11 GMT
This is an automated comment as this bug is open for more then 2 years. Please reply if you still experience this bug otherwise this issue will be closed after 1 month.

Loading...