Re: libpcap detection and linking in tcpdump

[Guy Harris via tcpdump-workers <tcpdump-workers () lists tcpdump org>]

> --- Begin Message ---
>
>
> From: Guy Harris <gharris () sonic net>
>
> Date: Thu, 7 Jan 2021 15:21:46 -0800
>
> On Sep 9, 2020, at 9:07 AM, Denis Ovsienko via tcpdump-workers <tcpdump-workers () lists tcpdump org> wrote:
>
> > The "Found pcap-config" message means that FindPCAP.cmake has not found
> > libpcap by means of pkg-config, and the lack of the message means the
> > pkg-config method worked. A few weeks ago Ubuntu 18.04 systems started
> > to have the libpcap.pc file in the libpcap0.8-dev package, so on such a
> > system "pkg-config --libs libpcap" now prints "-lpcap" and "pkg-config
> > --cflags libpcap" prints an empty string, which makes sense.
>
> It makes sense if you want to build with the *system* libpcap.
>
> It does *not* make sense if you want to build with the libpcap that was built, and installed under /tmp, in the Travis 
> build.  For *that*, you'd want "pkg-config --libs libpcap" to print "-L/tmp/lib -lpcap" and you'd want pkg-config 
> --cflags libpcap" to print "-I /tmp/include".
>
> That's because it's finding the libpcap.pc file in the libpcap0.8-dev package, not the one in /tmp/lib/pkgconfig; the 
> latter one should do what we want.
>
> I changed .travis.yml to run CMake with PKG_CONFIG_PATH=/tmp/lib/pkgconfig, which appears to make it find the right .pc 
> file and thus to find the right libpcap.
>
> > What does not make sense is that CMake seems to use non-pkg-config
> > flags to tell if a specific feature is available,
>
> It *should* be using the pkg-config flags - the code that tests for features just does
>
>         #
>         # libpcap/WinPcap/Npcap.
>         # First, find it.
>         #
>         find_package(PCAP REQUIRED)
>         include_directories(${PCAP_INCLUDE_DIRS})
>
>         cmake_push_check_state()
>
>         #
>         # Now check headers.
>         #
>         set(CMAKE_REQUIRED_INCLUDES ${PCAP_INCLUDE_DIRS})
>
>         #
>         # Check whether we have pcap/pcap-inttypes.h.
>         # If we do, we use that to get the C99 types defined.
>         #
>         check_include_file(pcap/pcap-inttypes.h HAVE_PCAP_PCAP_INTTYPES_H)
>
>         #
>         # Check for various functions in libpcap/WinPcap/Npcap.
>         #
>         cmake_push_check_state()
>         set(CMAKE_REQUIRED_LIBRARIES ${PCAP_LIBRARIES})
>
>         #
>         # Check for "pcap_list_datalinks()" and use a substitute version if
>         # it's not present.  If it is present, check for "pcap_free_datalinks()";
>         # if it's not present, we don't replace it for now.  (We could do so
>         # on UN*X, but not on Windows, where hilarity ensues if a program
>         # built with one version of the MSVC support library tries to free
>         # something allocated by a library built with another version of
>         # the MSVC support library.)
>         #
>         check_function_exists(pcap_list_datalinks HAVE_PCAP_LIST_DATALINKS)
>
>                 ...
>
>         cmake_pop_check_state()
>
> which doesn't care whether PCAP_INCLUDE_DIRS and PCAP_LIBRARIES were set from pkg-config or pcap-config or manually 
> poking the system.
>
> > but uses pkg-config
> > flags to compile and link the source,
>
> *However*, the CMake documentation says about CMAKE_PREFIX_PATH:
>
>         Semicolon-separated list of directories specifying installation prefixes to be searched by the find_package(), 
> find_program(), find_library(), find_file(), and find_path() commands.
>
> and we're setting CMAKE_PREFIX_PATH, so if any of the include or library checks use CMAKE_PREFIX_PATH, then they might 
> find headers for the libpcap installed in /tmp/libpcap, even though the build itself will use flags from the system 
> libpcap.pc.
>
> The CMake 3.19 documentation for FindPkgConfig, which is the module for using pkg-config:
>
>         https://cmake.org/cmake/help/v3.19/module/FindPkgConfig.html
>
> says:
>
>         PKG_CONFIG_USE_CMAKE_PREFIX_PATH
>                 Specifies whether pkg_check_modules() and pkg_search_module() should add the paths in the 
> CMAKE_PREFIX_PATH, CMAKE_FRAMEWORK_PATH and CMAKE_APPBUNDLE_PATH cache and environment variables to thepkg-config 
> search path.
>
>                 If this variable is not set, this behavior is enabled by default if CMAKE_MINIMUM_REQUIRED_VERSION is 
> 3.1 or later, disabled otherwise.
>
> So we should either 1) require CMake 3.1 or later or 2) forcibly set PKG_CONFIG_USE_CMAKE_PREFIX_PATH to YES.  For now, 
> my inclination is to do the latter.
>
> Once all the other stuff I've checked in passes Travis, I'll try that instead of explicitly setting PKG_CONFIG_PATH. 
> and see if that works.
>
> > and when the system has one
> > libpcap version installed as a package and another version that the
> > user wants to build with, that very easily breaks (and even if it does
> > not, the end result is not what the user was expecting).
> >
> > Here are my steps to reproduce:
> >
> > libpcap$ ./configure --enable-remote --prefix=/tmp/libpcap
> > libpcap$ make
> > libpcap$ make install
> > tcpdumpbuild$ cmake -DWITH_CRYPTO="yes"
> > -DCMAKE_PREFIX_PATH=/tmp/libpcap -DCMAKE_INSTALL_PREFIX=/tmp/libpcap
> > /path/to/tcpdump_git_clone
>
> Try that with
>
>         PKG_CONFIG_PATH=/tmp/libpcap/lib/pkgconfig cmake -DWITH_CRYPTO="yes" -DCMAKE_PREFIX_PATH=/tmp/libpcap 
> -DCMAKE_INSTALL_PREFIX=/tmp/libpcap
>
> and see if that works.
>
> > The above fails if the package libpcap0.8-dev is installed. After
> > removing the package the above works and tcpdump links
> > with /tmp/libpcap/lib/libpcap.so.1 as expected.
> >
> > Clearly, there are cases where using pkg-config would be the right
> > thing to do. But is seems wrong to try it always as the first choice.
> > Perhaps it would be better to try pkg-config after the user-specified
> > prefix did not result in a usable libpcap.
> >
> > Autoconf is not entirely consistent with CMake, in that it tries
> > pkg-config before libpcap in the specified prefix directory too, but
> > there are differences:
> > * Before these two methods autoconf checks for libpcap in ../libpcap
> >  and links it statically if it is available. CMake does not do that.
>
> That's because, as per a message I sent to François:
>
> "local libpcap" for tcpdump assumes, in effect, that you're doing in-tree libpcap builds, as it assumes that the 
> headers and library are in ../libpcap.
>
> With CMake, either subtree or out-of-tree builds are generally preferred - I do in-tree builds for software using 
> autotools, but I do subtree builds, in a "build" subdirectory, with CMake, as 1) CMake leaves behind in the build 
> directory a lot more stuff than autotools and 2) it doesn't seem to have a Make/Ninja/etc. rule to remove all that 
> stuff (no equivalent of "make distclean"), and doing a subtree build means I can do that with "rm -rf build".
>
> So CMake doesn't do a special hack for local libpcap.
>
> In addition, if you do a subtree or out-of-tree libpcap build, the include files won't be in the same directory as the 
> libraries, so you can't just manually set PCAP_ROOT to the libpcap build directory (it won't find the headers) or to 
> the libpcap source directory (it won't find the library).
>
> About the only ways to make it work are
>
>         1) do an in-tree libpcap build
>
> or
>
>         2) do an install after doing the libpcap build (even if the install is into a directory in /tmp, which I think 
> is what we do on Travis) and point CMake at that directory.
>
> The latter is what we're doing on Travis.
>
> > * If the above was not the case, the pkg-config method works in
> >  autoconf, so tcpdump is linked with the system libpcap. CMake build
> >  breaks in this case.
> > * If the above was not the case and ./configure is given a prefix, it
> >  fails to find libpcap under that prefix. CMake gets that right.
>
> That would probably involve adding PKG_CONFIG_PATH=${prefix}/lib/pkgconfig to the pkg-config command in the configure 
> script, so that pkg-config finds the libpcap.pc file in /tmp/libpcap/pkgconfig, just as with CMake.
>
> *However*, that's not necessary if you did an in-tree libpcap build and you have a libpcap source directory at the same 
> level as the tcpdump source directory, as it'll find that rather than the one installed in /tmp or the system one.
>
> Note also that --prefix= doesn't change the search path for commands or for pkg-config files; --prefix= sets where the 
> package you're *building* installs things, but doesn't affect where it searches for *other* things.  That tends to be 
> done on a per-thing basis with --with-thing={directory}; we could add a --with-pcap= flag to tcpdump for that purpose - 
> obviously, it'd fail if you do --without-pcap....  (At least at one point I think *I* believed it would do that, but 
> no.)
>
> I think --prefix= is the equivalent of CMAKE_INSTALL_PREFIX, not the equivalent of CMAKE_PREFIX_PATH; I'm not sure 
> autoconf *has* an equivalent of CMAKE_PREFIX_PATH - you just have to add the appropriate directories to the appropriate 
> path environment variables (PATH, PKG_CONFIG_PATH, etc.).
>
> --- End Message ---
_______________________________________________
tcpdump-workers mailing list
tcpdump-workers () lists tcpdump org
https://lists.sandelman.ca/mailman/listinfo/tcpdump-workers