[cfe-dev] Clang7 UBSan + -fvisibility-hidden

[Philipp Berger via cfe-dev]

Dear all,

We are currently investigating a possible undefined behaviour in our
program that is flagged by clang7 UBSan in combination with
boost::program_option from boost 1.69.0. We have created the following
working example that can we compiled and run with clang++ -std=c++17
-fsanitize=undefined -fno-omit-frame-pointer -lboost_program_options
debug.cpp  && UBSAN_OPTIONS=print_stacktrace=1 ./a.out

#include <iostream>
#include <boost/program_options.hpp>

namespace po = boost::program_options;

int main() {
    std::string test_string = "";
    po::options_description desc("test");
    desc.add_options()
        ("string", po::value<std::string>(&test_string))
    ;

    constexpr char *argv[] = {"test", "--string", "test"};
    constexpr int argc = sizeof(argv) / sizeof(char*);

    po::variables_map vm;
    po::store(po::parse_command_line(argc, argv, desc), vm);
    std::cerr << "Before notify" << std::endl;
    po::notify(vm);

    std::cout << "string -> " << test_string << std::endl;
}

We get the following output:

/usr/include/boost/any.hpp:249:17: runtime error: downcast of address
0x5638e42892e0 which does not point to an object of type
'any::holder<typename remove_cv<basic_string<char, char_traits<char>,
allocator<char> > >::type>' (aka
'holder<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > >')
0x5638e42892e0: note: object is of type
'boost::any::holder<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > >'
 00 00 00 00  00 55 f9 63 28 7f 00 00  f8 92 28 e4 38 56 00 00  04 00 00
00 00 00 00 00  74 65 73 74
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for
'boost::any::holder<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > >'
    #0 0x5638e1b06d6e in std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >*
boost::any_cast<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > >(boost::any*)
(/home/gereon/carl/src/tests/carl-settings/a.out+0x4cd6e)
    #1 0x5638e1b0672c in std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const*
boost::any_cast<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > >(boost::any const*)
(/home/gereon/carl/src/tests/carl-settings/a.out+0x4c72c)
    #2 0x5638e1b03625 in
boost::program_options::typed_value<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >, char>::notify(boost::any
const&) const (/home/gereon/carl/src/tests/carl-settings/a.out+0x49625)
    #3 0x7f2863f6a71e in boost::program_options::variables_map::notify()
(/usr/lib/libboost_program_options.so.1.69.0+0x5771e)
    #4 0x5638e1afa2ae in main
(/home/gereon/carl/src/tests/carl-settings/a.out+0x402ae)
    #5 0x7f2863a13222 in __libc_start_main (/usr/lib/libc.so.6+0x24222)
    #6 0x5638e1ad33ad in _start
(/home/gereon/carl/src/tests/carl-settings/a.out+0x193ad)

/usr/include/boost/any.hpp:249:114: runtime error: member access within
address 0x5638e42892e0 which does not point to an object of type
'any::holder<typename remove_cv<basic_string<char, char_traits<char>,
allocator<char> > >::type>' (aka
'holder<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > >')
0x5638e42892e0: note: object is of type
'boost::any::holder<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > >'
 00 00 00 00  00 55 f9 63 28 7f 00 00  f8 92 28 e4 38 56 00 00  04 00 00
00 00 00 00 00  74 65 73 74
              ^~~~~~~~~~~~~~~~~~~~~~~
              vptr for
'boost::any::holder<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > >'
    #0 0x5638e1b06e36 in std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >*
boost::any_cast<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > >(boost::any*)
(/home/gereon/carl/src/tests/carl-settings/a.out+0x4ce36)
    #1 0x5638e1b0672c in std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > const*
boost::any_cast<std::__cxx11::basic_string<char, std::char_traits<char>,
std::allocator<char> > >(boost::any const*)
(/home/gereon/carl/src/tests/carl-settings/a.out+0x4c72c)
    #2 0x5638e1b03625 in
boost::program_options::typed_value<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> >, char>::notify(boost::any
const&) const (/home/gereon/carl/src/tests/carl-settings/a.out+0x49625)
    #3 0x7f2863f6a71e in boost::program_options::variables_map::notify()
(/usr/lib/libboost_program_options.so.1.69.0+0x5771e)
    #4 0x5638e1afa2ae in main
(/home/gereon/carl/src/tests/carl-settings/a.out+0x402ae)
    #5 0x7f2863a13222 in __libc_start_main (/usr/lib/libc.so.6+0x24222)
    #6 0x5638e1ad33ad in _start
(/home/gereon/carl/src/tests/carl-settings/a.out+0x193ad)

string -> test

As we can see it runs fine (test_string has the correct value
afterwards). We have investigated into boost::program_options and
boost::any and found the following: In
program_options::typed_value::notify() (#2) we get a boost::any and cast
it to the type it is supposed to be by calling boost::any_cast().

Considering clang's output (downcast of address 0x5638e42892e0 which
does not point to an object of type 'any::holder<typename
remove_cv<basic_string<char, char_traits<char>, allocator<char> >
> ::type>' (aka 'holder<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > >') and object is of type
'boost::any::holder<std::__cxx11::basic_string<char,
std::char_traits<char>, std::allocator<char> > >') we can only conclude
that this boost::any instance in fact holds the correct member... in
which case the downcast should be legal.

UBSan however states that no false positives are to be expected.

Interestingly we are not able to reproduce this error without
program_options in the mix:

#include <boost/any.hpp>
#include <iostream>

void any_cast(const boost::any& a) {
    const std::string* t2 = boost::any_cast<std::string>(&a);
    std::cout << *t2 << std::endl;
}

int main() {
    boost::any a;
    a = std::string("test");
    any_cast(a);
}

Hence we suspect that (the precompiled) program_options may somehow mess
with the std::string type.

By the way, valgrind does not report any errors for both programs.

We have tried this on Arch linux (with boost 1.69.0-1, clang 7.0.1-1
using libstd++) and CentOS 7.6 (clang 7.0.0 using libc++ and boost
1.69.0 build with the same clang).

We have bisected this UBSan behavior to boost commit
16f52cede5bb0665f36c13dfd76c6badd719f21b, which activated
-fvisibility-inlines-hidden and -fvisibility=hidden.

Is there any common knowledge around visibility-hidden +
visibility-inlines-hidden + UBSan?


Kind regards,

Philipp