[misc] Statically linked raylib exposes functions from src/external/*

[ListeriaM]

Environment

• GNU/Linux
• raylib built using make

Issue description

Some of the single header libraries used by raylib expose a macro that affects the linkage of functions defined in them (i.e: MA_API for miniaudio.h), but these are not used by raylib. The result is that all of their functions are included in the final library (static or shared), this is evidenced by the output from objdump below. In the case of the shared library these are not accessible due of the use of -fvisibility=hidden in the Makefile (Linux, BSD & OS X), but they are still included in it, even if unused (and inaccessible in the case of the shared library). All of these functions bloat raylib considerably.

Code Example

Take ma_version() from miniaudio.h for example, it is not used by raylib, yet objdump -j.text --disassemble=ma_version libraylib.so shows its definition (we would see similar output from libraylib.a):

libraylib.so:     file format elf64-x86-64


Disassembly of section .text:

00000000000fcb7f <ma_version>:
   fcb7f:       48 85 ff                test   %rdi,%rdi
   fcb82:       74 06                   je     fcb8a <ma_version+0xb>
   fcb84:       c7 07 00 00 00 00       movl   $0x0,(%rdi)
   fcb8a:       48 85 f6                test   %rsi,%rsi
   fcb8d:       74 06                   je     fcb95 <ma_version+0x16>
   fcb8f:       c7 06 0b 00 00 00       movl   $0xb,(%rsi)
   fcb95:       48 85 d2                test   %rdx,%rdx
   fcb98:       74 06                   je     fcba0 <ma_version+0x21>
   fcb9a:       c7 02 15 00 00 00       movl   $0x15,(%rdx)
   fcba0:       c3                      ret

The fact that they are visible in the statically linked version means that we can do something like this:

// ma_version.c
#include <stdio.h>

int main(void) {
	// raylib does not provide a prototype
	typedef unsigned ma_uint32;
	extern void ma_version(ma_uint32 *, ma_uint32 *, ma_uint32 *);

	ma_uint32 major, minor, revision;

	ma_version(&major, &minor, &revision);
	printf("%u.%u.%u\n", major, minor, revision);

	return 0;
}

and get different output using the static or the shared library:

$ cc ma_version.c -l:libraylib.a -lm -o ma_version && ./ma_version # static
0.11.21
$ cc ma_version.c -lraylib -lm -o ma_version && ./ma_version # shared
/usr/bin/ld: /tmp/ccBPPu2C.o: in function `main':
ma_version.c:(.text+0x2a): undefined reference to `ma_version'
collect2: error: ld returned 1 exit status

[Peter0x44]

AFAIK, -fvisibility=hidden does not function for static libraries.
How do you propose raylib should hide them?
For the smallest possible executables, you should use LTO, which can remove the "bloat". You could also use -ffunction-sections and -Wl,--gc-sections.

[ListeriaM]

Many of the functions from external libraries can be defined to have internal linkage using macros like MA_API for miniaudio.h (although some libraries don't expose such macros), i.e:

#define MA_API static

this would let the compiler discard the unused ones at compile time and produce better code for those which are used. Since this would produce a lot of noise from all the unused function warnings, depending on the compiler, something like __attribute__((unused)) can be used (function attributes are a GNU C feature, I don't know if MSVC supports something similar), i.e:

#ifdef __GNUC__
# define MA_API __attribute__((unused)) static
#endif

This can be generalized in a similar way as the RLAPI macro, and have a RLAPI_PRIVATE macro (or whatever you may call it) which would be used in the external libraries that allow it.

For external libraries that don't allow it, I haven't tried yet but I have some ideas involving the use of the retain or section function attributes for raylib functions and partial linking (something like ld -r --gc-sections -u<some function> ...) which should keep all raylib functions and their dependencies.

[Peter0x44]

That proposal makes sense, thanks for the details. Most of this can be avoided with LTO for static libraries but it would help in some cases for the shared lib too.

[raysan5]

@ListeriaM What is exactly the bloat added from those functions? Personally I like them exposed because I use them sometimes on my user code just including the required .h and knowing the implementation is available in static raylib.

What I find more worrying is the shared library not exposing them.

What was the proposed changed/solution?

[ListeriaM]

What I was refering to as bloat is precisely the hidden functions in the shared library, they are defined but they cannot be used. If they were removed (or exposed), then static and shared raylib would be interchangeable.

So the problem is not necessarily that they're accessible in static raylib, but rather the difference in the APIs from static and shared.

The solution depends on the decision of removing or exposing them. I think removing them is the better option, since I don't consider them part of the raylib API, and I was surprised when I tried to use one of these headers in my code and got a linker error. If they were not exposed using them in your code would be as simple as defining *_IMPLEMENTATION before including the corresponding header, which is expected of stb-style headers.

In any case the first step would be to use the *API macros together with RLAPI or the proposed RLAPI_PRIVATE to make sure these functions have external or internal linkage regardless of the use of the -fvisibility flag.

[Peter0x44]

Just to note, I made the change to use -fvisibility=hidden but there would still be a problem with windows dlls, which effectively act that way by default. I did it to hide glfw.

So there would still be inconsistency between the so and a dll, even if using -fvisibility=hidden was reverted.

[Peter0x44]

The solution depends on the decision of removing or exposing them.
I think removing them is the better option, since I don't consider them part of the raylib API

I agree with this reasoning. Especially for header-only libraries, it is okay to have multiple copies, I think whatever solution is proposed for static linking should make it okay to just define the implementation in your own code, regardless of what raylib has.

[raysan5]

I think removing them is the better option, since I don't consider them part of the raylib API

Unfortunately, afaik, not all the internally used libraries expose some way to remove the symbols and that would imply some symbols removed and some exposed. Would it be possible to expose all the symbols in both static and shared libraries?

If not possible to find a really consistent solution, then I'd just left as inconsistent as it is now, I think most user will never case about it.

[orcmid]

Is there possibly a confusion here between debug and release builds?

For static linking, unreachable functions can be eliminated from the executable and that should relieve any concern about bloat. There's no way to know for a shared library, but I would think that externally-unreachable (i.e., static) functions would not have symbols in the shared library, although I can understand why they might have to.

[Peter0x44]

Linkers cannot remove symbols, even when static linking, most of the time, certainly not without extra flags.
For gcc you can use -ffunction-sections and -Wl--gc-sections to kind of get this effect, but the better way is using LTO, which is eliminating the unused functions from the compiler side more than the linker side.

It is something you need to request and arrange to happen - not default in most circumstances.

[orcmid]

@Peter0x44

It is something you need to request and arrange to happen - not default in most circumstances.

Got it. With the Windows Visual Studio Build Tools you do need to specify an option to the compiler and another in the static linking. I do that habitually with raylib apps, and the raylib *.c files. I assumed something similar for gcc.