| Elliott Hughes | ecaa192 | 2024-06-04 18:20:05 +0000 | [diff] [blame] | 1 | # bionic maintainer overview |
| Elliott Hughes | 0bfcbaf | 2017-08-28 09:18:34 -0700 | [diff] [blame] | 2 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 3 | [bionic](https://en.wikipedia.org/wiki/Bionic_(software)) is Android's |
| 4 | C library, math library, and dynamic linker. |
| Elliott Hughes | 0bfcbaf | 2017-08-28 09:18:34 -0700 | [diff] [blame] | 5 | |
| Elliott Hughes | ecaa192 | 2024-06-04 18:20:05 +0000 | [diff] [blame] | 6 | This document is a high-level overview of making changes to bionic itself. |
| 7 | If you're trying to _use_ bionic, or want more in-depth information about |
| 8 | some part of the implementation, see [all the bionic documentation](docs/). |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 9 | |
| 10 | ## What are the big pieces of bionic? |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 11 | |
| Dan Albert | 472cce5 | 2014-10-10 17:14:37 -0700 | [diff] [blame] | 12 | #### libc/ --- libc.so, libc.a |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 13 | |
| Dan Albert | 472cce5 | 2014-10-10 17:14:37 -0700 | [diff] [blame] | 14 | The C library. Stuff like `fopen(3)` and `kill(2)`. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 15 | |
| Dan Albert | 472cce5 | 2014-10-10 17:14:37 -0700 | [diff] [blame] | 16 | #### libm/ --- libm.so, libm.a |
| 17 | |
| 18 | The math library. Traditionally Unix systems kept stuff like `sin(3)` and |
| 19 | `cos(3)` in a separate library to save space in the days before shared |
| 20 | libraries. |
| 21 | |
| 22 | #### libdl/ --- libdl.so |
| 23 | |
| 24 | The dynamic linker interface library. This is actually just a bunch of stubs |
| 25 | that the dynamic linker replaces with pointers to its own implementation at |
| 26 | runtime. This is where stuff like `dlopen(3)` lives. |
| 27 | |
| 28 | #### libstdc++/ --- libstdc++.so |
| 29 | |
| 30 | The C++ ABI support functions. The C++ compiler doesn't know how to implement |
| 31 | thread-safe static initialization and the like, so it just calls functions that |
| 32 | are supplied by the system. Stuff like `__cxa_guard_acquire` and |
| 33 | `__cxa_pure_virtual` live here. |
| 34 | |
| 35 | #### linker/ --- /system/bin/linker and /system/bin/linker64 |
| 36 | |
| 37 | The dynamic linker. When you run a dynamically-linked executable, its ELF file |
| 38 | has a `DT_INTERP` entry that says "use the following program to start me". On |
| 39 | Android, that's either `linker` or `linker64` (depending on whether it's a |
| 40 | 32-bit or 64-bit executable). It's responsible for loading the ELF executable |
| 41 | into memory and resolving references to symbols (so that when your code tries to |
| 42 | jump to `fopen(3)`, say, it lands in the right place). |
| 43 | |
| 44 | #### tests/ --- unit tests |
| 45 | |
| 46 | The `tests/` directory contains unit tests. Roughly arranged as one file per |
| Elliott Hughes | 32c9572 | 2023-05-08 19:14:28 +0000 | [diff] [blame] | 47 | publicly-exported header file. `tests/headers/` contains compile-only tests |
| 48 | that just check that things are _in_ the headers, whereas the "real" tests |
| 49 | check actual _behavior_. |
| Dan Albert | 472cce5 | 2014-10-10 17:14:37 -0700 | [diff] [blame] | 50 | |
| 51 | #### benchmarks/ --- benchmarks |
| 52 | |
| Elliott Hughes | 0bfcbaf | 2017-08-28 09:18:34 -0700 | [diff] [blame] | 53 | The `benchmarks/` directory contains benchmarks, with its own [documentation](benchmarks/README.md). |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 54 | |
| 55 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 56 | ## What's in libc/? |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 57 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 58 | ``` |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 59 | libc/ |
| 60 | arch-arm/ |
| 61 | arch-arm64/ |
| 62 | arch-common/ |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 63 | arch-x86/ |
| 64 | arch-x86_64/ |
| 65 | # Each architecture has its own subdirectory for stuff that isn't shared |
| 66 | # because it's architecture-specific. There will be a .mk file in here that |
| 67 | # drags in all the architecture-specific files. |
| 68 | bionic/ |
| 69 | # Every architecture needs a handful of machine-specific assembler files. |
| 70 | # They live here. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 71 | string/ |
| 72 | # Most architectures have a handful of optional assembler files |
| 73 | # implementing optimized versions of various routines. The <string.h> |
| 74 | # functions are particular favorites. |
| 75 | syscalls/ |
| 76 | # The syscalls directories contain script-generated assembler files. |
| 77 | # See 'Adding system calls' later. |
| 78 | |
| 79 | include/ |
| 80 | # The public header files on everyone's include path. These are a mixture of |
| 81 | # files written by us and files taken from BSD. |
| 82 | |
| 83 | kernel/ |
| Elliott Hughes | cfc30908 | 2024-01-11 00:09:25 +0000 | [diff] [blame] | 84 | # The kernel uapi header files. The "libc" headers that developers actually |
| 85 | # use are a mixture of headers provided by the C library itself (which, |
| 86 | # for bionic, are in bionic/libc/include/) and headers provided by the |
| 87 | # kernel. This is because ISO C and POSIX will say things like "there is |
| 88 | # a constant called PROT_NONE" or "there is a type called struct stat, |
| 89 | # and it contains a field called st_size", but they won't necessarily say |
| 90 | # what _value_ that constant has, or what _order_ the fields in a type |
| 91 | # are in. Those are left to individual kernels' ABIs. In an effort to -- |
| 92 | # amongst other things, see https://lwn.net/Articles/507794/ for more |
| 93 | # background -- reduce copy & paste, the Linux kernel makes all the types |
| 94 | # and constants that make up the "userspace API" (uapi) available as |
| 95 | # headers separate from their internal-use headers (which contain all kinds |
| 96 | # of extra stuff that isn't available to userspace). We import the latest |
| 97 | # released kernel's uapi headers in external/kernel-headers/, but we don't |
| 98 | # use those headers directly in bionic. The bionic/libc/kernel/ directory |
| 99 | # contains scrubbed copies of the originals from external/kernel-headers/. |
| 100 | # The generate_uapi_headers.sh script should be used to go from a kernel |
| 101 | # tree to external/kernel-headers/ --- this takes care of the |
| 102 | # architecture-specific details. The update_all.py script should then be |
| 103 | # used to regenerate bionic's copy from external/kernel-headers/. |
| 104 | # The files in bionic must not be edited directly because any local changes |
| 105 | # will be overwritten by the next update. "Updating kernel header files" |
| 106 | # below has more information on this process. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 107 | |
| 108 | private/ |
| 109 | # These are private header files meant for use within bionic itself. |
| 110 | |
| Calin Juravle | bd33537 | 2014-02-28 16:31:04 +0000 | [diff] [blame] | 111 | dns/ |
| 112 | # Contains the DNS resolver (originates from NetBSD code). |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 113 | |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 114 | upstream-freebsd/ |
| 115 | upstream-netbsd/ |
| 116 | upstream-openbsd/ |
| Elliott Hughes | 7468ac2 | 2024-01-10 00:25:30 +0000 | [diff] [blame] | 117 | # These directories contain upstream source with no local changes. |
| 118 | # Any time we can just use a BSD implementation of something unmodified, |
| 119 | # we should. Ideally these should probably have been three separate git |
| 120 | # projects in external/, but they're here instead mostly by historical |
| 121 | # accident (because it wouldn't have been easy to import just the tiny |
| 122 | # subset of these operating systems that -- unlike Android -- just have |
| 123 | # one huge repository rather than lots of little ones and a mechanism |
| 124 | # like our `repo` tool). |
| 125 | # The structure under these directories mimics the relevant upstream tree, |
| 126 | # but in order to actually be able to compile this code in our tree |
| 127 | # _without_ making modifications to the source files directly, we also |
| 128 | # have the following subdirectories in each one that aren't upstream: |
| Elliott Hughes | d39f3f2 | 2014-04-21 17:13:46 -0700 | [diff] [blame] | 129 | android/ |
| 130 | include/ |
| 131 | # This is where we keep the hacks necessary to build BSD source |
| 132 | # in our world. The *-compat.h files are automatically included |
| 133 | # using -include, but we also provide equivalents for missing |
| 134 | # header/source files needed by the BSD implementation. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 135 | |
| 136 | bionic/ |
| 137 | # This is the biggest mess. The C++ files are files we own, typically |
| 138 | # because the Linux kernel interface is sufficiently different that we |
| 139 | # can't use any of the BSD implementations. The C files are usually |
| 140 | # legacy mess that needs to be sorted out, either by replacing it with |
| 141 | # current upstream source in one of the upstream directories or by |
| 142 | # switching the file to C++ and cleaning it up. |
| 143 | |
| Christopher Ferris | 63860cb | 2015-11-16 17:30:32 -0800 | [diff] [blame] | 144 | malloc_debug/ |
| 145 | # The code that implements the functionality to enable debugging of |
| 146 | # native allocation problems. |
| 147 | |
| Elliott Hughes | 3ad8ecb | 2014-07-21 16:35:24 -0700 | [diff] [blame] | 148 | stdio/ |
| 149 | # These are legacy files of dubious provenance. We're working to clean |
| 150 | # this mess up, and this directory should disappear. |
| 151 | |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 152 | tools/ |
| 153 | # Various tools used to maintain bionic. |
| 154 | |
| 155 | tzcode/ |
| 156 | # A modified superset of the IANA tzcode. Most of the modifications relate |
| 157 | # to Android's use of a single file (with corresponding index) to contain |
| Elliott Hughes | 31fc69f | 2023-06-20 15:36:11 -0700 | [diff] [blame] | 158 | # timezone data. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 159 | zoneinfo/ |
| Elliott Hughes | 31fc69f | 2023-06-20 15:36:11 -0700 | [diff] [blame] | 160 | # Android-format timezone data. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 161 | # See 'Updating tzdata' later. |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 162 | ``` |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 163 | |
| 164 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 165 | ## Adding libc wrappers for system calls |
| Elliott Hughes | 1a1b57c | 2018-02-08 09:38:54 -0800 | [diff] [blame] | 166 | |
| 167 | The first question you should ask is "should I add a libc wrapper for |
| 168 | this system call?". The answer is usually "no". |
| 169 | |
| 170 | The answer is "yes" if the system call is part of the POSIX standard. |
| 171 | |
| 172 | The answer is probably "yes" if the system call has a wrapper in at |
| Elliott Hughes | a96306c | 2022-04-01 11:47:58 -0700 | [diff] [blame] | 173 | least one other C library (typically glibc/musl or Apple's libc). |
| Elliott Hughes | 1a1b57c | 2018-02-08 09:38:54 -0800 | [diff] [blame] | 174 | |
| 175 | The answer may be "yes" if the system call has three/four distinct |
| Elliott Hughes | a96306c | 2022-04-01 11:47:58 -0700 | [diff] [blame] | 176 | users in different projects, and there isn't a more specific higher-level |
| 177 | library that would make more sense as the place to add the wrapper. |
| Elliott Hughes | 1a1b57c | 2018-02-08 09:38:54 -0800 | [diff] [blame] | 178 | |
| 179 | In all other cases, you should use |
| Elliott Hughes | bbd39aa | 2024-08-13 20:59:16 +0000 | [diff] [blame] | 180 | [syscall(3)](https://man7.org/linux/man-pages/man2/syscall.2.html) instead. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 181 | |
| 182 | Adding a system call usually involves: |
| 183 | |
| Elliott Hughes | 6b8b22c | 2022-04-22 15:15:08 -0700 | [diff] [blame] | 184 | 1. Add an entry (or entries, in some cases) to SYSCALLS.TXT. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 185 | See SYSCALLS.TXT itself for documentation on the format. |
| Elliott Hughes | 6b8b22c | 2022-04-22 15:15:08 -0700 | [diff] [blame] | 186 | See also the notes below for how to deal with tricky cases like `off_t`. |
| 187 | 2. Find the right header file to work in by looking up your system call |
| 188 | on [man7.org](https://man7.org/linux/man-pages/dir_section_2.html). |
| 189 | (If there's no header file given, see the points above about whether we |
| 190 | should really be adding this or not!) |
| 191 | 3. Add constants (and perhaps types) to the appropriate header file. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 192 | Note that you should check to see whether the constants are already in |
| 193 | kernel uapi header files, in which case you just need to make sure that |
| Elliott Hughes | 6b8b22c | 2022-04-22 15:15:08 -0700 | [diff] [blame] | 194 | the appropriate header file in libc/include/ `#include`s the relevant |
| 195 | `linux/` file or files. |
| 196 | 4. Add function declarations to the appropriate header file. Don't forget |
| 197 | to include the appropriate `__INTRODUCED_IN()`, with the right API level |
| 198 | for the first release your system call wrapper will be in. See |
| 199 | libc/include/android/api_level.h for the API levels. |
| 200 | If the header file doesn't exist, copy all of libc/include/sys/sysinfo.h |
| 201 | into your new file --- it's a good short example to start from. |
| 202 | |
| 203 | Note also our style for naming arguments: always use two leading |
| 204 | underscores (so developers are free to use any of the unadorned names as |
| 205 | macros without breaking things), avoid abbreviations, and ideally try to |
| 206 | use the same name as an existing system call (to reduce the amount of |
| 207 | English vocabulary required by people who just want to use the function |
| 208 | signatures). If there's a similar function already in the C library, |
| 209 | check what names it's used. Finally, prefer the `void*` orthography we |
| 210 | use over the `void *` you'll see on man7.org.) |
| 211 | 5. Add basic documentation to the header file. Again, the existing |
| 212 | libc/include/sys/sysinfo.h is a good short example that shows the |
| 213 | expected style. |
| 214 | |
| 215 | Most of the detail should actually be left to the man7.org page, with |
| 216 | only a brief one-sentence explanation (usually based on the description |
| 217 | in the NAME section of the man page) in our documentation. Always |
| 218 | include the return value/error reporting details (you can find out |
| 219 | what the system call returns from the RETURN VALUE of the man page), |
| 220 | but try to match the wording and style wording from _our_ existing |
| 221 | documentation; we're trying to minimize the amount of English readers |
| 222 | need to understand by using the exact same wording where possible). |
| 223 | Explicitly say which version of Android the function was added to in |
| 224 | the documentation because the documentation generation tool doesn't yet |
| 225 | understand `__INTRODUCED_IN()`. |
| 226 | |
| 227 | Explicitly call out any Android-specific changes/additions/limitations |
| 228 | because they won't be on the man7.org page. |
| 229 | 6. Add the function name to the correct section in libc/libc.map.txt; it'll |
| 230 | be near the end of the file. You may need to add a new section if you're |
| 231 | the first to add a system call to this version of Android. |
| 232 | 7. Add a basic test. Don't try to test everything; concentrate on just testing |
| Elliott Hughes | 4ece2d6 | 2022-04-11 13:36:18 -0700 | [diff] [blame] | 233 | the code that's actually in *bionic*, not all the functionality that's |
| 234 | implemented in the kernel. For simple syscalls, that's just the |
| 235 | auto-generated argument and return value marshalling. |
| 236 | |
| zijunzhao | e5e1e4e | 2022-05-10 21:22:00 +0000 | [diff] [blame] | 237 | Add a test in the right file in tests/. We have one file per header, so if |
| 238 | your system call is exposed in <unistd.h>, for example, your test would go |
| 239 | in tests/unistd_test.cpp. |
| 240 | |
| Elliott Hughes | 4ece2d6 | 2022-04-11 13:36:18 -0700 | [diff] [blame] | 241 | A trivial test that deliberately supplies an invalid argument helps check |
| 242 | that we're generating the right symbol and have the right declaration in |
| 243 | the header file, and that the change to libc.map.txt from step 5 is |
| 244 | correct. (You can use strace(1) manually to confirm that the correct |
| 245 | system call is being made.) |
| 246 | |
| 247 | For testing the *kernel* side of things, we should prefer to rely on |
| 248 | https://github.com/linux-test-project/ltp for kernel testing, but you'll |
| 249 | want to check that external/ltp does contain tests for the syscall you're |
| 250 | adding. Also check that external/ltp is using the libc wrapper for the |
| 251 | syscall rather than calling it "directly" via syscall(3)! |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 252 | |
| Elliott Hughes | a96306c | 2022-04-01 11:47:58 -0700 | [diff] [blame] | 253 | Some system calls are harder than others. The most common problem is a 64-bit |
| 254 | argument such as `off64_t` (a *pointer* to a 64-bit argument is fine, since |
| 255 | pointers are always the "natural" size for the architecture regardless of the |
| 256 | size of the thing they point to). Whenever you have a function that takes |
| 257 | `off_t` or `off64_t`, you'll need to consider whether you actually need a foo() |
| 258 | and a foo64(), and whether they will use the same underlying system call or are |
| 259 | implemented as two different system calls. It's usually easiest to find a |
| 260 | similar system call and copy and paste from that. You'll definitely need to test |
| Elliott Hughes | 4ece2d6 | 2022-04-11 13:36:18 -0700 | [diff] [blame] | 261 | both on 32-bit and 64-bit. (These special cases warrant more testing than the |
| 262 | easy cases, even if only manual testing with strace. Sadly it isn't always |
| 263 | feasible to write a working test for the interesting cases -- offsets larger |
| 264 | than 2GiB, say -- so you may end up just writing a "meaningless" program whose |
| 265 | only purpose is to give you patterns to look for when run under strace(1).) |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 266 | |
| zijunzhao | e5e1e4e | 2022-05-10 21:22:00 +0000 | [diff] [blame] | 267 | A general example of adding a system call: |
| 268 | https://android-review.googlesource.com/c/platform/bionic/+/2073827 |
| 269 | |
| 270 | ### Debugging tips |
| zijunzhao | a3696c1 | 2022-11-09 23:57:55 +0000 | [diff] [blame] | 271 | |
| Elliott Hughes | e64bec4 | 2025-03-04 12:38:54 -0800 | [diff] [blame] | 272 | If a test fails to build with an undefined symbol error, |
| 273 | this is most likely the _host_ reference test against glibc, |
| 274 | and you need to add an `#if defined(__GLIBC__)` to the test. |
| 275 | (Search for existing examples to copy & paste, |
| 276 | in particular to make sure you include the `GTEST_SKIP()`.) |
| zijunzhao | e5e1e4e | 2022-05-10 21:22:00 +0000 | [diff] [blame] | 277 | |
| Elliott Hughes | e64bec4 | 2025-03-04 12:38:54 -0800 | [diff] [blame] | 278 | When we switch to musl for the host libc, this should be less of a problem. |
| zijunzhao | e5e1e4e | 2022-05-10 21:22:00 +0000 | [diff] [blame] | 279 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 280 | ## Updating kernel header files |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 281 | |
| 282 | As mentioned above, this is currently a two-step process: |
| 283 | |
| 284 | 1. Use generate_uapi_headers.sh to go from a Linux source tree to appropriate |
| 285 | contents for external/kernel-headers/. |
| 286 | 2. Run update_all.py to scrub those headers and import them into bionic. |
| 287 | |
| Elliott Hughes | a57c878 | 2017-07-20 10:36:27 -0700 | [diff] [blame] | 288 | Note that if you're actually just trying to expose device-specific headers to |
| 289 | build your device drivers, you shouldn't modify bionic. Instead use |
| Elliott Hughes | 9c06d16 | 2023-10-04 23:36:14 +0000 | [diff] [blame] | 290 | `TARGET_DEVICE_KERNEL_HEADERS` and friends described in [config.mk](https://android.googlesource.com/platform/build/+/main/core/config.mk#186). |
| Elliott Hughes | a57c878 | 2017-07-20 10:36:27 -0700 | [diff] [blame] | 291 | |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 292 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 293 | ## Updating tzdata |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 294 | |
| Elliott Hughes | 47bdb6b | 2025-03-04 12:42:07 -0800 | [diff] [blame] | 295 | Tzdata updates are now handled by the libcore team because it needs to be |
| 296 | updated in sync with icu's copy of the data, and they own that. |
| 297 | |
| 298 | See |
| 299 | [system/timezone/README.android](https://android.googlesource.com/platform/system/timezone/+/main/README.android) |
| 300 | for more information. |
| Elliott Hughes | 560cee6 | 2014-02-18 22:08:56 -0800 | [diff] [blame] | 301 | |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 302 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 303 | ## Verifying changes |
| Dan Albert | e66d57f | 2014-11-12 17:08:38 -0800 | [diff] [blame] | 304 | |
| 305 | If you make a change that is likely to have a wide effect on the tree (such as a |
| 306 | libc header change), you should run `make checkbuild`. A regular `make` will |
| 307 | _not_ build the entire tree; just the minimum number of projects that are |
| 308 | required for the device. Tests, additional developer tools, and various other |
| 309 | modules will not be built. Note that `make checkbuild` will not be complete |
| 310 | either, as `make tests` covers a few additional modules, but generally speaking |
| 311 | `make checkbuild` is enough. |
| 312 | |
| 313 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 314 | ## Running the tests |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 315 | |
| Elliott Hughes | e6d6efd | 2024-02-21 15:52:10 +0000 | [diff] [blame] | 316 | The tests are all built from the tests/ directory. There is a separate |
| 317 | directory `benchmarks/` containing benchmarks, and that has its own |
| 318 | documentation on [running the benchmarks](benchmarks/README.md). |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 319 | |
| Elliott Hughes | f87b538 | 2025-03-04 13:36:01 -0800 | [diff] [blame] | 320 | ### Building |
| 321 | |
| 322 | We assume you've already checked out the Android source tree. |
| 323 | |
| 324 | To build, make sure you're in the right directory and you've set up your environment: |
| 325 | |
| 326 | $ cd main # Or whatever you called your Android source tree. |
| 327 | $ source build/envsetup.sh |
| 328 | |
| 329 | Then choose an appropriate "lunch". If you're not testing on a device, |
| 330 | two choices are particularly useful. |
| 331 | |
| 332 | If you want to be able to run tests and benchmarks directly on your x86-64 |
| 333 | host machine, use: |
| 334 | |
| 335 | $ lunch aosp_cf_x86_64_phone-trunk_staging-userdebug |
| 336 | |
| 337 | Alternatively, if you want to (say) check generated arm64 code without having |
| 338 | a specific device in mind, use: |
| 339 | |
| 340 | $ lunch aosp_cf_arm64_phone-trunk_staging-userdebug |
| 341 | |
| 342 | Note that in both cases, |
| 343 | these targets will also build the corresponding 32-bit variant. |
| 344 | See below for where the 64-bit and 32-bit files end up. |
| 345 | |
| 346 | See [Build Android](https://source.android.com/docs/setup/build/building) |
| 347 | for more details. |
| 348 | |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 349 | ### Device tests |
| 350 | |
| Elliott Hughes | f87b538 | 2025-03-04 13:36:01 -0800 | [diff] [blame] | 351 | Once you've completed that setup, you can build: |
| 352 | |
| 353 | $ cd bionic |
| 354 | $ mm |
| 355 | |
| 356 | This will build everything: bionic, the benchmarks, and the tests |
| 357 | (and all the dependencies). |
| 358 | |
| 359 | If you want to test on a device, |
| 360 | the first time after flashing your device, |
| 361 | you'll need to remount the filesystems to be writable: |
| 362 | |
| 363 | $ adb root |
| 364 | $ adb remount |
| 365 | $ adb reboot |
| 366 | $ adb wait-for-device |
| 367 | $ adb root |
| 368 | $ adb remount |
| 369 | |
| 370 | Then you can sync your locally built files across: |
| 371 | |
| 372 | $ adb sync |
| 373 | |
| 374 | And then you can run the 32-bit tests (dynamic or static): |
| 375 | |
| Sergii Piatakov | c3e3060 | 2017-11-02 13:41:32 +0200 | [diff] [blame] | 376 | $ adb shell /data/nativetest/bionic-unit-tests/bionic-unit-tests |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 377 | $ adb shell \ |
| Sergii Piatakov | c3e3060 | 2017-11-02 13:41:32 +0200 | [diff] [blame] | 378 | /data/nativetest/bionic-unit-tests-static/bionic-unit-tests-static |
| Elliott Hughes | f87b538 | 2025-03-04 13:36:01 -0800 | [diff] [blame] | 379 | |
| 380 | Or the 64-bit tests (dynamic or static): |
| 381 | |
| Sergii Piatakov | c3e3060 | 2017-11-02 13:41:32 +0200 | [diff] [blame] | 382 | $ adb shell /data/nativetest64/bionic-unit-tests/bionic-unit-tests |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 383 | $ adb shell \ |
| Sergii Piatakov | c3e3060 | 2017-11-02 13:41:32 +0200 | [diff] [blame] | 384 | /data/nativetest64/bionic-unit-tests-static/bionic-unit-tests-static |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 385 | |
| Elliott Hughes | 20758d5 | 2016-07-19 14:09:10 -0700 | [diff] [blame] | 386 | Note that we use our own custom gtest runner that offers a superset of the |
| 387 | options documented at |
| zijunzhao | a3696c1 | 2022-11-09 23:57:55 +0000 | [diff] [blame] | 388 | <https://github.com/google/googletest/blob/main/docs/advanced.md#running-test-programs-advanced-options>, |
| Elliott Hughes | 20758d5 | 2016-07-19 14:09:10 -0700 | [diff] [blame] | 389 | in particular for test isolation and parallelism (both on by default). |
| 390 | |
| Elliott Hughes | 0e8804e | 2016-12-02 13:22:21 -0800 | [diff] [blame] | 391 | ### Device tests via CTS |
| 392 | |
| 393 | Most of the unit tests are executed by CTS. By default, CTS runs as |
| 394 | a non-root user, so the unit tests must also pass when not run as root. |
| 395 | Some tests cannot do any useful work unless run as root. In this case, |
| 396 | the test should check `getuid() == 0` and do nothing otherwise (typically |
| 397 | we log in this case to prevent accidents!). Obviously, if the test can be |
| 398 | rewritten to not require root, that's an even better solution. |
| 399 | |
| 400 | Currently, the list of bionic CTS tests is generated at build time by |
| 401 | running a host version of the test executable and dumping the list of |
| 402 | all tests. In order for this to continue to work, all architectures must |
| 403 | have the same number of tests, and the host version of the executable |
| 404 | must also have the same number of tests. |
| 405 | |
| 406 | Running the gtests directly is orders of magnitude faster than using CTS, |
| 407 | but in cases where you really have to run CTS: |
| 408 | |
| 409 | $ make cts # In $ANDROID_ROOT. |
| 410 | $ adb unroot # Because real CTS doesn't run as root. |
| 411 | # This will sync any *test* changes, but not *code* changes: |
| 412 | $ cts-tradefed \ |
| 413 | run singleCommand cts --skip-preconditions -m CtsBionicTestCases |
| 414 | |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 415 | ### Host tests |
| 416 | |
| 417 | The host tests require that you have `lunch`ed either an x86 or x86_64 target. |
| Elliott Hughes | f87b538 | 2025-03-04 13:36:01 -0800 | [diff] [blame] | 418 | |
| 419 | (Obviously, in theory you could build for arm64 and run on an arm64 host, |
| 420 | but we currently only support x86-64 host builds.) |
| 421 | |
| 422 | For example: |
| 423 | |
| 424 | $ lunch aosp_cf_x86_64_phone-trunk_staging-userdebug |
| 425 | |
| 426 | Then build as normal: |
| 427 | |
| 428 | $ cd bionic |
| 429 | $ mm |
| 430 | |
| Josh Gao | 6cd1c92 | 2016-11-17 18:52:09 -0800 | [diff] [blame] | 431 | Note that due to ABI limitations (specifically, the size of pthread_mutex_t), |
| Elliott Hughes | f87b538 | 2025-03-04 13:36:01 -0800 | [diff] [blame] | 432 | 32-bit bionic requires PIDs less than 65536. |
| 433 | To enforce this, set /proc/sys/kernel/pid_max to 65536. |
| 434 | (The tests will remind you if you forget.) |
| 435 | |
| 436 | The easiest way to run is to use our provided script. |
| 437 | |
| 438 | To run the 32-bit tests on the host: |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 439 | |
| Elliott Hughes | 86f1e04 | 2016-08-01 13:16:37 -0700 | [diff] [blame] | 440 | $ ./tests/run-on-host.sh 32 |
| Elliott Hughes | f87b538 | 2025-03-04 13:36:01 -0800 | [diff] [blame] | 441 | |
| 442 | To run the 64-bit tests on the host: |
| 443 | |
| 444 | $ ./tests/run-on-host.sh 64 |
| Elliott Hughes | 86f1e04 | 2016-08-01 13:16:37 -0700 | [diff] [blame] | 445 | |
| 446 | You can supply gtest flags as extra arguments to this script. |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 447 | |
| Elliott Hughes | f87b538 | 2025-03-04 13:36:01 -0800 | [diff] [blame] | 448 | This script starts by running build/run-on-host.sh which -- despite the name -- |
| 449 | is actually a script to set up your host to look more like an Android device. |
| 450 | In particular, it creates a /system directory with appropriate symlinks to your |
| 451 | "out" directory. |
| 452 | |
| 453 | An alternative is to run the static binaries directly from your "out" directory. |
| 454 | |
| 455 | To run the static 32-bit tests: |
| 456 | |
| 457 | $ ../out/target/product/vsoc_x86_64/data/nativetest/bionic-unit-tests-static/bionic-unit-tests-static |
| 458 | |
| 459 | To run the static 64-bit tests: |
| 460 | |
| 461 | $ ../out/target/product/vsoc_x86_64/data/nativetest64/bionic-unit-tests-static/bionic-unit-tests-static |
| 462 | |
| 463 | |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 464 | ### Against glibc |
| 465 | |
| 466 | As a way to check that our tests do in fact test the correct behavior (and not |
| 467 | just the behavior we think is correct), it is possible to run the tests against |
| Elliott Hughes | 86f1e04 | 2016-08-01 13:16:37 -0700 | [diff] [blame] | 468 | the host's glibc. |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 469 | |
| Elliott Hughes | 86f1e04 | 2016-08-01 13:16:37 -0700 | [diff] [blame] | 470 | $ ./tests/run-on-host.sh glibc |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 471 | |
| Dan Albert | b9bc509 | 2023-07-19 18:37:43 +0000 | [diff] [blame] | 472 | ### Against musl |
| 473 | |
| 474 | Another way to verify test behavior is to run against musl on the host. glibc |
| 475 | musl don't always match, so this can be a good way to find the more complicated |
| 476 | corners of the spec. If they *do* match, bionic probably should too! |
| 477 | |
| 478 | $ OUT_DIR=$(ANDROID_BUILD_TOP)/musl-out ./tests/run-on-host.sh musl |
| 479 | |
| 480 | Note: the alternate OUT_DIR is used to avoid causing excessive rebuilding when |
| 481 | switching between glibc and musl. The first musl test run will be expensive |
| 482 | because it will not reuse any already built artifacts, but subsequent runs will |
| 483 | be cheaper than if you hadn't used it. |
| 484 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 485 | ## Gathering test coverage |
| Dan Albert | efee1ce | 2014-10-09 22:57:49 -0700 | [diff] [blame] | 486 | |
| Dan Albert | 837b683 | 2021-10-14 15:28:01 -0700 | [diff] [blame] | 487 | To get test coverage for bionic, use `//bionic/build/coverage.sh`. Before |
| 488 | running, follow the instructions at the top of the file to rebuild bionic with |
| 489 | coverage instrumentation. |
| Elliott Hughes | 0b1de06 | 2015-01-09 12:21:24 -0800 | [diff] [blame] | 490 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 491 | ## Attaching GDB to the tests |
| Dan Albert | 1af434c | 2015-09-18 13:17:02 -0700 | [diff] [blame] | 492 | |
| 493 | Bionic's test runner will run each test in its own process by default to prevent |
| 494 | tests failures from impacting other tests. This also has the added benefit of |
| 495 | running them in parallel, so they are much faster. |
| 496 | |
| 497 | However, this also makes it difficult to run the tests under GDB. To prevent |
| 498 | each test from being forked, run the tests with the flag `--no-isolate`. |
| 499 | |
| 500 | |
| Elliott Hughes | 84941fe | 2018-09-04 13:22:56 -0700 | [diff] [blame] | 501 | ## 32-bit ABI bugs |
| Elliott Hughes | 0b1de06 | 2015-01-09 12:21:24 -0800 | [diff] [blame] | 502 | |
| Elliott Hughes | 0bfcbaf | 2017-08-28 09:18:34 -0700 | [diff] [blame] | 503 | See [32-bit ABI bugs](docs/32-bit-abi.md). |