summaryrefslogtreecommitdiff
path: root/src
AgeCommit message (Collapse)AuthorLines
2019-07-28sigtimedwait: add time64 syscall support, decouple 32-bit time_tRich Felker-4/+24
time64 syscall is used only if it's the only one defined for the arch, or if the requested timeout length does not fit in 32 bits. on current 32-bit archs where time_t is a 32-bit type, this makes it statically unreachable. on 64-bit archs, there are only superficial changes to the code after preprocessing. on current 32-bit archs, the timeout is passed via an intermediate copy to remove the assumption that time_t is a 32-bit type.
2019-07-28mq_timedsend, mq_timedreceive: add time64, decouple 32-bit time_tRich Felker-0/+34
time64 syscall is used only if it's the only one defined for the arch, or if the requested absolute timeout does not fit in 32 bits. on current 32-bit archs where time_t is a 32-bit type, this makes it statically unreachable. on 64-bit archs, there is no change to the code after preprocessing. on current 32-bit archs, the timeout is passed via an intermediate copy to remove the assumption that time_t is a 32-bit type.
2019-07-28clock_nanosleep: add time64 syscall support, decouple 32-bit time_tRich Felker-0/+25
time64 syscall is used only if it's the only one defined for the arch, or if the requested time does not fit in 32 bits. on current 32-bit archs where time_t is a 32-bit type, this makes it statically unreachable. on 64-bit archs, there is no change to the code after preprocessing. on current 32-bit archs, the time is moved through an intermediate copy to remove the assumption that time_t is a 32-bit type.
2019-07-27implement settimeofday in terms of clock_settime, not old syscallRich Felker-1/+6
this is yet another place where special handling of time syscalls can and should be avoided by implementing legacy functions in terms of their modern replacements. in theory a fallback to SYS_settimeofday could be added to clock_settime, but SYS_clock_settime has been available since Linux 2.6.0 or earlier, i.e. all the way back to the minimum supported version.
2019-07-27internally, define plain syscalls, if missing, as their time64 variantsRich Felker-0/+83
this commit has no effect whatsoever right now, but is in preparation for a future riscv32 port and other future 32-bit archs that will be "time64-only" from the start on the kernel side. together with the previous x32 changes, this commit ensures that syscall call points that don't care about time (passing null timeouts, etc.) can continue to do so without having to special-case time64-only archs, and allows code using the time64 syscalls to uniformly test for the need to fallback with SYS_foo != SYS_foo_time64, rather than needing to check defined(SYS_foo) && SYS_foo != SYS_foo_time64.
2019-07-27fix and simplify futimesat fallback in utimensatRich Felker-6/+4
previously the fallback wrongly failed with EINVAL rather than ENOSYS when UTIME_NOW was used with one component but not both. commit dd5f50da6f6c3df5647e922e47f8568a8896a752 introduced this behavior when initially adding the fallback support. instead, detect the case where both are UTIME_NOW early and replace with a null times pointer; this may improve performance slightly (less copy from user), and removes the complex logic from the fallback case. it also makes things slightly simpler for adding time64 code paths.
2019-07-27refactor thrd_sleep and nanosleep in terms of clock_nanosleepRich Felker-5/+11
for namespace-safety with thrd_sleep, this requires an alias, which is also added. this eliminates all but one direct call point for nanosleep syscalls, and arranges that 64-bit time_t conversion logic will only need to exist in one file rather than three. as a bonus, clock_nanosleep with CLOCK_REALTIME and empty flags is now implemented as SYS_nanosleep, thereby working on older kernels that may lack POSIX clocks functionality.
2019-07-21use the correct stat structure in the fstat pathSamuel Holland-4/+4
commit 01ae3fc6d48f4a45535189b7a6db286535af08ca modified fstatat to translate the kernel's struct stat ("kstat") into the libc struct stat. To do this, it created a local kstat object, and copied its contents into the user-provided object. However, the commit neglected to update the fstat compatibility path and its fallbacks. They continued to pass the user-supplied object to the kernel, later overwiting it with the uninitialized memory in the local temporary.
2019-07-20refactor adjtime function using adjtimex function instead of syscallRich Felker-1/+1
this removes the assumption that userspace struct timex matches the syscall type and sets the stage for 64-bit time_t on 32-bit archs.
2019-07-20refactor adjtimex in terms of clock_adjtimeRich Felker-2/+4
this sets the stage for having the conversion logic for 64-bit time_t all in one file, and as a bonus makes clock_adjtime for CLOCK_REALTIME work even on kernels too old to have the clock_adjtime syscall.
2019-07-19fix inadvertent introduction of extern object stxRich Felker-1/+1
commit dfc81828f7ab41da08f744c44117a1bb20a05749 accidentally defined an instance of struct statx along with the struct declaration.
2019-07-18implement fstatat with SYS_statx, conditional on undersized kstat timeRich Felker-2/+68
this commit adds a new backend for fstatat (and thereby the whole stat family) using the SYS_statx syscall, but conditions the new code on the kernel stat structure's time fields being smaller than time_t. in principle that should make it all dead code at present, but mips64 has a broken stat structure with 32-bit time fields despite having 64-bit time_t elsewhere, so on mips64 it is a functional change that makes post-Y2038 filesystem timestamps accessible. whenever the 32-bit archs end up getting 64-bit time_t, regardless of how that happens, the changes in this commit will automatically take effect for them too.
2019-07-18cleanup includes now that stat, lstat no longer make direct syscallsRich Felker-2/+0
2019-07-18restore property that fstat(AT_FDCWD) fails with EBADFRich Felker-0/+1
AT_FDCWD is not a valid file descriptor, so POSIX requires fstat to fail with EBADF. if passed to fstatat, the call would spuriously succeed and return results for the working directory.
2019-07-18decouple struct stat from kernel typeRich Felker-4/+27
presently, all archs/ABIs have struct stat matching the kernel stat[64] type, except mips/mipsn32/mips64 which do conversion hacks in syscall_arch.h to work around bugs in the kernel type. this patch completely decouples them and adds a translation step to the success path of fstatat. at present, this is just a gratuitous copying, but it opens up multiple possibilities for future support for 64-bit time_t on 32-bit archs and for cleaned-up/unified ABIs. for clarity, the mips hacks are not yet removed in this commit, so the mips kstat structs still correspond to the output of the hacks in their syscall_arch.h files, not the raw kernel type. a subsequent commit will fix this.
2019-07-18refactor all stat functions in terms of fstatatRich Felker-23/+37
equivalent logic for fstat+O_PATH fallback and direct use of stat/lstat syscalls where appropriate is kept, now in the fstatat function. this change both improves functionality (now, fstatat forms equivalent to fstat/lstat/stat will work even on kernels too old to have the at functions) and localizes direct interfacing with the kernel stat structure to one file.
2019-07-16fix broken lseek on x32 (x86_64/ILP32) with offsets larger than LONG_MAXRich Felker-0/+15
this is analogous to commit 918c5fa0fc656e49b1ab9ce47183a23e3a36bc00 which fixed the corresponding issue for mips n32.
2019-07-16fix broken lseek on mipsn32 with offsets larger than LONG_MAXRich Felker-0/+20
mips n32 has 32-bit long, and generally uses long syscall arguments and return values, but provides only SYS_lseek, not SYS_llseek. we have some framework (syscall_arg_t, added for x32) to make syscall arguments 64-bit in such a setting, but it's not clear whether this could match the sign-extension semantics needed for 32-bit args to all the other syscalls, and we don't have any existing mechanism to allow the return value of syscalls to be something other than long. instead, just provide a custom mipsn32 version of the lseek function doing its own syscall asm with 64-bit arguments. as a result of commit 03919b26ed41c31876db41f7cee076ced4513fad, stdio will also get the new code, fixing fseeko/ftello too.
2019-07-16use namespace-safe __lseek for __stdio_seek instead of direct syscallRich Felker-10/+6
this probably saves a few bytes, avoids duplicating the clunky lseek/_llseek syscall convention in two places, and sets the stage for fixing broken seeks on x32 and mipsn32.
2019-07-13fix sigaltstack to ignore ss_size with SS_DISABLE, per POSIXJames Y Knight-1/+1
2019-07-10use the correct attributes for ___errno_locationSamuel Holland-0/+3
In the public header, __errno_location is declared with the "const" attribute, conditional on __GNUC__. Ensure that its internal alias has the same attributes. Maintainer's note: This change also fixes a regression in quality of code generation -- multiple references to errno in a single function started generating multiple calls again -- introduced by commit e13063aad7aee341d278d2a879a76ec7b59b2ad8.
2019-07-10fix restrict violations in internal use of several functionsSamuel Holland-10/+10
The old/new parameters to pthread_sigmask, sigprocmask, and setitimer are marked restrict, so passing the same address to both is prohibited. Modify callers of these functions to use a separate object for each argument.
2019-07-08prevent dup2 action for posix_spawn internal pipe fdRich Felker-0/+4
as reported by Tavian Barnes, a dup2 file action for the internal pipe fd used by posix_spawn could cause it to remain open after execve and allow the child to write an artificial error into it, confusing the parent. POSIX allows internal use of file descriptors by the implementation, with undefined behavior for poking at them, so this is not a conformance problem, but it seems preferable to diagnose and prevent the error when we can do so easily. catch attempts to apply a dup2 action to the internal pipe fd and emulate EBADF for it instead.
2019-07-02remove stray .end directives from powerpc[64] asmFangrui Song-2/+0
maintainer's note: these are not meaningful/correct/needed and the clang integrated assembler errors out upon seeing them.
2019-07-01ipc: prefer SYS_ipc when it is definedSzabolcs Nagy-12/+12
Linux v5.1 introduced ipc syscalls on targets where previously only SYS_ipc was available, change the logic such that the ipc code keeps using SYS_ipc which works backward compatibly on older kernels. This changes behaviour on microblaze which had both mechanisms, now SYS_ipc will be used instead of separate syscalls.
2019-07-01fix deadlock in synccall after threaded forkSamuel Holland-0/+1
synccall may be called by AS-safe functions such as setuid/setgid after fork. although fork() resets libc.threads_minus_one, causing synccall to take the single-threaded path, synccall still takes the thread list lock. This lock may be held by another thread if for example fork() races with pthread_create(). After fork(), the value of the lock is meaningless, so clear it. maintainer's note: commit 8f11e6127fe93093f81a52b15bb1537edc3fc8af and e4235d70672d9751d7718ddc2b52d0b426430768 introduced this regression. the state protected by this lock is the linked list, which is entirely replaced in the child path of fork (next=prev=self), so resetting it is semantically sound.
2019-06-28cap getdents length argument to INT_MAXRich Felker-0/+2
the linux syscall treats this argument as having type int, so passing extremely long buffer sizes would be misinterpreted by the kernel. since "short reads" are always acceptable, just cap it down. patch based on report and suggested change by Florian Weimer.
2019-06-25allow fmemopen with zero sizeRich Felker-1/+1
previously, POSIX erroneously required this to fail with EINVAL despite the traditional glibc implementation, on which the POSIX interface was based, allowing it. the resolution of Austin Group issue 818 removes the requirement to fail.
2019-06-21do not use _Noreturn for a function pointer in dynamic linkerMatthew Maurer-1/+1
_Noreturn is a C11 construct, and may only be used at the site of a function definition.
2019-06-14add riscv64 architecture supportRich Felker-0/+465
Author: Alex Suykov <alex.suykov@gmail.com> Author: Aric Belsito <lluixhi@gmail.com> Author: Drew DeVault <sir@cmpwn.com> Author: Michael Clark <mjc@sifive.com> Author: Michael Forney <mforney@mforney.org> Author: Stefan O'Rear <sorear2@gmail.com> This port has involved the work of many people over several years. I have tried to ensure that everyone with substantial contributions has been credited above; if any omissions are found they will be noted later in an update to the authors/contributors list in the COPYRIGHT file. The version committed here comes from the riscv/riscv-musl repo's commit 3fe7e2c75df78eef42dcdc352a55757729f451e2, with minor changes by me for issues found during final review: - a_ll/a_sc atomics are removed (according to the ISA spec, lr/sc are not safe to use in separate inline asm fragments) - a_cas[_p] is fixed to be a memory barrier - the call from the _start assembly into the C part of crt1/ldso is changed to allow for the possibility that the linker does not place them nearby each other. - DTP_OFFSET is defined correctly so that local-dynamic TLS works - reloc.h LDSO_ARCH logic is simplified and made explicit. - unused, non-functional crti/n asm files are removed. - an empty .sdata section is added to crt1 so that the __global_pointer reference is resolvable. - indentation style errors in some asm files are fixed.
2019-05-26optimize aarch64 dynamic tlsdesc function to spill fewer registersRich Felker-10/+7
with the glibc generation counter model for reusing dynamic tls slots after dlclose, it's really not possible to get away with fewer than 4 working registers. for us however it's always been possible, but tricky, and only became apparent after the switch to installing new dynamic tls at dlopen time. by merging the negated thread pointer into the addend early, the register holding the thread pointer can immediately be reused, bringing the working register count down to three. this allows saving/restoring via a single stp/ldp pair, since the return register x0 does not need to be saved. net reduction of 3 instructions, 2 of which were push/pop.
2019-05-16fix tls offsets when p_vaddr%p_align != 0 on TLS_ABOVE_TP targetsSzabolcs Nagy-1/+2
currently the bfd linker does not seem to create tls segments where p_vaddr%p_align != 0, but this is valid in ELF and then the runtime computed tls offset must satisfy offset%p_align == (base+p_vaddr)%p_align and in case of local exec tls (main executable) the smallest such offset must be used (otherwise it is incompatible with the offset computed by the static linker). the !TLS_ABOVE_TP case is handled correctly (the offset is negative then in the formula). the ldso code for TLS_ABOVE_TP is changed so the static tls offset of each module satisfies the formula.
2019-05-16fix format strings for uid/gid values in putpwent/putgrentRich Felker-2/+2
commit 648c3b4e18b2ce2b6af7d44783e42ca267ea49f5 omitted this change, which is needed to be able to use uid/gid values greater than INT_MAX with these interfaces. it fixes alpine linux bug #10460.
2019-05-05make fgetwc set error indicator for stream on encoding errorsRich Felker-2/+8
this is a requirement in POSIX that's omitted, and seemed potentially non-conforming, in the C standard. as such it was omitted here. however, as part of Austin Group issue #1170, the discrepancy was raised with WG14 and determined to be unintended; future versions of the C standard will require the error indicator to be set, as POSIX does.
2019-05-05allow archs to provide a 7-argument syscall if neededRich Felker-0/+1
commit 788d5e24ca19c6291cebd8d1ad5b5ed6abf42665 noted that we could add this if needed, and in fact it is needed, but not for one of the archs documented as having a 7th syscall arg register. rather, it's needed for mips (o32), where all but the first 4 arguments are passed on the stack, and the stack can accommodate a 7th.
2019-04-20make new math code compatible with unused variable warning/errorRich Felker-3/+6
commit b50d315fd23f0fbc4c11e2583801dd123d933745 introduced fp_force_eval implemented by default with a dead store to a volatile variable. unfortunately introduces warnings with -Wunused-variable and breaks the ability to use -Werror with the default warning options set by configure when warnings are enabled. we could just call fp_barrier instead, but that results in a spurious load after the store due to volatile semantics. the fix committed here avoids the load. it will still produce warnings without -Wno-unused-but-set-variable, but that's part of our default warning profile, and there are already other locations in the source where an unused variable warning will occur without it.
2019-04-17math: new powSzabolcs Nagy-303/+521
from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc The underflow exception is signaled if the result is in the subnormal range even if the result is exact. code size change: +3421 bytes. benchmark on x86_64 before, after, speedup: -Os: pow rthruput: 102.96 ns/call 33.38 ns/call 3.08x pow latency: 144.37 ns/call 54.75 ns/call 2.64x -O3: pow rthruput: 98.91 ns/call 32.79 ns/call 3.02x pow latency: 138.74 ns/call 53.78 ns/call 2.58x
2019-04-17math: new exp and exp2Szabolcs Nagy-480/+434
from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc TOINT_INTRINSICS and EXP_USE_TOINT_NARROW cases are unused. The underflow exception is signaled if the result is in the subnormal range even if the result is exact (e.g. exp2(-1023.0)). code size change: -1672 bytes. benchmark on x86_64 before, after, speedup: -Os: exp rthruput: 12.73 ns/call 6.68 ns/call 1.91x exp latency: 45.78 ns/call 21.79 ns/call 2.1x exp2 rthruput: 6.35 ns/call 5.26 ns/call 1.21x exp2 latency: 26.00 ns/call 16.58 ns/call 1.57x -O3: exp rthruput: 12.75 ns/call 6.73 ns/call 1.89x exp latency: 45.91 ns/call 21.80 ns/call 2.11x exp2 rthruput: 6.47 ns/call 5.40 ns/call 1.2x exp2 latency: 26.03 ns/call 16.54 ns/call 1.57x
2019-04-17math: new log2Szabolcs Nagy-106/+335
from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc code size change: +2458 bytes (+1524 bytes with fma). benchmark on x86_64 before, after, speedup: -Os: log2 rthruput: 16.08 ns/call 10.49 ns/call 1.53x log2 latency: 44.54 ns/call 25.55 ns/call 1.74x -O3: log2 rthruput: 15.92 ns/call 10.11 ns/call 1.58x log2 latency: 44.66 ns/call 26.16 ns/call 1.71x
2019-04-17math: new logSzabolcs Nagy-104/+454
from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc Assume __FP_FAST_FMA implies __builtin_fma is inlined as a single instruction. code size change: +4588 bytes (+2540 bytes with fma). benchmark on x86_64 before, after, speedup: -Os: log rthruput: 12.61 ns/call 7.95 ns/call 1.59x log latency: 41.64 ns/call 23.38 ns/call 1.78x -O3: log rthruput: 12.51 ns/call 7.75 ns/call 1.61x log latency: 41.82 ns/call 23.55 ns/call 1.78x
2019-04-17math: new powfSzabolcs Nagy-240/+232
from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc POWF_SCALE != 1.0 case only matters if TOINT_INTRINSICS is set, which is currently not supported for any target. SNaN is not supported, it would require an issignalingf implementation. code size change: -816 bytes. benchmark on x86_64 before, after, speedup: -Os: powf rthruput: 95.14 ns/call 20.04 ns/call 4.75x powf latency: 137.00 ns/call 34.98 ns/call 3.92x -O3: powf rthruput: 92.48 ns/call 13.67 ns/call 6.77x powf latency: 131.11 ns/call 35.15 ns/call 3.73x
2019-04-17math: new exp2f and expfSzabolcs Nagy-179/+193
from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc In expf TOINT_INTRINSICS is kept, but is unused, it would require support for __builtin_round and __builtin_lround as single instruction. code size change: +94 bytes. benchmark on x86_64 before, after, speedup: -Os: expf rthruput: 9.19 ns/call 8.11 ns/call 1.13x expf latency: 34.19 ns/call 18.77 ns/call 1.82x exp2f rthruput: 5.59 ns/call 6.52 ns/call 0.86x exp2f latency: 17.93 ns/call 16.70 ns/call 1.07x -O3: expf rthruput: 9.12 ns/call 4.92 ns/call 1.85x expf latency: 34.44 ns/call 18.99 ns/call 1.81x exp2f rthruput: 5.58 ns/call 4.49 ns/call 1.24x exp2f latency: 17.95 ns/call 16.94 ns/call 1.06x
2019-04-17math: new log2fSzabolcs Nagy-58/+108
from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc code size change: +177 bytes. benchmark on x86_64 before, after, speedup: -Os: log2f rthruput: 11.38 ns/call 5.99 ns/call 1.9x log2f latency: 35.01 ns/call 22.57 ns/call 1.55x -O3: log2f rthruput: 10.82 ns/call 5.58 ns/call 1.94x log2f latency: 35.13 ns/call 21.04 ns/call 1.67x
2019-04-17math: new logfSzabolcs Nagy-54/+109
from https://github.com/ARM-software/optimized-routines, commit 04884bd04eac4b251da4026900010ea7d8850edc, with minor changes to better fit into musl. code size change: +289 bytes. benchmark on x86_64 before, after, speedup: -Os: logf rthruput: 8.40 ns/call 6.14 ns/call 1.37x logf latency: 31.79 ns/call 24.33 ns/call 1.31x -O3: logf rthruput: 8.43 ns/call 5.58 ns/call 1.51x logf latency: 32.04 ns/call 20.88 ns/call 1.53x
2019-04-17math: add configuration macrosSzabolcs Nagy-0/+5
Musl currently aims to support non-nearest rounding mode and does not support SNaNs. These macros allow marking relevant code paths in case these decisions are changed later (they also help documenting the corner cases involved).
2019-04-17math: add macros for static branch prediction hintsSzabolcs Nagy-0/+9
These don't have an effectw with -Os so not useful with default settings other than documenting the expectation. With --enable-optimize=internal,malloc,string,math the libc.so code size increases by 18K on x86_64 and performance varies in -2% .. +10%.
2019-04-17math: add double precision error handling functionsSzabolcs Nagy-0/+35
2019-04-17math: add single precision error handling functionsSzabolcs Nagy-0/+37
These are supposed to be used in tail call positions when handling special cases in new code. (fp exceptions may be raised "naturally" by the common code path if special casing is more effort.) This implements the error handling apis used in https://github.com/ARM-software/optimized-routines without errno setting.
2019-04-17math: add eval_as_float and eval_as_doubleSzabolcs Nagy-0/+17
Previously type casts or assignments were used for handling excess precision, which assumed standard C99 semantics, but since it's a rarely needed obscure detail, it's better to use explicit helper functions to document where we rely on this. It also helps if the code is used outside of the libc in non-C99 compilation mode: with the default excess precision handling of gcc, explicit inline asm barriers are needed for narrowing on FLT_EVAL_METHOD!=0 targets. I plan to use this in new code with the existing style that uses double_t and float_t as much as possible. One ugliness is that it is required for almost every return statement since that does not drop excess precision (the standard changed this in C11 annex F, but that does not help in non-standard compilation modes or with old compilers).
2019-04-17math: add fp_arch.h with fp_barrier and fp_force_evalSzabolcs Nagy-6/+65
C99 has ways to support fenv access, but compilers don't implement it and assume nearest rounding mode and no fp status flag access. (gcc has -frounding-math and then it does not assume nearest rounding mode, but it still assumes the compiled code itself does not change the mode. Even if the C99 mechanism was implemented it is not ideal: it requires all code in the library to be compiled with FENV_ACCESS "on" to make it usable in non-nearest rounding mode, but that limits optimizations more than necessary.) The math functions should give reasonable results in all rounding modes (but the quality may be degraded in non-nearest rounding modes) and the fp status flag settings should follow the spec, so fenv side-effects are important and code transformations that break them should be prevented. Unfortunately compilers don't give any help with this, the best we can do is to add fp barriers to the code using volatile local variables (they create a stack frame and undesirable memory accesses to it) or inline asm (gcc specific, requires target specific fp reg constraints, often creates unnecessary reg moves and multiple barriers are needed to express that an operation has side-effects) or extern call (only useful in tail-call position to avoid stack-frame creation and does not work with lto). We assume that in a math function if an operation depends on the input and the output depends on it, then the operation will be evaluated at runtime when the function is called, producing all the expected fenv side-effects (this is not true in case of lto and in case the operation is evaluated with excess precision that is not rounded away). So fp barriers are needed (1) to prevent the move of an operation within a function (in case it may be moved from an unevaluated code path into an evaluated one or if it may be moved across a fenv access), (2) force the evaluation of an operation for its side-effect when it has no input dependency (may be constant folded) or (3) when its output is unused. I belive that fp_barrier and fp_force_eval can take care of these and they should not be needed in hot code paths.
generated by cgit v1.2.1 (git 2.18.0) at 2024-04-26 06:23:46 +0000