| Age | Commit message (Collapse) | Author | Lines |
|---|
|
In all cases this is just a change from two volatile int to one.
|
|
notes added by maintainer:
this function is a GNU extension. it was chosen over the similar BSD
function funopen because the latter depends on fpos_t being an
arithmetic type as part of its public API, conflicting with our
definition of fpos_t and with the intent that it be an opaque type. it
was accepted for inclusion because, despite not being widely used, it
is usually very difficult to extricate software using it from the
dependency on it.
calling pattern for the read and write callbacks is not likely to
match glibc or other implementations, but should work with any
reasonable callbacks. in particular the read function is never called
without at least one byte being needed to satisfy its caller, so that
spurious blocking is not introduced.
contracts for what callbacks called from inside libc/stdio can do are
always complicated, and at some point still need to be specified
explicitly. at the very least, the callbacks must return or block
indefinitely (they cannot perform nonlocal exits) and they should not
make calls to stdio using their own FILE as an argument.
|
|
previously, fgetwc left all but the first byte of an illegal sequence
unread (available for subsequent calls) when reading out of the FILE
buffer, but dropped all bytes contibuting to the error when falling
back to reading a byte at a time. neither behavior was ideal. in the
buffered case, each malformed character produced one error per byte,
rather than one per character. in the unbuffered case, consuming the
last byte that caused the transition from "incomplete" to "invalid"
state potentially dropped (and produced additional spurious encoding
errors for) the next valid character.
to handle both cases uniformly without duplicate code, revise the
buffered case to only cover situations where a complete and valid
character is present in the buffer, and fall back to byte-at-a-time
for all other cases. this allows using mbtowc (stateless) instead of
mbrtowc, which may slightly improve performance too.
when an encoding error has been hit in the byte-at-a-time case, leave
the final byte that produced the error unread (via ungetc) except in
the case of single-byte errors (for UTF-8, bytes c0, c1, f5-ff, and
continuation bytes with no lead byte). single-byte errors are fully
consumed so as not to leave the caller in an infinite loop repeating
the same error.
none of these changes are distinguished from a conformance standpoint,
since the file position is unspecified after encoding errors. they are
intended merely as QoI/consistency improvements.
|
|
fgetwc does not set the stream's error indicator on encoding errors,
making ferror insufficient to distinguish between error and eof
conditions. feof is also insufficient, since it will return true if
the file ended with a partial character encoding error.
whether fgetwc should be setting the error indicator itself is a
question with conflicting answers. the POSIX text for the function
states it as a requirement, but the ISO C text seems to require that
it not. this may be revisited in the future based on the outcome of
Austin Group issue #1170.
|
|
Update the buffer position according to the bytes consumed into st when
decoding an incomplete character at the end of the buffer.
|
|
this is mandated by C and POSIX standards and is in accordance with
glibc behavior.
|
|
commit c002668eb0352e619ea7064e4940b397b4a6e68d inadvertently moved
the check for unflushed write buffer outside of the scope of the
existing lock.
|
|
The switch statement has no 'default:' case and the function ends
immediately following the switch, so the extra comparison did not
communicate any extra information to the compiler.
|
|
commit 58e2396a9aa23c132faf4198ca4d779c84955b38 missed that the same
code was duplicated in implementation of vfwprintf.
|
|
the code being removed was written to optimize for size assuming the
compiler cannot collapse code paths for different types with the same
underlying representation. modern compilers sometimes succeed in
making this optimization themselves, but either way it's a small size
difference and not worth the source-level complexity or the UB
involved in this hack.
some incorrect use of va_arg still remains, particularly use of void *
where the actual argument has a different pointer type. fixing this
requires some actual code additions, rather than just removing cruft,
so I'm leaving it to be done later as a separate commit.
|
|
|
|
the swprintf write callback never reset its buffer pointers, so after
its 256-byte buffer filled up, it would keep repeating those bytes
over and over in the output until the destination buffer filled up. it
also failed to set the error indicator for the stream on EILSEQ,
potentially allowing output to continue after the error.
|
|
the old snprintf design setup the FILE buffer pointers to point
directly into the destination buffer; if n was actually larger than
the buffer size, the pointer arithmetic to compute the buffer end
pointer was undefined. this affected sprintf, which is implemented in
terms of snprintf, as well as some unusual but valid direct uses of
snprintf.
instead, setup the FILE as unbuffered and have its write function
memcpy to the destination. the printf core sets up its own temporary
buffer for unbuffered streams.
|
|
in nearest rounding mode exact halfway cases were not following the
round to even rule if the rounding happened at a base 1000000000 digit
boundary of the internal representation and the previous digit was odd.
e.g. printf("%.0f", 1.5) printed 1 instead of 2.
|
|
this patch fixes a large number of missed internal signed-overflow
checks and errors in determining when the return value (output length)
would exceed INT_MAX, which should result in EOVERFLOW. some of the
issues fixed were reported by Alexander Cherepanov; others were found
in subsequent review of the code.
aside from the signed overflows being undefined behavior, the
following specific bugs were found to exist in practice:
- overflows computing length of floating point formats with huge
explicit precisions, integer formats with prefix characters and huge
explicit precisions, or string arguments or format strings longer
than INT_MAX, resulted in wrong return value and wrong %n results.
- literal width and precision values outside the range of int were
misinterpreted, yielding wrong behavior in at least one well-defined
case: string formats with precision greater than INT_MAX were
sometimes truncated.
- in cases where EOVERFLOW is produced, incorrect values could be
written for %n specifiers past the point of exceeding INT_MAX.
in addition to fixing these bugs, we now stop producing output
immediately when output length would exceed INT_MAX, rather than
continuing and returning an error only at the end.
|
|
if the requested precision is close to INT_MAX, adding
LDBL_MANT_DIG/3+8 overflows. in practice the resulting undefined
behavior manifests as a large negative result, which is then used to
compute the new end pointer (z) with a wildly out-of-bounds value
(more overflow, more undefined behavior). the end result is at least
incorrect output and character count (return value); worse things do
not seem to happen, but detailed analysis has not been done.
this patch fixes the overflow by performing the intermediate
computation as unsigned; after division by 9, the final result
necessarily fits in int.
|
|
previously, fflush_unlocked was an alias for an internal backend that
was called by fflush, either for its argument or in a loop for each
file if a null pointer was passed. since the logic for the latter was
in the main fflush function, fflush_unlocked crashed when passed a
null pointer, rather than flushing all open files. since
fflush_unlocked is not a standard function and has no specification,
it's not clear whether it should be expected to accept null pointers
like fflush does, but a reasonable argument could be made that it
should.
this patch eliminates the helper function, simplifying fflush, and
makes fflush_unlocked an alias for fflush, which is valid because the
two functions agree in their behavior in all cases where their
behavior is defined (the unlocked version has undefined behavior if
another thread could hold locks).
|
|
commit b91cdbe2bc8b626aa04dc6e3e84345accf34e4b1, in fixing another
issue, changed the logic for how alt-form octal adds the leading zero
to adjust the precision rather than using a prefix character. this
wrongly suppressed the zero flag by mimicing an explicit precision
given by the format string. switch back to using a prefix character.
based on bug report and patch by Dmitry V. Levin, but simplified.
|
|
commit 7e816a6487932cbb3cb71d94b609e50e81f4e5bf (version 1.1.11
release cycle) moved the code that performs wchar_t to multibyte
conversion across code that used the resulting length in bytes,
thereby breaking the unget buffer space check in ungetwc and
clobbering up to three bytes below the start of the buffer.
for allocated FILEs (all read-enabled FILEs except stdin), the
underflow clobbers at most the FILE-specific locale pointer. no stores
are performed through this pointer, but subsequent loads may result in
a crash or mismatching encoding rule (UTF-8 multibyte vs byte-based).
for stdin, the buffer lies in .bss and the underflow may clobber
another object. in practice, for libc.so the adjacent object seems to
be stderr's buffer, which is completely unused, but this could vary
with linking options, or when static linking.
applications which do not attempt to use more than one character of
ungetwc pushback, or which do not use ungetwc, are not affected.
|
|
the comparison f->wpos > f->buf has undefined behavior when f->wpos is
a null pointer, despite the intuition (and actual compiler behavior,
for all known compilers) being that NULL > ptr is false for all valid
pointers ptr.
the purpose of the comparison is to determine if the write buffer is
non-empty, and the idiom used elsewhere for that is comparison against
f->wbase, which is either a null pointer when not writing, or equal to
f->buf when writing. in the former case, both f->wpos and f->wbase are
null; in the latter they are both non-null and point into the same
array.
|
|
the idiom fprintf(f, "%.*s", n, "") was wrongly used in vfwprintf as a
means of producing n spaces; instead it produces no output. the
correct form is fprintf(f, "%*s", n, ""), using width instead of
precision, since for %s the later is a maximum rather than a minimum.
|
|
internally, the idiom of passing nmemb=1 to fwrite and interpreting
the return value of fwrite (which is necessarily 0 or 1) as
failure/success is fairly widely used. this is not correct, however,
when the size argument is unknown and may be zero, since C requires
fwrite to return 0 in that special case. previously fwrite always
returned nmemb on success, but this was changed for conformance with
ISO C by commit 500c6886c654fd45e4926990fee2c61d816be197.
|
|
when the size argument was zero but nmemb was nonzero, these functions
were returning nmemb, despite no data having been written.
conceptually this is not wrong, but the standard requires a return
value of zero in this case.
|
|
when a write error occurred while flushing output due to a newline,
fwrite falsely reported all bytes up to and including the newline as
successfully written. in general, due to buffering such "spurious
success" returns are acceptable for stdio; however for line-buffered
mode it was subtly wrong. errors were still visible via ferror() or as
a short-write return if there was more data past the newline that
should have been written, but since the contract for line-buffered
mode is that everything up through the newline be written out
immediately, a discrepency was observable in the actual file contents.
|
|
previously, getdelim was allocating twice the space needed every time
it expanded its buffer to implement exponential buffer growth (in
order to avoid quadratic run time). however, this doubling was
performed even when the final buffer length needed was already known,
which is the common case that occurs whenever the delimiter is in the
FILE's buffer.
this patch makes two changes to remedy the situation:
1. over-allocation is no longer performed if the delimiter has already
been found when realloc is needed.
2. growth factor is reduced from 2x to 1.5x to reduce the relative
excess allocation in cases where the delimiter is not initially in the
buffer, including unbuffered streams.
in theory these changes could lead to quadratic time if the same
buffer is reused to process a sequence of lines successively
increasing in length, but once this length exceeds the stdio buffer
size, the delimiter will not be found in the buffer right away and
exponential growth will still kick in.
|
|
getdelim was updating *n, the caller's stored buffer size, before
calling realloc. if getdelim then failed due to realloc failure, the
caller would see in *n a value larger than the actual size of the
allocated block, and use of that value is unsafe. in particular,
passing it again to getdelim is unsafe.
now, temporary storage is used for the desired new size, and *n is not
written until realloc succeeds.
|
|
the buffer enlargement logic here accounted for the terminating null
byte, but not for the possibility of hitting the delimiter in the
buffer-refill code path that uses getc_unlocked, in which case two
additional bytes (the delimiter and the null termination) are written
without another chance to enlarge the buffer.
this patch and the corresponding bug report are by Felix Janda.
|
|
the specification for these functions requires that the buffer/size
exposed to the caller be valid after any successful call to fflush or
fclose on the stream. the implementation's approach is to update them
only at flush time, but that misses the case where fflush or fclose is
called without any writes having taken place, in which case the write
flushing callback will not be called.
to fix both the observable bug and the desired invariant, setup empty
buffers at open time and fail the open operation if no memory is
available.
|
|
this fixes a bug reported by Nuno Gonçalves. previously, calling
fclose on stdin or stdout resulted in deadlock at exit time, since
__stdio_exit attempts to lock these streams to flush/seek them, and
has no easy way of knowing that they were closed.
conceptually, leaving a FILE stream locked on fclose is valid since,
in the abstract machine, it ceases to exist. but to satisfy the
implementation-internal assumption in __stdio_exit that it can access
these streams unconditionally, we need to unlock them.
it's also necessary that fclose leaves permanent streams in a state
where __stdio_exit will not attempt any further operations on them.
fortunately, the call to fflush already yields this property.
|
|
tempnam uses an uninitialized buffer which is filled using memcpy and
__randname. It is therefore necessary to explicitly null-terminate it.
based on patch by Felix Janda.
|
|
functions which open in-memory FILE stream variants all shared a tail
with __fdopen, adding the FILE structure to stdio's open file list.
replacing this common tail with a function call reduces code size and
duplication of logic. the list is also partially encapsulated now.
function signatures were chosen to facilitate tail call optimization
and reduce the need for additional accessor functions.
with these changes, static linked programs that do not use stdio no
longer have an open file list at all.
|
|
this patch adjusts libc components which use the multibyte functions
internally, and which depend on them operating in a particular
encoding, to make the appropriate locale changes before calling them
and restore the calling thread's locale afterwards. activating the
byte-based C locale without these changes would cause regressions in
stdio and iconv.
in the case of iconv, the current implementation was simply using the
multibyte functions as UTF-8 conversions. setting a multibyte UTF-8
locale for the duration of the iconv operation allows the code to
continue working.
in the case of stdio, POSIX requires that FILE streams have an
encoding rule bound at the time of setting wide orientation. as long
as all locales, including the C locale, used the same encoding,
treating high bytes as UTF-8, there was no need to store an encoding
rule as part of the stream's state.
a new locale field in the FILE structure points to the locale that
should be made active during fgetwc/fputwc/ungetwc on the stream. it
cannot point to the locale active at the time the stream becomes
oriented, because this locale could be mutable (the global locale) or
could be destroyed (locale_t objects produced by newlocale) before the
stream is closed. instead, a pointer to the static C or C.UTF-8 locale
object added in commit commit aeeac9ca5490d7d90fe061ab72da446c01ddf746
is used. this is valid since categories other than LC_CTYPE will not
affect these functions.
|
|
commit 58165923890865a6ac042fafce13f440ee986fd9 added these optional
cancellation points on the basis that cancellable stdio could be
useful, to unblock threads stuck on stdio operations that will never
complete. however, the only way to ensure that cancellation can
achieve this is to violate the rules for side effects when
cancellation is acted upon, discarding knowledge of any partial data
transfer already completed. our implementation exhibited this behavior
and was thus non-conforming.
in addition to improving correctness, removing these cancellation
points moderately reduces code size, and should significantly improve
performance on i386, where sysenter/syscall instructions can be used
instead of "int $128" for non-cancellable syscalls.
|
|
the old idiom, f->mode |= f->mode+1, was adapted from the idiom for
setting byte orientation, f->mode |= f->mode-1, but the adaptation was
incorrect. unless the stream was alreasdy set byte-oriented, this code
incremented f->mode each time it was executed, which would eventually
lead to overflow. it could be fixed by changing it to f->mode |= 1,
but upcoming changes will require slightly more work at the time of
wide orientation, so it makes sense to just call fwide. as an
optimization in the single-character functions, fwide is only called
if the stream is not already wide-oriented.
|
|
this is undefined, but supported in our implementation of the normal
printf, so for consistency the wide variant should support it too.
|
|
|
|
|
|
aside from being invalid, the early check only optimized the error
case, and likely pessimized the common case by separating the
two branches on isascii(c) at opposite ends of the function.
|
|
these functions were written to handle clearing eof status, but failed
to account for the __toread function's handling of eof. with this
patch applied, __toread still returns EOF when the file is in eof
status, so that read operations will fail, but it also sets up valid
buffer pointers for read mode, which are set to the end of the buffer
rather than the beginning in order to make the whole buffer available
to ungetc/ungetwc.
minor changes to __uflow were needed since it's now possible to have
non-zero buffer pointers while in eof status. as made, these changes
remove a 'fast path' bypassing the function call to __toread, which
could be reintroduced with slightly different logic, but since
ordinary files have a syscall in f->read, optimizing the code path
does not seem worthwhile.
the __stdio_read function is also updated not to zero the read buffer
pointers on eof/error. while not necessary for correctness, this
change avoids the overhead of calling __toread in ungetc after
reaching eof, and it also reduces code size and increases consistency
with the fmemopen read operation which does not zero the pointers.
|
|
|
|
the equivalent checks for newly opened stdio output streams, used to
determine buffering mode, are also fixed.
on most archs, the TCGETS ioctl command shares a value with
SNDCTL_TMR_TIMEBASE, part of the OSS sound API which was apparently
used with certain MIDI and timer devices. for file descriptors
referring to such a device, TCGETS will not fail with ENOTTY as
expected; it may produce a different error, or may succeed, and if it
succeeds it changes the mode of the device. while it's unlikely that
such devices are in use, this is in principle very harmful behavior
for an operation which is supposed to do nothing but query whether the
fd refers to a tty.
TIOCGWINSZ, used to query logical window size for a terminal, was
chosen as an alternate ioctl to perform the isatty check. it does not
share a value with any other ioctl commands, and it succeeds on any
tty device.
this change also cleans up strace output to be less ugly and
misleading.
|
|
previously, aio operations were not tracked by file descriptor; each
operation was completely independent. this resulted in non-conforming
behavior for non-seekable/append-mode writes (which are required to be
ordered) and made it impossible to implement aio_cancel, which in turn
made closing file descriptors with outstanding aio operations unsafe.
the new implementation is significantly heavier (roughly twice the
size, and seems to be slightly slower) and presently aims mainly at
correctness, not performance.
most of the public interfaces have been moved into a single file,
aio.c, because there is little benefit to be had from splitting them.
whenever any aio functions are used, aio_cancel and the internal
queue lifetime management and fd-to-queue mapping code must be linked,
and these functions make up the bulk of the code size.
the close function's interaction with aio is implemented with weak
alias magic, to avoid pulling in heavy aio cancellation code in
programs that don't use aio, and the expensive cancellation path
(which includes signal blocking) is optimized out when there are no
active aio queues.
|
|
formally, it seems a sign is only required when the '+' modifier
appears in the format specifier, in which case either '+' or '-' must
be present in the output. but the specification is written such that
an optional negative sign is part of the output format anyway, and the
simplest approach to fixing the problem is removing the code that was
suppressing the sign.
|
|
previously, write errors neither stopped further output attempts nor
caused the function to return an error to the caller. this could
result in silent loss of output, possibly in the middle of output in
the event of a non-permanent error.
the simplest solution is temporarily clearing the error flag for the
target stream, then suppressing further output when the error flag is
set and checking/restoring it at the end of the operation to determine
the correct return value.
since the wide version of the code internally calls the narrow fprintf
to perform some of its underlying operations, initial clearing of the
error flag is suppressed when performing a narrow vfprintf on a
wide-oriented stream. this is not a problem since the behavior of
narrow operations on wide-oriented streams is undefined.
|
|
in this case there are two conflicting rules in play: that an explicit
precision of zero with the value zero produces no output, and that the
'#' modifier for octal increases the precision sufficiently to yield a
leading zero. ISO C (7.19.6.1 paragraph 6 in C99+TC3) includes a
parenthetical remark to clarify that the precision-increasing behavior
takes precedence, but the corresponding text in POSIX off of which I
based the implementation is missing this remark.
this issue was covered in WG14 DR#151.
|
|
commit 5345c9b884e7c4e73eb2c8bb83b8d0df20f95afb added a linked list to
track the FILE streams currently locked (via flockfile) by a thread.
due to a failure to fully link newly added members, removal from the
list could leave behind references which could later result in writes
to already-freed memory and possibly other memory corruption.
implicit stdio locking was unaffected; the list is only used in
conjunction with explicit flockfile locking.
this bug was not present in any releases; it was introduced and fixed
during the same release cycle.
patch by Timo Teräs, who discovered and tracked down the bug.
|
|
previously, fgets, fputs, fread, and fwrite completely omitted locking
and access to the FILE object when their arguments yielded a zero
length read or write operation independent of the FILE state. this
optimization was invalid; it wrongly skipped marking the stream as
byte-oriented (a C conformance bug) and exposed observably missing
synchronization (a POSIX conformance bug) where one of these functions
could wrongly complete despite another thread provably holding the
lock.
|
|
the C standard requires that "the contents of the array remain
unchanged" in this case.
this patch also changes the behavior on read errors, but in that case
"the array contents are indeterminate", so the application cannot
inspect them anyway.
|
|
this is analogous commit fffc5cda10e0c5c910b40f7be0d4fa4e15bb3f48
which fixed the corresponding issue for mutexes.
the robust list can't be used here because the locks do not share a
common layout with mutexes. at some point it may make sense to simply
incorporate a mutex object into the FILE structure and use it, but
that would be a much more invasive change, and it doesn't mesh well
with the current design that uses a simpler code path for internal
locking and pulls in the recursive-mutex-like code when the flockfile
API is used explicitly.
|
|
previously we detected this bug in configure and issued advice for a
workaround, but this turned out not to work. since then gcc 4.9.0 has
appeared in several distributions, and now 4.9.1 has been released
without a fix despite this being a wrong code generation bug which is
supposed to be a release-blocker, per gcc policy.
since the scope of the bug seems to affect only data objects (rather
than functions) whose definitions are overridable, and there are only
a very small number of these in musl, I am just changing them from
const to volatile for the time being. simply removing the const would
be sufficient to make gcc 4.9.1 work (the non-const case was
inadvertently fixed as part of another change in gcc), and this would
also be sufficient with 4.9.0 if we forced -O0 on the affected files
or on the whole build. however it's cleaner to just remove all the
broken compiler detection and use volatile, which will ensure that
they are never constant-folded. the quality of a non-broken compiler's
output should not be affected except for the fact that these objects
are no longer const and thus possibly add a few bytes to data/bss.
this change can be reconsidered and possibly reverted at some point in
the future when the broken gcc versions are no longer relevant.
|
