From 75cba9c67fde03421b96c1bcbaf666b4b348739d Mon Sep 17 00:00:00 2001 From: Rich Felker Date: Sun, 11 Feb 2018 20:48:14 -0500 Subject: fix incorrect overflow check for allocation in fmemopen when a null buffer pointer is passed to fmemopen, requesting it allocate its own memory buffer, extremely large size arguments near SIZE_MAX could overflow and result in underallocation. this results from omission of the size of the cookie structure in the overflow check but inclusion of it in the calloc call. instead of accounting for individual small contributions to the total allocation size needed, simply reject sizes larger than PTRDIFF_MAX, which will necessarily fail anyway. then adding arbitrary fixed-size structures is safe without matching up the expressions in the comparison and the allocation. --- src/stdio/fmemopen.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/src/stdio/fmemopen.c b/src/stdio/fmemopen.c index 7c193a57..2ce43d32 100644 --- a/src/stdio/fmemopen.c +++ b/src/stdio/fmemopen.c @@ -81,7 +81,7 @@ FILE *fmemopen(void *restrict buf, size_t size, const char *restrict mode) return 0; } - if (!buf && size > SIZE_MAX-sizeof(FILE)-BUFSIZ-UNGET) { + if (!buf && size > PTRDIFF_MAX) { errno = ENOMEM; return 0; } -- cgit v1.2.1