Arch Linux

Compiling with -O2 appears to generate code which mangles the return value from a gnulib function, causing the overwrite prompt to ignore 'n'.

To reproduce:
$ echo donttaze me >out
$ gzip out
$ echo overwritten >out
$ gzip out

Gzip will prompt you...
gzip: out.gz already exists; do you wish to overwrite (y or n)?

But responding with 'n' will still overwrite the file:
$ gzip -cd out.gz
overwritten

When compiling with -O0, this doesn't occur, and responding 'n' causes gzip to reply with 'not overwritten'.

gdb shows the return value from the yesno() function being mangled when called from check_ofname():

(gdb) break /mnt/Gluttony/build/gzip/src/gzip-1.5/lib/yesno.c:59
Breakpoint 1 at 0x40d2de: file yesno.c, line 59.
(gdb) run
Starting program: /usr/bin/gzip out
warning: Could not load shared library symbols for linux-vdso.so.1.
Do you need "set solib-search-path" or "set sysroot"?
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
gzip: out.gz already exists; do you wish to overwrite (y or n)? n

Breakpoint 1, yesno () at yesno.c:60
60 }
(gdb) l
55 while (c != '\n' && c != EOF)
56 c = getchar ();
57 #endif
58
59 return yes;
60 }
(gdb) print (int)yes
$1 = 0 <---- this is fine!
(gdb) step
check_ofname () at gzip.c:1674
1674 if (!ok) {
(gdb) print ok
$2 = -134367744 <---- what the fuck?
(gdb) l
1669 if (foreground && (presume_input_tty || isatty(fileno(stdin)))) {
1670 fprintf(stderr, " do you wish to overwrite (y or n)? ");
1671 fflush(stderr);
1672 ok = yesno(); <-- this is where ok is assigned from yesno()
1673 }
1674 if (!ok) { <--- and the breakpoint where $3 comes from
1675 fprintf(stderr, "\tnot overwritten\n");
1676 if (exit_code == OK) exit_code = WARNING;
1677 return ERROR;
1678 }
(gdb)

I notice via disassembly that yesno() is never inlined at either optimization level, and that check_ofname() is inlined at both optimization levels. I'm attaching the full disassembly at -O2 and -O0 for reference.

I think I've been able to rule out any wrongdoing in the yesno() function, but I'll mention 2 points of interest:

1) The bizzare while() loop you see at the top of the gdb output (looping until newline or EOF) does seem to have some effect on this. That is, removing while loop even with optimizations will result in the correct behavior. It does not result in yesno() being inlined (I suppose because it remains in a separate translation unit and we're not using LTO).
2) Replacing getchar() with the builtin _IO_getc(stdin) has no effect.