In an application to be run on Linux I want every output character
to be flushed immediately through stdout onto the screen.

Currently I have a couple explicit fflush() calls in my C program.
I wanted to avoid these calls and configure the output channel on
OS level to immediately output any arriving character automatically.

It seems I need something like the tcsetattr(3) 'FLUSHO', but the
man page says "FLUSHO (not in POSIX; not supported under Linux)".

What would then be the way to achieve that on Linux (if possible)?

Janis

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
> In an application to be run on Linux I want every output character
> to be flushed immediately through stdout onto the screen.
>
> Currently I have a couple explicit fflush() calls in my C program.
> I wanted to avoid these calls and configure the output channel on
> OS level to immediately output any arriving character automatically.
>
> It seems I need something like the tcsetattr(3) 'FLUSHO', but the
> man page says "FLUSHO (not in POSIX; not supported under Linux)".
>
> What would then be the way to achieve that on Linux (if possible)?

setvbuf(3) if you want to use stdio. But if you don't want any
buffering, you can also just use write(2) directly.

In article <uojh53$8bse$1@dont-email.me>,
Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>In an application to be run on Linux I want every output character
>to be flushed immediately through stdout onto the screen.
>
>Currently I have a couple explicit fflush() calls in my C program.
>I wanted to avoid these calls and configure the output channel on
>OS level to immediately output any arriving character automatically.
>
>It seems I need something like the tcsetattr(3) 'FLUSHO', but the
>man page says "FLUSHO (not in POSIX; not supported under Linux)".
>
>What would then be the way to achieve that on Linux (if possible)?
>
>Janis

#include <stdio.h>
#include <unistd.h>

int main(int argc,char **argv)
{ setvbuf(stdout,NULL,_IONBF,0);
printf("This is a test...\t");
sleep(10);
puts("Done!");
}

--
Christianity is not a religion.

- Rick C Hodgin -

On Sun, 21 Jan 2024 17:35:46 +0100, Janis Papanagnou wrote:

> In an application to be run on Linux I want every output character to be
> flushed immediately through stdout onto the screen.

Don’t want buffering, don’t use the buffered interface, then. Use write(2)
calls directly to FD 1, or STDOUT_FILENO, if you want to be symbolic about
it.

On 1/21/24 11:35, Janis Papanagnou wrote:
> In an application to be run on Linux I want every output character
> to be flushed immediately through stdout onto the screen.
>
> Currently I have a couple explicit fflush() calls in my C program.
> I wanted to avoid these calls and configure the output channel on
> OS level to immediately output any arriving character automatically.
>
> It seems I need something like the tcsetattr(3) 'FLUSHO', but the
> man page says "FLUSHO (not in POSIX; not supported under Linux)".
>
> What would then be the way to achieve that on Linux (if possible)?

Would setvbuf(stdout, _IONBF, null) do what you want?

On 21.01.2024 17:35, Janis Papanagnou wrote:
> In an application to be run on Linux I want every output character
> to be flushed immediately through stdout onto the screen.
>
> Currently I have a couple explicit fflush() calls in my C program.
> I wanted to avoid these calls and configure the output channel on
> OS level to immediately output any arriving character automatically.
>
> [...]

The task was delayed, so I'm coming back to this a bit late...

I used a slightly modified test program based on Kenny's code
that outputs single characters with a sleep delay, both with
write(2) and setvbuf(3). I also switched buffering from mode
to mode, to see the effects.
Both suggestions in this thread work well.[*] - Thank you!

There's one point, though, that unsettles me a bit. The setvbuf()
man page says:

"The setvbuf() function may only be used after opening a stream
and before any other operations have been performed on it."

If taken literally, the consequence would be that we can use that
function just once at the beginning of a program (and never again).
Especially switching modes would not be allowed or be [reliably]
possible. That would greatly restrict it's usefulness beyond the
initial (and then persistent) definition of the channel buffering
(unless closing and then reopening the channel, in my case stdout,
which is not exactly what I'd strive for).

In my code I've experimented; I made a couple initial outputs on
stdout, made a fflush() to serve my paranoia, and then switched to
unbuffered output. I also switched forth and back from unbuffered
to line-buffered output in a loop. - It *seems* to work at least.

Any experiences with the restriction documented in the man page,
or opinions and suggestions on the mode-switching I intend to do?

Janis

[*] With write I hadn't been sure whether there's another hidden
stream buffer involved that would have to be configured. But no,
a single character is output, that's fine.

On 2024-02-09, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
> There's one point, though, that unsettles me a bit. The setvbuf()
> man page says:
>
> "The setvbuf() function may only be used after opening a stream
> and before any other operations have been performed on it."

As of C99, the freopen function allows the path argument to be null.

You should be able to do this:

FILE *new_stream = freopen(NULL, "r", old_stream);

After this the old_stream object is toast. The new_stream object
is open to the same file/device.

Since no operations have been done on new_stream at this point,
we should be able to change the buffering mode to whatever
we want.

The downside is that we have to replace all copies of the old_stream
pointer with new_stream. (Easly done if we have some context structure
that holds the stream, where only that context structure is shared).

> In my code I've experimented; I made a couple initial outputs on
> stdout, made a fflush() to serve my paranoia, and then switched to
> unbuffered output. I also switched forth and back from unbuffered
> to line-buffered output in a loop. - It *seems* to work at least.

I would say all bets are off. Even if it works, there could be a memory
leak. The requirements allow a FILE stream to lazily allocate the
buffer. E.g. there can be a null pointer there, and when the first
operation notices this, it mallocs a buffer. Thus setvbuf can just
blindly store a pointer, without trying to free anything, such that it
results in a memory leak if it is used after a buffer is allocated.

To test that we can write a loop which repeatedly opens a stream,
performs some I/O, and then changes the buffer and closes,
monitoring the program for leaks.

> Any experiences with the restriction documented in the man page,
> or opinions and suggestions on the mode-switching I intend to do?

If it's a Linux man page, do not trust that. Linux man pages
are second-hand information cribbed from multiple sources.

Note that Linux systems have more than one C library, yet
those pages from the Linux man page project are the same!

Sometimes it's just copy and paste or paraphrasing from POSIX
or ISO C.

If you're on glibc, the documentation for setvbuf is this:

https://sourceware.org/glibc/manual/html_mono/libc.html#Controlling-Buffering

It doesn't mention anything about this issue, which means it remains
undefined behavior (i.e. glibc isn't providing a documented extension
whereby buffering can be changed after other operations have been done).

glibc is actually implementing POSIX, but regarding setvbuf, POSIX
defers all the requirements to ISO C. It carries forward the
restriction.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
> On 21.01.2024 17:35, Janis Papanagnou wrote:
>> In an application to be run on Linux I want every output character
>> to be flushed immediately through stdout onto the screen.
>>
>> Currently I have a couple explicit fflush() calls in my C program.
>> I wanted to avoid these calls and configure the output channel on
>> OS level to immediately output any arriving character automatically.
>>
>> [...]
>

[...]

> I used a slightly modified test program based on Kenny's code
> that outputs single characters with a sleep delay, both with
> write(2) and setvbuf(3). I also switched buffering from mode
> to mode, to see the effects.
> Both suggestions in this thread work well.[*] - Thank you!
>
> There's one point, though, that unsettles me a bit. The setvbuf()
> man page says:
>
> "The setvbuf() function may only be used after opening a stream
> and before any other operations have been performed on it."
>
> If taken literally, the consequence would be that we can use that
> function just once at the beginning of a program (and never again).
> Especially switching modes would not be allowed or be [reliably]
> possible.

IMHO, using stdio for unbuffered output is even more insane than using
stdio at all (the locking model forced into it is downright
horrible). Even if you want buffered single character/ small string
output, this can be implemented on stop of write in a sane way with very
little code. And the easiest to do unbuffered output is to make the
corresponding system calls instead of using stdio as
proxy-system-caller.

On 2024-02-09, Kaz Kylheku <433-929-6894@kylheku.com> wrote:
> On 2024-02-09, Janis Papanagnou <janis_papanagnou+ng@hotmail.com> wrote:
>> There's one point, though, that unsettles me a bit. The setvbuf()
>> man page says:
>>
>> "The setvbuf() function may only be used after opening a stream
>> and before any other operations have been performed on it."
>
> As of C99, the freopen function allows the path argument to be null.
>
> You should be able to do this:
>
> FILE *new_stream = freopen(NULL, "r", old_stream);

Unfortunately, freopen(NULL, ...) is kind of buggered.

If old_stream is connected to a file, it will reopen the file, which
means that the stream is repositioned at the beginning. Whereas if the
stream is a pipe or socket, then repositioning won't take place.

I'm playing with the function on glibc, and seeing the following.

If you use the "w" mode on a file stream, the file is reopened, and the
truncation implied by "w" takes place!

In other words, this sequence of operations:

FILE *f = fopen("foo", "w");
fputs("abcdefg\n", f);
fflush(f);

f = freopen(NULL, "w", f);
fputs("xyz\n", f);

fclose(f);

will end with "foo" containing just the line "xyz\n".

The "abcdefg\n" line is lost because because the freopen
truncates the file!

freopen is not a good solution if all you want is to fiddle with
buffering settings without upsetting anything else, or even moving the
read/write position.

Maybe it's better behaved for pipes and such, which cannot be truncated
or change position. But it's not clear whether it's supported at all for
something that was the result of fdopen on a pipe or a popen.

Probably about all freopen is good for is changing stdin or stdout from
binary to text mode at close to program startup. (Since there is no API
for doing that like for doing buffering)

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

On 09.02.2024 20:02, Rainer Weikusat wrote:
>
> IMHO, using stdio for unbuffered output is even more insane than using
> stdio at all (the locking model forced into it is downright
> horrible). Even if you want buffered single character/ small string
> output, this can be implemented on stop of write in a sane way with very
> little code. And the easiest to do unbuffered output is to make the
> corresponding system calls instead of using stdio as
> proxy-system-caller.

I have to admit that I'm a bit confused about your statement since
in your initial reply you proposed both, setvbuf(3) and write(2).
This is somewhat in contrast to the strong wording "insane" you used
here. (I mean, the stdio designers must have had some idea in mind
when they invented 'setvbuf()' with '_IONBF'.)

But okay, here you are suggesting to avoid stdio, which effectively
means that I replace, for example, all printf(...) formatted output
by a sequence of sprintf(buf,...); write(...,buf,...) and similarly
also replace all fputs() in favour of 'write' (and certainly also not
to mix them), i.e. to stay with stdio for formatting only, and for
the plain output process to use only system calls. Right? - This way
issues like the one I quoted from the man page[*], which can be
considered being a "proxy"-effect, could be prevented. Makes sense.

Janis

[*] "The setvbuf() function may only be used after opening a stream
and before any other operations have been performed on it."

Janis Papanagnou <janis_papanagnou+ng@hotmail.com> writes:
> On 09.02.2024 20:02, Rainer Weikusat wrote:
>>
>> IMHO, using stdio for unbuffered output is even more insane than using
>> stdio at all (the locking model forced into it is downright
>> horrible). Even if you want buffered single character/ small string
>> output, this can be implemented on stop of write in a sane way with very
>> little code. And the easiest to do unbuffered output is to make the
>> corresponding system calls instead of using stdio as
>> proxy-system-caller.
>
> I have to admit that I'm a bit confused about your statement since
> in your initial reply you proposed both, setvbuf(3) and write(2).
> This is somewhat in contrast to the strong wording "insane" you used
> here. (I mean, the stdio designers must have had some idea in mind
> when they invented 'setvbuf()' with '_IONBF'.)

stdio is part of C, write and read are UNIX interfaces. Hence, code
using stdio is portable to systems which don't support UNIX interfaces.

The 'insane' is my personal opinion on the thread-safe locking model
stdio is nowadays required to follow on UNIX. That's based on the
assumption that all stdio-streams will be used uncoordinately from
different threads and that because of this, every stdio-operation has to
acquire a lock on the stream before doing anyhing. People who don't use
stdio in this way may prefix _unlocked to each stdio-call their code is
making.

IMHO, it would have been much more sensible to declare that streams are
not thread-safe and that application need to apply whatever locking
suits their needs when sharing them between threads.

On 2024-02-11, Rainer Weikusat <rweikusat@talktalk.net> wrote:
> The 'insane' is my personal opinion on the thread-safe locking model
> stdio is nowadays required to follow on UNIX.

On Glibc/Linux, a single-threaded process that isn't linked with
-lpthread doesn't suffer from the locking. It is magically switched
on when you link in the threading library.