I've seen people trying to read files like this in a lot of posts lately.

Code

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char **argv)
{
    char * path = argc > 1 ? argv[1] : "input.txt";

    FILE * fp = fopen(path, "r");
    if( fp == NULL ) {
        perror(path);
        return EXIT_FAILURE;
    }

    while( !feof(fp) ) {  /* THIS IS WRONG */
        /* Read and process data from file… */
    }
    fclose(fp);
    return EXIT_SUCCESS;
}

What is wrong with this while( !feof(fp)) loop?

10 upvote
  flag
Consider splitting your question in 2. One part as the question proper and the other part as an answer; then put that 2nd part down there ... in the answer section of SO :) – pmg
11 upvote
  flag
upvote
  flag
2 upvote
  flag
Short answer: It tries to use a status reporting function to predict whether a future operation will succeed. – David Schwartz

5 Answers 11

No it's not always wrong. If your loop condition is "while we haven't tried to read past end of file" then you use while (!feof(f)). This is however not a common loop condition - usually you want to test for something else (such as "can I read more"). while (!feof(f)) isn't wrong, it's just used wrong.

1 upvote
  flag
I wonder ... f = fopen("A:\\bigfile"); while (!feof(f)) { /* remove diskette */ } or (going to test this) f = fopen(NETWORK_FILE); while (!feof(f)) { /* unplug network cable */ } – pmg
upvote
  flag
@pmg: As said, "not a common loop condition" hehe. I can't really think of any case I've needed it, usually I'm interested in "could I read what I wanted" with all that implies of error handling – Erik
upvote
  flag
@pmg: As said, you rarely want while(!eof(f)) – Erik
5 upvote
  flag
More accurately, the condition is "while we haven't tried to read past the end of the file and there was no read error" feof is not about detecting end of file; it is about determining whether a read was short because of an error or because the input is exhausted. – William Pursell

It's wrong because (in the absence of a read error) it enters the loop one more time than the author expects. If there is a read error, the loop never terminates.

Consider the following code:

/* WARNING: demonstration of bad coding technique*/

#include <stdio.h>
#include <stdlib.h>

FILE *Fopen( const char *path, const char *mode );
int
main( int argc, char **argv )
{
    FILE *in;
    unsigned count;

    in = argc > 1 ? Fopen( argv[ 1 ], "r" ) : stdin;
    count = 0;

    /* WARNING: this is a bug */
    while( !feof( in )) {  /* This is WRONG! */
        (void) fgetc( in );
        count++;
    }
    printf( "Number of characters read: %u\n", count );
    return EXIT_SUCCESS;
}

FILE *
Fopen( const char *path, const char *mode )
{
    FILE *f = fopen( path, mode );
    if( f == NULL ) {
        perror( path );
        exit( EXIT_FAILURE );
    }
    return f;
}

This program will consistently print one greater than the number of characters in the input stream (assuming no read errors). Consider the case where the input stream is empty:

$ ./a.out < /dev/null
Number of characters read: 1

In this case, feof() is called before any data has been read, so it returns false. The loop is entered, fgetc() is called (and returns EOF), and count is incremented. Then feof() is called and returns true, causing the loop to abort.

This happens in all such cases. feof() does not return true until after a read on the stream encounters the end of file. The purpose of feof() is NOT to check if the next read will reach the end of file. The purpose of feof() is to distinguish between a read error and having reached the end of the file. If fread() returns 0, you must use feof/ferror to decide. Similarly if fgetc returns EOF. feof() is only useful after fread has returned zero or fgetc has returned EOF. Before that happens, feof() will always return 0.

It is always necessary to check the return value of a read (either an fread(), or an fscanf(), or an fgetc()) before calling feof().

Even worse, consider the case where a read error occurs. In that case, fgetc() returns EOF, feof() returns false, and the loop never terminates. In all cases where while(!feof(p)) is used, there must be at least a check inside the loop for ferror(), or at the very least the while condition should be replaced with while(!feof(p) && !ferror(p)) or there is a very real possibility of an infinite loop, probably spewing all sorts of garbage as invalid data is being processed.

So, in summary, although I cannot state with certainty that there is never a situation in which it may be semantically correct to write "while(!feof(f))" (although there must be another check inside the loop with a break to avoid a infinite loop on a read error), it is the case that it is almost certainly always wrong. And even if a case ever arose where it would be correct, it is so idiomatically wrong that it would not be the right way to write the code. Anyone seeing that code should immediately hesitate and say, "that's a bug". And possibly slap the author (unless the author is your boss in which case discretion is advised.)

10 upvote
  flag
Mutlitple downvotes today: any explanation? If you disagree, please explain your reasons. – William Pursell
6 upvote
  flag
Sure it's wrong -- but aside from that it isn't "gratingly ugly". – nobar
66 upvote
  flag
You should add an example of correct code, as I imagine lots of people will come here looking for a quick fix. – jleahy
1 upvote
  flag
Is this different from file.eof()? – Thomas
upvote
  flag
@Thomas: I'm not a C++ expert, but I believe file.eof() returns effectively the same result as feof(file) || ferror(file), so it is very different. But this question is not intended to be applicable to C++. – William Pursell
upvote
  flag
I suppose correct code could be: change while(!feof(in)) into do { ... } while (!feof(in)); and add a conditioned break to avoid infinite loop. – m-ric
5 upvote
  flag
@m-ric that's not right either, because you'll still try to process a read that failed. – Mark Ransom
1 upvote
  flag
this is the actual correct answer. feof() is used to know the outcome of previous read attempt. Thus probably you don't want to use it as your loop break condition. +1 – Jack

feof() indicates if one has tried to read past the end of file. That means it has little predictive effect: if it is true, you are sure that the next input operation will fail (you aren't sure the previous one failed BTW), but if it is false, you aren't sure the next input operation will succeed. More over, input operations may fail for other reasons than the end of file (a format error for formatted input, a pure IO failure -- disk failure, network timeout -- for all input kinds), so even if you could be predictive about the end of file (and anybody who has tried to implement Ada one, which is predictive, will tell you it can complex if you need to skip spaces, and that it has undesirable effects on interactive devices -- sometimes forcing the input of the next line before starting the handling of the previous one), you would have to be able to handle a failure.

So the correct idiom in C is to loop with the IO operation success as loop condition, and then test the cause of the failure. For instance:

while (fgets(line, sizeof(line), file)) {
    /* note that fgets don't strip the terminating \n, checking its
       presence allow to handle lines longer that sizeof(line), not showed here */
    ...
}
if (ferror(file)) {
   /* IO failure */
} else if (feof(file)) {
   /* format error (not possible with fgets, but would be with fscanf) or end of file */
} else {
   /* format error (not possible with fgets, but would be with fscanf) */
}
2 upvote
  flag
Getting to the end of a file is not an error, so I question the phrasing "input operations may fail for other reasons than the end of file". – William Pursell
upvote
  flag
@WilliamPursell, reaching the eof isn't necessarily an error, but being unable to do an input operation because of eof is one. And it is impossible in C to detect reliably the eof without having made an input operation fails. – AProgrammer
upvote
  flag
Agree last else not possible with sizeof(line) >= 2 and fgets(line, sizeof(line), file) but possible with pathological size <= 0 and fgets(line, size, file). Maybe even possible with sizeof(line) == 1. – chux
upvote
  flag
All that "predictive value" talk... I never thought about it that way. In my world, feof(f) does not PREDICT anything. It states that a PREVIOUS operation has hit the end of the file. Nothing more, nothing less. And if there was no previous operation (just opened it), it does not report end of file even if the file was empty to start with. So, apart of the concurrency explanation in another answer above, I do not think there is any reason not to loop on feof(f). – BitTickler

Great answer, I just noticed the same thing because I was trying to do a loop like that. So, it's wrong in that scenario, but if you want to have a loop that gracefully ends at the EOF, this is a nice way to do it:

#include <stdio.h>
#include <sys/stat.h>
int main(int argc, char *argv[])
{
  struct stat buf;
  FILE *fp = fopen(argv[0], "r");
  stat(filename, &buf);
  while (ftello(fp) != buf.st_size) {
    (void)fgetc(fp);
  }
  // all done, read all the bytes
}
upvote
  flag
This is an interesting approach, but does not work on a fifo. It doesn't seem to offer any benefit over while( fgetc(fp) != EOF ) – William Pursell
1 upvote
  flag
True, but sometimes you don't use fgetc() to read files. For example, reading structured records, I have a read function (in this example where there's a fgetc) which detects errors and reads exactly one record, but it doesn't know how many records are in the file. Yes, it's wrong for fifos, or any other file that might change while you have it open. – tesch1
upvote
  flag
while( read_structured_record( fp ) == 1 ) { ... would be an idiomatic way to write that (assuming read_structured_record returns the number of records read). – William Pursell
upvote
  flag
File I/O errors may happen anytime, thus by not checking the result of fgetc(fp), one may miss that and not read all the bytes. – chux
upvote
  flag
Files can grow while you are reading them; they can also shrink. The size given by the one-time stat() operation is not reliable over the long term. – Jonathan Leffler
up vote 301 down vote accepted

I'd like to provide an abstract, high-level perspective.

Concurrency and simultaneity

I/O operations interact with the environment. The environment is not part of your program, and not under your control. The environment truly exists "concurrently" with your program. As with all things concurrent, questions about the "current state" don't make sense: There is no concept of "simultaneity" across concurrent events. Many properties of state simply don't exist concurrently.

Let me make this more precise: Suppose you want to ask, "do you have more data". You could ask this of a concurrent container, or of your I/O system. But the answer is generally unactionable, and thus meaningless. So what if the container says "yes" – by the time you try reading, it may no longer have data. Similarly, if the answer is "no", by the time you try reading, data may have arrived. The conclusion is that there simply is no property like "I have data", since you cannot act meaningfully in response to any possible answer. (The situation is slightly better with buffered input, where you might conceivably get a "yes, I have data" that constitutes some kind of guarantee, but you would still have to be able to deal with the opposite case. And with output the situation is certainly just as bad as I described: you never know if that disk or that network buffer is full.)

So we conclude that it is impossible, and in fact unreasonable, to ask an I/O system whether it will be able to perform an I/O operation. The only possible way we can interact with it (just as with a concurrent container) is to attempt the operation and check whether it succeeded or failed. At that moment where you interact with the environment, then and only then can you know whether the interaction was actually possible, and at that point you must commit to performing the interaction. (This is a "synchronisation point", if you will.)

EOF

Now we get to EOF. EOF is the response you get from an attempted I/O operation. It means that you were trying to read or write something, but when doing so you failed to read or write any data, and instead the end of the input or output was encountered. This is true for essentially all the I/O APIs, whether it be the C standard library, C++ iostreams, or other libraries. As long as the I/O operations succeed, you simply cannot know whether further, future operations will succeed. You must always first try the operation and then respond to success or failure.

Examples

In each of the examples, note carefully that we first attempt the I/O operation and then consume the result if it is valid. Note further that we always must use the result of the I/O operation, though the result takes different shapes and forms in each example.

  • C stdio, read from a file:

    for (;;) {
        size_t n = fread(buf, 1, bufsize, infile);
        consume(buf, n);
        if (n < bufsize) { break; }
    }
    

    The result we must use is n, the number of elements that were read (which may be as little as zero).

  • C stdio, scanf:

    for (int a, b, c; scanf("%d %d %d", &a, &b, &c) == 3; ) {
        consume(a, b, c);
    }
    

    The result we must use is the return value of scanf, the number of elements converted.

  • C++, iostreams formatted extraction:

    for (int n; std::cin >> n; ) {
        consume(n);
    }
    

    The result we must use is std::cin itself, which can be evaluated in a boolean context and tells us whether the stream is still in the good() state.

  • C++, iostreams getline:

    for (std::string line; std::getline(std::cin, line); ) {
        consume(line);
    }
    

    The result we must use is again std::cin, just as before.

  • POSIX, write(2) to flush a buffer:

    char const * p = buf;
    ssize_t n = bufsize;
    for (ssize_t k = bufsize; (k = write(fd, p, n)) > 0; p += k, n -= k) {}
    if (n != 0) { /* error, failed to write complete buffer */ }
    

    The result we use here is k, the number of bytes written. The point here is that we can only know how many bytes were written after the write operation.

  • POSIX getline()

    char *buffer = NULL;
    size_t bufsiz = 0;
    ssize_t nbytes;
    while ((nbytes = getline(&buffer, &bufsiz, fp)) != -1)
    {
        /* Use nbytes of data in buffer */
    }
    free(buffer);
    

    The result we must use is nbytes, the number of bytes up to and including the newline (or EOF if the file did not end with a newline).

    Note that the function explicitly returns -1 (and not EOF!) when an error occurs or it reaches EOF.

You may notice that we very rarely spell out the actual word "EOF". We usually detect the error condition in some other way that is more immediately interesting to us (e.g. failure to perform as much I/O as we had desired). In every example there is some API feature that could tell us explicitly that the EOF state has been encountered, but this is in fact not a terribly useful piece of information. It is much more of a detail than we often care about. What matters is whether the I/O succeeded, more-so than how it failed.

  • A final example that actually queries the EOF state: Suppose you have a string and want to test that it represents an integer in its entirety, with no extra bits at the end except whitespace. Using C++ iostreams, it goes like this:

    std::string input = "   123   ";   // example
    
    std::istringstream iss(input);
    int value;
    if (iss >> value >> std::ws && iss.get() == EOF) {
        consume(value);
    } else {
        // error, "input" is not parsable as an integer
    }
    

    We use two results here. The first is iss, the stream object itself, to check that the formatted extraction to value succeeded. But then, after also consuming whitespace, we perform another I/O/ operation, iss.get(), and expect it to fail as EOF, which is the case if the entire string has already been consumed by the formatted extraction.

    In the C standard library you can achieve something similar with the strto*l functions by checking that the end pointer has reached the end of the input string.

The answer

while(!eof) is wrong because it tests for something that is irrelevant and fails to test for something that you need to know. The result is that you are erroneously executing code that assumes that it is accessing data that was read successfully, when in fact this never happened.

upvote
  flag
The list item header '• C stdio, scanf' is wrong (or incomplete, at least). scanf is actually C stdio, however its use context is C++: C does not allow declaring variables in the for() initialization expression. – CiaPan
13 upvote
  flag
@CiaPan: I don't think that's true. Both C99 and C11 allow this. – Kerrek SB
7 upvote
  flag
But ANSI C does not. – CiaPan
upvote
  flag
@KerrekSB I'm using an ifstream and trying to test good as my loop condition. Can you help give me an example of why this is a bad idea: //allinonescript.com/q/28299761/2642059 – Jonathan Mee
upvote
  flag
@JonathanMee: It's bad for all the reasons I mention: you cannot look into the future. You cannot tell what will happen in the future. – Kerrek SB
upvote
  flag
@KerrekSB So I'm not expecting that good means my next read will work. But I am expecting that good means my previous read worked. (Or that the stream itself is readable.) In such a situation it's OK to use good? – Jonathan Mee
1 upvote
  flag
@JonathanMee: Yes, that would be appropriate, though usually you can combine this check into the operation (since most iostreams operations return the stream object, which itself has a boolean conversion), and that way you make it obvious that you're not ignoring the return value. – Kerrek SB
upvote
  flag
@Jonathan Leffler: I think your edit breaks the code: You have to call consume(buf, n) after a short read, otherwise you lose data! Roll back? – Kerrek SB
upvote
  flag
@KerrekSB: OK; I see what you mean...my change could be appropriate if, for sake of argument, the call were n = fread(&object, sizeof(object), 1, fp); where you'd only get 0 or 1 returned. I'll revert the lines to the old order; sorry about that. – Jonathan Leffler
upvote
  flag
The std::cin and scanf examples happily treat bad input (something worth raising an error) as EOF (which just means leave the loop). – ex-bart
upvote
  flag
Isn't C++'s design flawed? Suppose you read some data - which succeeds, but before you're able to check the good flag, an asynchronous failing read occurs. Now you good returns false for you even though the read succeeded – WorldSEnder
upvote
  flag
@WorldSEnder: That sounds like a general problem with modifying shared state concurrently. If you're not the only one accessing some shared state, then you can never know what that state is "at the moment"; in fact, the very notion of "at the moment" stops being meaningful. – Kerrek SB
upvote
  flag
The Posix write example is wrong. write returns -1 if there is an error which will lead to a segmentation violation when p eventually gets down to zero or p + n references unmapped memory (once k becomes -1, your loop won't terminate but will continually subtract one from p and add one to n). There's also a more subtle problem that returning zero is not necessarily an error - e.g. with non blocking IO. – JeremyP
upvote
  flag
@JeremyP: Thanks, I fixed that. I'm not going to get into non-blocking I/O, since you wouldn't generally be able to write that as a single, local loop. – Kerrek SB

Not the answer you're looking for? Browse other questions tagged or ask your own question.