Skip to content

TCP Disconnection and Timeouts in C++

A peer can finish sending, reset a connection or simply stop responding. Observe each case, then set a deadline that expires even when bytes keep arriving.

Course examples on GitHub

Open this lesson's sample files · CMake build instructions.

Before you start

Complete Message Boundaries in a TCP Stream. Run the commands in two terminals on the Pi, both in examples/network.

Outcome: distinguish EOF from a reset and from a local receive deadline, without discarding unread bytes or restarting the time budget after every read.

These are one-client loopback experiments. No root access, firewall changes or permanent service is needed.

Three different ways to stop waiting

Observation Meaning What it does not prove
recv() > 0 These bytes are available now. The whole application response has arrived.
recv() == 0 The peer has finished its sending direction, after queued bytes. The response is valid or complete.
recv() == -1, errno == ECONNRESET The connection was reset. Previously unread bytes can still be recovered.
Our clock reaches the deadline Our application stops waiting. The peer is dead, or TCP has closed itself.

A clean EOF can still truncate a length-prefixed message. Keep the previous lesson's parser checks; transport termination and application success are separate decisions.

sequenceDiagram accTitle: EOF, reset and deadline are different outcomes accDescr: The client sends a question mark to start the fixture. A clean peer sends hello then finishes its write side. A reset aborts the connection. A silent peer sends nothing, so the client's own clock ends the wait. participant C as Client participant S as Fixture server C->>S: One question-mark byte Note over C: Start receive deadline once alt Clean end of stream S->>C: hello S-->>C: Finish sending Note over C: Read queued bytes, then recv = 0 else Abortive close S--xC: Reset connection Note over C: recv fails with ECONNRESET else No response before deadline Note over S: Connection stays open Note over C: Local clock expires, close socket end

This is an application-level sequence, not a packet trace. EOF concerns one direction: shutdown(SHUT_WR) finishes sending without itself disabling reception.

Build and try the fixture server

1
2
cmake --preset native-debug
cmake --build --preset native-debug --parallel 2

In terminal A, start the clean-EOF mode:

1
./build/native/debug/tcp_lifecycle_server 9002 eof

Within 30 seconds, run in terminal B:

1
./build/native/debug/tcp_lifecycle_client 9002

Expected client output:

1
eof bytes=5 data=hello

The client uses a four-byte buffer, so it must read again to collect all five bytes. It continues until EOF rather than assuming the first read is the whole response.

Restart terminal A's server for each row below, replacing its last argument. Keep the same client command:

Server mode Server behaviour after receiving ? Typical client output Exit code
eof Send hello, finish sending, then close. eof bytes=5 data=hello 0
reset Abort the accepted connection with Linux SO_LINGER={1,0}. reset bytes=0 data= 2
idle Send nothing, keep open for about two seconds. deadline bytes=0 data= 3
drip Send a, b, c about 400 ms apart, then wait. deadline bytes=3 data=abc 3

For example, terminal A:

1
./build/native/debug/tcp_lifecycle_server 9002 drip

Terminal B:

1
2
./build/native/debug/tcp_lifecycle_client 9002
echo $?

The client normally stops after about one second of receiving, even in drip mode. Scheduling can change how many bytes it reads before that point; the invariant is that progress does not renew the deadline. These are not real-time timing guarantees.

The server handles one connection and exits. Wait for its finished mode=... line before restarting it. If port 9002 is occupied, choose another unprivileged port in both terminals.

Reset is a test fixture, not a cleanup recipe

The reset mode deliberately uses abortive close to reproduce a Linux TCP error. Do not add this option to normal cleanup: it can discard data. A crashed process does not necessarily produce a reset; ordinary descriptor closure can produce a clean EOF.

One receive deadline, not a timeout per read

Earlier samples used SO_RCVTIMEO to bound each blocking call. A peer sending some bytes before each call's timeout can keep a loop running far longer than that timeout.

Here, the client sets one deadline after connecting and sending the one-byte request:

sequenceDiagram accTitle: Incoming bytes consume the same receive budget accDescr: The default receive budget starts once at roughly zero milliseconds. Bytes a, b and c arrive around zero, four hundred and eight hundred milliseconds. The client stops around one thousand milliseconds, even though the server remains open. participant S as Drip server participant C as Client clock Note over C: Start once, budget = 1000 ms S->>C: a, around 0 ms Note over C: Remaining budget decreases S->>C: b, around 400 ms S->>C: c, around 800 ms Note over C: Around 1000 ms, deadline Note over S: Still open, no EOF yet

Read the receive loop in this order:

  1. steady_clock supplies a clock unaffected by wall-clock adjustments.
  2. Compute deadline - now before every poll(); never replace deadline after progress.
  3. Round a positive remainder up to milliseconds for poll(), then check the clock again when it returns.
  4. Read with MSG_DONTWAIT. If readiness changes, EAGAIN or EWOULDBLOCK means try the same deadline again, not EOF.
  5. After EINTR, recompute the remaining budget instead of starting a fresh wait.

POLLHUP may arrive alongside unread data. Do not exit immediately on that flag: read the queued bytes first and let recv() distinguish bytes, EOF and an error.

This budget covers only the receive phase, not DNS, connecting or sending. There is no DNS lookup here. Linux SO_SNDTIMEO separately bounds the blocking setup calls to about three seconds each; it does not set the poll() timeout. A whole-request deadline would need the same clock budget carried through all phases.

Read the standalone client

Complete file: app/tcp-lifecycle/client.cpp. All socket calls and descriptor cleanup are visible here. The one-byte request synchronizes the fixture so it does not reset the connection before the client has sent its request.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
#include <arpa/inet.h>
#include <array>
#include <cerrno>
#include <charconv>
#include <chrono>
#include <cstdint>
#include <iostream>
#include <poll.h>
#include <stdexcept>
#include <string>
#include <string_view>
#include <sys/socket.h>
#include <sys/time.h>
#include <system_error>
#include <unistd.h>

int main(int argc, char** argv) {
    int fd = -1;
    int status = 1;
    try {
        if (argc < 2 || argc > 3) {
            throw std::invalid_argument(
                "usage: tcp_lifecycle_client port [receive_ms]");
        }
        unsigned int port = 0;
        const std::string_view port_text = argv[1];
        const auto [port_end, port_error] = std::from_chars(
            port_text.data(), port_text.data() + port_text.size(), port);
        if (port_error != std::errc{} ||
            port_end != port_text.data() + port_text.size() || port == 0 ||
            port > 65535) {
            throw std::invalid_argument("port must be 1..65535");
        }
        int receive_ms = 1000;
        if (argc == 3) {
            const std::string_view text = argv[2];
            const auto [end, error] = std::from_chars(
                text.data(), text.data() + text.size(), receive_ms);
            if (error != std::errc{} || end != text.data() + text.size() ||
                receive_ms < 1 || receive_ms > 10000) {
                throw std::invalid_argument("receive_ms must be 1..10000");
            }
        }
        fd = ::socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0);
        if (fd < 0) {
            throw std::system_error(errno, std::generic_category(), "socket");
        }
        // Separate setup waits from the receive-phase deadline below.
        const timeval setup_timeout{3, 0};
        if (::setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, &setup_timeout,
                         sizeof(setup_timeout)) < 0) {
            throw std::system_error(errno, std::generic_category(),
                                    "setsockopt SO_SNDTIMEO");
        }
        sockaddr_in server{};
        server.sin_family = AF_INET;
        server.sin_port = htons(static_cast<std::uint16_t>(port));
        server.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
        if (::connect(fd, reinterpret_cast<const sockaddr*>(&server),
                      sizeof(server)) < 0) {
            throw std::system_error(errno, std::generic_category(), "connect");
        }
        // The server waits for this one byte before choosing its behaviour.
        const char request = '?';
        const auto sent = ::send(fd, &request, 1, MSG_NOSIGNAL);
        if (sent < 0) {
            throw std::system_error(errno, std::generic_category(), "send");
        }
        if (sent != 1) {
            throw std::runtime_error("request made no progress");
        }

        using Clock = std::chrono::steady_clock;
        const auto deadline =
            Clock::now() + std::chrono::milliseconds(receive_ms);
        std::array<char, 4> buffer{};
        std::string reply;
        std::string_view outcome;
        while (true) {
            const auto remaining = std::chrono::ceil<std::chrono::milliseconds>(
                                       deadline - Clock::now())
                                       .count();
            if (remaining <= 0) {
                outcome = "deadline";
                break;
            }
            pollfd ready{fd, POLLIN, 0};
            const int result = ::poll(&ready, 1, static_cast<int>(remaining));
            if (result < 0) {
                if (errno == EINTR) {
                    // Keep the original deadline after an interruption.
                    continue;
                }
                throw std::system_error(errno, std::generic_category(), "poll");
            }
            if (result == 0 || Clock::now() >= deadline) {
                continue; // Check the same deadline again.
            }
            if (ready.revents & POLLNVAL) {
                throw std::runtime_error("invalid socket descriptor");
            }
            // A hangup can coexist with unread data. Check with recv.
            const auto received =
                ::recv(fd, buffer.data(), buffer.size(), MSG_DONTWAIT);
            if (received < 0) {
                if (errno == EINTR || errno == EAGAIN || errno == EWOULDBLOCK) {
                    continue;
                }
                if (errno == ECONNRESET) {
                    outcome = "reset";
                    break;
                }
                throw std::system_error(errno, std::generic_category(), "recv");
            }
            if (received == 0) {
                outcome = "eof";
                break;
            }
            if (reply.size() + static_cast<std::size_t>(received) > 1024) {
                throw std::runtime_error("reply exceeds 1024 bytes");
            }
            reply.append(buffer.data(), static_cast<std::size_t>(received));
        }
        std::cout << outcome << " bytes=" << reply.size() << " data=" << reply
                  << '\n';
        status = outcome == "eof" ? 0 : outcome == "reset" ? 2 : 3;
    } catch (const std::exception& error) {
        std::cerr << error.what() << '\n';
    }
    if (fd >= 0) {
        ::close(fd);
    }
    return status;
}

Exit code 1 is reserved for other errors, such as invalid arguments, connection refusal or an oversized reply. This diagnostic client treats EOF as a completed observation, not proof of a valid application response.

How the server creates each outcome

The server consumes exactly the one-byte ? request, then selects the requested behaviour. eof sends all five bytes with a partial-write loop. reset changes only the accepted socket's linger option before cleanup. idle and drip use sleeping waits, not CPU-spinning loops, and finish after about two seconds of fixture activity.

Full standalone fixture: app/tcp-lifecycle/server.cpp 
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
#include <arpa/inet.h>
#include <cerrno>
#include <charconv>
#include <chrono>
#include <cstdint>
#include <iostream>
#include <stdexcept>
#include <string_view>
#include <sys/socket.h>
#include <sys/time.h>
#include <system_error>
#include <thread>
#include <unistd.h>

int main(int argc, char** argv) {
    int listener = -1;
    int connection = -1;
    int status = 0;
    try {
        if (argc != 3) {
            throw std::invalid_argument(
                "usage: tcp_lifecycle_server port eof|reset|idle|drip");
        }
        unsigned int port = 0;
        const std::string_view text = argv[1];
        const auto [end, error] =
            std::from_chars(text.data(), text.data() + text.size(), port);
        if (error != std::errc{} || end != text.data() + text.size() ||
            port > 65535) {
            throw std::invalid_argument("port must be 0..65535");
        }
        const std::string_view mode = argv[2];
        if (mode != "eof" && mode != "reset" && mode != "idle" &&
            mode != "drip") {
            throw std::invalid_argument(
                "mode must be eof, reset, idle or drip");
        }
        listener = ::socket(AF_INET, SOCK_STREAM | SOCK_CLOEXEC, 0);
        if (listener < 0) {
            throw std::system_error(errno, std::generic_category(), "socket");
        }
        const int reuse = 1;
        const timeval accept_timeout{30, 0};
        if (::setsockopt(listener, SOL_SOCKET, SO_REUSEADDR, &reuse,
                         sizeof(reuse)) < 0 ||
            ::setsockopt(listener, SOL_SOCKET, SO_RCVTIMEO, &accept_timeout,
                         sizeof(accept_timeout)) < 0) {
            throw std::system_error(errno, std::generic_category(),
                                    "setsockopt listener");
        }
        sockaddr_in local{};
        local.sin_family = AF_INET;
        local.sin_port = htons(static_cast<std::uint16_t>(port));
        local.sin_addr.s_addr = htonl(INADDR_LOOPBACK);
        if (::bind(listener, reinterpret_cast<const sockaddr*>(&local),
                   sizeof(local)) < 0 ||
            ::listen(listener, 1) < 0) {
            throw std::system_error(errno, std::generic_category(),
                                    "bind/listen");
        }
        socklen_t size = sizeof(local);
        if (::getsockname(listener, reinterpret_cast<sockaddr*>(&local),
                          &size) < 0) {
            throw std::system_error(errno, std::generic_category(),
                                    "getsockname");
        }
        std::cout << "listening 127.0.0.1:" << ntohs(local.sin_port)
                  << std::endl;
        connection = ::accept(listener, nullptr, nullptr);
        if (connection < 0) {
            throw std::system_error(errno, std::generic_category(), "accept");
        }
        ::close(listener);
        listener = -1;
        const timeval io_timeout{3, 0};
        if (::setsockopt(connection, SOL_SOCKET, SO_RCVTIMEO, &io_timeout,
                         sizeof(io_timeout)) < 0 ||
            ::setsockopt(connection, SOL_SOCKET, SO_SNDTIMEO, &io_timeout,
                         sizeof(io_timeout)) < 0) {
            throw std::system_error(errno, std::generic_category(),
                                    "setsockopt I/O timeout");
        }
        char request = 0;
        const auto received = ::recv(connection, &request, 1, 0);
        if (received < 0) {
            throw std::system_error(errno, std::generic_category(),
                                    "recv request");
        }
        if (received != 1 || request != '?') {
            throw std::runtime_error("expected one '?' request byte");
        }
        if (mode == "reset") {
            // Linux test fixture: abort instead of sending FIN.
            const linger abortive{1, 0};
            if (::setsockopt(connection, SOL_SOCKET, SO_LINGER, &abortive,
                             sizeof(abortive)) < 0) {
                throw std::system_error(errno, std::generic_category(),
                                        "setsockopt SO_LINGER");
            }
        } else if (mode == "idle") {
            std::this_thread::sleep_for(std::chrono::seconds(2));
        } else {
            const std::string_view reply = mode == "eof" ? "hello" : "abc";
            std::size_t offset = 0;
            while (offset < reply.size()) {
                const auto count = mode == "drip" ? 1 : reply.size() - offset;
                const auto sent = ::send(connection, reply.data() + offset,
                                         count, MSG_NOSIGNAL);
                if (sent < 0) {
                    throw std::system_error(errno, std::generic_category(),
                                            "send");
                }
                if (sent == 0) {
                    throw std::runtime_error("send made no progress");
                }
                offset += static_cast<std::size_t>(sent);
                if (mode == "drip") {
                    std::this_thread::sleep_for(std::chrono::milliseconds(400));
                }
            }
            if (mode == "drip") {
                std::this_thread::sleep_for(std::chrono::milliseconds(800));
            } else if (::shutdown(connection, SHUT_WR) < 0) {
                throw std::system_error(errno, std::generic_category(),
                                        "shutdown");
            }
        }
        std::cout << "finished mode=" << mode << '\n';
    } catch (const std::exception& error) {
        std::cerr << error.what() << '\n';
        status = 1;
    }
    if (connection >= 0) {
        ::close(connection);
    }
    if (listener >= 0) {
        ::close(listener);
    }
    return status;
}

The listener waits at most about 30 seconds for a client. The accepted socket's three-second per-call I/O timeouts prevent a client that never sends ? from leaving the fixture waiting indefinitely. Interrupted setup calls are reported as errors; the receive-deadline loop in the client handles interruptions separately.

To compile without CMake:

1
2
3
4
5
mkdir -p build/direct
c++ -std=c++20 -Wall -Wextra -Wpedantic -pthread \
    app/tcp-lifecycle/server.cpp -o build/direct/tcp_lifecycle_server
c++ -std=c++20 -Wall -Wextra -Wpedantic \
    app/tcp-lifecycle/client.cpp -o build/direct/tcp_lifecycle_client

Run the direct-build pair with the same arguments. -pthread selects platform thread support for the fixture's sleeping waits.

Change the budget, not the protocol

Restart the drip server, then allow three seconds for receiving:

1
./build/native/debug/tcp_lifecycle_client 9002 3000

It should now reach the server's clean close and print:

1
eof bytes=3 data=abc

The optional argument is a receive budget in milliseconds, from 1 to 10000. A longer budget changes how long this client waits, not the server's behaviour.

A deadline does not generate a special TCP timeout message. This program chooses to close its own descriptor afterwards. Do not blindly resend a real request: a peer may have performed the operation before the reply was delayed, and a retry could duplicate it.

Troubleshooting

  • Connection refused: start the server first and use the same port in both terminals. This is a connection-establishment failure, not a receive deadline.
  • Server reports an invalid request: use tcp_lifecycle_client, not the previous echo or framing client; their wire formats are different.
  • Deadline with some data: expected for drip; do not treat a partial reply as a complete message.
  • Different byte count under load: inspect the outcome first. The example does not promise exact read timing or a fixed number of TCP packets.

Check your understanding

A peer sends one byte every 400 ms forever. Would a one-second timeout on each receive call guarantee that the entire receive loop finishes within one second?

Compare your reasoning

No. Each successful call could be followed by another one-second wait. A fixed deadline makes the remaining budget shrink despite incoming bytes. The deadline is a local policy; it does not tell you whether a remote operation succeeded.

Continue the course

Previous: Message Boundaries in a TCP Stream · Next: Multiple TCP Connections · Course overview

References: receive results and errors, poll readiness and hangups, Linux socket timeouts, TCP connection errors.

Donate