os-test
Legend
Unanimous | Result kind 1 | Result kind 2 | Result kind 3 | Result kind 4 | Result kind 5 | |
---|---|---|---|---|---|---|
Good | Good unanimous | Good result kind 1 | Good result kind 2 | Good result kind 3 | Good result kind 4 | Good result kind 5 |
Unknown | Unknown unanimous | Unknown result kind 1 | Unknown result kind 2 | Unknown result kind 3 | Unknown result kind 4 | Unknown result kind 5 |
Bad | Bad unanimous | Bad result kind 1 | Bad result kind 2 | Bad result kind 3 | Bad result kind 4 | Bad result kind 5 |
Good. A cell is good if its output belongs to the set of expected valid outputs for that test, as determined by the applicable standards, specifications, expected behavior, or the interpretation of the os-test authors. If all the good results in a row have the same output, all the good result cells are colored in the unanimous color. Otherwise, each different kind of good outcome is colored in an unique color.
Unknown. A cell is unknown if the output is known, but it has not yet been determined if the output is a good result. If all the unknown results in a row have the same output, all the unknown result cells are colored in the unanimous color. Otherwise, each different kind of unknown outcome is colored in an unique color.
Bad. A cell is bad if its output is neither good nor unknown. If all the bad results in a row have the same output, all the bad result cells are colored in the unanimous color. Otherwise, each different kind of bad outcome is colored in an unique color.
§. The § link on the left of each row links to that row.
Suites
os-test currently contains these suites:
- udp - This suite tests the UDP stack.
udp
This suite tests the UDP stack.
Open questions:
- What should happen if you bind happen to the broadcast address 255.255.255.255? It is an explicitly forbidden source address. Should it fail with EADDRNOTAVAIL, or some other error, or succeed (and in which case, what does it mean?).
- What should happen if you connect to the any address 0.0.0.0?
- Should it be possible to shutdown a socket without having connected it?
- Should reading on a socket shutdown for read return 0 or should it fail with EWOULDBLOCK?
- Should writing to socket shutdown for write send SIGPIPE in addition to failing with ESPIPE?
- After shutdown for read, should data already received be avalilable for read? Some systems deliver already received asyncronous errors after shutdown, yet don’t deliver already received data after shutdown.
- After shutdown for read, should data be received? Should POLLIN be set if data is received yet recv of it would return 0.
- Should POLLOUT be set after shutdown for write? Should POLLHUP be set?
- Should POLLIN be set after shutdown for write if there’s already received data?
- If an asynchronous socket error is pending, which of POLLIN, POLLOUT, POLLERR should be set?
- Should asynchronous socket errors be delivered on connect?
- What should happen if you connect to an address on the loopback interface and then reconnect to the public internet? What happens to the locally bound address? Is the socket rebound? Or does the connect fail because there’s no longer a valid route from the loopback interface to the public internet.
- If a socket is connected, can you supply a destination address to sendto? What if the destination address is the current remote address?
- What is the local socket name after connect to loopback and then unconnect?
- How does unconnecting work? Can you pass a
sa_family_t
set toAF_UNSPEC
to connect, or do you need to wrap it in astruct sockaddr
, or astruct sockaddr_in
? - What happens if you bind after unconnect?
- If a socket is shutdown for read, should recv return any data received after the shutdown.
- If you receive on freshly made socket, what address should getsockname return?
Running this suite has the following side effects:
- Datagrams containing just the byte ‘x’ are sent to 8.8.8.8:53 (Google DNS)
because I needed a legitimate address that would not send back any ICMP
connection refused address. Please configure this address in udp/udp.h using
BLACKHOLE_HOST
andBLACKHOLE_PORT
to a site of your choice. - Datagrams containing just the byte ‘x’ are sent to port 65534 and port 65535 on the loopback address 127.0.0.1.
- This suite requires port 65535 on the loopback address 127.0.0.1 to be unused.
- UDP sockets will be bound to the loopback interface and on the interface leading to the public internet.
haiku Haiku 1 x86_64 |
|
---|---|
§ accept-nonblock | haiku: bad accept: EINVAL |
§ accept | haiku: bad accept: EINVAL |
§ bind-any-0-getpeername | haiku: good getsockname: ENOTCONN |
§ bind-any-0-getsockname | haiku: good 0.0.0.0:non-zero |
§ bind-any-0 | haiku: good exit: 0 |
§ bind-broadcast-0-getpeername | haiku: bad connect: General system error |
§ bind-broadcast-0-getsockname | haiku: bad connect: General system error |
§ bind-connect-self-send-poll | haiku: good 0 | POLLIN | POLLOUT |
§ bind-connect-self-send-recv | haiku: good x |
§ bind-connect-self-send-shutdown-r-poll | haiku: bad shutdown: ENOTSUP |
§ bind-connect-self-send-shutdown-r-recv-recv | haiku: bad shutdown: ENOTSUP |
§ bind-connect-self-send-shutdown-r-recv | haiku: bad shutdown: ENOTSUP |
§ bind-connect-self-send-shutdown-rw-poll | haiku: bad shutdown: ENOTSUP |
§ bind-connect-self-send-shutdown-rw-recv-recv | haiku: bad shutdown: ENOTSUP |
§ bind-connect-self-send-shutdown-rw-recv | haiku: bad shutdown: ENOTSUP |
§ bind-connect-self-send | haiku: good exit: 0 |
§ bind-connect-self-shutdown-r-send-poll | haiku: bad shutdown: ENOTSUP |
§ bind-connect-self-shutdown-r-send-recv | haiku: bad shutdown: ENOTSUP |
§ bind-connect-self | haiku: good exit: 0 |
§ bind-loopback-0-getpeername | haiku: good getsockname: ENOTCONN |
§ bind-loopback-0-getsockname | haiku: good 127.0.0.1:non-zero |
§ bind-rebind | haiku: bad exit: 0 |
§ bind-sendto-self-recv | haiku: good x |
§ bind-sendto-self | haiku: good exit: 0 |
§ bind-socket-sendto-first-poll | haiku: good 0 | POLLIN | POLLOUT |
§ bind-socket-sendto-first-recv | haiku: good x |
§ bind-socket-sendto-first-shutdown-r-poll | haiku: bad shutdown: ENOTSUP |
§ bind-socket-sendto-first-shutdown-r-recv | haiku: bad shutdown: ENOTSUP |
§ bind-socket-sendto-first-shutdown-rw-poll | haiku: bad shutdown: ENOTSUP |
§ bind-socket-sendto-first-shutdown-rw-recv | haiku: bad shutdown: ENOTSUP |
§ bind-socket-sendto-first-shutdown-w-poll | haiku: bad shutdown: ENOTSUP |
§ bind-socket-sendto-first-shutdown-w-recv | haiku: bad shutdown: ENOTSUP |
§ bind-socket-shutdown-r-sendto-first-poll | haiku: bad shutdown: ENOTSUP |
§ bind-socket-shutdown-r-sendto-first-recv | haiku: bad shutdown: ENOTSUP |
§ bind-socket-shutdown-rw-sendto-first-poll | haiku: bad shutdown: ENOTSUP |
§ bind-socket-shutdown-rw-sendto-first-recv | haiku: bad shutdown: ENOTSUP |
§ bind-socket-shutdown-w-sendto-first-poll | haiku: bad shutdown: ENOTSUP |
§ bind-socket-shutdown-w-sendto-first-recv | haiku: bad shutdown: ENOTSUP |
§ connect-any-getpeername | haiku: bad 127.0.0.1:0 |
§ connect-getpeername | haiku: good 127.0.0.1:65535 |
§ connect-loopback-get-so-bindtodevice | haiku: good getsockopt: SO_BINDTODEVICE: ENOPROTOOPT |
§ connect-loopback-getsockname | haiku: good 127.0.0.1:non-zero |
§ connect-loopback-reconnect-loopback-getsockname | haiku: good 127.0.0.1:same port |
§ connect-loopback-reconnect-wan-getsockname | haiku: bad 192.168.2.101:same port |
§ connect-loopback-unconnect-rebind-any-getsockname | haiku: unknown 0.0.0.0:non-zero |
§ connect-loopback-unconnect-rebind-loopback-getsockname | haiku: unknown 127.0.0.1:non-zero |
§ connect-poll | haiku: good 0 | POLLOUT |
§ connect-reconnect-any-getpeername | haiku: bad 127.0.0.1:0 |
§ connect-reconnect-getpeername | haiku: good 127.0.0.1:65534 |
§ connect-reconnect-same | haiku: good exit: 0 |
§ connect-reconnect | haiku: good exit: 0 |
§ connect-recv | haiku: good recv: EWOULDBLOCK |
§ connect-send-error-accept | haiku: bad accept: EINVAL |
§ connect-send-error-getpeername | haiku: good exit: 0 |
§ connect-send-error-getsockname | haiku: good exit: 0 |
§ connect-send-error-listen | haiku: good listen: ENOTSUP |
§ connect-send-error-poll-poll | haiku: unknown first poll: 0 | POLLIN | POLLOUT | POLLERR second poll: 0 | POLLIN | POLLOUT | POLLERR |
§ connect-send-error-poll-so-error-poll | haiku: unknown first poll: 0 | POLLIN | POLLOUT | POLLERR SO_ERROR: ECONNREFUSED second poll: 0 | POLLOUT |
§ connect-send-error-poll | haiku: unknown 0 | POLLIN | POLLOUT | POLLERR |
§ connect-send-error-reconnect | haiku: unknown exit: 0 |
§ connect-send-error-recv | haiku: good recv: ECONNREFUSED |
§ connect-send-error-send-send | haiku: bad second send: no error third send: no error |
§ connect-send-error-send | haiku: bad second send: no error |
§ connect-send-error-shutdown-r-recv | haiku: bad shutdown: ENOTSUP |
§ connect-send-error-shutdown-r-send | haiku: bad shutdown: ENOTSUP |
§ connect-send-error-shutdown-rw-recv | haiku: bad shutdown: ENOTSUP |
§ connect-send-error-shutdown-rw-send | haiku: bad shutdown: ENOTSUP |
§ connect-send-error-shutdown-rw-so-error | haiku: bad shutdown: ENOTSUP |
§ connect-send-error-shutdown-w-recv | haiku: bad shutdown: ENOTSUP |
§ connect-send-error-shutdown-w-send | haiku: bad shutdown: ENOTSUP |
§ connect-send-error-so-error-send-send | haiku: bad SO_ERROR: ECONNREFUSED second send: no error third send: no error |
§ connect-sendto-null | haiku: bad sendto: EINVAL |
§ connect-sendto-other | haiku: unknown sendto: EISCONN |
§ connect-sendto-same | haiku: unknown sendto: EISCONN |
§ connect-shutdown-r-recv | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-r-send | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-r-unconnect-recv | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-r-unconnect-send | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-reconnect | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-rw-reconnect-send | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-rw-recv | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-rw-send | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-rw-unconnect-recv | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-rw-unconnect-send | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-w-recv | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-w-send | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-w-unconnect-recv | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown-w-unconnect-send | haiku: bad shutdown: ENOTSUP |
§ connect-shutdown | haiku: bad shutdown: ENOTSUP |
§ connect-unconnect-getpeername | haiku: bad 0.0.0.0:0 |
§ connect-unconnect-getsockname | haiku: unknown 127.0.0.1:non-zero |
§ connect-unconnect-sa-family | haiku: unknown exit: 0 |
§ connect-unconnect-shutdown-r-recv | haiku: bad shutdown: ENOTSUP |
§ connect-unconnect-shutdown-r-send | haiku: bad shutdown: ENOTSUP |
§ connect-unconnect-shutdown-rw-recv | haiku: bad shutdown: ENOTSUP |
§ connect-unconnect-shutdown-rw-send | haiku: bad shutdown: ENOTSUP |
§ connect-unconnect-shutdown-w-recv | haiku: bad shutdown: ENOTSUP |
§ connect-unconnect-shutdown-w-send | haiku: bad shutdown: ENOTSUP |
§ connect-unconnect-sockaddr-in | haiku: good exit: 0 |
§ connect-unconnect-sockaddr-storage | haiku: good exit: 0 |
§ connect-unconnect-sockaddr | haiku: unknown exit: 0 |
§ connect-unconnect-unconnect | haiku: good exit: 0 |
§ connect-unconnect | haiku: good exit: 0 |
§ connect-wan-get-so-bindtodevice | haiku: good getsockopt: SO_BINDTODEVICE: ENOPROTOOPT |
§ connect-wan-getsockname | haiku: bad 192.168.2.101:non-zero |
§ connect-wan-send-reconnect-loopback-send | haiku: unknown exit: 0 |
§ connect-wan-unconnect-rebind-any-getsockname | haiku: bad bind: General system error |
§ connect-wan-unconnect-rebind-same-getsockname | haiku: bad 192.168.2.101:non-zero |
§ get-so-bindtodevice | haiku: good getsockopt: SO_BINDTODEVICE: ENOPROTOOPT |
§ getpeername | haiku: good getpeername: ENOTCONN |
§ getsockname | haiku: bad getsockname: ENOTCONN |
§ listen | haiku: good listen: ENOTSUP |
§ pair-send-poll | haiku: good 0 | POLLIN | POLLOUT |
§ pair-send-recv | haiku: good x |
§ pair-send-shutdown-r-poll | haiku: bad shutdown: ENOTSUP |
§ pair-send-shutdown-r-recv | haiku: bad shutdown: ENOTSUP |
§ pair-send-shutdown-rw-poll | haiku: bad shutdown: ENOTSUP |
§ pair-send-shutdown-rw-recv | haiku: bad shutdown: ENOTSUP |
§ pair-send-shutdown-w-poll | haiku: bad shutdown: ENOTSUP |
§ pair-send-shutdown-w-recv | haiku: bad shutdown: ENOTSUP |
§ pair-shutdown-r-send-poll | haiku: bad shutdown: ENOTSUP |
§ pair-shutdown-r-send-recv | haiku: bad shutdown: ENOTSUP |
§ pair-shutdown-rw-send-poll | haiku: bad shutdown: ENOTSUP |
§ pair-shutdown-rw-send-recv | haiku: bad shutdown: ENOTSUP |
§ pair-shutdown-w-send-poll | haiku: bad shutdown: ENOTSUP |
§ pair-shutdown-w-send-recv | haiku: bad shutdown: ENOTSUP |
§ poll | haiku: unknown 0 | POLLOUT |
§ recvfrom-getsockname | haiku: bad getsockname: ENOTCONN |
§ sendto-getsockname | haiku: good 0.0.0.0:non-zero |
§ shutdown-r-recv | haiku: bad shutdown: ENOTSUP |
§ shutdown-r-send | haiku: bad shutdown: ENOTSUP |
§ shutdown-rw-recv | haiku: bad shutdown: ENOTSUP |
§ shutdown-rw-send | haiku: bad shutdown: ENOTSUP |
§ shutdown-w-recv | haiku: bad shutdown: ENOTSUP |
§ shutdown-w-send | haiku: bad shutdown: ENOTSUP |
§ shutdown | haiku: bad shutdown: ENOTSUP |
§ so-error | haiku: good SO_ERROR: no error |
§ trio-connect-send-right-x-poll | haiku: good 0 | POLLIN | POLLOUT |
§ trio-connect-send-right-x-recv | haiku: good x |
§ trio-connect-send-wrong-y-poll | haiku: good 0 | POLLOUT |
§ trio-connect-send-wrong-y-recv | haiku: good recv: EWOULDBLOCK |
§ trio-connect-send-wrong-y-send-right-x-poll | haiku: good 0 | POLLIN | POLLOUT |
§ trio-connect-send-wrong-y-send-right-x-recv | haiku: good x |
§ trio-send-right-x-poll | haiku: good 0 | POLLIN | POLLOUT |
§ trio-send-right-x-recv | haiku: good x |
§ trio-send-right-x-send-wrong-y-poll | haiku: good 0 | POLLIN | POLLOUT |
§ trio-send-right-x-send-wrong-y-recv | haiku: good x |
§ trio-send-wrong-y-connect-poll | haiku: bad 0 | POLLIN | POLLOUT |
§ trio-send-wrong-y-connect-recv | haiku: bad y |
§ trio-send-wrong-y-connect-send-right-x-poll | haiku: good 0 | POLLIN | POLLOUT |
§ trio-send-wrong-y-connect-send-right-x-recv | haiku: bad y |
§ unconnect | haiku: good exit: 0 |