os-test

Legend

Unanimous Result kind 1 Result kind 2 Result kind 3 Result kind 4 Result kind 5
Good Good unanimous Good result kind 1 Good result kind 2 Good result kind 3 Good result kind 4 Good result kind 5
Unknown Unknown unanimous Unknown result kind 1 Unknown result kind 2 Unknown result kind 3 Unknown result kind 4 Unknown result kind 5
Bad Bad unanimous Bad result kind 1 Bad result kind 2 Bad result kind 3 Bad result kind 4 Bad result kind 5

Good. A cell is good if its output belongs to the set of expected valid outputs for that test, as determined by the applicable standards, specifications, expected behavior, or the interpretation of the os-test authors. If all the good results in a row have the same output, all the good result cells are colored in the unanimous color. Otherwise, each different kind of good outcome is colored in an unique color.

Unknown. A cell is unknown if the output is known, but it has not yet been determined if the output is a good result. If all the unknown results in a row have the same output, all the unknown result cells are colored in the unanimous color. Otherwise, each different kind of unknown outcome is colored in an unique color.

Bad. A cell is bad if its output is neither good nor unknown. If all the bad results in a row have the same output, all the bad result cells are colored in the unanimous color. Otherwise, each different kind of bad outcome is colored in an unique color.

§. The § link on the left of each row links to that row.

Suites

os-test currently contains these suites:

udp

This suite tests the UDP stack.

Open questions:

Running this suite has the following side effects:

haiku
Haiku 1 x86_64
§ accept-nonblock haiku: bad
accept: EINVAL
§ accept haiku: bad
accept: EINVAL
§ bind-any-0-getpeername haiku: good
getsockname: ENOTCONN
§ bind-any-0-getsockname haiku: good
0.0.0.0:non-zero
§ bind-any-0 haiku: good
exit: 0
§ bind-broadcast-0-getpeername haiku: bad
connect: General system error
§ bind-broadcast-0-getsockname haiku: bad
connect: General system error
§ bind-connect-self-send-poll haiku: good
0 | POLLIN | POLLOUT
§ bind-connect-self-send-recv haiku: good
x
§ bind-connect-self-send-shutdown-r-poll haiku: bad
shutdown: ENOTSUP
§ bind-connect-self-send-shutdown-r-recv-recv haiku: bad
shutdown: ENOTSUP
§ bind-connect-self-send-shutdown-r-recv haiku: bad
shutdown: ENOTSUP
§ bind-connect-self-send-shutdown-rw-poll haiku: bad
shutdown: ENOTSUP
§ bind-connect-self-send-shutdown-rw-recv-recv haiku: bad
shutdown: ENOTSUP
§ bind-connect-self-send-shutdown-rw-recv haiku: bad
shutdown: ENOTSUP
§ bind-connect-self-send haiku: good
exit: 0
§ bind-connect-self-shutdown-r-send-poll haiku: bad
shutdown: ENOTSUP
§ bind-connect-self-shutdown-r-send-recv haiku: bad
shutdown: ENOTSUP
§ bind-connect-self haiku: good
exit: 0
§ bind-loopback-0-getpeername haiku: good
getsockname: ENOTCONN
§ bind-loopback-0-getsockname haiku: good
127.0.0.1:non-zero
§ bind-rebind haiku: bad
exit: 0
§ bind-sendto-self-recv haiku: good
x
§ bind-sendto-self haiku: good
exit: 0
§ bind-socket-sendto-first-poll haiku: good
0 | POLLIN | POLLOUT
§ bind-socket-sendto-first-recv haiku: good
x
§ bind-socket-sendto-first-shutdown-r-poll haiku: bad
shutdown: ENOTSUP
§ bind-socket-sendto-first-shutdown-r-recv haiku: bad
shutdown: ENOTSUP
§ bind-socket-sendto-first-shutdown-rw-poll haiku: bad
shutdown: ENOTSUP
§ bind-socket-sendto-first-shutdown-rw-recv haiku: bad
shutdown: ENOTSUP
§ bind-socket-sendto-first-shutdown-w-poll haiku: bad
shutdown: ENOTSUP
§ bind-socket-sendto-first-shutdown-w-recv haiku: bad
shutdown: ENOTSUP
§ bind-socket-shutdown-r-sendto-first-poll haiku: bad
shutdown: ENOTSUP
§ bind-socket-shutdown-r-sendto-first-recv haiku: bad
shutdown: ENOTSUP
§ bind-socket-shutdown-rw-sendto-first-poll haiku: bad
shutdown: ENOTSUP
§ bind-socket-shutdown-rw-sendto-first-recv haiku: bad
shutdown: ENOTSUP
§ bind-socket-shutdown-w-sendto-first-poll haiku: bad
shutdown: ENOTSUP
§ bind-socket-shutdown-w-sendto-first-recv haiku: bad
shutdown: ENOTSUP
§ connect-any-getpeername haiku: bad
127.0.0.1:0
§ connect-getpeername haiku: good
127.0.0.1:65535
§ connect-loopback-get-so-bindtodevice haiku: good
getsockopt: SO_BINDTODEVICE: ENOPROTOOPT
§ connect-loopback-getsockname haiku: good
127.0.0.1:non-zero
§ connect-loopback-reconnect-loopback-getsockname haiku: good
127.0.0.1:same port
§ connect-loopback-reconnect-wan-getsockname haiku: bad
192.168.2.101:same port
§ connect-loopback-unconnect-rebind-any-getsockname haiku: unknown
0.0.0.0:non-zero
§ connect-loopback-unconnect-rebind-loopback-getsockname haiku: unknown
127.0.0.1:non-zero
§ connect-poll haiku: good
0 | POLLOUT
§ connect-reconnect-any-getpeername haiku: bad
127.0.0.1:0
§ connect-reconnect-getpeername haiku: good
127.0.0.1:65534
§ connect-reconnect-same haiku: good
exit: 0
§ connect-reconnect haiku: good
exit: 0
§ connect-recv haiku: good
recv: EWOULDBLOCK
§ connect-send-error-accept haiku: bad
accept: EINVAL
§ connect-send-error-getpeername haiku: good
exit: 0
§ connect-send-error-getsockname haiku: good
exit: 0
§ connect-send-error-listen haiku: good
listen: ENOTSUP
§ connect-send-error-poll-poll haiku: unknown
first poll: 0 | POLLIN | POLLOUT | POLLERR
second poll: 0 | POLLIN | POLLOUT | POLLERR
§ connect-send-error-poll-so-error-poll haiku: unknown
first poll: 0 | POLLIN | POLLOUT | POLLERR
SO_ERROR: ECONNREFUSED
second poll: 0 | POLLOUT
§ connect-send-error-poll haiku: unknown
0 | POLLIN | POLLOUT | POLLERR
§ connect-send-error-reconnect haiku: unknown
exit: 0
§ connect-send-error-recv haiku: good
recv: ECONNREFUSED
§ connect-send-error-send-send haiku: bad
second send: no error
third send: no error
§ connect-send-error-send haiku: bad
second send: no error
§ connect-send-error-shutdown-r-recv haiku: bad
shutdown: ENOTSUP
§ connect-send-error-shutdown-r-send haiku: bad
shutdown: ENOTSUP
§ connect-send-error-shutdown-rw-recv haiku: bad
shutdown: ENOTSUP
§ connect-send-error-shutdown-rw-send haiku: bad
shutdown: ENOTSUP
§ connect-send-error-shutdown-rw-so-error haiku: bad
shutdown: ENOTSUP
§ connect-send-error-shutdown-w-recv haiku: bad
shutdown: ENOTSUP
§ connect-send-error-shutdown-w-send haiku: bad
shutdown: ENOTSUP
§ connect-send-error-so-error-send-send haiku: bad
SO_ERROR: ECONNREFUSED
second send: no error
third send: no error
§ connect-sendto-null haiku: bad
sendto: EINVAL
§ connect-sendto-other haiku: unknown
sendto: EISCONN
§ connect-sendto-same haiku: unknown
sendto: EISCONN
§ connect-shutdown-r-recv haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-r-send haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-r-unconnect-recv haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-r-unconnect-send haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-reconnect haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-rw-reconnect-send haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-rw-recv haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-rw-send haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-rw-unconnect-recv haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-rw-unconnect-send haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-w-recv haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-w-send haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-w-unconnect-recv haiku: bad
shutdown: ENOTSUP
§ connect-shutdown-w-unconnect-send haiku: bad
shutdown: ENOTSUP
§ connect-shutdown haiku: bad
shutdown: ENOTSUP
§ connect-unconnect-getpeername haiku: bad
0.0.0.0:0
§ connect-unconnect-getsockname haiku: unknown
127.0.0.1:non-zero
§ connect-unconnect-sa-family haiku: unknown
exit: 0
§ connect-unconnect-shutdown-r-recv haiku: bad
shutdown: ENOTSUP
§ connect-unconnect-shutdown-r-send haiku: bad
shutdown: ENOTSUP
§ connect-unconnect-shutdown-rw-recv haiku: bad
shutdown: ENOTSUP
§ connect-unconnect-shutdown-rw-send haiku: bad
shutdown: ENOTSUP
§ connect-unconnect-shutdown-w-recv haiku: bad
shutdown: ENOTSUP
§ connect-unconnect-shutdown-w-send haiku: bad
shutdown: ENOTSUP
§ connect-unconnect-sockaddr-in haiku: good
exit: 0
§ connect-unconnect-sockaddr-storage haiku: good
exit: 0
§ connect-unconnect-sockaddr haiku: unknown
exit: 0
§ connect-unconnect-unconnect haiku: good
exit: 0
§ connect-unconnect haiku: good
exit: 0
§ connect-wan-get-so-bindtodevice haiku: good
getsockopt: SO_BINDTODEVICE: ENOPROTOOPT
§ connect-wan-getsockname haiku: bad
192.168.2.101:non-zero
§ connect-wan-send-reconnect-loopback-send haiku: unknown
exit: 0
§ connect-wan-unconnect-rebind-any-getsockname haiku: bad
bind: General system error
§ connect-wan-unconnect-rebind-same-getsockname haiku: bad
192.168.2.101:non-zero
§ get-so-bindtodevice haiku: good
getsockopt: SO_BINDTODEVICE: ENOPROTOOPT
§ getpeername haiku: good
getpeername: ENOTCONN
§ getsockname haiku: bad
getsockname: ENOTCONN
§ listen haiku: good
listen: ENOTSUP
§ pair-send-poll haiku: good
0 | POLLIN | POLLOUT
§ pair-send-recv haiku: good
x
§ pair-send-shutdown-r-poll haiku: bad
shutdown: ENOTSUP
§ pair-send-shutdown-r-recv haiku: bad
shutdown: ENOTSUP
§ pair-send-shutdown-rw-poll haiku: bad
shutdown: ENOTSUP
§ pair-send-shutdown-rw-recv haiku: bad
shutdown: ENOTSUP
§ pair-send-shutdown-w-poll haiku: bad
shutdown: ENOTSUP
§ pair-send-shutdown-w-recv haiku: bad
shutdown: ENOTSUP
§ pair-shutdown-r-send-poll haiku: bad
shutdown: ENOTSUP
§ pair-shutdown-r-send-recv haiku: bad
shutdown: ENOTSUP
§ pair-shutdown-rw-send-poll haiku: bad
shutdown: ENOTSUP
§ pair-shutdown-rw-send-recv haiku: bad
shutdown: ENOTSUP
§ pair-shutdown-w-send-poll haiku: bad
shutdown: ENOTSUP
§ pair-shutdown-w-send-recv haiku: bad
shutdown: ENOTSUP
§ poll haiku: unknown
0 | POLLOUT
§ recvfrom-getsockname haiku: bad
getsockname: ENOTCONN
§ sendto-getsockname haiku: good
0.0.0.0:non-zero
§ shutdown-r-recv haiku: bad
shutdown: ENOTSUP
§ shutdown-r-send haiku: bad
shutdown: ENOTSUP
§ shutdown-rw-recv haiku: bad
shutdown: ENOTSUP
§ shutdown-rw-send haiku: bad
shutdown: ENOTSUP
§ shutdown-w-recv haiku: bad
shutdown: ENOTSUP
§ shutdown-w-send haiku: bad
shutdown: ENOTSUP
§ shutdown haiku: bad
shutdown: ENOTSUP
§ so-error haiku: good
SO_ERROR: no error
§ trio-connect-send-right-x-poll haiku: good
0 | POLLIN | POLLOUT
§ trio-connect-send-right-x-recv haiku: good
x
§ trio-connect-send-wrong-y-poll haiku: good
0 | POLLOUT
§ trio-connect-send-wrong-y-recv haiku: good
recv: EWOULDBLOCK
§ trio-connect-send-wrong-y-send-right-x-poll haiku: good
0 | POLLIN | POLLOUT
§ trio-connect-send-wrong-y-send-right-x-recv haiku: good
x
§ trio-send-right-x-poll haiku: good
0 | POLLIN | POLLOUT
§ trio-send-right-x-recv haiku: good
x
§ trio-send-right-x-send-wrong-y-poll haiku: good
0 | POLLIN | POLLOUT
§ trio-send-right-x-send-wrong-y-recv haiku: good
x
§ trio-send-wrong-y-connect-poll haiku: bad
0 | POLLIN | POLLOUT
§ trio-send-wrong-y-connect-recv haiku: bad
y
§ trio-send-wrong-y-connect-send-right-x-poll haiku: good
0 | POLLIN | POLLOUT
§ trio-send-wrong-y-connect-send-right-x-recv haiku: bad
y
§ unconnect haiku: good
exit: 0