Unix Socket Programming

shutdown() is useful for deliniating when you are done providing a request to a server using TCP. A typical use is to send a request to a server followed by a shutdown(). The server will read your request followed by an EOF (read of 0 on most unix implementations). This tells the server that it has your full request. You then go read blocked on the socket. The server will process your request and send the necessary data back to you followed by a close. When you have finished reading all of the response to your request you will read an EOF thus signifying that you have the whole response. It should be noted the TTCP (TCP for Transactions -- see R. Steven's home page) provides for a better method of tcp transaction management.

/*

*The quick example below is a fragment of

* a very simple client process.

* After establishing the connection with

* the server it forks. Then

* child sends the keyboard input to the

* server until EOF is received and

* the parent receives answers from the server.

*

* variables declarations and error handling are omitted

*/

s=connect(...);

if( fork() ){

/*The child, it copies its stdin to the socket*/

while( gets(buffer) >0)

write(s,buf,strlen(buffer));

close(s);

exit(0);

}

else {

/* The parent, it receives answers */

while( (l=read(s,buffer,sizeof(buffer)){

do_something(l,buffer);

/* Connection break from the server is assumed */

/* ATTENTION: deadlock here */

wait(0); /* Wait for the child to exit */

exit(0);

}

What do we expect? The child detects an EOF from its stdin, it closes the socket (assuming connection break) and exits. The server in its turn detects EOF, closes connection and exits. The parent detects EOF, makes the wait() system call and exits. What do we see instead? The socket instance in the parent process is still opened for writing and reading, though the parent never writes. The server never detects EOF and waits for more data from the client forever. The parent never sees the connection is closed and hangs forever and the server hangs too. Unexpected deadlock!

You should change the client fragment

as follows:

if( fork() ) {

/* The child */

while( gets(buffer) }

write(s,buffer,strlen(buffer));

shutdown(s,1);

/* Break the connection

for writing, The server will detect EOF now.

Note: reading from the socket is still allowed.

The server may send some more data

after receiving EOF, why not? */

exit(0);

}

If the peer calls close() or exits, without

having messed with SO_LINGER, then our

calls to read() should return 0. It is

less clear what happens to write() calls

in this case; I would expect EPIPE, not

on the next call, but the one after.

If the peer reboots, or sets l_onoff = 1,

l_linger = 0 and then closes, then we

should get ECONNRESET (eventually) from

read(), or EPIPE from write().

When write() returns EPIPE, it also

raises the SIGPIPE signal - you never

see the EPIPE error unless you

handle or ignore the signal.

If the peer remains unreachable,

we should get some other error.

Don't think that write() can legitimately

return 0. read() should return 0 on

receipt of a FIN from the peer, and on

all following calls.

So yes, you must expect read() to return 0.

As an example, suppose you are receiving

a file down a TCP link; you

might handle the return from read() like this:

rc = read(sock,buf,sizeof(buf));

if (rc > 0)

{

write(file,buf,rc);

/* error checking on file omitted */

}

else if (rc == 0)

{

close(file);

close(sock);

/* file received successfully */

}

else /* rc < 0 */

{

/* close file and delete it, since data

is not complete report error, or whatever */

}

Use the getservbyname() routine. This will return a pointer to a servent structure. You are interested in the s_port field, which contains the port number, with correct byte ordering (so you don't need to call htons() on it). Here is a sample routine:

/* Take a service name, and a service type, and return a port number. If the service name is not found, it tries it as a decimal number. The number returned is byte ordered for the network. */

int atoport(char *service, char *proto)

{

int port;

long int lport;

struct servent *serv;

char *errpos;

/* First try to read it from /etc/services */

serv = getservbyname(service, proto);

if (serv != NULL)

port = serv->s_port;

else {

/* Not in services, maybe a number? */

lport = strtol(service,&errpos,0);

if ( (errpos[0] != 0) || (lport < 1)

|| (lport > 5000) )

return -1;

/* Invalid port address */

port = htons(lport);

}

return port;

}

First, create the socket and put it into non-blocking mode, then call connect(). There are three possibilities:

o connect succeeds: the connection has been successfully made (this usually only happens when connecting to the same machine)

o connect fails: obvious

o connect returns -1/EINPROGRESS. The connection attempt has begun, but not yet completed.

If the connection succeeds:

o the socket will select() as writable (and will also select as readable if data arrives)

If the connection fails:

o the socket will select as readable *and* writable, but either a read or write will return the error code from the connection attempt. Also, you can use getsockopt(SO_ERROR) to get the error status - but be careful; some systems return the error code in the result parameter of getsockopt(), but others (incorrectly) cause the getsockopt call itself to fail with the stored value as the error.

In the traditional BSD socket implementation, sockets that are atomic such as UDP keep received data in lists of mbufs. An mbuf is a fixed size buffer that is shared by various protocol stacks. When you set your receive buffer size, the protocol stack keeps track of how many bytes of mbuf space are on the receive buffer, not the number of actual bytes. This approach is used because the resource you are controlling is really how many mbufs are used, not how many bytes are being held in the socket buffer. (A socket buffer isn't really a buffer in the traditional sense, but a list of mbufs).

For example: Lets assume your UNIX has a small mbuf size of 256 bytes. If your receive socket buffer is set to 4096, you can fit 16 mbufs on the socket buffer. If you receive 16 UDP packets that are 10 bytes each, your socket buffer is full, and you have 160 bytes of data. If you receive 16 UDP packets that are 200 bytes each, your socket buffer is also full, but contains 3200 bytes of data. FIONREAD returns the total number of bytes, not the number of messages or bytes of mbufs. Because of this, it is not a good indicator of how full your receive buffer is.

Additionaly, if you receive UDP messages that are 260 bytes, you use up two mbufs, and can only recieve 8 packets before your socket buffer is full. In this case, only 2080 bytes of the 4096 are held in the socket buffer.

This example is greatly simplified, and the real socket buffer algorithm also takes into account some other parameters. Note that some older socket implementations use a 128 byte mbuf.

You have to design your protocol to expect a confirmation back from the destination when a message is received. Of course is the confirmation is sent by UDP, then it too is unreliable and may not make it back to the sender. If the sender does not get confirmation back by a certain time, it will have to re-transmit the message, maybe more than once. Now the receiver has a problem because it may have already received the message, so some way of dropping duplicates is required. Most protocols use a message numbering scheme so that the receiver can tell that it has already processed this message and return another confirmation. Confirmations will also have to reference the message number so that the sender can tell which message is being confirmed.

UDP is good for sending messages from one system to another when the order isn't important and you don't need all of the messages to get to the other machine. This is why I've only used UDP once to write the example code for the faq. Usually TCP is a better solution. It saves you having to write code to ensure that messages make it to the desired destination, or to ensure the message ordering. Keep in mind that every additional line of code you add to your project in another line that could contain a potentially expensive bug.

If you find that TCP is too slow for your needs you may be able to get better performance with UDP so long as you are willing to sacrifice message order and/or reliability.

UDP must be used to multicast messages to more than one other machine at the same time. With TCP an application would have to open separate connections to each of the destination machines and send the message once to each target machine. This limits your application to only communicate with machines that it already knows about.

This question is usually asked by people who are testing their server with telnet, and want it to process their keystrokes one character at a time. The correct technique is to use a psuedo terminal (pty). More on that in a minute.

You can have your server send a sequence of control characters: 0xff 0xfb 0x01 0xff 0xfb 0x03 0xff 0xfd 0x0f3, which translates to IAC WILL ECHO IAC WILL SUPPRESS-GO-AHEAD IAC DO SUPPRESS-GO-AHEAD. For more information on what this means, check out std8, std28 and std29. Roger also gave the following tips:

o This code will suppress echo, so you'll have to send the characters the user types back to the client if you want the user to see them.

o Carriage returns will be followed by a null character, so you'll have to expect them.

o If you get a 0xff, it will be followed by two more characters. These are telnet escapes.

Use of a pty would also be the correct way to execute a child process and pass the i/o to a socket.

SO_REUSEADDR allows your server to bind to an address which is in a TIME_WAIT state. It does not allow more than one server to bind to the same address. It was mentioned that use of this flag can create a security risk because another server can bind to a the same port, by binding to a specific address as opposed to INADDR_ANY. The SO_REUSEPORT flag allows multiple processes to bind to the same address provided all of them use the SO_REUSEPORT option.

This is a newer flag that appeared in the 4.4BSD multicasting code (although that code was from elsewhere, so I am not sure just who invented the new SO_REUSEPORT flag).

What this flag lets you do is rebind a port that is already in use, but only if all users of the port specify the flag. I believe the intent is for multicasting apps, since if you're running the same app on a host, all need to bind the same port. But the flag may have other uses. For example the following is from a post in February:

SO_REUSEPORT is also useful for eliminating the try-10-times-to-bind hack in ftpd's data connection setup routine. Without SO_REUSEPORT, only one ftpd thread can bind to TCP (lhost, lport, INADDR_ANY, 0) in preparation for connecting back to the client. Under conditions of heavy load, there are more threads colliding here than the try-10-times hack can accomodate. With SO_REUSEPORT, things work nicely and the hack becomes unnecessary.

I have also heard that DEC OSF supports the flag. Also note that under 4.4BSD, if you are binding a multicast address, then SO_REUSEADDR is condisered the same as SO_REUSEPORT (p. 731 of "TCP/IP Illustrated, Volume 2"). I think under Solaris you just replace SO_REUSEPORT with SO_REUSEADDR.

From a later Stevens posting, with minor editing:

Basically SO_REUSEPORT is a BSD'ism that arose when multicasting was added, even thought it was not used in the original Steve Deering code. I believe some BSD-derived systems may also include it (OSF, now Digital Unix, perhaps?). SO_REUSEPORT lets you bind the same address *and* port, but only if all the binders have specified it. But when binding a multicast address (its main use), SO_REUSEADDR is considered identical to SO_REUSEPORT (p. 731, "TCP/IP Illustrated, Volume 2"). So for portability of multicasting applications I always use SO_REUSEADDR.

After accept()ing a connection, use getpeername() to get the address of the client. The client's address is of course, also returned on the accept(), but it is essential to initialise the address-length parameter before the accept call for this will work.

int t;

int len;

struct sockaddr_in sin;

struct hostent *host;

len = sizeof sin;

if (getpeername(t, (struct sockaddr *)

&sin, &len) < 0)

perror("getpeername");

else {

if ((host = gethostbyaddr((char *)

&sin.sin_addr,sizeof sin.sin_addr,

AF_INET)) == NULL)

perror("gethostbyaddr");

else printf("remote host is '%s'n",

host->h_name);

}