mariadb/Docs/net_doc.txt

 MySQL Client - Server Protocol Ducumentation


Introduction
------------


This paper has an objective of a through description of the
client - server protocol which is embodied in MySQL. Particularly,
this paper aims to document and describe:

- manner in which MySQL server detects client connection requests and
  creates connection
- manner in which MySQL client C API call connects to server - the
  entire protocol of sending / receiving data by MySQL server and C API
  code
- manner in which queries are sent by client C API calls to server
- manner in which query results are sent by server
- manner in which query results are resolved by server
- sending and receiving of error messages


This paper does not have the goal or describing nor documenting other
related MySQL issues, like usage of thread libraries, MySQL standard
library set, MySQL strings library and other MySQL specific libraries,
type definitions and utilities.
				
Issues that are covered by this paper are contained in the following
source code files:

- client/net.c and sql/net_serv.c, the two being identical
- client/libmysql.c (not entire file is covered)
- include/mysql_com.h
- include/mysql.h 
- sql/mysqld.cc (not entire file is covered)
- sql/net_pkg.cc
- sql/sql_base.cc (not entire file is covered)	
- sql/sql_select.cc (not entire file is covered)
- sql/sql_parse.cc  (not entire file is covered)

Beside this introduction this paper presents basic definitions,
constants, structures and global variables, all related functions in
server and in C API. Textual description of the entire protocol
functioning is described in the last chapter of this paper.


Constants, structures and global variables
------------------------------------------

This chapter will  describe all constants, structures and
global variables relevant to client - server protocol. 

Constants

They are important as they contain default values, the ones
that are valied if options are not set in any other way. Beside that
MySQL source code does not contain a single non-defined constant in
it's code. This description of constants does not include
configuration and conditional compilation #defines.
				
NAME_LEN	  - field and table name length, current value 64
HOSTNAME_LENGTH   - length of the host name, current value 64
USERNAME_LENGTH   - user name length, current vluae 16
MYSQL_PORT        - default TCP/IP port number, current value 	3306
MYSQL_UNIX_ADDR   - full path of the default Unix socket file, current value
		    "/tmp/mysql.sock"
MYSQL_NAMEDPIPE   - full path of the default NT pipe file, current value
		    "MySQL"
MYSQL_SERVICENAME - name of the MySQL Service on NT, current value  "MySql"
NET_HEADER_SIZE   - size of the network header, when no
                    compression is used, current value  4
COMP_HEADER_SIZE - additional size of network header when
		    compression is used, current value 3

What follows are set of constants, defined in source only, which
define capabilities of the client built with that version of C
API. Simply, when some new feature is added in client, that client
feature is defined, so that server can detect what capabilities a
client program has.

CLIENT_LONG_PASSWORD	- client supports new more secure passwords
CLIENT_LONG_FLAG	- client uses longer flags
CLIENT_CONNECT_WITH_DB	- client can specify db on connect 
CLIENT_COMPRESS		- client can use compression protocol 
CLIENT_ODBC		- ODBC client 
CLIENT_LOCAL_FILES	- client can use LOAD DATA INFILE LOCAL 
CLIENT_IGNORE_SPACE	- client can  Ignore spaces before '('
CLIENT_CHANGE_USER	- client supports the mysql_change_user() 

What follows are other constants, pertaining to timeouts and sizes

MYSQL_ERRMSG_SIZE	- maximum size of error message string, current value 200
NET_READ_TIMEOUT	- read timeout, current value  30 sec.
NET_WRITE_TIMEOUT       - write timeout, current value 60 sec.
NET_WAIT_TIMEOUT	- wait for new query timeout, current value 8*60*60
			  sec. i.e. 8 hours
packet_error		- value returned in case of socket errors, current
			  value -1
TES_BLOCKING		- used in debug mode for setting up blocking testing 
RETRY COUNT		 - number of times network read and write will be
                           retried, current value 1 

There are also error messages for last_errno, which depict system
errors, and are used on the server only.

ER_NET_PACKAGE_TOO_LARGE  - packet is larger then max_allowed_packet
ER_OUT_OF_RESOURCES	  - practically no more memory
ER_NET_ERROR_ON_WRITE     - error in writing to NT Named Pipe
ER_NET_WRITE_INTERRUPTED  - some signal or interrupt happened
                            during write
ER_NET_READ_ERROR_FROM_PIPE - error in reading from  NT Named Pipe
ER_NET_FCNTL_ERROR	  - error in trying to set fcntl on socket
                            descriptor
ER_NET_PACKETS_OUT_OF_ORDER - packet numbers on client and
                              server side differ
ER_NET_UNCOMPRESS_ERROR   - error in uncompress of compressed packet

				
		Structs and eunms


struct NET

this is MySQL's network handle structure, used in all client / server
read/write functions. On the server it is initialized and preserved in
each thread. On the client, it is a part of MYSQL struct, which is
MySQL handle used in all C API functions. This structure uniquely
identifies a connection, either on the server or client side. 
This structure consists of the following
fields:

			  Vio* vio; - explained above
				HANDLE hPipe - Handle for NT Named Pipe file
				my_socket fd - file descriptor used for both tcp socket and Unix socket file 
			  int fcntl - contains info on fcntl options used on fd. Mostly
                    used for saving info if  blocking is used or not
				unsigned char *buff - network buffer used for storing data for
                              reading from  / writing to socket
				unsigned char,*buff_end - points to the end of buff
				unsigned char *write_pos - present writing position in buff
				unsigned char *read_pos - present reading postiion in
                                  buff. This pointer is used for
                                  reading data after calling
                                  my_net_read function  and function
                                  that are just it's  wrappers
				char last_error[MYSQL_ERRMSG_SIZE] - holds last error message
				unsigned int last_errno - holds last error code of the network 
                                  protocol. It's ossible values are
                                  listed in above constants. It is
																	used only  on the server side
				unsigned int max_packet - holds current value of buff size 
				unsigned int timeout - stores read timeout value for that connection 
				unsigned int pkt_nr - stores a value of the current packet
                              number in a batch of packets. Used
                              primarily for detection of protocol
                              errors resulting in a mismatch
				my_bool error - holds either 1 or 0 depending on the error condition
				my_bool return_errno - if it's value != 0 then there is an
															 error in protocol mismatch between
															 client and server
				my_bool compress -  if true compression is used in the protocol
				unsigned long remain_in_buf - used only in reading compressed
                                      packets. Explained in my_net_read
				unsigned long length - used only for storing a length of the
															 read packet. Explained in my_net_read
				unsigned long buf_length -  used only in reading compressed
                                    packets. Explained in my_net_read
				unsigned long where_b - used only in reading compressed
                                packets. Explained in my_net_read
				short int  more - used for reporting in mysql_list_processes
				char save_char - used in reading compressed packets for saving 
												 chars in order to make zero-delimited
                         strings. Explained in my_net_read

Few typedefs will be defined for easier understanding of the text that 
follows.

				typedef char **MYSQL_ROW - data containing one row of values
				typedef unsigned int MYSQL_FIELD_OFFSET - offset in bytes of
																								  the current field
				typedef MYSQL_ROWS *MYSQL_ROW_OFFSET - offset in bytes of
																							 the current row


				struct MYSQL_FIELD - contains all info on the attributes of a
specific column in a result set, plus info on lengths of the column in
a result set. This struct is tagged as st_mysql_field. This structure
consists of the following fields:

				char *name - name of column 
				char *table -	table of column if column was a field  and not
											expression or constant 
				char *def - default value (set by mysql_list_fields) 
				enum enum_field_types type -  see above
				unsigned int length -  width of column  in a current row
				unsigned int max_length - maximum width of that column in
																  entire result set
				unsigned int flags - corresponding to Extra in DESCRIBE
				unsigned int decimals - number of decimals in field 


				struct MYSQL_ROWS - a node for each row in the single linked
list forming entire result set. This struct is tagged as
st_mysql_rows, and has two fields:

				struct st_mysql_rows *next - pointer to a next one
				MYSQL_ROW data - see above


				struct MYSQL_DATA - contains all rows from result set. It is
tagged as st_mysql_data and has following fields:

			  my_ulonglong rows - how many rows
			  unsigned int fields - how many columns
				MYSQL_ROWS *data - see above. This is a first node of the
                           linked list 
				MEM_ROOT alloc -  MEM_ROOT is MySQL memory allocation
												  structure, and this field is used to store
												  all fields and rows.   

				
				struct st_mysql_options - holds various client options, and
contains following fields:

				unsigned int connect_timeout - time in sec. for cennection 
				unsigned int client_flag - used to cold client capabilities
				my_bool compress - boolean for  compression 
				my_bool named_pipe - is Named Pipe used on NT 
				unsigned int port - what TCP port is used
				char *host - host to connect to
				char *init_command - command to be executed upon connection 
				char *user - account name on MySQL server
				char *password - password for the above
				char *unix_socket - full path for Unix socket file
				char *db - default database
				char *my_cnf_file - optional configuration file
				char *my_cnf_group - optional header for options


				struct MYSQL - MySQL client's handle. Required for any
operation issed from client to server. Tagged as st_mysql and having
following fields:

				NET	 net - see above
				char *host - host on which MySQL server is running
				char *user - MySQL user name
				char *passwd - password for above 
				char *unix_socket- full path of Unix socket file
				char *server_version - version of the server
				char *host_info - contains info on how has connection been
                          established, TCP port, socket or Named Pipe
				char *info - used to store information on the query results,
										 like number of rows affected etc.
				char *db - current database
				unsigned int	port -= TCP port in use
				unsigned int client_flag - client capabilities
				unsigned int server_capabilities - server capabilities
				unsigned int	protocol_version - version of the protocl
				unsigned int	field_count - used for storing number of fields
                                    immidiately upon execution of a
                                    query, but before fetching rows
				unsigned long thread_id - server thread to which this connection is attached
				my_ulonglong affected_rows - used for storing number of rows
                                    immidiately upon execution of a
                                    query, but before fetching rows
				my_ulonglong insert_id - fetching LAST_INSERT_ID() through
																 client C API
				my_ulonglong extra_info - used by mysqlshow 
				unsigned long packet_length - saving size of the first packet
																			upon execution of a query
				enum mysql_status status - see above
				MYSQL_FIELD	*fields - see above
				MEM_ROOT	field_alloc - memory used for storing previous field 
                                (fields)
				my_bool	free_me - boolean that flags if MYSQL was allocated in 
												  mysql_init
				my_bool	reconnect -  used to automatically reconnect
				struct st_mysql_options options - see above
				char	 scramble_buff[9] - key for scrambling password before
																  sending it to server

				
				struct MYSQL_RES - tagged as st_mysql_res and used to store
entire result set from a single query. Contains following fields:

				my_ulonglong row_count - number of rows
				unsigned int	field_count - number of columns  
				unsigned int current_field - cursor for fetching fields
				MYSQL_FIELD	*fields - see above
				MYSQL_DATA	*data - see above, and used in buffered reads,
                            i.e.  mysql_store_result only
				MYSQL_ROWS	*data_cursor - pointing to the field  of above "data"
				MEM_ROOT	field_alloc - memory allocation for above "fields"
				MYSQL_ROW	row - used for storing row by row in unbuffered
												reads, i.e. in  mysql_use_result
				MYSQL_ROW	current_row - cursor to the current row for buffered 
                                reads
				unsigned long *lengths -  column lengths of current row 
				MYSQL		*handle - see above, used in unbuffered reads, i.e. in 
												  mysql_use_resultq
				my_bool	eof - used my mysql_fetch_row as a marker for end of data




				Global variables


			  unsigned long max_allowed_packet - maximum allowable value of
                                           network buffer. Default
                                           value - 1 Mb

				unsigned long net_buffer_length - default, starting  value of
																					network buffer - 8 Kb

				unsigned long bytes_sent - total number of bytes written since 
                                   startup of the server

				unsigned long bytes_received - total number of bytes read
                                       since startup of the server


				Synopsis of the basic client - server protocol 
				----------------------------------------------

				Purpose of this chapter is to provide a complete picture of
the basic client - server protocol implemented in MySQL. It was felt
it is necessary after writting descriptions for all of the functions
involved in basic protocol. There are at present 11 functions
involved, with several structures, many constants etc, which are all
described in detail. But as a forest could not be seen from the trees, 
so a concept of the protocol could not be deciphered easily from a
thourough documentation on minutae.

				Althouch concept of the protocol was not changed with the
introduction of vio system, embodied in violate.cc source file and VIO
system, the introduction of these has changed a code substantially. Before
VIO was introduced, functions for reading from / writing to network
connection had to deal with various network standards. So, these functions
depended on whether TCP port or Unix socket file or NT Named Pipe file is
used. This is all changed now and single vio_ functions are called, while
all this diversity is covered by vio_ functions.

				In MySQL a specific buffered network input / output transport
model has been implemented. Although each operating system mah have
it's own buffering for network connections, MySQL has added it's own
buffering model. This same for each of the three transport protocol
types that are used in MySQL client - server communications, which are 
tcp sockets (on all systems), Unix socket files on Unix and Unix-like
operating systems and  Named Pipe files on NT. Alghouth tcp sockets
are omnipresent, the later two types have been added for local
connections. Those two connection types can be used in local modes
only, i.e. when both client and server reside on the same host, and
are introduced because they enable better speeds for local
connections. This is especially usefull for WWW type of
applications. Startup options of MySQl server allow that either tcp
sockets or local connection (OS dependent) can be disallowed. 

				In order to be able to implement buffered input / output MySQL 
allocates a buffer. A starting size of this buffer is determined by a
value of the global variable net_buffer_length, which can be changed
at MySQL server startup. This is, as explained only a startup length
of MySQL network buffer. As  a signle  item that has to be read /
written can be larger then that value, MySQL will increase buffer size 
as long as that size reaches value of global variable
max_aallowed_packet, which is also settable at server startup. Maximum 
value of this variable is limited by the way MySQL stores / reads
sizes of packets to be sent / read, which means by the way MySQL
formats packages.

				Basically each packet consists of two parts, a header and
data. In the case when compression is not used, header consists of 4
bytes of which 3 contain a length of the packet to be sent and one
holds a packet number. When  compression is used there are
onother 3 bytes which store a size of uncompressed data. Because of
the way MySQL packs length into 3 bytes, plus due to the usage of some 
special values in the most significant byte, maximum size of
max_allowed_packet is limited to 24 Mb at present. So, if compression
is not used, at first 4 bytes are written to the buffer and then data
itself. As MySQL buffers I/O logical packets are packet together until 
packets fill up entire  size of the buffer. That size  no less then
net_buffer_size, but no greater  then max_allowed_packet. So, actuall
writting to the network is done when this buffer is filled up. As
frequently sequence of buffers make a logicall unit, like a result
set, then at the end of sending data, even if buffer is not full, data 
is written, i.e. flushed  to the connection with a call of the
net_flush function. In order to maintain that no single packet can be
larger then this value, checks are made throughout a code, so that not 
signle field or command could exceed that value. 

				In order to maintain coherency in consicutive packets, each
packet is numbered and their number stored as a part of a header, as
explained above. Packets start with 0, so whenever a logical packet is 
written, that number is incremented. On the other side when packets
are read, value that is fetched is compared with a value stored and if 
there is no mismatch that value is incremented too. Packet number is
reset on the client side when unwanted connections are removed from
the connection and on the server side when a new command hsa been
started. 


				So, before writting, a buffer contains a sequence of logical
packets, consisting of header plus data consequtively. In the case
that compression is used, packet numbers are not stored in each header
of the logical packets, but a whole buffer, or a part of it if
flushing is done, containing one or more logical packets are
compressed. In that case a new larger header, is formed, and all
logical packets contained in the buffer are compressed together. This
way only one packet is formed which makes several logical packets,
which improves both speed and compression ratio. On the other side,
when this large compressed packet is read, it is furst uncompressed,
and then logical packets are sent, one by one, to the calling
functions. 

				
				All this functionality is described in detail in the following 
chapter. It does not contain functions that form logical packets, or
that read and write to connections but also functions that are used
for  initialisation, clearing of connections. There are functions at
higher level dealing with sending fields, rows, establishing
connections, sending commands, but those are not explained in the
following chapter.
				

				Functions utilized in client - server protocol 
				----------------------------------------------

				First of all, functions are described that are involved in
praparing/ reading / writing data over TCP port , socket or named pipe
file, and functions directly related to those. All of these functions
are used both in server and client. Server and client specific code
segments will be documented in each function description. Each MySQl
function checks for errors in memory allocation / freeing, as well as
in every OS call, like the one dealing with files and sockets, and for
errors in indeginous MySQL function calls. This is expected, but has
to be said, as not to repeat it in every function description.

				Older versions of MySQL have utilized the following macros for 
reading from  / writing to socket. 

				raw_net_read  - calls OS function  recv function that reads N
												 bytes from a socket into a buffer. Number of
												 bytes read is returned. 

        raw_net_write - calls OS funciton  send to write N bytes from 
												 a buffer to socket. Number of bytes written
												 is  returned.

				These macros are replaced with VIO (Virtual I/O) functions.


				Function name: my_net_init
				
				Parameters: struct NET *, enum_net_type, struct Vio

				Return value : 1 for error, 0 for success

				Function purpose: To initialize properly all NET fields,
                          allocate memory and set socket options

				Function description

				First of all, buff field of NET struct is allocated to the
size of net_buffer_lenght, and on failure function exits with 0. All
fields in NET are set to their default or starting values. As
net_buffer_length and max_allowed_packet are configurable,
max_allowed_packet is set equal to net_buffer_length it the later one
is greater. max_packet if set for that NET to net_buffer_lenght, and
buff_end points to buff end. vio feild is set to the second parameter.
If it is a real connection, which is a case when second parameter is
not null, then fd field is set by calling vio_fd function.read_pos and
write_pos to buff, while remaining integers are set to 0. If function
is run on the MySQL server on  Unix  and server is started in a test
mode that would require testing of blocking, then vio_blocking
function is called. Last, fast throughput mode is set by a call to
vio_fastsend function.

				

				Function name: net_end
				
				Parameters: struct NET *

				Return value : void

				Function purpose: To release memory alloceted to buff

				

				Function name: net_realloc (private, static function)
				
				Parameters: struct NET, unlong (unsigned long)

				Return value : 1 for error, 0 for success

				Function purpose: To change memory allocated to buff

				Function description

				New length of buff field of NET struct is passed as second
parameter. It is first checked versus max_allowd_packet and if greater 
error is returned. New length is aligned to 4096 boundary.  Then , buff
is reallocated,  buff_end, max_packet and write_pas reset to the same
values as in my_net_init.



				Function name: net_clear (used on client side only)
				
				Parameters: struct NET *

				Return value : void

				Function purpose: To read unread packets

				Function description

				This function is used on client side only, and is executed
only if a program is not started in test mode. This function reads
unread packets without processing them. First, non-blocking mode is
set on systems that have not non-blocking mode defined.  This is
performed by checking a mode with vio_is_blocking function. and
setting non-blocking mode by vio_blocking function. If this operation
was successfull, then packets are read by vio_read function, to which
vio field of NET is passed together with buff and max_packet field
values.  field of the same struct at a length of max_packet. If
blocking was active before reading is performed, blocking is set with
vio_blocking function.  AFter reading has been performed, pkt_nr is
reset to 0 and write_pos reset to buff. In order to clarify some
matters non-blocking mode enables executing program to dissociate from 
a connection, so that error in connection would not hang entire
program or it's thread. 

				Function name: net_flush 
				
				Parameters: struct NET *

				Return value : 1 for error, 0 for success

				Function purpose: To write remaining bytes in buff to socket

				Function description

				net_real_write (described below) is performed is write_pos
differs from buff, both being fields of the only parameter. write_pos
is reset to buff. This function has to be used, as MySQL uses buffered 
writes, as it will be more explained in a function net_write_buff.


				Function name: my_net_write
				
				Parameters: struct NET *, const char *, ulong

				Return value : 1 for error, 0 for success

				Function purpose: Write a logical packet in a first parameter
                          of second parameter length

				Function description

				The purpose of this function is to prepare a logical packet
such that entire content of data, pointed to by second parametar and
in length of third parameter is sent to the other side. In case of
server, it is used for sending result sets, and in case of client it
is used for sending local data. This function foremost prepares a
header for the packet. Normal, header consists of 4 bytes, of which
first 3 bytes contain a length of the packet, thereby limiting a
maximum allowable length of a packet to 16 Mb, while a fourth byte
contains a packet number, which is used when one large packet has to
be divided into sequence of packets. This way each sub-packet gets
it's number which should be matched on the other side. When
compression is used another three bytes are added to packet header,
thus packet header is in that case increased to 7 bytes. Additional
three bytes are used to save a length of compressed data. As in
connection that uses compression option, code packs packets together,,
a header prepared by this function is later not used in writting to /
reading from network, but only to distinguish logical packets within a
buffered read operation.


				This function, first stores a value for third parameter into a
first 3 bytes of local char variable of NET_HEADER_SIZE size by usage
of function int3store. Then, at this point, if compression is not
used, pkt_nr is increased, and it's value stored in the last byte of
the said local char[] variable. If compression is used 0 is stored in
both values. Then those four bytes are sent to other side by the usage 
of the function net_write_buff(to be explained later on), and
successfull, entire packet in second parameter of the length described 
in third parameter is sent by the usage of the same function.


				Function name: net_write_command
				
				Parameters: struct NET *, char, const char *, ulong

				Return value : 1 for error, 0 for success

				Function purpose: Send a command with a  packet as in previous 
                          function 

				Function description

				This function is very similar to the previous one. The only
difference is that first packet is enlarged by one byte, so that a
command precedes a packet to be sent. This is implemented by
increasing fist packet by one byte, which contains a command code. As
command codes do not use a range of values that are used by chararcter 
sets, so when the other side receives a packet, first byte after
header contains a command code. This function is used by client for
sending all commands and queries, and by server in connection process
and for sending errors.
				

				Function name: net_write_buff (private, static function)
				
				Parameters: struct NET *, const char *, uint

				Return value : 1 for error, 0 for success

				Function purpose: To write a packet of vany size by cutting it 
                          and using next function for writing it

				Function description

			  This function was created  after compression feature has been
added to MySQL. This function supposes that packets have already been
properly formatted, regarding packet header etc. Principal reason for
this function existst because a packet that is sent by client or
server does not have to be less then max_packet. So this function
first calculeates how much data has been left in a buff, by getting a
difference between buff_end and write_pos and storing it to local
variable left_length. Then a loop is run as long as a length to be
sent is greater then length of left bytes (left_length). In a loop
data from second parameter is copied to buff at write_pos, as much as
it can be, i.e. by left_length. Then net_real_write function is called 
(see below) with NET, buff, and max_packet parameters. This function
is the lowest level fucntion that writes data over established
connection. In a loop, write_pos is reset to buff, a pointer to data
(second parameter) is moved by teh amount of data sent (left_length),
length of data to be sent (third parameter) is decreased by the amount 
sent (left_length) and left_length is reset to max_packet value, which 
ends a loop. This logic was necessary, as there could have been some
data yet unsent (write_pos != buf), while data to be sent could be as
large as necessary, thus requiring many loops. At the end of function, 
remaining data in second parameter are copied to buff at write_pos, by 
the remaining length of data to be sent (third parameter). So, in the
next call of this function remaining data will be sent, as buff is
used in a call to net_real_write. It is very important to note that if 
a packet to be sent is less then a number of bytes that are still
available in buff, then there will be no writing over network, but
only logical packets will be added one after another. This will
accelerate network traffic, plus if a compression is used,  the
expected compression rate would be higher. That is why server or
client functions that sends data uses at the end of data net_flush
function described above. 


				Function name: net_real_write 
				
				Parameters: struct NET *, const char *, ulong

				Return value : 1 for error, 0 for success

				Function purpose: To write data to a socket or pipe, with
                          compression if used

				Function description

				First, more field is set to 2, to enable reporting in
mysql_list_processes. Then if compression is enabled on that
connection, a new local buffer (variable b) is initialized to the
length of total header (normal header + compression header) and if no
memory is available error is returned. This buffer (b) is used for
holding a fineal, compressed packet to be written ove
connection. Furthermore in compressiion initialization, Second
parameter at length of third parameter is copied to the local buffer
b, and MySQL's wrapped zlib's compression function is run at total
header offset of the local buffer. Please, do note that this function
does not test effectiveness of compression. If compression is turned
on in some connection, it is used all of the time. Also, it is very
important to be cognizant of the fact that this algorithm makes
possible that a single compressed packet contains several logical
packets. In this way compression rate is increased and network
throughput is increased as well. However, this algorithm has
consequences on the other sided, that reads compressed packet, which
is covered in my_net_read function. After compression is done, a full
compression header is properly formed with a packet number,
compressed and uncompressed lengths. At the end of compression code,
third parameter is increased by total header length, as the original
header is not used (see above), and second parameter, pointer to data,
is set to point to local buffer b, in order that a further flow of
function is independent of compression.  . If a function is executed
on server side, a thread alarm initialized and if non-blocking is
active set at NET_WRITE_TIMEOUT. Two local (char *) pointers are
initialized, pos at beginning of second parameter, and end at end of
data. Then a loop is run as long as all data is written, which means
as long as pos != end.  First vio_write function is called, with
parameters of vio field, pos and size of data (end - pos). Number of
bytes written over connection is saved in local variable (length). If
error is returned local bool variable (interrupted) is set according
to the return value of the vio_should_retry called with vio field as
parameter. This bool variable indicates whether writing was
interrupted in some way or not.  Firther, error from vio_write is
differently treated on Unix versus other OS's (Win32 or OS/2). On Unix
an alarm is set if one is not in use, no bytes have been written and
there has been no interruption. Also, in that case, if connection is
not in blocking mode, a sub - loop is run as long as blocking is not
set with vio_blocking function. Withing a loop another run of above
vio_write is run based on return value of vio_is_retry function,
-provided number of repeated writes is less then RETRY_COUNT. If that
is not a case, error field of struct NET. is set to 1 and function
exits. At the exit of sub-llop number of reruns already executed is
reset to zero and another run of above vio_write function is
attempted. If a function is run on Win32 and OS/2, and in the case
that function flow was not interrupted and thread alarm is not in use,
again a main loop is continued until pos != end. In the case that this
function is executed on thread safe client program, a communication
flow is tested on EINTR, caused by context switching, by use of
vio_errno function, in which case a loop is continued. At the end of
processing of the error from vio_write, error field of struct NET is
set, and if on server last_errno field is set to
ER_NET_WRITE_INTERRUPTED in the case thatb local bool variable
(interrupted) is true or to ER_NET_ERROR_ON_WRITE. Before the end of
loop, in order to make possible evaluation of the loop condition, pos
is increased by a value writen in last iteration (length). Also global
variable bytes_sent is increased by the same value, for status
purposes.  At the end of the functions more fields is reset, in case
of compression, combression buffer (b) memory is released and if
thread is still in use, it is ended and blocking state is reset to
it's original state, and function returns error is all bytes are not
written.



				Function name: my_real_read (private, static function)
				
				Parameters: struct NET *,  ulong *

				Return value :  length of bytes read

				Function purpose: low level network connection read function

				Function description

				This function has made as a separate one when compression was
introduced in MySQL client - server protocol . It contains basic, low
level network reading functionality, while all dealings with
compressed packets are handled in next function. Compression in this
function is only handled in as much to unfold a length of uncompressed
data. First blocking state of connection is saved in local bool
variable net_blocking, and field more is set 1 for deteiled reporting
in mysqld_list_processes. A new thread alarm is initialized, in order
to enable read timout handling, and if on server and a connection can
block a program, the alarm is set at a value of timeout field. Local
pointer is set to the position of the next logical packet, with it's
header skipped, which is at field where_b offset from buff. Next, a
two time run code is entered. A loop is run exactly two times becase
first time number of bytes to be fetched (remain) are set to the
header size, which is different when compression is used or not used
on the connection. After first fetch has been done, number of packets
that will be received in second iteration is well known, as fetched
header contains a size of packet, packet number ,and in a case of
compression a size of uncompressed packet.  Then as long, as there are
bytes to read a loop is entered with ffirst reading data from network
connection with vio_read function, called with parameters of field
vio, current position and remaining number of bytes, which value is
hold by local variable (remain) initialized at a value of header size,
which differs if compression is used. Number of bytes read are
returned in local length variable.  If error is returned local bool
variable (interrupted) is set according to the return value of the
vio_should_retry called with vio field as parameter. This bool
variable indicates whether reading was interrupted in some way or not.
Firther, error from vio_read is differently treated on Unix versus
other OS's (Win32 or OS/2). On Unix an alarm is set if one is not in
use, no bytes have been read and there has been no interruption. Also,
in that case, if connection is not in blocking mode, a sub - loop is
run as long as blocking is not set with vio_blocking function. Withing
a loop another run of above vio_read is run based on return value of
vio_is_retry function, -provided number of repeated writes is less
then RETRY_COUNT. If that is not a case, error field of struct NET. is
set to 1 and function exits. At the exit of sub-llop number of reruns
already executed is reset to zero and another run of above vio_read
function is attempted. If a function is run on Win32 and OS/2, and in
the case that function flow was not interrupted and thread alarm is
not in use, again a main loop is continued as long as there are bytes
remaining. In the case that this function is executed on thread safe
client program, then if a another run should be made, which is decided
by the output of vio_should_retry function, in which case a loop is
continued. At the end of processing of the error from vio_read, error
field of struct NET is set, and if on server last_errno field is set
to ER_NET_READ_INTERRUPTED in the case thatb local bool variable
(interrupted) is true or to ER_NET_ERROR_ON_READ. In case of such an
error this function exits and returns error. In a case when there is
no error, number of remaining bytes (remain) is decreased by a number
of bytes read, which should be zero, but in case it is not the entire
code is still in while (remain > 0) loop, which will be exited
immediately if it is. This has been done to accomodate errors in the
traffic level and for the very slow connections. Current position in
field buff is also moved by the amount of bytes read by vio_read
funciton, and global variable bytes_received is increased by the same
value in a thread safe manner. When a loop that is run until necessary 
bytes are read (remain) is finished, then if external loop is in it's
first run, of the two, packet sequencing is tested on consistency by
comparing a number contained at 4th byte in header with pkt_nre
field. Header location is found at where_b offset to field_b. Usage of 
where_b is obligatory due to the possible compression usage. If there
is no compression on a connection, then where_b is always 0. If there
is a discrepancy, then first byte of the header is checked whether it
is equal to 255, because when error is sent by a server, or by a
client if it is sending data (like in LOAD DATA INFILE LOCAL...), then 
first byte in header is set to 255. If it is not 255, then an error on 
packets being out of order is printed. In any case, on server,
last_errno field is set to ER_NET_PACKETS_OUT_OF_ORDER and a function
returns with the error, i.e. value returned is packet_error. If a
check on serial number of packet is successful, pkt_nr field is
incremented in order to enable checking packet order with next packet
and if compression is used, uncompressed length is extracted from a
proper position in header and returned in the second parameter of this 
function. Length of the packet is saved, for the purpose of a proper
return value from this function. Still in the first iteration of the
main loop, a check must be made if field buff could accomodate entire
package that comes, in it's compressed or uncompressed form. This is
done in such a way, because zlib's compress and uncompress functions
use a same memory area for compression / uncompression. Necessary
field buff length is equal to current offset where data are (where_b
which is zero for non-compression), plus the larger value of
compressed or uncompressed package to be read in a second run. If this 
value is larger then a current length of field buff, which is read
from field max_packet, then feild buff has to be reallocated. IF
reallocation with net_realloc function fails, function is returned
with error. Before a second loop is started, length to be read is set
to the length of expected data and current position (pos) is set at
where_b offset from field buff. At the end of function, if alarm is
set, which is a case if it is run on server or on a client  if a
function is interrupted and another run of vio_read is attempted,
alarm is ended and blocking state is resotred from the saved local bool
variable net_blocking. Function returns number of bytes read or the
error (packet_error). 
				
				


				Function name: my_net_read 
				
				Parameters: struct NET *

				Return value : length of bytes read

				Function purpose: Highest level general purpose reading function 

				Function description
					
				First, if compression is not used, my_real_read is called,
with struct NET * a first parameter, and pointer to local ulong
complen as a second parameter, but it's value is not used here.
Number of bytes read is returned in local ulong variable len. read_pos
field is set to an offset of value of where_b field from field
buff. where_b field actually denotes where in field buff is a current
packet. If returned number of bytes read (local variable len) does not
signal that an error in packet trnasmission occured, i.e. it is not
set to packet_error, then a string contained in read_pos is zero
terminated. Simply, the end of a string starting at read_pos, and
ending at read_pos + len, is set to zero. This is done in that way,
because mysql_use_result expects a zero terminated string, and
function returns with a value local variable len. This ends this
function in the case that compression is not used and a remaining code
is executed only if compression is enabled on the connection.  In
order to explain how is compressed packet logically cut into
meningfull packets, a full meaning of several NET feilds should be
explained. First of all, fields in NET are used and not local
variables, as all values should be saved between consecutive calls of
this function. Simply, this function is called in order to return
logical packets, but this function does not need to call my_real_read
function everytime, because when a large packet is uncompressed, it
may, but not necessarily so, contain several logical
packets. Therefore, in oreder to preserve data on logical packets
local variables are not used. Instead fields in NET struct are
used. Field remain_in_buf denotes how many bytes of entire
uncompressed packets is still contained withing buff. field buf_length
saves a value of the length of entire uncompressed packet. field
save_char is used to save a character at a position where a packet
ends, which character has to be replaced with a zero, '\0', in order
to make a logical packet zero delimited, for mysql_use_result. Field
length stores a value of the length of compressed packet. Field
read_pos as usual, points to the current reading position. This char *
pointer is used by all fucntion calling this function in order to
fetch their data. Field buff is not used for that purpose, but
read_pos is used instead. This change was introduced with compression,
when algorihtm accomodated grouping of several packets together.  Now,
that meanings of all relevant NET fields is complained, we can proceed
with a flow of this functinn in case when compression is
active. First, if there are remaining portions of compressed packet in
a field buff, saved character value is set at a position where zero
char '\0' was inserted to enable a string do be zero delimited for
mysql_use_result. Then a loop is started. In the first part of the
loop, if there are remaining bytes, local uchar *pos variable is set
at a current position in field buff where a new packet starts. This
position is an (buf_length - remain_in_buf) offset in field buff. As
it is possible that next logical packet is not read to the full length
in the remaining of the field buf, several things had to be
inspected. It should be noted that data that is read from
net_rweal_read contains only logical packets containing 4 byte headers
only, being 4 byte headers prepared by my_net_write or
net_write_command. But, when written, logical packet could be so
divided that only a part of header is read in. Therefore after pointer
to the start of the next packet has been saved, a check is made
whether number of remaining bytes in buffer is less then 4, being 3
bytes for length and one byte for packet number. If it is greater,
then a length of the logical packet is extracted and saved a length
field. Then a check is made whether entire packet is contained withing
a buf, i.e. a check is made that a logical packet is fully contained
in a buffer. In that case, number of bytes remaining in buffer is
decreased by a full length of logical packet ( 4 + length field),
read_pos is moved forward by 4 bytes to skip header and be set at a
beginning of data in logical packet, length field is saved for a value
to be returned in function and a loop is exited.  In a case that
entire logical packet is not contained within a buffer, then if length
of the entire buffer differs from remaining length of logical packet,
it (logical packet) is moved to the beginning of the field buff. If
length of the entire buffer equals the remaining length of logical
packet, where_b and buf_length fields are set to 0. This is done so
that in both cases buffer is ready to accept next part of packet. In
order to get a next part of a packet, still within a loop,
my_real_read function is called and length of compressed packet is
returned to a local len variable, and length of compressed data is
returned in complen variable. In a case of non-compression value of
complen is zero.  If packet_error is from my_real_read function, this
function returns also with packet_error. If it is not a packet_error
my_uncompress function is called to uncompress data. It is called with
offset of where_b data from field buff, as it is a postion where
compressed packet starts, and with len and complen values, being
lengths of compressed and uncompressed data.  If there is no
compression, 0 is returned for uncompressed size from my_real_read
function, and my_uncompress wrapper function is made to skip zlib
uncompress in that case.  If error is returned fom my_uncompress,
error field is set to 1, if on server last_errno is set to
ER_NET_UNCOMPRESS_ERROR and loop is exited and function returns with
packet_error. If not, buf_length and reamin_in_buf fields are set to
the uncompressed size of buffer and a loop is continued. When a loop
is exited save_char field is used to save a char at end of a logical
packet, which is an offset of field len from position in field buff
pointed by field read_os, in order that zero char is set at the same
position, for mysql_use_result. Function returns a length of the
logical packet without it's header.