Commit graph

139 commits

Author SHA1 Message Date
8a66b424f3 Manual merge with Conflicts:
sql_udf.cc
2010-01-25 10:55:05 +08:00
He Zhenxing
6bf8c119fe Backport Bug#37148 to 5.1 2010-01-24 15:03:23 +08:00
25a436bdc4 Bug #49132 Replication failure on temporary table + DDL
In RBR, DDL statement will change binlog format to non row-based
format before it is binlogged, but the binlog format was not be
restored, and then manipulating a temporary table can not reset binlog
format to row-based format rightly. So that the manipulated statement
is binlogged with statement-based format.

To fix the problem, restore the state of binlog format after the DDL
statement is binlogged.
2010-01-22 17:38:21 +08:00
Georgi Kodinov
8f6f3dba21 Bug #40877: multi statement execution fails in 5.1.30
Implemented the server infrastructure for the fix:

1. Added a function LEX_STRING *thd_query_string(THD) to return
a LEX_STRING structure instead of char *.
This is the function that must be called in innodb instead of 
thd_query()

2. Did some encapsulation in THD : aggregated thd_query and 
thd_query_length into a LEX_STRING and made accessor and mutator 
methods for easy code updating. 

3. Updated the server code to use the new methods where applicable.
2009-10-16 13:29:42 +03:00
Ignacio Galarza
2d9421c3bb Bug#29125 Windows Server X64: so many compiler warnings
- Remove bothersome warning messages.  This change focuses on the warnings 
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
2009-02-13 11:41:47 -05:00
Ignacio Galarza
2b85c64d65 Bug#29125 Windows Server X64: so many compiler warnings
- Remove bothersome warning messages.  This change focuses on the warnings 
that are covered by the ignore file: support-files/compiler_warnings.supp.
- Strings are guaranteed to be max uint in length
2009-02-10 17:47:54 -05:00
Sergey Glukhov
515de310f0 Bug#37428 Potential security issue with UDFs - linux shellcode execution.
plugin_dir option backported from 5.1
2008-08-25 17:11:59 +05:00
Sergey Petrunia
eb8d8d3e3a Merge fix for BUG#35478 into 5.1 2008-07-15 21:46:02 +04:00
Sergey Petrunia
2951f00be4 BUG#35478: sort_union() returns bad data when sort_buffer_size is hit
- In QUICK_INDEX_MERGE_SELECT::read_keys_and_merge: when we got table->sort from Unique,
  tell init_read_record() not to use rr_from_cache() because a) rowids are already sorted
  and b) it might be that the the data is used by filesort(), which will need record rowids
  (which rr_from_cache() cannot provide).
- Fully de-initialize the table->sort read in QUICK_INDEX_MERGE_SELECT::get_next(). This fixes BUG#35477.
(bk trigger: file as fix for BUG#35478).
2008-07-15 18:13:21 +04:00
kostja@dipika.(none)
acf9b1f346 A fix and a test case for Bug#12713 "Error in a stored function called from
a SELECT doesn't cause ROLLBACK of statem".

The idea of the fix is to ensure that we always commit the current
statement at the end of dispatch_command(). In order to not issue
redundant disc syncs, an optimization of the two-phase commit
protocol is implemented to bypass the two phase commit if
the transaction is read-only.
2008-02-19 14:43:01 +03:00
gluh@eagle.(none)
58336411c9 Merge mysql.com:/home/gluh/MySQL/Merge/5.1
into  mysql.com:/home/gluh/MySQL/Merge/5.1-opt
2007-11-14 17:30:16 +04:00
istruewing@stella.local
d7ce48a9bb Merge bk-internal.mysql.com:/home/bk/mysql-5.1-engines
into  stella.local:/home2/mydev/mysql-5.1-bug31210
2007-11-10 11:58:41 +01:00
kaa@polly.(none)
5ab2d821d1 Merge polly.(none):/home/kaa/src/opt/bug32020/my51-bug31445
into  polly.(none):/home/kaa/src/opt/mysql-5.1-opt
2007-11-09 13:43:42 +03:00
kaa@polly.(none)
6320cdedb2 Merge polly.(none):/home/kaa/src/opt/bug32020/my50-bug31445
into  polly.(none):/home/kaa/src/opt/mysql-5.0-opt
2007-11-09 13:41:50 +03:00
kaa@polly.(none)
ccd8710958 Merge polly.(none):/home/kaa/src/opt/bug32020/my50-bug31445
into  polly.(none):/home/kaa/src/opt/bug32020/my51-bug31445
2007-11-09 13:31:48 +03:00
kaa@polly.(none)
e703c6a78c Fix for bug #32020: loading udfs while --skip-grant-tables is enabled
causes out of memory errors

The code in mysql_create_function() and mysql_drop_function() assumed
that the only reason for UDFs being uninitialized at that point is an
out-of-memory error during initialization. However, another possible 
reason for that is the --skip-grant-tables option in which case UDF 
initialization is skipped and UDFs are unavailable.

The solution is to check whether mysqld is running with
--skip-grant-tables and issue a proper error in such a case.
2007-11-09 13:29:43 +03:00
istruewing@stella.local
3eaf82a175 Bug#31210 - INSERT DELAYED crashes server when used on
partitioned table

Trying INSERT DELAYED on a partitioned table, that has not been
used right before, crashes the server. When a table is used for
select or update, it is kept open for some time. This period I
mean with "right before".

Information about partitioning of a table is stored in form of
a string in the .frm file. Parsing of this string requires a
correctly set up lexical analyzer (lex). The partitioning code
uses a new temporary instance of a lex. But it does still refer
to the previously active lex. The delayd insert thread does not
initialize its lex though...

Added initialization for thd->lex before open table in the delayed
thread and at all other places where it is necessary to call
lex_start() if all tables would be partitioned and need to parse
the .frm file.
2007-11-05 16:25:40 +01:00
malff@lambda.hsd1.co.comcast.net.
a4c66084a8 Merge lambda.hsd1.co.comcast.net.:/home/malff/TREE/mysql-5.0-runtime
into  lambda.hsd1.co.comcast.net.:/home/malff/TREE/mysql-5.1-rt-merge
2007-10-16 15:43:16 -06:00
malff@lambda.hsd1.co.comcast.net.
a9c6ed46e2 Implementing code review comments 2007-10-16 11:16:31 -06:00
monty@mysql.com/nosik.monty.fi
e53a73e26c Fixed a lot of compiler warnings and errors detected by Forte C++ on Solaris
Faster thr_alarm()
Added 'Opened_files' status variable to track calls to my_open()
Don't give warnings when running mysql_install_db
Added option --source-install to mysql_install_db

I had to do the following renames() as used polymorphism didn't work with Forte compiler on 64 bit systems
index_read()      -> index_read_map()
index_read_idx()  -> index_read_idx_map()
index_read_last() -> index_read_last_map()
2007-08-13 16:11:25 +03:00
malff/marcsql@weblab.(none)
c7bbd8917c WL#3984 (Revise locking of mysql.general_log and mysql.slow_log)
Bug#25422 (Hang with log tables)
Bug 17876 (Truncating mysql.slow_log in a SP after using cursor locks the
          thread)
Bug 23044 (Warnings on flush of a log table)
Bug 29129 (Resetting general_log while the GLOBAL READ LOCK is set causes
           a deadlock)

Prior to this fix, the server would hang when performing concurrent
ALTER TABLE or TRUNCATE TABLE statements against the LOG TABLES,
which are mysql.general_log and mysql.slow_log.

The root cause traces to the following code:
in sql_base.cc, open_table()
  if (table->in_use != thd)
  {
    /* wait_for_condition will unlock LOCK_open for us */
    wait_for_condition(thd, &LOCK_open, &COND_refresh);
  }
The problem with this code is that the current implementation of the
LOGGER creates 'fake' THD objects, like
- Log_to_csv_event_handler::general_log_thd
- Log_to_csv_event_handler::slow_log_thd
which are not associated to a real thread running in the server,
so that waiting for these non-existing threads to release table locks
cause the dead lock.

In general, the design of Log_to_csv_event_handler does not fit into the
general architecture of the server, so that the concept of general_log_thd
and slow_log_thd has to be abandoned:
- this implementation does not work with table locking
- it will not work with commands like SHOW PROCESSLIST
- having the log tables always opened does not integrate well with DDL
operations / FLUSH TABLES / SET GLOBAL READ_ONLY

With this patch, the fundamental design of the LOGGER has been changed to:
- always open and close a log table when writing a log
- remove totally the usage of fake THD objects
- clarify how locking of log tables is implemented in general.

See WL#3984 for details related to the new locking design.

Additional changes (misc bugs exposed and fixed):

1)

mysqldump which would ignore some tables in dump_all_tables_in_db(),
 but forget to ignore the same in dump_all_views_in_db().

2)

mysqldump would also issue an empty "LOCK TABLE" command when all the tables
to lock are to be ignored (numrows == 0), instead of not issuing the query.

3)

Internal errors handlers could intercept errors but not warnings
(see sql_error.cc).

4)

Implementing a nested call to open tables, for the performance schema tables,
exposed an existing bug in remove_table_from_cache(), which would perform:
  in_use->some_tables_deleted=1;
against another thread, without any consideration about thread locking.
This call inside remove_table_from_cache() was not required anyway,
since calling mysql_lock_abort() takes care of aborting -- cleanly -- threads
that might hold a lock on a table.
This line (in_use->some_tables_deleted=1) has been removed.
2007-07-27 00:31:06 -06:00
gkodinov/kgeorge@magare.gmz
bef15b279b Bug #26418: Slave out of sync after
CREATE/DROP TEMPORARY TABLE + ROLLBACK on master

The transaction ability of the storage engines of
the tables on the replication master and the replication
slave must generally be the same.
When the storage engine type of the slave is 
non-transactional then transactions on the master that 
mix update of transactional and non-transactional tables
should be avoided because they will cause inconsistency of
the data between the master's transactional table and the
slave's non-transactional table.

The effect described by this bug is actually expected.
A detailed test case is added (to be merged later to
the updated rpl_ddl.test), as there was no coverage 
by the existing tests. 
Some code cleanup is also added by this change.
2007-06-19 14:27:53 +03:00
jani@linux-th5m.site
f1807e1f25 Merge jamppa@bk-internal.mysql.com:/home/bk/mysql-5.1
into  linux-th5m.site:/home/my/mysql-5.1-marvel
2007-05-24 15:26:10 +03:00
svoj@mysql.com/june.mysql.com
e3f7947da0 BUG#28341 - Security issue still in library loading
UDF can be created from any library in any part of the server
LD_LIBRARY_PATH.

Allow to load udfs only from plugin_dir.
On windows, refuse to open udf in case it's path contains a slash.

No good test case for this bug because of imperfect error message
that includes error code and error string when it fails to dlopen a
library.
2007-05-18 16:23:46 +05:00
monty@mysql.com/narttu.mysql.fi
088e2395f1 WL#3817: Simplify string / memory area types and make things more consistent (first part)
The following type conversions was done:

- Changed byte to uchar
- Changed gptr to uchar*
- Change my_string to char *
- Change my_size_t to size_t
- Change size_s to size_t

Removed declaration of byte, gptr, my_string, my_size_t and size_s. 

Following function parameter changes was done:
- All string functions in mysys/strings was changed to use size_t
  instead of uint for string lengths.
- All read()/write() functions changed to use size_t (including vio).
- All protocoll functions changed to use size_t instead of uint
- Functions that used a pointer to a string length was changed to use size_t*
- Changed malloc(), free() and related functions from using gptr to use void *
  as this requires fewer casts in the code and is more in line with how the
  standard functions work.
- Added extra length argument to dirname_part() to return the length of the
  created string.
- Changed (at least) following functions to take uchar* as argument:
  - db_dump()
  - my_net_write()
  - net_write_command()
  - net_store_data()
  - DBUG_DUMP()
  - decimal2bin() & bin2decimal()
- Changed my_compress() and my_uncompress() to use size_t. Changed one
  argument to my_uncompress() from a pointer to a value as we only return
  one value (makes function easier to use).
- Changed type of 'pack_data' argument to packfrm() to avoid casts.
- Changed in readfrm() and writefrom(), ha_discover and handler::discover()
  the type for argument 'frmdata' to uchar** to avoid casts.
- Changed most Field functions to use uchar* instead of char* (reduced a lot of
  casts).
- Changed field->val_xxx(xxx, new_ptr) to take const pointers.

Other changes:
- Removed a lot of not needed casts
- Added a few new cast required by other changes
- Added some cast to my_multi_malloc() arguments for safety (as string lengths
  needs to be uint, not size_t).
- Fixed all calls to hash-get-key functions to use size_t*. (Needed to be done
  explicitely as this conflict was often hided by casting the function to
  hash_get_key).
- Changed some buffers to memory regions to uchar* to avoid casts.
- Changed some string lengths from uint to size_t.
- Changed field->ptr to be uchar* instead of char*. This allowed us to
  get rid of a lot of casts.
- Some changes from true -> TRUE, false -> FALSE, unsigned char -> uchar
- Include zlib.h in some files as we needed declaration of crc32()
- Changed MY_FILE_ERROR to be (size_t) -1.
- Changed many variables to hold the result of my_read() / my_write() to be
  size_t. This was needed to properly detect errors (which are
  returned as (size_t) -1).
- Removed some very old VMS code
- Changed packfrm()/unpackfrm() to not be depending on uint size
  (portability fix)
- Removed windows specific code to restore cursor position as this
  causes slowdown on windows and we should not mix read() and pread()
  calls anyway as this is not thread safe. Updated function comment to
  reflect this. Changed function that depended on original behavior of
  my_pwrite() to itself restore the cursor position (one such case).
- Added some missing checking of return value of malloc().
- Changed definition of MOD_PAD_CHAR_TO_FULL_LENGTH to avoid 'long' overflow.
- Changed type of table_def::m_size from my_size_t to ulong to reflect that
  m_size is the number of elements in the array, not a string/memory
  length.
- Moved THD::max_row_length() to table.cc (as it's not depending on THD).
  Inlined max_row_length_blob() into this function.
- More function comments
- Fixed some compiler warnings when compiled without partitions.
- Removed setting of LEX_STRING() arguments in declaration (portability fix).
- Some trivial indentation/variable name changes.
- Some trivial code simplifications:
  - Replaced some calls to alloc_root + memcpy to use
    strmake_root()/strdup_root().
  - Changed some calls from memdup() to strmake() (Safety fix)
  - Simpler loops in client-simple.c
2007-05-10 12:59:39 +03:00
msvensson@pilot.blaudden
a4ee7302e6 Merge pilot.blaudden:/home/msvensson/mysql/my51-m-mysql_upgrade
into  pilot.blaudden:/home/msvensson/mysql/mysql-5.1-maint
2007-04-24 11:11:45 +02:00
gluh@mysql.com/eagle.(none)
2d47f0cb1b Bug#21432 Database/Table name limited to 64 bytes, not chars, problems with multi-byte 2007-04-03 16:13:27 +05:00
cbell/Chuck@mysql_cab_desk.
a591514a67 Merge cbell@bk-internal.mysql.com:/home/bk/mysql-5.1-new-rpl
into  mysql_cab_desk.:C:/source/c++/mysql-5.1-new-rpl
2007-03-29 09:54:59 -04:00
serg@janus.mylan
68fd66e853 wl#3700 - post-review fixes:
s/ulonglong/key_part_map/, comments
2007-03-17 00:13:25 +01:00
cbell/Chuck@mysql_cab_desk.
3e44599c11 WL#3629 - Replication of Invocation and Invoked Features
This changeset adds replication of events and user-defined functions. 
There are several bug reports involved in this change:

BUG#16421, BUG#17857, BUG#20384:
This patch modifies the mysql.events table to permit the addition of
another enum value for the status column. The column now has values
of ('DISABLED','SLAVESIDE_DISABLED','ENABLED'). A status of
SLAVESIDE_DISABLED is set on the slave during replication of events.
This enables users to determine which events werereplicated from the 
master and to later enable them if they promote the slave to a master.
The CREATE, ALTER, and DROP statements are binlogged.
A new test was added for replication of events (rpl_events).

BUG#17671:
This patch modifies the code to permit logging of user-defined functions.
Note: this is the CREATE FUNCTION ... SONAME variety. A more friendly error 
message to be displayed should a replicated user-defined function not be
found in the loadable library or if the library is missing from the
slave.The CREATE andDROP statements are binlogged. A new test was added 
for replication of user-defined functions (rpl_udf). 

The patch also adds a new column to the mysql.event table named
'originator' that is used to store the server_id of the server that
the event originated on. This enables users to promote a slave to a 
master and later return the promoted slave to a slave and disable the
replicated events.
2007-03-16 09:56:57 -04:00
msvensson@pilot.blaudden
189f592551 Bug#24248 Problem updating from 5.0.15 to 5.1.12 running under Win XP SP2
- Fix error message to say that "mysql_upgrade" should be run when
 a particular error message is found to be missing
 - Dont's set default value on blob's - it will not be used anyway.
2007-03-08 15:43:54 +01:00
serg@janus.mylan
a07fd5fa8f WL#3700: Handler API change: all index search methods - that is,
index_read(), index_read_idx(), index_read_last(), and
records_in_range() - instead of 'uint keylen' argument take
'ulonglong keypart_map', a bitmap showing which keyparts are
present in the key value.
Fallback method is provided for handlers that are lagging behind.
2007-01-29 10:40:26 +01:00
gkodinov/kgeorge@macbook.local
a30c09c33f Merge macbook.local:/Users/kgeorge/mysql/work/mysql-5.0-opt
into  macbook.local:/Users/kgeorge/mysql/work/merge-5.1-opt
2007-01-08 12:32:48 +02:00
gkodinov/kgeorge@rakia.gmz
47c044e418 Merge bk-internal:/home/bk/mysql-5.0-opt
into  rakia.gmz:/home/kgeorge/mysql/work/B15439-5.0-opt
2007-01-03 18:29:39 +02:00
gkodinov/kgeorge@rakia.gmz
770608cf26 sql_udf.cc:
Addendum to the fix for bug#15439 : valgrind pb failed
2007-01-03 18:26:39 +02:00
holyfoot/hf@mysql.com/hfmain.(none)
79361c655a Merge mysql.com:/d2/hf/common/my50-common
into  mysql.com:/d2/hf/opt/my50-opt
2007-01-03 11:13:01 +04:00
kent@kent-amd64.(none)
be15e3bc15 Merge mysql.com:/home/kent/bk/main/mysql-5.0
into  mysql.com:/home/kent/bk/main/mysql-5.1
2006-12-23 20:20:40 +01:00
kent@mysql.com/kent-amd64.(none)
226a5c833f Many files:
Changed header to GPL version 2 only
2006-12-23 20:17:15 +01:00
gkodinov/kgeorge@macbook.gmz
d56d3a6971 Bug #15439: UDF name case handling forces DELETE FROM mysql.func to remove
the UDF
When deleting a user defined function MySQL must remove it from both the
in-memory hash table and the mysql.proc system table.
Finding (and removal therefore) from the internal hash table is case 
insensitive (or whatever the default charset is), whereas finding and 
removal from the system table is case sensitive.
As a result if you supply a function name that is not in the same character
case to DROP FUNCTION the server will remove the function only from the
in-memory hash table and will keep the row in mysql.proc system table.
This will cause inconsistency between the two structures (that is fixed
only by restarting the server).
Fixed by using the name in the precise case (from the in-memory hash table)
to delete the row in the mysql.proc system table.
2006-12-15 11:38:30 +02:00
kostja@bodhi.local
04c97488f9 Merge bodhi.local:/opt/local/work/tmp_merge
into  bodhi.local:/opt/local/work/mysql-5.1-runtime-merge
2006-08-12 21:06:51 +04:00
kostja@bodhi.local
73189969f3 Merge bodhi.local:/opt/local/work/tmp_merge
into  bodhi.local:/opt/local/work/mysql-5.1-runtime-merge
2006-07-26 23:33:25 +04:00
msvensson@neptunus.(none)
c83b9a5c1e Merge bk-internal:/home/bk/mysql-5.0-maint
into  neptunus.(none):/home/msvensson/mysql/mysql-5.0-maint
2006-07-18 09:32:49 +02:00
msvensson@neptunus.(none)
0efd88bd25 Merge bk-internal:/home/bk/mysql-5.0-maint
into  neptunus.(none):/home/msvensson/mysql/mysql-5.0-maint
2006-07-10 12:48:09 +02:00
kostja@bodhi.local
2ab1e065e4 Fix compiler warnings in sql_udf.h: ISO C++ forbids casting
between pointer to function and pointer to object.
2006-07-09 13:03:51 +04:00
konstantin@bodhi.netgear
01bc761690 Merge bodhi.netgear:/opt/local/work/tmp_merge
into  bodhi.netgear:/opt/local/work/mysql-5.1-runtime-merge-with-5.0
2006-07-06 22:55:48 +04:00
konstantin@mysql.com
117b76a562 A fix and a test case for
Bug#19022 "Memory bug when switching db during trigger execution"
 Bug#17199 "Problem when view calls function from another database."
 Bug#18444 "Fully qualified stored function names don't work correctly in
            SELECT statements"

 Documentation note: this patch introduces a change in behaviour of prepared
 statements.

 This patch adds a few new invariants with regard to how THD::db should
 be used. These invariants should be preserved in future:

  - one should never refer to THD::db by pointer and always make a deep copy
    (strmake, strdup)
  - one should never compare two databases by pointer, but use strncmp or
    my_strncasecmp
  - TABLE_LIST object table->db should be always initialized in the parser or
    by creator of the object.

    For prepared statements it means that if the current database is changed
    after a statement is prepared, the database that was current at prepare
    remains active. This also means that you can not prepare a statement that
    implicitly refers to the current database if the latter is not set.
    This is not documented, and therefore needs documentation. This is NOT a
    change in behavior for almost all SQL statements except:
     - ALTER TABLE t1 RENAME t2 
     - OPTIMIZE TABLE t1
     - ANALYZE TABLE t1
     - TRUNCATE TABLE t1 --
     until this patch t1 or t2 could be evaluated at the first execution of
     prepared statement. 

     CURRENT_DATABASE() still works OK and is evaluated at every execution
     of prepared statement.

     Note, that in stored routines this is not an issue as the default
     database is the database of the stored procedure and "use" statement
     is prohibited in stored routines.

  This patch makes obsolete the use of check_db_used (it was never used in the
  old code too) and all other places that check for table->db and assign it
  from THD::db if it's NULL, except the parser.

 How this patch was created: THD::{db,db_length} were replaced with a
 LEX_STRING, THD::db. All the places that refer to THD::{db,db_length} were
 manually checked and:
  - if the place uses thd->db by pointer, it was fixed to make a deep copy
  - if a place compared two db pointers, it was fixed to compare them by value
    (via strcmp/my_strcasecmp, whatever was approproate)
 Then this intermediate patch was used to write a smaller patch that does the
 same thing but without a rename.

 TODO in 5.1:
   - remove check_db_used
   - deploy THD::set_db in mysql_change_db

 See also comments to individual files.
2006-06-27 00:47:52 +04:00
msvensson@neptunus.(none)
2e0cad5404 Bug#20159 Inconsistant generation of generic error messages
- Use hardcoded error message instead of ER_GET_ERRNO
2006-06-07 14:07:11 +02:00
monty@mysql.com
74cc73d461 This changeset is largely a handler cleanup changeset (WL#3281), but includes fixes and cleanups that was found necessary while testing the handler changes
Changes that requires code changes in other code of other storage engines.
(Note that all changes are very straightforward and one should find all issues
by compiling a --debug build and fixing all compiler errors and all
asserts in field.cc while running the test suite),

- New optional handler function introduced: reset()
  This is called after every DML statement to make it easy for a handler to
  statement specific cleanups.
  (The only case it's not called is if force the file to be closed)

- handler::extra(HA_EXTRA_RESET) is removed. Code that was there before
  should be moved to handler::reset()

- table->read_set contains a bitmap over all columns that are needed
  in the query.  read_row() and similar functions only needs to read these
  columns
- table->write_set contains a bitmap over all columns that will be updated
  in the query. write_row() and update_row() only needs to update these
  columns.
  The above bitmaps should now be up to date in all context
  (including ALTER TABLE, filesort()).

  The handler is informed of any changes to the bitmap after
  fix_fields() by calling the virtual function
  handler::column_bitmaps_signal(). If the handler does caching of
  these bitmaps (instead of using table->read_set, table->write_set),
  it should redo the caching in this code. as the signal() may be sent
  several times, it's probably best to set a variable in the signal
  and redo the caching on read_row() / write_row() if the variable was
  set.

- Removed the read_set and write_set bitmap objects from the handler class

- Removed all column bit handling functions from the handler class.
  (Now one instead uses the normal bitmap functions in my_bitmap.c instead
  of handler dedicated bitmap functions)

- field->query_id is removed. One should instead instead check
  table->read_set and table->write_set if a field is used in the query.

- handler::extra(HA_EXTRA_RETRIVE_ALL_COLS) and
  handler::extra(HA_EXTRA_RETRIEVE_PRIMARY_KEY) are removed. One should now
  instead use table->read_set to check for which columns to retrieve.

- If a handler needs to call Field->val() or Field->store() on columns
  that are not used in the query, one should install a temporary
  all-columns-used map while doing so. For this, we provide the following
  functions:

  my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->read_set);
  field->val();
  dbug_tmp_restore_column_map(table->read_set, old_map);

  and similar for the write map:

  my_bitmap_map *old_map= dbug_tmp_use_all_columns(table, table->write_set);
  field->val();
  dbug_tmp_restore_column_map(table->write_set, old_map);

  If this is not done, you will sooner or later hit a DBUG_ASSERT
  in the field store() / val() functions.
  (For not DBUG binaries, the dbug_tmp_restore_column_map() and
  dbug_tmp_restore_column_map() are inline dummy functions and should
  be optimized away be the compiler).

- If one needs to temporary set the column map for all binaries (and not
  just to avoid the DBUG_ASSERT() in the Field::store() / Field::val()
  methods) one should use the functions tmp_use_all_columns() and
  tmp_restore_column_map() instead of the above dbug_ variants.

- All 'status' fields in the handler base class (like records,
  data_file_length etc) are now stored in a 'stats' struct. This makes
  it easier to know what status variables are provided by the base
  handler.  This requires some trivial variable names in the extra()
  function.

- New virtual function handler::records().  This is called to optimize
  COUNT(*) if (handler::table_flags() & HA_HAS_RECORDS()) is true.
  (stats.records is not supposed to be an exact value. It's only has to
  be 'reasonable enough' for the optimizer to be able to choose a good
  optimization path).

- Non virtual handler::init() function added for caching of virtual
  constants from engine.

- Removed has_transactions() virtual method. Now one should instead return
  HA_NO_TRANSACTIONS in table_flags() if the table handler DOES NOT support
  transactions.

- The 'xxxx_create_handler()' function now has a MEM_ROOT_root argument
  that is to be used with 'new handler_name()' to allocate the handler
  in the right area.  The xxxx_create_handler() function is also
  responsible for any initialization of the object before returning.

  For example, one should change:

  static handler *myisam_create_handler(TABLE_SHARE *table)
  {
    return new ha_myisam(table);
  }

  ->

  static handler *myisam_create_handler(TABLE_SHARE *table, MEM_ROOT *mem_root)
  {
    return new (mem_root) ha_myisam(table);
  }

- New optional virtual function: use_hidden_primary_key().
  This is called in case of an update/delete when
  (table_flags() and HA_PRIMARY_KEY_REQUIRED_FOR_DELETE) is defined
  but we don't have a primary key. This allows the handler to take precisions
  in remembering any hidden primary key to able to update/delete any
  found row. The default handler marks all columns to be read.

- handler::table_flags() now returns a ulonglong (to allow for more flags).

- New/changed table_flags()
  - HA_HAS_RECORDS	    Set if ::records() is supported
  - HA_NO_TRANSACTIONS	    Set if engine doesn't support transactions
  - HA_PRIMARY_KEY_REQUIRED_FOR_DELETE
                            Set if we should mark all primary key columns for
			    read when reading rows as part of a DELETE
			    statement. If there is no primary key,
			    all columns are marked for read.
  - HA_PARTIAL_COLUMN_READ  Set if engine will not read all columns in some
			    cases (based on table->read_set)
 - HA_PRIMARY_KEY_ALLOW_RANDOM_ACCESS
   			    Renamed to HA_PRIMARY_KEY_REQUIRED_FOR_POSITION.
 - HA_DUPP_POS              Renamed to HA_DUPLICATE_POS
 - HA_REQUIRES_KEY_COLUMNS_FOR_DELETE
			    Set this if we should mark ALL key columns for
			    read when when reading rows as part of a DELETE
			    statement. In case of an update we will mark
			    all keys for read for which key part changed
			    value.
  - HA_STATS_RECORDS_IS_EXACT
			     Set this if stats.records is exact.
			     (This saves us some extra records() calls
			     when optimizing COUNT(*))
			    

- Removed table_flags()
  - HA_NOT_EXACT_COUNT     Now one should instead use HA_HAS_RECORDS if
			   handler::records() gives an exact count() and
			   HA_STATS_RECORDS_IS_EXACT if stats.records is exact.
  - HA_READ_RND_SAME	   Removed (no one supported this one)

- Removed not needed functions ha_retrieve_all_cols() and ha_retrieve_all_pk()

- Renamed handler::dupp_pos to handler::dup_pos

- Removed not used variable handler::sortkey


Upper level handler changes:

- ha_reset() now does some overall checks and calls ::reset()
- ha_table_flags() added. This is a cached version of table_flags(). The
  cache is updated on engine creation time and updated on open.


MySQL level changes (not obvious from the above):

- DBUG_ASSERT() added to check that column usage matches what is set
  in the column usage bit maps. (This found a LOT of bugs in current
  column marking code).

- In 5.1 before, all used columns was marked in read_set and only updated
  columns was marked in write_set. Now we only mark columns for which we
  need a value in read_set.

- Column bitmaps are created in open_binary_frm() and open_table_from_share().
  (Before this was in table.cc)

- handler::table_flags() calls are replaced with handler::ha_table_flags()

- For calling field->val() you must have the corresponding bit set in
  table->read_set. For calling field->store() you must have the
  corresponding bit set in table->write_set. (There are asserts in
  all store()/val() functions to catch wrong usage)

- thd->set_query_id is renamed to thd->mark_used_columns and instead
  of setting this to an integer value, this has now the values:
  MARK_COLUMNS_NONE, MARK_COLUMNS_READ, MARK_COLUMNS_WRITE
  Changed also all variables named 'set_query_id' to mark_used_columns.

- In filesort() we now inform the handler of exactly which columns are needed
  doing the sort and choosing the rows.

- The TABLE_SHARE object has a 'all_set' column bitmap one can use
  when one needs a column bitmap with all columns set.
  (This is used for table->use_all_columns() and other places)

- The TABLE object has 3 column bitmaps:
  - def_read_set     Default bitmap for columns to be read
  - def_write_set    Default bitmap for columns to be written
  - tmp_set          Can be used as a temporary bitmap when needed.
  The table object has also two pointer to bitmaps read_set and write_set
  that the handler should use to find out which columns are used in which way.

- count() optimization now calls handler::records() instead of using
  handler->stats.records (if (table_flags() & HA_HAS_RECORDS) is true).

- Added extra argument to Item::walk() to indicate if we should also
  traverse sub queries.

- Added TABLE parameter to cp_buffer_from_ref()

- Don't close tables created with CREATE ... SELECT but keep them in
  the table cache. (Faster usage of newly created tables).


New interfaces:

- table->clear_column_bitmaps() to initialize the bitmaps for tables
  at start of new statements.

- table->column_bitmaps_set() to set up new column bitmaps and signal
  the handler about this.

- table->column_bitmaps_set_no_signal() for some few cases where we need
  to setup new column bitmaps but don't signal the handler (as the handler
  has already been signaled about these before). Used for the momement
  only in opt_range.cc when doing ROR scans.

- table->use_all_columns() to install a bitmap where all columns are marked
  as use in the read and the write set.

- table->default_column_bitmaps() to install the normal read and write
  column bitmaps, but not signaling the handler about this.
  This is mainly used when creating TABLE instances.

- table->mark_columns_needed_for_delete(),
  table->mark_columns_needed_for_delete() and
  table->mark_columns_needed_for_insert() to allow us to put additional
  columns in column usage maps if handler so requires.
  (The handler indicates what it neads in handler->table_flags())

- table->prepare_for_position() to allow us to tell handler that it
  needs to read primary key parts to be able to store them in
  future table->position() calls.
  (This replaces the table->file->ha_retrieve_all_pk function)

- table->mark_auto_increment_column() to tell handler are going to update
  columns part of any auto_increment key.

- table->mark_columns_used_by_index() to mark all columns that is part of
  an index.  It will also send extra(HA_EXTRA_KEYREAD) to handler to allow
  it to quickly know that it only needs to read colums that are part
  of the key.  (The handler can also use the column map for detecting this,
  but simpler/faster handler can just monitor the extra() call).

- table->mark_columns_used_by_index_no_reset() to in addition to other columns,
  also mark all columns that is used by the given key.

- table->restore_column_maps_after_mark_index() to restore to default
  column maps after a call to table->mark_columns_used_by_index().

- New item function register_field_in_read_map(), for marking used columns
  in table->read_map. Used by filesort() to mark all used columns

- Maintain in TABLE->merge_keys set of all keys that are used in query.
  (Simplices some optimization loops)

- Maintain Field->part_of_key_not_clustered which is like Field->part_of_key
  but the field in the clustered key is not assumed to be part of all index.
  (used in opt_range.cc for faster loops)

-  dbug_tmp_use_all_columns(), dbug_tmp_restore_column_map()
   tmp_use_all_columns() and tmp_restore_column_map() functions to temporally
   mark all columns as usable.  The 'dbug_' version is primarily intended
   inside a handler when it wants to just call Field:store() & Field::val()
   functions, but don't need the column maps set for any other usage.
   (ie:: bitmap_is_set() is never called)

- We can't use compare_records() to skip updates for handlers that returns
  a partial column set and the read_set doesn't cover all columns in the
  write set. The reason for this is that if we have a column marked only for
  write we can't in the MySQL level know if the value changed or not.
  The reason this worked before was that MySQL marked all to be written
  columns as also to be read. The new 'optimal' bitmaps exposed this 'hidden
  bug'.

- open_table_from_share() does not anymore setup temporary MEM_ROOT
  object as a thread specific variable for the handler. Instead we
  send the to-be-used MEMROOT to get_new_handler().
  (Simpler, faster code)



Bugs fixed:

- Column marking was not done correctly in a lot of cases.
  (ALTER TABLE, when using triggers, auto_increment fields etc)
  (Could potentially result in wrong values inserted in table handlers
  relying on that the old column maps or field->set_query_id was correct)
  Especially when it comes to triggers, there may be cases where the
  old code would cause lost/wrong values for NDB and/or InnoDB tables.

- Split thd->options flag OPTION_STATUS_NO_TRANS_UPDATE to two flags:
  OPTION_STATUS_NO_TRANS_UPDATE and OPTION_KEEP_LOG.
  This allowed me to remove some wrong warnings about:
  "Some non-transactional changed tables couldn't be rolled back"

- Fixed handling of INSERT .. SELECT and CREATE ... SELECT that wrongly reset
  (thd->options & OPTION_STATUS_NO_TRANS_UPDATE) which caused us to loose
  some warnings about
  "Some non-transactional changed tables couldn't be rolled back")

- Fixed use of uninitialized memory in ha_ndbcluster.cc::delete_table()
  which could cause delete_table to report random failures.

- Fixed core dumps for some tests when running with --debug

- Added missing FN_LIBCHAR in mysql_rm_tmp_tables()
  (This has probably caused us to not properly remove temporary files after
  crash)

- slow_logs was not properly initialized, which could maybe cause
  extra/lost entries in slow log.

- If we get an duplicate row on insert, change column map to read and
  write all columns while retrying the operation. This is required by
  the definition of REPLACE and also ensures that fields that are only
  part of UPDATE are properly handled.  This fixed a bug in NDB and
  REPLACE where REPLACE wrongly copied some column values from the replaced
  row.

- For table handler that doesn't support NULL in keys, we would give an error
  when creating a primary key with NULL fields, even after the fields has been
  automaticly converted to NOT NULL.

- Creating a primary key on a SPATIAL key, would fail if field was not
  declared as NOT NULL.


Cleanups:

- Removed not used condition argument to setup_tables

- Removed not needed item function reset_query_id_processor().

- Field->add_index is removed. Now this is instead maintained in
  (field->flags & FIELD_IN_ADD_INDEX)

- Field->fieldnr is removed (use field->field_index instead)

- New argument to filesort() to indicate that it should return a set of
  row pointers (not used columns). This allowed me to remove some references
  to sql_command in filesort and should also enable us to return column
  results in some cases where we couldn't before.

- Changed column bitmap handling in opt_range.cc to be aligned with TABLE
  bitmap, which allowed me to use bitmap functions instead of looping over
  all fields to create some needed bitmaps. (Faster and smaller code)

- Broke up found too long lines

- Moved some variable declaration at start of function for better code
  readability.

- Removed some not used arguments from functions.
  (setup_fields(), mysql_prepare_insert_check_table())

- setup_fields() now takes an enum instead of an int for marking columns
   usage.

- For internal temporary tables, use handler::write_row(),
  handler::delete_row() and handler::update_row() instead of
  handler::ha_xxxx() for faster execution.

- Changed some constants to enum's and define's.

- Using separate column read and write sets allows for easier checking
  of timestamp field was set by statement.

- Remove calls to free_io_cache() as this is now done automaticly in ha_reset()

- Don't build table->normalized_path as this is now identical to table->path
  (after bar's fixes to convert filenames)

- Fixed some missed DBUG_PRINT(.."%lx") to use "0x%lx" to make it easier to
  do comparision with the 'convert-dbug-for-diff' tool.


Things left to do in 5.1:

- We wrongly log failed CREATE TABLE ... SELECT in some cases when using
  row based logging (as shown by testcase binlog_row_mix_innodb_myisam.result)
  Mats has promised to look into this.

- Test that my fix for CREATE TABLE ... SELECT is indeed correct.
  (I added several test cases for this, but in this case it's better that
  someone else also tests this throughly).
  Lars has promosed to do this.
2006-06-04 18:52:22 +03:00
msvensson@neptunus.(none)
dc749e9ad7 Bug#11835 CREATE FUNCTION crashes server
- Update for 5.1
 - Add "have_dlopen"
 - Remove requirement to load udf's only in "opt_plugin_dir"
2006-03-21 10:54:26 +01:00
msvensson@neptunus.(none)
0811648e31 Merge neptunus.(none):/home/msvensson/mysql/mysql-5.0
into  neptunus.(none):/home/msvensson/mysql/bug11835/my50-bug11835
2006-03-20 21:35:05 +01:00