Commit graph

1184 commits

Author SHA1 Message Date
Michael Widenius
3c78bfe7f1 Added progress reporting for alter table, LOAD DATA INFILE and for aria tables: check table, repair table, analyze table.
- The client gets a progress report message that triggers a callback function if requested with mysql_options(MYSQL_PROGRESS_CALLBACK, function)
- Added Progress field last to 'show processlist'
- Stage, Max_stage and Progress field added to information_schema.progresslist
- The 'mysql' client by defaults enables progress reports when the output is a tty.
- Added progress_report_time time variable to configure how often progress reports is sent to client
Added read only system variable 'in_transaction' which is 1 if we have executed a BEGIN statement.


client/client_priv.h:
  Added OPT_REPORT_PROGRESS
client/mysql.cc:
  Added option --progress-reports (on by default if not batch mode)
  Progress reports is written to stdout for long running commands
include/Makefile.am:
  Added mysql/service_progress_report.h
include/myisamchk.h:
  Added variables to be able to do progress reporting in Aria and later in MyISAM
include/mysql.h:
  Added new mysql_options() parameter: MYSQL_PROGRESS_CALLBACK
include/mysql.h.pp:
  Added new mysql_options() parameter: MYSQL_PROGRESS_CALLBACK
include/mysql/plugin.h:
  Added functions for reporting progress.
include/mysql/plugin_auth.h.pp:
  Added functions for reporting progress.
include/mysql_com.h:
  Added CLIENT_PROGRESS mysql_real_connect() flag.
include/sql_common.h:
  Added callback function for reporting progress
mysql-test/r/old-mode.result:
  Ensure that SHOW PROGRESSLIST doesn't have the Progress column in old mode.
mysql-test/suite/funcs_1/datadict/datadict_priv.inc:
  Added new column
mysql-test/suite/funcs_1/datadict/processlist_priv.inc:
  Test all new PROCESSLIST columns
mysql-test/suite/funcs_1/r/is_columns_is.result:
  Updated results
mysql-test/suite/funcs_1/r/is_columns_is_embedded.result:
  Updated results
mysql-test/suite/funcs_1/r/is_columns_mysql_embedded.result:
  Updated results
mysql-test/suite/funcs_1/r/is_tables_is_embedded.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_priv_no_prot.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_priv_ps.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_val_no_prot.result:
  Updated results
mysql-test/suite/funcs_1/r/processlist_val_ps.result:
  Updated results
mysql-test/suite/pbxt/r/pbxt_locking.result:
  Updated results
mysql-test/suite/pbxt/r/skip_name_resolve.result:
  Updated results
mysql-test/t/old-mode.test:
  Ensure that SHOW PROGRESSLIST doesn't have the Progress column in old mode.
plugin/handler_socket/handlersocket/Makefile.am:
  Added -lmysqlservices
scripts/mytop.sh:
  Made 'State' field width dynamic.
  Added 'Progress' to process list display.
sql-common/client.c:
  Added handling of progress messages.
  Removed check_license() function.
sql/mysql_priv.h:
  Added opt_progress_report_time
sql/mysqld.cc:
  Added progress_report_time time variable to configure how often progress reports is sent to client
sql/protocol.cc:
  Added net_send_progress_packet()
sql/protocol.h:
  New prototypes
sql/set_var.cc:
  Added variables progress_report_time and in_transaction
sql/sql_acl.cc:
  Safety fix: Made client_capabilities ulonglong
sql/sql_class.cc:
  Added interface functions for progress reporting
sql/sql_class.h:
  Added varibles in THD for progress reporting.
  Added CF_REPORT_PROGRESS
sql/sql_load.cc:
  Added progress reporting for LOAD DATA INFILE
sql/sql_parse.cc:
  Added CF_REPORT_PROGRESS for top level commands for which it's safe to send progress reports to client
sql/sql_show.cc:
  Added Progress field last to 'show processlist'
  Stage, Max_stage and Progress field added to information_schema.progresslist
sql/sql_table.cc:
  Added progress reporting for ALTER TABLE
  Added THD as argument to copy_data_between_tables()
storage/maria/ha_maria.cc:
  Added progress reporting for check table, repair table, analyze table
  Fixed a bug in start_bulk_insert() that caused alter table to always run with all keys enabled.
storage/maria/ma_check.c:
  Added progress reporting
  Remember old state before starting repair. This removes some warnings from optimize_table if create-with-sort fails.
storage/maria/ma_check_standalone.h:
  Added dummy reporting function for standalone Aria programs.
storage/maria/ma_sort.c:
  Added progress reporting
storage/maria/maria_chk.c:
  Updated version
storage/maria/maria_def.h:
  Added new prototypes
tests/mysql_client_test.c:
  Added test case for progress reporting
2011-07-01 15:08:30 +03:00
Michael Widenius
5e876bd300 Aria bug fixes and improvements:
- Fixed performance bug in alter table with Aria; Aria didn't use disable keys + enable keys
- Fixed wrong warning about 'Wrong CRC on datapage' from REPAIR TABLE with aria block tables.
- Fixed bug in aria_chk that disabled performance counters.
- Added --translog_buffer_size to maria_read_log.

storage/maria/ha_maria.cc:
  Fixed performance bug in alter table with Aria; Aria didn't use disable keys + enable keys
storage/maria/ma_bitmap.c:
  Added some DBUG_ASSERT()'s and made code more uniform
storage/maria/ma_check.c:
  Fixed wrong warning about 'Wrong CRC on datapage' from REPAIR TABLE with aria block tables.
storage/maria/ma_pagecache.c:
  Copy read pages to buffer even if there was an error (to be able to detect zero filled pages)
storage/maria/maria_chk.c:
  Fixed bug in aria_chk that disabled performance counters.
storage/maria/maria_read_log.c:
  Added option to set translog_buffer_size
2011-06-30 00:37:12 +03:00
Michael Widenius
ba9a890f0c New status variables: Rows_tmp_read, Handler_tmp_update and Handler_tmp_write
Split status variable Rows_read to Rows_read and Rows_tmp_read so that one can see how much real data is read.
Same was done with with Handler_update and Handler_write.
Fixed bug in MEMORY tables where some variables was counted twice.
Added new internal handler call 'ha_close()' to have one place to gather statistics.
Fixed bug where thd->open_options was set to wrong value when doing admin_recreate_table()


mysql-test/r/status.result:
  Updated test results and added new tests
mysql-test/r/status_user.result:
  Udated test results
mysql-test/t/status.test:
  Added new test for temporary table status variables
sql/ha_partition.cc:
  Changed to call ha_close() instead of close()
sql/handler.cc:
  Added internal_tmp_table variable for easy checking of temporary tables.
  Added new internal handler call 'ha_close()' to have one place to gather statistics.
  Gather statistics for internal temporary tables.
sql/handler.h:
  Added handler variables internal_tmp_table, rows_tmp_read.
  Split function update_index_statistics() to two.
  Added ha_update_tmp_row() for faster tmp table handling with more statistics.
sql/item_sum.cc:
  ha_write_row() -> ha_write_tmp_row()
sql/multi_range_read.cc:
  close() -> ha_close()
sql/mysqld.cc:
  New status variables: Rows_tmp_read, Handler_tmp_update and Handler_tmp_write
sql/opt_range.cc:
  close() -> ha_close()
sql/sql_base.cc:
  close() -> ha_close()
sql/sql_class.cc:
  Added handling of rows_tmp_read
sql/sql_class.h:
  Added new satistics variables.
  rows_read++  ->  update_rows_read() to be able to correctly count reads to internal temp tables.
  Added handler::ha_update_tmp_row()
sql/sql_connect.cc:
  Added comment
sql/sql_expression_cache.cc:
  ha_write_row() -> ha_write_tmp_row()
sql/sql_select.cc:
  close() -> ha_close()
  ha_update_row() -> ha_update_tmp_row()
sql/sql_show.cc:
  ha_write_row() -> ha_write_tmp_row()
sql/sql_table.cc:
  Fixed bug where thd->open_options was set to wrong value when doing admin_recreate_table()
sql/sql_union.cc:
  ha_write_row() -> ha_write_tmp_row()
sql/sql_update.cc:
  ha_write_row() -> ha_write_tmp_row()
sql/table.cc:
  close() -> ha_close()
storage/heap/ha_heap.cc:
  Removed double counting of statistic variables.
  close -> ha_close() to get tmp table statistics.
storage/maria/ha_maria.cc:
  close -> ha_close() to get tmp table statistics.
2011-06-27 19:07:24 +03:00
Sergey Petrunya
7880039fc0 Merge 5.2 -> 5.3
(testcase for #798597 now crashes)
2011-06-24 21:43:31 +04:00
Sergey Petrunya
9f4739a3fa Fix compile failure 2011-06-24 16:43:25 +04:00
Michael Widenius
424e9a888d Fixed several errors in Aria discovered by test case for lp:727869 ma_pagecache.c:2103: find_block: Assertion `block->rlocks == 0
- Fixed assert in transaction log handler when aria_check was run on block-record table that was much bigger than expected.
- Fixed warnings about wrong mutex order between bitmap and intern_lock
- Fixed error in bitmap that could cause two rows to use same block for a block record.
- Fixed wrong test that could cause error if last page for a bitmap was used by a blob.
- Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks (very unlikely case).


mysql-test/suite/maria/r/maria-recovery3.result:
  Updated results
sql/mysqld.cc:
  Allow mi_check() to send information messages for log file
storage/maria/ma_bitmap.c:
  Fixed problem with wrong mutex order when bitmap was the first page that was flushed out of page cache
  - Fixed by introducing _ma_bitmap_mark_file_changed() that marks file changed without a bitmap lock.
  - Fixed one case in _ma_change_bitmap_page() where we didn't mark the bitmap changed. This could cause to rows to reuse same block if this was the only change to the bitmap.
  - Split _ma_bitmap_get_page_bits() in two parts to not take a bitmap lock when we already have it
  - Fixed bug in _ma_bitmap_set_full_page_bits() that caused an error if last page for a bitmap was used by a blob
storage/maria/ma_check.c:
  Better handling of wrong file length.
  Fixed bug when we tried to write to transaction log when it was not opened (happened when block record file was bigger than expected)
storage/maria/ma_pagecache.c:
  Fixed several bugs in pagecache for the case where pagecase had very few blocks and there was a lot of threads competing to get the blocks:
  - In link_block() mark a block given to another thread with PCBLOCK_REASSIGNED to ensure that no other threads can start re-using the block
    before the thread that requsted a block.
  - In free_block(), don't reset status for a block that is in re-assign by link_block() (we don't want to loose the PCBLOCK_REASSIGNED flag).
  - Added call to wait_for_flush() when we got a new block in find_block() to ensure that we don't use a block that is beeing flushed by another thread.
  - Moved setting of hits_left and last_hit_time in find_block() to where we assign the block.
  
  
  Code cleanup and making code uniform:
  - Changed a lot of KEYCACHE_DBUG_PRINT to use DBUG_PRINT
  - Streamlined all reporting of 'signal' and 'wait' between threads to be identical.
  - Use thread name instead of thread number (for each match against --debug)
  - Added more DBUG_ENTER, DBUG_PRINT and DBUG_ASSERT()
  - Added more comments
storage/myisam/ha_myisam.cc:
  Only print information about that we make a backup if we are really making a backup.
storage/myisam/mi_check.c:
  Inform mysqld that we are creating a backup of the data file (for inclusion in error log).
2011-06-24 12:08:45 +03:00
Michael Widenius
5407f511c2 Fixes to aria
- Fixed error when writing a blob to the last page on the bitmap.
- Marked bitmap changed in once case that could cause two rows to use the same blob page.
2011-06-24 10:10:50 +03:00
Igor Babaev
078b59f5bc Merge of mwl #106 into 5.3. 2011-06-15 21:48:38 -07:00
Michael Widenius
bf0e1f44c6 Fixed portability problem with partiton_error.test
Added option to aria_read_log to crash recovery at certain points in the recovery process.
Fixed bug that caused future recovery attempts to fail if we got a crash/got killed during closing of tables at end of recovery process.


mysql-test/mysql-test-run.pl:
  Don't abort if 'var' points to stale place; Just remove it.
mysql-test/suite/maria/r/maria.result:
  Fixed wrong indentation
mysql-test/t/partition_error.test:
  Fixed portability problem with partiton_error.test
storage/maria/ma_close.c:
  More DBUG_PRINT info
storage/maria/ma_pagecache.c:
  Copy flush_log_callback when writing to page cache. This fixes problem in recovery when switching from mode of file
storage/maria/ma_recovery.c:
  Added option to aria_read_log to crash recovery at certain points in the recovery process.
storage/maria/ma_recovery.h:
  Added option to aria_read_log to crash recovery at certain points in the recovery process.
storage/maria/maria_chk.c:
  Align aria_chk -d output
  Don't write warning Aria table '...' is usable but should be fixed if the table was before marked as crashed but now is ok
storage/maria/maria_read_log.c:
  Added option to aria_read_log to crash recovery at certain points in the recovery process.
2011-06-13 12:46:11 +03:00
Vladislav Vaintroub
4171483b53 Backport Fix for Bug#24509 - 2048 file descriptor limit on windows needs increasing.
The patch replaces the use of the POSIX I/O interfaces in mysys on Windows with 
the Win32 API calls (CreateFile, WriteFile, etc). The Windows HANDLE for the open
 file is stored in the my_file_info struct, along with a flag for append mode 
(because the Windows API does not support opening files in append mode in all cases)
The default max open files has been increased to 16384 and can be increased further
by setting --max-open-files=<value> during the server start.

Noteworthy benefit of this patch is that it removes limits from the table_cache size - 
allowing for more simultaneus users
2011-06-12 15:52:07 +02:00
Michael Widenius
e5403ec95a Fixed build failures with maria3.test and widows build
mysql-test/suite/maria/t/maria3.test:
  Fix when compiling with safemalloc
storage/maria/ha_maria.cc:
  Fix that MAX_FILE_SIZE is not limited on 32 bit machines.
2011-06-11 11:41:46 +03:00
Michael Widenius
3516b4b5d1 Merge with 5.2 2011-06-10 12:09:21 +03:00
Michael Widenius
13f55aac67 A bit better fix for tmp-table problem
Fixed reference to not initialized memory detected by valgrind

sql/sql_select.cc:
  A bit better fix for tmp-table problem:
  Use only dynamic_record format for group by and distinct.
storage/maria/ma_create.c:
  DYNAMIC_RECORD format doesn't pack VARCHAR fields.
  This change fixes a non-fatal uninitialized memory copy.
2011-06-10 02:02:58 +03:00
Michael Widenius
9e3570ea6c Removed depricated --maria-options from mysqld. 2011-06-09 21:39:31 +03:00
Igor Babaev
f03a3ee54f Merged the code of mwl 106 into the latest 5.3 with mwl 90 pushed.
Resolved all conflicts and failures.
2011-06-04 19:56:06 -07:00
Vladislav Vaintroub
b519f2b626 Fix compile errors and warnings and test errors introduced by microseconds push.
Also, change windows timespec definition to be Unix-ish - simplifies handling a lot.
2011-05-28 16:57:58 +02:00
Michael Widenius
f197991f41 Merge with 5.1-microseconds
A lot of small fixes and new test cases.

client/mysqlbinlog.cc:
  Cast removed
client/mysqltest.cc:
  Added missing DBUG_RETURN
include/my_pthread.h:
  set_timespec_time_nsec() now only takes one argument
mysql-test/t/date_formats.test:
  Remove --disable_ps_protocl as now also ps supports microseconds
mysys/my_uuid.c:
  Changed to use my_interval_timer() instead of my_getsystime()
mysys/waiting_threads.c:
  Changed to use my_hrtime()
sql/field.h:
  Added bool special_const_compare() for fields that may convert values before compare (like year)
sql/field_conv.cc:
  Added test to get optimal copying of identical temporal values.
sql/item.cc:
  Return that item_int is equal if it's positive, even if unsigned flag is different.
  Fixed Item_cache_str::save_in_field() to have identical null check as other similar functions
  Added proper NULL check to Item_cache_int::save_in_field()
sql/item_cmpfunc.cc:
  Don't call convert_constant_item() if there is nothing that is worth converting.
  Simplified test when years should be converted
sql/item_sum.cc:
  Mark cache values in Item_sum_hybrid as not constants to ensure they are not replaced by other cache values in compare_datetime()
sql/item_timefunc.cc:
  Changed sec_to_time() to take a my_decimal argument to ensure we don't loose any sub seconds.
  Added Item_temporal_func::get_time() (This simplifies some things)
sql/mysql_priv.h:
  Added Lazy_string_decimal()
sql/mysqld.cc:
  Added my_decimal constants max_seconds_for_time_type, time_second_part_factor
sql/table.cc:
  Changed expr_arena to be of type CONVENTIONAL_EXECUTION to ensure that we don't loose any items that are created by fix_fields()
sql/tztime.cc:
  TIME_to_gmt_sec() now sets *in_dst_time_gap in case of errors
  This is needed to be able to detect if timestamp is 0
storage/maria/lockman.c:
  Changed from my_getsystime() to set_timespec_time_nsec()
storage/maria/ma_loghandler.c:
  Changed from my_getsystime() to my_hrtime()
storage/maria/ma_recovery.c:
  Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/maria/unittest/trnman-t.c:
  Changed from my_getsystime() to mmicrosecond_interval_timer()
storage/xtradb/handler/ha_innodb.cc:
  Added support for new time,datetime and timestamp
unittest/mysys/thr_template.c:
  my_getsystime() -> my_interval_timer()
unittest/mysys/waiting_threads-t.c:
  my_getsystime() -> my_interval_timer()
2011-05-28 05:11:32 +03:00
Igor Babaev
89cf840cd9 Merge 2011-05-20 12:47:39 -07:00
Vladislav Vaintroub
2c1c18ed86 Fix compile warning. 2011-05-18 12:28:17 +02:00
Igor Babaev
704f97035f Merged the code of MWL#106 into 5.3
Resolved all conflicts, bad merges and fixed a few minor bugs in the code.
Commented out the queries from multi_update, view, subselect_sj, func_str,
derived_view, view_grant that failed either with crashes in ps-protocol or
with wrong results.
The failures are clear indications of some bugs in the code and these bugs
are to be fixed.
2011-05-16 22:39:43 -07:00
unknown
157a945318 Fix of compiler errors.
storage/maria/ma_loghandler.c:
  Awoiding signed and unsigned comporison.
storage/maria/ma_loghandler.h:
  debug function.
storage/maria/unittest/ma_loghandler_examples.c:
  debug function.
2011-05-16 16:13:34 +03:00
Michael Widenius
f458e198c0 Merge with 5.2 2011-05-16 14:05:45 +03:00
Michael Widenius
8c5e18c9cf Added more asserts to find out how maria.maria-big could have failed in buildbot.
storage/maria/ma_loghandler.c:
  Added assert if we give wrong type to translog_write_record().
  (This should be impossible, but the failure record in buildbot also looks impossible so we need more information...)
storage/maria/ma_loghandler.h:
  Added prototype for check_translog_description_table()
storage/maria/unittest/ma_loghandler_examples.c:
  Call check_translog_description_table() to register number of log_record_type_descriptor[] events.
2011-05-16 12:16:08 +03:00
Michael Widenius
5933619ceb Made test-unit run in parlallel. This was achived by having all aria tests that uses temporary files to create the temporary file in an unique tempdirectory.
aria_chk now returns 1 if one got any warnings during check and 2 if one got errors.
lp:728919 maria_chk should fail on all detected corruptions

mysql-test/lib/My/SafeProcess.pm:
  Removed ^M lines from the file
storage/maria/ma_check.c:
  Fixed indentation
storage/maria/maria_chk.c:
  aria_chk now returns 1 if one got any warnings during check and 2 if one got errors.
storage/maria/unittest/ma_control_file-t.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_maria_log_cleanup.c:
  Added help function create_tmpdir()
storage/maria/unittest/ma_pagecache_consist.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_pagecache_rwconsist.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_pagecache_rwconsist2.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_pagecache_single.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_test_all-t:
  Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler-t.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_first_lsn-t.c:
  Use unique sub directory for test results
  remove delete of files that are already deleted by maria_log_remove()
storage/maria/unittest/ma_test_loghandler_max_lsn-t.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_noflush-t.c:
  Use unique sub directory for test results
  remove delete of files that are already deleted by maria_log_remove()
storage/maria/unittest/ma_test_loghandler_nologs-t.c:
  Use unique sub directory for test results
storage/maria/unittest/ma_test_loghandler_pagecache-t.c:
  Use unique sub directory for test results
  remove delete of files that are already deleted by maria_log_remove()
storage/maria/unittest/ma_test_loghandler_purge-t.c:
  Use unique sub directory for test results
support-files/compiler_warnings.supp:
  Supress some warnings
unittest/unit.pl:
  Run 4 unittests in parallel
2011-05-13 16:22:05 +03:00
Michael Widenius
f34be18938 Merge with MariaDB 5.2 2011-05-10 18:17:43 +03:00
Michael Widenius
28f708e1e3 Automatic merge with 5.1 2011-05-08 12:37:38 +03:00
Michael Widenius
d5caa00161 Improved 'make test-unit' time slightly
storage/maria/unittest/ma_test_loghandler-t.c:
  Don't sync during test
storage/maria/unittest/ma_test_loghandler_multigroup-t.c:
  Don't sync during test
storage/maria/unittest/ma_test_loghandler_multithread-t.c:
  Don't sync during test
unittest/mysys/bitmap-t.c:
  Don't test all bit combinations (not needed)
unittest/mysys/thr_template.c:
  Remove sleep as old bug should be fixed nowadays
unittest/mysys/waiting_threads-t.c:
  Only run test with --big
unittest/mytap/tap.c:
  Print total time at end of test.
unittest/unit.pl:
  Use TAP::Harness instead of Test::Harness (recommended according to manual)
  Add times to tests.
2011-05-05 14:51:01 +03:00
Michael Widenius
1be5462d59 Merge with MariaDB 5.1 2011-05-03 19:10:10 +03:00
Michael Widenius
e415ba0fb2 Merge with MySQL 5.1.57/58
Moved some BSD string functions from Unireg
2011-05-02 20:58:45 +03:00
Michael Widenius
35352d4da6 mysql-test/suite/funcs_1/r/storedproc.result:
Fixed test failure
mysql-test/suite/funcs_1/t/storedproc.test:
  Fixed test failure
  (The error changed as syntax for alter changed with ALTER ONLINE)
storage/maria/ha_maria.cc:
  fixed compiler warning
2011-05-02 13:25:53 +03:00
Michael Widenius
2721e912ba Added ALTER ONLINE TABLE, which will give an error if the change can't be done 'instantly' (without a table copy)
mysql-test/r/alter_table_online.result:
  Test new feature
mysql-test/t/alter_table_online.test:
  Test new feature
sql/handler.cc:
  Added comment
sql/lex.h:
  Added ONLINE keyword
sql/mysql_priv.h:
  Added option to alter table to require online operation
sql/share/errmsg.txt:
  Added error message if ONLINE can't be done
sql/sql_lex.h:
  Added online option
sql/sql_parse.cc:
  Added online option to mysql_alter_table()
sql/sql_table.cc:
  Added test that gives error if table can't be done instantly when requsted to be online.
  Fixed wrong test if table includes a VARCHAR
  Fixed wrong (but unlikely) handling of error conditions in ALTER table
sql/sql_yacc.yy:
  Added ALTER ONLINE TABLE syntax
storage/maria/ha_maria.cc:
  Fixed bug where 'start_bulk_insert' used too small buffer if used with unknown number of rows
2011-04-28 19:56:10 +03:00
Michael Widenius
6da8ac5f71 Added option "AND DISABLE CHECKPOINT" to "FLUSH TABLES WITH READ LOCK"
This makes it possible to do safe multi volume snapshots as long as one snapshots the volume with the transaction logs last.


include/mysql_com.h:
  Added REFRESH_CHECKPOINT
mysql-test/r/flush.result:
  Added test of new FLUSH TABLES syntax + calls to checkpoint_status handler calls
mysql-test/t/flush.test:
  Added test of new FLUSH TABLES syntax + calls to checkpoint_status handler calls
sql/handler.cc:
  Added code to call checkpoint_state for all handlertons that supports it
sql/handler.h:
  Added new checkpoint_state() handlerton call to temporarly disable checkpoints.
sql/lex.h:
  Added CHECKPOINT keyword
sql/sql_yacc.yy:
  Added support for FLUSH TABLES WITH READ LOCK AND DISABLE CHECKPOINT
storage/maria/ha_maria.cc:
  Added handlerton call to disable checkpoints.
storage/maria/ma_checkpoint.c:
  Don't do checkpoint if checkpoints are disabled.
storage/maria/ma_static.c:
  Added maria_checkpoint_disabled
storage/maria/maria_def.h:
  Added maria_checkpoint_disabled
storage/xtradb/handler/ha_innodb.cc:
  Added handlerton call to disable checkpoints.
storage/xtradb/include/log0log.h:
  Added option to log_checkpoint() to allow one to ignore not critical checkpoints during the time checkpoints are disabled.
storage/xtradb/log/log0log.c:
  Added code to allow one to disable checkpoints during a FLUSH TABLES ... DISABLE CHECKPOINT
  This was done by adding a new argument to log_checkpoint() which tells us when the checkpoint is called by srv_master_thread (which are safe to ignore)
storage/xtradb/srv/srv0srv.c:
  Tell log_checkpoint() that checkpoints from srv_master_thread() are safe to ignore (will just delay recovery time a bit).
2011-04-28 18:02:26 +03:00
Sergei Golubchik
0accbd0364 lots of post-merge changes 2011-04-25 17:22:25 +02:00
Sergey Petrunya
997445bc8e Make EXPLAIN better at displaying MRR/BKA:
- "Using MRR" is no longer shown with range access.
- Instead, both range and BKA accesses will show one of the following:
  = "Rowid-ordered scan"
  = "Key-ordered scan"
  = "Key-ordered Rowid-ordered scan"
depending on whether DS-MRR implementation will do scan keys in order, rowids in order,
or both.
- The patch also introduces a way for other storage engines/MRR implementations to
  pass information to EXPLAIN output about the properties of employed MRR scans.
2011-04-02 14:04:45 +04:00
Vladislav Vaintroub
75625f5652 merge mwl#55 2011-03-11 15:20:24 +01:00
Vladislav Vaintroub
41d432462b merge 5.2 2011-03-10 09:39:14 +01:00
Michael Widenius
139a2b64bf Merge with 5.2 2011-03-09 15:47:59 +02:00
Sergey Petrunya
e6bd643c75 MRR interface: change range_info's type from char* to range_id_t typedef. The goals are:
- cleaner code
- ability to change from using pointers to offsets at some point
2011-03-04 12:06:03 +03:00
Michael Widenius
9f2194090c Fixed wrong filenames in maria unittest that caused unittest to fail
mysql-test/suite/sphinx/suite.pm:
  Update test to support 1.10-beta
storage/maria/unittest/ma_test_all-t:
  Fixed old filenames
2011-03-01 13:37:29 +02:00
Michael Widenius
3358cdd504 Merge with 5.1 to get in changes from MySQL 5.1.55 2011-02-28 19:39:30 +02:00
Michael Widenius
869f5d0e81 Merge with alias as String 2011-02-28 13:16:17 +02:00
Michael Widenius
ff3da0f963 Change TABLE->alias to String for less memory reallocation
Changed some String.ptr() -> String.c_ptr() for String that are not guaranteed to end with \0
Removed some c_ptr() usage from parameters to functions that takes ptr & length
Use preallocate buffers to avoid calling malloc() for most operations. 


sql/event_db_repository.cc:
  alias is now a String
sql/event_scheduler.cc:
  c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
sql/events.cc:
  c_ptr -> c_ptr_safe() to avoid warnings from valgrind.
  c_ptr -> ptr() as function takes ptr & length
sql/field.cc:
  alias is now a String
sql/field.h:
  alias is now a String
sql/ha_partition.cc:
  alias is now a String
sql/handler.cc:
  alias is now a String
  ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/item.cc:
  Store error parameter in separarte buffer to ensure correct error message
sql/item_func.cc:
  ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
sql/item_sum.h:
  Use my_strtod() instead of my_atof() to not have to make string \0 terminated
sql/lock.cc:
  alias is now a String
sql/log.cc:
  c_ptr() -> ptr() as function takes ptr & length
sql/log_event.cc:
  c_ptr_quick() -> ptr() as we only want to get the pointer to String buffer
sql/opt_range.cc:
  ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/opt_table_elimination.cc:
  alias is now a String
sql/set_var.cc:
  ptr() -> c_ptr() as string is not guaranteed to be \0 terminated
  c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
  c_ptr() -> ptr() as function takes ptr & length
  Simplify some code.
sql/sp.cc:
  c_ptr() -> ptr() as function takes ptr & length
sql/sp_rcontext.cc:
  alias is now a String
sql/sql_base.cc:
  alias is now a String.
  Here we win a realloc() for most alias usage.
sql/sql_class.cc:
  Use size descriptor for printf() to avoid accessing bytes outside of buffer
sql/sql_insert.cc:
  Change allocation of TABLE as it's now contains a String
  _ptr() -> ptr() as function takes ptr & length
sql/sql_load.cc:
  Use preallocate buffers to avoid calling malloc() for most operations.
sql/sql_parse.cc:
  Use c_ptr_safe() to ensure string is \0 terminated.
sql/sql_plugin.cc:
  c_ptr_quick() -> ptr() as function takes ptr & length
sql/sql_select.cc:
  alias is now a String
sql/sql_show.cc:
  alias is now a String
sql/sql_string.h:
  Added move() function to change who owns the string (owner does the free)
sql/sql_table.cc:
  alias is now a String
  c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
sql/sql_test.cc:
  c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
  alias is now a String
sql/sql_trigger.cc:
  c_ptr() -> c_ptr_safe() to avoid warnings from valgrind.
  Use field->init() to setup pointers to alias.
sql/sql_update.cc:
  alias is now a String
sql/sql_view.cc:
  ptr() -> c_ptr_safe() as string is not guaranteed to be \0 terminated
sql/sql_yacc.yy:
  r() -> c_ptr() as string is not guaranteed to be \0 terminated
sql/table.cc:
  alias is now a String
sql/table.h:
  alias is now a String
storage/federatedx/ha_federatedx.cc:
  Remove extra 1 byte alloc that is automaticly done by strmake()
  Ensure that error message ends with \0
storage/maria/ha_maria.cc:
  alias is now a String
storage/myisam/ha_myisam.cc:
  alias is now a String
2011-02-28 12:48:50 +02:00
Michael Widenius
07b5d554ee - maria/ma_test_all.sh can now be run with --tmpdir=/dev/shm for faster testing
- Fixed mysql-test-run failures on window
- Fixed compiler warnings from my last push (sorry about that)
- Fixed that maria_chk --repair --extended works again
- Fixed compiler warnings about using not unitialized data

mysql-test/mysql-test-run.pl:
  Better output
mysql-test/suite/parts/inc/partition_check_drop.inc:
  Use remove_files_wildcard instead of rm
mysys/safemalloc.c:
  Fixed argument to printf
storage/maria/ma_cache.c:
  Don't give errors when running maria_chk
storage/maria/ma_dynrec.c:
  Don't give errors when running maria_chk
storage/maria/ma_rt_test.c:
  Added option --datadir for where to put logs and test data
storage/maria/ma_test1.c:
  Added option --datadir for where to put logs and test data
storage/maria/ma_test2.c:
  Added option --datadir for where to put logs and test data
storage/maria/maria_chk.c:
  If --datadir is used but --logdir is not, set --logdir from --datadir
  (this reflects how --help said how things should work)
storage/maria/maria_read_log.c:
  Changed short option for 'maria-log-dir-path' from -l to -h to be same as mysqld, maria_chk, ma_test1 etc..
storage/maria/unittest/ma_test_all-t:
  Allow one to specify --tmpdir for where to store logs and data
storage/xtradb/buf/buf0buf.c:
  Fixed compiler warnings about using not unitialized data
storage/xtradb/row/row0upd.c:
  Fixed compiler warnings about using not unitialized data
storage/xtradb/srv/srv0srv.c:
  Fixed compiler warnings about using not unitialized data
2011-02-25 14:55:40 +02:00
Michael Widenius
4ad6d6b6c9 Merge in new handler and handlersocket code into 5.3 main 2011-02-22 11:15:47 +02:00
Michael Widenius
58bb0769bd Merge with MySQL 5.1.55
- Fixed some issues with partitions and connection_string, which also fixed lp:716890 "Pre- and post-recovery crash in Aria"
- Fixed wrong assert in Aria

Now need to merge with latest xtradb before pushing 

sql/ha_partition.cc:
  Ensure that m_ordered_rec_buffer is not freed before close.
sql/mysqld.cc:
  Changed to use opt_stack_trace instead of opt_pstack.
  Removed references to pstack
sql/partition_element.h:
  Ensure that connect_string is initialized
storage/maria/ma_key_recover.c:
  Fixed wrong assert
2011-02-20 18:51:43 +02:00
Michael Widenius
2813f568b7 Merge with bugfix
sql/multi_range_read.cc:
  Added printing of error if something goes wrong in get_next()
  (Not critical for this bug fix, but this was something that I noticed while testing and found missing)
storage/myisam/mi_rkey.c:
  Fixed wrong error number in mi_yield_and_check_if_killed()
2011-02-18 21:45:32 +02:00
Michael Widenius
b12e3796dc Fix for lp:711565 "Index Condition Pushdown can make a thread hold MyISAM locks as well as be unKILLable for long time"
- In Maria/MyISAM: Release/re-acquire locks to give queries that wait on them a chance to make progress
- In Maria/MyISAM: Change from numeric constants to ICP_RES values.
- In Maria: Do check index condition in maria_rprev() (was lost in the merge/backport?)
- In Maria/MyISAM/XtraDB: Check if the query was killed, and return immediately if it was.

Added new storage engine error: HA_ERR_ABORTED_BY_USER, for handler to signal that it detected a kill of the query and aborted

Authors: Sergey Petrunia & Monty

include/my_base.h:
  Added HA_ERR_ABORTED_BY_USER, for handler to signal that it detected a kill of the query and aborted
include/my_handler.h:
  Added comment
mysql-test/r/myisam_icp.result:
  Updated test
mysql-test/t/myisam_icp.test:
  Drop used tables at start of test
  Added test case that can help with manual testing of killing index condition pushdown query.
mysys/my_handler_errors.h:
  Text for new storage engine error
sql/handler.cc:
  If engine got HA_ERR_ABORTED_BY_USER, send kill message.
sql/multi_range_read.cc:
  Return error code
storage/maria/ha_maria.cc:
  Added ma_killed_in_mariadb() to detect kill.
  Ensure that file->external_ref points to TABLE object.
storage/maria/ma_extra.c:
  Dummy test-if-killed for standalone
storage/maria/ma_key.c:
  If ma_check_index_cond() fails, set my_errno and info->cur_row.lastpos
storage/maria/ma_rkey.c:
  Release/re-acquire locks to give queries that wait on them a chance to make progress
  Check if the query was killed, and return immediately if it was
storage/maria/ma_rnext.c:
  Check if the query was killed, and return immediately if it was
  Added missing fast_ma_writeinfo(info)
storage/maria/ma_rnext_same.c:
  Check if the query was killed, and return immediately if it was
  Added missing fast_ma_writeinfo(info)
storage/maria/ma_rprev.c:
  Check if the query was killed, and return immediately if it was
  Added missing fast_ma_writeinfo(info) and ma_check_index_cond()
storage/maria/ma_search.c:
  Give error message if we find a wrong key
storage/maria/ma_static.c:
  Added pointer to test-if-killed function
storage/maria/maria_def.h:
  New prototypes
storage/myisam/ha_myisam.cc:
  Added mi_killed_in_mariadb()
  Ensure that file->external_ref points to TABLE object.
storage/myisam/mi_extra.c:
  Dummy test-if-killed for standalone
storage/myisam/mi_key.c:
  If ma_check_index_cond() fails, set my_errno and info->lastpos
storage/myisam/mi_rkey.c:
  Ensure that info->lastpos= HA_OFFSET_ERROR in case of error
  Release/re-acquire locks to give queries that wait on them a chance to make progress
  Check if the query was killed, and return immediately if it was
  Reorder code to do less things in case of error.
  Added missing fast_mi_writeinfo()
storage/myisam/mi_rnext.c:
  Check if the query was killed, and return immediately if it was
  Simplify old ICP code
  Added missing fast_ma_writeinfo(info)
storage/myisam/mi_rnext_same.c:
  Check if the query was killed, and return immediately if it was
  Added missing fast_mi_writeinfo(info)
storage/myisam/mi_rprev.c:
  Check if the query was killed, and return immediately if it was
  Simplify error handling of ICP
  Added missing fast_mi_writeinfo(info)
storage/myisam/mi_search.c:
  Give error message if we find a wrong key
storage/myisam/mi_static.c:
  Added pointer to test-if-killed function
storage/myisam/myisamdef.h:
  New prototypes
storage/xtradb/handler/ha_innodb.cc:
  Added DB_SEARCH_ABORTED_BY_USER and ha_innobase::is_thd_killed()
  Check if the query was killed, and return immediately if it was
storage/xtradb/handler/ha_innodb.h:
  Added prototype
storage/xtradb/include/db0err.h:
  Added DB_SEARCH_ABORTED_BY_USER
storage/xtradb/include/row0mysql.h:
  Added possible ICP errors
storage/xtradb/row/row0sel.c:
  Use ICP errors instead of constants.
  Detect if killed and return B_SEARCH_ABORTED_BY_USER
2011-02-18 17:43:59 +02:00
Vladislav Vaintroub
18ad13ef81 Fix Aria engine build
- add forgotten source file
2011-02-12 15:43:24 +01:00
Vladislav Vaintroub
4c4c0226a0 Fix build : declaration in the middle of statement in C file. 2011-02-10 21:04:29 +01:00
Michael Widenius
6db663d614 Fixes for Aria storage engine:
- Fixed bug lp:624099 ma_close.c:75: maria_close: Assertion `share->in_trans == 0' failed on UNLOCK TABLES
- Fixed bug that caused table to be marked as not closed (crashed) during recovery testing.
- Use maria_delete_table_files() instead of maria_delete_table() to delete tempoary tables (faster and safer)
- Added checks to ensure that bitmap and internal mutex are always taken in right order.
- For transactional tables, only mark the table as changed before page for table is to be written to disk (and thus the log is flushed).
  This speeds up things a bit and fixes a problem where open_count was incremented on disk but there was no log entry to fix it during recovery -> table was crashed.
- Fixed a bug in repair() where table was not automaticly repaired.
- Ensure that state->global_changed, share->changed and share->state.open_count are set and reset properly.
- Added option --ignore-control-file to maria_chk to be able to run maria_chk even if the control file is locked.


mysql-test/suite/maria/r/maria-recover.result:
  Test result changed as we now force checkpoint before copying table, which results in pagecache getting flushed and we have more rows to recover.
mysql-test/suite/maria/r/maria.result:
  Added new tests
mysql-test/suite/maria/t/maria-recover.test:
  Force checkpoint before copying table.
  This is needed as now the open-count is increased first when first page is flushed.
mysql-test/suite/maria/t/maria.test:
  Added tests to verify fix for lp:624099
storage/maria/ha_maria.cc:
  Use table->in_use instead of current_thd (trivial optimization)
  Use maria_delete_table_files() instead of maria_delete_table() to delete tempoary tables (faster and safer)
  More DBUG_ASSERT()
  Reset locked tables count after locked tables have been moved to new transaction. This fixed lp:624099
storage/maria/ma_bitmap.c:
  Temporarly unlock bitmap mutex when calling _ma_mark_file_changed() and pagecache_delete_pages() to ensure right mutex lock order.
  Call _ma_bitmap_unpin_all() when bitmap->non_flusable is set to 0. This fixed a case when bitmap was not proparly unpinned.
  More comments
  Added DBUG_ASSERT() for detecting wrong share->bitmap usage
storage/maria/ma_blockrec.c:
  More DBUG_ASSERT()
  Moved code around in _ma_apply_redo_insert_row_head_or_tail() to make things safer on error conditions.
storage/maria/ma_check.c:
  Changed parameter for _ma_set_uuid()
  Corrected test for detecting if we lost many rows. This fixed some cases where auto-recovery failed.
  share->changed need to be set if state.open_count is changed.
  Removed setting of share->changed= 0 as called function sets it.
storage/maria/ma_close.c:
  - Added code to properly decrement open_count and have it written by _ma_state_info_write() for transactional tables.
    (This is more correct and also saves us one extra write by _ma_decrement_open_count() at close.
  - Added DBUG_ASSERT() to detect if open_count is wrong at maria_close().
storage/maria/ma_delete.c:
  Updated argument to _ma_mark_file_changed()
storage/maria/ma_delete_all.c:
  Updated argument to _ma_mark_file_changed()
  For transactional tables, call _ma_mark_file_changed() after log entry has been written (to allow recover to fix open_count)
  Reset more needed variables in _ma_reset_status()
storage/maria/ma_delete_table.c:
  Moved deletion of Aria files to maria_delete_table_files().
  Remove RAID usage (old not working code)
storage/maria/ma_extra.c:
  Set share->changed=1 when state needs to be updated on disk.
  Don't reset share->changed after call to _ma_state_info_write() as this calls sets share->changed.
  Set share->state.open_count to 1 to force table to be auto repaired if drop fails.
  Set share->global_changed before call to _ma_flush_table_files() to ensure that we don't try to mark the table changed during flush.
  Added DBUG_ENTER
storage/maria/ma_locking.c:
  Split _ma_mark_file_changed() into two functions to delay marking transactional tables as changed on disk until first disk write.
  Added argument to _ma_decrement_open_count() to tell if we should call ma_lock_database() or not.
  Don't decrement open count for transactional tables during _ma_decrement_open_count(). This will be done during close.
  Changed parameter for _ma_set_uuid()
storage/maria/ma_open.c:
  Set share->open_count_not_zero_on_open if state.open_count is not zero.
  This is needed for DBUG_ASSERT() in maria_close() that is there to enforce that open_count is always 0 at close.
  This test doesn't however work for tables that had open_count != 0 already on disk (ie, crashed tables).
  Enforce right mutex order for share->intern_lock and share->bitmap.bitmap_lock
  Don't set share->changed to 0 if share->state.open_count != 0, as state needs to be be written at close
storage/maria/ma_pagecache.c:
  Moved a bit of code in find_block() to avoid one if.
  More DBUG_ASSERT()
  (I left a comment in the code for Sanja to look at;  The code probably works but we need to check if it's optimal)
storage/maria/ma_pagecrc.c:
  For transactional tables, just before first write to disk, but after log is flushed, mark the file changed.
  This fixes some cases where recovery() did not detect that table was marked as changed and could thus not recover the marker.
storage/maria/ma_recovery.c:
  Set share->changed when share->global_changed is set.
storage/maria/ma_update.c:
  Updated parameter for _ma_mark_file_changed()
storage/maria/ma_write.c:
  Updated parameter for _ma_mark_file_changed()
storage/maria/maria_chk.c:
  Added option --ignore-control-file to be able to run maria_chk even if the control file is locked.
storage/maria/maria_def.h:
  Updated function prototypes.
  Added open_count_not_zero_on_open to MARIA_SHARE.
storage/myisam/ha_myisam.cc:
  current_thd -> table->in_use
2011-02-10 20:33:51 +02:00