Commit graph

6960 commits

Author SHA1 Message Date
Michael Widenius
53d44ad18b Added some fixes that should make MyISAM & Aria REPAIR work with more than 4G records
- If one specifies --force twice to myisamchk and aria_chk, then we will try to finnish the repair even if sort_buffer would be too small.
  This was done by dynamically allocate buffer handler objects as long as memory lasts.
- New option for myisamchk and aria_chk: create-missing-keys
- Changed default size of myisam_sort_buffer_size from 8M to 128M.
- Changed default size of sort_buffer_size in aria_chk from 128M to 256M.
- Increased information in error message about 'sort_buffer_size' beeing to small.
- Print also to 'show warnings' if repair was retried.
- Increased size of internal sort-buffer-readers from 16K to 128K
- Changed printing of 'number of records' to use %ll instead of casting to long
- Changed buffer sizes for myisam and aria to use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines.





include/my_global.h:
  Added MY_ALIGN_DOWN() to get previous alignment (for big memory areas)
include/myisam.h:
  Increased size of types to be able to handle more records
include/myisamchk.h:
  Increased size of types to be able to handle more records
  Added T_FORCE_SORT_MEMORY to force repair to work even if sort_buffer would not be big enough
mysql-test/r/myisam.result:
  Updated result
mysql-test/r/mysqld--help.result:
  Updated result
mysql-test/r/repair.result:
  Updated result
mysql-test/suite/maria/maria.result:
  Added test cases for some fixed bugs in MyISAM to verify that Aria doesn't have them too.
mysql-test/suite/maria/maria.test:
  Added test cases for some fixed bugs in MyISAM to verify that Aria doesn't have them too.
mysql-test/suite/maria/maria3.result:
  Updated result after sort buffer size increase
mysql-test/suite/sys_vars/r/aria_sort_buffer_size_basic.result:
  Updated result after sort buffer size increase
mysql-test/suite/sys_vars/r/myisam_sort_buffer_size_basic.result:
  Updated result after sort buffer size increase
mysql-test/suite/sys_vars/t/myisam_sort_buffer_size_basic.test:
  Updated result after sort buffer size increase
mysql-test/t/myisam.test:
  Fixed error messages to not print system specific data
mysql-test/t/repair.test:
  Fixed error messages to not print system specific data
storage/maria/ha_maria.cc:
  Print also to 'show warnings' if repair was retried
  Changed default size of sort_buffer_size from 128M to 256M (same as in mysqld)
storage/maria/ma_check.c:
  Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/ma_sort.c:
  Increased size of internal sort-buffer-readers from 16K to 128K
  Increased size of types to be able to handle more records
  Added support for T_FORCE_SORT_MEMORY
  Don't allocate too many extra BUFFPEK at a time (they are probably not needed)
  Improved error message for "sort_buffer_size is too small"
  Changed printing of 'number of records' to use %ll instead of casting to long
  Fixed bug where maria_update_key_parts() was called too early.
  Fixed bug in detecting result from read_to_buffer().
  Added 'out of memory' checking when calling 'alloc_dynamic()'.
storage/maria/maria_chk.c:
  Added --create-missing-keys
  If one specifies --force twice then we will try to finnish the repair even if sort_buffer would be too small.
  check_param.sort_buffer_length varialble was used with wrong type.
storage/maria/maria_def.h:
  Increased size of types to be able to handle more records
  Use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines
  Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/maria_ftdump.c:
  Renamed USE_BUFFER_INIT -> PAGE_BUFFER_INIT
storage/maria/maria_read_log.c:
  Use PAGE_BUFFER_INIT for page cache
storage/myisam/ha_myisam.cc:
  Changed default size of myisam_sort_buffer_size from 8M to 128M
storage/myisam/mi_check.c:
  Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisam_ftdump.c:
  Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisamchk.c:
  Added --create-missing-keys
  If one specifies --force twice then we will try to finnish the repair even if sort_buffer would be too small.
  check_param.sort_buffer_length varialble was used with wrong type.
  Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/myisamdef.h:
  Increased SORT_BUFFER_INIT to 64M
  (speeds up repair a lot and most machines have nowadays a lot of memory)
  Use MY_ALIGN_DOWN() to get same number of bytes allocated on different machines
  Renamed USE_BUFFER_INIT -> KEY_BUFFER_INIT
storage/myisam/sort.c:
  Increased size of internal sort-buffer-readers from 16K to 128K
  Increased size of types to be able to handle more records
  Added support for T_FORCE_SORT_MEMORY
  Don't allocate too many extra BUFFPEK at a time (they are probably not needed)
  Improved error message for "sort_buffer_size is too small"
  Changed printing of 'number of records' to use %ll instead of casting to long
  Fixed bug in detecting result from read_to_buffer().
  Added 'out of memory' checking when calling 'alloc_dynamic()'.
2013-05-10 12:32:34 +03:00
Michael Widenius
5333dafa84 Fixed errors and compiler warnings found by buildbot
Solaris fixes:
- Fixed that wait_timeout_func and wait_timeout tests works on solaris
- We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO).
- Fixed that compile-solaris-amd64-debug works (before that we got a wrong ELF class: ELFCLASS64 on linkage)
- Added missing sync_with_master
Other bug fixes:
- Free memory for rpl_global_gtid_binlog_state before exit() to avoid 'accessing uninitalized mutex' error.



BUILD/FINISH.sh:
  Fixed issues on Solaris with ksh
BUILD/compile-solaris-amd64-debug:
  Added missing -m64 flag
configure.cmake:
  We have to compile without NO_ALARM on Solaris as Solaris doesn't support timeouts on sockets with setsockopt(.. SO_RCVTIMEO)
mysql-test/suite/rpl/t/rpl_gtid_mdev4473.test:
  - Added missing sync_with_master (fix by knielsen)
sql-common/client.c:
  Added () to get rid of compiler warning
sql/item_strfunc.cc:
  Fixed compiler warning
sql/log.cc:
  Free memory for static variable rpl_global_gtid_binlog_state before exit()
  - If we are compiling with safemalloc, we would try to call sf_free() for some members after sf_terminate() was called, which would result of trying to access the uninitalized mutex 'sf_mutex'
sql/multi_range_read.cc:
  Fixed compiler warnings of converting double to ulong.
sql/opt_range.cc:
  Fixed compiler warnings of converting double to ulong or uint
  - Better to have all variables that can be number of rows as 'ha_rows'
sql/rpl_gtid.cc:
  Added rpl_binlog_state::free() to be able to free memory for static objects before exit()
sql/rpl_gtid.h:
  Added rpl_binlog_state::free() to be able to free memory for static objects before exit()
sql/set_var.cc:
  Fixed compiler warning
sql/sql_join_cache.cc:
  Fixed compiler warnings of converting double to uint
sql/sql_show.cc:
  Added cast to get rid of compiler warning
sql/sql_statistics.cc:
  Remove code that didn't do anything.
  (store_record() with record[0] is a no-op)
storage/xtradb/os/os0file.c:
  Added  __attribute__ ((unused))
support-files/compiler_warnings.supp:
  Ignore warnings from atomic_add_64_nv
  (was not able to fix this with a cast as the macro is a bit different between systems)
vio/viosocket.c:
  Added more DBUG_PRINT
2013-05-05 21:39:31 +03:00
Vladislav Vaintroub
305e3dea0d MDEV-4398 - remove incorrect fix, replace with correct one - change default to OFF for innodb_use_fallocate 2013-04-19 14:52:05 +02:00
Michael Widenius
8b714e507f Changed the client library to only mark memory as THREAD_SPECIFIC if one has called
mysql_options() with MYSQL_OPT_USE_THREAD_SPECIFIC_MEMORY

include/mysql.h:
  Added MYSQL_OPT_USE_THREAD_SPECIFIC_MEMORY
include/mysql.h.pp:
  Updated file
sql-common/client.c:
  Marked client memory as THREAD_SPECIFIC
sql/event_db_repository.cc:
  Fixed compiler warning
sql/slave.cc:
  Marked client memory as THREAD_SPECIFIC
storage/federatedx/federatedx_io_mysql.cc:
  Marked client memory as THREAD_SPECIFIC
storage/federatedx/ha_federatedx.cc:
  Marked client memory as THREAD_SPECIFIC
storage/sphinx/ha_sphinx.cc:
  Marked client memory as THREAD_SPECIFIC
2013-04-17 23:37:06 +03:00
Michael Widenius
8e52698580 Fixed compiler warnings and a not critical memory leak
sql/keycaches.cc:
  Added free_all_rpl_filters() to be able to free all filters at cleanup
sql/keycaches.h:
  Added prototype
sql/rpl_rli.cc:
  Fixed compiler warning
sql/slave.cc:
  Free all rpl_filters at cleanup
sql/sp.cc:
  Fixed compiler warning when not all struct elements was initialized
sql/sql_acl.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_events_waits_summary.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_ews_global_by_event_name.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_instances.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_file_summary.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_performance_timers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_consumers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_instruments.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_setup_timers.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_sync_instances.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/perfschema/table_threads.cc:
  Fixed compiler warning when not all struct elements was initialized
storage/xtradb/os/os0file.c:
  Fixed compiler warning when not all struct elements was initialized
2013-04-17 22:33:33 +03:00
unknown
585d7bbfb4 Merge 5.5->10.0-base 2013-04-17 09:39:40 +02:00
unknown
0508f766b4 Fixes for stuff seen in buildbot:
- Fix embedded build

 - Backport disable of fallocate, it creates too short ibdata1
   when used with O_DIRECT on old kernels

 - Do not disable innodb during .deb install, we need it for
   mysql.rpl_slave_state table.
2013-04-16 17:36:40 +02:00
unknown
eb95ebf917 Fix race in test case. 2013-04-16 17:33:47 +02:00
unknown
665a31af2b MDEV-26: Global transaction ID. First alpha release.
Merge of 10.0-mdev26 feature tree into 10.0-base.

Global transaction ID is prepended to each event group in the binlog.

Slave connect can request to start from GTID position instead of specifying
file name/offset of master binlog. This facilitates easy switch to a new
master.

Slave GTID state is stored in a table mysql.rpl_slave_state, which can be
InnoDB to get crash-safe slave state.

GTID includes a replication domain ID, allowing to keep track of distinct
positions for each of multiple masters.
2013-04-15 10:55:27 +02:00
Sergei Golubchik
fc17efc297 compiler warnings 2013-04-14 16:48:16 +02:00
Sergei Golubchik
f57ecb7786 5.5 merge 2013-04-14 10:04:07 +02:00
Sergei Golubchik
3ed6b24d82 Linking problem on Windows 2013-04-10 07:25:13 +02:00
Sergei Golubchik
775e82638b put status variables in the proper pluginname_ scope
(but support the scopeless mysql style too).
always output status/system variables in the correct lettercase
2013-04-09 23:27:33 +02:00
Sergei Golubchik
5f293dd694 MDEV-4022 table attributes with sysvar as a default value 2013-04-09 23:27:24 +02:00
Sergei Golubchik
3ad01d00f2 error messages: name the storage engine explicitly,
instead of "used storage engine" and similar changes.
2013-04-09 23:27:07 +02:00
Sergei Golubchik
ffbd15eb32 post-review comments and other minor edits 2013-04-09 16:20:59 +02:00
Sergei Golubchik
3a8e1a2205 mysql-test fixes 2013-04-09 16:19:22 +02:00
Sergei Golubchik
ee92b2114d assisted discovery in federatedx 2013-04-09 16:19:18 +02:00
Sergei Golubchik
60aed41222 fix internal plugin names 2013-04-09 16:19:10 +02:00
Sergei Golubchik
9c089b04b3 TABLE_SHARE::free_frm_image() method to free the memory
allocated by the same allocator as in TABLE_SHARE::read_frm_image()
2013-04-09 16:18:51 +02:00
Sergei Golubchik
02c712aa54 * frm extra2 segment.
* persistent table versions in the extra2
* ha_archive::frm_compare using TABLE_SHARE::tabledef_version
* distinguish between "important" and "optional" extra2 frm values
* write engine-defined attributes (aka "table options") to extra2, not to extra,
  but still read from the old location, if they're found there.
2013-04-09 16:18:33 +02:00
Sergei Golubchik
13b7bcd5d4 sequence engine 2013-04-09 16:17:16 +02:00
Sergei Golubchik
2d788ff5ca test_sql_discovery storage engine
use it to test main test cases with need_full_discover_for_existence > 0
2013-04-09 16:07:35 +02:00
Sergei Golubchik
4853c7192d discovery using sql CREATE TABLE statement 2013-04-09 16:07:17 +02:00
Sergei Golubchik
e06cb31719 CREATE TABLE and frm-less discovering engines.
Now CREATE TABLE does not write the frm file on disk,
if the engine can discover it
2013-04-09 15:57:09 +02:00
Sergei Golubchik
f597a6a694 HA_ERR_TABLE_DEF_CHANGED support in ha_archive 2013-04-09 15:50:55 +02:00
Sergei Golubchik
2481db063f move writing of the frm into init_from_binary_frm_image() 2013-04-09 15:49:21 +02:00
Sergei Golubchik
5d364e53cb handlerton::discover_table_names() can not discover only
"unknown" tables. Duplicates are possible - deal with them.
2013-04-09 15:49:00 +02:00
Sergei Golubchik
e5a323e107 single table discovery: handlerton::discover_table() method.
fixes for need_full_discover_for_existence mode
2013-04-09 15:45:31 +02:00
Sergei Golubchik
163882665e * don't use 1-8 numbers for open_table_error codes, use an enum.
* print "table doesn't exist in engine" when a table doesn't exist in the engine,
  instead of "file not found" (if no file was involved)
* print a complete filename that cannot be found ('t1.MYI', not 't1')
* it's not an error for a DROP if a table doesn't exist in the engine (or some table
  files cannot be found) - if the DROP succeeded regardless
2013-04-09 15:34:17 +02:00
Sergei Golubchik
03dfb2c121 file-based table discovery for discovering storage engines
also:
* move few tests from archive/archive.test to archive/discover.test
* fix (unintentionally) bug#34104, updated result files
2013-04-09 15:31:29 +02:00
Sergei Golubchik
69ba585a1e remove #if MYSQL_VERSION_ID
there is no "upstream" OQGraph version that needs to stay compatible
with different MariaDB releases.
2013-04-07 17:09:05 +02:00
Sergei Golubchik
58e62d701e * move bas_ext from the handler to the handlerton
* provide a default bas_ext value of the empty list
2013-04-07 17:08:49 +02:00
Sergei Golubchik
ccc9238335 s/st_ha_create_information/HA_CREATE_INFO/ 2013-04-07 15:35:39 +02:00
Sergei Golubchik
bbbd7cedf5 my_dir() cleanup
* replace pointer acrobatics with a struct
* make sorting explicit: MY_DONT_SORT -> MY_WANT_SORT
(if you want something to be done - say it. fixes all places where
my_dir() was used without thinking)
* typo s/number_off_files/number_of_files/
* directory_file_name() doesn't need to be extern
* remove #ifdef __BORLANDC__
* ignore '.' and '..' entries
2013-04-07 15:19:45 +02:00
Sergei Golubchik
e56cad4b15 small cleanup 2013-04-07 14:03:43 +02:00
Sergei Golubchik
4ec6fe10e5 remove ULL() and LL(), because they're totally unnecessary
and sometimes harmful (used with expressions)
2013-04-07 14:00:16 +02:00
Vladislav Vaintroub
ed949df15a MDEV-4338 - Support FusionIO/directFS atomic writes 2013-04-06 00:36:10 +02:00
Vladislav Vaintroub
0aa607a01a MDEV-4338 - Support FusionIO/directFS atomic writes 2013-04-06 00:35:45 +02:00
Sergei Golubchik
2901497b18 MDEV-4243 Warnings/errors while compiling with clang 2013-03-28 20:04:14 +01:00
Sergei Golubchik
993ea79f2d 5.5 merge 2013-03-27 23:41:02 +01:00
Sergei Golubchik
6599fd3e9c 5.3 merge 2013-03-27 10:03:28 +01:00
Sergey Petrunya
fb2501e2d4 MDEV-4277: Crash inside mi_killed_in_mariadb() with myisammrg
- Set MI_INFO::external_ref for MyISAM tables that are parts of myisamMRG table.
2013-03-21 11:06:27 +04:00
Sergei Golubchik
e73f13a707 extend check_global_access() to avoid my_error when it's not needed
(in INFORMATION_SCHEMA).
2013-03-19 15:25:58 +01:00
Sergei Golubchik
d24c8d9af1 fix innodb failures on solaris 2013-03-11 21:00:08 +01:00
Sergei Golubchik
9d8ee74b38 MDEV-4186 Test case main.myisampack fails on ppc32 (only)
fix the declaration to use the correct type for st_handler_check_param::sort_buffer_length.
remove redundant casts.
2013-03-08 19:09:45 +01:00
Sergei Golubchik
0d9a6d52d7 merge with XtraDB as of Percona-Server-5.5.30-rel30.1 2013-03-08 19:08:45 +01:00
Michael Widenius
4cace76d4d Automatic merge 2013-03-01 18:09:06 +02:00
Sergei Golubchik
9d72bbf84c merge with XtraDB as of Percona-Server-5.5.29-rel30.0 2013-02-28 23:56:17 +01:00
Sergei Golubchik
213f1c76a0 5.3->5.5 merge 2013-02-28 22:47:29 +01:00