lock_move_rec_list_start(): Relax a too strict assertion.
This function can be invoked on the leftmost leaf page, after all.
So, the first record of each page can be a 'default row' record,
but the 'default row' record must never be locked.
Contrary to what the comment said, trx_resurrect_table_locks()
does associate table locks with every recovered transaction that
modified any records, ever since this bug fix in MySQL 5.6.12:
Bug#16593427 ROLLBACK OF RECOVERED TRANSACTION CORRUPTS NON-ONLINE ADD INDEX
Actually there are 2 issues in the case of invisible columns
1st `select fields from t1` will have more fields then `select * from t1`.
So instead of `select * from t1` we are using `select a,b,invisible from t1`
these fields are supplied from `select fields from t1`.
2nd We are using --complete-insert when we detect that this table is using
invisible columns.
Feature Definition:-
This feature adds invisible column functionality to server.
There is 4 level of "invisibility":
1. Not invisible (NOT_INVISIBLE) — Normal columns created by the user
2. A little bit invisible (USER_DEFINED_INVISIBLE) — columns that the
user has marked invisible. They aren't shown in SELECT * and they
don't require values in INSERT table VALUE (...). Otherwise
they behave as normal columns.
3. More invisible (SYSTEM_INVISIBLE) — Can be queried explicitly,
otherwise invisible from everything. Think ROWID sytem column.
Because they're invisible from ALTER TABLE and from CREATE TABLE
they cannot be created or dropped, they're created by the system.
User cant not create a column name which is same as of
SYSTEM_INVISIBLE.
4. Very invisible (COMPLETELY_INVISIBLE) — as above, but cannot be
queried either. They can only show up in EXPLAIN EXTENDED (might
be possible for a very invisible indexed virtual column) but
otherwise they don't exist for the user.If user creates a columns
which has same name as of COMPLETELY_INVISIBLE then
COMPLETELY_INVISIBLE column is renamed again. So it is completely
invisible from user.
Invisible Index(HA_INVISIBLE_KEY):-
Creation of invisible columns require a new type of index which
will be only visible to system. User cant see/alter/create/delete
this index. If user creates a index which is same name as of
invisible index then it will be renamed.
Syntax Details:-
Only USER_DEFINED_INVISIBLE column can be created by user. This
can be created by adding INVISIBLE suffix after column definition.
Create table t1( a int invisible, b int);
Rules:-
There are some rules/restrictions related to use of invisible columns
1. All the columns in table cant be invisible.
Create table t1(a int invisible); \\error
Create table t1(a int invisible, b int invisble); \\error
2. If you want invisible column to be NOT NULL then you have to supply
Default value for the column.
Create table t1(a int, b int not null); \\error
3. If you create a view/create table with select * then this wont copy
invisible fields. So newly created view/table wont have any invisible
columns.
Create table t2 as select * from t1;//t2 wont have t1 invisible column
Create view v1 as select * from t1;//v1 wont have t1 invisible column
4. Invisibility wont be forwarded to next table in any case of create
table/view as select */(a,b,c) from table.
Create table t2 as select a,b,c from t1; // t2 will have t1 invisible
// column(b), but this wont be invisible in t2
Create view v1 as select a,b,c from t1; // v1 will have t1 invisible
// column(b), but this wont be invisible in v1
Implementation Details:-
Parsing:- INVISIBLE_SYM is added into vcol_attribute(so its like unique
suffix), It is also added into keyword_sp_not_data_type so that table
can have column with name invisible.
Implementation detail is given by each modified function/created function.
(Some function are left as they were self explanatory)
(m= Modified, n= Newly Created)
mysql_prepare_create_table(m):- Extra checks for invisible columns are
added. Also some DEBUG_EXECUTE_IF are also added for test cases.
mysql_prepare_alter_table(m):- Now this will drop all the
COMPLETELY_INVISIBLE column and HA_INVISIBLE_KEY index. Further
Modifications are made to stop drop/change/delete of SYSTEM_INVISIBLE
column.
build_frm_image(m):- Now this allows incorporating field_visibility
status into frm image. To remain compatible with old frms
field_visibility info will be only written when any of the field is
not NOT_INVISIBLE.
extra2_write_additional_field_properties(n):- This will write field
visibility info into buffer. We first write EXTRA2_FIELD_FLAGS into
buffer/frm , then each next char will have field_visibility for each
field.
init_from_binary_frm_image(m):- Now if we get EXTRA2_FIELD_FLAGS,
then we will read the next n(n= number of fields) chars and set the
field_visibility. We also increment
thd->status_var.feature_invisible_columns. One important thing to
note if we find out that key contains a field whose visibility is
> USER_DEFINED_INVISIBLE then , we declare this key as invisible
key.
sql_show.cc is changed accordingly to make show table, show keys
correct.
mysql_insert(m):- If we get to know that we are doing insert in
this way insert into t1 values(1,1); without explicitly specifying
columns, then we check for if we have invisible fields if yes then
we reset the whole record, Why ? Because first we want hidden columns
to get default/null value. Second thing auto_increment has property
no default and no null which voilates invisible key rule 2, And
because of this it was giving error. Reseting table->record[0]
eliminates this issue. More info put breakpoint on handler::write_row
and see auto_increment value.
fill_record(m):- we continue loop if we find invisible column because
this is already reseted/will get its value if it is default.
Test cases:- Since we can not directly add > USER_DEFINED_INVISIBLE
column then I have debug_dbug to create it in mysql_prepare_create_table.
Patch Credit:- Serg Golubchik
trx_roll_must_shutdown(): During the rollback of recovered transactions,
report progress and check if the rollback should be interrupted because
of a pending shutdown.
trx_roll_max_undo_no, trx_roll_progress_printed_pct: Remove, along with
the messages that were interleaved with other messages.
row_undo_step(), trx_rollback_active(): Abort the rollback of a
recovered ordinary transaction if fast shutdown has been initiated.
trx_rollback_resurrected(): Convert an aborted-rollback transaction
into a fake XA PREPARE transaction, so that fast shutdown can proceed.
trx_rollback_resurrected(): If shutdown was initiated, fake all
remaining active transactions to XA PREPARE state, so that shutdown
can proceed. Also, make the parameter "all" an output that will be
assigned to FALSE in this case.
trx_rollback_or_clean_recovered(): Remove the shutdown check
(it was moved to trx_rollback_resurrected()).
trx_undo_free_prepared(): Relax assertions.
row_quiesce_table_start(), row_quiesce_table_complete():
Use the more appropriate predicate srv_undo_sources for skipping
purge control. (This change alone is insufficient; it is possible
that this predicate will change during the call to trx_purge_stop()
or trx_purge_run().)
trx_purge_stop(), trx_purge_run(): Tolerate PURGE_STATE_EXIT.
It is very well possible to initiate shutdown soon after the statement
FLUSH TABLES FOR EXPORT has been submitted to execution.
srv_purge_coordinator_thread(): Ensure that the wait for purge_sys->event
in trx_purge_stop() will terminate when the coordinator thread exits.
The loop in read_xml_field(), unlike the same loop in read_sep_field(),
cannot end with item<>NULL, as it does not have any "break" statements.
The entire block "if (item) {...}" was a dead code.
The fixes for these bugs:
Bug#27586 Wrong autoinc value assigned by LOAD DATA in the NO_AUTO_VALUE_ON_ZERO mode
Bug#22372 Disable spatial key, load data, enable spatial key, crashes table
fixed only LOAD DATA INFILE, but did not fix LOAD XML INFILE.
This patch does for LOAD XML FILE what patches for Bug#27586 and Bug#22372
earlier did for LOAD DATA INFILE.
1. Fixing the auto_increment problem:
a. table->auto_increment_field_not_null is not set to TRUE
anymore when a column does not have a corresponding XML tag.
b. Adding "table->auto_increment_field_not_null= false"
in the end of read_xml_field().
These two changes resemble the patch for Bug#27586.
2. Fixing the GEOMETRY problem:
The result for "reset()" was not tested for errors in read_xml_field(),
which made it possible for empty string to sneak into a "GEOMETRY NOT NULL"
column when this column does not have a corresponding XML tag with data.
After this patch the result of reset() is tested and and an error is
returned in such cases.
This change effectively resembles the patch for Bug#22372
3. Spliting the code into a new virtual method Field::load_data_set_null().
Rationale:
a. To avoid duplicate code in read_sep_field() and read_xml_field():
Changes #1 and #2 made the code handling NULL values for Field
exactly the same in read_sep_field() and read_xml_field().
b. To avoid tests for field_type(), which is not friendly to
upcoming data type plugins.
This change makes it possible for data type plugins
to implement their own special way for handling NULL values in LOAD DATA
by overriding Field_xxx::load_data_set_null(),
like Field_geom and Field_timestamp do.
Instead of updating global counter, calculate Threads_running on the fly.
All threads having command != COM_SLEEP are included.
Behaviour changes:
Previously SHOW STATUS and SHOW GLOBAL STATUS returned the same values
representing global status. Now SHOW STATUS always returns 1 indicating that
current session has 1 thread running.
Previously only threads that were executing dispatch_command() or running events
were accounted by Threads_running. Now it is rough equivalent of
SELECT COUNT(*) FROM INFORMATION_SCHEMA.PROCESSLIST WHERE state!='Sleep'
ha_print_info(): Remove.
srv_printf_innodb_monitor(): Do not acquire btr_search_latches[]
Add the equivalent functionality that was part of the non-debug
version of ha_print_info().
The problem was that multi_range_read_info_const() called
multi_range_key_create_key() which changed m_part_spec.start_part,
while there was an activ table scan ongoing.
Fixed by copying and restoring m_part_spec around
multi_range_key_create_calls()
first, and we do not not care whether client has received all data.
This is a TCP optimization to avoid TIME_WAIT in TCP connection teardown.
This patch would abort connection on timeout, which usually happens when
client reads a large result set, at slower pace then the server can
write.
The patch also cleans up socket timeout handling, so that Windows
is consistent with another platforms (using nonblocking socket IO
+ waiting in poll/select on single socket, rather than setsockopt).
This makes identifying timeouts easier.
Also removed the superficial shutdown() before closesocket() in a few
places where it was used, because it was never needed , and
reportedly breaks SO_LINGER on Windows.
When the transaction isolation level is SERIALIZABLE, or when
a locking read is performed in the REPEATABLE READ isolation level,
InnoDB must lock delete-marked records in order to prevent another
transaction from inserting something.
However, at READ UNCOMMITTED or READ COMMITTED isolation level or
when the parameter innodb_locks_unsafe_for_binlog is set, the
repeatability of the reads does not matter, and there is no need
to lock any records.
row_search_mvcc(): Skip locks on delete-marked committed records upfront,
instead of invoking row_unlock_for_mysql() afterwards. The unlocking
never worked for secondary index records.
dict_stats_rename_table(): After DB_LOCK_WAIT_TIMEOUT
or DB_DUPLICATE_KEY, reset the trx->error_state before retrying.
Also, properly treat DB_DEADLOCK as a hard error.
Part2: make MyRocks add its directory into @@ignore_db_dirs when starting.
This is necessary because apparently not everybody are using plugin's my.cnf
So load ha_rocksdb.{so,dll} manually and then hit MDEV-12451, MDEV-14461
etc.
