FOREIGN DATA WRAPPER to work with CONNECT engine
modified: storage/connect/tabjdbc.cpp
- Add a function to retrieve User variable value (DEVELOPMENT only)
modified: storage/connect/ha_connect.cc
modified: storage/connect/jsonudf.cpp
modified: storage/connect/jsonudf.h
modified: storage/connect/tabjdbc.cpp
Suppress unused parameter from PlugSubSet
modified: storage/connect/global.h
modified: storage/connect/plugutil.cpp
modified: storage/connect/jsonudf.cpp
modified: storage/connect/tabjson.cpp
modified: storage/connect/user_connect.cc
- Fix a bug making column catalog XML tables fail
modified: storage/connect/tabxml.cpp
- Comment out wrong message
modified: storage/connect/ha_connect.cc
- Update error message when sorting an ODBC table fails
modified: storage/connect/tabodbc.cpp
- Add error message when gettting an address
from an OEM fails.
modified: storage/connect/reldef.cpp
- Make some modifications useful for OEM module writting
Export discovery functions for CSV, JDBC and XML
Remove unuseful include from tabjson.h
Move TDBXML::data_charset function from header file to source
modified: storage/connect/tabfmt.h
modified: storage/connect/tabjson.h
modified: storage/connect/tabxml.cpp
modified: storage/connect/tabxml.h
- Update test result
modified: storage/connect/mysql-test/connect/r/jdbc_oracle.result
modified: storage/connect/connect.cc
modified: storage/connect/ha_connect.cc
modified: storage/connect/ha_connect.h
modified: storage/connect/tabjdbc.cpp
modified: storage/connect/tabmysql.cpp
modified: storage/connect/tabodbc.cpp
- MDEV-17212: Test if NumResultCols is implemented by the data source
modified: storage/connect/odbconn.cpp
- Change error type in Optimize
modified: storage/connect/ha_connect.cc
- Update version date
modified: storage/connect/ha_connect.cc
- Fix truncating error messages on first unrecognized latin1 character
modified: storage/connect/ha_connect.cc
- Fix MDEV-17343
Reject multi-table UPDATE/DELETE commands that crash on some systems
modified: storage/connect/ha_connect.cc
modified: storage/connect/tabext.cpp
- Try fix some failing tests
modified: storage/connect/mysql-test/connect/r/jdbc_postgresql.result
modified: storage/connect/mysql-test/connect/r/odbc_postgresql.result
- Typo
modified: storage/connect/global.h
filamtxt.cpp: DOSFAM::RenameTempFile: Change sprintf to snprintf.
filamvct.cpp: VECFAM::RenameTempFile: Change sprintf to snprintf.
javaconn.cpp:
Add JAVAConn::GetUTFString function.
Use it instead of env->GetStringUTFChars.
Fix wrong identation.
javaconn.h: Add GetUTFString declaration.
jdbconn.cpp:
Use GetUTFString function instead of env->GetStringUTFChars.
jmgoconn.cpp:
Use GetUTFString function instead of env->GetStringUTFChars.
Fix wrong identation.
jsonudf.cpp: change 139 to BMX line 4631.
tabjmg.cpp:
Add ReleaseStringUTF.
Fix wrong identation.
tabpivot.cpp: Fix wrong identation.
tabutil.cpp: TDBPRX::GetSubTable: Change sprintf to snprintf.
modified: storage/connect/filamtxt.cpp
modified: storage/connect/filamvct.cpp
modified: storage/connect/javaconn.cpp
modified: storage/connect/javaconn.h
modified: storage/connect/jdbconn.cpp
modified: storage/connect/jmgoconn.cpp
modified: storage/connect/jsonudf.cpp
modified: storage/connect/tabjmg.cpp
modified: storage/connect/tabpivot.cpp
modified: storage/connect/tabutil.cpp
- Fix MDEV-16895 CONNECT engine's get_error_message can cause buffer
overflow and server crash with long queries
ha_connect_cc: Update version.
get_error_message: Remove charset conversion.
modified: storage/connect/ha_connect.cc
- Fix a server crash on inserting bigint to a JDBC table
JDBConn::SetUUID:
Suppress check on ctyp that causes a server crash because ctyp
can be negative and this triggers an DEBUG_ASSERT on return.
modified: storage/connect/jdbconn.cpp
- Delete an assert(qrp) from JCATPARM *AllocCatInfo that is called with
qrp=NULL from JDBConn::SetUUID. Also delete a clone of this function
that was duplicated in javaconn.cpp.
modified: storage/connect/javaconn.cpp
modified: storage/connect/jdbconn.cpp
- Update some (disabled) tests and results to avoid failure
modified: storage/connect/mysql-test/connect/r/jdbc.result
modified: storage/connect/mysql-test/connect/r/json_java_2.result
modified: storage/connect/mysql-test/connect/r/json_java_3.result
modified: storage/connect/mysql-test/connect/r/mongo_java_2.result
modified: storage/connect/mysql-test/connect/r/mongo_java_3.result
modified: storage/connect/mysql-test/connect/r/xml2.result
modified: storage/connect/mysql-test/connect/t/json_java_2.test
modified: storage/connect/mysql-test/connect/t/json_java_3.test
modified: storage/connect/mysql-test/connect/t/mongo_java_2.test
modified: storage/connect/mysql-test/connect/t/mongo_java_3.test
modified: storage/connect/mysql-test/connect/t/xml2.test
This reverts commit 72deed5988
which was merged as commit 29d4ac2ceb.
The change caused regressions, such as mysql-test-run failing
to run at all with --valgrind, or aborting on Windows on the
first test failure.
The code passing positions in the query to constructors of
Rewritable_query_parameter descendants (e.g. Item_splocal)
was not reliable. It used various Lex_input_stream methods:
- get_tok_start()
- get_tok_start_prev()
- get_tok_end()
- get_ptr()
to find positions of the recently scanned tokens.
The challenge was mostly to choose between get_tok_start()
and get_tok_start_prev(), taking into account to the current
grammar (depending if lookahead takes place before
or after we read the positions in every particular rule).
But this approach did not work at all in combination
with token contractions, when MYSQLlex() translates
two tokens into one token ID, for example:
WITH ROLLUP -> WITH_ROLLUP_SYM
As a result, the tokenizer is already one more token ahead.
So in query fragment:
"GROUP BY d, spvar WITH ROLLUP"
get_tok_start() points to "ROLLUP".
get_tok_start_prev() points to "WITH".
As a result, it was "WITH" who was erroneously replaced
to NAME_CONST() instead of "spvar".
This patch modifies the code to do it a different way.
Changes:
1. For keywords and identifiers, the tokenizer now
returns LEX_CTRING pointing directly to the query
fragment. So query positions are now just available using:
- $1.str - for the beginning of a token
- $1.str+$1.length - for the end of a token
2. Identifiers are not allocated on the THD memory root
in the tokenizer any more. Allocation is now done
on later stages, in methods like LEX::create_item_ident().
3. Two LEX_CSTRING based structures were added:
- Lex_ident_cli_st - used to store the "client side"
identifier representation, pointing to the
query fragment. Note, these identifiers
are encoded in @@character_set_client
and can have broken byte sequences.
- Lex_ident_sys_st - used to store the "server side"
identifier representation, pointing to the
THD allocated memory. This representation
guarantees that the identifier was checked
for being well-formed, and is encoded in utf8.
4. To distinguish between two identifier types
in the grammar, two Bison types were added:
<ident_cli> and <ident_sys>
5. All non-reserved keywords were marked as
being of the type <ident_cli>.
All reserved keywords are still of the type NONE.
6. All curly brackets in rules collecting
non-reserved keywords into non-terminal
symbols were removed, e.g.:
Was:
keyword_sp_data_type:
BIT_SYM {}
| BOOLEAN_SYM {}
Now:
keyword_sp_data_type:
BIT_SYM
| BOOLEAN_SYM
This is important NOT to have brackets here!!!!
This is needed to make sure that the underlying
Lex_ident_cli_ststructure correctly passes up to
the calling rule.
6. The code to scan identifiers and keywords
was moved from lex_one_token() into new
Lex_input_stream methods:
scan_ident_sysvar()
scan_ident_start()
scan_ident_middle()
scan_ident_delimited()
This was done to:
- get rid of enormous amount of references to &yylval->lex_str
- and remove a lot of references like lip->xxx
7. The allocating functionality which puts identifiers on the
THD memory root now resides in methods of Lex_ident_sys_st,
and in THD::to_ident_sys_alloc().
get_quoted_token() was removed.
8. Cleanup: check_simple_select() was moved as a method to LEX.
9. Cleanup: Some more functionality was moved from *.yy
to new methods were added to LEX:
make_item_colon_ident_ident()
make_item_func_call_generic()
create_item_qualified_asterisk()
trx_undof_page_add_undo_rec_log(): Write the MLOG_UNDO_INSERT
record instead of the equivalent MLOG_2BYTES and MLOG_WRITE_STRING.
This essentially reverts commit 9ee8917dfd.
In MariaDB 10.3, I attempted to simplify the crash recovery code
by making use of lower-level redo log records. It turns out that
we must keep the redo log parsing code in order to allow crash-upgrade
from older MariaDB versions (MDEV-14848).
Now, it further turns out that the InnoDB redo log record format is
suboptimal for logging multiple changes to a single page. This simple
change to the redo logging of undo log significantly affects the
INSERT and UPDATE performance.
Essentially, we wrote
(space_id,page_number,MLOG_2BYTES,2 bytes)
(space_id,page_number,MLOG_WRITE_STRING,N+4 bytes)
instead of the previously written
(space_id,page_number,MLOG_UNDO_INSERT,N+2 bytes)
The added redo log volume caused a single-threaded INSERT
(without innodb_adaptive_hash_index) of
1,000,000 rows to consume 11 seconds instead of 9 seconds,
and a subsequent UPDATE of 30,000,000 rows to consume 64 seconds
instead of 58 seconds. If we omitted all redo logging for the
undo log, the INSERT would consume only 4 seconds.
The trx_t::undo_mutex covered both some main-memory data structures
(trx_undo_t) and access to undo pages. The trx_undo_t is only
accessed by the thread that is associated with a running transaction.
Likewise, each transaction has its private set of undo pages.
The thread that is associated with an active transaction may
lock multiple undo pages concurrently, but no other thread may
lock multiple pages of a foreign transaction.
Concurrent access to the undo logs of an active transaction is possible,
but trx_undo_get_undo_rec_low() only locks one undo page at a time,
without ever holding any undo_mutex.
It seems that the trx_t::undo_mutex would have been necessary if
multi-threaded execution or rollback of a single transaction
had been implemented in InnoDB.
In the merge of commit e7f4e61f6e
the call fil_flush_file_spaces(FIL_TYPE_LOG) is necessary.
Tablespaces will be flushed as part of the redo log
checkpoint, but the redo log will not necessarily
be flushed, depending on innodb_flush_method.
InnoDB takes a lot of time to perform null updates. The reason is that
even though an empty update vector was created, InnoDB will go on to
write undo log records and update the system columns
DB_TRX_ID and DB_ROLL_PTR in the clustered index, and of course write
redo log for all this.
This could have been fixed properly in
commit 54a492ecac more than 10 years ago.
there was a problem with "next_free_value >= reserved_until" condition:
SEQUENCE::set_value handle next_free_value & reserved_until after adjust_values() call, so it is incorect to put assert on it in adjust_values()
Differences:
MariaDB doesn't support a JSON type therefore the crc32 on those values
are different.
JSON extract syntax is different.
loaddata_utf8 has 3 duplicate lines removed compared to MySQL version.
From mysql-server:
09fdfad50764ff6809e7dd5300e9ce1ab727b62a
e90ae1707e0ca46abc775d1680d1856c4be38b66
described in http://github.com/mysql/mysql-server/pull/157
Apart from external contribution I have added few more additional testcases
for CRC32() function, which are given below.
New Testcases added:
->Verify the crc value of various numeric and string data types(int,
double, blob, text, json, enum, set)
->Verify the crc value when expressions having comparison_operators
and logical_operators
->Verify the crc value for the expression having string_functions,
arithmetic_functions, json_functions
->Verify the crc value for the expression having Geometry functions
like POINT, LINESTRING, MULTILINESTRING, POLYGON, MULTIPOLYGON
->Verify the crc value generated from stored procedures, functions,
triggers, prepare statement, views.
Fix:
Patch based on contribution by Daniel Black (Github user: grooverdan)
Reviewed-by: Anitha Gopi anitha.gopi@oracle.com
Reviewed-by: Srikanth B R srikanth.b.r@oracle.com
RB: 17294
Remove the local variable srv_buf_pool_size_org, which was always 0.
In MySQL 5.7, InnoDB was made a mandatory storage engine, which would
force InnoDB to start up when executing
mysqld --verbose --help
which is what mysql-test-run.pl is doing as a first step. With a
large innodb_buffer_pool_size, this would take a long time.
So, MySQL 5.7 includes a hack that starts up InnoDB with a smaller
buffer pool when the option --verbose is present.
MariaDB uses HAVE_LZO, not HAVE_LZO1X (which was never defined).
Also, the variable srv_lzo_disabled was never defined or read
(only declared and assigned to, in unreachable code).
The InnoDB system table column SYS_TABLES.MIX_LEN was repurposed
in InnoDB Plugin for MySQL 5.1, in
commit 91111174ee (MySQL 5.1.46).
Until MySQL 5.6, it only contained a flag DICT_TF2_TEMPORARY.
MySQL 5.6 introduced a number of flags that were transient
in nature. One of these was introduced in 5.6.5, originally
called DICT_TF2_USE_TABLESPACE and later renamed to
DICT_TF2_USE_FILE_PER_TABLE. MySQL 5.7.6 introduced logic
that insists that the flag be set for any table that does not
reside in a shared tablespace, breaking upgrade from MySQL 5.5.
MariaDB does not support shared tablespaces other than the
InnoDB system tablespace. Also, some dependencies on
SYS_TABLES.MIX_LEN were removed in an earlier fix:
MDEV-13084 MariaDB 10.2 crashes on corrupted SYS_TABLES.MIX_LEN field
(commit e813fe8622).
dict_check_sys_tables(): Remove a bogus debug assertion, and add a
comment that explains how DICT_TF2_USE_FILE_PER_TABLE is used.
dict_table_is_file_per_table(): Remove a bogus debug assertion.
