ca86986a14
This commit is the part I of the series of four that addresses
MDEV-31949 in two main directions which are xa parallel slave
performance and xa transaction crash-recovery.
This one improves upon MDEV-742 design's XA binlogging to
facilitate to the crash-recovery (the actual binlog-based recovery
is coming in the part IV of MDEV-33168 et al).
With the refactoring changes, when binlog is ON, handling of execution of
a XA transaction, including binlogging, is made conceptually uniform
with the normal BEGIN-COMMIT transaction.
That is At XA-PREPARE the transaction first is prepared in engines and
after that accumulated replication events are written to the binary
log, naturally without any completion event as it's unknown yet.
When later XA-"COMPLETE" that is XA-COMMIT and XA-PREPARE follows up,
the binary logging of respective Query event takes place first.
One can perceive such scheme as if a normal transaction logging
is split in the middle into two parts (and nothing really happens
in between of them but time passed by). And after the second chunk is sent
to binlog the transaction gets committed (or rolled back) in engine.
With binlog is enabled both phases' loggings go through binlog-group-commit,
where XA-PREPARE "sub-transaction" merely groups for binary logging
so skips the engine action while XA-"COMPLETE" does both, that is the logging
and an ordered "complete". This behavior is also consistent between
completions from the native and external connections.
Being a participant of binlog-group-commit designates either XA phase
is recoverable (not implemented here) from active binlogs determined
by binlog-checkpoint.
For the latter specifically this patch removes custom unlogging of XA-prepare
group. See
entry.need_unlog= 0
et al in MYSQL_BIN_LOG::write_transaction_to_binlog().
In addition to the above a corner case of engine read-only XA transaction
is addressed. Previously it was streamlined with logging an empty XA-PREPARE
group of binlog events concluded by XA-"COMPLETE" query-event.
Now when a preparing XA transaction is found to have only read-only engine
branches or none it is marked for rollback as XA_RDONLY optimization:
- nothing gets logged at the prepare time an XA_RDONLY note is generated and
- it's rolled back at disconnect
For XA-COMPLETE to tell whether the prepare phase was logged or
not the XID state object is extended with a boolean flag which is a
part internal interface for recovery implementation.
The flag is normally raised by XA-prepare at flushing to binlog and
also at binlog recovery (will be done so it's fully implemented).
Notable changes:
sql/handler.cc
- ha_prepare()
a. is ensured to execute binlog_hton::prepare() as
the last XA's branch for preparing;
b. engine read-only is marked in the xid state to rollback and
ER_XA_RDONLY *note* is generated.
- conversely ha_rollback_trans() executes binlog hton::rollback() as first
branch (the commit method was already equipped to do so)
- ditto to the external completion of XA via ha_commit_or_rollback_by_xid();
the function is made a sort of recursive. It may be first be invoked
on a top level to take on the binlog hton "completion" to be
called from its stack once again now having is_xap_binlogged() false,
so to carry out the engine commit.
- xarecover_handlerton() now only simulates successful find of the user
xid in binlog.
sql/log.cc
- binlog_commit,rollback() et al are simplified and cleaned up (like
binlog_complete_by_xid() introduction).
In particular binlog_{commit,rollback}() are rendered to retain
just a single piece of XA footprint in either. The methods recognize
naturally empty transaction caches at XA completion to proceed anyway
into binlog-group-commit thickness;
the binlog_commit's binlog_commit_flush_trx_cache() decides which
type of transaction and which XA phase is being handled so a proper
group event closure is computed.
- MYSQL_BIN_LOG::trx_group_commit_with_engines() takes care to
raise or drop XID::binlogged flag via xid_cache_update_xa_binlog_state().
- the new run_xa_complete_ordered() encapsulates XA specifics at
execution of the engine ordered commit. It's defined with asserts due to
MDEV-32455.
It's not done as TC_LOG::member because of the scope of this work is
limited. The new function mirrors the logic of the normal run_commit_ordered()
in that it skips engine completion for those that lack
hton::commit_ordered() in favor of doing that on the top level
of ha_commit,rollback_trans().
sql/log_event_server.cc
- use the transaction cache at XA-PREPARE handling (specifically
when XA-END Query event is logged).
sql/xa.cc
- XID_cache_element extended with is-binlogged meaning flag and
few rating functions added to use by binlogging and recovery
(xid_cache_update_xa_binlog_state());
- trans_xa_commit,rollback() external action branches are converted
into calls of a new largely common function.
sql/xa.h
- xid_cache_insert(XID *xid, bool is_binlogged) parameter list is
extended for xa binlog recovery.
|
||
|---|---|---|
| .. | ||
| charsets | ||
| CMakeLists.txt | ||
| errmsg-utf8.txt | ||
| insert_translations_into_errmsg.py | ||
| README.md | ||
A quicker way for adding new language translations to the errmsg-utf8.txt file
Summary
To generate a new language translation of MariaDB use the following pull request (PR) as a template for your work:
You will notice as part of your translation work, you will have to add your language translations to the file sql/share/errmsg-utf8.txt which is found in the current directory. This file is long with many sections which can make the translation work tedious. In this README, we explain a procedure and provide a script insert_translations_into_errmsg.py that cuts down the amount of tedium in accomplishing the task.
Procedure
-
Start by grepping out all the english translations from errmsg-utf8.txt using the following grep command, and redirecting the output to a file:
grep -P "^\s*eng\s" errmsg-utf8.txt > all_english_text_in_errmsg-utf8.txt
-
Next use Google translate to obtain a translation of this file. Google translate provides the ability to upload whole files for translation. For example, this technique was used to obtain Swahili translations which yielded a file with output similar to the below (output is truncated for clarity):
sw "hashchk" sw "isamchk" sw "LA" sw "NDIYO" sw "Haiwezi kuunda faili '% -.200s' (kosa: %iE)" sw "Haiwezi kuunda jedwali %
s.%s (kosa: %iE)" sw "Haiwezi kuunda hifadhidata '% -.192s' (kosa: %iE)" sw "Haiwezi kuunda hifadhidata '% -.192s'; hifadhidata ipo"
Note that Google translate removes the leading whitespace in the translation file it generates. DO NOT add that leading whitespace back!
-
Give the translated file an appropriate name (e.g.
all_swahili_text_in_errmsg-utf8.txt) and store it in the same directory witherrmsg-utf8.txtandall_english_text_in_errmsg-utf8.txt. These 3 files will be used by the script insert_translations_into_errmsg.py. -
Proof check the auto-translations in the file you downloaded from Google translate. Note that Google might omit formatting information that will cause the compilation of MariaDB to fail, so pay attention to these.
-
Reintegrate these translations into the errmsg-utf8.txt by running the insert_translations_into_errmsg.py script as follows:
chmod ugo+x insert_translations_into_errmsg.py # Make the script executable if it is not.
./insert_translations_into_errmsg.py <errmsg-utf8.txt file>
For example, for the swahili translation, we ran the following:
./insert_translations_into_errmsg.py errmsg-utf8.txt all_english_text_in_errmsg-utf8.txt all_swahili_text_in_errmsg-utf8.txt
The script uses the
errmsg-utf8.txtfile and the grepped english file to keep track of each new translation. It then creates a file in the same directory aserrmsg-utf8.txtwith the nameerrmsg-utf8-with-new-language.txt. -
Check that the reintegration of the new translations into
errmsg-utf8-with-new-language.txtwent OK, and if it did, renameerrmsg-utf8-with-new-language.txttoerrmsg-utf8.txt:mv errmsg-utf8-with-new-language.txt errmsg-utf8.txt
-
In the header of errmsg-utf8.txt make sure to add your language long form to short form mapping. E.g. for Swahili, add:
swahili=sw