mariadb/sql/wsrep_server_service.h
Teemu Ollakka 3f59bbeeae MDEV-29293 MariaDB stuck on starting commit state
The problem seems to be a deadlock between KILL command execution
and BF abort issued by an applier, where:
* KILL has locked victim's LOCK_thd_kill and LOCK_thd_data.
* Applier has innodb side global lock mutex and victim trx mutex.
* KILL is calling innobase_kill_query, and is blocked by innodb
  global lock mutex.
* Applier is in wsrep_innobase_kill_one_trx and is blocked by
  victim's LOCK_thd_kill.

The fix in this commit removes the TOI replication of KILL command
and makes KILL execution less intrusive operation. Aborting the
victim happens now by using awake_no_mutex() and ha_abort_transaction().
If the KILL happens when the transaction is committing, the
KILL operation is postponed to happen after the statement
has completed in order to avoid KILL to interrupt commit
processing.

Notable changes in this commit:
* wsrep client connections's error state may remain sticky after
  client connection is closed. This error message will then pop
  up for the next client session issuing first SQL statement.
  This problem raised with test galera.galera_bf_kill.
  The fix is to reset wsrep client error state, before a THD is
  reused for next connetion.
* Release THD locks in wsrep_abort_transaction when locking
  innodb mutexes. This guarantees same locking order as with applier
  BF aborting.
* BF abort from MDL was changed to do BF abort on server/wsrep-lib
  side first, and only then do the BF abort on InnoDB side. This
  removes the need to call back from InnoDB for BF aborts which originate
  from MDL and simplifies the locking.
* Removed wsrep_thd_set_wsrep_aborter() from service_wsrep.h.
  The manipulation of the wsrep_aborter can be done solely on
  server side. Moreover, it is now debug only variable and
  could be excluded from optimized builds.
* Remove LOCK_thd_kill from wsrep_thd_LOCK/UNLOCK to allow more
  fine grained locking for SR BF abort which may require locking
  of victim LOCK_thd_kill. Added explicit call for
  wsrep_thd_kill_LOCK/UNLOCK where appropriate.
* Wsrep-lib was updated to version which allows external
  locking for BF abort calls.

Changes to MTR tests:
* Disable galera_bf_abort_group_commit. This test is going to
  be removed (MDEV-30855).
* Record galera_gcache_recover_manytrx as result file was incomplete.
  Trivial change.
* Make galera_create_table_as_select more deterministic:
  Wait until CTAS execution has reached MDL wait for multi-master
  conflict case. Expected error from multi-master conflict is
  ER_QUERY_INTERRUPTED. This is because CTAS does not yet have open
  wsrep transaction when it is waiting for MDL, query gets interrupted
  instead of BF aborted. This should be addressed in separate task.
* A new test galera_kill_group_commit to verify correct behavior
  when KILL is executed while the transaction is committing.

Co-authored-by: Seppo Jaakola <seppo.jaakola@iki.fi>
Co-authored-by: Jan Lindström <jan.lindstrom@galeracluster.com>
Signed-off-by: Julius Goryavsky <julius.goryavsky@mariadb.com>
2023-05-22 00:39:43 +02:00

102 lines
3.3 KiB
C++

/* Copyright 2018 Codership Oy <info@codership.com>
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; version 2 of the License.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA */
#ifndef WSREP_SERVER_SERVICE_H
#define WSREP_SERVER_SERVICE_H
/* wsrep-lib */
#include "wsrep/server_service.hpp"
#include "wsrep/exception.hpp" // not_impemented_error(), remove when finished
#include "wsrep/storage_service.hpp"
class Wsrep_server_state;
/* wsrep::server_service interface implementation */
class Wsrep_server_service : public wsrep::server_service
{
public:
Wsrep_server_service(Wsrep_server_state& server_state)
: m_server_state(server_state)
{ }
wsrep::storage_service* storage_service(wsrep::client_service&);
wsrep::storage_service* storage_service(wsrep::high_priority_service&);
void release_storage_service(wsrep::storage_service*);
wsrep::high_priority_service*
streaming_applier_service(wsrep::client_service&);
wsrep::high_priority_service*
streaming_applier_service(wsrep::high_priority_service&);
void release_high_priority_service(wsrep::high_priority_service*);
void background_rollback(wsrep::unique_lock<wsrep::mutex> &,
wsrep::client_state &);
void bootstrap();
void log_message(enum wsrep::log::level, const char*);
void log_dummy_write_set(wsrep::client_state&, const wsrep::ws_meta&)
{ throw wsrep::not_implemented_error(); }
void log_view(wsrep::high_priority_service*, const wsrep::view&);
void recover_streaming_appliers(wsrep::client_service&);
void recover_streaming_appliers(wsrep::high_priority_service&);
wsrep::view get_view(wsrep::client_service&, const wsrep::id& own_id);
wsrep::gtid get_position(wsrep::client_service&);
void set_position(wsrep::client_service&, const wsrep::gtid&);
void log_state_change(enum wsrep::server_state::state,
enum wsrep::server_state::state);
bool sst_before_init() const;
std::string sst_request();
int start_sst(const std::string&, const wsrep::gtid&, bool);
int wait_committing_transactions(int);
void debug_sync(const char*);
private:
Wsrep_server_state& m_server_state;
};
/**
Helper method to create new streaming applier.
@param orig_thd Original thd context to copy operation context from.
@param ctx Context string for debug logging.
*/
class Wsrep_applier_service;
Wsrep_applier_service*
wsrep_create_streaming_applier(THD *orig_thd, const char *ctx);
/**
Helper method to create new storage service.
@param orig_thd Original thd context to copy operation context from.
@param ctx Context string for debug logging.
*/
class Wsrep_storage_service;
Wsrep_storage_service*
wsrep_create_storage_service(THD *orig_thd, const char *ctx);
#endif /* WSREP_SERVER_SERVICE */