mirror of
https://github.com/MariaDB/server.git
synced 2025-01-17 12:32:27 +01:00
7cffb5f6e8
The buffer pool refactoring in MDEV-15053 and MDEV-22871 shifted
the performance bottleneck to the page flushing.
The configuration parameters will be changed as follows:
innodb_lru_flush_size=32 (new: how many pages to flush on LRU eviction)
innodb_lru_scan_depth=1536 (old: 1024)
innodb_max_dirty_pages_pct=90 (old: 75)
innodb_max_dirty_pages_pct_lwm=75 (old: 0)
Note: The parameter innodb_lru_scan_depth will only affect LRU
eviction of buffer pool pages when a new page is being allocated. The
page cleaner thread will no longer evict any pages. It used to
guarantee that some pages will remain free in the buffer pool. Now, we
perform that eviction 'on demand' in buf_LRU_get_free_block().
The parameter innodb_lru_scan_depth(srv_LRU_scan_depth) is used as follows:
* When the buffer pool is being shrunk in buf_pool_t::withdraw_blocks()
* As a buf_pool.free limit in buf_LRU_list_batch() for terminating
the flushing that is initiated e.g., by buf_LRU_get_free_block()
The parameter also used to serve as an initial limit for unzip_LRU
eviction (evicting uncompressed page frames while retaining
ROW_FORMAT=COMPRESSED pages), but now we will use a hard-coded limit
of 100 or unlimited for invoking buf_LRU_scan_and_free_block().
The status variables will be changed as follows:
innodb_buffer_pool_pages_flushed: This includes also the count of
innodb_buffer_pool_pages_LRU_flushed and should work reliably,
updated one by one in buf_flush_page() to give more real-time
statistics. The function buf_flush_stats(), which we are removing,
was not called in every code path. For both counters, we will use
regular variables that are incremented in a critical section of
buf_pool.mutex. Note that show_innodb_vars() directly links to the
variables, and reads of the counters will *not* be protected by
buf_pool.mutex, so you cannot get a consistent snapshot of both variables.
The following INFORMATION_SCHEMA.INNODB_METRICS counters will be
removed, because the page cleaner no longer deals with writing or
evicting least recently used pages, and because the single-page writes
have been removed:
* buffer_LRU_batch_flush_avg_time_slot
* buffer_LRU_batch_flush_avg_time_thread
* buffer_LRU_batch_flush_avg_time_est
* buffer_LRU_batch_flush_avg_pass
* buffer_LRU_single_flush_scanned
* buffer_LRU_single_flush_num_scan
* buffer_LRU_single_flush_scanned_per_call
When moving to a single buffer pool instance in MDEV-15058, we missed
some opportunity to simplify the buf_flush_page_cleaner thread. It was
unnecessarily using a mutex and some complex data structures, even
though we always have a single page cleaner thread.
Furthermore, the buf_flush_page_cleaner thread had separate 'recovery'
and 'shutdown' modes where it was waiting to be triggered by some
other thread, adding unnecessary latency and potential for hangs in
relatively rarely executed startup or shutdown code.
The page cleaner was also running two kinds of batches in an
interleaved fashion: "LRU flush" (writing out some least recently used
pages and evicting them on write completion) and the normal batches
that aim to increase the MIN(oldest_modification) in the buffer pool,
to help the log checkpoint advance.
The buf_pool.flush_list flushing was being blocked by
buf_block_t::lock for no good reason. Furthermore, if the FIL_PAGE_LSN
of a page is ahead of log_sys.get_flushed_lsn(), that is, what has
been persistently written to the redo log, we would trigger a log
flush and then resume the page flushing. This would unnecessarily
limit the performance of the page cleaner thread and trigger the
infamous messages "InnoDB: page_cleaner: 1000ms intended loop took 4450ms.
The settings might not be optimal" that were suppressed in
commit
|
||
|---|---|---|
| .. | ||
| atomic | ||
| mysql | ||
| aria_backup.h | ||
| assume_aligned.h | ||
| big_endian.h | ||
| byte_order_generic.h | ||
| byte_order_generic_x86.h | ||
| byte_order_generic_x86_64.h | ||
| CMakeLists.txt | ||
| decimal.h | ||
| dur_prop.h | ||
| errmsg.h | ||
| ft_global.h | ||
| handler_ername.h | ||
| handler_state.h | ||
| hash.h | ||
| heap.h | ||
| ilist.h | ||
| json_lib.h | ||
| keycache.h | ||
| lf.h | ||
| little_endian.h | ||
| m_ctype.h | ||
| m_string.h | ||
| ma_dyncol.h | ||
| maria.h | ||
| my_alarm.h | ||
| my_alloc.h | ||
| my_atomic.h | ||
| my_atomic_wrapper.h | ||
| my_attribute.h | ||
| my_base.h | ||
| my_bit.h | ||
| my_bitmap.h | ||
| my_byteorder.h | ||
| my_check_opt.h | ||
| my_compare.h | ||
| my_compiler.h | ||
| my_context.h | ||
| my_counter.h | ||
| my_cpu.h | ||
| my_crypt.h | ||
| my_dbug.h | ||
| my_decimal_limits.h | ||
| my_default.h | ||
| my_dir.h | ||
| my_getopt.h | ||
| my_global.h | ||
| my_handler_errors.h | ||
| my_libwrap.h | ||
| my_list.h | ||
| my_md5.h | ||
| my_net.h | ||
| my_nosys.h | ||
| my_pthread.h | ||
| my_rdtsc.h | ||
| my_rnd.h | ||
| my_service_manager.h | ||
| my_stack_alloc.h | ||
| my_stacktrace.h | ||
| my_sys.h | ||
| my_time.h | ||
| my_tree.h | ||
| my_uctype.h | ||
| my_user.h | ||
| my_valgrind.h | ||
| my_xml.h | ||
| myisam.h | ||
| myisamchk.h | ||
| myisammrg.h | ||
| myisampack.h | ||
| mysql.h | ||
| mysql.h.pp | ||
| mysql_async.h | ||
| mysql_com.h | ||
| mysql_com_server.h | ||
| mysql_embed.h | ||
| mysql_time.h | ||
| mysql_version.h.in | ||
| mysqld_default_groups.h | ||
| mysys_err.h | ||
| pack.h | ||
| password.h | ||
| pfs_file_provider.h | ||
| pfs_idle_provider.h | ||
| pfs_memory_provider.h | ||
| pfs_metadata_provider.h | ||
| pfs_socket_provider.h | ||
| pfs_stage_provider.h | ||
| pfs_statement_provider.h | ||
| pfs_table_provider.h | ||
| pfs_thread_provider.h | ||
| pfs_transaction_provider.h | ||
| probes_mysql.d.base | ||
| probes_mysql.h | ||
| probes_mysql_nodtrace.h.in | ||
| queues.h | ||
| rijndael.h | ||
| scope.h | ||
| service_versions.h | ||
| source_revision.h.in | ||
| span.h | ||
| sql_common.h | ||
| ssl_compat.h | ||
| sslopt-case.h | ||
| sslopt-longopts.h | ||
| sslopt-vars.h | ||
| t_ctype.h | ||
| thr_alarm.h | ||
| thr_lock.h | ||
| thr_timer.h | ||
| thread_pool_priv.h | ||
| typelib.h | ||
| violite.h | ||
| waiting_threads.h | ||
| welcome_copyright_notice.h | ||
| wqueue.h | ||
| wsrep.h | ||