Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL

column is used for ORDER BY

Problem: filesort isn't meant for null length sort data
(e.g. char(0)), that leads to a server crash.

Fix: disregard sort order if sort data record length is 0 (nothing
to sort).


mysql-test/r/select.result:
  Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL 
  column is used for ORDER BY
    - test result.
mysql-test/t/select.test:
  Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL 
  column is used for ORDER BY
    - test case.
sql/filesort.cc:
  Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL 
  column is used for ORDER BY
    - assert added as filesort cannot handle null length sort data.
sql/sql_select.cc:
  Fix for bug#49897: crash in ptr_compare when char(0) NOT NULL 
  column is used for ORDER BY
    - don't sort null length data e.g. in case of ORDER BY CHAR(0).
This commit is contained in:
Ramil Kalimullin 2010-01-29 13:17:57 +04:00
commit 172af3722e
4 changed files with 119 additions and 0 deletions

View file

@ -134,6 +134,8 @@ ha_rows filesort(THD *thd, TABLE *table, SORT_FIELD *sortorder, uint s_length,
error= 1;
bzero((char*) &param,sizeof(param));
param.sort_length= sortlength(thd, sortorder, s_length, &multi_byte_charset);
/* filesort cannot handle zero-length records. */
DBUG_ASSERT(param.sort_length);
param.ref_length= table->file->ref_length;
param.addon_field= 0;
param.addon_length= 0;