Followup: remove this line from get_column_range_cardinality()
set_if_bigger(res, col_stats->get_avg_frequency());
and make sure it is only used with the binary histograms.
For JSON histograms, it makes the estimates unnecessarily imprecise.
Previous JSON parser was using an API which made the parsing
inefficient: the same JSON contents was parsed again and again.
Switch to using a lower-level parsing API which allows to do
parsing in an efficient way.
- Make Histogram_json_hb::range_selectivity handle singleton buckets
specially when computing selectivity of the max. endpoint bound.
(for min. endpoint, we already do that).
- Also, fixed comments for Histogram_json_hb::find_bucket
- Use String::c_ptr_safe() instead of String::c_ptr
- Do proper datatype conversions in Histogram_json_hb::parse
- Remove Histogram_json_hb::Bucket::end_value. Introduce
get_end_value() instead.
Basic ideas:
1. Store "popular" values in their own buckets.
2. Also store ndv (Number of Distinct Values) in each bucket.
Because of #1, the buckets are now variable-size, so store the size in
each bucket.
Adjust selectivity estimation functions accordingly.
