CHANGELOG.md

Changelog

27.0.0 (2022-11-11)

Full Changelog

Breaking changes:

• Recurse into Dictionary value type in DataType::is_nested #3083 [arrow] (tustvold)
• early type checks in RowConverter #3080 [arrow] (crepererum)
• Add Decimal128 and Decimal256 to downcast_primitive #3056 [arrow] (viirya)
• Replace remaining _generic temporal kernels with _dyn kernels #3046 [arrow] (viirya)
• Replace year_generic with year_dyn #3041 [arrow] (viirya)
• Validate decimal256 with i256 directly #3025 [arrow] (viirya)
• Hadoop LZ4 Support for LZ4 Codec #3013 [parquet] (marioloko)
• Replace hour_generic with hour_dyn #3006 [arrow] (viirya)
• Accept any &dyn Array in nullif kernel #2940 [arrow] (tustvold)

Implemented enhancements:

• Row Format: Option to detach/own a row #3078 [arrow]
• Row Format: API to check if datatypes are supported #3077 [arrow]
• Deprecate Buffer::count_set_bits #3067 [arrow]
• Add Decimal128 and Decimal256 to downcast_primitive #3055 [arrow]
• Improved UX of creating TimestampNanosecondArray with timezones #3042 [arrow]
• Cast decimal256 to signed integer #3039 [arrow]
• Support casting Date64 to Timestamp #3037 [arrow]
• Check overflow when casting floating point value to decimal256 #3032 [arrow]
• Compare i256 in validate_decimal256_precision #3024 [arrow]
• Check overflow when casting floating point value to decimal128 #3020 [arrow]
• Add macro downcast_temporal_array #3008 [arrow]
• Replace hour_generic with hour_dyn #3005 [arrow]
• Replace temporal _generic kernels with dyn #3004 [arrow]
• Add RowSelection::intersection #3003 [parquet]
• I would like to round rather than truncate when casting f64 to decimal #2997 [arrow]
• arrow::compute::kernels::temporal should support nanoseconds #2995 [arrow]
• Release Arrow 26.0.0 (next release after 25.0.0) #2953 [parquet] [arrow] [arrow-flight]
• Add timezone offset for debug format of Timestamp with Timezone #2917 [arrow]
• Support merge RowSelectors when creating RowSelection #2858 [parquet]

Fixed bugs:

• Inconsistent Nan Handling Between Scalar and Non-Scalar Comparison Kernels #3074 [arrow]
• Debug format for timestamp ignores timezone #3069 [arrow]
• Row format decode loses timezone #3063 [arrow]
• binary operator produces incorrect result on arrays with resized null buffer #3061 [arrow]
• RLEDecoder Panics on Null Padded Pages #3035 [parquet]
• Nullif with incorrect valid_count #3031 [arrow]
• RLEDecoder::get_batch_with_dict may panic on bit-packed runs longer than 1024 #3029 [parquet]
• Converted type is None according to Parquet Tools then utilizing logical types #3017
• CompressionCodec LZ4 incompatible with C++ implementation #2988 [parquet]

Documentation updates:

• Mark parquet predicate pushdown as complete #2987 [parquet] (tustvold)

Merged pull requests:

• Improved UX of creating TimestampNanosecondArray with timezones #3088 [arrow] (src255)
• Remove unused range module #3085 [parquet] (tustvold)
• Make intersect_row_selections a member function #3084 [parquet] (tustvold)
• Update hashbrown requirement from 0.12 to 0.13 #3081 [parquet] [arrow] (dependabot[bot])
• feat: add OwnedRow #3079 [arrow] (crepererum)
• Use ArrowNativeTypeOp on non-scalar comparison kernels #3075 [arrow] (viirya)
• Add missing inline to ArrowNativeTypeOp #3073 [arrow] (tustvold)
• fix debug information for Timestamp with Timezone #3072 [arrow] (waitingkuo)
• Deprecate Buffer::count_set_bits (#3067) #3071 [arrow] (tustvold)
• Add compare to ArrowNativeTypeOp #3070 [arrow] (tustvold)
• Minor: Improve docstrings on WriterPropertiesBuilder #3068 [parquet] (alamb)
• Faster f64 inequality #3065 [arrow] (tustvold)
• Fix row format decode loses timezone (#3063) #3064 [arrow] (tustvold)
• Fix null_count computation in binary #3062 [arrow] (viirya)
• Faster f64 equality #3060 [arrow] (tustvold)
• Update arrow-flight subcrates (#3044) #3052 [arrow-flight] (tustvold)
• Minor: Remove cloning ArrayData in with_precision_and_scale #3050 [arrow] (viirya)
• Split out arrow-json (#3044) #3049 [arrow] (tustvold)
• Move intersect_row_selections from datafusion to arrow-rs. #3047 [parquet] (Ted-Jiang)
• Split out arrow-csv (#2594) #3044 [arrow] (tustvold)
• Move reader_parser to arrow-cast (#3022) #3043 [arrow] (tustvold)
• Cast decimal256 to signed integer #3040 [arrow] (viirya)
• Enable casting from Date64 to Timestamp #3038 [arrow] (gruuya)
• Fix decoding long and/or padded RLE data (#3029) (#3035) #3036 [parquet] (tustvold)
• Fix nullif when existing array has no nulls #3034 [arrow] (tustvold)
• Check overflow when casting floating point value to decimal256 #3033 [arrow] (viirya)
• Update parquet to depend on arrow subcrates #3028 [parquet] (tustvold)
• Make various i256 methods const #3026 [arrow] (tustvold)
• Split out arrow-ipc #3022 [arrow] (tustvold)
• Check overflow while casting floating point value to decimal128 #3021 [arrow] (viirya)
• Update arrow-flight #3019 [arrow-flight] (tustvold)
• Move ArrowNativeTypeOp to arrow-array (#2594) #3018 [arrow] (tustvold)
• Support cast timestamp to time #3016 [arrow] (naosense)
• Add filter example #3014 [arrow] (tustvold)
• Check overflow when casting integer to decimal #3009 [arrow] (viirya)
• Add macro downcast_temporal_array #3007 [arrow] (viirya)
• Parquet Writer: Make column descriptor public on the writer #3002 [parquet] (pier-oliviert)
• Update chrono-tz requirement from 0.7 to 0.8 #3001 [arrow] (dependabot[bot])
• Round instead of Truncate while casting float to decimal #3000 [arrow] (waitingkuo)
• Support Predicate Pushdown for Parquet Lists (#2108) #2999 [parquet] (tustvold)
• Split out arrow-cast (#2594) #2998 [arrow] (tustvold)
• arrow::compute::kernels::temporal should support nanoseconds #2996 [arrow] (comphead)
• Add RowSelection::from_selectors_and_combine to merge RowSelectors #2994 [parquet] (Ted-Jiang)
• Simplify Single-Column Dictionary Sort #2993 [arrow] (tustvold)
• Minor: Add entry to changelog for 26.0.0 RC2 fix #2992 (alamb)
• Fix ignored limit on lexsort_to_indices #2991 [arrow] (alamb)
• Add clone and equal functions for CastOptions #2985 [arrow] (askoa)
• minor: remove redundant prefix #2983 [arrow] [arrow-flight] (jackwener)
• Compare dictionary decimal arrays #2982 [arrow] (viirya)
• Compare dictionary and non-dictionary decimal arrays #2980 [arrow] (viirya)
• Add decimal comparison kernel support #2978 [arrow] (viirya)
• Move concat kernel to arrow-select (#2594) #2976 [arrow] (tustvold)
• Specialize interleave for byte arrays (#2864) #2975 [arrow] (tustvold)
• Use unary function for numeric to decimal cast #2973 [arrow] (viirya)
• Specialize filter kernel for binary arrays (#2969) #2971 [arrow] (tustvold)
• Combine take_utf8 and take_binary (#2969) #2970 [arrow] (tustvold)
• Faster Scalar Dictionary Comparison ~10% #2968 [arrow] (tustvold)
• Move byte_size from datafusion::physical_expr #2965 [arrow] (avantgardnerio)
• Pass decompressed size to parquet Codec::decompress (#2956) #2959 [parquet] (marioloko)
• Add Decimal Arithmetic #2881 [arrow] (tustvold)

* This Changelog was automatically generated by github_changelog_generator