Fix/type preservation empty dataframes

[dan-corneanu]

It looks like manipulating a column in an empty data frame defaults the result to a type of :string.

[kou]

Could you also add a fix to this PR?

diff --git a/lib/red_amber/data_frame_variable_operation.rb b/lib/red_amber/data_frame_variable_operation.rb
index 7a5179e..62b0706 100755
--- a/lib/red_amber/data_frame_variable_operation.rb
+++ b/lib/red_amber/data_frame_variable_operation.rb
@@ -675,9 +675,18 @@ module RedAmber
           raise DataFrameArgumentError, "Data size mismatch (#{data.size} != #{size})"
         end
 
-        a = Arrow::Array.new(data.is_a?(Vector) ? data.to_a : data)
+        if data.respond_to?(:to_arrow_chunked_array)
+          chunked_array = data.to_arrow_chunked_array
+        else
+          if data.respond_to?(:to_arrow_array)
+            a = data.to_arrow_array
+          else
+            a = Arrow::Array.new(data)
+          end
+          chunked_array = Arrow::ChunkedArray.new([a])
+        end
         fields[i] = Arrow::Field.new(key, a.value_data_type)
-        arrays[i] = Arrow::ChunkedArray.new([a])
+        arrays[i] = chunked_array
       end
       [fields, arrays]
     end
diff --git a/lib/red_amber/vector.rb b/lib/red_amber/vector.rb
index 7237807..5267eb6 100644
--- a/lib/red_amber/vector.rb
+++ b/lib/red_amber/vector.rb
@@ -198,6 +198,22 @@ module RedAmber
     alias_method :values, :to_ary
     alias_method :entries, :to_ary
 
+    # Convert to an Arrow::Array.
+    #
+    # @return [Arrow::Array]
+    #   Apache Arrow array representation.
+    def to_arrow_array
+      @data.to_arrow_array
+    end
+
+    # Convert to an Arrow::ChunkedArray.
+    #
+    # @return [Arrow::ChunkedArray]
+    #   Apache Arrow chunked array representation.
+    def to_arrow_chunked_array
+      @data.to_arrow_chunked_array
+    end
+
     # Indeces from 0 to size-1 by Array.
     #
     # @return [Array]

[dan-corneanu]

diff --git a/lib/red_amber/data_frame_variable_operation.rb b/lib/red_amber/data_frame_variable_operation.rb
index 7a5179e..62b0706 100755
--- a/lib/red_amber/data_frame_variable_operation.rb
+++ b/lib/red_amber/data_frame_variable_operation.rb
@@ -675,9 +675,18 @@ module RedAmber
           raise DataFrameArgumentError, "Data size mismatch (#{data.size} != #{size})"
         end
 
-        a = Arrow::Array.new(data.is_a?(Vector) ? data.to_a : data)
+        if data.respond_to?(:to_arrow_chunked_array)
+          chunked_array = data.to_arrow_chunked_array
+        else
+          if data.respond_to?(:to_arrow_array)
+            a = data.to_arrow_array
+          else
+            a = Arrow::Array.new(data)
+          end
+          chunked_array = Arrow::ChunkedArray.new([a])
+        end
         fields[i] = Arrow::Field.new(key, a.value_data_type)
-        arrays[i] = Arrow::ChunkedArray.new([a])
+        arrays[i] = chunked_array
       end
       [fields, arrays]
     end

@kou if if data.respond_to?(:to_arrow_chunked_array) is true, then a will be nil in this line fields[i] = Arrow::Field.new(key, a.value_data_type).

I don't completely understand how chucked arrays are manipulated but could we replace that line with fields[i] = Arrow::Field.new(key, chunked_array.value_data_type) ?

[kou]

Ah, sorry. We can use chunked_array.value_data_type instead of a.value_data_type.

[dan-corneanu]

@kou FYI I had to update this test after the change

[kou]

I'll fix build failures on Linux in upstream. Please wait for a while...