Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to do a change_table to change partition key #64

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

louiseGrandjonc
Copy link
Contributor

@louiseGrandjonc louiseGrandjonc commented Sep 11, 2019

This PR allows a user to create a migration to do

  change_table :project_categories, partition_key: :account_id do |t|
  end

Which will create the primary key (partittion_key, id).

Things this PR does not do:

  • Allow the user to reverse his decision (go back to pkey: id instead of partition_key, id). This should be okay as users rarely undistribute a table, and they could just drop the migration
  • Check that the primary key already has the right columns before dropping and creating the pkey. Though it checks the number of column.

"ON kcu.constraint_name = tco.constraint_name " \
"AND kcu.constraint_schema = tco.constraint_schema "\
"WHERE tco.constraint_type = 'PRIMARY KEY' " \
"AND tco.constraint_name = '#{table_name}_pkey'")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use a bind parameter here, instead of string substitution?

Whilst its unlikely that someone will do SQL injection here, it seems better to avoid problems with table_name called '; DROP TABLE xyz; SELECT '

columns_result.values.each do |c| columns.push(c[0]) end

if columns.length != pkey_columns.length
execute "ALTER TABLE #{table_name} DROP CONSTRAINT IF EXISTS #{table_name}_pkey"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here regarding SQL injection - I think there are methods to escape identifier names in ActiveRecord::Base that we can use

Copy link
Contributor Author

@louiseGrandjonc louiseGrandjonc Sep 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I tried looking into it, the method to sanitize raw sql is sanitize_sql_array, except here, we are taking the name of the table in the ALTER TABLE which ends up like this:

query = ActiveRecord::Base::sanitize_sql_array(["ALTER TABLE $1 DROP CONSTRAINT IF EXISTS $2_pkey;", table_name, table_name])
execute(query)

And this triggers errors because SQL doesn't suppose that format

If you have an idea how to do change the queries, let me know

def create_table(table_name, options = {}, &block)
ret = orig_create_table(table_name, options.except(:partition_key), &block)
if options[:partition_key] && options[:partition_key].to_s != 'id'
execute "ALTER TABLE #{table_name} DROP CONSTRAINT #{table_name}_pkey"
execute "ALTER TABLE #{table_name} ADD PRIMARY KEY(id, \"#{options[:partition_key]}\")"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course, to be fair my own code was not up to my comments :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

haha, noted, I'll work on avoiding SQL injection ;)

spec/schema.rb Outdated
create_distributed_table :allowed_places, :account_id
create_reference_table :categories

change_table :project_categories, partition_key: :account_id do |t|
end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can not pass the block at all, unless that throws an error:

change_table :project_categories, partition_key: :account_id


if columns_result.present?
columns = []
columns_result.values.each do |c| columns.push(c[0]) end
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For one liners usually in Ruby you'd use the { .. } syntax:

columns_result.values.each { |c| columns.push(c[0]) }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there anyway to do this without initializing a constant? in python you can do
columns = [e[0] for e in columns_result.values]

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, you could do it like this:

columns = columns_result.values.map { |c| c[0] } 

or without the block, utilizing the fact that there is a first method on arrays:

columns = columns_result.values.map(&:first)

@louiseGrandjonc louiseGrandjonc changed the title allow to do a change_table to change partition key [WIP] allow to do a change_table to change partition key Sep 12, 2019
@louiseGrandjonc louiseGrandjonc changed the title [WIP] allow to do a change_table to change partition key Allow to do a change_table to change partition key Sep 17, 2019
@brunoprietog
Copy link

Hi, are you planning to merge this?

@serprex
Copy link
Collaborator

serprex commented Nov 18, 2022

No, both Louis & Lukas are no longer with Citus

If someone else wants this to get through they can create a new PR based on this one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants