Include extra input variables into block cross-validation in Verde and using polygons as coordinates

[AAzzam91]

Hi @leouieda,

I came across this interesting tool and wanted to use some of the functions in this package for my master thesis for handling spatial data. I would like to ask two questions:

The first question that I would like to ask is about the train_test_split with spacing and the cross validation (BlockKFold and cross_val_score) functions. I have read the documentation and source code but could not find a direct approach to the following:
How can I include extra independent or feature variables (input) into the following functions:

• verde.train_test_split(coordinates, data, weights=None, spacing=None, shape=None, **kwargs)
• verde.BlockKFold() with verde.cross_val_score(estimator, coordinates, data, weights=None, cv=None, client=None,
  delayed=False, scoring=None)

In the documentation examples I only see that the coordinates = X (two coordinate variables) and data = y (one target variable).
Would it be possible to add extra input variables (X) besides the coordinates and still having the data split into blocks?
If this would be possible, should those extra variables be added into the data argument (y) and then be splitted into X_train, y_train, X_test, y_test? or is there another approach to handle this?

The second question that I would like to ask is about the coordinates input type.
Could for example polygons be used as coordinates inputs instead of points? or is it only possible to input coordinates such as x, y or lon, lat?

The dataset that I am using has:

• around 10 numerical independent X variables
• geometry variable (polygon grid)
• centroid x, y variable from polygon grid
• 1 numerical target variable

Thank you for your advice and help.

Kind regards,

A.Azzam

[leouieda]

👋🏾 Hi @AAzzam91 thanks for asking this! I'm going to transfer this to our Forum instead since it's more of a questions. You don't have to do anything and should get automatically redirected there.

[leouieda]

Done! Now this is a bit better for this type of question than the issues.

[leouieda]

How can I include extra independent or feature variables (input) into the following functions:

That was a bit of an oversight on my part. It should be possible to make verde.BlockKFold more flexible so that it can take X with more than 2 features. Right now that doesn't work because of this line: https://github.com/fatiando/verde/blob/main/verde/base/base_classes.py#L81

Making it more flexible would mean remove that restriction and passing an argument to tell the class which features are the coordinates used for blocking. This is an interesting idea and I opened this issue to keep track of progress on that: fatiando/verde#398 Thanks!

The second question that I would like to ask is about the coordinates input type.
Could for example polygons be used as coordinates inputs instead of points? or is it only possible to input coordinates such as x, y or lon, lat?

The Verde block CV is a bit more limited in terms of scope and is mostly meant for point data. Handling geometries gets into things like geopandas and a bit outside of our expertise. While that's a great use case, it would be a bit outside the scope of what we could do in a single package.

So the answer here is "no". Only points such as x, y or lon, lat are allowed. But this may be doable or maybe in scope for the spacv package by @SamComber: https://github.com/SamComber/spacv Worth asking there.

---

Sorry that this may not be a satisfying answer but I hope it gives you some ideas to move forward 🙂

[AAzzam91]

Hi @leouieda, Thank you for the reply and clear answers.

Concerning the first answer: I am not entirely sure whether it is correct, but it worked for me to include more variables in the 'y' input. After splitting the data on the coordinates ('X' input), I have indexed the train test blocks to retrieve a X_train, y_train, X_test and y_test. For instance, see below example:

#BLOCK SPLIT
coordinatesx = a tuple of my x and y coordinates
valuesttestlagknn = a tuple containing my extra X input variables and at the end my output (target) y variable

trainblock, testblock = vd.train_test_split(coordinatesx, valuesttestlagknn, test_size=0.25, spacing = 2000, random_state=0)
X_train_b = np.stack(trainblock[1][:-1], axis=1)
X_test_b = np.stack(testblock[1][:-1], axis=1)
y_train_b = trainblock[1][-1]
y_test_b = testblock[1][-1]

I have inspected the output of these splits and it seemed that the blocks that were split on coordinates also line up with the correct X and y variables. However, I am not entirely sure if this could be a correct approach but for me it worked. Do you think that this makes sense?

Concerning the second answer: Thank you for sharing the interesting spacv package. I have not yet inspected it but will look at it soon.
I have kept using points as inputs for now.

Kind regards,

A.Azzam