Data visualization on Plotly ~~ animated subplots

[Aubert-Antoine]

Hello everyone!

I'm currently working on a K-Nearest Neighbor (KNN) regression project, and I'm having some difficulties visualizing the results using Plotly.

Context:
In this project, I am using the KNN algorithm to predict the value of a continuous target variable. The dataset consists of 20 features and 1 target variable. After running the KNN algorithm with different numbers of neighbors (k), I obtain the Mean Squared Error (MSE) and the predicted vs. actual values for each k.

Objective:
My objective is to create an animated Plotly figure that displays two subplots side-by-side.

1. The first subplot should be a scatter plot of the predicted vs. actual values for the current k, with a trendline that changes color according to the current MSE value. I would also like to add a diagonal line representing the "best case" scenario where predicted = actual.
2. The second subplot should be a line plot of the MSE value for each k, with a marker indicating the current k value and a color that changes according to the MSE value.

Difficulties:

1. IDK if it is possible… (chat-GPT is not able to provide any good piece of code)
2. I really want the 2 sub-plots are animated in the same time
3. The type of my plots is maybe not compatible together.

Existing code :

fig = make_subplots(rows=1, cols=2, column_widths=[0.7, 0.3])

# Create an animated scatter plot with a slider for k_n
scatter_fig = px.scatter(df, x='val_obs', y='val_pred', animation_frame='k_n',
                         color='MSE',
                         range_x=[df['val_obs'].min(), df['val_obs'].max()],
                         range_y=[df['val_pred'].min(), df['val_pred'].max()],
                         title="KNN Regression",
                         color_continuous_scale='Viridis')

fig.add_trace(scatter_fig.data[0], row=1, col=1)

fig.add_shape(type='line', x0=df['val_obs'].min(), y0=df['val_obs'].min(),
              x1=df['val_obs'].max(), y1=df['val_obs'].max(),
              line=dict(color='green', width=1, dash='dash'), row=1, col=1)

fig.add_trace(go.Scatter(x=df['val_obs'], y=df['val_obs'],
                         mode='lines', line=dict(color='orange'), name='Regression Line'),
              row=1, col=1)

fig.add_trace(go.Scatter(x=df['k_n'], y=df['MSE']),
              row=1, col=2)

# Update the layout to show the slider
fig.update_layout(updatemenus=[dict(type='buttons',
                                    showactive=False,
                                    buttons=[dict(label='Play',
                                                  method='animate',
                                                  args=[None,
                                                        dict(frame=dict(duration=50),
                                                             fromcurrent=True,
                                                             transition=dict(duration=0))
                                                        ])
                                             ]
                                    )
                               ])

# Show the plot
fig.show()

This is the output I would like, but it is not working

My df :

k_n	MSE	val_obs	val_pred
count	4620.000000	4620.000000	4620.000000
mean	10.500000	19.706844	-0.039782
std	5.766905	8.799771	1.020814
min	1.000000	11.359861	-1.616203
25%	5.750000	17.912610	-0.948785
50%	10.500000	18.686920	-0.524334
75%	15.250000	19.903318	0.873971
max	20.000000	55.991032	2.062035

Thanks for any help
Antoine