The actual size of fold for 5-fold cross validation

[povolann]

Hello,

if I understood correctly when training the data, 1/5 of the training data set is used for validation without being trained and in the same moment the rest of the training data set is used for the training.
So, I tried to run training with nnUNetTrainerV2 for the kits19 challenge data but I had only 21 cases in the VALIDATION KEYS and 80 cases in the TRAINING KEYS. And I was expecting 42 cases for validation and 168 cases for training. I am sure that I'm missing something, but I can't figure out what.

Thank you for your answer!
Anya

[FabianIsensee]

Honestly no idea what is going on. In my experiments I have the correct number of cases.

Fold 0:

2020-01-07 17:21:18.764138: TRAINING KEYS:
odict_keys(['case_00000', 'case_00001', 'case_00002', 'case_00003', 'case_00004', 'case_00005', 'case_00006', 'case_00007', 'case_00008', 'case_00009', 'case_00010', 'case_00011', 'case_00012', 'case_00013', 'case_00015', 'case_00018', 'case_00021', 'case_00022', 'case_00023', 'case_00024', 'case_00025', 'case_00026', 'case_00028', 'case_00029', 'case_00031', 'case_00032', 'case_00034', 'case_00035', 'case_00036', 'case_00037', 'case_00038', 'case_00039', 'case_00040', 'case_00041', 'case_00042', 'case_00043', 'case_00044', 'case_00045', 'case_00046', 'case_00048', 'case_00050', 'case_00051', 'case_00053', 'case_00055', 'case_00057', 'case_00058', 'case_00059', 'case_00060', 'case_00061', 'case_00062', 'case_00063', 'case_00064', 'case_00065', 'case_00066', 'case_00069', 'case_00071', 'case_00072', 'case_00073', 'case_00074', 'case_00075', 'case_00076', 'case_00077', 'case_00079', 'case_00080', 'case_00081', 'case_00082', 'case_00083', 'case_00084', 'case_00085', 'case_00086', 'case_00087', 'case_00088', 'case_00090', 'case_00091', 'case_00093', 'case_00095', 'case_00096', 'case_00097', 'case_00098', 'case_00100', 'case_00101', 'case_00102', 'case_00104', 'case_00105', 'case_00106', 'case_00107', 'case_00108', 'case_00109', 'case_00111', 'case_00112', 'case_00113', 'case_00114', 'case_00115', 'case_00117', 'case_00118', 'case_00119', 'case_00120', 'case_00121', 'case_00122', 'case_00123', 'case_00124', 'case_00125', 'case_00126', 'case_00127', 'case_00129', 'case_00130', 'case_00131', 'case_00132', 'case_00133', 'case_00134', 'case_00135', 'case_00136', 'case_00137', 'case_00138', 'case_00140', 'case_00141', 'case_00142', 'case_00143', 'case_00144', 'case_00145', 'case_00146', 'case_00147', 'case_00150', 'case_00151', 'case_00154', 'case_00156', 'case_00157', 'case_00158', 'case_00159', 'case_00161', 'case_00162', 'case_00163', 'case_00164', 'case_00166', 'case_00167', 'case_00168', 'case_00169', 'case_00170', 'case_00171', 'case_00172', 'case_00173', 'case_00174', 'case_00175', 'case_00176', 'case_00177', 'case_00182', 'case_00183', 'case_00184', 'case_00185', 'case_00186', 'case_00187', 'case_00189', 'case_00190', 'case_00191', 'case_00192', 'case_00193', 'case_00195', 'case_00197', 'case_00198', 'case_00199', 'case_00200', 'case_00201', 'case_00202', 'case_00203', 'case_00206', 'case_00207', 'case_00208', 'case_00209'])
2020-01-07 17:21:18.773306: VALIDATION KEYS:
odict_keys(['case_00014', 'case_00016', 'case_00017', 'case_00019', 'case_00020', 'case_00027', 'case_00030', 'case_00033', 'case_00047', 'case_00049', 'case_00052', 'case_00054', 'case_00056', 'case_00067', 'case_00068', 'case_00070', 'case_00078', 'case_00089', 'case_00092', 'case_00094', 'case_00099', 'case_00103', 'case_00110', 'case_00116', 'case_00128', 'case_00139', 'case_00148', 'case_00149', 'case_00152', 'case_00153', 'case_00155', 'case_00160', 'case_00165', 'case_00178', 'case_00179', 'case_00180', 'case_00181', 'case_00188', 'case_00194', 'case_00196', 'case_00204', 'case_00205'])

Maybe your preprocessing was incomplete? How many cases are in your preprocessed folder (number of npz files)? Maybe delete splits_final.pkl?

[povolann]

Thank you for the super fast answer :) So, my understading of the 5-fold cross validation is actually correct?

nnUNetData_plans_v2.1_stage0 - in this file I have 210 npz files
nnUNetData_plans_v2.1_stage1 - in this file I have 101 npz files, corresponds to the 80+21 cases

In this case...should I run the preprocessing again?

[FabianIsensee]

Your 3d_fullres preprocessing broke. nnUNetData_plans_v2.1_stage1 is incomplete.
Please reduce the number of threads with -tf then running nnUNet_plan_and_preprocess to ensure that doesn't happen

[povolann]

Ok, thank you.