PARTITION Procedure
Example 24.4 k-Fold Partitioning
(View the complete code for this example.)
This example demonstrates how to use PROC PARTITION to perform stratified k-fold partitioning to partition the data; it uses the same data table as is used in Example 24.1.
You can load the sampsio.hmeq data set into your CAS session by naming your CAS engine libref in the first statement of the following DATA step. This DATA step assumes that your CAS engine libref is named mylib, but you can substitute any appropriately defined CAS engine libref.
data mylib.hmeq;
set sampsio.hmeq;
run;
The following statements perform the partitioning:
proc partition data=mylib.hmeq kfold=10 seed=10 nthreads=3;
by BAD;
output out=mylib.out5 copyvars=(job reason loan value delinq derog);
run;
proc print data=mylib.out5(obs=20);
run;
The KFOLD=10 option requests that the input data be partitioned to 10 equal-size folds within each level of the BY variable BAD. The SEED= option specifies 10 as the random seed to be used in the partitioning process. The OUTPUT statement requests that the sampled data be stored in a table named mylib.out5, and the COPYVARS= option lists the variables to be copied from mylib.hmeq to mylib.out5. The output data table, mylib.out5, includes an indicator that shows which fold each observation is selected to.
Output 24.4.1 shows the frequency information for each level of the BY variable BAD in the mylib.out5 data table.
Output 24.4.1: Frequency Information Table
| k-Fold Partition Frequency | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Index | BAD | Number of Obs | Fold Size 1 | Fold Size 2 | Fold Size 3 | Fold Size 4 | Fold Size 5 | Fold Size 6 | Fold Size 7 | Fold Size 8 | Fold Size 9 | Fold Size 10 |
| 0 | 0 | 4771 | 478 | 477 | 477 | 477 | 477 | 477 | 477 | 477 | 477 | 477 |
| 1 | 1 | 1189 | 119 | 118 | 119 | 119 | 119 | 119 | 119 | 119 | 119 | 119 |
Output 24.4.2 shows the first 20 output sample observations in mylib.out5; the _Fold_ column shows which fold the observation is selected to.
Output 24.4.2: k-Fold Partitioning Output Data Table
| Obs | JOB | REASON | LOAN | VALUE | DELINQ | DEROG | _Fold_ |
|---|---|---|---|---|---|---|---|
| 1 | Other | HomeImp | 1100 | 39025 | 0 | 0 | 1 |
| 2 | Other | HomeImp | 1300 | 68400 | 2 | 0 | 5 |
| 3 | Other | HomeImp | 1500 | 16700 | 0 | 0 | 3 |
| 4 | 1500 | . | . | . | 3 | ||
| 5 | Office | HomeImp | 1700 | 112000 | 0 | 0 | 10 |
| 6 | Other | HomeImp | 1700 | 40320 | 0 | 0 | 7 |
| 7 | Other | HomeImp | 1800 | 57037 | 2 | 3 | 9 |
| 8 | Other | HomeImp | 1800 | 43034 | 0 | 0 | 6 |
| 9 | Other | HomeImp | 2000 | 46740 | 2 | 0 | 5 |
| 10 | Sales | HomeImp | 2000 | 62250 | 0 | 0 | 4 |
| 11 | 2000 | . | . | . | 6 | ||
| 12 | Office | HomeImp | 2000 | 29800 | 1 | 0 | 2 |
| 13 | Other | HomeImp | 2000 | 55000 | 0 | 0 | 4 |
| 14 | Mgr | 2000 | 87400 | 0 | 0 | 2 | |
| 15 | Other | HomeImp | 2100 | 83850 | 1 | 0 | 3 |
| 16 | Other | HomeImp | 2200 | 34687 | 1 | 0 | 1 |
| 17 | Mgr | HomeImp | 2200 | 102600 | 6 | 2 | 4 |
| 18 | 2200 | . | . | . | 5 | ||
| 19 | Other | HomeImp | 2300 | 40150 | 0 | 0 | 4 |
| 20 | Office | HomeImp | 2300 | 120953 | 0 | 0 | 4 |