Data Science Pilot Action Set

Mutual-Information-Based Analysis of Missing Patterns Using the analyzeMissingPatterns Action

This section contains PROC CAS code.

Note: Input data must be accessible in your CAS session, either as a CAS table or as a transient-scope table. A CAS table has a two-level name: the first level is your CAS engine libref, and the second level is the table name. You refer to this table in the CAS procedure by specifying only the second level. For more information about two-level names, see Chapter 2, Shared Concepts (SAS Visual Data Mining and Machine Learning: Procedures). A transient-scope table is called directly from the action and exists in memory for the duration of the action. For more information about accessing data, see SAS Viya: System Programming Guide. For more information about PROC CAS and programming in CASL, see SAS Cloud Analytic Services: CASL Programmer’s Guide and SAS Cloud Analytic Services: CASL Reference.

The following DATA step creates the reference data table mycas.dmagecr in your CAS session, keeping only the age, amount, coapp, duration, foreign, job, and good_bad variables. To illustrate how you can use the analyzeMissingPatterns action to analyze missing patterns in a data table, artificially created and highly correlated missing patterns are created in the reference data table. These statements assume that your CAS engine libref is named mycas, but you can substitute any appropriately defined CAS engine libref.

data mycas.dmagecr;
   set sampsio.dmagecr;
   keep age amount coapp duration foreign job good_bad;
   call streaminit(12345);
   r = rand("Uniform");
   if r < 0.1 then
      do;
         age = .;
         amount = .;
      end;
   else if r < 0.2 then
      do;
         duration = .;
      end;
run;

The following statements run the analyzeMissingPatterns action to analyze the missing patterns in the input variables. The action outputs a data table that contains the mutual information of the variables among themselves and versus the target variable. It outputs the results of the analysis in an output data table named miss_pattern_out.

proc cas;
   loadactionset "dataSciencePilot";
   dataSciencePilot.analyzeMissingPatterns
                            / table     = "DMAGECR"
                              casOut    = {name    = "MISS_PATTERN_OUT",
                                           replace = True}
                              inputs    = {{name   = "age"},
                                           {name   = "amount"},
                                           {name   = "coapp"},
                                           {name   = "duration"},
                                           {name   = "foreign"},
                                           {name   = "job"}}
                              nominals  = {"coapp",  "foreign", "job"}
                              ;
   run;
   fetch / table = "MISS_PATTERN_OUT";
   run;
quit;

The table parameter names the input reference data table. The casOut parameter names the output data table. The inputs parameter specifies that the age, amount, coapp, duration, foreign, and job variables be used as inputs. The nominals parameter specifies that the coapp, foreign, and job variables be used as nominal variables.

The CAS output table is shown in Figure 1.

Figure 1: Missing Pattern

Results from table.fetch

Selected Rows from Table MISS_PATTERN_OUT
_Index_	FirstVariable	SecondVariable	Type	MI	NormMI	SU	EntropyPerChange
1	age	coapp	_mindmind_1	0	0	0	.
2	age	foreign	_mindmind_2	0	0	0	.
3	age	job	_mindmind_3	0	0	0	.
4	age	coapp	_mindnonm_1	0.0003542873	0.026614351	0.0006968001	0.065795892
5	age	foreign	_mindnonm_2	0.0008488509	0.0411856961	0.0024019622	0.3717095636
6	age	job	_mindnonm_3	0.0015108119	0.0549278009	0.0015971884	0.1068916187
7	age	amount	_mindnonm_5	0	0	0	0
8	age	duration	_mindnonm_6	0.0093443331	0.1360702612	0.0045803774	0.2619951725
9	amount	coapp	_mindmind_1	0	0	0	.
10	amount	foreign	_mindmind_2	0	0	0	.
11	amount	job	_mindmind_3	0	0	0	.
12	amount	age	_mindmind_4	0.4784338655	0.7847967318	1	100
13	amount	coapp	_mindnonm_1	0.0003542873	0.026614351	0.0006968001	0.065795892
14	amount	foreign	_mindnonm_2	0.0008488509	0.0411856961	0.0024019622	0.3717095636
15	amount	job	_mindnonm_3	0.0015108119	0.0549278009	0.0015971884	0.1068916187
16	amount	age	_mindnonm_4	0	0	0	0
17	amount	duration	_mindnonm_6	0.0093443331	0.1360702612	0.0045803774	0.2619951725
18	duration	coapp	_mindmind_1	0	0	0	.
19	duration	foreign	_mindmind_2	0	0	0	.
20	duration	job	_mindmind_3	0	0	0	.

Mutual-Information-Based Analysis of Missing Patterns Using the analyzeMissingPatterns Action

This section contains Lua code for the analysis in the CASL version of this example, which contains details about the results.

Note: In order to run this code, the data that are described in the CASL version need to be accessible to the CAS server. One way to do this is to convert the dmagecr data to the comma-separated-value (CSV) file dmagecr.csv and then use the following code to load the CSV file into CAS:

s:loadtable{casLib="casuser", path="dmagecr.csv"}

For more information about coding in Lua, see Getting Started with SAS Viya for Lua and SAS Viya: System Programming Guide.

s:loadactionset{actionset="dataSciencePilot"}

s:dataSciencePilot_analyzeMissingPatterns {
                              table     = {name ="DMAGECR"},
                              casOut    = {name    = "MISS_PATTERN_OUT",
                                           replace = True},
                              inputs    = {{name   = "age"},
                                           {name   = "amount"},
                                           {name   = "coapp"},
                                           {name   = "duration"},
                                           {name   = "foreign"},
                                           {name   = "job"}},
                              nominals  = {"coapp",  "foreign", "job"}
}

s:fetch{table="MISS_PATTERN_OUT"}

For example output from the action, see the CASL code examples.

Mutual-Information-Based Analysis of Missing Patterns Using the analyzeMissingPatterns Action

This section contains R code for the analysis in the CASL version of this example, which contains details about the results.

m <- cas.read.csv(s, "dmagecr.csv", casOut=list(name="dmagecr"))

For more information about coding in R, see Getting Started with SAS Viya for R and SAS Viya: System Programming Guide.

cas.loadActionSet(s,"dataSciencePilot")

cas.dataSciencePilot.analyzeMissingPatterns(
   s,
   table     = list(name ="DMAGECR"),
   casOut    = list(name = "MISS_PATTERN_OUT",
                    replace = True),
   inputs    = list("age",
                    "amount",
                    "coapp",
                    "duration",
                    "foreign",
                    "job"),
   nominals  = list("coapp",
                    "foreign",
                    "job")
)

cas.fetch(s, table = list(name = "MISS_PATTERN_OUT"))

For example output from the action, see the CASL code examples.

Mutual-Information-Based Analysis of Missing Patterns Using the analyzeMissingPatterns Action

This section contains Python code for the analysis in the CASL version of this example, which contains details about the results.

s.upload_file('dmagecr.csv')

For more information about coding in Python, see Getting Started with SAS Viya for Python and SAS Viya: System Programming Guide.

s.loadactionset(actionset="dataSciencePilot")
s.dataSciencePilot.analyzeMissingPatterns(
     table     = {"name" : "DMAGECR"},
     inputs    = ["age", "amount", "coapp", "duration", "foreign", "job"],
     nominals  = ["coapp", "foreign", "job"],
     casOut    = {"name" : "MISS_PATTERN_OUT", "replace" : True}
 )
s.fetch(table = {"name": "MISS_PATTERN_OUT"})

For example output from the action, see the CASL code examples.

Last updated: November 04, 2020