The TEXTMINE Procedure

Example 24.4 Adding Part-of-Speech Tagging

This example uses the data table that is generated in Example 24.1. The following statements run PROC TEXTMINE to parse the documents. Because the NOTAGGING option is not specified in the PARSE statement, PROC TEXTMINE uses context clues to determine a term’s part of speech.

/* create data table */
data mycas.CarNominations;
infile datalines delimiter='|' missover;
length text $70 ;
input text$ i;
datalines;
   The Ford Taurus is the World Car of the Year. |1
   Hyundai won the award last year. |2
   Toyota sold the Toyota Tacoma in bright green. |3
   The Ford Taurus is sold in all colors except for lime green. |4
   The Honda Insight was World Car of the Year in 2008. |5
   ;
run;

proc textmine data=mycas.CarNominations;
doc_id i;
var text;
parse
   termwgt   = none
   cellwgt   = none
   reducef   = 1
   entities  = std
   outparent = mycas.outparent
   outterms  = mycas.outterms
   outchild  = mycas.outchild
   outconfig = mycas.outconfig
   ;
run;
data outterms; set mycas.outterms; run;
proc print data= outterms; run;

Output 24.4.1 shows the content of the mycas.outterms data table. Compared to Output 24.3.1, the mycas.outterms data table also contains the part-of-speech tag for the terms.

Output 24.4.1: The mycas.outterms Data Table with Part-of-Speech Tagging

ObsTermRoleAttributeFreqnumdocs_keepKeyParentParent_id_isparWeight
1isVAlpha22Y302626.1
2wasVAlpha11Y312626.1
3brightAAlpha11Y1.1 1
4taurusNAlpha22Y2.2 1
5wonVAlpha11Y321717.1
6forPPOSAlpha11Y3.3 1
7lime greennlpNounGroupAlpha11Y4.4 1
8limeAAlpha11Y5.5 1
9in 2008nlpDateEntity11Y6.6 1
10theDETAlpha85Y7.7 1
11exceptVAlpha11Y8.8 1
12bright greennlpNounGroupAlpha11Y9.9 1
13colorNAlpha11Y10.10+1
14inPPOSAlpha33Y11.11 1
15hyundainlpOrganizationEntity11Y12.12 1
16soldVAlpha22Y332828.1
17toyotanlpOrganizationEntity21Y13.13 1
18last yearnlpDateEntity11Y14.14 1
19fordnlpOrganizationEntity22Y15.15 1
20allAAlpha11Y16.16 1
21winVAlpha11Y17.17+1
22carPNAlpha22Y18.18 1
23colorsNAlpha11Y341010.1
24awardNAlpha11Y19.19 1
25insightPNAlpha11Y20.20 1
26ofPPOSAlpha22Y21.21 1
27hondaPNAlpha11Y22.22 1
28worldPNAlpha22Y23.23 1
29lastAAlpha11Y24.24 1
30greenNAlpha22Y25.25 1
31beVAlpha33Y26.26+1
32tacomaPNAlpha11Y27.27 1
33sellVAlpha22Y28.28+1
34yearNAlpha33Y29.29 1


Last updated: November 11, 2020