The TEXTMINE Procedure
Example 24.4 Adding Part-of-Speech Tagging
This example uses the data table that is generated in Example 24.1. The following statements run PROC TEXTMINE to parse the documents. Because the NOTAGGING option is not specified in the PARSE statement, PROC TEXTMINE uses context clues to determine a term’s part of speech.
/* create data table */
data mycas.CarNominations;
infile datalines delimiter='|' missover;
length text $70 ;
input text$ i;
datalines;
The Ford Taurus is the World Car of the Year. |1
Hyundai won the award last year. |2
Toyota sold the Toyota Tacoma in bright green. |3
The Ford Taurus is sold in all colors except for lime green. |4
The Honda Insight was World Car of the Year in 2008. |5
;
run;
proc textmine data=mycas.CarNominations;
doc_id i;
var text;
parse
termwgt = none
cellwgt = none
reducef = 1
entities = std
outparent = mycas.outparent
outterms = mycas.outterms
outchild = mycas.outchild
outconfig = mycas.outconfig
;
run;
data outterms; set mycas.outterms; run;
proc print data= outterms; run;
Output 24.4.1 shows the content of the mycas.outterms data table. Compared to Output 24.3.1, the mycas.outterms data table also contains the part-of-speech tag for the terms.
Output 24.4.1: The mycas.outterms Data Table with Part-of-Speech Tagging
| Obs | Term | Role | Attribute | Freq | numdocs | _keep | Key | Parent | Parent_id | _ispar | Weight |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | is | V | Alpha | 2 | 2 | Y | 30 | 26 | 26 | . | 1 |
| 2 | was | V | Alpha | 1 | 1 | Y | 31 | 26 | 26 | . | 1 |
| 3 | bright | A | Alpha | 1 | 1 | Y | 1 | . | 1 | 1 | |
| 4 | taurus | N | Alpha | 2 | 2 | Y | 2 | . | 2 | 1 | |
| 5 | won | V | Alpha | 1 | 1 | Y | 32 | 17 | 17 | . | 1 |
| 6 | for | PPOS | Alpha | 1 | 1 | Y | 3 | . | 3 | 1 | |
| 7 | lime green | nlpNounGroup | Alpha | 1 | 1 | Y | 4 | . | 4 | 1 | |
| 8 | lime | A | Alpha | 1 | 1 | Y | 5 | . | 5 | 1 | |
| 9 | in 2008 | nlpDate | Entity | 1 | 1 | Y | 6 | . | 6 | 1 | |
| 10 | the | DET | Alpha | 8 | 5 | Y | 7 | . | 7 | 1 | |
| 11 | except | V | Alpha | 1 | 1 | Y | 8 | . | 8 | 1 | |
| 12 | bright green | nlpNounGroup | Alpha | 1 | 1 | Y | 9 | . | 9 | 1 | |
| 13 | color | N | Alpha | 1 | 1 | Y | 10 | . | 10 | + | 1 |
| 14 | in | PPOS | Alpha | 3 | 3 | Y | 11 | . | 11 | 1 | |
| 15 | hyundai | nlpOrganization | Entity | 1 | 1 | Y | 12 | . | 12 | 1 | |
| 16 | sold | V | Alpha | 2 | 2 | Y | 33 | 28 | 28 | . | 1 |
| 17 | toyota | nlpOrganization | Entity | 2 | 1 | Y | 13 | . | 13 | 1 | |
| 18 | last year | nlpDate | Entity | 1 | 1 | Y | 14 | . | 14 | 1 | |
| 19 | ford | nlpOrganization | Entity | 2 | 2 | Y | 15 | . | 15 | 1 | |
| 20 | all | A | Alpha | 1 | 1 | Y | 16 | . | 16 | 1 | |
| 21 | win | V | Alpha | 1 | 1 | Y | 17 | . | 17 | + | 1 |
| 22 | car | PN | Alpha | 2 | 2 | Y | 18 | . | 18 | 1 | |
| 23 | colors | N | Alpha | 1 | 1 | Y | 34 | 10 | 10 | . | 1 |
| 24 | award | N | Alpha | 1 | 1 | Y | 19 | . | 19 | 1 | |
| 25 | insight | PN | Alpha | 1 | 1 | Y | 20 | . | 20 | 1 | |
| 26 | of | PPOS | Alpha | 2 | 2 | Y | 21 | . | 21 | 1 | |
| 27 | honda | PN | Alpha | 1 | 1 | Y | 22 | . | 22 | 1 | |
| 28 | world | PN | Alpha | 2 | 2 | Y | 23 | . | 23 | 1 | |
| 29 | last | A | Alpha | 1 | 1 | Y | 24 | . | 24 | 1 | |
| 30 | green | N | Alpha | 2 | 2 | Y | 25 | . | 25 | 1 | |
| 31 | be | V | Alpha | 3 | 3 | Y | 26 | . | 26 | + | 1 |
| 32 | tacoma | PN | Alpha | 1 | 1 | Y | 27 | . | 27 | 1 | |
| 33 | sell | V | Alpha | 2 | 2 | Y | 28 | . | 28 | + | 1 |
| 34 | year | N | Alpha | 3 | 3 | Y | 29 | . | 29 | 1 |
Copyright © SAS Institute Inc. All rights reserved.
Last updated: November 11, 2020