SMC Procedure

SMC Evaluating

The marginal likelihood is defined by

p Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript 1 colon t Baseline right-parenthesis equals integral p Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript 1 colon t Baseline comma bold upper X Subscript 1 colon t Baseline right-parenthesis d bold upper X Subscript 1 colon t Baseline

Monte Carlo methods can directly estimate this integral, but the approximation is usually poor in practice. Alternatively, the marginal likelihood (t greater-than 1) is decomposed as

p Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript 1 colon t Baseline right-parenthesis equals p Subscript bold-italic theta Baseline left-parenthesis bold upper Y 1 right-parenthesis product Underscript j equals 2 Overscript t Endscripts p Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis

For j equals 2 comma ellipsis comma t, it is straightforward to have

StartLayout 1st Row 1st Column Blank 2nd Column p Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis 2nd Row 1st Column equals 2nd Column integral p Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Baseline right-parenthesis p Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j minus 1 Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis d bold upper X Subscript j minus 1 3rd Row 1st Column equals 2nd Column integral integral g Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j Baseline right-parenthesis f Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Baseline right-parenthesis p Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j minus 1 Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis d bold upper X Subscript j minus 1 Baseline d bold upper X Subscript j EndLayout

These terms can be estimated by the SMC algorithms with the weighted particles at time step j minus 1 as

ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis equals sigma-summation Underscript k equals 1 Overscript upper N Endscripts w overTilde Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline integral g Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j Baseline right-parenthesis f Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Baseline right-parenthesis d bold upper X Subscript j Baseline comma j greater-than 1

SIR Algorithm

In the SIR algorithm, this approximation can be further simplified by sampling bold upper X Subscript j Superscript left-parenthesis k right-parenthesis (j greater-than 1) from q Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline comma bold upper Y Subscript j Baseline right-parenthesis for k equals 1 comma ellipsis comma upper N:

StartLayout 1st Row 1st Column ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis 2nd Column equals sigma-summation Underscript k equals 1 Overscript upper N Endscripts upper V Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline StartFraction g Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis f Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Superscript left-parenthesis k right-parenthesis Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline right-parenthesis Over q Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline comma bold upper Y Subscript j Baseline right-parenthesis EndFraction equals sigma-summation Underscript k equals 1 Overscript upper N Endscripts w Subscript j Baseline left-parenthesis bold upper X Subscript 1 colon j Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis EndLayout

Moreover, p Subscript bold-italic theta Baseline left-parenthesis bold upper Y 1 right-parenthesis can be written as the following integral with the importance sampling scheme:

p Subscript bold-italic theta Baseline left-parenthesis bold upper Y 1 right-parenthesis equals integral StartFraction mu Subscript bold-italic theta Baseline left-parenthesis bold upper X 1 right-parenthesis g Subscript bold-italic theta Baseline left-parenthesis bold upper Y 1 vertical-bar bold upper X 1 right-parenthesis Over q Subscript bold-italic theta Superscript 1 Baseline left-parenthesis bold upper X 1 vertical-bar bold upper Y 1 right-parenthesis EndFraction q Subscript bold-italic theta Superscript 1 Baseline left-parenthesis bold upper X 1 vertical-bar bold upper Y 1 right-parenthesis d bold upper X 1

In the SIR algorithm, left-brace bold upper X 1 Superscript left-parenthesis k right-parenthesis Baseline right-brace Subscript k equals 1 Superscript upper N are sampled from q Subscript bold-italic theta Superscript 1 Baseline left-parenthesis bold upper X 1 vertical-bar bold upper Y 1 right-parenthesis. Therefore, the basic Monte Carlo approximation of p Subscript bold-italic theta Baseline left-parenthesis bold upper Y 1 right-parenthesis is

ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y 1 right-parenthesis equals StartFraction 1 Over upper N EndFraction sigma-summation Underscript k equals 1 Overscript upper N Endscripts w 1 left-parenthesis bold upper X 1 Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis

BF Algorithm

In the BF algorithm, this approximation can be simplified by sampling bold upper X Subscript j Superscript left-parenthesis k right-parenthesis (j greater-than 1) from f Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Superscript left-parenthesis k right-parenthesis Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline right-parenthesis for k equals 1 comma ellipsis comma upper N:

StartLayout 1st Row 1st Column ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis 2nd Column equals sigma-summation Underscript k equals 1 Overscript upper N Endscripts StartFraction 1 Over upper N EndFraction StartFraction g Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis f Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Superscript left-parenthesis k right-parenthesis Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline right-parenthesis Over f Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Superscript left-parenthesis k right-parenthesis Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline right-parenthesis EndFraction equals StartFraction 1 Over upper N EndFraction sigma-summation Underscript k equals 1 Overscript upper N Endscripts w Subscript j Baseline left-parenthesis bold upper X Subscript 1 colon j Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis EndLayout

Moreover, the ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y 1 right-parenthesis in the BF algorithm is computed in the same way as the one in the SIR algorithm.

APF Algorithm

In the APF algorithm, because v Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline equals w overTilde Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis, the approximation can be written as

ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis equals sigma-summation Underscript k equals 1 Overscript upper N Endscripts StartFraction v Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline sigma-summation Underscript i equals 1 Overscript upper N Endscripts v Subscript j minus 1 Superscript left-parenthesis i right-parenthesis Baseline Over left-parenthesis sigma-summation Underscript i equals 1 Overscript upper N Endscripts v Subscript j minus 1 Superscript left-parenthesis i right-parenthesis Baseline right-parenthesis ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis EndFraction integral g Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j Baseline right-parenthesis f Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis d bold upper X Subscript j Baseline comma j greater-than 1

The auxiliary index variables left-brace upper A Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline right-brace Subscript k equals 1 Superscript upper N are sampled according to left-brace StartFraction v Subscript j minus 1 Superscript left-parenthesis k right-parenthesis Baseline Over sigma-summation Underscript i equals 1 Overscript upper N Endscripts v Subscript j minus 1 Superscript left-parenthesis i right-parenthesis Baseline EndFraction right-brace Subscript k equals 1 Superscript upper N, and bold upper X Subscript j Superscript left-parenthesis k right-parenthesis are sampled from q Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline comma bold upper Y Subscript j Baseline right-parenthesis. The weight for each auxiliary index is StartFraction 1 Over upper N EndFraction. The preceding approximation is thus simplified as

StartLayout 1st Row 1st Column ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis 2nd Column equals sigma-summation Underscript k equals 1 Overscript upper N Endscripts StartFraction 1 Over upper N EndFraction left-parenthesis sigma-summation Underscript i equals 1 Overscript upper N Endscripts v Subscript j minus 1 Superscript left-parenthesis i right-parenthesis Baseline right-parenthesis StartFraction g Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis f Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Superscript left-parenthesis k right-parenthesis Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline right-parenthesis Over ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline right-parenthesis q Subscript bold-italic theta Baseline left-parenthesis bold upper X Subscript j Superscript left-parenthesis k right-parenthesis Baseline vertical-bar bold upper X Subscript j minus 1 Superscript left-parenthesis upper A Super Subscript j minus 1 Super Superscript left-parenthesis k right-parenthesis Superscript right-parenthesis Baseline comma bold upper Y Subscript j Baseline right-parenthesis EndFraction 2nd Row 1st Column Blank 2nd Column equals StartFraction 1 Over upper N EndFraction left-parenthesis sigma-summation Underscript i equals 1 Overscript upper N Endscripts v Subscript j minus 1 Superscript left-parenthesis i right-parenthesis Baseline right-parenthesis left-parenthesis sigma-summation Underscript k equals 1 Overscript upper N Endscripts w Subscript j Baseline left-parenthesis bold upper X Subscript 1 colon j Superscript left-parenthesis k right-parenthesis Baseline right-parenthesis right-parenthesis period EndLayout

It is further simplified to ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y Subscript j Baseline vertical-bar bold upper Y Subscript 1 colon j minus 1 Baseline right-parenthesis equals sigma-summation Underscript i equals 1 Overscript upper N Endscripts v Subscript j minus 1 Superscript left-parenthesis i right-parenthesis if you use APF(ADP). Moreover, the ModifyingAbove p With caret Subscript bold-italic theta Baseline left-parenthesis bold upper Y 1 right-parenthesis in the APF algorithm is computed in the same way as the one in the SIR algorithm.

Last updated: July 09, 2026