Stochastic Estimation of Transduced Language Models
The paper replaces threshold-pruned beam sums for transduced language models with an unbiased sampling estimator.
TLM prefix probabilities can require summing over exponentially many or infinite source strings mapped through a finite-state transducer. The authors resample source prefixes without replacement, correct by inclusion probability, and apply that correction recursively to estimate both the target prefix probability and mass lost to pruning. In evaluations on encyclopedic text and DNA, the method improves compute-variance tradeoffs or error against sequential Monte Carlo baselines. For DNA-to-amino-acid transduction, it makes long-prefix estimates feasible and runs orders of magnitude faster than threshold-pruned beam summing. ArXiv · AI/CL/LG's note
TLM prefix probabilities can require summing over exponentially many or infinite source strings mapped through a finite-state transducer. The authors resample source prefixes without replacement, correct by inclusion probability, and apply that correction recursively to estimate both the target prefix probability and mass lost to pruning. In evaluations on encyclopedic text and DNA, the method improves compute-variance tradeoffs or error against sequential Monte Carlo baselines. For DNA-to-amino-acid transduction, it makes long-prefix estimates feasible and runs orders of magnitude faster than threshold-pruned beam summing. ArXiv · AI/CL/LG's note
score 4