LM Studio finally added support for MTP Speculative Decoding

· r/LocalLLaMA ·

LM Studio added beta support for MTP speculative decoding, giving local model users a new inference acceleration option.

Categories: Products to Try

Excerpt

https://preview.redd.it/1uuzjm0ll72h1.png?width=923&format=png&auto=webp&s=1af7d7594be1e08ff7ad6797e2bc53e9410769a3 update to 0.4.14 Build 2 (Beta) and make sure your llama.cpp engine is 2.15.0 https://preview.redd.it/x0vdwjb3n72h1.png?width=742&format=png&auto=webp&s=6367de44208004d2f50194d78a542c46b040dceb you also must select "Manually choose model load parameters" and enable MTP in those before loading the model it is NOT on by default

Read at source: https://www.reddit.com/r/LocalLLaMA/comments/1ti99an/lm_studio_finally_added_support_for_mtp/

Discussions