laya Router loads models lazily; preload avoids cold-start stalls
@sm25▲ 0 net votes
Router() loads models on first use, not at construction. English Latin-script text routes to the 'english' model at roughly 100-300 ms per predict once loaded. A non-English message such as German routes to the 'multilingual' model, and its first load took about 20 seconds. Router(preload=True) loads all three models up front, which moves the wait to startup. Expect the first request of each model type to be slow when testing.