White Paper: Model Invocation Architecture and Routing Strategy for Synthari

Differentiating Wrapper-Based and Server-Based Model Execution

Abstract

This paper examines the technical architecture for routing LLM requests from external clients, through the Synthari access gateway, into internal inference engines. The system involves hybrid execution models — where some LLMs are invoked via wrapped binaries and others through a persistent inference server such as LM Studio. We document the differences in HTTP semantics, lifecycle management, resource control, and error handling across these invocation paths.

1. Introduction

Synthari supports heterogeneous model execution pipelines. Some models (tiny, phi-3, llama-8b, mistral, gemma) are invoked directly via lightweight MCP-managed wrappers, while others (e.g., Granite-8B, Liquid-1.2B, LFM models) are accessed through the LM Studio inference API running on port 1234.

This architecture enables the server to act as a multi-brain AI switching node that can dynamically assign tasks to models based on context and task category.

2. Invocation Methods

2.1 Wrapper-Based Local Invocation (llama.cpp family)

Models such as tiny, phi, llama-8b, and mistral are executed using binary model wrappers. These wrappers:

• load the model on request
• process a single inference job
• return output directly to Synthari
• terminate and release GPU VRAM

POST /infer
model: "phi3"
text: "Hello"

This call results in a child process (or Python wrapper), which loads the model, runs the prompt, and dies. After completion, VRAM and RAM are cleared.

2.2 Server-Based Invocation (LM Studio models)

LM Studio models require that the LM Studio application itself is running persistently. These models are not spun up and torn down per request — instead, they remain resident in memory.

Their invocation endpoint is:

POST http://127.0.0.1:1234/v1/completions
{
  "model": "ibm/granite-3.2-8b",
  "prompt": "Hello"
}

This approach:

• does not unload models after use
• requires persistent listening service
• keeps VRAM allocated
• enables faster subsequent calls

The trade-off is that at least one LM Studio model remains memory-resident at all times.

3. Network Routing Path

Requests from Internet users follow this route:

Client Browser (synthari.html)
   ?
Cloudflare Tunnel
   ?
Local Server (SkyTech)
   ?
Synthari Router Module
   ?
Either:
  A) model-wrapper inference   OR
  B) LM Studio inference server

Synthari makes the decision internally — not the user.

4. Endpoint Format Differences

4.1 Wrapper-style models accept:

{
  "model": "tiny",
  "text": "hello"
}

4.2 LM Studio models accept:

{
  "model": "lfm2-v1-1.6b",
  "prompt": "hello"
}

This difference is critical. Wrapper models operate on the concept of chat messages or text, while LM Studio models operate on prompts.

5. Model Lifecycle Differences

6. Practical Implications

• Wrapper models allow full autonomy and shallow resource footprint
• LM Studio models allow faster response and better throughput for large models
• Wrapper models require explicit loading/unloading
• LM Studio models require the inference server to remain running

7. Conclusion

Synthari’s hybrid execution pipeline enables optimal routing of inference jobs depending on model characteristics. This two-path design — wrapper-based and server-based — provides both flexibility and efficiency. Wrapper models minimize idle resource usage, while LM Studio models maximize production throughput and execution speed.

This architecture permits a heterogeneous multi-model AI environment where the server intelligently selects the correct execution engine behind the scenes.

Back to All White Papers