Devotd Github
NewInference & serving

miaai-lab/glm-5.3-flash-exl3-2x-dgx-sparks-tensorfold — A Serving Recipe for Two DGX Sparks

A repository that lays out a setup for serving GLM-5.3-Flash EXL3 on two DGX Sparks with TensorFold.

Repositorymiaai-lab/glm-5.3-flash-exl3-2x-dgx-sparks-tensorfold ↗

A repository that lays out a setup for serving GLM-5.3-Flash EXL3 on two DGX Sparks with TensorFold.

GitHub description: GLM-5.3-Flash EXL3 on 2x DGX Spark with TensorFold

stars
★ 499
Forks
95
Language
Python
License
Apache-2.0
Read original source ↗View on GitHub ↗

miaai-lab/glm-5.3-flash-exl3-2x-dgx-sparks-tensorfold is a deployment recipe for serving GLM-5.3-Flash, quantized with EXL3, on two DGX Sparks via TensorFold. The repository provides model deployment and start/stop scripts for this setup, plus materials for prompt reuse and tool-call inspection. There are no public releases, and the repository description presents a GLM-5.3-Flash serving configuration.

The value of this material lies less in the model itself than in examining an inference server configuration that spans two devices. The README describes an OpenAI-compatible API, image and video input, tool calling, long-context handling, and concurrent processing of multiple requests. The configuration includes EXL3 weights and an FP8 KV cache. The KV cache is the area that holds the computed results of previous tokens. The draft generation path and the communication between the two devices are also introduced as components. It is a Python repository under the Apache-2.0 license.

The README's performance table presents throughput and time to first token by number of concurrent requests, but these are results recorded on the specific hardware and configuration described in the document. Since those measurements have not been independently reproduced, they are hard to accept as a general performance comparison. This repository is therefore closer to a reference for a serving recipe using two DGX Sparks, and there is no basis for expecting the same results in other environments.