Devotd Github
NewInference & serving

midagedev/bloomery depth-serve — waits for the previous server process to exit

The depth-serve script in midagedev/bloomery was modified to wait until the previous server's process disappears from the process table before starting the next server.

Repositorymidagedev/bloomery ↗

The depth-serve script in midagedev/bloomery was modified to wait until the previous server's process disappears from the process table before starting the next server.

GitHub description: Hybrid GPU + CPU inference for mixture-of-experts models, written in Rust down to the CUDA kernels. A drop-in llama-server.

stars
★ 26
Forks
5
Language
Rust
License
MIT
Latest release
v0.3.3
Read original source ↗View on GitHub ↗

midagedev/bloomery is a library that handles GPU and CPU inference for mixture (MoE) models together in a single project. It is written in Rust and implements even the CUDA kernels itself, and it advertises itself as a drop-in server that replaces llama-server. The commit 1a7bba2 of November 5, 2026 touched only a single file, tools/ref/depth-serve.sh. The commit link is https://github.com/midagedev/bloomery/commit/1a7bba264cdbc1df2bc7819855f5b15f619ddcde. It adds 18 lines and deletes 6, and its content is the fixing of one wait condition. The wait before starting the next server now includes, in addition to 'the port is empty' and 'there is no process holding the timing card', also 'the previous server's process is no longer present in the process table'. Previously the wait was ended by only the first two conditions.

The reason this condition is needed lies in the failure mode the commit message carries. A server terminated by the wrapper's KILL remains in a terminating state for several seconds while it cleans up roughly 262 GB of the host set. During that time the port is closed and the card context also disappears, so the CPU probe in front of the next server read that process as a co-tenant, and the result row became invalid as [cpu-busy] bloomery-serve 99.7 % over 6.5 s. The same failure repeated twice in a row across release sittings. That is why ds_serve_procs() was added to the script. It uses pgrep -x to collect the processes whose comm name is bloomery-serve or llama-server into the wait queue as pid and name. ds_prev_gone checks the port, the timing card, and the process table together, and keeps the identity of what it waits on by name; if it exceeds PORT_WAIT_S seconds it treats the case as a failure with the name 'its process has not exited'. The self-check recorded by the commit is 199 checks ok.

That said, this change is limited to a single place — the repository's depth-serve script — and not to the inference engine or the CUDA kernels. No performance figures or throughput changes are presented. The situation dealt with is also only one wait condition: the case where a server terminated by KILL is still cleaning up. The commit trailer still carries a co-author credit for Claude Fable 5.1 and a session link. Judging from this, it is a fix that is needed for people who want to reproduce the repository's release measurements as they are, and for people who use the MoE inference path itself, this commit touches nothing.