Bagtyyar

Gonka — AI Inference Infrastructure

Active build

Specified, deployed, and supported a 24-node GPU cluster (192× RTX 4080) serving open-weight LLMs on the Gonka decentralized AI inference network — plus an OpenAI-compatible API gateway built on top of it.

  • GPU Infrastructure
  • RTX 4080 Clusters
  • Open-Weight LLMs
  • Qwen
  • vLLM
  • Decentralized Inference
  • Express
  • TypeScript
  • Zod
  • SSE
  • Docker
  • Vitest

Outcome

24-node / 192-GPU inference cluster specified, deployed, and supported for 3 months on the Gonka decentralized AI network, serving Qwen 235B-class models — plus GonkaProvider, a public OpenAI-compatible API gateway with streaming validation and an upstream vLLM fix.

Context

Gonka is a decentralized network for AI inference: hosts run GPU servers serving open-weight models and earn network tokens for verified compute. During the network's early phase, a host engaged me to take their operation from zero to production: hardware selection, procurement guidance (servers sourced from China), deployment, and ongoing operations — delivered with a 3-month support guarantee. Client identity and commercial figures are confidential.

Scope Delivered

  • Hardware specification: advised on server selection for a 24-node cluster, 8× RTX 4080 per node — 192 GPUs in total — balancing inference throughput, memory bandwidth for large models, power, and cost.
  • Model serving: deployed open-weight LLM inference at scale, including Qwen 235B-class instruction models, during the network's early development phase.
  • Operations: node setup, network onboarding, monitoring, and a 3-month support engagement covering incident response and tuning.

GonkaProvider — API gateway (own work, public)

On top of the infrastructure work, I built and open-sourced GonkaProvider: an OpenAI-compatible Express/TypeScript gateway that proxies chat completions to Gonka ML nodes via the signed gonka-openai client.

  • Strict TypeScript + Zod validation across the boundary.
  • SSE stream validation, reasoning and tool-call aggregation, multimodal normalization.
  • ADRs, unit tests, and integration tests.
  • Includes an upstream fix for a vLLM chunk-validation bug found while integrating.

Outcomes

  • 192-GPU inference operation taken from hardware shopping list to revenue-earning production on a live decentralized network.
  • 3-month guaranteed support delivered to completion.
  • Public gateway codebase demonstrating AI-integration engineering: streaming, validation, and provider abstraction.

Public repository — GonkaProvider