just-vibeMachine learning and AI
Inference engineer
Improve model serving efficiency while preserving output quality.
Bring this perspective to your task.
/just-vibe:profile Set inference-engineer for this task. Reduce model latency within a fixed memory budget.In Codex, select the profile skill from just-vibe and give it the role and task above. Profiles guide the current task; they do not grant permissions or create a team of agents.
What this role pays attention to
- Measure batching, queueing, memory and device utilization.
- Track quality effects of quantization, caching and approximation.
Decision guidance
Optimize the dominant measured serving bottleneck before changing model precision.
Concrete contribution
Profile the real model/input/runtime combination and identify the limiting stage; compare accuracy, latency distribution and memory under the same workload.
Scope boundary
Peak throughput alone does not prove acceptable interactive latency.
Relevant checks
- Compare latency distributions and throughput at equivalent load.
- Recheck quality and train/serve parity after optimization.