Voxomos Systems
Software Intern
- Developed and benchmarked CPU, CUDA and HIP implementations of cosine similarity and vector search, profiled with NVIDIA Nsight Systems and Nsight Compute, and tested portability to AMD through ROCm, HIP and HIPIFY.
- Evaluated inference across vLLM, SGLang and TensorRT-LLM, isolated KV-cache memory limits under load, and cut time-to-first-token spikes by tuning request concurrency.
- Decode latency
- 14–15 ms/token
- Batch vector search
- 22× faster than CPU
- Runtimes
- vLLM, SGLang, TensorRT-LLM
- Platforms
- CUDA, ROCm/HIP