A personal timeline of how programming has evolved over the years. Plus a fun 2030 prediction.
LLM APIs can rack up costs fast—especially in background jobs, serverless functions, or cron-based pipelines running as part of CI/CD or due to programmatic errors doing retries. Here’s how to keep usage in check without overengineering a solution.
Start with basic alert guardrails:
These won’t stop usage, but they give early warning.
Create different API keys with distinct limits:
This prevents accidental overuse during testing and helps in bifurcation.
Use cloud billing APIs to track LLM spend:
Against service name once get granular cost usage and return boolean stating if LLM api calls can be made or not.
Check usage every Nth requests or every Nth min elapsed based of your worker being stateless or stateful. Stop LLM usage if api returns false.
LLM calls often run outside of web requests. Add usage checks in:
With these effort, you can avoid budget surprises while keeping your LLM-powered systems running smoothly.
]]>With the advent of viral real estate listings on social media comes a barrage of call center inquiries about the projects and their details. The UAE, in particular, receives calls from across the globe.
The client was already using an AI SaaS product that would take call recordings and extract:
The system also tagged phone numbers and other metadata, automatically raising CRM tickets. These tickets were routed to agents over WhatsApp within seconds, along with a dossier containing extracted details, a transcript, and a playable audio file link. This enabled agents to exercise judgment and prioritize callbacks effectively.
The pricing with the SaaS provider was a fixed cost of 7 AED per call. My task was to replicate this functionality at a lower cost, while ensuring the client’s in-house tech team could take over and fully own the resulting IP.
Using ffmpeg, we processed each audio file to:
Running ffprobe on the optimized files showed that we reduced audio file sizes by approximately 60% on average. This resulted in downstream savings in both storage and processing costs.
After experimenting with optimized versions of OpenAI Whisper and various Hugging Face models, I ultimately chose the reliable Google Speech-to-Text API. One key reason was the need to handle multi-lingual conversations — many East Asian callers switched between their native language and English, e.g. Hindi/Urdu -> English.
Google’s API allowed us to specify a primary language along with up to four alternative languages.
Another useful trick was hosting audio files in Google Cloud Storage. This appeared to speed up transcription time — my hunch is that Google avoids copying files already hosted within their cloud infrastructure (as opposed to fetching from S3 or a local server).
I handed over a Dockerized version of the pipeline, micro-batching audio files in 3-minute windows, and ran a worker pool of 20 concurrent workers. ffmpeg was the most CPU-intensive part of the process. The client now runs the entire pipeline within Kubernetes, where the number of messages in Redis streams dynamically determines the scaling of the worker pool.
Their cost is now equal to the reserved instance + google transcription cost + Gemini 2.5 Pro. When spread across the no of calls they are now saving approx. 92% in cost in June alone.
Tech stack:
A few months ago, at our 25th school reunion, I was reminded how everyone remembered my early obsession with computers. Some even recalled the DBase III Plus program I built for a school project. That passion never really left me—and perhaps that’s why most of my career has never felt like “work.”
Even before formal education, I was curious about how computers worked. That curiosity finally found structure through courses in Computer Architecture and Operating Systems during my master’s.
It’s been 18 years since I started my journey in IT—from Software Engineer to Manager of Managers. I’ve been fortunate to work with some brilliant people and gain experience across both large enterprises and startups.
Life has kept me busy, but I’m currently on a career break to support my wife as she recovers from health issues. It feels like the right time to restart this blog—both to explore technical interests and reconnect with my roots in coding and writing.
]]>