أحدث منشورات Paolo Perrone على لينكدإن

Stack Overflow copy-paste was the original vibe coding. We just upgraded the source.

I've replaced $4,000/month in LLM infrastructure costs with 8 open-source repos. $48,000/year. Gone. Here's the swap list: 1️⃣ Paid serving API → vLLM (74K ⭐) Self-hosted inference. PagedAttention, continuous batching. $0.03/token → $0.002/token overnight. https://lnkd.in/eeT_HM2B 2️⃣ Cloud fine-tuning platform…

Someone just made NVIDIA and Apple talk to each other for inference. 12 machines. 2.4TB unified memory. One problem. 7 DGX Sparks: 896 GB, brutal at compute. 5 Mac Studios: 1.5 TB, brutal at bandwidth. Mac Studios hold trillion-parameter models but prefill at 400 tok/s. A 900K context load takes 30+ minutes to…

Someone just made NVIDIA and Apple talk to each other for inference. 12 machines. 2.4TB unified memory. One problem. 7 DGX Sparks: 896 GB, brutal at compute. 5 Mac Studios: 1.5 TB, brutal at bandwidth. Mac Studios hold trillion-parameter models but prefill at 400 tok/s. A 900K context load takes 30+ minutes to…

A YC-backed CTO told me his team deleted every engineering metric except two. Revenue per line of code. Revenue per token. His 4-person team outships companies with 40 engineers. He says these metrics are why. Here's what they actually track: 1️⃣ Revenue per line of code = lifetime cost of ownership Every line y…

I skipped hiring a research analyst for Q4. I verify inference latency claims before I commit to hardware. I tried rotating this across the team. It wasn't sustainable: → Pulling benchmarks from new papers → Cross-referencing vendor specs → PRs delayed, expensive engineering hours burned Instead of adding headcou…