The I in LLM is for intelligence

 

I've finally got DeepSeek running on my Nvidia 4070 and working through VS Code.

Yea, yea, the cloud platforms are way faster and much better that trying to do this with a locally hosted model but. I have a i9-11900KB with an RTX 4070 as a daily driver but can't afford Yet Another Subscription.

Originally, I planned to host Ollama and spent hours in vain trying to get that to live inside VS Code. Bottom line is that VS Code would never execute the tooling regardless of which plugin installed. I'd been running LM Studio on my laptop (Ultra 9 185H w/ 64G) and it worked.... v... e... r... r... y... s... l... o... w... l... y... So I tried that with the RTX 4070. This combination is workable. with a few gotchas.

Of course, CoPilot will stupidly refuse to work with anything 'not cloud' hosted no matter what tweaks you try out. I get the feeling this is a design choice and not an oversight. There is always one critical piece missing or in the way (in the name of security). And no, self-signed certificate authorities are not acceptable, nor is a local host proxy (not sure how it figured that out).

Conntinue.dev does work but not with config.yaml for some reason. You must use config.json; the old, deprecated configuration mechanism. I should probably look to see if there is a bug report on that yet.

For now, I'm running LM Studio on Ubuntu 24.4 (bleck) as a desktop app. llmster is not behaving well and I don't have time this week to look at it. Because of my limited VRAM on the 4070 I'm running:
- deepseek-r1-distill-qwen-14b for planning, chat, and analysis of existing code.
- qwen2.5-coder:9b for a coding assistant along with
- qwen2.5-coder:1.5b for code completion

I can't help but wonder if the creators of deepseek are fans of the Hitch Hiker's Guide series who ran a fowl of Vogon copyright law.