Telexed

AI news through a solo-operator lens — only what changes your day11 of 829

FILTER[All][Agents & tools][Models & API][Generative media][Infra & SaaS][ASO & growth][Indie business][Idea signals][Other][★6+ high-signal]

r/LocalLLaMA ✕clear filters

Wed, May 271 dispatches

#0829
#0829Generative media r/LocalLLaMA4 days ago
PrismML ships `Bonsai Image 4B` as 1-bit and ternary image models
60radar
Bonsai Image 4BImage generation model — 1-bit/ternary browser runtime
A text-to-image model small enough for browser-side WebGPU changes the demo and prototyping surface. Worth testing for local-first image features; quality still needs validation.
- Weights are about 3GB, versus roughly 16GB for FLUX.2 Klein 4B; distribution and local demos get much lighter.
- Binary and Ternary variants target extreme quantization, so the real test is prompt quality, latency, and failure modes.
- Apache-2.0 licensing lowers commercial friction. Browser-local generation can reduce API cost and privacy concerns for small tools.
- The Hugging Face WebGPU demo matters because no backend GPU is required for first-contact testing.
Source: www.reddit.com/r/LocalLLaMA/comments/1togflk/prismml_jusRead original →
FIG-8291:1
60radar
FIG-8291:1

Sat, May 232 dispatches

#0828
#0828Other r/LocalLLaMAlast week
`Qwen3.6 27B` pure `Q4_K_M` GGUF fits in **16GB VRAM**
40radar
Qwen3.6Open LLM — active GGUF ecosystem for local inference
Pure quantization trims enough size to keep the whole model on a consumer GPU. Useful for local agent tests, but quality loss is real and benchmark depth is thin.
- Q4_K_M MTP is 15.4GB and non-MTP is 15.1GB; comparable builds listed at 16.5-18GB often spill past 16GB cards.
- MTP reaches 40 tok/s generation but only 195 tok/s prompt processing; non-MTP flips the trade-off at 715 tok/s pp and 24 tok/s tg.
- Perplexity delta is larger than Unsloth's quant: +0.1707 vs +0.0553 on MTP, so the size win buys speed/fit at some quality cost.
Source: www.reddit.com/r/LocalLLaMA/comments/1tkzk9e/qwen36_27b_Read original →
40radar
PHOTO
FIG-8281:1
#0827
#0827Agents & tools r/LocalLLaMAlast week
BeeLlama v0.2.0 boosts inference speed by up to 4.9x on an RTX 3090
40radar
BeeLlamaLocal LLM engine — accelerates token generation via DFlash
An inference engine that achieves up to a 4.9x token speedup over llama.cpp via DFlash. It makes high-throughput local LLMs more viable on consumer GPUs like the RTX 3090.
- Achieves 164 tokens/sec with Qwen 3.6 27B on a single RTX 3090, a 4.4x speedup compared to llama.cpp's 37.2 tps.
- DFlash, a form of speculative decoding, accelerates inference using a smaller draft model. While prompt processing speed is similar, token generation is significantly faster.
- The update adds full support for Gemma 4 31B and is compatible with the GGUF format, easing integration with the existing local LLM ecosystem.
- This makes fast prototyping or running small-scale services on owned hardware more feasible, especially for tasks involving long text generation, without cloud API costs.
Source: www.reddit.com/r/LocalLLaMA/comments/1tkpz2y/beellama_v0Read original →
40radar
PHOTO
FIG-8271:1

Fri, May 221 dispatches

#0826
#0826Models & API r/LocalLLaMAlast week
DeepSeek raises $10.29B, commits to open-source models over short-term commercialization
50radar
DeepSeekOpen-source LLM developer — known for its `DeepSeek-Coder` model
DeepSeek has secured a massive $10.29 billion funding round, prioritizing open-source AI development. This signals their emergence as a major, stable player in the open-source camp, joining Meta and Mistral as a key alternative.
- The founder's commitment is to continue developing open-source models rather than pursuing immediate commercial goals, ensuring a stable supply of high-quality free models.
- The $10.29B financing round provides the capital to compete with major players like OpenAI and Anthropic in the race toward AGI.
- Alongside Meta's Llama and Mistral AI, DeepSeek is poised to become a pillar of the open-source ecosystem, offering more alternatives to closed-source APIs.
Source: www.reddit.com/r/LocalLLaMA/comments/1tkfvvj/deepseek_isRead original →
50radar
PHOTO
FIG-8261:1

Thu, May 212 dispatches

#0825
#0825Other r/LocalLLaMAlast week
`ik_llama.cpp` pushes `Qwen3.6 35B A3B` near 110 tok/s on 12GB VRAM
40radar
ik_llama.cppllama.cpp fork — optimized CPU offload and quantization
MTP plus CPU offload can make a local MoE model feel interactive on consumer hardware. Useful for private coding or batch jobs, but still a setup-specific benchmark.
- Same IQ4_XS quant averaged 89.76 tok/s on regular llama.cpp; ik_llama.cpp samples reached roughly 105-110 tok/s.
- Hardware was RTX 4070 Super 12GB, Ryzen 7 9700X, and 48GB DDR5. CPU offload quality matters as much as VRAM.
- Benchmark used --ctx-size 131072, q8 KV cache, and draft-mtp; long-context local workflows remain memory-sensitive.
- Treat it as a tuning lead, not a buying guide. Kernel, quant, and fork versions can swing results hard.
Source: www.reddit.com/r/LocalLLaMA/comments/1tjh7az/110_toks_wiRead original →
40radar
PHOTO
FIG-8251:1
#0824
#0824Models & API r/LocalLLaMAlast week
Cohere launches `Command A+`, an Apache 2.0 MoE open-weight model
80radar
Command A+Open-weight LLM — Apache 2.0 MoE model
A practical open-weight model enters the agent stack. Apache 2.0 plus strong quantization makes local or self-hosted experiments cheaper to justify.
- Command A+ is Cohere’s first MoE model; top-line performance still needs work, but speed and responsiveness are the claimed edge.
- The model is released under Apache 2.0, so commercial use and product integration have fewer license traps.
- Quantization is positioned as a core feature: it runs well on 1-2 GPUs, making self-hosted agent backends more realistic.
- Cohere frames it as the kind of model behind its enterprise agents, not just a benchmark artifact.
Source: www.reddit.com/r/LocalLLaMA/comments/1tizmar/re_what_eveRead original →
FIG-8241:1
80radar
FIG-8241:1

Wed, May 202 dispatches

#0823
#0823Models & API r/LocalLLaMAlast week
`Qwen3.7 Max` hits 5th on Artificial Analysis; 27B/35B still pending
60radar
Qwen3.7 MaxLarge language model — high-end Alibaba Qwen variant
Artificial Analysis puts it near GPT 5.4 xhigh and above Gemini 3.5 Flash. Strong benchmark signal, but migration waits on API price and smaller-model results.
- Ranked 5th on Artificial Analysis, roughly tied with GPT 5.4 xhigh; credible enough to add to model eval lists.
- Gemini 3.5 Flash sits one step lower in the cited ranking, so latency/price will decide the practical winner.
- Qwen3.6 27B trails Max by 6 points; the 27B/35B Qwen3.7 results matter for local or cheaper deployment.
Source: www.reddit.com/r/LocalLLaMA/comments/1tie6gy/qwen37_max_Read original →
60radar
PHOTO
FIG-8231:1
#0822
#0822Generative media r/LocalLLaMA2 weeks ago
`Nova3D` Generates Articulated 3D Objects via Blender Code
50radar
Nova3DOpen-source 3D generation tool — preserves parts and pivots
Instead of mesh blobs, the pipeline asks an LLM to compile native Blender Python scene graphs. Useful as a prompt-to-code pattern, but local models still break complex transforms.
- Nova3D exports multi-part GLB files with transform nodes and pivot axes preserved, so parts can rotate or articulate.
- The core bet is prompt-to-code over diffusion: edit a part node instead of regenerating the whole object from text.
- Frontend uses Flutter plus a Three.js viewport for browser rendering and node manipulation; hosted API is default.
- Local models still hallucinate Blender matrix math on complex transforms, so BYOK Gemini is suggested for better output.
Source: www.reddit.com/r/LocalLLaMA/comments/1thucyj/a_tool_i_buRead original →
FIG-8221:1
50radar
FIG-8221:1

Tue, May 191 dispatches

#0821
#0821Agents & tools r/LocalLLaMA2 weeks ago
Agent Shell Access Hit the `rm -rf /` Failure Mode
40radar
An agent tried rm -rf / while testing a shell-command block. The block worked, but sandboxing must come before shell access.
- The whitelist blocked the harmful command, so damage was zero, aside from operational panic.
- bubblewrap isolation came after the whitelist; that ordering is backward for any agent with shell execution.
- Command allowlists help, but they are a second layer. Filesystem isolation and disposable workspaces should be default.
Source: www.reddit.com/r/LocalLLaMA/comments/1thosnt/got_my_firsRead original →
40radar
PHOTO
FIG-8211:1

Mon, May 181 dispatches

#0820
#0820Agents & tools r/LocalLLaMA2 weeks ago
`SmallCode` hits 87/100 coding-agent tasks with an active 4B model
50radar
SmallCodeLocal coding agent — compound tools for small models
Reliability comes from the harness, not raw model size. The benchmark is self-reported, but the agent patterns are immediately reusable for local-first coding tools.
- Compound tools collapse search-read-edit-verify into one call, cutting the multi-step drift that breaks small models after 3+ tool calls.
- The fix loop runs compile/lint immediately after edits and feeds errors back, so the model only needs to repair concrete failures.
- On repeated failure, tasks shrink from broad file edits to line-level fixes; that is a practical recipe for weaker local models.
- Cloud escalation is scoped to the stuck task when an OpenAI or Claude key exists, keeping most work local without hard failure.
Source: www.reddit.com/r/LocalLLaMA/comments/1tgecrq/i_built_a_cRead original →
FIG-8201:1
50radar
FIG-8201:1

Sun, May 171 dispatches

#0819
#0819Other r/LocalLLaMA2 weeks ago
`llama.cpp` fork enables quantized KV cache with tensor split
50radar
llama.cppLocal LLM inference engine — supports GGUF and CUDA backends
Tensor parallelism becomes usable with quantized KV cache on dual GPUs. Still a fork with MoE caveats, so it is a test-only local inference tweak.
- Benchmarked Qwen3.5 27B Q4_K_M at 30.05 tok/s with -sm tensor vs 21.22 tok/s without it for generation.
- The command uses -ctk q8_0 -ctv q8_0, removing the old tensor-split tradeoff of falling back to non-quantized KV cache.
- Author reports real use rising from about 25 tok/s to 40 tok/s on 3060 12GB + 4070 Super 12GB.
- MoE models currently break with -sm tensor; dense models like Qwen 27B/9B are the safer test target.
Source: www.reddit.com/r/LocalLLaMA/comments/1tflngz/dual_gpu_llRead original →
50radar
PHOTO
FIG-8191:1

PrismML ships `Bonsai Image 4B` as 1-bit and ternary image models

`Qwen3.6 27B` pure `Q4_K_M` GGUF fits in **16GB VRAM**

BeeLlama v0.2.0 boosts inference speed by up to 4.9x on an RTX 3090

DeepSeek raises $10.29B, commits to open-source models over short-term commercialization

`ik_llama.cpp` pushes `Qwen3.6 35B A3B` near 110 tok/s on 12GB VRAM

Cohere launches `Command A+`, an Apache 2.0 MoE open-weight model

`Qwen3.7 Max` hits 5th on Artificial Analysis; 27B/35B still pending

`Nova3D` Generates Articulated 3D Objects via Blender Code

Agent Shell Access Hit the `rm -rf /` Failure Mode

`SmallCode` hits 87/100 coding-agent tasks with an active 4B model

`llama.cpp` fork enables quantized KV cache with tensor split

`Qwen3.6 27B` pure `Q4_K_M` GGUF fits in 16GB VRAM