Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
alphabell-research 
posted an update 4 days ago
Post
2,994
One number missing from LFM3's on-device story: memory at 256K context. Liquid AI reports 1.90 GB peak on iPhone 17 Pro at 32K. From the pinned config, f16 K/V cache alone would be 0.41 GB at 32K, 1.61 GB at 128K and 3.22 GB at 256K, before weights and runtime buffers. The chart shows the calculation, not a phone measurement. If you have run longer contexts on-device, share peak memory and time to first token here: LiquidAI/LFM3-1.7B#5

Model: LiquidAI/LFM3-1.7B
alimirew123222

999sx is very awesome game

Twu31

The reported iPhone 17 Pro results in the linked discussion add a useful distinction: the Q4_0 run handles a 256K prompt with about 5.1 GB peak process memory, but cold time to first token is around 16 minutes. That makes KV-cache reuse for a stable prefix a practical consideration for local agents. Linking that measurement beside the estimate would make the trade-off clearer.
Wl6adams

thats easily solved with some representational math especially on frequently similar calculations through the weights, the issue is not the model or weights but runtime you use for it

In this post