banner
merve.bsky.social
merve
@merve.bsky.social
8.4K followers 680 following 240 posts
proud mediterrenean 🧿 open-sourceress at hugging face 🤗 multimodality, zero-shot vision, vision language models, transformers
Posts Media Videos Starter Packs
Pinned
merve.bsky.social
merve @merve.bsky.social · Nov 1
I did a 1 hr speed-run on multimodal computer vision (VLMs, multimodal retrieval, zero-shot vision) in MIT AI Visions

it's up on youtube by popular demand www.youtube.com/embed/_TlhKH...
6 9 110
merve.bsky.social
merve @merve.bsky.social · May 11
llama.cpp has vision language model support now! ❤️‍🔥

get started with sota VLMs (gemma 3, Qwen2.5VL, InternVL3 & more) and serve them wherever you want 🤩
learn more github.com/ggml-org/lla... 📖
2 5 44
merve.bsky.social
merve @merve.bsky.social · May 2
If you want to ✨ speed-up & harden ✨ your RAG pipelines, use visual document retrieval models ⬇️

We have shipped a how-to guide for VDR models in Hugging Face transformers 🤗📖 huggingface.co/docs/transfo...
3 3 27
merve.bsky.social
merve @merve.bsky.social · Apr 15
here's a good blog on successful DSE model MCDSE, compression and more huggingface.co/blog/marco/a...
Visually Multilingual: Introducing mcdse-2b
A Blog post by Marco Cimolai on Hugging Face
huggingface.co
3
merve.bsky.social
merve @merve.bsky.social · Apr 15
Why do people sleep on DSE multimodal retrieval models? 👀

They're just like ColPali, but highly scalable, fast and you can even make them more efficient with binarization or matryoshka with little degradation 🪆⚡️

I collected some here huggingface.co/collections/...
1 1 12
merve.bsky.social
merve @merve.bsky.social · Apr 15
I'm so hooked on @hf.co Inference Providers (specifically Qwen2.5-VL-72B) for multimodal agentic workflows with smolagents 🥹

get started ⤵️
> filter models provided by different providers
> test them through widget or Python/JS/cURL
2 10
merve.bsky.social
merve @merve.bsky.social · Apr 14
my weekly summary on what's released in open AI is up on @hf.co huggingface.co/posts/merve/...

collection is here huggingface.co/collections/...
1 18
merve.bsky.social
merve @merve.bsky.social · Apr 14
fan-favorite open-source PDF rendering model OlmOCR goes faster and more efficient ⚡️

RolmOCR-7B follows same recipe with OlmOCR, builds on Qwen2.5VL with training set modifications and improves accuracy & performance 🤝

huggingface.co/reducto/Rolm...
16
merve.bsky.social
merve @merve.bsky.social · Apr 12
Hello friends 👋🏼

If visit Turkey this summer, know that millions of Turkish people are doing a boycott, once a week not buying anything and rest of the week only buying necessities

if you have plans, here's a post that summarizes where you should buy stuff from www.instagram.com/share/BADrkS...
Login • Instagram
Welcome back to Instagram. Sign in to check out what your friends, family & interests have been capturing & sharing around the world.
www.instagram.com
2 28
Reposted by merve
merve.bsky.social
merve @merve.bsky.social · Apr 9
SmolVLM paper is out and it's packed with great findings on training a good smol vision LM!

Andi summarized them below, give it a read if you want to see more insights 🤠
andimara.bsky.social
Andi @andimara.bsky.social · Apr 8
Today, we share the tech report for SmolVLM: Redefining small and efficient multimodal models.
🔥 Explaining how to create a tiny 256M VLM that uses less than 1GB of RAM and outperforms our 80B models from 18 months ago!
huggingface.co/papers/2504....
Paper page - SmolVLM: Redefining small and efficient multimodal models
Join the discussion on this paper page
huggingface.co
4 29
merve.bsky.social
merve @merve.bsky.social · Apr 11
the model also has impressive OCR capabilities ⬇️
5
merve.bsky.social
merve @merve.bsky.social · Apr 11
we'll give this model a test on agentic capabilities but here's an example from paper:
1 2
merve.bsky.social
merve @merve.bsky.social · Apr 11
This model consists of a dynamic res handling MoonViT encoder, a projection layer and a 16B MoE decoder (with 2.8B active params)

the paper introduces an interesting pre-training pipeline to handle long context and the model saw 4.4T tokens arxiv.org/pdf/2504.07491
1 1
merve.bsky.social
merve @merve.bsky.social · Apr 11
DO NOT SLEEP ON THIS MODEL

Kimi-VL-A3B-Thinking is the first ever capable open-source reasoning VLM with MIT license ❤️
> it has only 2.8B activated params 👏
> it's agentic 🔥 works on GUIs
> surpasses gpt-4o

I've put it to test (see below ⤵️) huggingface.co/spaces/moons...
1 2 30
merve.bsky.social
merve @merve.bsky.social · Apr 11
InternVL3 is out 💥

> 7 ckpts with various sizes (1B to 78B)
> Built on InternViT encoder and Qwen2.5VL decoder, improves on Qwen2.5VL
> Can do reasoning, document tasks, extending to tool use and agentic capabilities 🤖
> easily use with Hugging Face transformers 🤗 huggingface.co/collections/...
2 12
Reposted by merve
simonwillison.net
Simon Willison @simonwillison.net · Apr 9
Model Context Protocol has prompt injection security problems
simonwillison.net/2025/Apr/9/m...
Model Context Protocol has prompt injection security problems
As more people start hacking around with implementations of MCP (the Model Context Protocol, a new standard for making tools available to LLM-powered systems) the security implications of tools built ...
simonwillison.net
10 23 120
Reposted by merve
jsulz.com
jsulz @jsulz.com · Apr 9
Xet infra now backs 1000s of repos on @hf.co , which means we get to put on our researcher hats and peer into the bytes 👀 🤓

Xet clients chunk files (~64KB) and skip uploads of duplicate content, but what if those chunks are already in _another_ repo? We skip those too.
From Chunks to Blocks: Accelerating Uploads and Downloads on the Hub
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
1 5 18
merve.bsky.social
merve @merve.bsky.social · Apr 9
SmolVLM paper is out and it's packed with great findings on training a good smol vision LM!

Andi summarized them below, give it a read if you want to see more insights 🤠
andimara.bsky.social
Andi @andimara.bsky.social · Apr 8
Today, we share the tech report for SmolVLM: Redefining small and efficient multimodal models.
🔥 Explaining how to create a tiny 256M VLM that uses less than 1GB of RAM and outperforms our 80B models from 18 months ago!
huggingface.co/papers/2504....
Paper page - SmolVLM: Redefining small and efficient multimodal models
Join the discussion on this paper page
huggingface.co
4 29
merve.bsky.social
merve @merve.bsky.social · Apr 6
X'in politikaları sebebiyle işimle alakalı post'ları burada da paylaşıyor olacağım, takip edebilirsiniz 😊
1 1 31
merve.bsky.social
merve @merve.bsky.social · Mar 6
icymi I shipped a tutorial on fine-tuning vision language models on videos ⏯️

learn how to fine-tune SmolVLM2 on Video Feedback dataset 📖 github.com/merveenoyan/...
smol-vision/Fine_tune_SmolVLM2_on_Video.ipynb at main · merveenoyan/smol-vision
Recipes for shrinking, optimizing, customizing cutting edge vision models. 💜 - merveenoyan/smol-vision
github.com
1 3 33
merve.bsky.social
merve @merve.bsky.social · Feb 26
All the multimodal document retrieval models (ColPali, DSE et al) are now under visual document retrieval at @hf.co 📝🤗

take your favorite VDR model out for multimodal RAG 🤝
19
Reposted by merve
andimara.bsky.social
Andi @andimara.bsky.social · Jan 23
Smol but mighty:
• 256M delivers 80% of the performance of our 2.2B model.
• 500M hits 90%.
Both beat our SOTA 80B model from 17 months ago! 🎉

Efficiency 🤝 Performance

Explore the collection here: huggingface.co/collections/...
Blog: huggingface.co/blog/smolervlm
1 2 16
Reposted by merve
andimara.bsky.social
Andi @andimara.bsky.social · Jan 23
Introducing the smollest VLMs yet! 🤏
SmolVLM (256M & 500M) runs on <1GB GPU memory.
Fine-tune it on your laptop and run it on your toaster. 🚀
Even the 256M model outperforms our Idefics 80B (Aug '23).
How small can we go? 👀
1 7 48
merve.bsky.social
merve @merve.bsky.social · Jan 17
Everything that was released passed week in open AI 🤠

> Link to all models, datasets, demos huggingface.co/collections/...
> Text-readable version is here huggingface.co/posts/merve/...
1 3 32
merve.bsky.social
merve @merve.bsky.social · Jan 13
Learn more from their blog post here huggingface.co/blog/vdr-2b-... 📖
Visual Document Retrieval Goes Multilingual
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
huggingface.co
1 1 8
merve.bsky.social
merve @merve.bsky.social · Jan 13
there's a new multimodal retrieval model in town 🤠
@llamaindex.bsky.social released vdr-2b-multi-v1
> uses 70% less image tokens, yet outperforming other dse-qwen2 based models
> 3x faster inference with less VRAM 💨
> shrinkable with matryoshka 🪆
huggingface.co/collections/...
1 2 46