James Hall
@mrjameshall.bsky.social
Founder of Parallax, author of jsPDF
Getting your head around sampling (and constrained sampling - like used for Structured Outputs) is a good area to get stuck into.
January 4, 2025 at 12:45 PM
Getting your head around sampling (and constrained sampling - like used for Structured Outputs) is a good area to get stuck into.
- Built a basic UI on top of this to start exploring.
Some interesting advantages to having greater insight into underlying probabilities is knowing when it might be wrong, debugging models issues, and for learning purposes.
Some interesting advantages to having greater insight into underlying probabilities is knowing when it might be wrong, debugging models issues, and for learning purposes.
January 4, 2025 at 12:45 PM
- Built a basic UI on top of this to start exploring.
Some interesting advantages to having greater insight into underlying probabilities is knowing when it might be wrong, debugging models issues, and for learning purposes.
Some interesting advantages to having greater insight into underlying probabilities is knowing when it might be wrong, debugging models issues, and for learning purposes.
What I've done:
- Pulled Llama3.3 locally from HuggingFace, converted it to GGUF for llama.cpp.
- Quantized it so it doesn't crash my M3 Mac.
- Started a llama.cpp server, and use the raw (non-OpenAI-compatible) API to delve into the underlying logprob data.
- Pulled Llama3.3 locally from HuggingFace, converted it to GGUF for llama.cpp.
- Quantized it so it doesn't crash my M3 Mac.
- Started a llama.cpp server, and use the raw (non-OpenAI-compatible) API to delve into the underlying logprob data.
January 4, 2025 at 12:45 PM
What I've done:
- Pulled Llama3.3 locally from HuggingFace, converted it to GGUF for llama.cpp.
- Quantized it so it doesn't crash my M3 Mac.
- Started a llama.cpp server, and use the raw (non-OpenAI-compatible) API to delve into the underlying logprob data.
- Pulled Llama3.3 locally from HuggingFace, converted it to GGUF for llama.cpp.
- Quantized it so it doesn't crash my M3 Mac.
- Started a llama.cpp server, and use the raw (non-OpenAI-compatible) API to delve into the underlying logprob data.
Add your own words or compare predefined sets.
I'm working on a way to visualise the famous king - man + woman = queen example. Any other ideas?
I'm working on a way to visualise the famous king - man + woman = queen example. Any other ideas?
December 27, 2024 at 5:13 PM
Add your own words or compare predefined sets.
I'm working on a way to visualise the famous king - man + woman = queen example. Any other ideas?
I'm working on a way to visualise the famous king - man + woman = queen example. Any other ideas?