Perfect Tool Calling : Post in Locallama about user's experience with "711".
https://www.reddit.com/r/LocalLLaMA/comments/1v275de/davidau_somehow_managed_to_improve_qwen_36_27b/
Snippet below:
So I hopped on the HF page and noticed a few things that stood out.
The model is a collaboration between different people. This model isn't a one man show like most of his models iirc.
There are benchmarks. And some of them have large improvements and none of them show regression. Too good to be true. But benchmarks are not the full story. But what this showed is that whatever is going on, didn't hurt the model.
There is a focus on agentic performance, but I think adding fable in the name might be a bit of a stretch since we don't have the true thought traces for fable.
--
Previously, I'm using IQ4XS by unsloth with preserve thinking. I need 262K context on my Hermes agent so I do have to run Q4 KV on my 3090 (I got two more GPUs on the way)
Upon loading DavidAU's model and using my unsloth settings, a few things immediately stood out to me.
The model is not broken.
The model reasoning is much more efficient.
The reasoning of this model is different than the stock model. I often see it make a structured plan in the reasoning block the stock model doesn't do. There seems to be more nuance in the reasoning as well.
--
The immediate vibe is obvious. This feels better than stock model. But vibes aint shit. So I put it through my personal crons through my Hermes agent.
I have a long context setup where I need the agent to log into my work scheduling software, locate overtime bonus pay shifts (with very specific filters) and notify me. The website is quite complex with various traps and JS elements.
I've ran it 10 times. With full context clear each time. The unsloth stock model fails 2/10 times. This model did not fail. No failed tool calls either at KV Q4.
This model is going to be interesting at unquanted KV and Q8 weights. Y'all gotta try it. DavidAU cooked for once on this model.
FYI: use the vision mmprog from unsloth. it's half the size and I had zero issues with it.
I've been using this model as well. So far it's been better than the stock model.
The OP does point out that the unsloth mmprog is smaller and still works well. You probably should switch yours out. I got some free VRAM back when I swapped.
David, I saw in the other thread that the source of this is posted elsewhere, but that model's MTP head layer is broken. Can you upload the unquantized version of this so that people can try it with autoround? llama.cpp is noticably worse than vllm on dual graphics card setups
I fixed the source and added the missing MTP layer and files.
-G
omg its coding 1 shot games on my pc. no erros i love you david!
Interesting that others are showing better tool calls. I have been having a lot of problems with looping tool calls in Hermes with this model. It may certainly be something I am doing incorrectly or poorly.
Tool calling is very dependent on Jinja template functions. Hermes may require some Jinja template adjustment[s] or settings adjustments.
Also, tool calling in general: Q6 or higher is strongly suggested.
You might be able to get away with IQ4XS/Q4s ; but the rule of thumb (all models) is Q6 , or Q5s min quant.
Interesting that others are showing better tool calls. I have been having a lot of problems with looping tool calls in Hermes with this model. It may certainly be something I am doing incorrectly or poorly.
Use default 1.0 temperature.
Thanks for the feedback. Unfortunately I don't have the VRAM to run Q5 or Q6 quants. I am able to sqeeze by with Q4 in most cases. This looping has started recently with MTP and uncensored models. I don't recall it being a problem when I was using a more traditional Unsloth version, but it's hard to be sure. Good information though, so thank you very much.