Apple fm: Can the free AI on your Mac sort customer messages?

macOS 27 brought a free AI model to Macs that runs without an internet connection; you call it in Terminal with the fm command. Within days, posts saying “every Mac now has free AI” were going around.

At the time we were getting ready to use Jev to sort incoming customer messages. Jev is a paid decision model that puts a probability next to each answer; we do not use it in live work yet. We tried the free model first and shared the result on LinkedIn (in Turkish). When a reader asked for a real comparison, we gave the same messages to Jev as well. This note brings both tests, the setup steps and our conclusion together.

What is Apple fm, and is it on every Mac?

fm is a command that lets you reach Apple’s on-device language model from Terminal. Behind the command is the on-device model that Apple offers developers as Foundation Models and that sits at the core of Apple Intelligence. It runs without an internet connection, there is no usage fee and your message does not leave the computer.

The “every Mac” line is not quite right, though. According to Apple’s Apple Intelligence page, you need a Mac with an M1 or later chip, and the device language and Siri language must be set to the same supported language; Turkish is one of them. On our Mac, the model also had to be downloaded once and a licence had to be accepted on first use.

What did we ask about 18 Turkish messages?

We wrote the messages to look like what could land in the inbox of a software company, a web agency and a restaurant: a team getting an error at sign-in, a subscriber asking for a refund, a shop that wants to sell coffee beans online, a spam offering a backlink package, a job seeker sending a CV, a customer writing that they got food poisoning after a meal.

The answer to yes/no and multiple-choice questions is usually spelled out in the message. Graded questions are harder: we drew the line between “medium” and “high” urgency, and someone else might draw it differently. The wrong answers in that column should be read with this in mind.

fm’s result: 37 of 48 questions

fm answered 16 of the 18 messages. It got 37 of the 48 questions on those 16 messages right, which is 77 percent. It got 13 of 16 yes/no questions, 14 of 16 multiple-choice questions and 10 of 16 graded questions right.

Two of the mistakes stand out. fm called every refund message “high urgency”; we had expected “medium” for both refund messages. It also took a software developer who sent a CV and asked about open positions for a real customer who wanted a company website, and rated the warmth “warm”. It got all three questions on this message wrong, the same way, in all 5 runs.

The two unanswered messages were harmless technical support messages. One came from a team that was getting an error at sign-in and had a client presentation in two hours, asking for urgent help; the other reported that charts on the reports page sometimes came up empty. When we asked for the answer in a template with predefined options (a JSON schema), fm stopped both at its safety filter in all 5 runs and left them unanswered. Asked the same messages in plain text, it answered correctly. The --guardrails permissive-content-transformations option, which should loosen the filter, did not help.

The same messages on a paid model: the Jev comparison

We gave Jev the same 18 messages with the same questions. On the 16 messages fm answered, Jev got 42 of the 48 questions right, which is 88 percent. Across all 18 messages it got 47 of 54 right.

Jev made no mistakes on yes/no and multiple-choice questions. All 7 of its mistakes across the 18 messages were on graded questions: on four support messages it rated urgency one step too high, it rated the warmth of the spam message and the job application “warm” instead of “cold”, and it rated the risk of a mixed restaurant review “medium” instead of “low”. fm was not the only one to overrate the urgency of refund messages; Jev also called both refund messages “high”.

The four messages below show the difference best. The fm column shows in how many of 5 runs the same answer came back; the Jev column shows the answer’s score. For yes/no questions this score is the probability of “yes”. The messages are translated from Turkish.

The real difference: how far can you trust an answer?

When you connect a model to a workflow, the real question is which answers are applied automatically and which ones a person checks. For that, the model has to be able to say “I’m not sure about this”.

With fm, the only signal we had was how often it gave the same answer when we asked the same question 5 times. That signal did not work: almost all of fm’s wrong answers came back the same in all 5 runs. Repeating an answer does not show that it is right.

Jev’s score, on the other hand, really did separate the wrong answers:

The rule we had planned for Jev was this: answers scored above 0.8 are applied automatically, a person looks at the ones below. 0.8 is not a value we measured but a threshold we chose. Even so, the table shows why sending low-scored answers to a person works. We could not build this rule with fm.

Try it on your own Mac: setup and first commands

We tried the commands below on 2 November 2026 on a Mac running macOS 27. You can open Terminal and type them in order.

1. 01

   Check that the model is ready

   Type fm available. If the model is ready, it replies “System model available”; if not, read the message it shows.

2. 02

   Accept the licence

   On first use, open a chat with fm chat, accept the licence by typing yes, then exit. You can read the licence text with fm license.

3. 03

   Ask a single question

   fm respond 'your question' answers one question; echo 'your question' | fm respond works too. Add -i 'instruction' to give the model an instruction.

4. 04

   Get the answer in a fixed format

   With --schema file you get the answer as JSON with set fields. In our test, two harmless messages hit the safety filter only in this mode; if something comes back unanswered, try the same question in plain text.

5. 05

   Measure the length of a text

   fm count-tokens 'text' prints how many tokens a text takes, and -q prints only the number. The Turkish sentence “Merhaba, bugün hava nasıl?” (“Hello, how’s the weather today?”) is 7 tokens.

6. 06

   Optionally, run a local server

   fm serve --host 127.0.0.1 --port PORT runs the model as a local server on this computer (replace PORT with a free port number). It rejects requests from a page at another address (403); to use it from a browser you need a small proxy server running at the same address. Ours is about 90 lines of Python. fm --help lists all commands.

Which jobs would we use it for, and which not?

Reading both tests together, the decision depends on what a wrong answer costs.

When you look at your own inbox, the question to ask is: if a message goes to the wrong place, what do you lose? If all you lose is a label, the free model is enough. If you lose a lead, or the result is a wrong refund, ask for a reliable score next to the answer and send low-scored answers to a person.

The limits of this test

The test is small: 18 messages, 54 answers. We wrote both the messages and the expected answers; on graded questions the expected answer is open to debate. We tested fm on a single macOS version and ran each model on one day. The result shows a direction; it is not statistical proof. We also draw no general conclusion such as “fm is bad at Turkish” or “good at Turkish”; we only describe what happened with these 18 messages.