If you have been following my articles about comparing AI services, you’d know that, through some ‘rule of thumb’ reasoning I was able to determine the following ranking of Ai services:

1. Deepseek

2. M365 Copilot

3. Copilot Researcher

4. Gemini

5. Copilot Studio

6. ChatGPT deep research

7. ChatGPT

The problem is that I used the same AI services to potentially evaluate the results that they in fact generated. Could that result in bias? Unsure, but I’d suggest probably, if you look at the results.

What I therefore decided to do was have the original articles evaluated by two AI services that were not on my original list, Claude and Grok. Here’s the result of jus these two:

If I now incorporate these results in the overall results I get the following:

That changes the ranking slightly to:

1. M365 Copilot

2. Deepseek

3. Gemini

4. Copilot Studio

5. Copilot Researcher

6. ChatGPT deep research

7. ChatGPT

with the average score being 20, which most services exceed. ChatGPT still lags, even after this! Interesting, huh?

I think my original conclusion remains valid – most AI services, except for ChatGPT, seem to produce very similar quality on average when prompted in the same way.

I started out to provide an indication of the differences between different AI services here:

Testing the differences between AI services

I did a quick comparison here:

An analysis of how AI services vary

I then did a deep analysis of all the generated articles using:

Copilot Researcher

Gemini Deep Thinking

ChatGPT Deep Thinking

If you now take those three results and assign a score of 7 = highest and 1 = lowest recommendations of each and total them up, you end up with this ranking table:

The winner then appears to be, on average, Deepseek. However, you will note that most AI services tested, except ChatGPT have similar scores, with the ‘average’ score being 12, which most services, except again ChatGPT, scored at or above.

This analysis is far from perfect or ideal or for that matter without bias. There are so many variables that possibly come into play that it very difficult, if not impossible, to get a true ‘apples vs apples’ comparison of AI services. However, I think this result still does provide value if you are looking to answer the question of the ‘best’ AI service. That answer seems to largely be that most AI services, apart from ChatGPT, are pretty much the on par when it comes to prompting, so choosing from amongst these simply based on their response to prompts, doesn’t seem to matter all that much.

Of course, there are plenty of other factors, aside from prompt results, that should be considered. The quality of the generated results also is greatly affected by the actual prompts used and I am sure that also varies across the AI services as well.

What I’ll now be interested to see is what the ‘click’ rate is on each article after a period of time. Will the Google AI service generate more article ‘hits’ than the other articles? Time will tell and I’ll report back once enough time has elapsed. These results also make a good benchmark to potentially test again down the track to see if things have changed at all and the progress these AI agents have made.

Interesting time ahead.

Recently, I have been analysing the results produced from the same prompt in carious AI services. After having the various AI services generate answers I compared their value using Copilot Researcher and Gemini. To provide a final alternate analysis of the articles I used ChatGPT Deep Research and received the following analysis, summarised here:

————————–

Rankings (Value to SMB Owner): Based on clarity, practicality, and depth of recommendations, we rank the articles as follows (1 = most valuable):

1. Deepseek (Aug 31) – Provides a step-by-step guide on replacing firewall functions with M365 features, with concrete examples (Safe Links, web filtering, Conditional Access) and even a cost comparison. Its clear bullet format and action-oriented advice make it highly accessibleblog.ciaops.comblog.ciaops.com.

2. ChatGPT (Sep 2) – Gives an ultra-concise Q&A answer summarizing the essentials. The “Answer in short” explicitly states that for most SMBs a basic firewall suffices and expensive NGFWs add little compared to built-in M365 protectionsblog.ciaops.com. Its brevity makes it very easy to digest (2‑minute read), though it is simple.

3. Copilot Studio (Aug 30) – A bullet-point summary that quickly lists the key protections in M365 (endpoint EDR, email filtering, conditional access, DLP, etc.)blog.ciaops.comblog.ciaops.com. Its Q&A style (“Short answer”) is user-friendly and covers the main points succinctly.

4. M365 Copilot (Sep 3) – Offers a short answer plus a configuration blueprint (steps to enforce MFA, Intune baselines, Defender settings, etc.) focused on identity/device security. This balances brevity with practical steps. It clearly states that for remote-first SMBs, a basic router is enough and effort should go into Intune/M365 policiesblog.ciaops.comblog.ciaops.com.

5. ChatGPT (Aug 28) – A traditional blog-style article with multiple sections on built-in security, Zero Trust, host firewalls, etc. It thoroughly covers many features of Business Premiumblog.ciaops.comblog.ciaops.com. While detailed, its long paragraphs (7‑min read) and technical tone may be harder for a busy non-expert to quickly follow.

6. Gemini (Sep 1) – A formal, research-style analysis (13‑min) with executive summary, threat modeling, TCO, and compliance discussionblog.ciaops.comblog.ciaops.com. It is extremely comprehensive (notably covering things like PCI DSS) but in heavy academic tone. This depth is impressive but likely more than a typical SMB owner needs.

7. Copilot Researcher (Aug 29) – An exhaustively detailed report (20‑min) with an executive summary, configuration guide, feature comparisons, and cost analysisblog.ciaops.comblog.ciaops.com. While it contains valuable info, its length and technicality make it the hardest to consume. It also concludes (like the others) that M365 BP can often replace expensive firewallsblog.ciaops.com, but the sheer volume can overwhelm small teams.

In summary, the shorter Q&A and bullet-list articles (Deepseek, ChatGPT Sep2, Copilot Studio, M365 Copilot) score highest for SMB owners because they deliver clear guidance quickly. The longer, more technical write-ups (Gemini, Copilot Researcher) score lower despite depth.

Overall Best: Deepseek’s article stands out as the most practical: it clearly explains why and how to use M365 features instead of buying pricey firewalls, with concrete steps and examplesblog.ciaops.comblog.ciaops.com. This level of clarity makes it particularly valuable for a small-business owner deciding on cost-effective security.

Recently, I have been analysing the results produced from the same prompt in carious AI services. After having the various AI services generate answers I compared their value using Copilot Researcher. To provide an alternate analysis of the articles I used Gemini Deep Research and received the following analysis, summarised here:

————————–

The seven articles provided by the user all arrive at a similar, well-reasoned conclusion regarding the strategic superiority of M365 Business Premium for the typical SMB. However, they vary in their clarity, depth, and how well they frame the discussion for a business-focused audience. The ranking below is based on an assessment of these qualitative factors, determining which AI tool provided the most valuable and actionable report for an SMB owner.

Ranking Criteria

• Clarity & Structure: How easy is the argument to follow? Is the report well-structured and free of jargon?
• Comprehensiveness: Does the report cover all key aspects of both solutions and the modern security landscape?
• Depth of Insight: Does the report go beyond a simple feature list to discuss total cost of ownership, strategic alignment, and the new security model?
• Actionability: Does the report provide a clear, practical recommendation for the SMB owner?

AI Tool Rankings – M365 vs Hardware Firewalls