My Adventure with Local LLMs on a New M4 Mac mini

Hey there, tech enthusiasts! I’ve been having a blast with my shiny new M4 Mac mini, and it’s all about to spill here because I want you to join me in this exciting journey with local large language models (LLMs). Let’s dive into why I love my Mac mini and how you can set up your own local LLMs using tools like Ollama, Gemini, and DeepSeek.

First off, what’s an LLM? These fancy bit of code can understand and generate human-like text, making them super useful for tasks like content creation, data analysis, language translation, and more. I’ve been mainly using them for coding of the last few days as I try and create an app that will help me keep a track of the characters, locations and magical doohickies in the series of books I’m working on.

If your privacy conscious like me you may be wondering how you can run one of these LLMs without giving all your data away to companies like Google, Facebook or Open AI. Well you have two options really a PC with a massively powerful graphics card or one of Apple’s M Series computers. Considering you can pick up a base model M4 Mac mini with 16GB of RAM for around £600 rather than paying at least double that just for a GPU I would recommend this route.

So how do you go about getting Started with Local LLMs on Your Mac mini?

First, you’ll need to make sure your Mac mini is equipped with at least an M1 chip or newer. The reason for this is the built in Neural Engine in the M Series chips. The tiny bits of silicone are designed for the specific task of running AI models.

After you have your hardware it’s all about setting up the environment by downloading and installing something to run you LLMs I opened for LM Studio as it really easy to use. Search for your LLM, download, select at the top of the app and you can start chatting right away. There are other options such as Ollama which give more features but this is perfect for me just now.

I’ve mainly been focusing on the Gemini, and DeepSeek LLMs for the time being as I’ve heard good things about all three. Each model has unique powers that can be used in various scenarios:

• Content Creation: Ollama and Gemini are great for generating content quickly.
• Data Analysis: DeepSeek is excellent for processing large datasets and extracting insights.
• Language Translation: Both Gemini and DeepSeek can translate text from one language to another, making global communication easier.
• Customer Service: Use sentiment analysis with DeepSeek to gain insights into customer feedback and improve services.
• Educational Tools: LLMs can serve as personalized tutors, generating tailored explanations for students’ questions.

So now that we have an idea of which LLMs can do what it’s worth talking about the Day-to-Day benefits of running LLMs locally. Running LLMs locally on your M Series Mac can boost your productivity by automating routine tasks, enhance creativity with idea generation, and improve data analysis capabilities.

For example I used it to help write this blog post giving it a rude to what I wanted to talk about. It then gave me a draft that I have gone through and change a lot and turned into the post you are reading now. I’ve also been using to generate code for apps and helping me generate ideas for my books.

Embarking on this journey with local LLMs has been incredibly rewarding. By setting up my own local environment using tools like LM Studio, I’ve gained access to powerful AI capabilities that has transform the way I work.

Whether you’re a content creator, data analyst, or business professional, these models are your companions in navigating the complexities of modern digital information. So go ahead, unleash the potential of large language models on your M4 Mac mini and see how it can revolutionise your daily workflow!