Clustering Three Minisforum MS-02 Ultra Mini Workstations: Monte Carlo, OpenFOAM, and a 671B-Parameter LLM
イチケン / ICHIKEN
Minisforum is best known for mini PCs, but it also sells a compact workstation, the MS-02 Ultra. The video tests the top model, the MS-02 Ultra 285HX. It has a 24-core CPU, 192 GB of DDR5 ECC memory and dual 25G SFP28 networking in a chassis that is only slightly larger than a mini PC. The presenter wanted to find out what that hardware can do under heavy load. Three units were linked into a cluster and given three jobs: a Monte Carlo calculation of pi, a fluid simulation that works every CPU core, and inference with a very large language model that fills the memory.
Exterior and Networking
The presenter calls it quite small for a workstation and one of the smallest normal computers, though bigger than a mini PC. The MS-02 Ultra comes in three models with different specifications, and the one tested is the top model.
The front has a power button and USB ports, including two USB4 V2 Type-C ports. The presenter notes that the standard allows up to 80 Gbps. These ports support USB4 over Ethernet, so two PCs connected with a USB-C cable can form a network. The rear also has display output.
The rear panel has USB, HDMI, USB4, and two RJ45 LAN ports, one 2.5 GbE and one 10 GbE. There are also two 25 GbE SFP28 ports. These come from a network expansion card, not from the motherboard. The presenter describes the networking as "quite luxurious."
The SFP28 ports use an Intel E810 Ethernet controller. The presenter considers its main advantage to be support for RDMA (Remote Direct Memory Access), which lets networked computers exchange data directly between their memories without going through the OS or CPU, so transfers are fast. The presenter expected RDMA to matter for the cluster simulations. They also speculated that Minisforum may have picked an RDMA-capable card because it designed the machine for cluster computing from the start.
Inside the Chassis
The case is aluminum. The presenter guesses this helps heat dissipation in such a small chassis. It looks seamless, and the presenter wondered whether it was extruded, but did not confirm it. The interior is tightly packed. From the bottom up it holds the power supply and network card, then the CPU and a free PCIe slot. The slot can take a low-profile graphics card. Space is limited, though, and the presenter says that makes it a little hard to decide what to install there.
Removing a cover exposes the CPU cooler, which has one fan at the intake and one at the exhaust. The CPU is an Intel Core Ultra 9 285HX with 8 Performance cores and 16 Efficient cores, 24 threads in total. It also includes an NPU for AI processing. The presenter's view is that its 13 TOPS is "honestly not enough," so it is best treated as a bonus. It could handle image classification, object detection, or running a small LLM, but the presenter thinks adding a GPU would give more satisfying results.
Memory slots sit on both sides of the board. Two are under the heat sink and two more are on the back, which also holds the Wi-Fi module and the factory-installed Kioxia SSD. The presenter used the second, empty M.2 slot on the back for an extra SSD to hold a separate boot OS. The network card carries two more M.2 slots, so the machine can take up to four SSDs in total.
Cluster Setup
Each node has one 24-core CPU that runs 24 threads. With three identical units, the cluster runs 72 threads. Parallel work across nodes needs data exchange and synchronization, and the presenter used two MPI implementations for this: Open MPI and Intel MPI. Intel MPI can use RDMA, so the presenter also planned to measure the difference with and without it.
The nodes were connected to a switch over the 25G SFP ports with direct-attach cables. The presenter flagged one concern: the switch runs at only 10 Gbps. They did not yet know whether bandwidth or latency matters more for these workloads, and went ahead with that switch. The parallel code and MPI setup instructions are published on GitHub and in a blog post.
Monte Carlo Calculation of Pi
The first test was a basic parallel job. Dots are placed at random inside a square, and the share that lands inside the circle gives an estimate of pi. The presenter explains that Monte Carlo methods suit parallel computing because each trial is independent. Results do not affect each other, and no thread or node waits for an earlier step to finish, so the CPU's resources are used efficiently. More threads should therefore mean a shorter run.
The run used 200 million samples. Results:
- 1 thread: 42.3 seconds, with only CPU 0 at 100% utilization
- 2 threads: 22.8 seconds, roughly half
- 24 threads (one full node): 2.77 seconds
- 48 threads (two nodes): 1.41 seconds
- 72 threads (all three nodes): 0.97 seconds
The presenter observed that time falls sharply as threads increase but is not exactly inversely proportional to the thread count. They attribute the gap to time spent on work other than the calculation itself.
Fluid Simulation with OpenFOAM
The second test was numerical fluid simulation, such as computing how water flows. The presenter used OpenFOAM, which is open source and free.
The presenter gave a simplified explanation. The object is divided into a mesh, and the differential equations are solved algebraically by adding and subtracting values between neighboring cells. A finer mesh on the same model captures more detail but has more cells. Keeping the cell size and enlarging the simulated area also increases the cell count. Either way, more cells means more computation time, so parallel computing helps here too.
Unlike the Monte Carlo case, adding threads and nodes did not always make OpenFOAM faster. The presenter showed timings for a two-dimensional dam-break simulation. On a single node, run time generally fell as threads increased. The presenter had expected that connecting more nodes would cut the time further, but admits it was "not that naive." Adding nodes made the simulation take much longer.
Further testing showed two things. First, if each thread handles too few mesh cells, time spent on other processing outweighs the calculation. Second, the simulation finished faster when it did not use all 24 threads on a node. The presenter thinks this happens because the CPU and memory get congested within each machine, and communication between nodes also becomes congested. Their rule of thumb is that parallelizing only pays off when each thread gets roughly several hundred thousand mesh cells. At that size, calculation time is large relative to overhead.
Intel MPI and RDMA
Up to this point, inter-node communication used Open MPI. Because the E810 supports Intel MPI and RDMA, the presenter tried Intel MPI to see if it would speed things up. Intel MPI cut run time by 17% compared with Open MPI.
The presenter noted that the E810 appears to support RDMA with Open MPI as well. Setting that up looked difficult, so they gave up for this video and would like to try again later.
Their main conclusion from the OpenFOAM work is that, for this kind of workload, the difference between 10 Gbps and 25 Gbps bandwidth "is not actually very important." They believe network latency has a much larger effect.
Running DeepSeek-R1 671B on CPU and RAM
The last test used the large memory. LLM inference normally relies on GPUs and high-bandwidth memory. This time the presenter deliberately used only CPUs and system RAM. The model was a Q4_K_M quantization of DeepSeek-R1, a full-size model with 671 billion parameters. The model files alone are about 400 GB, and they were loaded into RAM across the three nodes.
After loading, the first node used 184 GB of its 192 GB (96%), the second 170 GB, and the third 169 GB. The presenter said they had never seen that much memory in use. Because free memory was pushed so close to the limit, the context size could not be increased and was set to about 20,000 tokens.
During a chat, the main node's CPU was busy and its Ethernet traffic stayed at about 100 Mbps. The other nodes showed no visible CPU processing, but they also kept up constant Ethernet traffic of about 100 Mbps. At one point Ethernet usage suddenly dropped, and the presenter was unsure why. The first prompt asked for a long talk, which was too slow, so the presenter changed it to ask for about 400 words. The output rate was about 1.1 tokens per second. The presenter reran the earlier prompt to see what had gone wrong and watched the model's reasoning appear.
The presenter admits the speed is poor next to something like ChatGPT, which is unavoidable for a local model. Seeing such a large model run at home is romantic, they say, but the speed brings you back to reality. They ended with an open question: is there a large model that runs faster on CPU?
Conclusions
For numerical computing, the presenter sees real value in the high core count and large RAM. Compared with workstations built for serious numerical work, though, it has fewer lanes connecting the CPU and memory, so it does not match them. For LLMs, the presenter still considers GPUs the proper tool. Inference works on CPU, but the presenter did not find it very practical. The remaining advantage is that a single unit with 192 GB of memory can hold a large model at home.
Hello. Today I'll use a small computer packed with a lot of hardware. Here it is: Minisforum's MS-02 Ultra. Minisforum itself is famous for its mini PCs, but it also offers workstation computers like this. As you'd expect from a mini PC manufacturer, this workstation is quite small. As you can see, it's small.
Inside, it has a 24-core CPU, 192GB of DDR5 ECC memory, and dual 25G SFP28 ports. The specifications are quite high-end.
In today's video, I'll try experiments that use the full performance of this mini workstation. To use up a large amount of memory, I'll try inference with a large language model, or LLM. To use the multicore CPU, I'll try fluid simulation. This time, I'll use three units, not just one, as a computing cluster. First, let's see how much is packed into the MS-02's small chassis.
Let's start with the exterior. It's quite small for a workstation. It's bigger than a mini PC, though. It's among the smallest regular computers. The MS-02 Ultra comes in three models with different specifications. This is the top-of-the-line MS-02 Ultra 285HX.
The front has a common configuration, including a power button and USB ports. This USB-C section has two USB4 V2 ports. The standard supports up to 80 Gbps, so it's quite fast. Ethernet over USB4 is supported. For example, connecting compatible PCs with a USB-C cable also lets you create a network. You can output video from the back of the unit, and this also serves as a display output.
Next, the back: USB, two RJ45 ports for LAN cables, HDMI, USB4, and SFP28. The two RJ45 ports support 2.5 and 10 gigabit Ethernet. There are also two 25-gigabit SFP28 Ethernet ports. These come from a network expansion card rather than the motherboard. The networking specifications are quite generous.
The SFP28 ports use an Intel E810 Ethernet controller. What's good about this controller is that it supports RDMA, or remote direct memory access. This technology exchanges data directly between the memories of networked computers. Its advantage is high-speed data exchange without involving the OS or CPU. This time, I'll run simulations on a cluster of three of these computers, and RDMA works very well for this. Minisforum may have selected a network card supporting RDMA with this kind of cluster computing in mind from the beginning.
Next, let's look inside. The case is made of aluminum. I'm guessing this is for heat dissipation, since the chassis is so small. It looks seamless. Is this made by extrusion? It's quite intricately made.
It's also very compact inside. From the bottom, there's the power supply and network card, then the CPU and free PCIe slots. The available PCIe slots have enough room for a low-profile graphics card. With so little space, it's a bit hard to decide what to install.
You can remove this cover to see the CPU cooler. It has two fans, one at the intake and one at the exhaust. The CPU is an Intel Core Ultra 9 285HX, with eight Performance cores and 16 Efficient cores, for a total of 24 threads.
It also has an NPU for AI processing, but honestly, 13 TOPS isn't much, so it's best to think of it as an added bonus. It's used for image classification, object detection, and running small LLMs. But adding a GPU seems more satisfactory than using the NPU.
Hidden under this heat sink are two DDR memory modules. Next, the back side has two more DDR memory modules, a Wi-Fi module, and the originally installed SSD. I have a Kioxia SSD here. This was an empty expansion slot, and this time we added a drive to keep the boot operating systems separate.
For SSD expansion, there are actually M.2 slots on the network card, allowing two additional drives. That means you can install up to four SSDs in total. We've now looked at a rough overview of the hardware. From here, we'll actually use it for computing.
First, let's summarize the available computing resources. This computer has one CPU, which can run 24 threads on 24 cores. Those are the computing resources for one node, or one PC. There are two more identical nodes, so we can run a total of 72 threads across three nodes, forming a single cluster. With this many threads, it should be strong at parallel computing.
Parallel computing requires data exchange and synchronization between nodes. This time, we decided to use Open MPI and Intel MPI for that. Intel MPI can use RDMA, so we'll also test how much difference RDMA makes.
For the network connection, we'll basically use the 25-gigabit SFP ports with these direct-attach cables. I do have a concern: this switch actually runs at 10 gigabits. Which matters more for the computation, bandwidth or latency? I'm still trying to figure that out, so I'll proceed with this switch for now.
The parallel computing code used in today's experiments and instructions for setting up MPI are all collected on GitHub and in a blog post linked in the description. If you're interested, please check them out.
First, as an introductory multithreaded parallel computing task, we'll calculate pi using the Monte Carlo method. You place dots randomly inside this rectangle, count how many fall inside the circle, and calculate the proportion.
Monte Carlo methods work well with parallel computing because each calculation is independent. The results of individual calculations don't affect each other. Threads and nodes also don't have to wait for a previous calculation step to finish, so the CPU's computing resources can be used efficiently. In other words, the more threads you have for parallel Monte Carlo calculations, the sooner you can finish.
Let's try it. First, we'll use only one thread. I'm starting the calculation now. As you can see, only CPU 0 has 100% utilization. We'll leave it like this for a while. The calculation has finished. With one thread, it took 42.3 seconds.
Next, let's see what happens with two threads. Looking at CPU utilization, the two CPUs are staying at 100%. They're working hard on the calculation. We've set the number of Monte Carlo trials to 200 million this time. The calculation has finished. It took 22.8 seconds, roughly half the time.
Next is 24 threads, the maximum available on a single node. All CPU utilization is now at 100%, but the calculation finished almost immediately. That's fast. With 24 threads, it took 2.77 seconds. Using more threads considerably reduces the calculation time, although the relationship doesn't seem perfectly inversely proportional. That's because some time is spent on things other than computation. This is what it looks like.
Next, 48 threads. The calculation will probably finish in a moment. It's done: 1.41 seconds. Finally, we'll use all three nodes for a total of 72 threads. The calculation is done: 0.97 seconds. Parallel computation can make computation times very short for various tasks.
Next, I'll try fluid simulation as a numerical calculation. In plain English, this calculates water flow like this. There are various fluid simulation programs. This time, we used OpenFOAM, which is open source and free to use.
To explain the calculation very simply, you divide the object into a mesh like this, then perform additions and subtractions between neighboring mesh cells to solve differential equations algebraically. If you keep the model the same size and make the mesh finer, you can calculate numerical values down to very fine details, but the number of mesh cells increases.
Even if the mesh cell size stays the same, you might want to simulate a larger area. You can then simulate a larger model over a wider region, but the number of mesh cells still increases. More mesh cells mean more computation time. Parallel computing is effective for these numerical calculations, too.
However, unlike the Monte Carlo method described earlier, simply parallelizing the computation isn't enough. In OpenFOAM, increasing the number of threads or nodes didn't necessarily shorten the calculation time.
These results make it easy to see. This compares the time taken with different thread counts for the two-dimensional dam-break calculation shown earlier. With only one node, the simulation tends to finish sooner as the number of threads increases. I thought connecting more nodes and increasing the thread count would save time, but it wasn't that simple. Instead, the simulation took much longer.
Through various tests, we found that when the number of mesh cells assigned to each thread is too small, the time spent on other processing becomes relatively large compared with the computation time. We also found that the simulation takes less time when we don't use all 24 threads. I think this is because of the complicated interactions involving the CPU, memory, and communication between nodes.
So here's what I'd like to do: calculate with around several hundred thousand mesh cells per thread. This makes the time spent computing relatively longer than the time spent on other tasks, so parallel computation becomes worthwhile.
Up to this point, I've used Open MPI for parallel computation between nodes. However, the Intel E810 network card supports Intel MPI and RDMA, so we'll see whether we can speed up the calculation further. Here are the results: Intel MPI reduced the time by 17% compared with Open MPI.
The Intel E810 network card also seems to support RDMA with Open MPI. I gave up on that this time because the optimization setup looked difficult. I'd like to try again if I get the chance.
One thing I learned this time is that when using OpenFOAM and similar software, whether the network bandwidth is 10 or 25 gigabits isn't actually that important. Network latency seems to have a much greater effect.
Finally, as an experiment using the large memory capacity, I'll try inference with a large language model, or LLM. LLM inference generally seems to use GPU power and high-bandwidth memory. This time, we'll deliberately use the CPU and RAM.
We'll use the DeepSeek-R1 Q4_K_M quantized model. This is the full-size 671B model, with 671 billion parameters. The model alone takes up 400GB. We'll load it into RAM and actually run it.
I've now finished loading it into memory. We're using 184GB of the available 192GB, or 96%. The other two units are also nearly out of memory. This one is using 170GB, and the third is using 169GB. That's impressive. I've never seen this much memory used.
We're very close to the memory limit, so we can't increase the context size. I've adjusted it to around 20,000. Now let's actually chat about something.
As you can see, the CPU is getting used quite a bit. We're also continuously communicating over Ethernet at about 100 Mbps. Let's look at the other nodes. The other machines aren't doing any CPU processing, but there is some sort of continuous Ethernet communication at about 100 Mbps. Suddenly, Ethernet usage dropped. What's going on?
It's too slow, so I'll change the prompt. Maybe "long talk" isn't such a good idea. Let's ask it to talk about something in about 400 words. Sorry, this one hasn't quite finished yet. Now the final response, "Hello," has appeared. This is how it came out.
It's pretty slow, about 1.1 tokens per second. I'll redo the previous one and take another look. Something went wrong earlier. It's doing a lot of thinking internally like this.
Compared with ChatGPT or something similar, it's still slow, although that's understandable because it's running locally. There's an appeal to seeing such a large model running, but the speed suddenly brings you back to reality. Is there a larger model that runs faster on the CPU?
Today, I used Minisforum's MS-02 Ultra mini workstation for numerical calculations and running LLMs. For numerical calculations, I could see how its large number of cores and large RAM capacity could be useful. However, compared with a workstation intended for serious numerical computing, it falls short in areas such as the number of lanes connecting the CPU and memory.
LLMs are also primarily GPU territory. They work on the CPU, but I didn't find it very practical. Still, having 192GB of memory in one unit gives you the advantage of being able to fit a large model in memory. If you're interested in mini workstations, please consider this one as a candidate.
If you found this video useful, please give it a like and subscribe to the channel. If you have any questions or concerns, please let us know in the comments. Thank you for watching until the end.
Article published · Updated