Hardware
16 minutes read (About 2367 words)

Homemade Cluster Hardware.

Introduction.

This morning, I decided to build a real cluster for my researchs. Why?

In 2012 I started to working projects related with bigdata technologies and from 2014 in 100% bigdata projects.

In all this time I have had the necessity to test the performance in real clusters. In my last project, I am working 100% with AWS, so we create and destroy clusters continuously without buy any type of hardware.

But what about my personal projects? The Cloud and AWS is a fantastic tool to do it, but:

  1. I am not a company with a big budget, so I can't expend hundreds of euros per month only for enjoy. If I do it, my wife kill me.
  2. I am a romantic, so I prefer hardware.

Target.

With the proliferation of new card-sizes single-boards computers, there are a lot of people that is starting to create clusters using this kinds of cheap computers. But is really the best option?

In this post, I am going to try to build a cluster with a budget of €500 (around $520/£426)

To do it, I am going to compare:

I discarded other computers focus in robotics so they don't have enough memory, performance, etc. Also, I excluded solutions that exceeded €100, because I want buy at least 5 computers.

It isn't going to be a production environment, but the idea is to use the cluster for distributed and big data application.

So I am going to analysis the following list of  points:

  • CPU / GPU: The performance per thread and the number of cores is important.

  • Memory: It's important to have enough memory per every job/task executed or node working. 1G of RAM is not a lot, but it is the minimum requirement that I used to exclude a lot of card size computers. Different use cases need different amount of memory. For example:

    • For Elasticsearch cluster to index the OpenStreetMap full planet network, we need a lot of memory.
    • For a Spark or AKKA cluster to process the OpenStreetMap full planet to create a network, where every paralleling block size is around 32M of ram, we don't need a lot of memory.

  • Storage: Always important to store temporal data, also important if the cluster is a storage system like Hdfs, Hbase, Cassandra or Elasticsearch. The minimum requirement will be 32GB and at least 30MB/s.
    Maybe, the best option will be to use an external USB pendrive.
    I am going to consider few different options, comparing the transfer rate in MB/s:

    • microSD write speed (read is faster)

      • UHS-II Speed Class 3 : theoretical maximum rate of 240Mbps (30MB/s)
      • UHS-I Speed Class 1 :  theoretical maximum rate of 80Mbps (10MB/s)
      • Speed Class 10 : theoretical maximum rate of 80Mbps (10MB/s)
      • Speed Class 6 : theoretical maximum rate of 48Mbps (6MB/s)
      • Speed Class 4 : theoretical maximum rate of 32Mbps (4MB/s)
      • Speed Class 2 : theoretical maximum rate of 16Mbps (2MB/s)

    • USB for external drivers:

      • USB 2.0 theoretical maximum rate of 480 Mbps (60MB/s)
      • USB 3.0 theoretical maximum rate of 5 Gbps (640MB/s)
      • USB 3.1 theoretical maximum rate of 10 Gbps (1280MB/s)

    • Thunderbolt theoretical maximum rate of 10 Gbps (1280MB/s)

    • eMMC: Few times faster than the SD.

    • eSATA: theoretical maximum rate of 6 Gbps (768MB/s)

  • Network: I think that 1Gbps Ethernet is the minimum requirement.
  • OS: I need to be able to execute a full distribution of Linux. I prefer Ubuntu.
  • Power: Different systems need different power supply. It is better if the consume is low and it is not necessary one power supply per node.
  • Physical space for allocation: I don't want my office full of hardware without cases!
  • Price: Of course, one of the must important.

I am going to exclude from the comparison all components not related directly with the CPU. For example, USB HUB, Ethernet Switch and cables, etc.

Comparison

Rasperry Pi 3 Odroid XU4 Odroid C2 Banana Pi M64 PINE A64+ 2GB Mini PC UP Squared
CPU 1.2GHz 64 Bit Quad-Core ARMv8 Cortex A53 Samsung Exynos-5422 : Cortex™-A15 and Cortex™-A7 Amlogic ARM® Cortex®-A53(ARMv8) 1.5Ghz 1.2GHz Allwinner 64 Bit Quad-Core ARM Cortex A53 1.2 Ghz 64-Bit Quad-Core ARM Cortex A53 AMD APU Sempron 3850 Quad Core Celeron™ 2C 2.4 GHz N3350
Memory 1GB SDRAM LPDDR2 @ 900 MHz 2GB LPDDR3 2Gbyte DDR3 SDRAM 2GB LPDDR3 2GB LPDDR3 4GB DDR3 2GB LPDDR 4
GPU Broadcom VideoCore IV 3D Mali-T628 MP6 Mali™-450 GPU Dual core Mali 400 MP2 Dual core Mali 400 MP2 AMD Radeon™ R Intel Gen 9 HD<
Video Output HDMI 1.3 HDMI 1.4a HDM 2.0 HDMI 1.4 HDMI 1.4a HDMI 1x Full HDMI 1.4b – 4K @ 30 hz, 1x DP 1.2 – 4K @ 60 hz
microSD MicroSDHC (up to 32 GB) MicroSDHC ?? MicroSDHC ?? microSDXC ( up to 256GB ) microSDXC ( up to 256GB )
USB USB 2.0 PORT (x4) USB 3.0 (x2) USB 2.0 (x1) SB 2.0 PORT (x2) USB OTG (x1) USB 2.0 PORT (x2) USB OTG (x1) USB 2.0 PORT (x2) USB 3.0 (x2) USB 2.0 (x4) USB 3.0 Type A(x3) USB OTG Micro B (x1) USB 2.0 (x2)
eMMC eMMC5.0 eMMC5.0 8G - 32GB eMMC
SATA SSD - - 250GB SATA3 (SATA 6Gb/s) Full-size SATA3 (SATA 6Gb/s)
Ether 100Mbit/s 1Gbit/s 1Gbit/s 1Gbit/s 1Gbit/s 1Gbit/s 1Gbit/s x2 dedicated
SO Raspbian,Ubuntu,Android Things, etc. Android and Linux Ubuntu 16.04 or Android 5.1 Lollipop based on Kernel3.14LTS Android and Linux Android and Linux Windows / Linux Windows / Linux
Power Rec. 5V@2.5A Power rating: 800 mA (4.0 W) 5V@4A input via DC Power 5V@2A via DC Power and/or Micro USB (OTG) 5V@2A via DC Power and/or Micro USB (OTG) 5V@2A via DC Power and/or Micro USB (OTG) 5V@4A via DC Power and/or Micro USB (OTG)
Size 85.60x56.5mm 82x58mm 82x56mm 92x60mm 127x79mm 285x88x320mm 86x90mm
Board price US$35 US$74 US$40 US$74 US$29.00 -
Case US$12 - US$10
Storage US$15 - MicroSDHC Memory Card 32 GB Class 10 UHS-1 U1 - -
Total Price US$62 US$101 US$67 US$101 US$56 US$168 US$168
Total Price €59 €96 €64 €96 €53 €160 €160
Total Price £50 £82 £54 £82 £46 £137 £137

Rasperry Pi 3

Official site: www.raspberrypi.org/ Wiki: https://en.wikipedia.org/wiki/Raspberry_Pi​

It could be the most famous card-size computer, with the biggest community.

The last version in this moment is the Raspberry Pi 3, with 1.2GHz 64-bit quad-core ARMv8 Cortex-A53 CPU, a VideoCore IV 3D GPU and only 1GB SDRAM @ 400 MHz it is not the more powerful computer in the list compared.

Also, a big handicap is the 100Mbps Ethernet and the storage system.

Other extra features: 40 pins GPIO, 802.11n wireless and bluetooth 4.1

Banana Pi M64

Official site:https://www.banana-pi.org/m64.html Wiki: https://en.wikipedia.org/wiki/Banana_Pi#Banana_Pi_M64

This computer board has the same 1.2GHz 64-bit quad-core ARMv8 Cortex-A53 CPU and an equivalent Dual core Mali 400 MP2 GPU.

The big difference is the 1Gbps ethernet, the 2GB LPDDR3 RAM the microSDXC and the 8GB eMMC included in the board.

The problem, the price. It's more than the double of the Raspberry Pi 3. So per every Banana I can buy two Raspberries :)

Pine A64+ 2GB

Official Site: https://www.pine64.org/?page_id=1194 Wiki:https://en.wikipedia.org/wiki/Pine64 This computer board has the same 1.2GHz 64-bit quad-core ARMv8 Cortex-A53 CPU and an equivalent Dual core Mali 400 MP2 GPU.

The big difference is the 1Gbps ethernet, the 2GB LPDDR3 RAM and the microSDXC.

The price is really good. Better than the Raspeberry and cheaper: $30

The bad part, no wifi or bluetooth without an extra $10. Also, old HDMI 1.4

Odroid XU4 and C2

Official site: http://www.hardkernel.com

Wiki: https://en.wikipedia.org/wiki/ODROID

Both are good alternatives.

Both with 2GB Ram, 1GBit/s Ethernet eMMC5.0 that is faster (expensive and difficult to find) than a microsd.

XU4 has a older CPU with a 32bits architecture, but the performance is good enough. Also, has a USB 3.0 port, so you can use really cheap 128GB 100MB/s scandisk pendrives. The problem is that it is one of the more expensive boards.

C2 is the newest one, with new A53 CPU with a 64bits architecture but with USB 2.0 ports.

Mini ITX homemade cheap PCs

Selected configuration: https://uk.pcpartpicker.com/user/angelcervera/saved/6xfmGX

It is going to be more powerful than any other combination, but I can not buy more than 3 servers.

In this field there are a lot of option, from £65 with only 2GB of RAM and 32GB of ROM to £140 with 4GB of RAM and 60GB SSD SATA3:

UP Squared

Site: http://www.up-board.org/upsquared/specifications-up2

The specification are really good. All advantages of a Intel PC and all advantages of a card-size board.

But this comparison it to find a low cost server (no for robotics) and at the end, the price is the same that a Mini ITX.

Conclusion.

Really, there is not a winner.

In my opinion:

  • ​If you have enough money or your wife does not manage it: A custom miniPC is the best option.
  • If you don't want expend a lot, Pine A64+ 2G with a good microSD car is the best option.

Other conclusion is that Rasberry Pi is the more important in the market with the biggest community, but looks that it is not the best relation price/specs.

Reference:

# ,
13€ Recycled 20 cores Homemade Cluster
Docker + UFW = Hidden trap.
Your browser is out-of-date!

Update your browser to view this website correctly. Update my browser now

×