There is no one answer that fits all. Where cloud will always be cheaper is data storage.

If you were to host everything on-prem, that would be a lot of capex. It would cost to maintain that as well. For on-prem, you have to think more about electricity, redundancy, backups, security, and so on. Anything you would need to do to build out a data center. Once you have it set up though, yes it would be cheaper.

For tech companies, this is already a non starter as they want to scale and scale fast. They also can’t just spend all their investors money, so they convert capex into opex instead.

Also, historically, IT is slow. Very slow. This is why there is a world of DevOps because developers became increasingly frustrated with how slow it is to provision infrastructure for them. To fix this, you could probably hire more people, but again, that’s an extra expensive that you can just now offset to cloud.

With cloud you can set up something in multiple data centers within minutes. If on-prem, you would need to have multiple physical locations of your own.

Another option is to rent out space in a data center, then you just buy your own hardware and do not have to worry about 80% of what would go into a data center. You would still need to set up these systems in a way that can scale for future use, which means more capex up front.

At the end of the day, there is no one size fits all. As you mentioned, most businesses could benefit in the long run by hosting their own stuff. I will say though, managing things like your own email server has become a nightmare. This is just a lot easier to let someone else manage. Then again, you have the concern of data storage, this is just easier and cheaper to host in cloud. Something like Google workspace or m365.

To put it another way, go to your boss and tell him you need to pay $2,000,000 up front for IT hardware. Now tell him you’ll need to pay $250,000 a year for the same services in cloud. What do you think they will go with?

I do hate that it’s come to this though, because I feel like people are losing knowledge. Only the people that build data centers these days will have that IT knowledge and you have people that can no longer tinker like we used to.

LLMs are smart, they are just not intelligent

Vivaldi said they eventually will, but will keep it around as long as possible. So, short answer, yes.

I have to ask, why start with 0? I never understood this with infrastructure. I would do something like 00000 if I did numbers so it would be easy to sort, but I always started with 1. I’m just curious.

You will get different answers. Some people like proxmox with ZFS. You can run vms and lxc containers pretty easily. Some people like running everything in a container and using podman or docker. Some people like to raw dog it and just install everything on bare metal ( I don’t recommend this approach though).

The setup I currently have are 3 servers. One server for compute. This is where I run all my services from. 1 server for storage. 1 server for backup storage.

The compute server is set up with an NFS share that connects to the storage server. These all have a 10gbe nic on a 10gbe switch.

If I could go back and redo this setup again, I would make a few changes. I do have a few NVMe drives in my storage server for the NFS share. The compute server has the user home directories on there, as well as the permanent files for the containers that have volumes. This makes it easy for me to backup that data to the other server as well.

With that said, I kinda wish I went with less storage and built out a server using mostly nvmes. My mobo doesn’t do bifurcation on its x16 slots and so I can only get 1 NVMe per slot. It’s a waste. Nvmes can run somewhat hot, but are smaller and easier to cool than platters. Plus it’s faster to rebuild if something were to happen. You could probably get away with using 1 parity drive because of this.

I would still need a few big drives for my media, but that data is not as critical to me in the event I lost something there.

What I would look for in a storage system are the following:

• mobo with rdimm memory
• bifurcation pcie slots to add adapter cards for NVMe drives or lots of NVMe slots on the mobo.
• if doing 10gbe, use sfp+ nics and a sfp+ switch (runs cooler). Then you would just get sfp cables instead of cat6/6e.
• management port (ipmi)
• as much memory as you can afford

With those requirements in mind, something like an ASRock server motherboard using an AMD epyc would normally fit the bill. I have seen bundles go for about 600-700 on AliExpress.

As far as the OS. I treat the storage server as an appliance. I have truenas on there. This is also the reason I have a separate computer server as it makes it easier for me to manage services the way I want, without trying to hack the truenas box. This makes it easy to replicate to my backup since that is also truenas. I have snapshots every hour and those get backed up. I also have cloud backup for critical data every hour.

Last, but not least, I have a vps server so I can access my services from the internet. This uses a wireguard tunnel and forwards from the vps to the compute server.

For the compute server, I am managing mostly everything with saltbox. Which uses ansible and docker containers for most services.

No matter what you choose, I highly recommend ZFS for your data. Good luck!

That sounds like a really good idea. You basically get the best of everything.

The cool thing about ZFS is the pool information is stored on the disks themselves. You can just plug them in and import the pools.

I decided instead to use ZFS. Better protection than just letting something sit there. Your backups are only as good as your restores. So, if you are not testing your restores, those backups may be useless anyway.

ZFS with snapshots, replicated to another ZFS box. The replicated data also stores the snapshots and they are read-only. I have snapshots running every hour.

I have full confidence that my data is safe and recoverable.

With that said, you could always use M-disk.

Any reason why that board? Not 100% sure what you are trying to do, but it seems like an expensive board for a home NAS. I feel like you could get more value with other hardware. Again, you don’t need a raid controller these days. They are a pain to deal with and provide less protection when compared to software raid these days. It looks like the x16 can be split on that board to be 8/8, so if needed you can add an adapter to add 2 nvmes.

You can just get an HBA card and add a bunch of drives to that as well if you need more data ports.

I would recommend doing a bit more research on hardware and try and figure out what you need ahead of time. Something like an ASRock motherboard might better in this case. The epyc CPU is fine. But maybe get something with rdimm memory. I would just make sure it has a Management port like ipmi on the supermicro.

1. You don’t need zfs cache. Stay away from it. This isn’t going to help with what you want to do anyway. Just have enough RAM.

2. You need to backup your stuff. Follow the 3-2-1 rule. RAID is not a backup.

3. Don’t use hardware raids, there are many benefits to using software these days.

With that said, let’s dig into it. You don’t really need NVMe drives tbh. SATA is probably going to be sufficient enough here. With that said, having mirrored drives will be sufficient enough as long as you are backing up your data. This also depends on how much space you will need.

I just finished building out my backup and storage solution and ended up wanting NVMe drives for certain services that run. I just grabbed a few 1 TB drives and mirrors them. Works great and I do get better performance, even with other bottlenecks. This is then replicated to another server for backup and also to cloud backup.

You also haven’t said what hardware you are currently using or if you are using any software for the raid. Are you currently using zfs? Unraid? What hardware do you have? You might be able to use a pice slot to install multiple NVMe drives in the same slot. This requires bifurcation though.

Just rename the model like they renamed gulf of Mexico.

The same thing happened with me. This was probably going to be my last year anyway, but i noped out real quick after the increase. Only reason I still had it was because I had some stuff in OneDrive that I was slowly backing up elsewhere. That just gave me the motivation to take care of it finally.

Was this in the Bible or something? Why is it immoral?

Let me ask this. Imagine 1 person owned many farms of food. They sell their food and they own a huge house on top of the hill. There is more than enough food to feed every person in town. The only way for anyone to get food is to buy it from this one person since he owns all of the farm land and if anyone tries to farm their own food, he uses his money to push them around and makes them stop.

A family is struggling to find work. The father asks the farm owner if he could get some food to eat. The farm owner obviously says no. Pay or no food, he says. The family ends up starving to death.

Would it be wrong for the family to steal food in this case so they can survive? Or is that immoral? Is the farm owner immoral for not helping them? He has plenty of money to last him 100 lifetimes, his belly is full, but he keeps eating. Who is wrong here?

Can you further expand on why you think it’s bad? I’m generally curious.

Where can you get free water? Unless you have a well or something.

paste. I don’t think a lot of people know this command, but it can be handy at times

Ok but containers generally have a lot less dependencies. If you are making your own images, then you know exactly how to rebuild them. In the event something happens, it makes it much easier to get up and running again and also remember what you did to get the service running. The only other thing that would be better is Nix.

If you use an image that someone is maintaining, this makes it even easier and there are services out there that will keep your containers up to date when a new image is available. You can also just automate your image builds to run nightly and keep it up to date.

Yeah I agree. I just got 20tb in mine. Decided to just z2, which in my case should be fine. But was contemplating the same thing. Going to have to start doing z2 with 3 drives in each vdev lol.

No.

I’ll have to give this another try. I tried it before and I stopped using it for a reason that I don’t remember. It looked promising though.

I know what you mean. There are tools I see everyday and I ask, but why? I have started to just ask, why not? There doesn’t always have to be a use case and sometimes people just want to create shit. They don’t even care if others use it, but want to share it anyway in case there is that one other person that does.