While we were in California I was reading random books I downloaded during the pandemic and got sucked into a book about bioinformatics, (Applied Bioinformatics, Paul M. Selzer). It was way over my head in many ways, but one thing I did understand is that there are a number of public databases of genetic data, some of which are hosted by the U.S. government.
Given the state of things, I thought it would be prudent to back these up.
Starting with www.ncbi.nlm.nih.gov I found that all of this data is available via FTP, which makes the process of backing it up very simple, however after I started I ran into a snag: it's huge.
I assumed it would be a lot of data but I didn't expect terabytes. I couldn't even pull-down the first directory with the space I had on my laptop, and I'm not sure I even have enough room on the pod for it.
Looks like it's time to grow the pod.
I've been using 1TB SATA SSD's to rebuild the pod. I could have sworn that the last one I bought was under $100, but right now the cheapest I can find is $125, not bad but surprising. This has me questioning the choice to use only SSD in this pod.
The rationale behind going pure SSD is that they use less power, make less noise and in general put less stress on the system than mechanical drives. They have less capacity, but I figured that 45TB raw capacity would be enough for anything I'm doing in the lab.
But if the price keeps going up (perhaps driven indirectly by AI datacenter demands?) maybe I should reconsider mechanical drives.
Then there's the matter of expanding the pod on the software size. I'm using ZFS, but it is an old version so not all the options available today are on the table. My understanding is that I'll need to create a new raidz vdev, which means getting at least three new disks before I can do anything.
Getting enough disks for a new pool gets me thinking about the potential of sending the existing pool over to the new one and fixing a lot of problems with the current setup. The pod is running an ancient version of Debian, for long boring reasons. I had intended to remedy this before putting it into production use but well... didn't. Now it is storing about 2TB of data, too much to simply copy to any other single machine that I have online, so I don't want to go messing-around with upgrading the operating system and all that.
But if I had a second pool that I could send this one to, I could safely rebuild it, and even rebuild the ZFS setup so that it's more amicable to being expanded, etc., maybe even experiment with compression, deduplication and all the other cool ZFS stuff.
...of course, this makes me think maybe it's time to finally go beyond ZFS and use JSFS... but that means getting JSFS up-to-date, and probably writing a FUSE or some other sort of thing that will let me mount it like a normal filesystem for the applications I run that can't be adapted to a REST filesystem...
What a rabbit hole.
I think what I might do today is simply park the effort to backup the genome and come up with a plan to build a new storage array big enough to back the current contents of the pod up to. The outcome of that projet will inform all these other choices, and might move a few other efforts forward as well.
Jason J. Gullickson, 2026