Terabyte(s) SSD On Servers and Now On Laptops

by Jon Davis 3. April 2009 00:29

http://www.engadget.com/tag/pcie%2Cssd

I first found out about 1TB super-super-SUPER fast SSD storage when EVE Online announced that they upgraded their database server with these puppies.

Tonight I just spotted this mention of a laptop with such a thing.
http://www.engadget.com/2009/03/02/asus-slaps-1tb-ssd-within-lamborghini-vx5-laptop/
http://www.engadget.com/2009/03/03/hands-on-asus-lamborghini-vx5-laptop/

"A state of disbelief" doesn't state my feelings enough.

Why Not Big RAM, RAM Disk, and 10 Gigabit Switches and Adapters?

by Jon Davis 9. November 2007 14:54

This is just something I've been pondering lately. I've been doing a lot of work this year with Lucene.Net (a port of Lucene, which is written in Java, to .NET) to manage a search engine. In our configuration, it uses RAMDirectory objects to retain the indexes in memory, then it searches the indexed content as though it was on disk. It takes up a lot of RAM, but it's very performant. A search query, including network load of transfering the XML-based query and the XML-based result set (over Windows Communication Framework), is typically in the range of about 0.05 seconds over a gigabit switch using standard, low-end, modern server hardware.

We don't just spider our sites with this stuff as with Google (or Nutch). We manually index our content using real field names and values per index, very similar to SQL Server tables except that you can have multiple same-name fields with different values in the same index record ("document") which is great for multiple keywords. If we could get it to properly join index on fields like in SQL Server you can join tables on fields, as well as to perform arbitrary functions or delegates as query parameters (which is DOABLE!!), we'd have ourselves something that is useful enough for us to consider throwing SQL Server completely out the window for read-only tasks and get a hundredfold performance boost. Yes, I just said, that!!

Because of the load we put on RAM, trying to keep the I/O off the SCSI adapter and limit it to the memory bus, all of this has led me to question why network and RAM capacities have not evolved nearly as fast as hard drive capacities. It seems to me that a natural and clean way of managing the performance of any high-traffic, database-driven web site is to minimize the I/O contention, period. I hear about people spending big money on redundant database servers with all these terabytes of storage space, but then only, say, 16 GB of RAM and gigabit switch. And that's fine, I guess, considering how when the scale goes much higher than that, the prices escalate out of control.

That, then, is my frustration. I want 10 gigabit switches and adapters NOW. I want 128GB RAM on a single motherboard NOW. I want 512GB solid state drives NOW. And I want it all for less than fifteen grand. Come on, industry. Hurry up. :P

But assuming that the hardware became available, this kind of architectural shift would be a shift, indeed, that would also directly affect how server-side software is constructed. Microsoft Windows and SQL Server, in my opinion, should be overhauled. Windows should natively support RAM disks. Microsoft yanked an in-memory OLE-DB database provider a few years ago and I never understood why. And while I realize that SQL Server needs to be rock-solid for reliably persisting committed database transaction to long-term storage, there should be greater design flexibility in the database configuration and greater runtime flexibility, such as in the Transact-SQL language, that determines how transactions persist (lately or atomically).

Maybe I missed stuff that's already there, which is actually quite likely. I'm not exactly an extreme expert on SQL Server. I just find these particular aspects of data service optimizations an area of curiosity.


 

Powered by BlogEngine.NET 1.4.5.0
Theme by Mads Kristensen

About the author

Jon Davis (aka "stimpy77") has been a programmer, developer, and consultant for web and Windows software solutions professionally since 1997, with experience ranging from OS and hardware support to DHTML programming to IIS/ASP web apps to Java network programming to Visual Basic applications to C# desktop apps.
 
Software in all forms is also his sole hobby, whether playing PC games or tinkering with programming them. "I was playing Defender on the Commodore 64," he reminisces, "when I decided at the age of 12 or so that I want to be a computer programmer when I grow up."

Jon was previously employed as a senior .NET developer at a very well-known Internet services company whom you're more likely than not to have directly done business with. However, this blog and all of jondavis.net have no affiliation with, and are not representative of, his former employer in any way.

Contact Me 


Tag cloud

Calendar

<<  December 2014  >>
MoTuWeThFrSaSu
24252627282930
1234567
891011121314
15161718192021
22232425262728
2930311234

View posts in large calendar