Windows has a technology I just stumbled upon that should make a few *nix folks jealous. It’s called Esent. It’s been around since Windows 2000 and is still alive and strong in Windows 7.
What is Esent, you ask? It’s a database engine. No, not like SQL Server, it doesn’t do the T-SQL language. But it is an ISAM database, and it’s got several features of a top-notch database engine. According to this page,
ESENT is an embeddable, transactional database engine. It first shipped with Microsoft Windows 2000 and has been available for developers to use since then. You can use ESENT for applications that need reliable, high-performance, low-overhead storage of structured or semi-structured data. The ESENT engine can help with data needs ranging from something as simple as a hash table that is too large to store in memory to something more complex such as an application with tables, columns, and indexes.
Many teams at Microsoft—including The Active Directory, Windows Desktop Search, Windows Mail, Live Mesh, and Windows Update—currently rely on ESENT for data storage. And Microsoft Exchange stores all of its mailbox data (a large server typically has dozens of terrabytes of data) using a slightly modified version of the ESENT code.
Significant technical features of ESENT include:
- ACID transactions with savepoints, lazy commits, and robust crash recovery.
- Snapshot isolation.
- Record-level locking (multi-versioning provides non-blocking reads).
- Highly concurrent database access.
- Flexible meta-data (tens of thousands of columns, tables, and indexes are possible).
- Indexing support for integer, floating point, ASCII, Unicode, and binary columns.
- Sophisticated index types, including conditional, tuple, and multi-valued.
- Columns that can be up to 2GB with a maximum database size of 16TB.
Note: The ESENT database file cannot be shared between multiple processes simultaneously. ESENT works best for applications with simple, predefined queries; if you have an application with complex, ad-hoc queries, a storage solution that provides a query layer will work better for you.
Wowza. I need 16TB databases, I use those all the time. LOL.
My path to stumbling upon Esent was first by looking at RavenDB, which is rumored to be built upon Esent as its storage engine. Searching for more info on Esent, I came across ManagedEsent, which provides a crazy-cool PersistentDictionary and exposes the native Esent with an API wrapper.
To be quite honest, the Jet-prefixed API points look to be far too low-level for my interests, but some of the helper classes are definitely a step in the right direction in making this API more C#-like.
I’m particularly fascinated, however, by the PersistentDictionary. It’s a really neat, simple way to persist ID’d serializable objects to the hard drive very efficiently. Unfortunately it is perhaps too simple; it does not do away with NoSQL services that provide rich document querying and indexing.
Looks like someone over there at Microsoft who plays with Esent development is blogging: http://blogs.msdn.com/b/laurionb/