The original post: /r/linux by /u/diagraphic on 2024-12-24 12:33:55.

Hello, my fellow Linux users! I and others have been working on an open-source storage engine called TidesDB for a couple of months now, and we are at v0.5.0b, which is a fairly big milestone for me—halfway to TidesDB 1. TidesDB is an LSM(log structured merge) tree-based storage engine designed from the ground up to be simple, fast, and efficient.

Here are some features!

  • ACID transactions are atomic, consistent, isolated, and durable. Transactions are tied to their respective column family.
  • Concurrent multiple threads can read and write to the storage engine. Column families use a read-write lock thus allowing multiple readers and a single writer per column family. Transactions on commit block other threads from reading or writing to the column family until the transaction is completed. A transaction is thread safe.
  • Column Families store data in separate key-value stores. Each column family has their own memtable and sstables.
  • Atomic Transactions commit or rollback multiple operations atomically. When a transaction fails, it rolls back all operations.
  • Cursor iterate over key-value pairs forward and backward.
  • WAL write-ahead logging for durability. Column families replay WAL on startup. This reconstructs memtable if the column family did not reach threshold prior to shutdown.
  • Multithreaded Compaction manual multi-threaded paired and merged compaction of sstables. When run for example 10 sstables compacts into 5 as their paired and merged. Each thread is responsible for one pair - you can set the number of threads to use for compaction.
  • Bloom Filters reduce disk reads by reading initial blocks of sstables to check key existence.
  • Compression compression is achieved with Snappy, or LZ4, or ZSTD. SStable entries can be compressed as well as WAL entries.
  • TTL time-to-live for key-value pairs.
  • Configurable column families are configurable with memtable flush threshold, data structure, if skip list max level, if skip list probability, compression, and bloom filters.
  • Error Handling API functions return an error code and message.
  • Easy API simple and easy to use api.
  • Multiple Memtable Data Structures memtable can be a skip list or hash table.
  • Multiplatform Linux, MacOS, and Windows support.

https://github.com/tidesdb/tidesdb

I hope you get a chance to check it out! do let me know your thoughts, questions, etc. Cheers!