API Reference

All methods exported by the native addon execute synchronously to avoid microtask queue scheduling overhead in Node.js, ensuring maximum throughput. Custom high-level utilities like getOrSet seamlessly support both synchronous and asynchronous operations.

---

CacheManager

The central manager responsible for provisioning and isolating individual caches.

TIP

Cross-Thread Sharing: All cache instances are shared process-wide across all Node.js worker_threads. You can instantiate a new CacheManager inside a worker thread and retrieve a cache created by the main thread using manager.getCache(name).

const { CacheManager } = require('offheap');
const manager = new CacheManager();

new CacheManager()

Instantiates a new central cache manager.

• Returns: CacheManager instance.

createCache(name, config)

Creates and returns an isolated cache instance.

const cache = manager.createCache('products', {
  shards: 16,
  eviction: {
    policy: 'w-tinylfu',
    capacity: 10000,
    maxBytes: 100 * 1024 * 1024 // 100 MB byte-capacity limit
  },
  compression: {
    enabled: true,
    minSizeBytes: 1024
  },
  l1: {
    enabled: true
  },
  ttl: {
    defaultMs: 60000
  }
});

• Parameters:
  • name (string): Unique name/namespace for the cache.
  • config (CacheConfig):
    • shards (number, optional): Number of internal lock shards. High concurrency workloads benefit from larger shard numbers (e.g. 16 or 32). Default: 8.
    • eviction (object, optional): Configuration for eviction and capacity limits:
      • policy ("lru" | "arc" | "w-tinylfu" | "tinylfu"): The eviction policy. "w-tinylfu" is the default and preferred policy name; "tinylfu" is accepted as an alias.
      • capacity (number): The maximum number of entries allowed in the cache. Default: 10000.
      • maxBytes (number, optional): The maximum memory size of keys and values combined in bytes. When this threshold is crossed, entries are evicted according to the active policy.
    • compression (object, optional): Configuration for LZ4 compression:
      • enabled (boolean): Enable or disable payload compression. Default: false.
      • algorithm ("lz4", optional): Compression algorithm to use. Default: "lz4".
      • minSizeBytes (number, optional): Min bytes size required to compress JSON payloads. Default: 1024 (1 KB).
    • l1 (object, optional): Configures the fast JS-level L1 cache layer:
      • enabled (boolean): Enable or disable the V8-heap L1 layer. Default: true.
      • capacity (number, optional): Max L1 capacity. Default: 10% of L2 capacity (capped at 10000).
    • ttl (object, optional): Configures time-to-live settings:
      • defaultMs (number, optional): Default expiration in milliseconds.
      • mode ("absolute" | "sliding", optional): Expiry mode. Default: "absolute".
    • Legacy Compatibility Options (can be defined directly at the root config object):
      • policy (maps to eviction.policy)
      • capacity (maps to eviction.capacity)
      • maxBytes (maps to eviction.maxBytes)
      • l1Capacity (maps to l1.capacity)
      • compressionEnabled (maps to compression.enabled)
• Returns: Cache instance.
• Throws: Error if a cache with the specified name already exists.

getCache(name)

Retrieves an existing cache by name.

• Parameters:
  • name (string): The cache namespace.
• Returns: Cache | null (returns null if the cache does not exist).

deleteCache(name)

Deletes a cache instance from the manager.

• Parameters:
  • name (string): The cache namespace to delete.
• Returns: boolean (true if the cache was successfully deleted).

clear()

Deletes all cache instances managed by this instance.

• Returns: void

dispose()

Releases all cache instances immediately, releasing all underlying native memory.

• Returns: void

---

Cache

An isolated, thread-safe cache instance.

get(key)

Retrieves a value from the cache.

const value = cache.get('prod_101');

• Parameters:
  • key (string): The lookup key.
• Returns: Buffer | string | object | number | boolean | undefined
  • Returns Buffer if the value was stored as a Buffer or Uint8Array.
  • Returns string if the value was stored as a string.
  • Returns object | array | number | boolean if the value was stored as a JSON-serializable type.
  • Returns undefined if the key is missing or expired.

set(key, value, ttlMsOrOptions?)

Stores an entry in the cache. If the key already exists, its value is overwritten.

// A. Simple set with TTL (in milliseconds)
cache.set('key', { data: 'test' }, 60000); 

// B. Set with advanced options
cache.set('key', { data: 'test' }, {
  ttlMs: 60000,
  compression: true,       // Force compression overrides
  minSizeBytes: 512        // Compress JSON payload if >= 512 bytes
});

• Parameters:
  • key (string): The entry key.
  • value (Buffer | Uint8Array | string | any): The payload to store.
  • ttlMsOrOptions (number | SetOptions, optional):
    • If passed as a number: The time-to-live in milliseconds.
    • If passed as an object (SetOptions):
      • ttlMs (number, optional): Time-to-live in milliseconds.
      • compression (boolean, optional): Override global compression configuration for this key.
      • minSizeBytes (number, optional): Minimum byte size override to compress.
• Returns: Buffer | string | object | undefined (returns the old value if it was overwritten, or undefined).

has(key)

Checks if a key exists in the cache and is not expired, without deserializing the value.

if (cache.has('auth_session')) { ... }

• Parameters:
  • key (string): Key to check.
• Returns: boolean (true if key exists and is valid).

peek(key)

Retrieves a value without updating the eviction metadata (e.g., LRU order or frequency sketch count). Useful for logging, debugging, or health checks.

const debugVal = cache.peek('hot_key');

• Parameters:
  • key (string): Key to retrieve.
• Returns: Buffer | string | object | undefined

touch(key, ttl_ms)

Renews or changes the Time-To-Live (TTL) of a key without re-writing the cached value.

cache.touch('session_12', 30 * 60 * 1000); // Extend session by 30 min

• Parameters:
  • key (string): Key to renew.
  • ttl_ms (number, optional): New time-to-live in milliseconds. Use undefined to clear expiry.
• Returns: boolean (true if the key existed and TTL was updated).

increment(key, delta?, ttl_ms?)

Atomically increments a numeric counter key in memory (Tag 4). Ideal for rate limiters.

const requestCount = cache.increment('rate_limit:ip_127.0.0.1', 1, 60000);

• Parameters:
  • key (string): Key of the counter.
  • delta (number, optional): Value to increment by. Default: 1.
  • ttl_ms (number, optional): Time-to-live for the counter if it's created.
• Returns: number (the newly incremented counter value).

decrement(key, delta?, ttl_ms?)

Atomically decrements a numeric counter key in memory (Tag 4).

const remainingTokens = cache.decrement('api_tokens:user_88', 1);

• Parameters:
  • key (string): Key of the counter.
  • delta (number, optional): Value to decrement by. Default: 1.
  • ttl_ms (number, optional): Time-to-live.
• Returns: number (the newly decremented counter value).

mget(keys)

Performs a batch lookup for multiple keys in a single FFI boundary crossing, significantly improving throughput for multi-key lookups.

const items = cache.mget(['k1', 'k2', 'k3']); // Returns { k1: val1, k2: val2 }

• Parameters:
  • keys (string[]): Array of keys to retrieve.
• Returns: Record<string, any> (Object mapping found keys to their deserialized values).

mset(entries, ttl_ms?)

Performs a batch write of multiple key-value entries in a single FFI crossing.

cache.mset({ a: 1, b: 'hello', c: Buffer.from([1, 2]) }, 60000);

• Parameters:
  • entries (Record<string, any>): Object representing key-value entries to store.
  • ttl_ms (number, optional): Time-to-live in milliseconds for all written entries.

mdelete(keys)

Performs a batch delete of multiple keys in a single FFI crossing.

const deletedCount = cache.mdelete(['a', 'b', 'c']);

• Parameters:
  • keys (string[]): Array of keys to delete.
• Returns: number (number of deleted keys).

getOrSet(key, factory, ttl_ms?)

Implements a coalesced compute-on-miss cache access pattern. If two concurrent requests lookup the same missing key, they will await the same factory promise, preventing cache stampedes.

const product = await cache.getOrSet('prod_101', async () => {
  return await db.fetchProduct(101);
}, 60000);

• Parameters:
  • key (string): Key.
  • factory (() => any | Promise<any>): A callback that computes the value if missing. Can return a Promise or a synchronous value.
  • ttl_ms (number, optional): TTL in milliseconds for the computed value.
• Returns: any (the cached value, or the resolved promise result).

delete(key)

Deletes a specific key from the cache.

• Parameters:
  • key (string): The key to remove.
• Returns: boolean (true if the key existed and was deleted).

clear()

Removes all keys and resets stats for this cache instance.

• Returns: void

keys()

Returns an array of all active (non-expired) keys in the cache.

• Returns: string[]

stats()

Returns enriched telemetry statistics for the cache, incorporating lock-free lifetime counters, a shard load analysis, process RSS memory telemetry, and L1 cache layer usage.

const telemetry = cache.stats();
// Telemetry format:
// {
//   hits: 1452,
//   misses: 92,
//   capacity: 10000,
//   size: 4210,
//   bytesUsed: 541029,
//   sets: 1544,
//   deletes: 54,
//   evictions: 12,
//   expirations: 30,
//   hitRate: 0.9404,
//   uptimeMs: 125000,
//   shards: {
//     count: 8,
//     details: [
//       { size: 526, bytesUsed: 67320 },
//       ...
//     ],
//     sizeStdDev: 12.4
//   },
//   memory: {
//     payloadBytes: 541029,     // raw sum of off-heap key + value payloads
//     processRss: 124928300     // overall process RSS memory footprint (bytes)
//   },
//   l1: {
//     size: 421,
//     capacity: 1000
//   }
// }

• Returns: CacheStats object:
  • hits (number): Number of successful read queries.
  • misses (number): Number of queries for missing or expired keys.
  • capacity (number): Cache-wide capacity limit.
  • size (number): Current count of active entries.
  • bytesUsed (number): Current byte-capacity usage of stored keys and values (same as payloadBytes).
  • sets (number): Lifetime set operations.
  • deletes (number): Lifetime delete operations.
  • evictions (number): Lifetime capacity and memory size policy eviction events.
  • expirations (number): Lifetime lazy expiration events.
  • hitRate (number): Cache hit rate (between 0 and 1).
  • uptimeMs (number): Uptime of this cache instance in milliseconds.
  • shards (object): Shard statistics breakdown:
    • count (number): Number of internal locks shards.
    • details (object[]): Array containing the { size, bytesUsed } load of each shard.
    • sizeStdDev (number): The standard deviation of the entry counts across all shards. A high value suggests hashing imbalance or key pattern collisions.
  • memory (object): Memory footprint metrics:
    • payloadBytes (number): Theoretical memory allocated for key and value payloads.
    • processRss (number): System resident set size (RSS) memory footprint of the Node.js process (useful to compare payload consumption vs native memory pool allocation/fragmentation).
  • l1 (object): JS L1 cache layer usage:
    • size (number): Current number of items in the V8 heap L1 cache.
    • capacity (number): Capacity limit of the L1 cache.

monitor(callback, intervalMs?)

Initiates a pull-based background poller that tracks real-time statistics changes (deltas and operation rates) without blocking the hot path.

const stop = cache.monitor((snapshot) => {
  console.log(`Operations per second: ${snapshot.rates.opsPerSec}`);
  console.log(`Cache Hit Rate: ${snapshot.rates.hitRate}`);
}, 1000); // Poll metrics every 1 second

// Stop monitoring when done
stop();

• Parameters:
  • callback ((snapshot: StatusSnapshot) => void): Callback triggered on every polling interval.
  • intervalMs (number, optional): Real-time polling frequency in milliseconds.
    • Inherits the soft floor minIntervalMs from the configuration (defaulting to 500ms).
    • Rejects intervals below the hard safety floor limit of 16ms (throwing a RangeError).
• Returns: () => void (a disposer function to stop the background monitor timer).

dispose()

Explicitly disposes of the native sub-caches immediately, freeing all of its memory back to the OS.

• Returns: void