Features

We’ll show the different applications you can use Memorix for, and how to do so

Cache

Cache is the most basic use of an in-memory service, to use it simply add a line to your Cache scope in schema.memorix:

Cache {
  adminUser: {
    payload: {
      email: string
      password: string
    }
  }
}

And to use it in your project

await memorix.cache.adminUser.set({
  email: "me@mail.com",
  password: "Aa123456",
});
const adminUser = await memorix.cache.adminUser.get();
from src.generated_schema import Memorix, CacheAdminUser

...

memorix.cache.adminUser.set(CacheAdminUser(email="me@mail.com", password="Aa123456"))
admin_user = memorix.cache.adminUser.get()
memorix.cache.adminUser.set(
  memorix_generated::CacheAdminUser {
    email: "me@mail.com".to_string(),
    password: "Aa123456".to_string(),
  }
).await?;
let admin_user = memorix.cache.adminUser.get().await?;

Cache key

If we want to store and get multiple objects of the same kind that’s also possible!

Cache {
  user {
    key: number
    payload: {
      email: string
      password: string
    }
  }
}

And to use it in your project

await memorix.cache.user.set(1, {
  email: "me@mail.com",
  password: "Aa123456",
});
await memorix.cache.user.set(2, {
  email: "you@mail.com",
  password: "Aa123456",
});
const me = await memorix.cache.user.get(1);
from src.generated_schema import Memorix, CacheUser

...

memorix.cache.user.set(
  1,
  CacheAdminUser(email="me@mail.com", password="Aa123456"),
)
memorix.cache.user.set(
  2,
  CacheAdminUser(email="you@mail.com", password="Aa123456"),
)
me = memorix.cache.adminUser.get(1)
memorix.cache.user.set(
  &1,
  memorix_generated::CacheAdminUser {
    email: "me@mail.com".to_string(),
    password: "Aa123456".to_string(),
  },
).await?;
memorix.cache.user.set(
  &2,
  memorix_generated::CacheAdminUser {
    email: "you@mail.com".to_string(),
    password: "Aa123456",
  },
).await?;
let me = memorix.cache.adminUser.get(&1).await?;

Key can be any type you like, even a nested object

Cache options

You can define cache options in your schema to change it’s behaviour

Cache {
    hello {
        payload: string
        ttl: "10"
    }
    helloForever {
        payload: string
        ttl: "0"
    }
}

Here to defined that each cache item will expire in 5 seconds, but specifically hello will expire in 10 seconds and helloForever won’t expire.

nameTypeDefaultDescription
ttlstring or env value"0" - No expirationThe numerical value of how many seconds until the data is expired and can be deleted from the cache, 0 for no expiration
extend_on_getstring or env value"false"If is set to true, the item’s expiration will be reset each time get api is called

PubSub

PubSub (short for publish and subscribe) is a feature used to broadcast messages from one publisher to many subscribers.
Unlike Cache, the message isn’t saved anywhere and is just passed along.
To use it simply add a line to your PubSub scope in schema.memorix:

PubSub {
  message: {
    payload: string
  }
}

And to use it in your project

await { stop } = memorix.pubsub.message.subscribe(({ payload }) => {
  // Will be called twice with "hello" then "world"
  console.log("Got payload: " + payload);
});
await memorix.pubsub.message.publish("hello");
await memorix.pubsub.message.publish("world");
await stop();
await memorix.pubsub.message.publish("Will be published but no one is listening");

You can also subscribe to an Async iterable if you don’t pass a callback

const subscription = await memorix.pubsub.message.subscribe();
for await (const payload of subscription.asyncIterator) {
  console.log("Got payload: " + payload);
}

To use python pubsub, you need to subscribe on a Thread or a Process since it’s a blocking code, for example

import multiprocessing

def listen_to_message() -> None:
    for payload in memorix.pubsub.message.subscribe():
        # Will be called twice with "hello" then "world"
        print("Got payload: ", payload)

process = multiprocessing.Process(target=listen_to_message)

memorix.pubsub.message.publish(payload="hello")
memorix.pubsub.message.publish(payload="world")

process1.kill()

memorix.pubsub.message.publish(payload="Will be published but no one is listening")

To use rust pubsub, you need to subscribe on a different async function, for example

extern crate tokio;
extern crate futures_util;

use futures_util::StreamExt;

async fn listen_to_message(
    mut memorix: example_schema_generated::Memorix,
) -> Result<(), Box<dyn std::error::Error + Sync + Send>> {
  let subscription = memorix.pubsub.message.subscribe().await?
  loop {
    let payload = subscription
      .next()
      .await
      .expect("Subscription shouldn't end")?;
    println!("Got payload: {}", payload);
  }
  Ok(())
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Sync + Send>> {
  let mut memorix = example_schema_generated::Memorix::new("redis://localhost:6379/0").await?;

  let futures_v: Vec<
      std::pin::Pin<
          Box<dyn std::future::Future<Output = Result<(), Box<dyn std::error::Error + Sync + Send>>>>,
      >,
  > = vec![
    Box::pin(listen_to_message(memorix.clone())),
    Box::pin(async move {
      memorix.pubsub.message.publish(&"hello".to_string()).await?;
      memorix.pubsub.message.publish(&"world".to_string()).await?;
      Ok(())
    }),
  ];

  futures::future::select_all(futures_v).await.0?; // Run until one is complete
  memorix.pubsub.message.publish(&"Will be published but no one is listening".to_string()).await?;

  Ok(())
}

PubSub key

PubSub also supports key the same as Cache supports it, you can use it to publish a message to a specific subscriber or a group.

Task

Task is the last feature Memorix supports, it sends a message to a specific queue, and one of the listeners picks it up.

Unlike PubSub, even if no one is listening to a queue, it’s saved and will be passed to a single listener once it starts listening.

To use it simply add a line to your Task scope in schema.memorix:

Task {
  addMessage: {
    payload: string
  }
}

And to use it in your project

await memorix.task.addMessage.queue("hello");
await memorix.task.addMessage.queue("world");
await { stop } = memorix.task.addMessage.dequeue(async (payload) => {
  // Will be called twice with "hello" then "world"
  console.log("Got payload: " + payload);

  // To stop listening, just for example
  if (payload === "world") {
    await stop();
  }
});

// Emptys queue
await memorix.task.addMessage.empty();

You might want to consider also using a Thread or a Process just like we did with PubSub, this example won’t do that


memorix.task.addMessage.queue(payload="hello")
memorix.task.addMessage.queue(payload="world")

for res in memorix.task.addMessage.dequeque():
    # Will be called twice with "hello" then "world"
    print("Got payload: ", res.payload)

    # To stop listening, just for example
    if res.payload == "world"
      break

# Emptys queue
memorix.task.addMessage.empty()
memorix
        .task
        .pass_ball
        .addMessage
        .enqueue(&"hello".to_string())
        .await?;
memorix
        .task
        .pass_ball
        .addMessage
        .enqueue(&"world".to_string())
        .await?;

let mut async_iterator = memorix
    .task
    .addMessage
    .dequeue()
    .await?;

loop {
    let payload = async_iterator
        .next()
        .await
        .ok_or("Async Iterator shouldn't end")??;

    if payload == "world" {
      break;
    }
}
memorix.task.addMessage.empty().await?;

Task key

Task also supports key the same as Cache and PubSub support it, you can use put messages in different queues.

Task options

You can define task options globally or per task item in your schema to change it’s behaviour

Task {
  addMessage {
    payload: string
    queue_type: env(QUEUE_TYPE)
  }
}
nameTypeDefaultDescription
queue_typestring or env value"fifo"fifo (first in first out) or lifo (last in first out)

Namespace and imports

Namespaces are useful when using multiple schemas in order to avoid name collisions, also great for microservices to define their own schema.
To use it simply surround your desired scopes with a namespace scope

messages-schema.memorix

Namespace messages {
  PubSub {
    addItem: {
      payload: int
      public: [publish subscribe]
    }
  }
}

then you can import it in your main schema

schema.memorix

Config {
  import: [
    "<path-to>/messages-schema.memorix"
    "<path-to>/another-schema.memorix"
  ]
  export: {
    files: [
      {
        language: typescript
        path: "memorix.generated.ts"
      }
    ]
  }
}

PubSub {
  addItem {
    payload: string
  }
}

Even though we defined PubSub.addItem twice, since one is in a namespace, they won’t collide with each other.
To use it in your project

await memorix.pubsub.addItem.publish(12);
await memorix.messages.pubsub.addItem.publish("in 'messages' namespace");
memorix.pubusb.addItem.publish(payload=12)
memorix.messages.pubusb.addItem.publish(payload="woin 'messages' namespacerld")
memorix.pubusb.addItem.publish(&12).await?;
memorix.messages.pubusb.addItem.publish(&"woin 'messages' namespacerld").await?;

You can also see we defined public in the imported messages-schema.memorix, by default, all imported API is private. Here is a list of methods which we can define as public

  • Cache methods (can be defined as private / public)

    nameDescription
    getGetting the stored value
    setSetting the stored value
    deleteDeleting the stored value
  • PubSub methods (can be defined as private / public)

    nameDescription
    publishPublishing a value to all listeners
    subscribeSubscribing as a listener
  • Task methods (can be defined as private / public)

    nameDescription
    enqueueAdding a value to the queue
    dequeueGetting a value from the queue
    emptyClearing the queue
    get_lenGet current length of the queue