Features
We’ll show the different applications you can use Memorix for, and how to do so
Cache
Cache is the most basic use of an in-memory service, to use it simply add a line to your Cache scope in schema.memorix
:
Cache {
adminUser: {
payload: {
email: string
password: string
}
}
}
And to use it in your project
await memorix.cache.adminUser.set({
email: "me@mail.com",
password: "Aa123456",
});
const adminUser = await memorix.cache.adminUser.get();
from src.generated_schema import Memorix, CacheAdminUser
...
memorix.cache.adminUser.set(CacheAdminUser(email="me@mail.com", password="Aa123456"))
admin_user = memorix.cache.adminUser.get()
memorix.cache.adminUser.set(
memorix_generated::CacheAdminUser {
email: "me@mail.com".to_string(),
password: "Aa123456".to_string(),
}
).await?;
let admin_user = memorix.cache.adminUser.get().await?;
Cache key
If we want to store and get multiple objects of the same kind that’s also possible!
Cache {
user {
key: number
payload: {
email: string
password: string
}
}
}
And to use it in your project
await memorix.cache.user.set(1, {
email: "me@mail.com",
password: "Aa123456",
});
await memorix.cache.user.set(2, {
email: "you@mail.com",
password: "Aa123456",
});
const me = await memorix.cache.user.get(1);
from src.generated_schema import Memorix, CacheUser
...
memorix.cache.user.set(
1,
CacheAdminUser(email="me@mail.com", password="Aa123456"),
)
memorix.cache.user.set(
2,
CacheAdminUser(email="you@mail.com", password="Aa123456"),
)
me = memorix.cache.adminUser.get(1)
memorix.cache.user.set(
&1,
memorix_generated::CacheAdminUser {
email: "me@mail.com".to_string(),
password: "Aa123456".to_string(),
},
).await?;
memorix.cache.user.set(
&2,
memorix_generated::CacheAdminUser {
email: "you@mail.com".to_string(),
password: "Aa123456",
},
).await?;
let me = memorix.cache.adminUser.get(&1).await?;
Key can be any type you like, even a nested object
Cache options
You can define cache options in your schema to change it’s behaviour
Cache {
hello {
payload: string
ttl: "10"
}
helloForever {
payload: string
ttl: "0"
}
}
Here to defined that each cache item will expire in 5 seconds, but specifically hello
will expire in 10 seconds and helloForever
won’t expire.
name | Type | Default | Description |
---|---|---|---|
ttl | string or env value | "0" - No expiration | The numerical value of how many seconds until the data is expired and can be deleted from the cache, 0 for no expiration |
extend_on_get | string or env value | "false" | If is set to true, the item’s expiration will be reset each time get api is called |
PubSub
PubSub (short for publish and subscribe) is a feature used to broadcast messages from one publisher to many subscribers.
Unlike Cache
, the message isn’t saved anywhere and is just passed along.
To use it simply add a line to your PubSub scope in schema.memorix
:
PubSub {
message: {
payload: string
}
}
And to use it in your project
await { stop } = memorix.pubsub.message.subscribe(({ payload }) => {
// Will be called twice with "hello" then "world"
console.log("Got payload: " + payload);
});
await memorix.pubsub.message.publish("hello");
await memorix.pubsub.message.publish("world");
await stop();
await memorix.pubsub.message.publish("Will be published but no one is listening");
You can also subscribe to an Async iterable
if you don’t pass a callback
const subscription = await memorix.pubsub.message.subscribe();
for await (const payload of subscription.asyncIterator) {
console.log("Got payload: " + payload);
}
To use python pubsub, you need to subscribe on a Thread
or a Process
since it’s a blocking code, for example
import multiprocessing
def listen_to_message() -> None:
for payload in memorix.pubsub.message.subscribe():
# Will be called twice with "hello" then "world"
print("Got payload: ", payload)
process = multiprocessing.Process(target=listen_to_message)
memorix.pubsub.message.publish(payload="hello")
memorix.pubsub.message.publish(payload="world")
process1.kill()
memorix.pubsub.message.publish(payload="Will be published but no one is listening")
To use rust pubsub, you need to subscribe on a different async function, for example
extern crate tokio;
extern crate futures_util;
use futures_util::StreamExt;
async fn listen_to_message(
mut memorix: example_schema_generated::Memorix,
) -> Result<(), Box<dyn std::error::Error + Sync + Send>> {
let subscription = memorix.pubsub.message.subscribe().await?
loop {
let payload = subscription
.next()
.await
.expect("Subscription shouldn't end")?;
println!("Got payload: {}", payload);
}
Ok(())
}
#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Sync + Send>> {
let mut memorix = example_schema_generated::Memorix::new("redis://localhost:6379/0").await?;
let futures_v: Vec<
std::pin::Pin<
Box<dyn std::future::Future<Output = Result<(), Box<dyn std::error::Error + Sync + Send>>>>,
>,
> = vec![
Box::pin(listen_to_message(memorix.clone())),
Box::pin(async move {
memorix.pubsub.message.publish(&"hello".to_string()).await?;
memorix.pubsub.message.publish(&"world".to_string()).await?;
Ok(())
}),
];
futures::future::select_all(futures_v).await.0?; // Run until one is complete
memorix.pubsub.message.publish(&"Will be published but no one is listening".to_string()).await?;
Ok(())
}
PubSub key
PubSub also supports key the same as Cache
supports it, you can use it to publish a message to a specific subscriber or a group.
Task
Task is the last feature Memorix supports, it sends a message to a specific queue, and one of the listeners picks it up.
Unlike
PubSub
, even if no one is listening to a queue, it’s saved and will be passed to a single listener once it starts listening.
To use it simply add a line to your Task scope in schema.memorix
:
Task {
addMessage: {
payload: string
}
}
And to use it in your project
await memorix.task.addMessage.queue("hello");
await memorix.task.addMessage.queue("world");
await { stop } = memorix.task.addMessage.dequeue(async (payload) => {
// Will be called twice with "hello" then "world"
console.log("Got payload: " + payload);
// To stop listening, just for example
if (payload === "world") {
await stop();
}
});
// Emptys queue
await memorix.task.addMessage.empty();
You might want to consider also using a
Thread
or aProcess
just like we did withPubSub
, this example won’t do that
memorix.task.addMessage.queue(payload="hello")
memorix.task.addMessage.queue(payload="world")
for res in memorix.task.addMessage.dequeque():
# Will be called twice with "hello" then "world"
print("Got payload: ", res.payload)
# To stop listening, just for example
if res.payload == "world"
break
# Emptys queue
memorix.task.addMessage.empty()
memorix
.task
.pass_ball
.addMessage
.enqueue(&"hello".to_string())
.await?;
memorix
.task
.pass_ball
.addMessage
.enqueue(&"world".to_string())
.await?;
let mut async_iterator = memorix
.task
.addMessage
.dequeue()
.await?;
loop {
let payload = async_iterator
.next()
.await
.ok_or("Async Iterator shouldn't end")??;
if payload == "world" {
break;
}
}
memorix.task.addMessage.empty().await?;
Task key
Task also supports key the same as Cache
and PubSub
support it, you can use put messages in different queues.
Task options
You can define task options globally or per task item in your schema to change it’s behaviour
Task {
addMessage {
payload: string
queue_type: env(QUEUE_TYPE)
}
}
name | Type | Default | Description |
---|---|---|---|
queue_type | string or env value | "fifo" | fifo (first in first out) or lifo (last in first out) |
Namespace and imports
Namespaces are useful when using multiple schemas in order to avoid name collisions, also great for microservices to define their own schema.
To use it simply surround your desired scopes with a namespace scope
messages-schema.memorix
Namespace messages {
PubSub {
addItem: {
payload: int
public: [publish subscribe]
}
}
}
then you can import it in your main schema
schema.memorix
Config {
import: [
"<path-to>/messages-schema.memorix"
"<path-to>/another-schema.memorix"
]
export: {
files: [
{
language: typescript
path: "memorix.generated.ts"
}
]
}
}
PubSub {
addItem {
payload: string
}
}
Even though we defined PubSub.addItem
twice, since one is in a namespace, they won’t collide with each other.
To use it in your project
await memorix.pubsub.addItem.publish(12);
await memorix.messages.pubsub.addItem.publish("in 'messages' namespace");
memorix.pubusb.addItem.publish(payload=12)
memorix.messages.pubusb.addItem.publish(payload="woin 'messages' namespacerld")
memorix.pubusb.addItem.publish(&12).await?;
memorix.messages.pubusb.addItem.publish(&"woin 'messages' namespacerld").await?;
You can also see we defined public in the imported messages-schema.memorix
, by default, all imported API is private.
Here is a list of methods which we can define as public
Cache methods (can be defined as private / public)
name Description get Getting the stored value set Setting the stored value delete Deleting the stored value PubSub methods (can be defined as private / public)
name Description publish Publishing a value to all listeners subscribe Subscribing as a listener Task methods (can be defined as private / public)
name Description enqueue Adding a value to the queue dequeue Getting a value from the queue empty Clearing the queue get_len Get current length of the queue