Quick Start

Fluvio is an open-source data streaming platform that aggregates, correlates, and apply programmable intelligence to data in motion.

Written in Rust, Fluvio provides low-latency, high-performance programmable streaming on cloud-native architecture.

SmartModules, powered by WebAssembly, provide users the ability to perform inline computations on streaming data without leaving the cluster. – Learn more about SmartModules

Fluvio provides client libraries for several popular programming languages. – Learn more about Fluvio’s client libraries

This guide outlines using the Fluvio CLI for streaming data on your local machine or self-hosted environment.

We will cover:

Starting a cluster
Storing data with Topics
Sending and reading data with Producers and Consumers
Creating Connectors
Developing and using SmartModules

Requirements

Basic command-line experience for using the fluvio CLI
A Rust development environment for SmartModule development
The fluvio and smdk CLIs

Installing the CLI

Download and install the CLI by running:

$ curl -fsS https://hub.infinyon.cloud/install/install.sh | bash

fvm will be installed at ~/.fvm/bin, and will install fluvio and the rest of the development tools at ~/.fluvio/bin.

You will need to add these directories to your shell’s PATH environment variable.

$ echo 'export PATH="${HOME}/.fvm/bin:${HOME}/.fluvio/bin:${PATH}"' >> ~/.bashrc

$ echo 'export PATH="${HOME}/.fvm/bin:${HOME}/.fluvio/bin:${PATH}"' >> ~/.zshrc

$ fish_add_path ~/.fluvio/bin

Example output

$ curl -fsS https://hub.infinyon.cloud/install/install.sh | bash

☁️  Downloading fluvio version manager, fvm
   target arch aarch64-apple-darwin
⬇️ Installing fvm
done: FVM installed successfully at /Users/telant/.fvm
help: Add FVM to PATH using source $HOME/.fvm/env
fluvio install dir /Users/$USER/.fluvio
If version of fluvio is already installed, you can run 'fvm install' or 'fvm switch' to change versions
☁️ Installing fluvio
info: Downloading (1/5): fluvio@0.11.0
info: Downloading (2/5): fluvio-cloud@0.2.15
info: Downloading (3/5): fluvio-run@0.11.0
info: Downloading (4/5): cdk@0.11.0
info: Downloading (5/5): smdk@0.11.0
done: Installed fluvio version 0.11.0
done: Now using fluvio version 0.11.0
🎉 Install complete!
fluvio: 💡 You'll need to add '~/.fvm/bin' and ~/.fluvio/bin/' to your PATH variable
fluvio:     You can run the following to set your PATH on shell startup:
fluvio:       echo 'export PATH="${HOME}/.fvm/bin:${HOME}/.fluvio/bin:${PATH}"' >> ~/.zshrc

Start a cluster

$ fluvio cluster start

Create your first topic

Topics store and send data streams. Create one with:

$ fluvio topic create quickstart-topic

Produce data to your topic

Send data to your topic with:

$ fluvio produce quickstart-topic
> hello world!
Ok!

Exit the prompt with Ctrl+C. You can also send files or stream output:

fluvio produce -f ./path/to/file.txt
echo "hello world!" | fluvio produce quickstart-topic

Consume data from your topic

Read data from your topic with:

$ fluvio consume quickstart-topic -B -d

This will display data sent to the topic.

Create your first SmartModule

SmartModules are user-defined functions that process data streams.

Build a Filter SmartModule

Use the SmartModules Development Kit (smdk) binary to create your SmartModule project:

smdk generate
cd quickstart

A filter SmartModule might look like this:

use fluvio_smartmodule::{smartmodule, Result, Record};

#[smartmodule(filter)]
pub fn filter(record: &Record) -> Result<bool> {
    let string = std::str::from_utf8(record.value.as_ref())?;
    Ok(string.contains('a'))
}

Build and test it with:

smdk build
smdk test --text "cats"

Load it onto your cluster with:

smdk load

Verify it’s loaded:

fluvio smartmodule list

Using SmartModules with Topics

Create a topic and use SmartModules with producers and consumers:

fluvio topic create fruits
fluvio consume fruits --smartmodule=quickstart
fluvio produce fruits

Only records matching the SmartModule’s filter will be shown.

Edit this page