This Week in Fluvio #50
Fluvio is a distributed, programmable streaming platform written in Rust.
This Week in Fluvio is our weekly newsletter for development updates to Fluvio open source.
Welcome to the 50th edition of this week in Fluvio.
It’s been 18 weeks since we last published our weekly newsletter. It was our first Fluvio Newsletter winter!
We are rebooting the newsletter and there is a lot more to come…
https://github.com/infinyon/fluvio/releases/tag/v0.10.6
Connector Development Kit Launch: The goal of the Connector Development Kit (CDK) is to enable developers to write connectors to read data from data sources across different services using fluvio and Rust. We have been internally building our connectors using the CDK and are now ready to share it with the open source community.
Generating a project
$ cdk generate
🤷 Project Name: my-connector
🔧 Destination: ~/my-connector ...
🔧 project-name: my-connector ...
🔧 Generating template ...
✔ 🤷 Will your Connector be public? · false
🤷 Please set a group name: acme
✔ 🤷 Which type of Connector would you like [source/sink]? · source
Ignoring: /var/folders/r8/4x6_d2rn283946frzd1gc1pr0000gn/T/.tmptToFV3/cargo-generate.toml
[1/6] Done: Cargo.toml
[2/6] Done: Connector.toml
[3/6] Done: sample-config.yaml
[4/6] Done: src/config.rs
[5/6] Done: src/main.rs
[6/6] Done: src
🔧 Moving generated files into: `~/my-connector`...
💡 Initializing a fresh Git repository
✨ Done! New project created ~/my-connector
This is what the generated rust code for the connector looks like
If you’re interested in learning more, here are the docs to CDK
We are excited to see what you are going to build with the CDK!
We are currently working on a way to certify connectors built by the community and would love your inputs on our Discord channel.
Besides the Connector Development Kit, we have made other improvements to the packaging of Fluvio components.
Full changelog available here: https://github.com/infinyon/fluvio/blob/master/CHANGELOG.md
We are aware of the frustrations of getting started with fluvio due to the clunky installation process. We are considering a clean binary install, as well as installing using a package manager. However, we have been busy with the CDK, SMDK and other Cloud Features.
We would love to know what do you recommend we build next to make your development flow easier. Please comment in our Discord channel and let us know. Tag Deb (DRC) on your feedback and feature requests.
Enabled –file and –key-separator to be used together, fix –key handling when producing lines (#3092)
Carson Rajcan presented to our team a really cool contribution to the Fluvio OSS. In the project Carson developed the functionality to apply Smart Module transformation for Producer - PR #3014
As Winter is now over… We are planning community events to champion community contributors, as well as planning hackathons to build som cool stuff with the community using CDK, SMDK etc.
Let us know what events you are interested in our Discord channel.
We are putting the finishing touches on connector secrets for you to use the InfinyOn certified connectors built using CDK to interact with InfinyOn Cloud while useing passwords, API keys.
We are also shaping work for materialized views of time based aggregates from topics.
We are also exploring a developer user experience for our cloud platform. More updates on this in a couple of months.
Sr. Rust Cloud Software Engineer Developer Advocate
Get in touch with us on Github Discussions or join our Discord channel and come say hello!
For the full list of changes this week, be sure to check out our CHANGELOG.
