The fluvio topic subcommands are used to create and delete topics, as
well as to view basic information about existing topics.
This command is used to create new Fluvio topics.
Create a Topic with the given name
fluvio topic create [OPTIONS] [name]
Arguments:
[name]
The name of the Topic to create
Options:
-p, --partitions <partitions>
The number of Partitions to give the Topic
Partitions are a way to divide the total traffic of a single Topic into separate streams which may be processed independently. Data sent to different partitions may be processed by separate SPUs on different computers. By dividing the load of a Topic evenly among partitions, you can increase the total throughput of the Topic.
[default: 1]
-r, --replication <integer>
The number of full replicas of the Topic to keep
The Replication Factor describes how many copies of the Topic's data should be kept. If the Topic has a replication factor of 2, then all of the data in the Topic must be fully stored on at least 2 separate SPUs.
This applies to each Partition in the Topic. If we have 3 partitions and a replication factor of 2, then all 3 of the partitions must exist on at least 2 SPUs.
[default: 1]
-i, --ignore-rack-assignment
Ignore racks while computing replica assignment
-f, --replica-assignment <file.json>
Replica assignment file
-d, --dry-run
Validates configuration, does not provision
--retention-time <time>
Retention time (round to seconds) Ex: '1h', '2d 10s', '7 days' (default)
--segment-size <bytes>
Segment size (by default measured in bytes) Ex: `2048`, '2 Ki', '10 MiB', `1 GB`
--compression-type <compression>
Compression configuration for topic
--max-partition-size <bytes>
Max partition size (by default measured in bytes) Ex: `2048`, '2 Ki', '10 MiB', `1 GB`
-c, --config <PATH>
Path to topic configuration file
All configuration parameters can be specified either as command-line arguments or inside the topic configuration file in YAML format.
-h, --help
Print help (see a summary with '-h')
Example usage:
$ fluvio topic create greeting
topic "greeting" created
Retention is a policy for how data is cleaned up from a topic.
- For a time-based policy, use
--retention-time - For a segment-size based policy, use
--segment-size
Check the docs for more info about data retention
Example usage:
In this example, the last segment of 500k will be deleted after 30 days.
$ fluvio topic create my-topic --retention-time '30 days' --segment-size 500000
topic "my-topic" created
This configuration will set compression at a topic level. When set producers are forced to use a compression algorithm that matches with the topic configuration. The SPU will reject any Produce request that does not match with the topic configuration.
If --compression-type any is used, SPU will accept any compression algorithm.
possible values:
any(default)nonegziplz4snappy
Example usage:
$ fluvio topic create my-topic --compression-type gzip
topic "my-topic" created
In this example, the topic my-topic will be created with compression type gzip.
By default, Fluvio will automatically assign replicas to SPUs. However, you can manually assign replicas to SPUs by using the --replica-assignment flag.
Please refer to following replica sections for detail of replica assignment.
Note that in order to replication assignment to work, you need to have at least 2 SPUs in your cluster.
Example usage:
In this example, we assign first replica to SPU 0, second replica to SPU 1.
First we create replica assignment file replica.json.
[
{
"id": 0,
"replicas": [
0,
1
]
}
]
The replicas fields correspond to the SPU ids. You can get SPU ids by running fluvio cluster spu list.
Then we create topic with replica assignment file.
$ fluvio topic create my-topic --replica-assignment replica.json
topic "my-topic" created
Use partition commands to show that topic has been created with replica assignment.
$ fluvio partition list
TOPIC PARTITION LEADER REPLICAS RESOLUTION SIZE HW LEO LRS FOLLOWER OFFSETS
my-topic 0 0 [1] Online 0 B 0 0 0 0 [ReplicaStatus { spu: 1, hw: -1, leo: -1 }]
This command shows you all the existing topics in your cluster, as well as some basic information about them, including how many partitions a topic has and how many times it is replicated.
List all of the Topics in the cluster
fluvio topic list [OPTIONS]
Options:
-O, --output <type> Output [default: table] [possible values: table, yaml, json]
-h, --help Print help
Example usage:
$ fluvio topic list
NAME TYPE PARTITIONS REPLICAS IGNORE-RACK STATUS REASON
greeting computed 1 1 resolution::provisioned
This command prints more detailed information about a specific topic.
Print detailed information about a Topic
fluvio topic describe [OPTIONS] <name>
Arguments:
<name> The name of the Topic to describe
Options:
-O, --output <type> Output [default: table] [possible values: table, yaml, json]
-h, --help Print help
Example usage:
$ fluvio topic describe greeting
Name : greeting
Type : computed
Partition Count : 1
Replication Factor : 1
Ignore Rack Assignment : false
Status : provisioned
Reason :
-----------------
This command deletes an existing Fluvio topic and all data associated with it. This data may not be recovered, so use this with care.
Delete one or more Topics with the given name(s)
fluvio topic delete [OPTIONS] <name>...
Arguments:
<name>... One or more name(s) of the topic(s) to be deleted
Options:
-c, --continue-on-error Continue deleting in case of an error
-h, --help Print help
Example usage:
$ fluvio topic delete greeting
topic "greeting" deleted
