Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add SHOW TOPICS EXTENDED #1268

Closed
apurvam opened this issue May 2, 2018 · 2 comments · Fixed by #3183
Closed

Add SHOW TOPICS EXTENDED #1268

apurvam opened this issue May 2, 2018 · 2 comments · Fixed by #3183
Assignees
@cpettitt-confluent
Labels
enhancement good first issue

Comments

@apurvam
Copy link
Contributor

apurvam commented May 2, 2018 •
edited by miguno
Loading

Currently the SHOW TOPICS command does a lot of work:

  1. List topics -- 1 API call.
  2. List consumer groups -- 1 API call
  3. For each consumer group get number of active consumers -- 1 API call per consumer group.

This takes a long time, particularly when there are many consumer groups.

We should add a show topics extended command which returns the consumer group information.
@miguno
Copy link
Contributor

miguno commented May 3, 2018 •
edited
Loading

To clarify: I suppose you mean to (a) change SHOW TOPICS to only show 1 from above, and (b) add SHOW TOPICS EXTENDED to show 1-3?

@apurvam
Copy link
Contributor Author

apurvam commented May 3, 2018

Yes, that's what I mean.

@apurvam apurvam mentioned this issue May 7, 2018
Closed
@cpettitt-confluent cpettitt-confluent self-assigned this Aug 5, 2019
cpettitt-confluent added a commit to cpettitt-confluent/ksql that referenced this issue Aug 6, 2019
@cpettitt-confluent
feat: Add SHOW TOPICS EXTENDED
5b76e1f 
Fixes confluentinc#1268

Description
===========

This change splits "SHOW TOPICS" into two commands:

1. "SHOW TOPICS EXTENDED", which shows what was previously shown by
"SHOW TOPICS". Sample output:

    ksql> show topics extended;

     Kafka Topic                                                                                   | Partitions | Partition Replicas | Consumers | ConsumerGroups
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1                  | 1         | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1                  | 1         | 1

2. "SHOW TOPICS", which now no longer queries consumer groups and their
active consumers. Sample output:

    ksql> show topics;

     Kafka Topic                                                                                   | Partitions | Partition Replicas
    ---------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1

Notes
=====

This changeset primarily involved renaming KafkaTopicXXX classes to
KafkaTopicXXXExtended and then trimming out the consumer group info from
the original KafkaTopicXXX classes.
cpettitt-confluent added a commit to cpettitt-confluent/ksql that referenced this issue Aug 6, 2019
@cpettitt-confluent
feat: Add SHOW TOPICS EXTENDED
651da03 
Fixes confluentinc#1268

Description
===========

This change splits "SHOW TOPICS" into two commands:

1. "SHOW TOPICS EXTENDED", which shows what was previously shown by
"SHOW TOPICS". Sample output:

    ksql> show topics extended;

     Kafka Topic                                                                                   | Partitions | Partition Replicas | Consumers | ConsumerGroups
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1                  | 1         | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1                  | 1         | 1

2. "SHOW TOPICS", which now no longer queries consumer groups and their
active consumers. Sample output:

    ksql> show topics;

     Kafka Topic                                                                                   | Partitions | Partition Replicas
    ---------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1

Notes
=====

This changeset primarily involved renaming KafkaTopicXXX classes to
KafkaTopicXXXExtended and then trimming out the consumer group info from
the original KafkaTopicXXX classes.
cpettitt-confluent added a commit to cpettitt-confluent/ksql that referenced this issue Aug 6, 2019
@cpettitt-confluent
feat: Add SHOW TOPICS EXTENDED
9e95746 
Fixes confluentinc#1268

Description
===========

This change splits "SHOW TOPICS" into two commands:

1. "SHOW TOPICS EXTENDED", which shows what was previously shown by
"SHOW TOPICS". Sample output:

    ksql> show topics extended;

     Kafka Topic                                                                                   | Partitions | Partition Replicas | Consumers | ConsumerGroups
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1                  | 1         | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1                  | 1         | 1

2. "SHOW TOPICS", which now no longer queries consumer groups and their
active consumers. Sample output:

    ksql> show topics;

     Kafka Topic                                                                                   | Partitions | Partition Replicas
    ---------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1

Notes
=====

This changeset primarily involved renaming KafkaTopicXXX classes to
KafkaTopicXXXExtended and then trimming out the consumer group info from
the original KafkaTopicXXX classes.
cpettitt-confluent added a commit to cpettitt-confluent/ksql that referenced this issue Aug 6, 2019
@cpettitt-confluent
feat: Add SHOW TOPICS EXTENDED
df9a263 
Fixes confluentinc#1268

BREAKING CHANGE: "SHOW TOPICS" no longer includes the "Consumers" and
"ConsumerGroups" columns. You can use "SHOW TOPICS EXTENDED" to get the
output previous emitted from "SHOW TOPICS". See below for examples.

This change splits "SHOW TOPICS" into two commands:

1. "SHOW TOPICS EXTENDED", which shows what was previously shown by
"SHOW TOPICS". Sample output:

```
    ksql> show topics extended;

     Kafka Topic                                                                                   | Partitions | Partition Replicas | Consumers | ConsumerGroups
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1                  | 1         | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1                  | 1         | 1
```

2. "SHOW TOPICS", which now no longer queries consumer groups and their
active consumers. Sample output:

```
    ksql> show topics;

     Kafka Topic                                                                                   | Partitions | Partition Replicas
    ---------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1
```

This changeset primarily involved renaming KafkaTopicXXX classes to
KafkaTopicXXXExtended and then trimming out the consumer group info from
the original KafkaTopicXXX classes.
@cpettitt-confluent cpettitt-confluent mentioned this issue Aug 6, 2019
Merged
cpettitt-confluent added a commit to cpettitt-confluent/ksql that referenced this issue Aug 7, 2019
@cpettitt-confluent
feat: Add SHOW TOPICS EXTENDED
af66006 
Fixes confluentinc#1268

BREAKING CHANGE: "SHOW TOPICS" no longer includes the "Consumers" and
"ConsumerGroups" columns. You can use "SHOW TOPICS EXTENDED" to get the
output previous emitted from "SHOW TOPICS". See below for examples.

This change splits "SHOW TOPICS" into two commands:

1. "SHOW TOPICS EXTENDED", which shows what was previously shown by
"SHOW TOPICS". Sample output:

```
    ksql> show topics extended;

     Kafka Topic                                                                                   | Partitions | Partition Replicas | Consumers | ConsumerGroups
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1                  | 1         | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1                  | 1         | 1
```

2. "SHOW TOPICS", which now no longer queries consumer groups and their
active consumers. Sample output:

```
    ksql> show topics;

     Kafka Topic                                                                                   | Partitions | Partition Replicas
    ---------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1
```

This changeset primarily involved renaming KafkaTopicXXX classes to
KafkaTopicXXXExtended and then trimming out the consumer group info from
the original KafkaTopicXXX classes.
cpettitt-confluent added a commit to cpettitt-confluent/ksql that referenced this issue Aug 7, 2019
@cpettitt-confluent
feat: Add SHOW TOPICS EXTENDED
a6fe3b7 
Fixes confluentinc#1268

BREAKING CHANGE: "SHOW TOPICS" no longer includes the "Consumers" and
"ConsumerGroups" columns. You can use "SHOW TOPICS EXTENDED" to get the
output previous emitted from "SHOW TOPICS". See below for examples.

This change splits "SHOW TOPICS" into two commands:

1. "SHOW TOPICS EXTENDED", which shows what was previously shown by
"SHOW TOPICS". Sample output:

```
    ksql> show topics extended;

     Kafka Topic                                                                                   | Partitions | Partition Replicas | Consumers | ConsumerGroups
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1                  | 1         | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1                  | 1         | 1
```

2. "SHOW TOPICS", which now no longer queries consumer groups and their
active consumers. Sample output:

```
    ksql> show topics;

     Kafka Topic                                                                                   | Partitions | Partition Replicas
    ---------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1
```

This changeset primarily involved renaming KafkaTopicXXX classes to
KafkaTopicXXXExtended and then trimming out the consumer group info from
the original KafkaTopicXXX classes.
cpettitt-confluent added a commit that referenced this issue Aug 7, 2019
@cpettitt-confluent
feat: Add SHOW TOPICS EXTENDED (#3183)
dd3eb5f 
Fixes #1268

BREAKING CHANGE: "SHOW TOPICS" no longer includes the "Consumers" and
"ConsumerGroups" columns. You can use "SHOW TOPICS EXTENDED" to get the
output previous emitted from "SHOW TOPICS". See below for examples.

This change splits "SHOW TOPICS" into two commands:

1. "SHOW TOPICS EXTENDED", which shows what was previously shown by
"SHOW TOPICS". Sample output:

```
    ksql> show topics extended;

     Kafka Topic                                                                                   | Partitions | Partition Replicas | Consumers | ConsumerGroups
    --------------------------------------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1                  | 1         | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1                  | 1         | 1
```

2. "SHOW TOPICS", which now no longer queries consumer groups and their
active consumers. Sample output:

```
    ksql> show topics;

     Kafka Topic                                                                                   | Partitions | Partition Replicas
    ---------------------------------------------------------------------------------------------------------------------------------
     _confluent-command                                                                            | 1          | 1
     _confluent-controlcenter-5-3-0-1-actual-group-consumption-rekey                               | 1          | 1
```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@cpettitt-confluent cpettitt-confluent

Labels
enhancement good first issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

feat: Add SHOW TOPICS EXTENDED cpettitt-confluent/ksql
3 participants
@miguno @apurvam @cpettitt-confluent