-
-
Notifications
You must be signed in to change notification settings - Fork 656
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Download all messages in a topic to a csv file #1384
Download all messages in a topic to a csv file #1384
Conversation
Hi @tchiotludo, This pr is ready to be reviewed. Could I please have some feedback on it? Thanks in advance |
By this way the functionality will be more visible |
Thanks for the feedback @tchiotludo! I will make the changes. Thanks |
Hi @tchiotludo, Updated PR. Could you have a look when you have sometime? Thanks in advance |
Sorry @tchiotludo I had to fix a code quality issue. Could I get an approval again? Thanks |
Hi @tchiotludo Could you approve again? Sorry had to make a file change. I ran it locally, the build passes now. |
All builds have passed. Is it possible to merge these changes? thanks in advance |
hey @graciasrochelle, almost done but you need to move the button on the bottom bar please : thanks 👍 |
Sorry I must have missed that. I will make the change |
Hi @tchiotludo, I have updated my pr. It's ready to be reviewed again. Thanks in advance |
Hi @tchiotludo thank you for the amazing tool. can you please review this PR. |
Hi @tchiotludo I am interested in this feature as well and will help if you could please approve the PR. Thank you so much for AKHQ! |
Hi @tchiotludo Sorry to bother you. But any chance you can approve this PR. We wanted to use this feature once merged. |
Thank you so much for this feature @graciasrochelle and thank you @tchiotludo for approving the PR. :-) |
@graciasrochelle I think there is an issue with the download feature. allData.push('\n'); When I test it (CSV or JSON) it leads to wrong file format because there is extra commas for each line except the 1st one due to this \n. Moreover I think the csv or json option can be confusing because exporting in CSV or JSON produces the same file. It's just the extension that changes @tchiotludo FYI |
Hi @AlexisSouquiere Thanks for flagging this. Maybe I misunderstood the export to csv requirement. I will update to remove the new line. And also try and edit the csv file to be more meaningful. Will update it this weekend. Sorry for the inconvenience caused. |
…search-avro-schema-subject * commit '9e402c63949e868f877b5dc3940aa618ee89e901': feat(ui): download all messages in a topic to a csv file (tchiotludo#1384)
* commit '9e402c63949e868f877b5dc3940aa618ee89e901': feat(ui): download all messages in a topic to a csv file (tchiotludo#1384)
Hi @AlexisSouquiere, I was having a look at the CSV file, I am not sure if much can be done. I don't know if the headers are static or dynamic. If we know what the headers will be then the file can be made more relevant. What are your thought on this? |
For me headers are not important. If people wants raw data (not JSON I mean) to be exported, CSV is a good option we will just create a file with one record by row. Then people can use the file as they like. The file has a CSV extension but it can be CSV or plain text inside. JSON export can be interesting to export JSON/Avro records but the file content has to be JSON-compliant. So we should build it as a JSON array and each array item is one of the records you want to export [
{ "field1": 1, "field2", "xxx" }, // 1st record
{ "field1": 1, "field2", "xxx" } // 2nd record
] |
Hello! I opened the issue regarding the download of all messages (#1183) some time ago. Thanks @graciasrochelle for implementing this! I want to share my thoughts on the discussion regarding the messge format. I agree with @AlexisSouquiere that we should either create a simple plain CSV file without any JSON elements OR a JSON-compliant file. What we have now is better than nothing and I don't know how difficult it is to create a JSON-compliant file, but it would be perfect to have the option to choose between the two options and create two different files. Thanks again and kind regards Jens |
@graciasrochelle I have created #1442 to fix the issues I mentionned. @tchiotludo it was the last comment that I had regarding the next version release |
close #1183 Co-authored-by: rochelle <[email protected]>
* dev: (56 commits) chore(version): update to 0.24.0 chore(deps): update java deps fix(ui): fixing logo not showing on login form (tchiotludo#1449) fix(ui): fixing data download inconsistencies on JSON and CSV format (tchiotludo#1442) chore(docs): add GrtGaz as AKHQ user (tchiotludo#1439) feat(ui): download all messages in a topic to a csv file (tchiotludo#1384) fix(ui): clicking on Empty Topic button display blank screen (tchiotludo#1429) chore(landing): add baloise logo (tchiotludo#1427) fix(ui): don't disabled empty for compact,delete topic chore(deps): update all java deps fix(topicdata): Use the partition leader from partition info (tchiotludo#1388) fix(ui): harmonize utc labels (tchiotludo#1403) fix(ui): fix produce multi / tombstone not working and forms layout issue (tchiotludo#1396) fix(auth): fix regex header string for long strim (tchiotludo#1402) fix(webserver): reduce default micronaut log to warn (tchiotludo#1406) chore(webserver): warn if security group is wet without a jwt secret (tchiotludo#1412) chore(deps): update micronaut (tchiotludo#1405) feat(docs): list of available roles in documentation (tchiotludo#1394) chore(deps): update all java deps fix(ui): handle sche subject with special chars (tchiotludo#1354) ... # Conflicts: # build.gradle # gradle.properties # helm/akhq/Chart.yaml # src/test/java/org/akhq/controllers/KsqlDbControllerTest.java # src/test/java/org/akhq/repositories/KsqlDbRepositoryTest.java
* dev: (56 commits) chore(version): update to 0.24.0 chore(deps): update java deps fix(ui): fixing logo not showing on login form (tchiotludo#1449) fix(ui): fixing data download inconsistencies on JSON and CSV format (tchiotludo#1442) chore(docs): add GrtGaz as AKHQ user (tchiotludo#1439) feat(ui): download all messages in a topic to a csv file (tchiotludo#1384) fix(ui): clicking on Empty Topic button display blank screen (tchiotludo#1429) chore(landing): add baloise logo (tchiotludo#1427) fix(ui): don't disabled empty for compact,delete topic chore(deps): update all java deps fix(topicdata): Use the partition leader from partition info (tchiotludo#1388) fix(ui): harmonize utc labels (tchiotludo#1403) fix(ui): fix produce multi / tombstone not working and forms layout issue (tchiotludo#1396) fix(auth): fix regex header string for long strim (tchiotludo#1402) fix(webserver): reduce default micronaut log to warn (tchiotludo#1406) chore(webserver): warn if security group is wet without a jwt secret (tchiotludo#1412) chore(deps): update micronaut (tchiotludo#1405) feat(docs): list of available roles in documentation (tchiotludo#1394) chore(deps): update all java deps fix(ui): handle sche subject with special chars (tchiotludo#1354) ... # Conflicts: # build.gradle # docker-compose-dev.yml # docker-compose.yml # gradle.properties # helm/akhq/Chart.yaml # src/test/java/org/akhq/controllers/KsqlDbControllerTest.java # src/test/java/org/akhq/repositories/KsqlDbRepositoryTest.java
* dev: (56 commits) chore(version): update to 0.24.0 chore(deps): update java deps fix(ui): fixing logo not showing on login form (tchiotludo#1449) fix(ui): fixing data download inconsistencies on JSON and CSV format (tchiotludo#1442) chore(docs): add GrtGaz as AKHQ user (tchiotludo#1439) feat(ui): download all messages in a topic to a csv file (tchiotludo#1384) fix(ui): clicking on Empty Topic button display blank screen (tchiotludo#1429) chore(landing): add baloise logo (tchiotludo#1427) fix(ui): don't disabled empty for compact,delete topic chore(deps): update all java deps fix(topicdata): Use the partition leader from partition info (tchiotludo#1388) fix(ui): harmonize utc labels (tchiotludo#1403) fix(ui): fix produce multi / tombstone not working and forms layout issue (tchiotludo#1396) fix(auth): fix regex header string for long strim (tchiotludo#1402) fix(webserver): reduce default micronaut log to warn (tchiotludo#1406) chore(webserver): warn if security group is wet without a jwt secret (tchiotludo#1412) chore(deps): update micronaut (tchiotludo#1405) feat(docs): list of available roles in documentation (tchiotludo#1394) chore(deps): update all java deps fix(ui): handle sche subject with special chars (tchiotludo#1354) ... # Conflicts: # build.gradle # client/src/containers/Topic/Topic/Topic.jsx # client/src/containers/Topic/Topic/TopicData/TopicData.jsx # gradle.properties # helm/akhq/Chart.yaml # src/test/java/org/akhq/controllers/KsqlDbControllerTest.java # src/test/java/org/akhq/repositories/KsqlDbRepositoryTest.java
* dev: (62 commits) chore(docker): add unless-stopped (tchiotludo#1445) fix(connect): sort throw exception (tchiotludo#1431) chore(build): add idea-ext on build.gradle (tchiotludo#1446) fix(topicdata): no topic data when 1 record with newest sort (tchiotludo#1466) fix(topicdata): search can failed data pagination with many results (tchiotludo#1468) chore(schema): typos chore(version): update to 0.24.0 chore(deps): update java deps fix(ui): fixing logo not showing on login form (tchiotludo#1449) fix(ui): fixing data download inconsistencies on JSON and CSV format (tchiotludo#1442) chore(docs): add GrtGaz as AKHQ user (tchiotludo#1439) feat(ui): download all messages in a topic to a csv file (tchiotludo#1384) fix(ui): clicking on Empty Topic button display blank screen (tchiotludo#1429) chore(landing): add baloise logo (tchiotludo#1427) fix(ui): don't disabled empty for compact,delete topic chore(deps): update all java deps fix(topicdata): Use the partition leader from partition info (tchiotludo#1388) fix(ui): harmonize utc labels (tchiotludo#1403) fix(ui): fix produce multi / tombstone not working and forms layout issue (tchiotludo#1396) fix(auth): fix regex header string for long strim (tchiotludo#1402) ... # Conflicts: # .env # build.gradle # client/src/containers/Topic/Topic/TopicData/TopicData.jsx # gradle.properties # helm/akhq/Chart.yaml # src/main/java/org/akhq/controllers/SchemaController.java # src/test/java/org/akhq/controllers/KsqlDbControllerTest.java # src/test/java/org/akhq/repositories/KsqlDbRepositoryTest.java
Related open issues: #1183
Download all messages in a topic to a csv file:
Topics._.akhq.io.-.Google.Chrome.2023-03-15.23-19-57.mp4