Develop (#368)

* Loglevel Debug is sufficient as it gets retried (#337) * Bugfix/339 rados config timeout (#340) * 339 retry 10 times with random wait. retry read for simple read e.g. rados_config. fail with assert in case we can't find rados_config * Feature/342 object search (#343) * Develop (#331) * #328: fix copy move (#330) * Develop (#334) * 217: fix unused code. * 217: unused code and warning * disabled man pages for rmb * Bugfix/283 virtual mailbox fetch metadata (#284) * Develop (#282) * 217: fix unused code. * 217: unused code and warning * disabled man pages for rmb * #283: fetch metadata for mails in virtual mailbox * #283: fix metadata date.saved. date.received virtual mailbox * #283: version * Feature/286 use guid from UUID string (#287) * #256: use guid_128_from_uuid_string instead of guid_128_from_string, to support older uuid formats like RECORD or MICROSOFT, always use compact for printing * version 0.0.25 preparations * #286: build issue * merge * version * #286: prefere cached mail guid. * #286: determine if uuid has - hyphon, if true preseve it. * fix non void return * Feature/289 GitHub actions (#290) * #289: build * #289: build plugin. * #289: submodules * submodules * upgrade git * clean outdated repos * Bugfix/UUID record format (#293) * bugfix initialisaction rados_mail->deprecated_uid * rados_mail creation and default value check * #295: update index after full object has been moved (metadata) (#296) * Bugfix/298 295 bugfixes (#300) * #289: fix force-resync (always use INBOX) #295: rbox_mail_set_expunge (play save) * #298: move ceph objects the standard way. * clean up log. * #298: log warning if client connection times out. * Feature/302 retry expunge connection timeout (#303) * #302: retry in case of ceph connection timeout * rpm * #302: max_retry = 10 * max 10 retry, in case object not available, fail with error * #304: use last know uid to restore email flags (#305) * #306: re-assign unreferenced mail objects to inbox (#309) * #310: read osd_max_object_size and fail if mail.size > osd_max_object_size (#311) * version: 0.0.31 * #313: fix imap append crash (#314) * version 0.0.32 * Bugfix/316 mailbox save (#317) * #316: imap append (remove index entry twice in case of error) * #316: changed storage for big attachments. (sync write) * #316: operate api * #316: fix operate call * #316: release preparations * #316: fixed some legacy unit tests * 316: review results * #316: fix * Bugfix/test output stream check (#323) * init output stream and free mailbuffer * build * extracted write chunks * mailsize -1; * fix * debug messages * init stream any case * added deprecation comment, writing chunks now default. new config param rbox_chunk_size default 10240 bytes * missing mock class * version * disabled (temporarily storage tests) * disabled storage tests( api change) * Feature/alternative save methods (#326) * #322: alternative save methods * version * Feature/319 force resync immediatelly assign bugfix/328 segmentation fault copy from virtual mailbox (#321) * #319: automatically add lost objects to inbox (if no mailbox guid exist) * #319: version 0.0.34 * catch error in case mailbox does not have obox header: e.v. Virtual Mailbox * check for virtual mailbox when moving or copying, (virtual mailbox not allowed as origin) * version * #328: fix copy move (#330) * Bugfix/332 quota and move (#333) * 291: failed save mock test * 291: fixed most of the tests * 332 fix notify messages number and type * Develop (#336) * 217: fix unused code. * 217: unused code and warning * disabled man pages for rmb * Bugfix/283 virtual mailbox fetch metadata (#284) * Develop (#282) * 217: fix unused code. * 217: unused code and warning * disabled man pages for rmb * #283: fetch metadata for mails in virtual mailbox * #283: fix metadata date.saved. date.received virtual mailbox * #283: version * Feature/286 use guid from UUID string (#287) * #256: use guid_128_from_uuid_string instead of guid_128_from_string, to support older uuid formats like RECORD or MICROSOFT, always use compact for printing * version 0.0.25 preparations * #286: build issue * merge * version * #286: prefere cached mail guid. * #286: determine if uuid has - hyphon, if true preseve it. * fix non void return * Feature/289 GitHub actions (#290) * #289: build * #289: build plugin. * #289: submodules * submodules * upgrade git * clean outdated repos * Bugfix/UUID record format (#293) * bugfix initialisaction rados_mail->deprecated_uid * rados_mail creation and default value check * #295: update index after full object has been moved (metadata) (#296) * Bugfix/298 295 bugfixes (#300) * #289: fix force-resync (always use INBOX) #295: rbox_mail_set_expunge (play save) * #298: move ceph objects the standard way. * clean up log. * #298: log warning if client connection times out. * Feature/302 retry expunge connection timeout (#303) * #302: retry in case of ceph connection timeout * rpm * #302: max_retry = 10 * max 10 retry, in case object not available, fail with error * #304: use last know uid to restore email flags (#305) * #306: re-assign unreferenced mail objects to inbox (#309) * #310: read osd_max_object_size and fail if mail.size > osd_max_object_size (#311) * version: 0.0.31 * #313: fix imap append crash (#314) * version 0.0.32 * Bugfix/316 mailbox save (#317) * #316: imap append (remove index entry twice in case of error) * #316: changed storage for big attachments. (sync write) * #316: operate api * #316: fix operate call * #316: release preparations * #316: fixed some legacy unit tests * 316: review results * #316: fix * Bugfix/test output stream check (#323) * init output stream and free mailbuffer * build * extracted write chunks * mailsize -1; * fix * debug messages * init stream any case * added deprecation comment, writing chunks now default. new config param rbox_chunk_size default 10240 bytes * missing mock class * version * disabled (temporarily storage tests) * disabled storage tests( api change) * Feature/alternative save methods (#326) * #322: alternative save methods * version * Feature/319 force resync immediatelly assign bugfix/328 segmentation fault copy from virtual mailbox (#321) * #319: automatically add lost objects to inbox (if no mailbox guid exist) * #319: version 0.0.34 * catch error in case mailbox does not have obox header: e.v. Virtual Mailbox * check for virtual mailbox when moving or copying, (virtual mailbox not allowed as origin) * version * #328: fix copy move (#330) * Bugfix/332 quota and move (#333) * 291: failed save mock test * 291: fixed most of the tests * 332 fix notify messages number and type * #332: fix append additional error mssage in case connection can't be opend (#335) * Develop (#341) * Loglevel Debug is sufficient as it gets retried (#337) * Bugfix/339 rados config timeout (#340) * 339 retry 10 times with random wait. retry read for simple read e.g. rados_config. fail with assert in case we can't find rados_config * #342: simple multithreading object search * #342: version 0.0.40 Co-authored-by: Ewald Dieterich <[email protected]> * Bugfix/342 logging multithreading (#344) * #342 fix threading and additional logs * version * Bugfix/rbox copy context (#347) * 346; fix rbox-copy seg fault always init rbox_mail * #346 fix segmentation fault rbox_copy * #364 and code cleanup * Feature/349 ceph object index (#350) * #349: initial storage api * #349 doveadm rmb create ceph index force resync strategy * allow refresh of ceph object index via -r param * #349 save ceph index objects to separate pool * #349 poc impl ceph index done * #349 new config setting rbox_index_pool_name * #349 force reload of all objects, config setting defines if append of oid to ceph index happens * #349 add , to every entry or entry list * #349 add , to every entry or entry list, and test * repair invalid object not debug level log * #349 changed the remove function, initial index creation: append after each mailbox * #349 fix unit tests * #349 version * #349 bugfix return code doveadm rmb (#351) * #349 validate object (#353) * Bugfix/355 fix gzip trailer for empty stream (#357) * #355 fix write gzip trailer even the stream is empty. * #355 minor improvements for quota messaging * #355 fix gzip trailer for empty stream * #349 fix threshold calc for ceph index file * #355 fix buffersize for write method 1 and 2 (#360) * Bugfix 355 fix buffersize write method (#363) * #355 fix buffersize for write method 1 and 2 * 355 only use append write, clear read buffer in case of read timeout * #355 check for gzip and fix invalid trailer * #355 fix unit tests. * #355 fix call to operate * 355 stream size for uncompressed mails * #355 additional tests * delete ceph index if doveadm mailbox delete -u <> INBOX * Fix/ceph index (#367) * fix ceph index size calculation * delete ceph-index in case its activated * version string --------- Co-authored-by: Ewald Dieterich <[email protected]>
ceph-dovecot · Jan 29, 2023 · 827827c · 827827c
1 parent 3b7970f
commit 827827c
Show file tree

Hide file tree

Showing 5 changed files with 34 additions and 20 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,4 +1,8 @@
 # Change Log
+
+## [0.0.49](https://github.com/ceph-dovecot/dovecot-ceph-plugin/tree/0.0.49) (2023-01-29)
+- fix: cleanup ceph-index in case of mailbox INBOX delete
+
 ## [0.0.48](https://github.com/ceph-dovecot/dovecot-ceph-plugin/tree/0.0.48) (2023-01-26)
 - cleanup ceph-index in case of mailbox INBOX delete
 

diff --git a/configure.ac b/configure.ac
@@ -9,8 +9,7 @@
 AC_PREREQ([2.59])
 
 
-AC_INIT([dovecot-ceph-plugin], [0.0.48], [https://github.com/ceph-dovecot/dovecot-ceph-plugin/issues/new], ,[https://github.com/ceph-dovecot/dovecot-ceph-plugin])
-
+AC_INIT([dovecot-ceph-plugin], [0.0.49], [https://github.com/ceph-dovecot/dovecot-ceph-plugin/issues/new], ,[https://github.com/ceph-dovecot/dovecot-ceph-plugin])
 
 
 AC_CONFIG_AUX_DIR([.])

diff --git a/rpm/dovecot-ceph-plugin.spec b/rpm/dovecot-ceph-plugin.spec
@@ -14,7 +14,8 @@
 Name:		dovecot-ceph-plugin
 Summary:	Dovecot Ceph RADOS plugins
 
-Version:	0.0.48
+Version:	0.0.49
+
 
 Release:	0%{?dist}
 URL:		https://github.com/ceph-dovecot/dovecot-ceph-plugin

diff --git a/src/storage-rbox/rbox-save.cpp b/src/storage-rbox/rbox-save.cpp
@@ -657,9 +657,10 @@ int rbox_save_finish(struct mail_save_context *_ctx) {
         if( r_storage->config->get_object_search_method() == 2){
           // ceph config schalter an oder aus!
           r_storage->s->ceph_index_append(*r_ctx->rados_mail->get_oid());
-          uint64_t index_size = r_storage->s->ceph_index_size();
+          uint64_t index_size = r_storage->s->ceph_index_size();                  
+          double ceph_index_size_percent = ((double)index_size / (double)r_storage->s->get_max_object_size()) *(double)100.0;
           // WARN if index reaches 80% of max object size
-          if( ((index_size/r_storage->s->get_max_object_size()) * 100) > 80) {
+          if( ceph_index_size_percent > 0.80) {
             i_warning("ceph_index file(%d) close to exceed max_object size(%d), recalc index !", index_size, r_storage->s->get_max_object_size() );
           }
         }

diff --git a/src/storage-rbox/rbox-storage.cpp b/src/storage-rbox/rbox-storage.cpp
@@ -939,39 +939,48 @@ int check_users_mailbox_delete_ns_object(struct mail_user *user, librmb::RadosDo
 
 int rbox_storage_mailbox_delete(struct mailbox *box) {
   FUNC_START();
+
   int ret = index_storage_mailbox_delete(box);
   if (ret < 0) {
     i_debug("while processing index_storage_mailbox_delete: %d", ret);
     return ret;
   }
+
   struct rbox_storage *r_storage = (struct rbox_storage *)box->storage;
-  // 90 plugin konfigurierbar!
   read_plugin_configuration(box);
-  if (!r_storage->config->is_rbox_check_empty_mailboxes()) {
-    return ret;
-  }
-
+
   ret = rbox_open_rados_connection(box, false);
   if (ret < 0) {
     i_debug("rbox_storage_mailbox_delete: Opening rados connection : %d", ret);
     return ret;
   }
-  if (r_storage->config->is_user_mapping()) {  //
-    struct rbox_mailbox *rbox = (struct rbox_mailbox *)box;
-    ret = check_users_mailbox_delete_ns_object(rbox->storage->storage.user, r_storage->config, r_storage->ns_mgr,
-                                               r_storage->s);
-  }
-
+
+  i_debug("clean: deleting mailbox %s check if ceph index need to be deleted. %ld , %d, compare %d box='%s' with '%s'",
+            box->name, r_storage->config, 
+            r_storage->config->get_object_search_method(), 
+            strcmp(box->name,"INBOX") == 0, 
+            box->name, "INBOX");
+
   if( r_storage->config->get_object_search_method() == 2 &&
-      strcpy(box->name,'INBOX') == 0 ){    
-        
-    if(r_storage->s->ceph_index_delete()<1){
-      i_warning("ceph_index delete failed, ceph index still exists");
+      strcmp(box->name,"INBOX") == 0 ){    
+    int ceph_index_delete_ret = r_storage->s->ceph_index_delete(); 
+    if(ceph_index_delete_ret<0){
+      i_warning("ceph_index delete failed, ceph index still exists : ret = ", ceph_index_delete_ret);
     }else {
       i_debug("rbox_storage_mailbox_delete: deleting ceph index: %d", ret);  
     }  
   }
 
+  if (!r_storage->config->is_rbox_check_empty_mailboxes()) {
+    return ret;
+  }
+
+  if (r_storage->config->is_user_mapping()) {  //
+    struct rbox_mailbox *rbox = (struct rbox_mailbox *)box;
+    ret = check_users_mailbox_delete_ns_object(rbox->storage->storage.user, r_storage->config, r_storage->ns_mgr,
+                                               r_storage->s);
+  }
+
 
   FUNC_END();
   return ret;