failed to parse message body

[c0mputerking]

Hello again all :) it has been awhile since posting here, I upgraded my system to version 1.1 anyways now having issues with mail tickets, apparently they are not being parsed correctly see error below. Please forgive me if i am (hopefully) overlooking something simple like a missing dependency on my system which is Debian 9

00:00:00.918 [ThreadPoolTaskScheduler-1] ERROR io.lavagna.service.MailTicketService - failed to parse message body
javax.mail.FolderClosedException: null
        at javax.mail.internet.MimeBodyPart.getContent(MimeBodyPart.java:639) ~[mail-1.4.6.jar!/:1.4.6]
        at io.lavagna.service.MailTicketService.getTextFromMimeMultipart(MailTicketService.java:282) ~[classes/:?]
        at io.lavagna.service.MailTicketService.getTextFromMessage(MailTicketService.java:270) ~[classes/:?]
        at io.lavagna.service.MailTicketService.checkNew(MailTicketService.java:164) [classes/:?]
        at io.lavagna.service.MailTicketService$$FastClassBySpringCGLIB$$70991c3d.invoke(<generated>) [classes/:?]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) [spring-core-5.0.4.RELEASE.jar:5.0.4.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:685) [spring-aop-5.0.4.RELEASE.jar:5.0.4.RELEASE]
        at io.lavagna.service.MailTicketService$$EnhancerBySpringCGLIB$$e1e8ac9d.checkNew(<generated>) [classes/:?]
        at io.lavagna.service.Scheduler.checkMailTickets(Scheduler.java:87) [classes/:?]
        at sun.reflect.GeneratedMethodAccessor49.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_171]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_171]
        at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) [spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE]
        at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) [spring-context-5.0.4.RELEASE.jar:5.0.4.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_171]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_171]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_171]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]

[syjer]

Hi @c0mputerking , looks like some kind of bug in our code. I'll have a look.

[syjer]

looking at the code, I can't identify an easy culprit at the moment. Looks like it will be hard to track down. Maybe updating the dependency on our side will fix the bug.

[c0mputerking]

WOW that was quick, thank you so much for looking into this :) for me. As a side Note i had email support tickets working pretty good on 1.1 M8 ... however i rolled out a completely new Debian 9 system for the 1.1 upgrade, and just imported my database to the new version of Lavangna, which worked pretty nicely ... seemed to have missed mail ticket config setting however but not a biggy all my boards and an cards seemed import into the new system.

[syjer]

the exception is launched when calling bodyPart.getContent();

  if (bodyPart.isMimeType("text/plain")) {
    result = result + "\n" + bodyPart.getContent();
    break;
  }

(https://github.com/digitalfondue/lavagna/blob/master/src/main/java/io/lavagna/service/MailTicketService.java#L282).

The exception said that the Folder is already closed, but currently I can't see how it's possible.

[syjer]

looks like it's this issue: https://jira.spring.io/browse/INT-4299

[c0mputerking]

Yes i saw that while googling before i posted :) however it was a bit over my head ... Hmmm also was there some kind of change in versions that would have caused this moving from 1.1.M8 or is just my luck with new install/import?

[syjer]

@c0mputerking , we did update some dependencies, but currently I can't see how they are related to this issue.

[syjer]

As a first test, could you try this snapshot version with the updated dependencies?

https://oss.sonatype.org/content/repositories/snapshots/io/lavagna/lavagna/1.2-SNAPSHOT/lavagna-1.2-20180616.194012-1-distribution.zip

[syjer]

as an alternative, we will need to extend Pop3MailReceiver and ImapMailReceiver and then override/customize the .receive() method.

edit: Or look at what could be possible with HeaderMapper<MimeMessage> headerMapper

[c0mputerking]

I can try the snapshot but is it stable or can/will it mess up my database? or should it just be as simple as putting my lavagna.sh file in the new snapshot dir

[syjer]

it's stable, I've only updated the dependencies (see commit 0f6faa1 ). There are no others changes in the code.

[c0mputerking]

Sorry to be an amateur but i am guessing the dependencies are baked into Java somehow? ie my OS does not need any further packages to be installed?

[syjer]

you are right, all the dependencies are in the ".war" archive.

[c0mputerking]

Ok testing snapshot just waiting for it to pickup the new email not sure how often that happens?

[syjer]

should be each 5 minutes

[c0mputerking]

Ok just for testing i sent 2 emails one html and one plain text, i got the plain text one.
html email still gives error

/root/lavagna-1.2-SNAPSHOT/bin/lavagna.sh
Starting jetty server 9.4.11.v20180605
Server is listening at 0.0.0.0/0.0.0.0:8080
20:18:42.069 [main] INFO  io.lavagna.common.LavagnaEnvironment - For property datasource.dialect, the value is: MYSQL
20:18:42.091 [main] INFO  io.lavagna.common.LavagnaEnvironment - For property datasource.url, the value is: jdbc:mysql://localhost:3306/lavagna?useUnicode=true&characterEncoding=utf-8
20:18:42.091 [main] INFO  io.lavagna.common.LavagnaEnvironment - For property datasource.username, the value is: c0mputerking
20:18:42.092 [main] INFO  io.lavagna.common.LavagnaEnvironment - For property spring.profiles.active, the value is: prod
20:25:01.467 [ThreadPoolTaskScheduler-1] ERROR io.lavagna.service.MailTicketService - failed to parse message body
javax.mail.FolderClosedException: null
        at javax.mail.internet.MimeBodyPart.getContent(MimeBodyPart.java:639) ~[mail-1.4.6.jar!/:1.4.6]
        at io.lavagna.service.MailTicketService.getTextFromMimeMultipart(MailTicketService.java:282) ~[classes/:?]
        at io.lavagna.service.MailTicketService.getTextFromMessage(MailTicketService.java:270) ~[classes/:?]
        at io.lavagna.service.MailTicketService.checkNew(MailTicketService.java:164) [classes/:?]
        at io.lavagna.service.MailTicketService$$FastClassBySpringCGLIB$$70991c3d.invoke(<generated>) [classes/:?]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) [spring-core-5.0.7.RELEASE.jar:5.0.7.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:684) [spring-aop-5.0.7.RELEASE.jar:5.0.7.RELEASE]
        at io.lavagna.service.MailTicketService$$EnhancerBySpringCGLIB$$544b739c.checkNew(<generated>) [classes/:?]
        at io.lavagna.service.Scheduler.checkMailTickets(Scheduler.java:87) [classes/:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_171]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_171]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_171]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_171]
        at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65) [spring-context-5.0.7.RELEASE.jar:5.0.7.RELEASE]
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.0.7.RELEASE.jar:5.0.7.RELEASE]
        at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93) [spring-context-5.0.7.RELEASE.jar:5.0.7.RELEASE]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_171]
        at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_171]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_171]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_171]

[syjer]

thanks for testing, I'm implementing the alternative solution (ugly, but I think it will work).

[c0mputerking]

Ok will we be alerted here? please not a huge rush, do not make it to ugly :) also does this correlate with the other bug i submitted some time ago?

#100

[syjer]

well, the uglyness is only due to the fact that I'll unfortunately need to import some code and modify it, as they don't provide any point to "inject" my custom code. It's unfortunate, but I hope it's temporary :).

I'm pushing the code and a new snapshot in a few minutes. If you want to try :).

(About the #100, I don't think so).

[syjer]

New snapshot up: https://oss.sonatype.org/content/repositories/snapshots/io/lavagna/lavagna/1.2-SNAPSHOT/lavagna-1.2-20180616.204753-2-distribution.zip :)

It should work now, I hope :)

[c0mputerking]

Ok seems to be working now :) bit different output now but i got an html message

root@list:~# /root/lavagna-1.2-SNAPSHOT/bin/lavagna.sh
Starting jetty server 9.4.11.v20180605
Server is listening at 0.0.0.0/0.0.0.0:8080
20:54:12.416 [main] INFO  io.lavagna.common.LavagnaEnvironment - For property datasource.dialect, the value is: MYSQL
20:54:12.439 [main] INFO  io.lavagna.common.LavagnaEnvironment - For property datasource.url, the value is: jdbc:mysql://localhost:3306/lavagna?useUnicode=true&characterEncoding=utf-8
20:54:12.440 [main] INFO  io.lavagna.common.LavagnaEnvironment - For property datasource.username, the value is: c0mputerking
20:54:12.440 [main] INFO  io.lavagna.common.LavagnaEnvironment - For property spring.profiles.active, the value is: prod
20:55:01.515 [ThreadPoolTaskScheduler-1] INFO  io.lavagna.service.mailreceiver.ImapMailReceiver - attempting to receive mail from folder [INBOX] 

[syjer]

If it work, then I'll make a stable release tomorrow :).

Thank you for opening this issue (and testing, obviously) :D

[c0mputerking]

Looks like it is working ... however dis some testing
With an inserted image that did not quite work but i did get an inline text placeholder like [image: ihategmail.png] so i knew there should be an image
Also tested with an attached txt file which did not get attached, but actually got parsed which was kind of cool
Finally i tested with an attached png not working and got invisibly dropped so i would have no idea the sender even tried to upload attachment

[syjer]

Thank you for providing more information :).
About

• the inline image, I think it would be expected, as we don't do any special handling.
• the attached text file: as it process all the text/plain and text/html attachments, I think it's correct
• about the attached image, I'll have a look, it should ignore the data I think. I'm quite sure that it will not add any attachment to the issue though.

[c0mputerking]

would be cool if it added attachments to the cards as attachments may be a feature request ?

[syjer]

sure, I'll create a new issue for that feature request :)

[syjer]

done. I'll close this issue, thank you. :)