Support new firmware version

[marian-m12l]

Add support for the new firmware, which comes with great improvements:

• Packs are stored in a filesystem, which should be easier to manage than a custom usb driver
• Assets are now compressed on the device, which should free up plenty of storage! (my 4GB device went from 73% used to 9% !!)

Major update needed, though.

[chrobiche]

Can you clarify if the new firmware can work with version 0.2.0 or we need to wait to update?

[bourgouinadrien]

I've just upgraded the Lunii of one of my daughter and Lunii isn't detected anymore but Studio.
Also, story pack where like .pack files and they are now .pk files.
It's still just some archived, I was able to find 2 audio file in sf\000\

[marian-m12l]

Indeed, the newest firmware is not supported by STUdio. When upgraded, the Lunii become a completely different device (even the usb productId changed), using a filesystem partition instead of usb bulk operations, compressed assets and a new binary format to describe nodes.

I did some "manual" testing since opening this issue to try and make it work on my device, but as of today the packs only work partially. And I am unsure whether I can make it work in the future. I'm working (from time to time...) on integrating this work into STUdio (on a branch) to automate the generation of pack files and detect the new device / partition, but this will take some time...

[GossBust]

Hi,

I updated the firmware of my Lunii and lost all my custom stories, what a shame and a waste of time!
Is there a way to downgrade the firmware and come back to the latest firmware working with Studio?

By the way, mhank you Marian for Studio: incredible work!

[kaelhem]

@marian-m12l Do you need any help to support the new firmware? It would be a pleasure to lend a hand (even if I'm not a java guy). Did you manage to do reverse-engineering with the new binary format? Does the file transfer work?
I've almost finished my converter Moiki -> STUdio, so I'm really motivated :)

[GossBust]

Hi,

I updated the firmware of my Lunii and lost all my custom stories, what a shame and a waste of time!
Is there a way to downgrade the firmware and come back to the latest firmware working with Studio?

By the way, mhank you Marian for Studio: incredible work!

Does anybody think it is possible to copy the old firmware from the SD card of another Lunii?
Did anybody tried this manipulation?
Has someone a SD card image of the previous firmware please?

[ateliermeteorite]

great idea !!

[bourgouinadrien]

Hi,
I updated the firmware of my Lunii and lost all my custom stories, what a shame and a waste of time!
Is there a way to downgrade the firmware and come back to the latest firmware working with Studio?
By the way, mhank you Marian for Studio: incredible work!

Does anybody think it is possible to copy the old firmware from the SD card of another Lunii?
Did anybody tried this manipulation?
Has someone a SD card image of the previous firmware please?

I have 2 Lunii, one with old firmware and one with the new.
If it can help I can dd both sd card

[ateliermeteorite]

Hi,
I updated the firmware of my Lunii and lost all my custom stories, what a shame and a waste of time!
Is there a way to downgrade the firmware and come back to the latest firmware working with Studio?
By the way, mhank you Marian for Studio: incredible work!

Does anybody think it is possible to copy the old firmware from the SD card of another Lunii?
Did anybody tried this manipulation?
Has someone a SD card image of the previous firmware please?

I have 2 Lunii, one with old firmware and one with the new.
If it can help I can dd both sd card

yes, if you manage to make a dd of the old version

[kaelhem]

Unfortunately, I think the firmware is flashed on the board, not on the sd...

[marian-m12l]

I've still some work to do to "clean up" the new pack writer. Once it's done, i'll push it on a branch if you guys want to have a look at it. Most of the pack generation is done (the new format is pretty straightforward) but there are issues with the assets. Help and contributions are welcome :-)

[marian-m12l]

I just pushed the WIP pack writer on branch feature/firmware2dot4, if y'all want to have a look.

A few words of caution:

• This does not produce fully-working story packs
• This requires the default official story pack to be copied from the device (.content/4CDF38C6) into ~/.studio/fs/4CDF38C6 (the scrambled parts are copied from this existing story pack)
• The pack must be manually transferred into the device: simply copy the generated folder onto the device's .content folder, then edit the binary "pack index" file .pi to add your pack's 16-bytes UUID to the list (e.g. 00000000000000000000000012345678)
• Your source archive story pack must use mono/44100Hz MP3 files
• You'll need to call the pack conversion from your own program, e.g.:

import studio.core.v1.model.StoryPack;
import studio.core.v1.reader.archive.ArchiveStoryPackReader;
import studio.core.v1.utils.PackAssetsCompression;
import studio.core.v1.writer.fs.FsStoryPackWriter;

import java.io.FileInputStream;
import java.nio.file.Path;

public class Main {
    public static void main(String[] args) throws Exception {
        ArchiveStoryPackReader reader = new ArchiveStoryPackReader();
        StoryPack pack = reader.read(new FileInputStream("/path/to/pack.zip"));
        // Prepare assets (RLE-encoded BMP, audio must already be MP3)
        StoryPack packWithPreparedAssets = PackAssetsCompression.withPreparedAssetsFirmware2dot4(pack);
        // Write FS pack
        FsStoryPackWriter writer = new FsStoryPackWriter();
        writer.write(packWithPreparedAssets, Path.of("/tmp"));
    }
}

[GossBust]

Hi,

Can you explain this step please? "then edit the binary "pack index" file .pi to add your pack's 16-bytes UUID to the list (e.g. 00000000000000000000000012345678)"
Where is the .pi file please?

Thanks for the work

[marian-m12l]

@GossBust Should have made this clearer: .pi is a file at the root of the device partition that acts as a "pack index", listing the (ordered) uuids of all the "installed" packs. Apparently, only the last 8 hex characters are used by the device to find the pack folder within .content.

[GossBust]

@marian-m12l Thanks for the explanation. That is why my converted packs where not recognized by the Lunii.
Mounting the Lunii on Windows 7, I can not see this 'pack index' file at the root of the device partition using Windows Explorer. Should I use a specific tool to see it ? "Cmd" ? A binary viewer?

[noctanvol]

@marian-m12l thx for your hardwork on fixing Studio to work with the latest firmware.
Can I buy you a coffee or smt ?
regards

[marian-m12l]

@GossBust Didn't try on Windows. The file is probably hidden, though. You can use an hex editor to add your pack.

[marian-m12l]

@noctanvol Thanks for your kind words. There's still some way to go before it is correctly supported, though (if ever).

[GossBust]

Hi there,

Launching the pack conversion for most of my custom packs, I have an error "Image file is too big to fit into base image: 10876 > 9462". The 'rf' files and 'sf' files are not created in the 'new format' pack.
I guess there is some work on this new firmware version, I keep faith :p.

Waiting for that, do you think there is a way to come back to the previous firmware version?

See you

[marian-m12l]

@GossBust Indeed, support for the new firmware is still far from being done and reliable, it's pretty "hackish" and the assets processing is constrained by the existing story pack. Smaller images can be expanded, but bigger images that cannot fit cause errors. You can tweak your images by converting yourself to BMP format (4-bit depth, RLE encoding/compression) and making sure the files are small enough.
I've not looked into downgrading firmware, but I think it's written to the flash/SPI memory on the device, which (to my knowledge) is not writable via USB anymore.

[GossBust]

@marian-m12l Thanks again for your work and the explanations.

[Grumby]

Hello,

Thank you and well done for your work and the pleasure it can bring to our children and grandchildren.

I fetched https://github.com/marian-m12l/studio/tree/feature/firmware2dot4 via git and created the app with maven. This created a ~/studio folder and the application is in ~ /studio/web-ui/target. It is from this folder that I run the command
sh studio-linux.sh (or sh studio-macos.sh depending on the computer I'm using, although the two scripts are the same).

I have two concerns about your explanation for transferring my Stories Pack to the Lunii device.

1. in the .content/ directory of the Lunii, I don't have a 4CDF38C6 file but six files with 8 alphanumeric characters. Which one should I copy to ~/.studio/fs/ ? Or should I copy all 6 ?

2. when I write a Main.java script by taking your example and replacing (of course) the "/path/to/pack.zip" by my path to my pack and that I place it in the ~/studio/core/target/classes folder I run in a terminal the command:
   javac Main.java
   An error appears:

Main.java:1: error: cannot access StoryPack
import studio.core.v1.model.StoryPack;
bad class file: ./studio/core/v1/model/StoryPack.class
class file has wrong version 55.0, should be 52.0
Please remove or make sure it appears in the correct subdirectory of the classpath.

Am I in the correct folder to run the java script (knowing that in other folders I have the error that the packages cannot be imported because they do not exist) ? How to neutralize the error concerning the wrong version, knowing that I am under jdk 11.0.8 and that version 52 depends on jdk 8.x.x if I believe what I have read on the internet ?

Thanks for your help.

[baldator]

Hi,
I was able to build and push to my Lunii a custom story pack.
However, when I select the cover node, nothing happens, meaning that the menu node is not working. Is someone experiencing the same behavior?
@marian-m12l: in which file is the link between the cover and the menu node?
Thank you

[marian-m12l]

@Grumby
4CDF38C6 folder is supposed to hold the default story pack (Susanne et Gaston). Did you remove it from the device? It's also possible that there are several versions using different uuids.
To create an application of your own, the easiest way is probably to add it within the maven project, so you can benefit from the dependencies management. The error you mention, though, means you're probably compiling with JDK 8. Did you check which javac command is in your path?

[marian-m12l]

@baldator What do you mean "nothing happens"? You turn the wheel onto your pack and you get nothing? empty screen and no sound? no error either? As mentioned, the support is very shaky at this point, and I do not expect a complete story pack to work. The node navigation is basically a done deal (as long as the story pack is consistent), but assets cause issues.

[vpoulailleau]

For the second point I understand that for the moment I should not worry about it, as it only " cosmetic " in some sense so the issue is not coming from this error message, right ?

@xounamoun Right. I downloaded official.json several days after having used STUdio.

Just to be sure of the syntax, you wrote in the launch script:

FS_MOUNTPOINT=/media/ayman/LUNII

If yes, I don't see any problem. You may send the full log.

BTW, once STUdio is launched, you can turn on and off the device, it will appear/disappear in the interface. You don't have to worry about the order of operations.

[MPR-Git]

Merci pour ce travail !

C'est super !
(bon par contre, j'ai mis à jour ma lunii, du coup je vais devoir attendre un peu avant de pouvoir repackager les oli de france inter)

[djouln]

Edit : finalement j'ai fini par trouver d'où venait le pb comme un grand. Un chemin vers une vielle version de java dans la variable systeme "path" de windows10.

Bonjour tout le monde. Et bravo au passage pour le travail accompli.

Je viens vers vous car depuis (malencontreuse) mise à jour du lunni STUDio ne se lance plus. Du tout.
Si qqun peut m'aider.
J'étais en version 0.2. j'ai testé la beta 0.3 sans succes.
Je suis sous windows.
J'ai rajouté set FS_MOUNTPOINT=I:\ dans le .bat (I est bien la lettre de l'appareil) mais sans effet.

Je vous laisse le log, si ca peut aider.

Merci d'avance!

[xounamoun]

@vpoulailleau Weirdly enough I just retried and now it works. Just for fun I tried now to get the .json file working, and evertyhing's smooth with the token.

So ... I tried editing my first little story. I did a short one (with only 3 nodes and the same image / audio each time, just to check). What is really frustrating is that the " drag and drop " towards the Lunii is not working ! I don't get what I am doing wrong.

Does anyone experienced this before ? Could it be a problem of version ? I bought the device last autumn, so I am pretty sure it's the version 2, so I don't know if I am supposed to specify this when I edit the story.

EDIT : in fact, everything's is working fine :) I just have to drop in a specific space in the left bar.

[akelai]

Is a binary release available of a version that works with the new firmware?

[vpoulailleau]

Is a binary release available of a version that works with the new firmware?

You may try https://github.com/marian-m12l/studio/releases/tag/0.3.0-beta.2

[heylliott]

When i run studio, i can't find my lunii...
Anyone can help me ?
I'm on mac, with release 0.3.0 beta 2
Thank you much

[arabine]

Hello, in case of, I just published a beta version of my story teller software : https://github.com/OpenStoryTeller/application
Only available on Windows for now, I can deliver Linux or Mac version if needed. I will release an Android version probably next week.
The work on the electronics device box is undergoing, I will be able to show a first prototype in few months.

Marian >> can you contact me by mail to share some information; I'm planning to build two prototypes, one is for you!

[marian-m12l]

Hi all,

I just released a new beta, and hopefully the last one :-) : https://github.com/marian-m12l/studio/releases/tag/0.3.0-beta.3

It includes a few important (imo) usability improvements:

• automatically fetch the official metadata database. this is probably the most mentioned pain point
• automatically find the device mount point (for firmware 2.x). again, biggest issue w.r.t. support for the new firmware
• UX improvements w.r.t. conversion and grouped pack files, as mentioned in my previous message (yes, that was long ago...)

[bourgouinadrien]

Thanks again @marian-m12l for your hard work.
I'll try that in the next few days and will give you à feedback

[marian-m12l]

@DevauxS The error indicates an invalid "official.json" file. It should be located in your ~/.studio/db folder.

@kollibar Merci pour l'issue, même si je n'ai pas encore pris le temps de regarder le patch. Si c'est assez simple j'essaierai de l'inclure dans la prochaine release.

@tomtom923 La nouvelle version beta devrait détecter automatiquement le chemin de la lunii, donc plus besoin de configurer la "lettre". Pour les images PNG, en fonctions des réglages d'export PNG il n'est pas impossible que vous soyez tombé dans un cas mal supporté. Avez-vous essayé en JPEG ?

@T1nd4 Les packs peuvent être convertis entre les différents formats. L'action est rendue plus explicite avec la dernière UI. Si la conversion fonctionne il n'y a pas plus de risque de "brick" lors du transfert.

@heylliott @MPR-Git Give a try to the latest beta release!

@arabine Wow, thanks! Looks like plenty of fun in the making. I'll get in touch.

@vpoulailleau @bourgouinadrien Thanks for the "customer support"! :-D

[vpoulailleau]

@vpoulailleau @bourgouinadrien Thanks for the "customer support"! :-D

Avec plaisir ! J'ai eu la flemme de faire du Java, mais je voulais aider ton super projet qui me sert bien !

[Dedcap]

Thanks a lot for the great works I will try it those few days Dedcap Le dim. 25 avr. 2021 à 11:01, Vincent Poulailleau ***@***.***> a écrit :

…

@vpoulailleau <https://github.com/vpoulailleau> @bourgouinadrien <https://github.com/bourgouinadrien> Thanks for the "customer support"! :-D Avec plaisir ! J'ai eu la flemme de faire du Java, mais je voulais aider ton super projet qui me sert bien ! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#122 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASXVVBFVVZXUGK2PTGI3NRTTKPK6HANCNFSM4OPJBQAA> .

[akelai]

I just released a new beta, and hopefully the last one :-) : https://github.com/marian-m12l/studio/releases/tag/0.3.0-beta.3

Great. Tried it, but I'm not sure if what I get is normal or not: I see on the left a total of three cards, two of them are the actual only two official packs that I have on the device, and a third card (that I wouldn't expect) that is a weird one with a "?" as image thumbnail (i.e. no thumbnail) and a title that is just a long sequence of characters with no apparent meaning (while the other two cards have there the titles of the packs).

When I put the mouse pointer on the cards, it becomes a "no action" icon (the slashed red circle) when it goes on those two cards of the official packs, and a open-hand icon (i.e. card draggable - but I wouldn't drag it without knowing anything) when it goes on that weird card.

Does this sound normal to you, or is it working bad?

[bourgouinadrien]

So I only try to add a new story on a existing pack.
So far so good "almost".

I just had to add a line to my studio-windows.bat as for the previous version
SET PATH="C:\Program Files\Java\jdk-14.0.2\bin"
Because of this "bug" (I can correct that on windows sideside)
PS C:\Users\Adrien> .\Downloads\studio-web-ui-0.3.0-beta.3\studio-windows.bat 1 fichier(s) copié(s). 1 fichier(s) copié(s). 2021-04-25 17:29:53,295 [ERROR] [i.v.c.i.l.c.RunCommand]: Failed to deploy verticle studio.webui.MainVerticle java.lang.UnsupportedClassVersionError: studio/webui/MainVerticle has been compiled by a more recent version of the Java Runtime (class file version 55.0), this version of the Java Runtime only recognizes class file versions up to 52.0 at java.lang.ClassLoader.defineClass1(Native Method) at java.lang.ClassLoader.defineClass(Unknown Source) at java.security.SecureClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.defineClass(Unknown Source) at java.net.URLClassLoader.access$100(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at io.vertx.core.impl.JavaVerticleFactory.createVerticle(JavaVerticleFactory.java:37) at io.vertx.core.impl.DeploymentManager.createVerticles(DeploymentManager.java:243) at io.vertx.core.impl.DeploymentManager.lambda$doDeployVerticle$2(DeploymentManager.java:216) at io.vertx.core.impl.FutureImpl.dispatch(FutureImpl.java:105) at io.vertx.core.impl.FutureImpl.onComplete(FutureImpl.java:83) at io.vertx.core.impl.DeploymentManager.doDeployVerticle(DeploymentManager.java:183) at io.vertx.core.impl.DeploymentManager.doDeployVerticle(DeploymentManager.java:156) at io.vertx.core.impl.DeploymentManager.deployVerticle(DeploymentManager.java:145) at io.vertx.core.impl.VertxImpl.deployVerticle(VertxImpl.java:683) at io.vertx.core.impl.launcher.commands.VertxIsolatedDeployer.deploy(VertxIsolatedDeployer.java:42) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at io.vertx.core.impl.launcher.commands.ClasspathHandler.deploy(ClasspathHandler.java:168) at io.vertx.core.impl.launcher.commands.RunCommand.deploy(RunCommand.java:403) at io.vertx.core.impl.launcher.commands.RunCommand.run(RunCommand.java:270) at io.vertx.core.impl.launcher.VertxCommandLauncher.execute(VertxCommandLauncher.java:248) at io.vertx.core.impl.launcher.VertxCommandLauncher.dispatch(VertxCommandLauncher.java:383) at io.vertx.core.impl.launcher.VertxCommandLauncher.dispatch(VertxCommandLauncher.java:346) at io.vertx.core.Launcher.main(Launcher.java:45) PS C:\Users\Adrien> .\Downloads\studio-web-ui-0.3.0-beta.3\studio-windows.bat 1 fichier(s) copié(s). 1 fichier(s) copié(s). 2021-04-25 17:30:31,447 [INFO ] [s.w.s.StoryTellerService]: Setting up story teller driver

I was able to add a new story to as existing pack.

Cherry on the cake (and thanks a lot for this) if you have 2 Lunii, you don't have to regenerate the pack again to add it to the other Lunii \o/
You can add it to the other Lunii, it will "just" extract file to the second Lunii.
A lot of time gained :D

Thanks a lot @marian-m12l

edit : well it seems that it's not working as well as I thought bevause the pack didn't appear whent Lunii was started. I'll investigate on this :)

[marian-m12l]

@akelai the behaviour regarding the official story packs seems normal. However there shouldn't be another "unofficial" story pack unless you added it beforehand. What does the official luniistore application show?

[marian-m12l]

@bourgouinadrien The path issue probably depends on one's previous installation of java. Maybe I should improve the documentation. Thanks for mentioning the use of multiple devices, I just fixed the device-specific boot-file generation so that it is generated when transferring the story pack instead of during conversion (which now doesn't require the device to be plugged).

[akelai]

However there shouldn't be another "unofficial" story pack unless you added it beforehand. What does the official luniistore application show?

Ah! The official luniistore application shows those two official cards under "Luniistore albums" folder (I'm translating), and another card under "Studio Lunii" folder. So that "weird card" is probably this third one. But in the official luniistore application it's not weird, it has the picture thumbnail and its actual title. In fact, by looking at it now in its normal shape, I see it is a story I created with the official iPhone app.

[A02l01]

I just tried the beta version and it worked like a charm.
Thanks a lot for your work !

[viraxe91]

bonjour a tous,
tout d’abord un très grand merci à toutes celles et ceux qui ont participer aux développement et plus spécialement @marian-m12l .
je n'ai malheureusement pas encore réussi à connecter la lunii de mon fils mais l'outil me parait très bien fait et super utile.
De mon cote j'arrive à accéder au programme mais malheureusement pas à mon lunii V0.3.0 beta 3

Encore merci pour le temps de dev et merci d'avance pour votre aide.

[niamiot]

works like a charm with studio version 0.3.0-beta.3 and lumni in version 2.6
Thank you so much !!!!!

[Canimania]

Bonjour @marian-m12l

Merci de ton travail.

J'ai essayé avec la lunii v2.2 J'ai bien respecté les opérations. Cependant je n'arrive pas à la detecter sous win10

Est ce normal?

Bonne soirée

[marian-m12l]

@akelai The official luniistore gets your custom story metadata from your account, which STUdio does not, hence the "unknown story pack" behaviour.

@A02l01 @niamiot Thanks for the feedback!

@viraxe91 Connaissez-vous la version du firmware ? Lorsque la Lunii est branchée, est-ce qu'un nouveau "disque" apparaît?

@Canimania Pour le firmware 2.x il faut utiliser une version 0.3.0

[marian-m12l]

After giving some time to the latest beta version (with 350 downloads), I figure it must be stable enough, so I just released version 0.3.0

Thank you for your great feedback! And thank you @Aptustech obviously for the reverse engineering work!

Finally closing this issue :-)

But feel free to open new ones if you encounter problems with the latest version.

[DantSu]

Hi !

Does someone have "Suzan and Gaston" story on Lunii v2 ? I need BMP files to find the common key of the Lunii v3.

Edit : this one : Les mondes merveilleux de Suzanne et Gaston

[ex23]

Hi,

I have a Lunii with the story of "Suzanne et Gaston" as the default story. How can I send the files to you ?

[DantSu]

Hi,

I have a Lunii with the story of "Suzanne et Gaston" as the default story. How can I send the files to you ?

Thx for your reply. I have found someone with same stories as mine. We works on the v3 hack.